Lab: Skills - JUnit and EclEmma in Eclipse

Instructions: Answer the following questions one at a time. After answering each question, check your answer (by clicking on the check-mark icon if it is available) before proceeding to the next question.

Getting Ready: Before going any further, you should:

Setup your development environment.
Download the following files:

../experimenting_unit-testing/using-objects/GiftCard.java

to an appropriate directory/folder (e.g., the course downloads directory/folder). In most browsers/OSs, the easiest way to do this is by right-clicking/control-clicking on each of the links above and then selecting Save as... or Save link as....
If you don't already have one from earlier in the semester, create a project named eclipseskills.
Drag the file GiftCard.java into the default package (using the "Copy files" option).
Open GiftCard.java.

1. JUnit Basics: JUnit is is an open-source testing framework. It provides a way to write, organize, and run repeatable test. This part of the lab will help you become familiar with JUnit. (Help is also available on the CS wiki.)

Create an empty JUnit test named GiftCardTest in the default package by clicking on File-New-JUnit Test Case. Use the most recent version of JUnit; if necessary, add JUnit to the build path when asked.
Note: Normally, you should put tests in their own package(s). To keep things simple, we will break that rule.

Copy the following code into GiftCardTest, replacing the test that was added by Eclipse.

    @Test
    public void getIssuingStoreTest()
    {
        double       balance;        
        GiftCard     card;
        int          issuingStore;
        
        issuingStore = 1337;
        balance      = 100.00;
        card = new GiftCard(issuingStore, balance);

        assertEquals(issuingStore, card.getIssuingStore(), "getIssuingStore()");
    }

A JUnit test suite is a class, much like any other class. Tests are methods that are preceded with the annotation @Test. (Note: An annotation provides information about a program but is not part of the program. Annotations have no effect on the operation of the program. Instead, they are used to provide information to tools that might use the program as input.)
How many tests are in the test suite GiftCardTest?

.
One, the getIssuingStoreTest() method.
JUnit has an Assert class that has a static assertEquals() method with the following signature that is used to compare expected and actual results:
public static void assertEquals(int expected, int actual, String description)
where description is a human-readable String describing the test, expected is the expected result, and actual is the result of actually running the code being tested.

How would you call this method and pass it the String "getIssuingStore()", the int issuingStore, and the int returned by the card object's getIssuingStore() method?

.
Assert.assertEquals(issuingStore, card.getIssuingStore(), "getIssuingStore()");

How is this method actually being called in GiftCardTest?


                                            assertEquals(issuingStore, card.getIssuingStore(), "getIssuingStore()");

Why isn't the class name needed?
.

If you look at the top of the class you will see that the import statement for org.junit.jupiter.api.Assertions.* has a static modifier. This tells the compiler that the class name can be omitted from calls to static methods in this package.
This feature should be used sparingly because it makes code very difficult to read. In this case, writing Assert.assertEquals() seems redundant, so people frequently use a static import.
Execute GiftCardTest. Why is it somewhat surprising that you can execute GiftCardTest?
.

Because GiftCardTest doesn't have a main() method, and, in the past, we've only been able to execute classes that have a main() method.
In fact, JUnit has a class (called org.junit.runner.JUnitCore with a main() method and that class is what is being executed. It then calls the various tests in GiftCardTest. The class with the main() method is being hidden from us by Eclipse to make our lives easier.
What output was generated?
.

None, but test results appeared in a new window. That window indicates that a test suite was run and that all of the tests passed.
To see what happens when a test fails, modify the getIssuingStore() method in the GiftCard class so that it returns issuingStore + 1, save GiftCard.java, and re-run the test suite.
Now what happens?

.

The test fails.
In the "Failure Trace", interpret the line:
```
java.lang.AssertionError: getIssuingStore() expected:<1337> but was:<1338>
```
Note: You may have to scroll the "Failure Trace" window to the right to see the whole message.

.

It says that when the test described as "getIssuingStore()" was run, the result returned should have been 1337 but was 1338.
What mechanism is JUnit using to indicate an abnormal return?
.

It's throwing an exception (in this case, an AssertionFailedError).
Before you forget, correct the fault in the getIssuingStore() method.
The Assert class in JUnit also has a static assertEquals() method with the following signature:
public static void assertEquals(double expected, double actual, double tolerance, String description)
where tolerance determines how close two double values have to be in order to be considered "approximately equal".

Add a test named getBalanceTest() that includes a call to assertEquals() that can be used to test the getBalance() method in the card class (with a tolerance of 0.001).

.
@Test public void getBalanceTest() { double balance; GiftCard card; int issuingStore; issuingStore = 1337; balance = 100.00; card = new GiftCard(issuingStore, balance); assertEquals(balance, card.getBalance(), 0.001, "getBalance()"); }
How many tests are in your test suite now?
.

2
Suppose you had put both calls to assertEquals() in one method (named, say, getIssuingStoreTest()). How many tests would be in your test suite?
.

Unfortunately, different people/tools use different terminology. In JUnit parlance, there is only one test, corresponding to the method getIssuingStoreTest(). In our parlance, a set of inputs and expected outputs (which might require several assertions) defines a test. Nothing in JUnit prevents someone from putting more than one test (in our parlance) into one @Test-annotated method. This is one of the reasons it can be a little difficult to interpret the output from testing tools.
Suppose, for example, you have two @Test-annotated methods, one with 99 assertions and the other with 1. Whether 1 of the 99 assertions fails or 98 out of the 99 assertions fails, JUnit will report that you failed one of two tests (i.e., 50% of the tests).

So, unless you have a good reason to do so, you should not include multiple tests inside of an @Test-annotated method.
Re-compile and re-execute the test suite. How many tests were run?
.

Two, both of which passed.

The Assert class in JUnit also has a static assertEquals() method with the following signature:


    public static void assertEquals(String expected, String  actual, String description)

Using JUnit terminology, add a test named deductTest_Balance() to your test suite that can be used to test the deduct() method in the GiftCard class. Note: Be careful, the variables that are declared in the getIssuingStore() method are local.


                                        @Test
    public void deductTest_Balance()
    {
        double       balance;        
        GiftCard     card;
        int          issuingStore;
        String       s;
        
        
        issuingStore = 1337;
        balance      = 100.00;
        card = new GiftCard(issuingStore, balance);

        s = "Remaining Balance: " + String.format("%6.2f", 90.00);
        assertEquals(s, card.deduct(10.0), "deduct(10.00)");
    }

Execute the test suite.

2. Coverage: This part of the lab will help you understand coverage tools and coverage metrics.

Eclipse has a plugin called EclEmma (that uses a library called JaCoCo) that provides coverage metrics. Read the course "Help" page about using EclEmma.
If EclEmma is not installed, use the instructions on the course "Help" page about Installing EclEmma to install the course code coverage tool.
Run GiftCardTest using EclEmma (Hint: You must click on a different button.), click on the "Coverage" tab and expand the directories/packages until you can see GiftCard.java and GiftCardTest.java.
Why is the coverage of GiftCardTest.java 100% and why is this uninteresting/unimportant?

.

This just means that all of the tests were executed. That isn't at all surprising.
How many of the tests passed?
.

All three of them.
Does this mean that the GiftCard class is correct?
.

I hope so, but I'm guessing from the tone of the question that there may not be enough tests.
What is the statement coverage for GiftCard.java?
.

55.7% - that doesn't sound good.
Select the tab containing the source code for GiftCard.java. What is different about it?
.

Statements are now highlighted in different colors.
What do you think it means when a statement is highlighted in red?
.

The statement was never executed by the tests. In other words, the statement was not covered.
Hover your mouse over the icon to the left of the first if statement in the constructor. What information appears?
.

2 of 4 branches missed.

Add tests to your test suite so that it covers all of the statements and branches in the deduct() method in the GiftCard class.


                                        @Test
    public void deductTest_AmountDue()
    {
        double       balance;        
        GiftCard     card;
        int          issuingStore;
        String       s;
        
        
        issuingStore = 1337;
        balance      = 100.00;
        card = new GiftCard(issuingStore, balance);

        s = "Amount Due: " + String.format("%6.2f", 10.00);
        assertEquals(s, card.deduct(110.0), "deduct 110.00 from 100.00");
    }

    @Test
    public void deduct_InvalidTransaction()
    {
        double       balance;        
        GiftCard     card;
        int          issuingStore;
        String       s;
        
        
        issuingStore = 1337;
        balance      = 100.00;
        card = new GiftCard(issuingStore, balance);

        s = "Invalid Transaction";
        assertEquals(s, card.deduct(-10.0), "deduct -10.00 from 100.00");
    }

Your test suite still may not cover every statement in the GiftCard class. If this is the case, what is different about the statements that remain untested?
.

They all involve exceptions. (If you left other statements uncovered then you were careless.)

3. Testing Methods that Throw Exceptions: This part of the lab will help you learn how to test methods that throw exceptions.

The easiest (though not the most sophisticated) way to test for exceptions is to use include a try-catch statement in the test. For example, add the following test to your test suite.


                         @Test
    public void constructorTest_IncorrectBalance()
    {
        try
        {
            new GiftCard(1, -100.00);
            fail("Constructor, Negative Balance: Should have thrown an IllegalArgumentException");
        }
        catch (IllegalArgumentException iae)
        {
            // This is expected
        }
    }

Run GiftCardTest using EclEmma, click on the "Coverage" tab and expand the directories/packages until you can see GiftCardTest.java. Or, alternatively, click on the tab for GiftCardTest.java.
Why is the coverage of GiftCardTest.java NOT 100% and why is this unimportant?

.

There are statements in the test suite that should not be executed when GiftCard.java is correct. So, the fact that the test suite isn't completely covered doesn't matter -- after all, we're not trying to test the test suite

Add a test to your test suite named constructorTest_IncorrectID_Low() that covers the case when the storeID is less than 0.


                                        @Test
    public void constructorTest_IncorrectID_Low()
    {
        try
        {
            new GiftCard(-1, 100.00);
            fail("Conctructor, Low ID: Should have thrown an IllegalArgumentException");
        }
        catch (IllegalArgumentException iae)
        {
            // This is expected
        }
    }

4. Coverage and Completeness: This part of the lab will help you better understand code coverage and the completeness of test suites.

Run EclEmma on your current test suite. What is the statement coverage for GiftCard.java now?
.

100%
What branch does the test suite fail to test?
.

So far, the test suite is only covering the storeID < 0 branch and not the storeID > MAX_ID branch.

Add a test to your test suite named constructorTest_IncorrectID_High() that covers the other branch.


                                        @Test
    public void constructorTest_IncorrectID_High()
    {
        try
        {
            new GiftCard(100000, 100.00);
            fail("Conctructor, Low ID: Should have thrown an IllegalArgumentException");
        }
        catch (IllegalArgumentException iae)
        {
            // This is expected
        }
    }

Run EclEmma. What is the branch coverage now?
.

All branches are covered.
From a "white box" testing perspective, is your test suite complete? Conversely, can you think of tests that should be added?
.

I would certainly add a test that passed the constructor both an illegal storeID and an illegal openingBalance. Though each was handled properly individually, I can imagine situations where that might not be the case.
From a "black box" testing perspective, is your test suite complete? Conversely, can you think of tests that could be added?
.

It would probably be a good idea to test multiple different legal and illegal parameters, rather than just one (which is the case now). For example, maybe really big numbers or really small numbers will trigger a failure.