CS 139 Algorithm Development
Lab08A: Program Testing with JUnit

Objectives

• create a test class for use with JUnit
• write JUnit test methods
  
• run JUnit tests against a class or set of classes
  

Background     

• What is JUnit?
  
  JUnit is a simple, open source framework to write and run repeatable tests. JUnit features include:
  
      Assertions for testing expected results
      Test fixtures for sharing common test data
      Test runners for running tests
  
  JUnit was originally written by Erich Gamma and Kent Beck.
  
  
• Where is the JUnit home page?
  
  The official JUnit home page is http://junit.org.
  
  
• Where is the JUnit documentation?
  
  The following documents are included in the JUnit distribution in the doc directory:
  
      JUnit Test Infected: Programmers Love Writing Tests
      JUnit Cookbook
      JUnit - A Cook's Tour
      JUnit FAQ
  
  

New Terms

JUnit

A Java tool for automating the testing of Java programs. JUnit is a powerful tool that is used extensively in industry among Java software developers.

test class

A class that has the purpose of testing some other class, but is not part of the final application.

test method

A method that has the purpose of testing another method, but is not part of the final application.

assertion

A statement that should always be true. Assertions are used in testing to make claims about the correctness of programs, which are verified by testing tools such as JUnit.

Instructions

• Create a folder named lab08A in which to store your programs. It will be most convenient if this folder resides on your desktop or in your home directory.
• Follow the detailed directions below.
  
• At the end of the lab, upload this file to Canvas:
• BasicMath_Test.java

Part 1: Using JUnit for Testing

One thing we've learned through decades of software engineering is that developing correct software is very difficult! The best way to ensure correctness is to test thoroughly while software is being developed.

Routine testing of Java programs is made "easy" by a tool called "JUnit". JUnit is a framework that makes testing automatic by providing a standard format for tests and an easy way to execute them.

The basic JUnit pattern is very simple:

• For every class, X, there is a companion class named X_Test that is responsible for testing the correctness of X.
• Test classes must import junit.framework.* and they must extend TestCase (see the example below).
  
  
• For every method, m, in class X there is a companion method, testM, in class X_Test that is responsible for testing the correctness of m.
• Test methods are always of type void and they take no parameters.
  
• Each test method establishes an expected output, then runs its subject method (the one that it's testing) to get an actual output, and then uses the assertEquals method to assert the equivalence of the two.
  (Note: There are other assert statements that can also be used, but for now we will stick with assertEquals.)
  
  If the actual and expected values are not equivalent, then the assertion has failed, and an error message is generated during testing.
  The first parameter to the assertEquals method is an error message to be displayed in this case.
  

Here's a simple example. Note the correspondence in the class and method names.

               
public class BasicMath 
{
   public static double add(double addend1, double addend2) 
   {
      double sum;
      sum = addend1 + addend2;
      return sum;
   }
               
               
   public static double subtract(double subtractor, double minuend) 
   {
      double difference = minuend - subtractor;
      return difference;
   }         
}    
      
(Note: Comments have been omitted for compactness. You should comment your code.)
          

import junit.framework.*;
             
public class BasicMath_Test extends TestCase 
{
   public static void testAdd() 
   {
      double expectedSum = 15.5;
      double actualSum = BasicMath.add(7.2, 8.3);
      assertEquals("Error in BasicMath.add: ", expectedSum, actualSum); 
   }
             
   public static void testSubtract() 
   {
      double expectedDifference = 2.2;
      double actualDifference = BasicMath.subtract(1.3, 3.5);
      assertEquals("Error in BasicMath.subtract: ", 
         expectedDifference, actualDifference); 
   }        
} 

This example will execute one test case per method. More test cases can be added to the test methods (and usually are) to cover lots of different possibilities.

Step 1. Use an editor (of your choice) to create these two Java classes (BasicMath.java and BasicMath_Test.java) in your lab folder.  Then download the file junit.jar from Canvas into the same folder.

Step 2. Open a terminal window and compile the Java classes using javac. In order to incorporate the JUnit classes into the compilation, you'll need to add the classpath (-cp) option:

javac -cp .:junit.jar *.java


You might recall that the "*" is a wildcard character in the Unix command shell, and so the command means "compile everything that ends with .java".

Step 2. Enter the following command to run the tests. This command will execute JUnit, which will find all of the test methods in your test class (BasicMath_Test) and execute them.

java -cp .:junit.jar org.junit.runner.JUnitCore BasicMath_Test


You should see this message (with some minor variation possibly), indicating that your test was successful.

JUnit version 4.11-SNAPSHOT-20120805-1225
..
Time: 0.005

OK (2 tests)


Note: On Unix-based systems (such as the CS lab computers), you can enter the following command one time, after which it will no longer be necessary to add "-cp .:junit.jar" to your Java commands. This command sets an "environment variable", which the javac command will use. The punctuation and capitalization must be exactly as shown.

export CLASSPATH=.:junit.jar:$CLASSPATH

Windows-based systems require that environment variables be set through the Control Panel.

Step 3. Now you will induce an error into the methods and see the result of failed tests.  In each of the BasicMath methods, add "+ 1" at the end of the return statement. For example:

return sum + 1;

Then recompile the classes and rerun the JUnit tests. You should see an error listing something like this (several lines have been deleted):

There were 2 failures:
1) testAdd(BasicMath_Test)
junit.framework.AssertionFailedError: Error in BasicMath.add:  expected:<15.5> but was:<16.5>
    at junit.framework.Assert.fail(Assert.java:50)
...
2) testSubtract(BasicMath_Test)
junit.framework.AssertionFailedError: Error in BasicMath.subtract:  expected:<2.2> but was:<3.2>
...
FAILURES!!!
Tests run: 2,  Failures: 2

This message tells you that two tests have failed, and it displays the error information from the "assertEquals" statements.
 

Part 2: Testing with an IDE (Dr. Java)

All tests completed successfully.
  BasicMath_Test
    testAdd
    testSubtract

Step 2. Now reintroduce the "+ 1" errors in just the BasicMath add method and run the tests again using Dr.Java. You should see an indication that one test passed and the other failed.

Part 2: Testing with an IDE (JGrasp)

Step 1. Start the JGrasp IDE and open the two files from your lab folder. Correct the errors that you introduced in Step 3, and click the "Compile" button to recompile.

JGrasp has built-in support for JUnit, but you need to create the file from within JGrasp to have it "recognize" the file as a JUnit test, otherwise the "runner" will not run your tests. If you want to use an externally created test file, you need to help JGrasp to recognize the file as a test file. So in JGrasp, you need to build a small program to run the tests.

Step 2. Configure JGrasp to find the junit.jar file containing the testing framework. In Settings/Path/Classpath/Workspace, choose the Classpath tab andpoint to the junit.jar file.

Step 3. Create the following class, compile and run it (alternately, put a main into the test file you created before):

import junit.framework.*;
           
public class BasicMathTest_Test
{
    public static void main(String args[]) 
    {
       org.junit.runner.JUnitCore.main("BasicMath_Test");
    }
            
}

Notice that the name of the tester program is passed to the "runner's" main method and that the quotes are necessary because the file name is passed as a String. You can have it run additional tests by including them as arguments to main separated by commas ("Test1_Test", "Test2_Test", etc).

Step 4. Now reintroduce the "+ 1" errors in just the BasicMath add method and run the tests again using this program. You should see an indication that one test passed and the other failed.

Part 2: Testing with an IDE (Eclipse)

Step 1. Start the Eclipse IDE and open the two files from your lab folder. Correct the errors that you introduced in Step 3 insuring that you have no compiler errors.

Eclipse has built in support for JUnit testing.

Step 2. Import your files into a new Java Project. (File/Import/General/FileSystem).

Step 3. Right click your test file and choose Run As... JUnit Test.

Step 4. Now reintroduce the "+ 1" errors in just the BasicMath add method and run the tests again using this program. You should see an indication that one test passed and the other failed.

Part 3: Writing Basic Test Methods

Step 1. Add multiply and divide methods to your BasicMath class. Their signatures should look like these:

public static double multiply(double multiplier, double multiplicand)

public static double divide(double divisor, double dividend)

Step 2. Add testMultiply and testDivide methods to the BasicMath_Test class. Use the same pattern as in testAdd and testSubtract: establish an expected result and an actual result, then compare the two with an assertEquals statement with an appropriate error message.

Step 3. Run your new tests to validate the new methods. Then induce errors into the new methods, just as you did before with add and subtract, and run the test again to see if they are really working.

Part 4: More Testing

It's generally not enough to test a method just once. To really be sure that the method is correct, we need to test multiple times with multiple values.

Step 1. Add two more tests to the testAdd method to test the following cases. Create new variables, (actualSum2, expectedSum2, etc.), and a new assertEquals call for each test. 

case 1: addend1 = 0.0,  addend2 = 0.0
case 2: addend1 = -5.0, addend2 = +3.5

Step 2. Add two additional tests to each of the other three test methods. Design your own test cases for these methods. Don't test division by zero, however; You'll learn later how to deal with that type of error.

Part 5 (Optional): Testing without Source Code

Sometimes it's necessary to test compiled version of classes, without access to the source code. We refer to this as "black box testing", since we can't see inside of the class (i.e., it's source code); Only its behavior is visible.