• Definition:
  • Verifying that a program satisfies specifications by carefully reading the code
• Desk Checking for Students of Introductory Programming:
  • Did you forgetting to initialize variables?
  • Are your Boolean expressions correct?
  • Did you forget to check for a condition (e.g., divide by 0)?
  • Did you compare values appropriately (e.g., consider rounding errors, compare reference types)?
  • Did you misuse integer division?

• Definition:
  • Verifying that a program satisfies specifications by executing the code
• Testing for Students of Introductory Programming:
  • Unit testing should be performed when a module is completed
  • System testing should be performed after (the current version of) the program/application is completed

• Definition:
  • Locating and repairing logic errors
• Debugging for Students of Introductory Programming:
  • Don't worry about formal terms like failure, fault, and defect; use the informal term error instead
  • Do distinguish between the symptom (i.e., the observable manifestation of the error) and the error itself (the logic error in the code)

• The Process:
  1. Create test cases that can be used to identify symptoms (i.e., incorrect behaviors)
  2. Execute the code and identify incorrect behaviors
  3. Form a hypothesis about each incorrect behavior
  4. Instrument the relevant portion(s) of the code (using debug output statements or a debugger)
  5. Execute the instrumented code
  6. Prove the hypothesis (or create a new one and repeat)
  7. Fix the error
  8. Run the tests again
• Some Observations:
  • There is some creativity involved both in the creation of tests and in the formation of hypotheses
  • It is important to use a good process (and be able to describe the process you use); haphazard debugging is almost never successful

• Defined:
  • The inclusion of calls to print() that are designed to provide information about the program while it is executing
• Best Practices:
  • Include a description of what is being output in every call (otherwise it will be impossible to interpret)
  • Don't delete these statements (until the program is correct), just comment them out (because you'll need them later)

• White Box (a.k.a. Open Box and Clear Box)
  • You know how the algorithm works
  • You should try to devise tests that execute all statements/branches/control paths
• Black Box (a.k.a. Closed Box)
  • You only know the inputs and outputs, not the workings
  • You should generate random/representative test inputs (and tests inputs based on the specifications, if you have them)

• Statement Coverage:
  • Make sure every statement is executed
• Branch Coverage:
  • Make sure every "branch" (resulting from conditionals) of every statement is executed
• Path Coverage:
  • Make sure every "path" through the code is executed (which is often impractical in practice)

    public static int calculate(int x, int y)
    {
       int        a, b;

       a = 1;
       if (x > y)
       {
          a = 2;
       }

       x++;
       b = y * a;
       if (y <= 0)
       {
         b++;
       }
       
       return b;
    }
  

• One Test Can Cover Every Statement:
  • calculate(5, -2)
• Two Tests Can Cover Every Branch:
  • calculate( 5, 2)
  • calculate(-2, -1)

• A Seemingly Good Idea:
  • Use all possible inputs
• The Difficulty:
  • All outputs need to be verified
• A More Realistic Approach:
  • Some random tests
  • Some heuristic tests

• Random Testing:
  • Generate test cases randomly
• Heuristic Testing:
  • Generate test cases using "rules of thumb" (e.g., boundary value analysis)

• Value Heuristics:
  • Extreme values are thought to be the most likely to cause a failure (e.g., include a test case with a large positive value, a small positive value, 0, a large negative number, a small negative number)
  • Include type mismatches (e.g., characters for integers)
• Array Heuristics:
  • Include a small array, large array, 0-length array, 1-length array
  • Include unsorted, sorted ascending, and sorted descending arrays
  • Include arrays with one value (e.g., all negative, all positive, all 0)

• File Name Heuristics:
  • Include multiple files with the same prefix.
  • Include multiple files with the same suffix (i.e., file type).
  • Include files that reverse the prefix and suffix.
• String Heuristics:
  • Include strings that vary only in case (e.g., all upper case, all lower case, mixed case)
  • Include zero-length strings
  • Include long strings
  • Include short strings
  • Include strings that consist of one repeated character
  • Include strings that are in reverse order
  • Include strings that swap underscores and dashes
  • Include strings that swap ones and lower case ls
  • Include strings that swap zeros and Os

• Stub:
  • A program that simulates the activity of missing components
• Driver:
  • A program that passes test cases to a particular component
• Test Harness:
  • A system that facilitates the organization and execution of tests