Grammar & Testing

The Java Grammar

Review of the recursion lab - Look at the solution for the Directory program.

Terminology

Validation

Are we building the product that we are supposed to build?

Verification

Does the product do what it is supposed to do correctly?

Failure

Incorrect/missing behavior

Fault (defect)

An error/mistake

Trigger conditions

The conditions that cause a fault to result in a failure

Symptom

A characteristic of a failure that can help you recognize that a failure has occurred

Fault types

• Algorithmic - A block of code does not generate the proper output for a given input. (think submit)
  • Precision - not to the correct level of accuracy
  • Checking - inputs are not validated before the computation is performed
  • Stress/Overload - the fault occurs because of filling a data structure are filled past their capacity
  • Assignment - Falut in variable/data structure initialization
• Throughput/Performance - A block of code does not perform at the required speed
  • Timing/Coordination - Involves shared resources
• Recovery - A block of code does not respond to another fault appropriately

Testing

Confirming the presence of faults or defects

• Reliability testing - test cases should reflect normal operating conditions
• Robustness testing - test cases should cover a wide variety of conditions

Debugging

Locating and repairing faults/defects

Testing Approaches

Black box testing (closed)

The tester knows the form of input and the expected output, but knows nothing about the internals.

State box testing

The tester has information about the "pre" and "post" states as well as input and output.

White box testing (clear, open)

The tester knows the internal details of the component being tested.

Test plan

A formal document that describes the planned tests to verify the software (see stages).

Code Review

Desk Checking

The programmer reads through the program and traces the execution.

Code Walkthrough

The programmer gives the code to a review team and leads an informal discussion.

Code Inspection

A review team checks the code against a prepared list of concerns

Stages

Unit Testing (or component/module testing)

A component is tested in isolation from any other components.

Integration Testing

Components are tested in combination to ensure that the information is passed properly between them.

System Testing

The entire system is tested to ensure that it does what it is specified to do.

Test Case

A particular choice of inputs.

Test

A collection of test cases.

Followup Test

A test case that is created when a test case indicates a failure. (Is the failure reproducible?)

Discussion of different testing strategies

Black box

Ideally - test every possible input and combination of inputs.

Realistically - much too large for all but the most trivial problems.

Generally some combination of "normal cases" and "extreme cases" are used in testing. Does the software work as expected for normal values? What values will likely cause problems?

Stress testing - Put the system under a simulation of the expected load. Test for performance or synchronization issues.

Usually testing is done with both a combination of black box and white box testing.

Black Box Testing Strategies

Random Testing:

• Generate test cases randomly
• Heuristic Testing:
  • Generate test cases using "rules of thumb"
  • Extreme values are thought to be the most likely to cause a failure
  • Value Heuristics:
    • Extreme values are thought to be the most likely to cause a failure (e.g., include a test case with a large positive value, a small positive value, 0, a large negative number, a small negative number)
    • Include type mismatches (e.g., characters for integers)
  • Array Heuristics:
    • Include a small array, large array, 0-length array
    • Include unsorted, sorted ascending, and sorted descending arrays
    • Include arrays with one value (e.g., all negative, all positive, all 0)

White box (Clear box)- Note - when we state that we should test these statements, we are always providing specific test cases and comparing the results to the expected result

Statement testing

Make sure every statement is executed (if it can't be reached, it doesn't belong)

Branch testing

Make sure every branch (decision point) is executed (think of the number of test cases for some of the PA's)

Path testing

Make sure every path through the program (combinations of decisions and loops) is executed

An example - what examples would you use to clear box test this method.

public static int calculate(int x, int y)
{
   int a, b;
   do
   {
      a = 1;          // S1
      if (x > y)      // S2
      {
         a = 2;       // S3
      }
      x++;            // S4
      b = x * a;      // S5

   }while (b <= 0);   // S6
   
   return b;          // S7
}

Clear box test

• Test every statement
• Test every branch
• Test every path

Automated Testing

• Automated testing depends on our ability to detect failed tests automatically

• A Solution:
  • Have an oracle that knows the truth
• A Complication:
  • An imperfect oracle can lead to both Type I and Type II errors

"Bugs" Beginning Programmers Should Watch Out For

• Arithmetic Operators:
  • Divide by zero
  • Misuse of integer division
• Conditions:
  • Misuse of = for ==
  • Misuse of == for .equals()
• Loops:
  • Incorrect initialization (initializing to the wrong value or initializing the wrong variable)
  • Incorrect update (updating by the wrong amount or updating the wrong variable)
  • Incorrect termination (e.g., off by one)
  • break mistakes
• Arrays:
  • Uninitialized elements
  • Use of unallocated elements (i.e., oversteping array bounds)
• Methods/Functions:
  • Returning the wrong value/variable
  • Multiple return statements
• Classes:
  • Attributes inappropriately declared static
  • Attributes and local variables with the same name
  • Missing constructors
  • Confusing aliases, shallow copies, and deep copies
  • Misuse of overloaded methods
• Derived Classes:
  • Accidental inclusion of a shadow attribute
  • Failure to call the parent's constructor
  • Misuse of overridden methods