Testing

Terminology

Validation

Are we building the product that we are supposed to build?

Verification

Does the product do what it is supposed to do correctly?

Testing

Confirming the presence of defects (Note: this does not say confirming that the software works correctly).

Debugging

Locating and repairing defects - testing is one part of the debugging process

Testing Approaches

Black box testing (closed)

The tester knows the form of input and the expected output, but knows nothing about the internals.

White box testing (clear, open)

The tester knows the internal details of the component being tested.

Test plan

A formal document that describes the planned tests to verify the software.

Test

A component of the test plan. Consists of one or more test cases.

Test Case

A single set of input and its associated expected output.

Code Review

Desk Checking

The programmer reads through the program and traces the execution.

Code Walkthrough

The programmer gives the code to a review team and leads an informal discussion.

Code Inspection

A review team checks the code against a prepared list of concerns

Stages

Unit Testing (or component/module testing)

A component is tested in isolation from any other components.

Integration Testing

Components are tested in combination to ensure that the information is passed properly between them.

System Testing

The entire system is tested to ensure that it does what it is specified to do.

Followup Test

A test case that is created when a test case indicates a failure. (Is the failure reproducible?)

Discussion of different testing strategies

Ideally - test every possible input and combination of inputs.

Realistically - much too large for all but the most trivial problems.

Generally some combination of "normal cases" and "extreme cases" are used in testing. Does the software work as expected for normal values? What values will likely cause problems?

Stress testing - Put the system under a simulation of the expected load. Test for performance or synchronization issues.

Usually testing is done with both a combination of black box and white box testing.

Black Box Testing Strategies

• Generate test cases randomly
• Informed Testing:
  • Generate test cases using "rules of thumb"
  • Specialized Value Strategies:
    • Extreme values are thought to be the most likely to cause a failure (e.g., include a test case with a large positive value, a small positive value, 0, a large negative number, a small negative number)
    • Include the "special values" of 0, null, blank
    • Include type mismatches (e.g., characters for integers, mistyped input)
    • If different behavior is required for different kinds of input (choices, variants), then test some of those variants as well. Categories of differences.
  • Array Strategies:
    • Include a small array, large array, 0-length array
    • Include the null array (one that has not yet been instantiated)
    • Include unsorted, sorted ascending, and sorted descending arrays
    • Include arrays with one value (e.g., all negative, all positive, all 0)

An example: You are to develop test for the linked method. What test cases would you have?

Bank.html

White box - Note - when we state that we should test these statements, we are always providing specific test cases and comparing the results to the expected result

Statement testing

Make sure every statement is executed (if it can't be reached, it doesn't belong).

Branch testing

Make sure every branch (decision point) is executed.

Path testing

Make sure every path through the program (combinations of decisions and loops) is executed.

An example - what test cases would you use to white box test this method.

White box test

• Test every statement
• Test every branch
• Test every path

Testing.java