Software Quality Assurance

Software Quality Assurance
An Introduction

Prof. David Bernstein
James Madison University

Computer Science Department

bernstdh@jmu.edu

Getting Started

Definition:
- Quality assurance (QA) is a systematic pattern of activities intended to ensure that a product properly satisfies the needs and desires of its stakeholders.
Activities:
- Validation is the process of determining if a product (or its specification) satisfies stakeholders' needs and desires ("Are we building the right product?")
- Verification is the process of determining if a product (or its specification) satisfies those needs and desires properly ("Are we building the product right?")

An Example

The Setting:
- You are a newspaper reporter and you are told to write an article about homelessness (the product)
Validation:
- You must write about homelessness
- Note: All validation activities are product specific
Verification:
- You must write properly (e.g., use proper grammar, follow the newspaper's style guide, make a compelling argument)
- Note: Some verification activities are product-specific (e.g., the quality of the argument) and some are not (e.g., proper grammar)

Software Quality - History

Conformance to Specifications (Crosby, 1979):
- Items that are verified (i.e., the reliability, robustness, and safety of the product)
Customer Satisfaction (Feigenbaum, 1983):
- Items that are validated (i.e., the adequacy of the product)

History (cont.) - Boehm et al. (1978)

History (cont.)

The Modern Perspectives - ISO/IEC 25010

Functional Suitability:
- Functional completeness, correctness, and appropriateness
Performance Efficiency:
- Processing times and throughput rates, resources used, capacity
Compatability:
- The degree to which the product can co-exist with and interoperate with other products

The Modern Perspectives - ISO/IEC 25010 (cont.)

Software Products vs. Large-Scale Engineering Products

Examples:
- Structures (bridges, buildings, dams)
- Networks (e.g., roads, water, electrical, communication)
How is Quality Defined?
- In very similar ways

Software Products vs. Large-Scale Engineering Products (cont.)

(Courtesy of xkcd)

Software Products vs. Manufactured Products

Other Products:
- After the product is designed and engineered it must be produced/manufactured and each instance is different
- The concern is often with the quality of the instances of the product (sometimes called "quality control")
Software Products:
- After the product is designed and engineered each instance is identical
- The concern is with the quality of the product

Defect Elimination:

Defect Prevention

Process Guides:
- Standards and guidelines that describe the way everyone in the organization should behave
- Templates and checklists that make it easier to do so
Analysis and Design Methodologies:
- Well-codified approaches to understanding and solving software-related problems (e.g., OO)
Well-Studied Solutions:
- Reference architecture
- Design patterns

Defect Prevention (cont.)

Prototypes:
- Throwaway/Exploratory
- Evolutionary
Tools:
- Version/configuration management
- Computer-aided software engineering (CASE)

Defect Detection and Removal

Review and Correct:
- Automated tools
- Manual techniques (e.g., desk-checks, walk-throughs, inspections)
Test and Debug:
- Testing is a validation and verification process that makes use of the system/product (including prototypes) while it is in operation or being operated on

An Abstract View of Testing

Quality of the Process

An Observation:
- Many activities affect product quality
Questions One Might Ask:
- When are we finding particular validation problems and can we find them earlier?
- When are we finding particular verification problems and can we find them earlier?
Implications:
- The quality of the process can be an important determiner of the quality of the product

Organizing for Quality

Root Causes of Quality Problems:
- Management tends to blame workers
- Workers tend to blame managers
Implications (Feigenbaum, 1983):
- Quality shouldn't be the responsibility of a single department
- Quality must be part of the entire process

Approaches/Methodologies/Standards

Quality Circles:
- A small group that meets on a regular basis to consider quality issues for its "department" and make recommendations to management
Six Sigma (ISO 13053:2011):
- Plan-Do-Check-Act (Demming)
- Continuously monitor products and processes (engineering and management processes)
- Focus on achieving measurable improvements and confirming them with statistical analyses
- Well-Defined Rolls (Executives, Champions, Master Black Belts, Black Belts, Green Belts)
ISO 9001:
- Must have a formal statement of a quality policy that is understood and applied at all levels
- Must make decisions based on audited data
- Must communicate with the customer and obtain feedback from the customer

Approaches/Methodologies/Standards (cont.)

The Quality Tradeoff

Costs of Low Quality Software:
- Negligence and liability costs
- Security costs
- Disgruntled customers (hence lost sales)
Costs of High Quality Software:
- Prevention costs
- Appraisal costs
- Delays to market (hence lost sales)

The Quality Tradeoff (cont.)

(Courtesy of xkcd)