- Forward


Software Quality Assurance
An Introduction


Prof. David Bernstein
James Madison University

Computer Science Department
bernstdh@jmu.edu

Print

Getting Started
Back SMYC Forward
  • Definition:
    • Quality assurance (QA) is a systematic pattern of activities intended to ensure that a product properly satisfies the needs and desires of its stakeholders.
  • Activities:
    • Validation is the process of determining if a product (or its specification) satisfies stakeholders' needs and desires ("Are we building the right product?")
    • Verification is the process of determining if a product (or its specification) satisfies those needs and desires properly ("Are we building the product right?")
An Example
Back SMYC Forward
  • The Setting:
    • You are a newspaper reporter and you are told to write an article about homelessness (the product)
  • Validation:
    • You must write about homelessness
    • Note: All validation activities are product specific
  • Verification:
    • You must write properly (e.g., use proper grammar, follow the newspaper's style guide, make a compelling argument)
    • Note: Some verification activities are product-specific (e.g., the quality of the argument) and some are not (e.g., proper grammar)
Software Quality - History
Back SMYC Forward
  • Conformance to Specifications (Crosby, 1979):
    • Items that are verified (i.e., the reliability, robustness, and safety of the product)
  • Customer Satisfaction (Feigenbaum, 1983):
    • Items that are validated (i.e., the adequacy of the product)
History (cont.) - Boehm et al. (1978)
Back SMYC Forward
  • As-Is Utility:
    • Portability
    • Reliability
    • Efficiency
    • Human Engineering
  • Maintainability:
    • Testability
    • Understandability
    • Modifiability
History (cont.)
Back SMYC Forward
  • Garvin (1984):
    • Transcendental View - I know it when I see it
    • User View - fitness for purpose
    • Manufacturing View - conformance to specifications
    • Product View - tied to inherent product characteristics
    • Value-Based View - tied to what the customer is willing to pay
  • Garvin (1988):
    • Performance
    • Features
    • Reliability
    • Conformance
    • Durability
    • Serviceability
    • Aesthetics
    • Perceived Quality
The Modern Perspectives - ISO/IEC 25010
Back SMYC Forward
  • Functional Suitability:
    • Functional completeness, correctness, and appropriateness
  • Performance Efficiency:
    • Processing times and throughput rates, resources used, capacity
  • Compatability:
    • The degree to which the product can co-exist with and interoperate with other products
The Modern Perspectives - ISO/IEC 25010 (cont.)
Back SMYC Forward
  • Usability:
    • Understandability, learnability, and operability
  • Security:
    • Confidentiality, integrity, authentication, and accountability
  • Maintainability:
    • Modifiability/Adaptability and reusability
Software Products vs. Large-Scale Engineering Products
Back SMYC Forward
  • Examples:
    • Structures (bridges, buildings, dams)
    • Networks (e.g., roads, water, electrical, communication)
  • How is Quality Defined?
    • In very similar ways
Software Products vs. Large-Scale Engineering Products (cont.)
Back SMYC Forward
https://imgs.xkcd.com/comics/voting_software.png
(Courtesy of xkcd)
Software Products vs. Manufactured Products
Back SMYC Forward
  • Other Products:
    • After the product is designed and engineered it must be produced/manufactured and each instance is different
    • The concern is often with the quality of the instances of the product (sometimes called "quality control")
  • Software Products:
    • After the product is designed and engineered each instance is identical
    • The concern is with the quality of the product
Defect Elimination:
Back SMYC Forward
  • Defects:
    • Any undesirable aspect of a product
  • Defect Elimination Activities:
    • Prevention
    • Detection and Removal
Defect Prevention
Back SMYC Forward
  • Process Guides:
    • Standards and guidelines that describe the way everyone in the organization should behave
    • Templates and checklists that make it easier to do so
  • Analysis and Design Methodologies:
    • Well-codified approaches to understanding and solving software-related problems (e.g., OO)
  • Well-Studied Solutions:
    • Reference architecture
    • Design patterns
Defect Prevention (cont.)
Back SMYC Forward
  • Prototypes:
    • Throwaway/Exploratory
    • Evolutionary
  • Tools:
    • Version/configuration management
    • Computer-aided software engineering (CASE)
Defect Detection and Removal
Back SMYC Forward
  • Review and Correct:
    • Automated tools
    • Manual techniques (e.g., desk-checks, walk-throughs, inspections)
  • Test and Debug:
    • Testing is a validation and verification process that makes use of the system/product (including prototypes) while it is in operation or being operated on
An Abstract View of Testing
Back SMYC Forward

images/testing_abstract.gif
Quality of the Process
Back SMYC Forward
  • An Observation:
    • Many activities affect product quality
  • Questions One Might Ask:
    • When are we finding particular validation problems and can we find them earlier?
    • When are we finding particular verification problems and can we find them earlier?
  • Implications:
    • The quality of the process can be an important determiner of the quality of the product
Organizing for Quality
Back SMYC Forward
  • Root Causes of Quality Problems:
    • Management tends to blame workers
    • Workers tend to blame managers
  • Implications (Feigenbaum, 1983):
    • Quality shouldn't be the responsibility of a single department
    • Quality must be part of the entire process
Approaches/Methodologies/Standards
Back SMYC Forward
  • Quality Circles:
    • A small group that meets on a regular basis to consider quality issues for its "department" and make recommendations to management
  • Six Sigma (ISO 13053:2011):
    • Plan-Do-Check-Act (Demming)
    • Continuously monitor products and processes (engineering and management processes)
    • Focus on achieving measurable improvements and confirming them with statistical analyses
    • Well-Defined Rolls (Executives, Champions, Master Black Belts, Black Belts, Green Belts)
  • ISO 9001:
    • Must have a formal statement of a quality policy that is understood and applied at all levels
    • Must make decisions based on audited data
    • Must communicate with the customer and obtain feedback from the customer
Approaches/Methodologies/Standards (cont.)
Back SMYC Forward
  • Capability Maturity Model (CMM):
    1. Initial
    2. Repeatable (requirements management, project planning, project tracking)
    3. Defined (process focused, integrated management, peer reviews, coordination)
    4. Managed (quantitative)
    5. Optimizing (fault prevention, change management)
  • Software Process Improvement and Capability Determination (SPICE) - ISO15504:
    1. Performed informally
    2. Planned and tracked
    3. Well-defined
    4. Quantitatively controlled
    5. Continuously improving
The Quality Tradeoff
Back SMYC Forward
  • Costs of Low Quality Software:
    • Negligence and liability costs
    • Security costs
    • Disgruntled customers (hence lost sales)
  • Costs of High Quality Software:
    • Prevention costs
    • Appraisal costs
    • Delays to market (hence lost sales)
The Quality Tradeoff (cont.)
Back SMYC Forward
https://imgs.xkcd.com/comics/code_lifespan.png
(Courtesy of xkcd)
There's Always More to Learn
Back -