Quality

QA, QC

and

Testing


Darrow Cole - Aug 30, 2016

Testing


The process of exercising a
Thing
and watching for failure

  • Our "Thing" is Software
  • Testing helps us deliver quality software
  • Testing predates software

Testing is Mature


Many standards and techniques are in use

  • ISO - International Organization for Standardization
  • ASME - American Society of Mechanical Engineers
  • IEEE - Merging with ISO
  • Deming - Plan-Do-Check-Act

Testing in Perspective


Today, Testing falls under Quality

  • Testing is one component of Quality
  • and, Quality is ...

Quality


Two main categories

  • Quality Assurance
    • Planning, process management to prevent defects in the product: proactive
  • Quality Control
    • Activity to ensure quality in the product through testing and other activity: reactive

Mixed with V&V


QA and QC assist in determing

  • Verification
    • Does the system meet the requirements?
  • Validation
    • Does the system meet the needs of the user?

Detailed V&V


  • ASC
    • Verification: The process of confirming that a computer code correctly implements the algorithms that were intended
    • Validation: The process of confirming that the predictions of a code adequately represent measured physical phenomena
  • AIAA
    • Verification: The process of determining that a model implementation accurately represents the developer’s conceptual description of the model and the solution to the model
    • Validation: The process of determining the degree to which a model is an accurate representation of the real world from the perspective of the intended uses of the model
  • IEEE
    • Verification: (1) The process of evaluating a[software] system or component to determine whether the products of a given development phase satisfy the conditions imposed at the start of that phase. (2) Formal proof of program correctness.
    • Validation: The process of evaluating a [software] system or component during or at the end of the development process to determine whether it satisfies specified requirements.

Quality


Two main categories

Rt-Clk to download this chart

 
Your browser does not support SVG

End-to-End Testing


Where does it fit?

  • Dynamic (code is executed)
  • Functional (Black Box)
  • System (Does is meet specs?)
  • Falls into the Verification of V&V

More End-to-End


What is End-to-End Again?

Testing of the complete application environment in a way that mimics real-world useage such as interacting with the GUI, database and application functionality.

Test Plan


Next Step - Test Plan

The test plan should include:

  • Scope
  • Features to be tested
  • Features not being tested
  • Metrics
  • Deliverables ... and more!

STLC Methodology


Software Testing Life Cycle

The test plan is part of the STLC. See Software development process for more details.

Finally, Our Testing


What to test?

User story: A plain English description of what a user does, or needs to do, to achieve a specified result.

Our Testing S/W


End-to-End Testing

  • The Behat framework is at the core
  • Extended with
    • in-house php coding
    • Laravel framework and GUI

Basic Beehat


Single User Mode

  • Installed locally
  • Manual configuration
  • Run on the command line
  • Read results from the command line

More Basic Beehat


Installation

Giving Beehat Work


Simple User Story

  • Feature: Find Library Hours
  • Background:
    • Given I go to "https://www.library.ucla.edu"
    • Then I follow "Hours"
    • Then the url should match "/hours"

Beehat Output


Output for User Story

  • 1 scenario (1 passed)
  • 3 steps (3 passed)
  • 0m8.18s (13.07Mb)

Extended Framework


Eases Larger Projects

  • Authoring
  • Execution
  • Reporting

Authoring


Templating

  • Given I go to [/home]
  • Then I follow [/hours_link]
  • Then the url should match [/hours_url]

Execution


GUI

  • Specify groups of tests
  • Individualized config files
  • Results display

Reporting


MySQL and Jira

  • All results kept in db
  • Select results sent to Jira
  • Summary displayed in Confluence

Resources


Project Documentation

Available, tidying up in progress

Your browser does not support SVG
Your browser does not support SVG