Quality

QA, QC

and

Testing

Testing

The process of exercising a
Thing
and watching for failure

• Our "Thing" is Software
• Testing helps us deliver quality software
• Testing predates software

Testing is Mature

Many standards and techniques are in use

• ISO - International Organization for Standardization
• ASME - American Society of Mechanical Engineers
• IEEE - Merging with ISO
• Deming - Plan-Do-Check-Act

Testing in Perspective

Today, Testing falls under Quality

• Testing is one component of Quality
• and, Quality is ...

Quality

Two main categories

• Quality Assurance
  • Planning, process management to prevent defects in the product: proactive
• Quality Control
  • Activity to ensure quality in the product through testing and other activity: reactive

Mixed with V&V

QA and QC assist in determing

• Verification
  • Does the system meet the requirements?
• Validation
  • Does the system meet the needs of the user?

Detailed V&V

• ASC
  • Verification: The process of confirming that a computer code correctly implements the algorithms that were intended
  • Validation: The process of confirming that the predictions of a code adequately represent measured physical phenomena
• AIAA
  • Verification: The process of determining that a model implementation accurately represents the developer’s conceptual description of the model and the solution to the model
  • Validation: The process of determining the degree to which a model is an accurate representation of the real world from the perspective of the intended uses of the model
• IEEE
  • Verification: (1) The process of evaluating a[software] system or component to determine whether the products of a given development phase satisfy the conditions imposed at the start of that phase. (2) Formal proof of program correctness.
  • Validation: The process of evaluating a [software] system or component during or at the end of the development process to determine whether it satisfies specified requirements.

Quality

Two main categories

Rt-Clk to download this chart

End-to-End Testing

Where does it fit?

• Dynamic (code is executed)
• Functional (Black Box)
• System (Does is meet specs?)
• Falls into the Verification of V&V

More End-to-End

What is End-to-End Again?

Testing of the complete application environment in a way that mimics real-world useage such as interacting with the GUI, database and application functionality.

Test Plan

Next Step - Test Plan

The test plan should include:

• Scope
• Features to be tested
• Features not being tested
• Metrics
• Deliverables ... and more!

STLC Methodology

Software Testing Life Cycle

The test plan is part of the STLC. See Software development process for more details.

Finally, Our Testing

What to test?

User story: A plain English description of what a user does, or needs to do, to achieve a specified result.

Our Testing S/W

End-to-End Testing

• The Behat framework is at the core
• Extended with
  • in-house php coding
  • Laravel framework and GUI

Basic Beehat

Single User Mode

• Installed locally
• Manual configuration
• Run on the command line
• Read results from the command line

More Basic Beehat

Installation

• PHP 5.3+, Composer, Java, Selenium configuration
• Behat installation instructions

Giving Beehat Work

Simple User Story

• Feature: Find Library Hours
• Background:
  • Given I go to "https://www.library.ucla.edu"
  • Then I follow "Hours"
  • Then the url should match "/hours"

Beehat Output

Output for User Story

• 1 scenario (1 passed)
• 3 steps (3 passed)
• 0m8.18s (13.07Mb)

Extended Framework

Eases Larger Projects

• Authoring
• Execution
• Reporting

Authoring

Templating

• Given I go to [/home]
• Then I follow [/hours_link]
• Then the url should match [/hours_url]

Execution

GUI

• Specify groups of tests
• Individualized config files
• Results display

Reporting

MySQL and Jira

• All results kept in db
• Select results sent to Jira
• Summary displayed in Confluence

Resources

Project Documentation

Available, tidying up in progress

• Confluence
• Jira
• Github