Methodology:: Testing

From SK:: Wiki

1 General
2 Testing
3 Downloads
4 Your Comments

General

Warning If you think that you have discovered any copyright violation on page, please notify me immediately by e-mail: sergei@kostigoff.net. Thank you.

Status: Draft release.

Purpose

There are quite a lot of methodologies, books, articles, tools, etc. related to Testing. Goal of this section is not to repeat all of them nor provide an overview. Instead there are areas usually not enough highlighted from my point of view. Tests should be tested as well. In the section above there is a technique which allows estimate quality of the test during the testing (not quality of the product being tested).

Scope

There are "test-for-the-test" diagrams in the section.

Glossary and Abbreviations

None.

Testing

Testing:: Factors affecting test-case effectiveness

The Factors affecting test-case effectiveness are shown on Figure 1.

Fig.1: Testing:: Effectiveness Factors

There are 6 factors (in 3 groups) affecting test-case effectiveness:

Test-design phase factors
1. Incorrect test specifications
2. Incomplete test design
3. Incomplete test suite
Test-execution phase factors
1. Test execution problems
Test-planning phase factors
1. Incorrect functional specifications
2. Incomplete functional specifications

All these factors will reflect on test-case effectiveness.

Testing:: Test-Case Escape Classification

During the analysis of the test results one of tricks is classification of test-case escapes based on the logic shown on Figure 2.

Fig.2: Testing:: Test-Case Escape Classification

This will allow to analyze not only the quality of the product being tested but test and assumption quality during the test preparation.

By using algorithm shown on the Fig.2 all test escapes can be grouped into 6 groups corresponding to 6 factors above.

Testing:: Test Escape Analysis

Figure 3 gives an example of the test-case escape analysis. This is not a real example.

Fig.2: Testing:: Test-Case Escape Analysis

As we can see from the diagram, Test Escapes does not always indicate product fails, but caused by either incomplete functional specification, or by incorrect functional specification.

Test Escapes in the example shows importance of the Requirements Analysis at the beginning of the project.

By analysis of the Diagram shown on Figure 3 it is possible to significantly improve the Test Quality.

Reference

The above material is compiled from IT Professional Magazine published by IEEE

Downloads

Diagrams from the above Diagram Set could be downloaded in Zip format here:

testing_dia.zip

Diagrams are created in Dia format. For Unix/Linux users it is a standard diagrammer software. Windows users can obtain legal GPL-ed copy of DIA from http://dia-installer.de/index_en.html