RSS Feed

Software Testing

What is not Software Testing?

What is Software Testing?

Overview of Test Methods

Static tests
The software is not executed but analyzed offline. In this category would be code inspections (e.g. Fagan inspections), Lint checks, cross reference checks, etc.

Dynamic tests
This requires the execution of the software or parts of the software (using stubs). It can be executed in the target system, an emulator or simulator. Within the dynamic tests the state of the art distinguishes between structural and functional tests.

Structural tests
These are so called "white-box tests" because they are performed with the knowledge of the source code details. Input interfaces are stimulated with the aim to run through certain predefined branches or paths in the software. The software is stressed with critical values at the boundaries of the input values or even with illegal input values. The behavior of the output interface is recorded and compared with the expected (predefined) values.

Functional tests
These are the so called "black-box" tests. The software is regarded as a unit with unknown content. Inputs are stimulated and the values at the output results are recorded and compared to the expected and specified values.

Test in Stages

The various tests are able to find different kinds of errors. Therefore it is not enough to rely on one kind of test and completely neglect the other. E.g. white-box tests will be able to find coding errors. To detect the same coding error in the system test is very difficult. The system malfunction which may result from the coding error will not necessarily allow conclusions about the location of the coding error. Test therefore should be progressive and supplement each other in stages in order to find each kind of error with the appropriate method.

Module test
A module is the smallest compilable unit of source code. Often it is too small to allow functional tests (black-box tests). However it is the ideal candidate for white-box tests. These have to be first of all static tests (e.g. Lint and inspections) followed by dynamic tests to check boundaries, branches and paths. This will usually require the employment of stubs and special test tools.

Component test
This is the black-box test of modules or groups of modules which represent certain functionality. There are no rules about what can be called a component. It is just what the tester defined to be a component, however it should make sense and be a testable unit. Components can be step by step integrated to bigger components and tested as such.

Integration test
The software is step by step completed and tested by tests covering a collaboration of modules or classes. The integration depends on the kind of system. E.g. the steps could be to run the operating system first and gradually add one component after the other and check if the black-box tests still run (the test cases of course will increase with every added component). The integration is still done in the laboratory. It may be done using simulators or emulators. Input signals may be stimulated.

Software / System test
This is a black-box test of the complete software in the target system. The environmental conditions have to be realistic (complete original hardware in the destination environment).

Which Test finds which Error?

Possible error Can be best found by Example
Syntax errors Compiler, Lint Missing semicolons, Values defined but not initalized or used, order of evaluation disregarded.
Data errors Software inspection, module tests Overflow of variables at calculation, usage of inappropriate data types, values not initialized, values loaded with wrong data or loaded at a wrong point in time, lifetime of pointers.
Algorithm and logical errors Software inspection, module tests Wrong program flow, use of wrong formulas and calculations.
Interface errors Software inspection, module tests, component tests. Overlapping ranges, range violation (min. and max. values not observed or limited), unexpected inputs, wrong sequence of input parameters.
Operating system errors, architecture and design errors Design inspection, integration tests Disturbances by OS interruptions or hardware interrupts, timing problems, lifetime and duration problems.
Integration errors Integration tests, system tests Resource problems (runtime, stack, registers, memory, etc.)
System errors System tests Wrong system behaviour, specification errors