Topic: incremental testing

topics > computer science > programming > Group: testing

Cleanroom software development
error safe systems
incremental development
incremental execution
execution with program stubs
testing testing


An incrementally developed program needs to be incrementally tested or the advantages of careful design will be lost. Incremental testing is made difficult or impossible when small units can not be isolated, when small units can not be directly tested, when dependency relationships are largely implicit, or when a module depends on many other modules. Incremental testing depends on certified testing of all component procedures, and control of all possible data inputs. Incremental testing may include syntactic checks of newly edited text, re-executing procedures with new values, or executing loops with several values. Direct execution of procedures simplifies incremental testing. Such an approach is used in Forth, Thesa, and several interpreted systems. (cbb 5/80)
Subtopic: test often up

Quote: test-driven development decreases the interval between writing tests and production code to minutes; write test before code, write code to pass test, repeat [»martRC5_2007]
Quote: code a little, test a little; test early, often, and automatically; bugs easier to fix [»huntA_2000]

Subtopic: separately testable modules up

Quote: using routines from untested modules can lead to systems that only work when everything works; e.g., a task scheduler that depends on the file system [»parnDL5_1978]
Quote: if write a system as a set of programs must understand and verify each part without knowing or using the details of the other parts [»parnDL10_1976]
Quote: components must rely on the correctness of any module it gets data from [»jackMA_1975]
Quote: a good design is made of building blocks which are separately testable [»postRM5_1987]
Quote: with frames, most errors are in the customized specification frames because the underlying frames are robust [»bassPG7_1987]

Subtopic: uses hierarchy up

Quote: 'A uses B' if the correct functioning of A depends on a correct implementation of B [»parnDL5_1978]
Quote: can test and use subsets of a level in the 'uses' hierarchy
Quote: each level of a 'uses' hierarchy is a testable and usable subset of the system [»parnDL5_1978]
Quote: unnecessary uses of routines and data from other modules increases the connectivity of a structure without improving its performance [»parnDL8_1971]

Subtopic: instruction testing up

Quote: with FORTH, one can execute any instruction from the terminal and observe its behavior; good for interactive program check-out [»bailGV8_1977]
Quote: use the command history to test previous commands with different values [»teitW3_1977]
Quote: no drivers needed for unit testing with interactive interpreters; just type a call to the function [»tichWF11_1987]

Subtopic: step through code up

Quote: the best way to catch errors is to step through the code immediately after writing or changing the code [»maguS_1993]
Quote: it is easy to simulate failures when stepping through code
Quote: step through critical code at the assembly language level

Subtopic: automated tests up

Quote: write automated tests for every manually identified bug; the same problem will arise again [»huntA_2000]

Subtopic: problems with unit testing up

Quote: unit verification by debugging may cause design faults when the fixed modules are combined [»cobbRH11_1990]
Quote: functional verification leaves 2-5 fixes/Kloc for later phases compared to 10-30 fixes/Kloc from unit testing [»cobbRH11_1990]
Quote: an adequate test of a program is not necessarily adequate for its components
Quote: antidecomposition of testing; even though a program is adequately tested, its components may not be adequately tested [»weyuEJ6_1988]

Subtopic: typestate avoids system testing up

Quote: NIL's typestate interfaces meant that system testing did not reveal new errors in unit-testing; locality of errors was assured [»stroRE5_1985]

Subtopic: syntactic error checking up

Quote: TOPD could check code fragments for syntactic correctness; quite useful [»hendP9_1975]
Quote: the compiler immediately detects and highlights errors in edited text; e.g., marks uses of undeclared variables [»teitT6_1979]

Subtopic: testing loops up

Quote: a few repetitions of loops and recursion will show errors; like mathematical induction
[»sattEH5_1975, OK]

Related Topics up

Topic: Cleanroom software development (38 items)
Topic: error safe systems (76 items)
Topic: incremental development (74 items)
Topic: incremental execution (22 items)
Topic: execution with program stubs (5 items)
Topic: testing testing
(13 items)

Updated barberCB 7/04
Copyright © 2002-2008 by C. Bradford Barber. All rights reserved.
Thesa is a trademark of C. Bradford Barber.