Topic: testing testing

topics > computer science > programming > Group: testing

software maintenance

execution profile
incremental testing
management of large software projects
quality assurance
software metrics
test data selection
testing by program mutation
statistical testing based on a usage profile


It is difficult to measure test progress or test completeness. Without objective measures, testing is usually declared finished when time or money runs out. Several measures have been suggested: mean time between detected errors, bebugging (does the test data catch mutated programs), and counting unexercised statements. If certified, incremental testing is possible, then modules-to-be-tested is an accurate measure. (cbb 5/80)
Subtopic: adequate testing up

Quote: an adequate test of a program is not necessarily adequate for its components
Quote: the adequacy of test data may be based on the program or on its specification [»weyuEJ6_1988]
Quote: derived test cases from formal specification; tested each precondition and postcondition; also white-box testing; only three faults during operation; acceptance testing found cosmetic faults [»tretJ9_2001]

Subtopic: path coverage up

Quote: check sheets should test every instruction of the program and all points of difficulty [»turiA3_1951]
Quote: statement and branch adequacy satisfies only five of the eight axioms; need mutation or modified size adequacy [»weyuEJ12_1986]
QuoteRef: kosyDK_1973 ;;87 Knuth and others: keep count of statement execution so check all paths.
Quote: modified size adequacy subsumes branch adequacy and mutation adequacy; such test sets distinguish a program from most other programs [»weyuEJ12_1986]
Quote: axioms of test data adequacy include testable, nontrivial, coverage, complete [»weyuEJ12_1986]

Subtopic: statistical tests up

Quote: measure testing effectiveness by error days, i.e., the number of days from creation of an error to its detection [»millHD_1979]

Subtopic: mutation tests up

Quote: use embedded test for self-testable components; evaluate quality with selective mutations [»jazeJM7_2001]
Quote: SWAT identified all 34 injected memory leaks (and a real leak) in a large interactive web application; identified 6 leaks in beta code [»chilTM10_2004]

Subtopic: problems wtih testing up

Quote: program testing can not show the absence of bugs [»dijkEW10_1972]
QuoteRef: cheaTE_1976 ;;48 can easily verify that a bridge is 95% complete but it is very difficult to do the same for an operating system

Related Topics up

Group: software maintenance   (14 topics, 368 quotes)

Topic: execution profile (43 items)
Topic: incremental testing (26 items)
Topic: management of large software projects (63 items)
Topic: quality assurance (22 items)
Topic: software metrics (32 items)
Topic: test data selection (39 items)
Topic: testing by program mutation (18 items)
Topic: statistical testing based on a usage profile
(27 items)

Updated barberCB 2/05
Copyright © 2002-2008 by C. Bradford Barber. All rights reserved.
Thesa is a trademark of C. Bradford Barber.