Group:
software maintenance
Topic:
execution profile
Topic:
incremental testing
Topic:
management of large software projects
Topic:
quality assurance
Topic:
software metrics
Topic:
test data selection
Topic:
testing by program mutation
Topic:
statistical testing based on a usage profile
| |
Summary
It is difficult to measure test progress or test completeness. Without objective measures, testing is usually declared finished when time or money runs out. Several measures have been suggested: mean time between detected errors, bebugging (does the test data catch mutated programs), and counting unexercised statements. If certified, incremental testing is possible, then modules-to-be-tested is an accurate measure. (cbb 5/80)
Subtopic: adequate testing 
| Quote: an adequate test of a program is not necessarily adequate for its components
| | Quote: the adequacy of test data may be based on the program or on its specification [»weyuEJ6_1988]
| | Quote: derived test cases from formal specification; tested each precondition and postcondition; also white-box testing; only three faults during operation; acceptance testing found cosmetic faults [»tretJ9_2001]
|
Subtopic: path coverage
| Quote: check sheets should test every instruction of the program and all points of difficulty [»turiA3_1951]
| | Quote: statement and branch adequacy satisfies only five of the eight axioms; need mutation or modified size adequacy [»weyuEJ12_1986]
| | QuoteRef: kosyDK_1973 ;;87 Knuth and others: keep count of statement execution so check all paths.
| | Quote: modified size adequacy subsumes branch adequacy and mutation adequacy; such test sets distinguish a program from most other programs [»weyuEJ12_1986]
| | Quote: axioms of test data adequacy include testable, nontrivial, coverage, complete [»weyuEJ12_1986]
|
Subtopic: statistical tests
| Quote: measure testing effectiveness by error days, i.e., the number of days from creation of an error to its detection [»millHD_1979]
|
Subtopic: mutation tests
| Quote: use embedded test for self-testable components; evaluate quality with selective mutations [»jazeJM7_2001]
| | Quote: SWAT identified all 34 injected memory leaks (and a real leak) in a large interactive web application; identified 6 leaks in beta code [»chilTM10_2004]
|
Subtopic: problems wtih testing
| Quote: program testing can not show the absence of bugs [»dijkEW10_1972]
| | QuoteRef: cheaTE_1976 ;;48 can easily verify that a bridge is 95% complete but it is very difficult to do the same for an operating system
|
Related Topics
Group: software maintenance (14 topics, 368 quotes)
Topic: execution profile (43 items)
Topic: incremental testing (26 items)
Topic: management of large software projects (63 items)
Topic: quality assurance (22 items)
Topic: software metrics (32 items)
Topic: test data selection (39 items)
Topic: testing by program mutation (18 items)
Topic: statistical testing based on a usage profile
(27 items)
|
