Topic: testing by program mutation

topics > computer science > programming > Group: testing

automated testing
sensitivity of software to change
testing testing
Subtopic: mutation testing, bebugging up

Quote: SWAT identified all 34 injected memory leaks (and a real leak) in a large interactive web application; identified 6 leaks in beta code [»chilTM10_2004]
Quote: if test data detects all simple mutations of a program, it also detects complex combinations [»buddTA_1980]
Quote: test data is adequate if it detects simple mutations of a program [»buddTA_1978]
QuoteRef: gilbT_1977 ;;28 bebugging-- automated insertion of known bugs in order to determine bug density and to test debugging techniques. presumably if 50% of bebugged bugs found than still 50% of real bugs around.
QuoteRef: gilbT_1977 ;;32 can measure bugs by artificially inserting a number of bugs and seeing how many of those bugs are found -- this should indicate total # of bugs

Subtopic: reliability testing up

Quote: tested reliability under system crash by injecting faults; random bit flips in kernel; imitate programming errors such as pointer corruption, copy overrun, off-by-one; most crashes happened within 15 seconds [»chenPM9_1996]
Quote: use embedded test for self-testable components; evaluate quality with selective mutations [»jazeJM7_2001]

Subtopic: mutation implementation up

Quote: one program mutation is replacing a binary operator with another [»buddTA_1980]
Quote: a program mutation system with 23 mutant operators; size = reference count * data items [»buddTA_1980]
Quote: mutation testing in PIMS by replacing constants, deleting statements, etc.; see if test data catches them [»buddTA_1978, OK]
Quote: test tolerance to unusual events by fault injections and a test for unsafe or unacceptable outcomes; e.g., change a variable's value [»voasF7_1997]

Subtopic: studies of mutation testing up

Quote: testing a mutant system successfully caught 80% of the errors [»buddTA_1980]
Quote: contract requirement: 90% of bebugged errors fixed in half an hour or less [»pariG_1980]

Subtopic: software review of mutations up

Quote: reviewers can miss program flaws even though they know that flaws have been introduced [»parnDL6_1990]

Subtopic: problems with mutation testing up

Quote: mutation testing assumes that incorrect programs are almost correct; only simple errors [»buddTA_1978]
Quote: mutation and perturbation testing are good for catching errors of commission but poor for errors of omission [»howdWE1_1990]
Quote: try to beat a mutation system by finding correct test data on which mutants of an incorrect program fail [»buddTA_1980]
Quote: a randomly altered program rarely serves a useful function; unsuited to evolution [»conrM5_1985]

Related Topics up

Topic: automated testing (25 items)
Topic: sensitivity of software to change (44 items)
Topic: testing testing
(13 items)

Updated barberCB 2/05
Copyright © 2002-2008 by C. Bradford Barber. All rights reserved.
Thesa is a trademark of C. Bradford Barber.