Group:
debugging
Topic:
automated tests of specifications and designs
Topic:
data-driven design
Topic:
debugging by usage rules
Topic:
debugging techniques
Topic:
model checker
Topic:
test hardware
Topic:
test data selection
Topic:
test scripts
Topic:
testing by program mutation
Topic:
testing by voting or N-version
| |
Subtopic: regression test 
| Quote: PL.8 optimizer bugs produce catastrophic results since optimizer applies to all code; regression test of 150 self-checking programs [»auslM6_1982]
| | Quote: need systematic regression testing on both distributing and receiving sites; can automate acceptance test; more reliable under change [»nowiDA8_1978]
|
Subtopic: crash diagnosis
| Quote: automatic diagnosis of Windows XP failures; 83% third-party failures, 12% hardware, and 5% Microsoft bugs [»murpB11_2004]
|
Subtopic: automated test generation
| Quote: generate SQL by random parse trees and delayed decisions; propagate state information downwards and decisions upwards [»slutD8_1998, OK]
| | Quote: must automate the generation of SQL tests; manually written tests cover a tiny fraction of the SQL input domain [»slutD8_1998, OK]
| | Quote: Cleanroom estimates the user input distribution to generate test cases
| | Quote: generate test cases by selecting operations randomly according to the operational profile and input states randomly with its domain [»musaJD3_1993]
| | Quote: TOPD's checker generates test cases from the model and determines if the procedure's result matches the model's result [»hendP9_1975]
|
Subtopic: usage rules
| Quote: find bugs by automatic extraction of usage rules; as templates (beliefs) with few contradictions [»englD10_2001]
| | Quote: use finite state model to check proper usage of uid-setting system calls; build a finite state model of the program; check for privileged regions [»chenH8_2002]
|
Subtopic: test bus
| Quote: over a century ago, telephone companies added a test bus to its phone switches; the built-in test access allowed automatic, nightly testing of every phone line; problems found and fixed before subscribers noticed [»martRC7_2005]
| | Quote: a software test bus is an API for unit and acceptance tests, using the same APIs as the user interface [»martRC7_2005]
|
Subtopic: test case reduction
| Quote: automated testing should include automated test case simplification [»zellA2_2002]
| | Quote: use Delta Debugging to help identify bugs tripped by a test; e.g., reduced an HTML page to a single line [»zellA11_2001]
| | Quote: automatic simplification of erroneous SQL; helped debug problems [»slutD8_1998, OK]
|
Subtopic: database testing
| Quote: use small, canned, in-memory databases for testing; create database just before test execution; faster, known content, repeatable, self-contained [»martRC7_2005]
|
Subtopic: random input
| Quote: wynot minimized the random input that caused a crashed program; crash identification by stack trace [»zellA2_2002]
|
Subtopic: exceptions
| Quote: automated test of exception-safety; ThisCanThrow() throws an exception when a global counter becomes zero; rerun with increasing values of the counter; operations must be exception-neutral [»abraD4_1998]
|
Subtopic: compare multiple implementations
| Quote: automated, stochastic testing of SQL across multiple database managers; randomly generate SQL statements [»slutD8_1998, OK]
| | Quote: compare SQL queries across multiple vendors; avoid sorting by comparing row counts and column checksums [»slutD8_1998, OK]
| | Quote: example of comparing random SQL across multiple database systems; 0.07% had possible errors [»slutD8_1998, OK]
| | Quote: SQL comparison across database systems not effective for testing NULL, strings, and numeric type coercion; needed for portability [»slutD8_1998, OK]
|
Subtopic: sampling
| Quote: use sampling to identify predicates that are always true for a bug; called deterministic bugs [»liblB6_2003]
| | Quote: use adaptive profiling to identify memory leaks in long running programs; sample code segments inversely to execution frequency; a leak is a stale object that is not accessed; SWAT has a low false positive rate [»chilTM10_2004]
|
Subtopic: problems with automated UI testing
Quote: automatic testing of software through its UI is slow, opaque, and dangerous; even tiny changes can cause may tests to fail or become inoperable
[»martRC7_2005]
|
Related Topics
Group: debugging (10 topics, 333 quotes)
Topic: automated tests of specifications and designs (12 items)
Topic: data-driven design (41 items)
Topic: debugging by usage rules (41 items)
Topic: debugging techniques (23 items)
Topic: model checker (49 items)
Topic: test hardware (5 items)
Topic: test data selection (39 items)
Topic: test scripts (13 items)
Topic: testing by program mutation (18 items)
Topic: testing by voting or N-version
(10 items)
|
