Topic: software review

topics > computer science > programming > Group: testing

program design
requirement specification
software engineering
software maintenance

change notification
Cleanroom software development
consistency testing
debugging by reading code
design documentation
design errors
management of large software projects
programming style
programming without errors
quality assurance
software management
specification errors
team programming
usability errors

Subtopic: advantages up

Quote: a design review produces a list of ideas, objections, concerns, and issues; produce a plan for addressing the issues and resolving all open items [»colwRP_2006]
Quote: program reviews improve quality, educate new programmers, provide overall view, and assist management review [»kralTM5_1975, OK]
Quote: inspections make code easier to understand and change; 60% of issues concerned maintainability; e.g., coding standards, comments, portability, documentation [»siyH12_2001]
Quote: a technical review tests the reviewed material, provides technical and leadership training, communicates standards, and develops community spirit [»weinGM9_1982]
Quote: need a rigorous review after programming and before debugging; often caused major improvements or revised design strategies [»corbFJ_1979]
Quote: review a program before compilation [»hoarCA_1974]
Quote: code inspection works for any product with well-defined completion criteria; successfully used for hardware development [»russGW1_1991]
Quote: visually inspect every bit of analysis and code; most errors are obvious [»roycWW8_1970]

Subtopic: inspection vs. testing up

Quote: inspect code before unit test; better motivation, less disruption, may catch design errors, may skip unit test [»wellEF9_1993]
Quote: don't test before code inspections; inspections are more productive; inspection requires a motivated team rewarded are detected errors [»russGW1_1991]
Quote: a design review and code inspection after testing can find many more errors than the testing itself [»joneTC4_1979]
Quote: code reading better than functional or structural testing for detecting software faults [»basiVR12_1987]
Quote: code inspections are two to four times more effective than formal designer testing and system testing [»russGW1_1991]
Quote: testing is better than code inspection for errors due to execution, timing, traffic, load, system interactions [»russGW1_1991]

Subtopic: meetings vs. preparation up

Quote: meetingless inspections finds more defects than those with meetings [»portA11_1997]
Quote: a low preparation rate (<200 lines/hour) is about 20% better than a high one [»wellEF9_1993]
Quote: when reviewing software, prefer one-on-one discussions instead of protracted group discussions [»parnDL8_1985]
Quote: found meetingless inspections better than those with meetings; very few defects for more often with meetings [»perrDE7_2002]
Quote: a second individual review is more effective than a review meeting in locating defects [»portAA3_1997]
Quote: two-sessions were no better than single session reviews and took twice as long [»portAA6_1997]
Quote: one reviewer plus author found fewer true defects than two- or four-reviewer teams
Quote: review meetings located 30% of the true defects; the remainder were found during preparation; like Humphrey, while Votta studied design inspections [»portAA6_1997]

Subtopic: review materials up

Quote: use HyperCode to reduce inspection delays; annotated, diff-marked code listings; online, automatically generated, no meetings [»perrDE7_2002]
Quote: excellent acceptance of HyperCode inspections; automatically generated reports, more defects reported, better defect descriptions with discussion, shorter meetings, widely used [»perrDE7_2002]
Quote: typeset drafts appear to get more thorough reviews then rough drafts [»joneTC4_1979b]
Quote: a module design document records the major design decisions for design reviews and documentation [»parnDL2_1986]
Quote: include all design alternatives in the documentation along with why they were considered and why they were rejected [»parnDL2_1986]

Subtopic: inspection team up

Quote: design and code inspection teams with moderator, designers, coders, and testers
Quote: formal design reviews with 10 to 20 other interested designers and observers [»colwRP_2006]
Quote: can use a librarian with software reviews; all communication (review kits) in writing through the librarian [»mcgoCL_1979]
Quote: 4-person inspection teams more than twice as effective as 3-person teams; better product knowledge, code producer less prominent [»wellEF9_1993]
Quote: random selection of software review treatments: 1,2, or 4 reviewers with 1 or 2 review sessions or 2 reviews with defect repair; all reviewers prepared ahead of time

Subtopic: testers up

Quote: testers are effective reviewers of formal design specifications; they use the specifications to write their tests

Subtopic: how to review up

Quote: in a design review describe the context, function, constraints, alternatives, block diagram, timing diagrams, protocols, and controllers [»colwRP_2006]
Quote: do not judge the validity of an issue during a review; all issues deserve attention [»colwRP_2006]

Subtopic: assumption-driven review up

Quote: when reviewing consistency between assumptions and functions, ask which assumptions allow a function's implementation [»parnDL8_1985]
Quote: each module includes basic assumptions which will not change and assumptions about correct usage [»parnDL8_1985]
Quote: design errors: inconsistencies, inefficiencies, ambiguities, inflexibilities [»parnDL8_1985]

Subtopic: questionnaire-driven review up

Quote: review software by asking questions; use the documents under review; e.g., write a program by reading the documentation [»parnDL8_1985]
Quote: focused design reviews on one type of error with questionnaires answered from the design documentation [»parnDL8_1985]
Quote: an active review consists of questions to be answered using the documentation; mostly individual work; generated a wealth of comments [»hoffDM_2001]

Subtopic: paraphrase inspection up

Quote: code inspections find the most errors at 150 lines of code paraphrase per hour; 4 times worse at 750 lines per hour [»russGW1_1991]
Quote: code inspection by paraphrasing it line by line; for detecting errors; use moderator, inspectors, author, two hour reviews [»russGW1_1991]
Quote: during a review, one reviewer paraphrases the code, one reviewer is the moderator, and the author compiles the master list of defects

Subtopic: scenario inspection up

Quote: scenarios better for software reviews than checklists and ad hoc reviews; checklists did not help [»perrDE7_2002]
Quote: reviewers that follow a scenario for specific classes of errors found more defects than ad hoc or checklist reviewers [»portA11_1997]

Subtopic: use case inspection up

Quote: inspection by usage-based reasoning validates design and requirement documents by manually executing high priority use cases [»thelT8_2003]
Quote: experiment found usage-based reasoning better than checklist-based reading; better for all faults, critical-to-user faults, and time to first fault [»thelT8_2003]

Subtopic: mental execution up

Quote: in checking a program, the program is de-programmed to see if it re-creates the original specification [»greeTR_1980]
Quote: mentally verify clear boxes by verifying a small number of correctness conditions; can check unanimously in a few seconds [»lingRC5_1993]
Quote: check sheets will catch most errors if performed conscientiously; forget purpose and method of the machine to avoid anticipating a result [»turiA3_1951]
Quote: use desk checking to verify that subroutines occupy distinct locations, specifications are satisfied, overwrites do not occur, and unpreserved registers are invalid [»wilkMV_1951]
Quote: algebraic notation makes programs easy to read and mentally check; reduces programming error and increases productivity [»glenAE2_1953]

Subtopic: module inspection up

Quote: with displays, a program made of many short programs can be checked one display at a time
Quote: used displays to inspect safety-critical software for a nuclear plant [»parnDL12_1994]
Quote: the tabular organization of displays allowed a step-by-step review process
Quote: checked that the displays satisfied the requirements document by transforming the display's tables into the corresponding requirement tables [»parnDL12_1994]
Quote: the decomposition phase of the review process should be communicated by the designer; the reviewer should not need to derive the decomposition [»parnDL12_1994]

Subtopic: error-prone module up

Quote: during code inspection, concentrate on error-prone modules [»fagaME3_1976, OK]

Subtopic: size of change up

Quote: review software according to change sizes; under 10 lines, 10-50, and over 50 lines [»puruR6_2005]

Subtopic: faults detected by inspection up

Quote: extra and missing statements account for almost half of all errors found by code inspections
Quote: issues reported across all code inspections: 22% false positives, 60% readability or coding standards, 18% true defects and inefficiencies; studied true defects [»portAA6_1997]

Subtopic: effectiveness of inspection up

Quote: Apache developers do not use QA testing; review code before or after checkin; most core developers review all changes [»mockA7_2002]
Quote: code inspections of 2.5 million lines of code detected about 1 defect per person-hour over all activities [»russGW1_1991]
Quote: catch 20% of bugs in code inspection; half of these from designing the test cases, half from the inspection [»yamaT11_1998]
Quote: in Cleanroom, used intensive team reviews to verify two formal grammars of 1500 productions each; no grammar errors found in field testing [»lingRC10_1988]
Quote: using function-theoretic design verification, a team review checked 1200 correctness questions in a few days; astonishing savings during test [»lingRC10_1988]
Quote: via functional verification inspections, the IBM Wheelwriter had no software errors after 6 years; 63 Kloc of code on three processors

Subtopic: inspection training up

Quote: BNR uses a self-paced video and workbook for teaching code inspections; includes good and bad meetings [»russGW1_1991]

Subtopic: examples up

Quote: a typical release of the 5ESS switch requires 1,500 inspections with four or more participants
Quote: study of 130 software reviews during the development of 55K new lines of C++ code; managed and observed by an inspection quality engineer [»portAA6_1997]

Subtopic: cost of inspection up

Quote: a 20 KLOC program takes 4-6.7 person-years to develop and about 1 person-year for code inspection [»russGW1_1991]
Quote: run 10 test cases per day during code inspection, and 25 per day during machine debugging

Subtopic: problems with inspection up

Quote: traditional design inspection uses mental program execution; needs long term memory, non-local reasoning, virtually infinite number of paths [»lingRC10_1988]
Quote: software reviews tend to go through code too quickly, lack focus on a single objective, and do not benefit from team synergy [»russGW1_1991]
Quote: the process has little effect on software inspections; most variation due to the reviewers, the code size, and its functionality [»portA11_1997]
Quote: conventional design reviews are often incomplete due to information overload [»parnDL8_1985]
Quote: reviewers can miss program flaws even though they know that flaws have been introduced [»parnDL6_1990]
Quote: software inspection is a double pain; requires documentation beforehand and inspection itself is painful; easily dropped [»berrDM10_2002]
Quote: check sheets miss errors due to preconceived ideas, insufficient test cases, not checking alterations, or alternating check sheet but not the program

Related Topics up

Group: program design   (13 topics, 454 quotes)
Group: requirement specification   (11 topics, 307 quotes)
Group: software engineering   (18 topics, 470 quotes)
Group: software maintenance   (14 topics, 368 quotes)

Topic: bugs (66 items)
Topic: change notification (19 items)
Topic: Cleanroom software development (38 items)
Topic: consistency testing (60 items)
Topic: debugging by reading code (11 items)
Topic: design documentation (43 items)
Topic: design errors (15 items)
Topic: management of large software projects (63 items)
Topic: programming style (47 items)
Topic: programming without errors (28 items)
Topic: quality assurance (22 items)
Topic: software management (28 items)
Topic: specification errors (10 items)
Topic: team programming (7 items)
Topic: usability errors
(6 items)

Updated barberCB 1/06
Copyright © 2002-2008 by C. Bradford Barber. All rights reserved.
Thesa is a trademark of C. Bradford Barber.