Topic: safety critical systems

topics > engineering > Group: process control

program proving

automated tests of specifications and designs
backup processor
collision avoidance by robots
defensive programming
embedded systems
error checking in robot programming
error safe systems
formal methods and languages
hard real time systems
logging data and events
operating system security
power fail recovery
preventing accidental errors
programming without errors
resourceful, redundant systems for reliability
testing by voting or N-version
type-safe and secure languages

Subtopic: reliability up

Quote: safety is a quality of the system in which the software is used; it is not a quality of the software itself; reused software may be unsafe [»leveNG7_1993]
Quote: designers of safety-critical systems must convince themselves, the customer, and certification authorities that the design and implementation are correct
Quote: automated control needed for a highly reliable storm surge barrier; humans have one in a thousand failure probability for complex tasks [»tretJ9_2001]
Quote: use one pattern for reporting errors and recovering failed processes
Quote: over half of system for simulators, test systems, and supporting software
Quote: companies building hazardous equipment should log, track, and analyze all safety-related failures and accidents [»leveNG7_1993]
Quote: should certify software engineers for safety-critical software [»leveNG7_1993]

Subtopic: coding rules up

Quote: coding rules for safety-critical applications in C; e.g., simple control flow, short functions, assertions, check args, avoid pointers, no warnings [»holzGJ6_2006]
Quote: for safety-critical code, do not use dynamic memory allocation after initialization
Quote: all loops should have a fixed upper bound

Subtopic: testing up

Quote: derived test cases from formal specification; tested each precondition and postcondition; also white-box testing; only three faults during operation; acceptance testing found cosmetic faults [»tretJ9_2001]

Subtopic: synchronous language up

Quote: use synchronous languages for real-time, embedded, safety critical applications [»benvA1_2003]

Subtopic: reliable despite failure up

Quote: how do engineers create reliable structures from imperfect mechanisms [»demiRA5_1979]
Quote: a process control system may lose major subsystems; must still be able to operate remotely or manually [»instrumentcontrol]

Subtopic: examples up

Quote: for digital flight-control use synchronous channels, sensor data to all channels, and exact-match majority voting; needs fault-tolerant clock synchronization, etc. [»rushJ12_1991]
Quote: validated BOS's communication protocol with PROMELA and SPIN; identified an unsafe situation where only one door closed; intuitive chart of message sequence [»tretJ9_2001]
Quote: study of the software-related accidents of the Therac-25; six over two years, including death and serious injury [»leveNG7_1993]
Quote: review of literature on formal models of Java safety; applications to smartcards; need better models [»hartPH12_2001]

Subtopic: problems with reliability up

Quote: very low safety probabilities seem hard to justify; do not rely on them [»leveNG7_1993]
Quote: a reliability model for ultrareliable software requires independent components; not reasonable for software/hardware design flaws [»butlRW12_1991]
Quote: failure rates of ultrareliable software can not be quantified because of interdependent software or hardware design flaws [»butlRW1_1993]

Subtopic: problems with N-version software up

Quote: N-version programs have coincident failures greatly exceeding that expected by chance; modest gain in reliability [»eckhDE7_1991]
Quote: coincident failures in N-version programs often due to common misunderstandings of a specification; exact fault differs [»eckhDE7_1991]
Quote: N-version voting was only partially effective at detecting failures due to a fault

Subtopic: problems with software up

Quote: hardware failure modes are more limited than software failures, so hardware interlocks should still be used
Quote: a common mistake is to put too much confidence in software; design errors are hard to find and eliminate [»leveNG7_1993]
Quote: no independent checks of Therac-25 malfunctioning; only patient reactions [»leveNG7_1993]
Quote: Therac-25 could not detect massive overdoses; the ion chambers were saturated and indicated low dosage
Quote: software errors can always be attributed to transient hardware errors [»leveNG7_1993]
Quote: with a software-controlled system, a program sequence error can be catastrophic [»instrumentcontrol]
Quote: a programmer on hearing that a dam had broke spent two days trying to verify his program; it wasn't his dam that failed, but he found many errors [»meerL_1978]
Quote: software reliability is poor because of design faults, radically new systems, and discontinuous input-to-output mappings [»littB11_1993]

Related Topics up

Group: program proving   (10 topics, 311 quotes)
Group: security   (23 topics, 874 quotes)

Topic: automated tests of specifications and designs (12 items)
Topic: backup processor (3 items)
Topic: collision avoidance by robots (7 items)
Topic: defensive programming (22 items)
Topic: embedded systems (26 items)
Topic: error checking in robot programming (6 items)
Topic: error safe systems (76 items)
Topic: formal methods and languages (53 items)
Topic: hard real time systems (64 items)
Topic: logging data and events (17 items)
Topic: operating system security (18 items)
Topic: power fail recovery (6 items)
Topic: preventing accidental errors (37 items)
Topic: programming without errors (28 items)
Topic: resourceful, redundant systems for reliability (38 items)
Topic: testing by voting or N-version (10 items)
Topic: type-safe and secure languages
(43 items)

Updated barberCB 1/05
Copyright © 2002-2008 by C. Bradford Barber. All rights reserved.
Thesa is a trademark of C. Bradford Barber.