27 ;;Quote: identifying a misconfiguration of a fully-racked, benchmark machine took nearly a week, full-time, around the globe

28 ;;Quote: modern, software systems can not be observed without modifying the software itself

28 ;;Quote: conditional trace and asserts bifurcates software; performance problems are found in production, but can only be understood in a development environment

30 ;;Quote: DTrace provides safe and heterogenous dynamic instrumentation of production systems; arbitrary actions, predicates, and in situ aggregation

30+;;Quote: DTrace providers create probes via the DTrace framework; allows tracing at any level of the system, even across protection boundaries

31 ;;Quote: DTrace actions execute in a virtual machine; no backwards branches, only defined subroutine calls, no arbitrary stores; arbitrary loads, including unmapped memory

31 ;;Quote: DTrace's quantize() generates a power-of-two histogram, in situ; for example, frequency of write system calls by size by application

33 ;;Quote: DTrace aggregates data at the source by CPU; keyed by arbitrary n-tuple; periodically aggregated across CPUs

33 ;;Quote: DTrace providers declare translators that translate from implementation-specific data structures into abstract ones; e.g., an io provider has 'start' and 'done' instead of 'fstat', etc.