abstract ;;Quote: Google's scalable distributed file system was designed for frequent component failure and huge, append-only files

abstract+;;Quote: Google co-designed its applications and file system API

2 ;;Quote: use file snapshot and record append for producer-consumer queues and many-way merging; minimizes synchronization overhead; record append at-least-once at a known offset

2 ;;Quote: a GFS file consists of fixed-size chunks identified by a globally unique 64-bit handle; a chunk is a Linux file

2+;;Quote: replicate GFS file chunks on multiple chunkservers and racks; usually three replicas

2 ;;Quote: a single GFS master manages the file system metadata for chunkservers; e.g., namespace, access control, mappings, garbage collection

2+;;Quote: use heartbeat messages for communication between master and chunkservers

2 ;;Quote: direct access to 64MB chunks with cached metadata; few master communications, persistent TCB connection to chunkserver, all metadata in master's memory

4 ;;Quote: GFS supports serial and concurrent writes and record appends; a mutation is replicated and may fail

4+;;Quote: a file region is consistent if all clients see the same data; it is also defined if clients see the entire mutation

5 ;;Quote: use leases to maintain order across replicas; master grants a chunk lease to the primary replica, and the primary orders the mutations

6 ;;Quote: chunkservers forward data immediately to the next replica over a precomputed path

6+;;Quote: from the GFS network topology, estimate distance by IP addresses

7 ;;Quote: use metadata to quickly snapshot a file or directory tree; copy-on-write to same chunkserver

7 ;;Quote: GFS uses a B-tree to map namespace to metadata; no directory nodes; 100 bytes per file

7+;;Quote: each GFS namespace node has a read-write lock; consistent total order to prevent deadlock

8 ;;Quote: use garbage collection to reclaim deleted files; mark deleted files by rename; on scan, delete from metadata; delete metadata for orphaned chunks

8+;;Quote: heartbeat messages sync chunkservers and master incrementally; deletes orphaned chunks

9 ;;Quote: master and chunkservers restart in seconds no matter how they are terminated; no abnormal termination

9+;;Quote: GFS uses shadow masters instead of mirrors; sub-second delay

9 ;;Quote: GFS uses shadow masters instead of mirrors; sub-second delay

11 ;;Quote: GFS aggregate read rate achieves 80% of physical link limit (6 MB/s per client); aggregate write rate is half that

15 ;;Quote: use checksumming to detect data corruption at disk level; with many disks, occurs frequently