Metadata updates, such as file creation and block allocation, have consistently been identified as a source of performance, integrity, security, and availability problems for file systems. Soft updates is an implementation technique for low-cost sequencing of fine-grained updates to write-back cache blocks. Using soft updates to track and enforce metadata update dependencies, a file system can safely use delayed writes for almost all file operations. This article describes soft updates, their incorporation into the 4.4BSD fast file system, and the resulting effects on the system. We show that a disk-based file system using soft updates achieves memory-based file system performance while providing stronger integrity and security guarantees than most disk-based file systems. For workloads that frequently perform updates on metadata (such as creating and deleting files), this improves performance by more than a factor of two and up to a factor of 20 when compared to the conventional synchronous write approach and by 4 -19% when compared to an aggressive write-ahead logging approach. In addition, soft updates can improve file system availability by relegating crash-recovery assistance (e.g., the fsck utility) to an optional and background role, reducing file system recovery time to less than one second. Permission to make digital / hard copy of part or all of this work for personal or classroom use is granted without fee provided that the copies are not made or distributed for profit or commercial advantage, the copyright notice, the title of the publication, and its date appear, and notice is given that copying is by permission of the ACM, Inc. To copy otherwise, to republish, to post on servers, or to redistribute to lists, requires prior specific permission and / or a fee. General Terms: Performance, Reliability, Design, Experimentation, Measurement, Security Additional Key Words and Phrases: File systems, metadata, integrity, failure recovery, disk caches, write-back caching, delayed writes In file systems, metadata (e.g., directories, inodes, and free block maps) gives structure to raw storage capacity. Metadata consists of pointers and descriptions for linking multiple disk sectors into files and identifying those files. To be useful for persistent storage, a file system must maintain the integrity of its metadata in the face of unpredictable system crashes, such as power interruptions and operating system failures. Because such crashes usually result in the loss of all information in volatile main memory, the information in nonvolatile storage (i.e., disk) must always be consistent enough to deterministically reconstruct a coherent file system state. Specifically, the on-disk image of the file system must have no dangling pointers to uninitialized space, no ambiguous resource ownership caused by multiple pointers, and no live resources to which there are no pointers. Maintaining these invariants generally requires sequencing (or atomic grouping) of updates to small on-disk metadata objects. From a performance standpoint, recent and predicted future technology trends result in a growing disparity between processing performance and disk access times. This disparity, combined with increasing main memory sizes, dictates that high-performance file systems aggressively employ caching techniques to avoid disk accesses and hide disk latencies. For metadata updates, which are characterized by strong spatial and temporal locality and by small sizes relative to the units of disk access, this means write-back caching. Write-back caching can substantially improve metadata update performance by combining multiple updates into a much smaller quantity of background disk writes. The savings come from two sources: multiples updates to a single metadata component (e.g., removal of a recently added directory entry) and multiple independent updates to a single block of metadata (e.g., several entries added to a directory block). This article describes soft updates, an implementation technique for low-cost sequencing of fine-grained updates to write-back cache blocks . The soft updates mechanism tracks dependencies among updates to cached (i.e., in-memory) copies of metadata and enforces these dependencies, via update sequencing, as the dirty metadata blocks are written back to nonvolatile storage. Because most metadata blocks contain many pointers, cyclic dependencies occur frequently when dependencies are recorded only at the block level. Therefore, soft updates tracks dependencies on a per-pointer basis and allows blocks to be written in any order. Any still-dependent updates in a metadata block are rolled-back before the block is written and rolled-forward afterward. Thus, dependency cycles are eliminated as an issue. With soft updates, applications always
The soft updates mechanism tracks dependencies among changes to cached (i.e., inmemory) copies of metadata and enforces these dependencies, via update sequencing, as the dirty metadata blocks are written back to nonvolatile storage REF .
Soft updates: a solution to the metadata update problem in file systems
TOCS
Computer Science