CrossFS: A Cross-layered Direct-Access File System

The storage access latency in modern storage devices is transitioning from milliseconds to microseconds [7]. While modern applications strive to increase I/O parallelism, storage software bottlenecks such as system call overheads, coarsegrained concurrency control, and the inability to exploit storage hardware concurrency continues to impact I/O performance. Several kernel-level, user-level, and firmware-level file systems have been designed to benefit from CPU parallelism [9], direct storage access [2, 3, 5], or computational capability embedded in the storage hardware [4]. However, these approaches are designed in isolation and fail to exploit modern, ultra-fast storage hardware. Kernel-level file systems (Kernel-FS) satisfy fundamental file system guarantees such as integrity, consistency, durability, and security. Despite years of research, Kernel-FS designs continue to suffer from three main bottlenecks. First, applications must enter and exit the OS for performing I/O, which could increase latency by 1-4μs [3]. Recently found security vulnerabilities have further amplified such costs. Second, even state-of-the-art designs enforce unnecessary serialization (e.g., inode-level read-write lock) when accessing disjoint portions of data in a file leading to high concurrent access overheads. Third, Kernel-FS designs fail to fully exploit storage hardwarelevel capabilities such as device-level compute, thousands of I/O queues, and firmware schedulers, which impacts I/O latency, throughput, and concurrency in I/O-heavy applications. As an alternative design point, there is an increasing focus towards designing user-level file systems (User-FS) for direct storage access bypassing the OS [1–3, 5, 7]. However, satisfying the fundamental file system guarantees from untrusted user-level is challenging [4]. While these designs have advanced the state of the art, some designs bypass the OS only for data-plane operations (without data sharing) [1, 3, 7]. In contrast, others provide full direct access by either sidestepping or inheriting coarse-grained and suboptimal concurrency control across threads and processes [2, 5], or even compromise correctness. Importantly, most User-FS designs fail to exploit the hardware capabilities of modern storage. At the other extreme is the exploration of firmware-level file systems (Firmware-FS) that embed the file system into the device firmware for direct-access [4]. The Firmware-FS acts as a central entity to satisfy fundamental file system properties. Although a first important step towards utilizing storage-level computational capability (with 4-8 cores in modern storage), current designs miss out on benefiting from host-level multicore parallelism. Additionally, these designs inherit inodecentric design for request queuing, concurrency control, and scheduling, leading to poor I/O scalability. In summary, current User-FS, Kernel-FS, and Firmware-FS designs lack a synergistic design across the user, the kernel, and the firmware layers, which is critical for achieving direct storage access and scaling concurrent I/O performance without compromising fundamental file system properties. To address the aforementioned bottlenecks, we propose Per-inode Interval Tree