Toward scalable and high-performance I/O with cross-layered storage design
Creators: name: Ren, Yujie
Publisher: name: No Publisher Supplied
Year: 2024

description: The storage technologies have been rocketing in the past decade, PM (persistent memory) offers nano-second scale latency while NVMe SSD provide bandwidth of tens of gigabytes. However, software innovations in managing storage devices fall behind such hardware advancement. Firstly, traditional I/O stack and file systems in operating systems, widely used in both industry and academia, suffer from high software overheads (e.g., system calls, data copies, coarse-grained locks), which are no longer negligible for emerging fast storage devices with micro-second hardware latency. Secondly, with the increasing compute power and memory within storage devices, near-data processing provides opportunities for redesigning software I/O stacks and interfaces to keep pace with fast storage hardware. However, traditional I/O stack in operating system and existing user-level solutions are not able to exploit such opportunities without an efficient I/O interface to bind I/O requests to their related data processing operations. Thirdly, exa-scale data with explosive growth motivates the collaborative use of heterogeneous storage devices with both emerging and existing storage media. Nevertheless, managing these devices with vast differences in bandwidth, asymmetrical read and write performance, and capacity gaps present significant challenges. This thesis addresses these problems with a cross-layered system design by dividing I/O software stack into user space, OS kernel, and storage firmware to capitalize on the advantages of each layer. The research scope in this thesis diverges in two dimensions. The first is to redesign I/O stacks and interfaces at a single storage level. The second is to develop methods of managing heterogeneous storage devices. At a single storage level, to reduce software overheads in traditional I/O stack in operating systems with emerging computational storage devices, the first work in this thesis proposes a cross-layered file system design that disaggregates file systems into user-space, OS, and storage devices to eliminate scalability bottlenecks in each layer and scaling I/O performance with fine-grained concurrency. We demonstrate the file descriptor instead of the inode should server as the abstraction to concurrently access files. By mapping file descriptors to dedicated hardware I/O queues, we efficientlyeliminate coarse-grained locks used in existing systems. On top of the first work, this thesis made the observation that there are computational tasks tightly coupled with I/O operations, causing unnecessary context switching and data copy overheads potentially. The second work proposes a novel CISC I/O interface compatible with POSIX which packs multiple I/O operations with related computational tasks in a single compound I/O operation to utilize the compute power in storage devices and reduces data copy overheads effectively. Scaling beyond the scope of a single storage, we target heterogeneous storage systems composed of different types of storage devices. Existing caching and tiering approaches for managing heterogeneous storage devices place a faster storage on top of the storage hierarchy, which fails to fully capitalize on the cumulative storage capabilities. The third work in this thesis presents a novel solution to exploit the collective hardware and software capabilities offered by multiple storage devices by delegating resource management to user space and retaining important properties such as permission enforcement and sharing in the OS kernel. In summary, motivated by the software bottlenecks in existing solutions, this thesis proposes a cross-layered design to disaggregate the components of the I/O stack across user space, OS kernel, and storage firmware (if applicable) to reduce the software overheads incurred from a monolithic I/O stack design in most existing solutions.