White Paper Secure Distributed and Parallel File Systems Based on Network-Attached Autonomous Disk Drives

The network-attached, autonomous disk drive is a revolutionary advance in disk drive and file system technologies that removes the server workstation bottleneck from networked file systems, embeds the file system into the disk drive and promotes the disk to a first class network device. By integrating security into the drive, network-attached disk drives provide file system integrity that is not dependant on the trustworthiness of the client. Minimizing the latency between disk drive and network is achieved by utilizing a low-power, low-cost, high-bandwidth embedded architecture specifically designed for data movement. Direct attachment to a network means that the disk drive can serve as a fundamental building block for highly-reliable, highly-available and fault-tolerant file systems providing low-cost, high-performance storage to meet the rapidly increasing needs of digital data storage applications. Goal of this white paper Our goal in distibuting this white paper is to identify industrial partners for this research agenda. We believe that raising the level of the storage system interface to that of a simple file system, treating the device as a first class network entity and integrating a security protocol at the drive level can open substantial opportunities for drive value-added features. Toward the exploitation of these opportunities we seek the experience and expertise of the storage industry. We would like to build close relationships with particular partners, leveraging the relationships that we currently enjoy through CMU’s Data Storage Systems Center and CMU’s Parallel Data Laboratory. Further, because widespread impact of the research we are embarking on will require industry consensus, we seek to establish a colaboration with a research group in the National Storage Industry Corporation. Professor Garth Gibson School of Computer Science Carnegie Mellon University 5000 Forbes Avenue Pittsburgh, PA 15213 garth@cs.cmu. edu (412)-268-5890 (phone) (412)-268-5576 (fax) Thi d d i h F M k 4 0 4 2 Next Generation Disk Drive Interfaces Our vision of scalable distributed and parallel storage architectures is guided by these themes: • to exploit new opportunities enabled by technology trends, • to bring applications closer to their data and give them greater control over its performance, availability, and consistency, • and to design security into the most basic levels of the system. We propose to develop a scalable, secure, high performance file system interface appropriate for commodity disks endowed with first class network ports. We will demonstrate this interface for stand-alone personal computers, traditional distributed file systems, and highly scalable parallel file systems for parallel computers. We will further develop the drive computing architecture capable of meeting the demands of this new role but still sufficiently cost-effective and powermeager to meet the competitive demands of the high-density, commodity drive marketplace. The employment of these drives as a basic system building block in scalable multicomputing environments reconceptualizes network architecture to eliminate file server workstations, endowing the network with data transfer characteristic of a system backplane and a trust model characteristic of an insecure heterogeneous environment. Enable new classes of performance optimizations by raising storage’s system interface The US digital data storage industry leads the world’s storage markets. It has survived the move from being able to rely on captive customers to having to compete in the price sensitive personal computing marketplace. It has responded to the pressure from DRAM density growth and increased its areal growth rates from 25% per year to 60% per year. It has developed perhaps the cheapest, most widely supported and highest bandwidth local area network, the SCSI bus, whose technology hiding aspects have enabled rapid introduction of new data rates and geometries. Moreover, SCSI has induced drive vendors to endow each drive with a device-optimized operating system implementing geometry-dependent scheduling and caching; this has been a particularly effective as added-value that distinguishes one product from another because of the slow evolution of client operating systems. But the physical layers of SCSI have been pushed to their limit; the next major evolution in disk architecture is the adoption of high-speed serial interconnects such as Fibrechannel, SSA, and Firewire. High-speed is needed because media data rates are approaching 10 MB/s and may grow at up to 20% per year. Serial is needed to lower cost, extend server-disk physical separation, and increase the number of ports on one server adapter. For example, Fibrechannel-attached drives will be available in 1995 and will burst transfer at 1 GB/s over a 2 km cable distance with up to 126 disks and servers attached. While these serial interconnects will free disk technology from the physical limitations of SCSI, they will not change the logical interfaces: SCSI’s abstraction of a drive as a single linear collection of fixed size sectors. In order to continue to add value through drive-embedded optimizations such as aggressive prefetching, dynamic allocation, or on-the-fly compression, we propose to