waashk/dblp huggingface dataset:
improving the write performance of an nfs server. the network file system (nfs) utilizes a stateless protocol between clients and servers; the major advantage of this statelessness is that nfs crash recovery is very easy. however, the protocol requires that data modification operations such as write be fully committed to stable storage before replying to the client. the cost of this is significant in terms of response latency and server cpu and i/o loading. this paper describes a write gathering technique that exploits the fact that there are often several write requests for the same file presented to the server at about the same time. with this technique the data portions of these writes are combined and a single metadata update is done that applies to them all. no replies are sent to the client until after this metadata update has been fully committed, thus the nfs crash recovery design is not violated. this technique can be used in most nfs server implementations and requires no client modifications.
rex: secure, extensible remote execution. the ubiquitous ssh package has demonstrated the importance of secure remote login and execution. as remote execution tools grow in popularity, users require new features and extensions, which are difficult to add to existing systems. rex is a remote execution utility with a novel architecture specifically designed for extensibility as well as security and transparent connection persistence in the face of network complexities such as nat and dynamic ip addresses. to achieve extensibility, rex bases much of its functionality on a single new abstraction--emulated file descriptor passing across machines. this abstraction is powerful enough for users to extend rex's functionality in many ways without changing the core software or protocol. rex addresses security in two ways. first, the implementation internally leverages file descriptor passing to split the server into several smaller programs, reducing both privileged and remotely exploitable code. second, rex selectively delegates authority to processes running on remote machines that need to access other resources. the delegation mechanism lets users incrementally construct trust policies for remote machines. finally, rex provides mechanisms for accessing servers without globally routable ip addresses, and for resuming sessions when a tcp connection aborts or an endpoint's ip address changes. measurements of the system demonstrate that rex's architecture does not come at the cost of performance.
the ferret document browser. the ferret document browser is a vehicle for exploring the design and use of document storage and retrieval systems. its distributed, modular structure allows independent information providers to control their data, yet make use of a common access and billing control facility. document images are distributed via a nationwide at&t corporate internet which consists mainly of ethernet networks interconnected by leased data circuits. the relatively low bandwidth of this networks is dealt with by compressing the documents for transmission, and by decompressing pages as requested on the workstation. a page image can be decompressed and displayed in less that a half second. a broadband version of the system makes use of the bbfs broadband file server, the hpc interconnect, the luckynet broadband network and the liaison network multimedia workstation. this system allows document browsing at rates up to 15 page images per second.
a flash-memory based file system. a flash memory device driver that supports a conventional unix file system transparently was designed. to avoid the limitations due to flash memory's restricted number of write cycles and its inability to be overwritten, this driver writes data to the flash memory system sequentially as a log-structured file system (lfs) does and uses a cleaner to collect valid data blocks and reclaim invalid ones by erasing the corresponding flash memory regions. measurements showed that the overhead of the cleaner has little effect on the performance of the prototype when utilization is low but that the effect becomes critical as the utilization gets higher, reducing the random write throughput from 222 kbytes/s at 30% utilization to 40 kbytes/s at 90% utilization. the performance of the prototype in the andrew benchmark test is roughly equivalent to that of the 4.4bsd pageable memory based file system (mfs).
a cooperative internet backup scheme. we present a novel peer-to-peer backup technique that allows computers connected to the internet to back up their data cooperatively: each computer has a set of partner computers, which collectively hold its backup data. in return, it holds a part of each partner's backup data. by adding redundancy and distributing the backup data across many partners, a highly-reliable backup can be obtained in spite of the low reliability of the average internet machine. because our scheme requires cooperation, it is potentially vulnerable to several novel attacks involving free riding (e.g., holding a partner's data is costly, which tempts cheating) or disruption. we defend against these attacks using a number of new methods, including the use of periodic random challenges to ensure partners continue to hold data and the use of disk-space wasting to make cheating unprofitable. results from an initial prototype show that our technique is feasible and very inexpensive: it appears to be one to two orders of magnitude cheaper than existing internet backup services.
building secure high-performance web services with okws. okws is a toolkit for building fast and secure web services. it provides web developers with a small set of tools that has proved powerful enough to build complex systems with limited effort. despite its emphasis on security, okws shows performance improvements compared to popular systems: when servicing fully dynamic, non-disk-bound database workloads, okws's throughput and responsiveness exceed that of apache 2 [3], flash [23] and haboob [44]. experience with okws in a commercial deployment suggests it can reduce hardware and system management costs, while providing security guarantees absent in current systems.
an analysis of trace data for predictive file caching in mobile computing. one way to provide mobile computers with access to the resources of a network, even in the absence of communication, is to predict which information will be used during disconnection and cache the appropriate data while still connected. to determine the feasibility of this approach, traces of file-access activity for three diverse application domains were collected for periods of over two months. analysis of these traces using traditional and new measures reveals that user working sets tend to be small compared to modern disk sizes, that users tend to reference the same files for several days or even weeks at a time, and that different users do not tend to write to the same file except in highly constrained circumstances. these factors encourage the conclusion that an automated caching system can be built for a wide variety of environments.
redundancy elimination within large collections of files. ongoing advancements in technology lead to ever-increasing storage capacities. in spite of this, optimizing storage usage can still provide rich dividends. several techniques based on delta-encoding and duplicate block suppression have been shown to reduce storage overheads, with varying requirements for resources such as computation and memory. we propose a new scheme for storage reduction that reduces data sizes with an effectiveness comparable to the more expensive techniques, but at a cost comparable to the faster but less effective ones. the scheme, called redundancy elimination at the block level (rebl), leverages the benefits of compression, duplicate block suppression, and delta-encoding to eliminate a broad spectrum of redundant data in a scalable and efficient manner. rebl generally encodes more compactly than compression (up to a factor of 14) and a combination of compression and duplicate suppression (up to a factor of 6.7). rebl also encodes similarly to a technique based on delta-encoding, reducing overall space significantly in one case. furthermore, rebl uses super-fingerprints, a technique that reduces the data needed to identify similar blocks while dramatically reducing the computational requirements of matching the blocks: it turns o(n2) comparisons into hash table lookups. as a result, using super-fingerprints to avoid enumerating matching data objects decreases computation in the resemblance detection phase of rebl by up to a couple orders of magnitude.
flexible and safe resolution of file conflicts. in this paper we describe the support provided by the coda file system for transparent resolution of conflicts arising from concurrent updates to a file in different network partitions. such partitions often occur in mobile computing environments. coda provides a framework for invoking customized pieces of code called application-specific resolvers (asrs) that encapsulate the knowledge needed for file resolution. if resolution succeeds, the user notices nothing more than a slight performance delay. only if resolution fails does the user have to resort to manual repair. our design combines a rule-based approach to asr selection with transactional encapsulation of asr execution. this paper shows how such an approach leads to flexible and efficient file resolution without loss of security or robustness.
a performance comparison of unix operating systems on the pentium. this paper evaluates the performance of three popular versions of the unix operating system on the x86 architecture: linux, freebsd, and solaris. we evaluate the systems using freely available micro- and application benchmarks to characterize the behavior of their operating system services. we evaluate the currently available major releases of the systems "asis," without any performance tuning. our results show that the x86 operating systems and system libraries we tested fail to deliver the pentium's full memory write performance to applications. on small-file workloads, linux is an order of magnitude faster than the other systems. on networking software, freebsd provides two to three times higher bandwidth than linux. in general, solaris performance usually lies between that of the other two systems. although each operating system out-performs the others in some area, we conclude that no one system offers clearly better overall performance. other factors, such as extra features, ease of installation, or freely available source code, are more convincing reasons for choosing a particular system.
design and implementation of a multimedia protocol suite in a bsd unix kernel. development of distributed multimedia applications requires support for coordination and temporal/causal synchronization of traffic over related streams. our current research involves investigation of appropriate os and communication abstractions to support such applications. towards this goal, we have designed and implemented mcp, a suite of transport and session layer protocols, in the framework of a standard bsd unix networking platform. mcp contains two new abstractions. first, mcp contains a token-based mechanism for coordination of traffic over a multipoint connection. second, mcp includes an abstraction called a multi-flow conversation that enforces both temporal and causal synchronization among related data streams. this paper discusses unix kernel implementation of mcp and describes our experience in using mcp.
mgtk: an sml binding of gtk+. we describe mgtk, a standard ml language binding for the gtk+ toolkit. gtk+ is a graphical toolkit for the x window system, and provides an object-oriented c language api. since standard ml is a mostly-functional language without object types, constructing a binding to gtk+ is not a trivial task. in mgtk, a single-inheritance class hierarchy is encoded using sml's type system. most of the mgtk binding is machine generated, to best utilize the limited manpower of the project. the goal of the mgtk project is "just" to present a type-safe interface to gtk+ for sml programmers. this contrasts with gui libraries for functional languages, which concentrate on producing good user interfaces: there are several sml graphical user interface libraries available for this task. with mgtk, sml applications have access to the mature, complete and familiar gtk+ user interface.
operation-based update propagation in a mobile file system. in this paper we describe a technique called operation-based update propagation for efficiently transmitting updates to large files that have been modified on a weakly connected client of a distributed file system. in this technique, modifications are captured above the file-system layer at the client, shipped to a surrogate client that is strongly connected to a server, re-executed at the surrogate, and the resulting files transmitted from the surrogate to the server. if re-execution fails to produce a file identical to the original, the system falls back to shipping the file from the client over the slow network. we have implemented a prototype of this mechanism in the coda file system on linux, and demonstrated performance improvements ranging from 40 percents to nearly three orders of magnitude in reduced network traffic and elapsed time. we also found a novel use of forward error correction in this context.
network programming for the rest of us. twisted is a high-level networking framework that is built around event-driven asynchronous i/o. it supports tcp, ssl, udp and other network transports. twisted supports a wide variety of network protocols (including imap, ssh, http, dns). it is designed in a way that makes it easy to use with other event-driven toolkits such as gtk+, qt, tk, wxpython and win32. implemented mostly in python, twisted makes it possible to create network applications without having to worry about low-level platform specific details. however, unlike many other network toolkits, twisted still allows developers to access platform specific features if necessary. twisted has been used to develop a wide variety of applications, including messaging clients, distributed hash tables, web applications and both open source projects and commercial applications.
the vinum volume manager. the vinum volume manager is a block device driver which implements virtual disk drives. it isolates disk hardware from the block device interface and maps data in ways which result in an increase in flexibility, performance and reliability compared to the traditional slice view of disk storage. vinum implements the raid-0, raid-1 and raid-5 models, both individually and in combination.
malloc() performance in a multithreaded linux environment. network servers make special demands that other types of applications may not make on memory allocators. we describe a simple malloc() microbenchmark suite that tests the ability of malloc() to divide its work efficiently among multiple threads and processors. the purpose of this suite is to determine the suitability of an operating system's heap allocator for use with network servers running in an smp environment.
the design of the dents dns server. dents is a server implementation of the internet's domain name system. dents main features are a modular driver architecture, a corba-based control facility, a replaceable tree system, a clean design and good karma. dents is free software, licensed under version 2 of the gpl. in this paper, i describe the design of dents, concentrating on the innovations and evolutions it embodies, and including the future directions in which we hope to take the server. i describe some of the problems we've had. finally, i summarize some lessons about server design which dents reflects.
a quantitative analysis of disk drive power management in portable computers. with the advent and subsequent popularity of portable computers, power management of system components has become an important issue. current portable computers implement a number of power reduction techniques to achieve a longer battery life. included among these is spinning down a disk during long periods of inactivity. in this paper, we perform a quantitative analysis of the potential costs and benefits of spinning down the disk drive as a power reduction technique. our conclusion is that almost all the energy consumed by a disk drive can be eliminated with little loss in performance. although on current hardware, reliability can be impacted by our policies, the next generation of disk drives will use technology (such as dynamic head loading) which is virtually unaffected by repeated spinups. we found that the optimal spindown delay time, the amount of time the disk idles before it is spun down, is 2 seconds. this differs significantly from the 3-5 minutes in current practice by industry. we will show in this paper the effect of varying the spindown delay on power consumption; one conclusion is that a 3-5 minute delay results in only half of the potential benefit of spinning down a disk.
drinking from the firehose: multicast usenet news. news transport and spooling systems of the last several years have concentrated on decreasing the resource load on news servers. one beneficial side effect has been the average decrease in time that a news system spends on a given article. this paper describes a novel usenet news transport protocol, which we call muse. the two major motivations behind muse are to reduce the average propagation delays of articles on usenet and to further reduce the resource load on a centralized news server. muse runs on top of the experimental internet multicast backbone, commonly referred to as the mbone. major design and implementation issues are discussed. security concerns of multicast news are discussed and our solution is examined. the problems of scaling news distribution to thousands of hosts are also addressed.
implementation of a modern web search engine cluster. yuntis is a fully-functional prototype of a complete web search engine with features comparable to those available in commercial-grade search engines. in particular, yuntis supports page quality scoring based on global web linkage graph, extensively exploits text associated with links, computes pages' keywords and lists of similar pages of good quality, and provides a very flexible query language. this paper reports our experiences in the three-year development process of yuntis, by presenting its design issues, software architecture, implementation details, and performance measurements.
eliminating receive livelock in an interrupt-driven kernel. most operating systems use interface interrupts to schedule network tasks. interrupt-driven systems can provide low overhead and good latency at low offered load, but degrade significantly at higher arrival rates unless care is taken to prevent several pathologies. these are various forms ofreceive livelock, in which the system spends all of its time processing interrupts, to the exclusion of other necessary tasks. under extreme conditions, no packets are delivered to the user application or the output of the system. to avoid livelock and related problems, an operating system must schedule network interrupt handling as carefully as it schedules process execution. we modified an interrupt-driven networking implementation to do so; this modification eliminates receive livelock without degrading other aspects of system performance. our modifications include the use of polling when the system is heavily loaded, while retaining the use of interrupts ur.jer lighter load. we present measurements demonstrating the success of our approach.
the restore-o-mounter: the file motel revisited. we present a scheme for referencing and accessing saved ((footnote 1: we use the word "save" to denote the super set of "backup" and "archive"; save is also easier to conjugate than backup.)) files in a manner that is transparent to unix" applications. the scheme requires no kernel modifications. instead, it uses a "mounted" process that allows users to change directories to the past and browse their saved files with their favorite utilities. the mounted process acts as a protocol gateway between nfs and a commercially available network backup product. time travel is supported; users may change directories to any moment in the past. any saved version (not just the most recent version) of any file can be viewed or recovered, even if the file has since been deleted. using this transparent method of retrieving saved files by naming their location in the past, a poor man's file migration scheme can be implemented by substituting a symbolic link to a saved location for a file. once a file is referenced, the symbolic link can be replaced with its original file. this migration scheme requires no kernel modifications yet remains transparent to unix applications and users.
performance implications of multiple pointer sizes. many users need 64-bit architectures: 32-bit systems cannot support the largest applications, and 64-bit systems perform better for some applications. however, performance on some other applications can suffer from the use of large pointers; large pointers can also constrain feasible problem size. such applications are best served by a 64-bit machine that supports the use of both 32-bit and 64-bit pointer variables. this paper analyzes several programs and programming techniques to understand the performance implications of different pointer sizes. many (but not all) programs show small but definite performance consequences, primarily due to cache and paging effects.
porting the sgi xfs file system to linux. the limitations of traditional linux file systems are becoming evident as new application demands for linux file systems arise. sgi has ported the xfs file system to the linux operating system to address these constraints. this paper describes the major technical areas that were addressed in this port, specifically regarding the file system interface to the operating system, buffer caching in xfs, and volume management layers. in addition, this paper describes some of the legal issues surrounding the porting of the xfs file system, and the encumbrance review process that sgi performed.
operating system support for distributed multimedia. multimedia applications place new demands upon processors, networks and operating systems. while some network designers, through atm for example, have considered revolutionary approaches to supporting multimedia, the same cannot be said for operating systems designers. most work is evolutionary in nature, attempting to identify additional features that can be added to existing systems to support multimedia. here we describe the pegasus project's attempt to build an integrated hardware and operating system environment from the ground up specifically targeted towards multimedia.
not quite nfs, soft cache consistency for nfs. there are some constraints inherent in the nfstm∈ protocol that result in performance limitations for high performance workstation environments. this paper discusses an nfs-like protocol named not quite nfs (nqnfs), designed to address some of these limitations. this protocol provides full cache consistency during normal operation, while permitting more effective client-side caching in an effort to improve performance. there are also a variety of minor protocol changes, in order to resolve various nfs issues. the emphasis is on observed performance of a preliminary implementation of the protocol, in order to show how well this design works and to suggest possible areas for further improvement.