waashk/dblp huggingface dataset:
lazy asynchronous i/o for event-driven servers. we introduce lazy asynchronous i/o (laio), a new asynchronous i/o interface that is well suited to event-driven programming. laio is general in the sense that it applies to all blocking i/o operations. furthermore, it is lazy in the sense that it creates a continuation only when an operation actually blocks, and it notifies the application only when a blocked operation completes in its entirety. these features make programming high-performance, event-driven servers using laio considerably easier than with previous interfaces. we describe a user-level implementation of laio, relying only on kernel support for scheduler activations, a facility present in many unix-like systems. we compare the performance of web servers implemented using laio to the performance obtained with previous interfaces. for workloads with an appreciable amount of disk i/o, laio performs substantially better than the alternatives, because it avoids blocking entirely. in one such case, the peak throughput with laio is 24% higher than the next best alternative. for in-memory workloads it performs comparably.
portable multithreading-the signal stack trick for user-space thread creation. this paper describes a pragmatic but portable fallback approach for creating and dispatching between the machine contexts of multiple threads of execution on unix systems that lack a dedicated user-space context switching facility. such a fallback approach for implementing machine contexts is a vital part of a user-space multithreading environment, if it has to achieve maximum portability across a wide range of unix flavors. the approach is entirely based on standard unix system facilities and ansi-c language features and especially does not require any assembly code or platform specific tricks at all. the most interesting issue is the technique of creating the machine context for threads, which this paper explains in detail. the described approach closely follows the algorithm as implemented by the author for the popular user-space multithreading library gnu portable threads (gnu pth, [25]) which this way quickly gained the status of one of the most portable user-space multithreading libraries.
atom: a flexible interface for building high performance program analysis tools. code instrumentation is a powerful mechanism for understanding program behavior. unfortunately, code instrumentation is extremely difficult, and therefore has been mostly relegated to building special purpose tools for use on standard industry benchmark suites. atom (analysis tools with om) provides a very flexible and efficient code instrumentation interface that allows powerful, high performance program analysis tools to be built with very little effort. this paper illustrates this flexibility by building five complete tools that span the interests of application programmers, computer architects, and compiler writers. this flexibility does not come at the expense of performance. because atom uses procedure calls as the interface between the application and the analysis routines, the performance of each tool is similar to or greatly exceeds the best known hand-crafted implementations.
design and implementation of a transaction-based filesystem on freebsd. transactional database management systems (dbms's) have special data integrity requirements that standard filesystems such as the berkeley fast filesystem do not address. this paper briefly describes the requirements a transactional dbms makes of a transaction-based filesystem, then goes on to describe the design and implementation of such a filesystem, referred to as a block repository, which is part of the sqrl dbms project. the implementation of sqrl's block repository is different than most traditional filesystems in that it is purposely implemented in user-land using raw devices and threads. its performance is more tunable to the needs of transaction processing than would be the case if it were integrated into the kernel.
new tricks for an old terminal driver. users expect more out of command lines than they did fifteen years ago, but the terminal interface has not evolved to keep up with their expectations. with a few modifications, though, the terminal driver can provide every program with support for the arrow keys and several common emacs commands.
distributed computing: moving from cgi to corba. in this paper, we document the evolution of a banner and delivery system from a simple cgi script written in perl running on a single host into a distributed computing application using corba. while corba has an established history in the enterprise-computing world, it is only recently that the open-source ® community has begun to embrace it. starting without any rpc programming experience, it took targetnet a little less than half a year to integrate corba into the apache web server and convert all their cgi programs into corba servers. performance of the system increased from 50 transactions per second to over 400 per second. thanks to the cross-platform capabilities of corba, future components can be developed on virtually any operating system and programming language. by adding inexpensive servers, the capacity of the system scales in a near-linear fashion. most importantly, the switch to corba didn't require a change of operating system or development environment - everything runs on a free operating system using opensource components.
an extensible protocol architecture for application-specific networking. plexus is a networking architecture that allows applications to achieve high performance with customized protocols. application-specific protocols are written in a typesafe language and installed dynamically into the operating system kernel. because these protocols execute within the kernel, they can access the network interface and other operating system services with low overhead. protocols implemented with plexus outperform equivalent protocols implemented on conventional monolithic systems. plexus runs in the context of the spin extensible operating system.
evolving mach 3.0 to a migrating thread model. we have modified mach 3.0 to treat cross-domain remote procedure call (rpc) as a single entity, instead of a sequence of message passing operations. with rpc thus elevated, we improved the transfer of control during rpc by changing the thread model. like most operating systems, mach views threads as statically associated with a single task, with two threads involved in an rpc. an alternate model is that of migrating threads, in which, during rpc, a single thread abstraction moves between tasks with the logical flow of control, and "server" code is passively executed. we have compatibly replaced mach's static threads with migrating threads, in an attempt to isolate this aspect of operating system design and implementation. the key element of our design is a decoupling of the thread abstraction into the execution context and the schedulable thread of control, consisting of a chain of contexts. a key element of our implementation is that threads are now "based" in the kernel, and temporarily make excursions into tasks via upcalls. the new system provides more precisely defined semantics for thread manipulation and additional control operations, allows scheduling and accounting attributes to follow threads, simplifies kernel code, and improves rpc performance. we have retained the old thread and ipc interfaces for backwards compatibility, with no changes required to existing client programs and only a minimal change to servers, as demonstrated by a functional unix single server and clients. the logical complexity along the critical rpc path has been reduced by a factor of nine. local rpc, doing normal marshaling, has sped up by factors of 1.7-3.4. we conclude that a migrating-thread model is superior to a static model, that kernel-visible rpc is a prerequisite for this improvement, and that it is feasible to improve existing operating systems in this manner.
the shell as a service. this paper explores the design history of the nmake shell coprocess. originally a special purpose uniprocessor executor, the coshell has evolved into a general purpose service that automatically executes shell actions on lightly loaded hosts in a local network. a major thrust of this work has been ease of use. the only privilege required for installation, administration or use is rsh access to the local hosts. nmake and gnu-make users can take advantage of network execution with no makefile modifications. shell level access is similar to but more efficient than rsh and allows host expression matching to replace the explicit host name argument. also provided is a c programming library interface with five primitive operations that follow the fork-exec-wait process model. beside the speedups attained by parallelizing computations in a homogeneous network, coshell also supports heterogeneous configurations. this presents novel solutions to traditional cross-compilation problems. it also allows the user to view a new network host as a compute engine rather than yet another architecture on which to port the home environment and tools. coshell runs on most s5r4 and bsd unix* operating system variants.
migrating an mvs mainframe application to a pc. due to advances in computer architecture, performance of the pc now exceeds that of the typical mainframe. however, computing cost on a mainframe continues to greatly exceed that on a pc. thus, migrating mainframe applications to pc can result in substantial savings. the major stumbling block in doing this is the cost of software migration itself. this paper discusses an experiment in using a software tool approach to migrate a large billing application from mvs running on a mainframe to unix running on a pc. we developed tools to port the application from mainframe to pc with minimal code rewriting and enable transferring data cheaply so that processing can be done without interrupting other ongoing mainframe operations. we were able to transfer data from the mvs mainframe to a linux pc and complete its processing in less total time than if done entirely on the original mainframe.
extended data formatting using sfio. the ansi-c standard defines stdio as the i/o library for c programs. despite its ubiquitous use, stdio has well-documented deficiencies in various areas including data formatting. the sfio library provides an alternative to stdio with improved functionality, robustness and performance. in particular, sfio extends the data formatting functions so that applications can deal with arbitrary scalar objects, avoid unsafe operations and even define their own conversion patterns. this paper discusses these formatting enhancements
the pebble component-based operating system. pebble is a new operating system designed with the goals of flexibility, safety, and performance. its architecture combines a set of features heretofore not found in a single system, including (a) a minimal privileged mode nucleus, responsible for switching between protection domains, (b) implementation of all system services by replaceable user-level components with minimal privileges (including the scheduler and all device drivers) that run in separate protection domains enforced by hardware memory protection, and (c) generation of code specialized for each possible cross-domain transfer. the combination of these techniques results in a system with extremely inexpensive cross-domain calls that makes it well-suited for both efficiently specializing the operating system on a per-application basis and supporting modern component-based applications.
trapeze/ip: tcp/ip at near-gigabit speeds. this paper presents experiences with high-speed tcp/ip networking on a gigabit-per-second myrinet network, using a myrinet messaging system called trapeze. we explore the effects of common optimizations above and below the tcp/ip protocol stack, including zero-copy sockets, large packets with scatter/gather i/o, checksum offloading, message pipelining, and interrupt suppression. our experiments use extended freebsd 4.0 kernels on a range of intel and compaq alpha platforms. these experiments give a snapshot of the freebsd tcp/ip implementation running at bandwidths as high as 956 mb/s. we also report some results using gigabit ethernet products from alteon networks, which yielded a tcp bandwidth of 988 mb/s using zero-copy sockets on a 500 mhz compaq alpha 21264 workstation.
embedded inodes and explicit grouping: exploiting disk bandwidth for small files. small file performance in most file systems is limited by slowly improving disk access times, even though current file systems improve on-disk locality by allocating related data objects in the same general region. the key insight for why current file systems perform poorly is that locality is insufficient -- exploiting disk bandwidth for small data objects requires that they be placed adjacently. we describe c-ffs (co-locating fast file system), which introduces two techniques, embedded inodes and explicit grouping, for exploiting what disks do well (bulk data movement) to avoid what they do poorly (reposition to new locations). with embedded inodes, the inodes for most files are stored in the directory with the corresponding name, removing a physical level of indirection without sacrificing the logical level of indirection. with explicit grouping, the data blocks of multiple small files named by a given directory are allocated adjacently and moved to and from the disk as a unit in most cases. measurements of our c-ffs implementation show that embedded inodes and explicit grouping have the potential to increase small file throughput (for both reads and writes) by a factor of 5-7 compared to the same file system without these techniques. the improvement comes directly from reducing the number of disk accesses required by an order of magnitude. preliminary experience with software-development applications shows performance improvements ranging from 10-300 percent.
emstar: a software environment for developing and deploying wireless sensor networks. many wireless sensor network (wsn) applications are composed of a mixture of deployed devices with varying capabilities, from extremely constrained 8-bit "motes" to less resource-constrained 32-bit "microservers". emstar is a software environment for developing and deploying complex wsn applications on networks of 32-bit embedded microserver platforms, and integrating with networks of motes. emstar consists of libraries that implement message-passing ipc primitives, tools that support simulation, emulation, and visualization of live systems, both real and simulated, and services that support networking, sensing, and time synchronization. while emstar's design has favored ease of use and modularity over efficiency, the resulting increase in overhead has not been an impediment to any of our current projects.
linux device driver emulation in mach. we describe the design and performance of code added to the mach microkernel (mach 4.0, version uk02p21) that permits one to build a mach kernel that includes unmodified linux device drivers. we have written emulation code to support all linux 1.3.35 network and scsi drivers for the isa and pci i/o buses. emulation increases latency, but very little. the degree depends on both device and operation, and varies from 2 microseconds for receiving small (60 byte) network packets up to 197 microseconds for writing 16kb to an isa scsi device.
multi-resident afs: an adventure in mass storage. the pittsburgh supercomputing center has been working to integrate distributed file system technology with hierarchical mass storage. we produced a system utilizing the andrew file system that can be interfaced to many mass storage systems. we retained the semantics of afs and compatibility with standard clients and servers. the architecture has a logical separation between the facility that provides the user interface and access semantics and the management of the storage systems that contain user data. support for file level replication is provided for high availability to data in a fashion that is transparent to users. this system is called multi-resident afs.
idleness is not sloth. many people have observed that computer systems spend much of their time idle, and various schemes have been proposed to use this idle time productively. the commonest approach is to off-load activity from busy periods to less-busy ones in order to improve system responsiveness. in addition, speculative work can be performed in idle periods in the hopes that it will be needed later at times of higher utilization, or non-renewable resource like battery power can be conserved by disabling unused resources. we found opportunities to exploit idle time in our work on storage systems, and after a few attempts to tackle specific instances of it in ad hoc ways, began to investigate general mechanisms that could be applied to this problem. our results include a taxonomy of idle-time detection algorithms, metrics for evaluating them, and an evaluation of a number of idleness predictors that we generated from our taxonomy.
the refdbms distributed bibliographic database system. refdbms is a database system for sharing bibliographic references among many users at sites on a wide-area network such as the internet. this paper describes our experiences in building and using refdbms for the last two years. it summarizes the collection of facilities that refdbms provides, and gives detailed information on how well refdbms functions as a collaborative, wide-area, distributed information system.
a 3-tier raid storage system with raid1, raid5, and compressed raid5 for linux. this paper presents the design and implementation of a host-based driver (a "volume manager") for a 3-tier raid storage system, currently with 3 tiers: a small raid1 tier and larger raid5 and compressed raid5 (craid5) tiers. based on access patterns ("temperature"), the driver automatically migrates frequently accessed data to raid1 while demoting not so frequently accessed data to raid5/craid5. the prototype system, called "temperature sensitive storage" (tss), provides reliable persistence semantics for data migration between the tiers using ordered updates or logging. mechanisms are separated from policies through an api so that any desired policy can be implemented in trusted user processes. we also discuss the problems faced while moving from the original implementation on the solaris platform to linux. finally, we present comparison of the performance of our design with comparable systems using striping or raid5.
adaptable binary programs. to accurately and comprehensively monitor a program's behavior, many performance measurement tools transform the program's executable representation or binary. by instrumenting binary programs to monitor program events, tools can precisely analyze compiler optimization effectiveness, memory system performance, pipeline interlocking, and other dynamic program characteristics that are fully exposed only at this level. binary transformation has also been used to support software-enforced fault isolation, debugging, machine re-targeting, and machine-dependent optimization. at present, binary transformation applications face a difficult trade-off. previous approaches to implementing robust transformations result in significant disk space and run-time overhead. to improve efficiency, some current systems sacrifice robustness, relying on heuristic assumptions about the program and recognition of compiler-dependent code generation idioms. in this paper we begin by investigating the run-time and disk space overhead of transformation strategies that do not require assumptions about the program's control flow or register usage. we then detail simple information about the binary program that can significantly reduce this overhead. for each type of information, we show how it enables a corresponding type of binary transformation. we call binary programs that contain such enabling information adaptable binaries. because adaptable binary information is simple, any compiler can generate it. despite its simplicity, adaptable binary information has the necessary and sufficient expressive power to support a rich set of binary transformations.
the multispace: an evolutionary platform for infrastructural services. this paper presents the architecture for a base, a clustered environment for building and executing highly available, scalable, but flexible and adaptable infrastructure services. our architecture has three organizing principles: addressing all of the difficult service fault-tolerance, availability, and consistency problems in a carefully controlled environment, building that environment out of a collection of execution environments that are receptive to mobile code, and using dynamically generated code to introduce run-time-generated levels of indirection separating clients from services. we present a prototype java implementation of a base called the multispace, and talk about two applications written on this prototype: the ninja jukebox (a cluster based music warehouse), and keiretsu (an instant messaging service that supports heterogeneous clients). we show that the multispace implementation successfully reduces the complexity of implementing services, and that the platform is conducive to rapid service evolution.
reducing file system latency using a predictive approach. despite impressive advances in file system through put resulting from technologies such as high-bandwidth networks and disk arrays, file system latency has not improved and in many cases has become worse. consequently, file system i/o remains one of the major bottlenecks to operating system performance [10]. this paper investigates an automated predictive approach towards reducing file latency. automatic prefetching uses past file accesses to predict future file systemrequests. the objective is to provide data in advance of the request for the data, effectively masking access latencies. we have designed and implement a system to measure the performance benefits of automatic prefetching. our current results, obtained from a trace-driven simulation, show that prefetching results in as much as a 280% improvement over lru especially for smaller caches. alternatively, prefetching can reduce cache size by up to 50%.
newscache - a high-performance cache implementation for usenet news. usenet news is reaching its limits as current traffic strains the available infrastructure. news data volume increases steadily and competition with other internet services has intensified. consequently bandwidth requirements are often beyond that provided by typical links and the processing power needed exceeds a single system's capabilities. a rapidly growing number of users, especially attracted by www, overloads communication links and makes bandwidth a scarce resource. while an elaborate caching infrastructure was adopted for the www, usenet news still uses most of its originally defined infrastructure. caching techniques have not yet been adopted on a large scale. we believe that this is due to the lack of efficient cache implementations. in this paper we present a high performance cache server for usenet news that helps to conserve network bandwidth, computing power, and disk storage and is compatible with the current infrastructure and standards. after a thorough comparison of existing news database formats and replacement strategies we designed and implemented newscache to remedy usenet news bottlenecks. we present an empirical comparison of different cache replacement strategies as well as an evaluation of the use of newscache as a news server.