waashk/dblp huggingface dataset:
cool job allocation: measuring the power savings of placing jobs at cooling-efficient locations in the data center. data center costs for computer power and cooling are staggering. because certain physical locations inside the data center are more efficient to cool than others, this suggests that allocating heavy computational workloads onto those servers that are in more efficient places might bring substantial savings. this simple idea raises two critical research questions that we address: (1) how should one measure and rank the cooling efficiency of different places in a data center? (2) how substantial is the savings? we performed a set of experiments in a thermally isolated portion of a real data center, and validated that the potential savings is substantial and therefore warrants further work in this area to exploit the savings opportunity.
reboots are for hardware: challenges and solutions to updating an operating system on the fly. patches to modern operating systems, including bug fixes and security updates, and the reboots and downtime they require, cause tremendous problems for system users and administrators. dynamic update allows an operating system to be patched without the need for a reboot or other service interruption. we have taken the approach of building dynamic update functionality directly into an existing operating system, k42. to determine the applicability of our update system, and to investigate the changes that are made to os code, we analysed k42's revision history. the analysis showed that our original system could only support half of the desired changes to k42. the main problem preventing more changes from being converted to dynamic updates was our system's inability to update interfaces. other studies, as well as our own investigations, have shown that change to interfaces is also prevalent in systems such as linux. thus, it is apparent that a dynamic update mechanism needs to handle interface changes to be widely applicable. in this paper, we describe how to support interface changes in a modular dynamic update system. with this improvement, approximately 79% of past performance and bug fix changes to k42 could be converted to dynamic updates, and we expect the proportion would be even higher if the fixes were being developed for dynamic update. measurements of our system show that the runtime overhead is very low, and the time to apply updates is acceptable. this paper makes the following contributions. we present a mechanism to handle interface changes for dynamic updates to an operating system. for performance-sensitive updates, we show how to apply changes lazily. we discuss lessons learned, including how an operating system can be structured to better support dynamic update. we also describe how our approach extends to other systems such as linux, that although structured modularly, are not strictly object-oriented like k42.
exploring recovery from operating system lockups. operating system lockup errors can render a computer unusable by preventing the execution other programs. watchdog timers can be used to recover from a lockup by resetting the processor and rebooting the system when a lockup is detected. this results in a loss of unsaved data in running programs. based on the observation that volatile memory is not affected when a processor a reset occurs, we present an approach to recover from a watchdog reset with minimal or zero loss of application state. we study the resolution of lockup conditions using thread termination and using exception dispatch. thread termination can still result in a usable system and is already used as a recovery strategy for other errors in linux. using exceptions allows developers to write code to handle a lockup within the erroneous thread and attempt application transparent recovery. fault injection experiments show that a significant percentage of lockups can be recovered by thread termination. exception handling further improves the recoverability of the operating system.
hyperion: high volume stream archival for retrospective querying. network monitoring systems that support data archiving and after-the-fact (retrospective) queries are useful for a multitude of purposes, such as anomaly detection and network and security forensics. data archiving for such systems, however, is complicated by (a) data arrival rates, which may be hundreds of thousands of packets per second on a single link, and (b) the need for online indexing of this data to support retrospective queries. at these data rates, both common database index structures and general-purpose file systems perform poorly. this paper describes hyperion, a system for archiving, indexing, and on-line retrieval of high-volume data streams. we employ a write-optimized stream file system for high-speed storage of simultaneous data streams, and a novel use of signature file indexes in a distributed multi-level index. we implement hyperion on commodity hardware and conduct a detailed evaluation using synthetic data and real network traces. our streaming file system, streamfs, is shown to be fast enough to archive traces at over a million packets per second. the index allows queries over hours of data to complete in as little as 10-20 seconds, and the entire system is able to index and archive over 200,000 packets/sec while processing simultaneous on-line queries.
configuration management at massive scale: system design and experience. the development and maintenance of network device configurations is one of the central challenges faced by large network providers. current network management systems fail to meet this challenge primarily because of their inability to adapt to rapidly evolving customer and provider-network needs, and because of mismatches between the conceptual models of the tools and the services they must support. in this paper, we present the presto configuration management system that attempts to address these failings in a comprehensive and flexible way. developed for and deployed over the last 4 years within a large isp network, presto constructs device-native configurations based on the composition of configlets representing different services or service options. configlets are compiled by extracting and manipulating data from external systems as directed by the presto configuration scripting and template language. we outline the configuration management needs of large-scale network providers, introduce the presto system and configuration language, and demonstrate the use, workflows, and ultimately the platform's flexibility via an example of vpn service. we conclude by considering future work and reflect on the operators' experiences with presto.
implementation and performance evaluation of fuzzy file block matching. the fuzzy file block matching technique (fuzzy matching for short), was first proposed for opportunistic use of content addressable storage. fuzzy matching aims to increase the hit ratio in the content-addressable storage providers, and thus can improve the performance of underlying distributed file storage systems by potentially saving significant network bandwidth and reducing file transmission costs. fuzzy matching employs shingling to represent the fuzzy hashing of file blocks for similarity detection, and error-correcting information to reconstruct the canonical content of a file block from some similar blocks. in this paper, we present the implementation details of fuzzy matching and a very basic evaluation of its performance. in particular, we show that fuzzy matching can recover new versions of gnu emacs source from older versions.
from trusted to secure: building and executing applications that enforce system security. commercial operating systems have recently introduced mandatory access controls (mac) that can be used to ensure system-wide data confidentiality and integrity. these protections rely on restricting the flow of information between processes based on security levels. the problem is, there are many applications that defy simple classification by security level, some of them essential for system operation. surprisingly, the common practice among these operating systems is simply to mark these applications as "trusted", and thus allow them to bypass label protections. this compromise is not a limitation of mac or the operating system services that enforce it, but simply a fundamental inability of any operating system to reason about how applications treat sensitive data internally--and thus the os must either restrict the data that they receive or trust them to handle it correctly. these practices were developed prior to the advent security-typed languages. these languages provide a means of reasoning about how the os's sensitive data is handled within applications. thus, applications can be shown to enforce system security by guaranteeing, in advance of execution, that they will adhere to the os's mac policy. in this paper, we provide an architecture for an operating system service, that integrate security-typed language with operating system mac services. we have built an implementation of this service, called siesta, which handles applications developed in the security-typed language, jif, running on the selinux operating system. we also provide some sample applications to demonstrate the security, flexibility and efficiency of our approach.
safestore: a durable and practical storage system. this paper presents safestore, a distributed storage system designed to maintain long-term data durability despite conventional hardware and software faults, environmental disruptions, and administrative failures caused by human error or malice. the architecture of safestore is based on fault isolation, which safe-store applies aggressively along administrative, physical, and temporal dimensions by spreading data across autonomous storage service providers (ssps). however, current storage interfaces provided by ssps are not designed for high end-to-end durability. in this paper, we propose a new storage system architecture that (1) spreads data efficiently across autonomous ssps using informed hierarchical erasure coding that, for a given replication cost, provides several additional 9's of durability over what can be achieved with existing black-box ssp interfaces, (2) performs an efficient end-to-end audit of ssps to detect data loss that, for a 20% cost increase, improves data durability by two 9's by reducing mttr, and (3) offers durable storage with cost, performance, and availability competitive with traditional storage systems. we instantiate and evaluate these ideas by building a safestore-based file system with an nfs-like interface.
events can make sense. tame is a new event-based system for managing concurrency in network applications. code written with tame abstractions does not suffer from the "stack-ripping" problem associated with other event libraries. like threaded code, tamed code uses standard control flow, automatically-managed local variables, and modular interfaces between callers and callees. tame's implementation consists of c++ libraries and a source-to-source translator; no platform-specific support or compiler modifications are required, and tame induces little runtime overhead. experience with tame in real-world systems, including a popular commercial web site, suggests it is easy to adopt and deploy.
a memory soft error measurement on production systems. memory state can be corrupted by the impact of particles causing single-event upsets (seus). understanding and dealing with these soft (or transient) errors is important for system reliability. several earlier studies have provided field test measurement results on memory soft error rate, but no results were available for recent production computer systems. we believe the measurement results on real production systems are uniquely valuable due to various environmental effects. this paper presents methodologies for memory soft error measurement on production systems where performance impact on existing running applications must be negligible and the system administrative control might or might not be available. we conducted measurements in three distinct system environments: a rack-mounted server farm for a popular internet service (ask.com search engine), a set of office desktop computers (univ. of rochester), and a geographically distributed network testbed (planetlab). our preliminary measurement on over 300 machines for varying multi-month periods finds 2 suspected soft errors. in particular, our result on the internet servers indicates that, with high probability, the soft error rate is at least two orders of magnitude lower than those reported previously. we provide discussions that attribute the low error rate to several factors in today's production system environments. as a contrast, our measurement unintentionally discovers permanent (or hard) memory faults on 9 out of 212 ask.com machines, suggesting the relative commonness of hard memory faults.
mapjax: data structure abstractions for asynchronous web applications. the current approach to developing rich, interactive web applications relies on asynchronous rpcs (remote procedure calls) to fetch new data to be displayed by the client. we argue that for the majority of web applications, this rpc-based model is not the correct abstraction: it forces programmers to use an awkward continuation-passing style of programming and to expend too much effort manually transferring data. we propose a new programming model, mapjax, to remedy these problems. mapjax provides the abstraction of data structures shared between the browser and the server, based on the familiar primitives of objects, locks, and threads. mapjax also provides additional features (parallel for loops and prefetching) that help developers minimize response times in their applications. map-jax thus allows developers to focus on what they do best-writing compelling applications-rather than worrying about systems issues of data transfer and callback management. we describe the design and implementation of the mapjax framework and show its use in three prototypical web applications: a mapping application, an email client, and a search-autocomplete application. we evaluate the performance of these applications under realistic internet latency and bandwidth constraints and find that the unoptimized mapjax versions perform comparably to the standard ajax versions, while mapjax performance optimizations can dramatically improve performance, by close to a factor of 2 relative to non-mapjax code in some cases.
supporting practical content-addressable caching with czip compression. content-based naming (cbn) enables content sharing across similar files by breaking files into position-independent chunks and naming these chunks using hashes of their contents. while a number of research systems have recently used custom cbn approaches internally to good effect, there has not yet been any mechanism to use cbn in a general-purposeway. in this paper, we demonstrate a practical approach to applying cbn without requiring disruptive changes to end systems. we develop czip, a cbn compression scheme which reduces data sizes by eliminating redundant chunks, compresses chunks using existing schemes, and facilitates sharing within files, across files, and across machines by explicitly exposing cbn chunk hashes. czip-aware caching systems can exploit the cbn information to reduce storage space, reduce bandwidth consumption, and increase performance, while content providers and middleboxes can selectively encode their most suitable content. we show that czip compares well to stand-alone compression schemes, that a cbn cache for czip is easily implemented, and that a czip-aware cdn produces significant benefits.
evaluating block-level optimization through the io path. this paper focuses on evaluation of the effectiveness of optimization at various layers of the io path, such as the file system, the device driver scheduler, and the disk drive itself. io performance is enhanced via effective block allocation at the file system, request merging and reordering at the device driver, and additional complex request reordering at the disk drive. our measurements show that effective combination of these optimization forms yields superior performance under specific workloads. in particular, the impact on io performance of technological advances in modern disk drives (i.e., reduction on head positioning times and deployment of complex request scheduling) is shown. for example, if the outstanding requests in the io subsystem can all be accommodated by the disk queue buffer then disk level request scheduling is as effective as to close any gaps in the performance between io request schedulers at the device driver level. even more, for disk drives with write through caches, large queue depths improve overall io throughput and when combined with the best performing disk scheduling algorithm at the device driver level, perform comparably with an io subsystem where disks have write-back caches.
dandelion: cooperative content distribution with robust incentives. content distribution via the internet is becoming increasingly popular. to be cost-effective, commercial content providers are considering the use of peer-to-peer (p2p) protocols such as bittorrent to save on bandwidth costs and to handle peak demands. however, when an online content provider uses a p2p protocol, it faces a crucial issue: how to incentivize its clients to upload to their peers. this paper presents dandelion, a system designed to address this issue in the case of paid content distribution. unlike previous solutions, most notably bittorrent, dandelion provides robust (provably non-manipulable) incentives for clients to upload to others. in addition, unlike systems with tit-for-tat-based incentives, a client is motivated to upload to its peers even if the peers do not have content that interests the client. a client that honestly uploads to its peers is rewarded with credit, which can be redeemed for various types of rewards, such as discounts on paid content. in designing dandelion, we trade scalability for the ability to provide robust incentives. the evaluation of our prototype system on planetlab demonstrates the viability of our approach. a dandelion server that runs on commodity hardware with a moderate access link is capable of supporting up to a few thousand clients. these clients can download content at rates comparable to those of bittorrent clients.
energy management for hypervisor-based virtual machines. current approaches to power management are based on operating systems with full knowledge of and full control over the underlying hardware; the distributed nature of multi-layered virtual machine environments renders such approaches insufficient. in this paper, we present a novel framework for energy management in modular, multi-layered operating system structures. the framework provides a unified model to partition and distribute energy, and mechanisms for energy-aware resource accounting and allocation. as a key property, the framework explicitly takes the recursive energy consumption into account, which is spent, e.g., in the virtualization layer or subsequent driver components. our prototypical implementation targets hypervisor-based virtual machine systems and comprises two components: a host-level subsystem, which controls machine-wide energy constraints and enforces them among all guest oses and service components, and, complementary, an energy-aware guest operating system, capable of fine-grained application-specific energy management. guest level energy management thereby relies on effective virtualization of physical energy effects provided by the virtual machine monitor. experiments with cpu and disk devices and an external data acquisition system demonstrate that our framework accurately controls and stipulates the power consumption of individual hardware devices, both for energy-aware and energyunaware guest operating systems.
potshards: secure long-term storage without encryption. users are storing ever-increasing amounts of information digitally, driven by many factors including government regulations and the public's desire to digitally record their personal histories. unfortunately, many of the security mechanisms that modern systems rely upon, such as encryption, are poorly suited for storing data for indefinitely long periods of time--it is very difficult to manage keys and update cryptosystems to provide secrecy through encryption over periods of decades. worse, an adversary who can compromise an archive need only wait for cryptanalysis techniques to catch up to the encryption algorithm used at the time of the compromise in order to obtain "secure" data. to address these concerns, we have developed potshards, an archival storage system that provides long-term security for data with very long lifetimes without using encryption. secrecy is achieved by using provably secure secret splitting and spreading the resulting shares across separately-managed archives. providing availability and data recovery in such a system can be difficult; thus, we use a new technique, approximate pointers, in conjunction with secure distributed raid techniques to provide availability and reliability across independent archives. to validate our design, we developed a prototype potshards implementation, which has demonstrated "normal" storage and retrieval of user data using indexes, the recovery of user data using only the pieces a user has stored across the archives and the reconstruction of an entire failed archive.
passwords for everyone: secure mnemonic-based accessible authentication. in many environments, a computer system is severely constrained to the extent that the practical input mechanisms are merely binary switches. requiring the user to remember a long random bit string and to authenticate by entering each bit in the available binary input mechanism, is completely impractical. this paper deals with the question of authentication in such environments where the inputs are constrained to be yes/no responses to statements displayed on the user's screen. we present passwit, a mnemonic-based system for such environments that combines good usability with high security, and has many additional features such as (to mention a few) resistance to phishing, keystroke-logging, and compatibility with currently deployed systems and password file formats (hence it can co-exist with existing login mechanisms).
wresting control from bgp: scalable fine-grained route control. today's internet users and applications are placing increased demands on internet service providers (isps) to deliver fine-grained, flexible route control. to assist network operators in addressing this challenge, we present the intelligent route service control point (irscp), a route control architecture that allows a network operator to flexibly control routing between the traffic ingresses and egresses within an isp's network, without modifying the isp's existing routers. in essence, irscp subsumes the control plane of an isp's network by replacing the distributed bgp decision process of each router in the network with a more flexible, logically centralized, appliction-controlled route computation. irscp supplements the traditional bgp decision process with an explicitly ranked decision process that allows route control applications to provide a per-destination, per-router explicit ranking of traffic egresses. we describe our implementation of irscp as well as a straightforward set of correctness requirements that prevents routing anomalies. to illustrate the potential of application-controlled route selection, we use our irscp prototype to implement a simple form of dynamic customer-traffic load balancing, and demonstrate through emulation that our implementation is scalable.