waashk/dblp huggingface dataset:
dynamic instrumentation of production systems. this paper presents dtrace, a new facility for dynamic instrumentation of production systems. dtrace features the ability to dynamically instrument both user-level and kernel-level software in a unified and absolutely safe fashion. when not explicitly enabled, dtrace has zero. probe effect--the system operates exactly as if dtrace were not present at all. dtrace allows for many tens of thousands of instrumentation points, with even the smallest of systems offering on the order of 30,000 such points in the kernel alone. we have developed a c-like high-level control language to describe the predicates and actions at a given point of instrumentation. the language features user-defined variables, including thread-local variables and associative arrays. to eliminate the need for most postprocessing, the facility features a scalable mechanism for aggregating data and a mechanism for speculative tracing. dtrace has been integrated into the solaris operating system and has been used to find serious systemic performance problems on production systems-problems that could not be found using pre-existing facilities.
dp: a library for building portable, reliable distributed applications. dp is a library of process management and communication tools for writing portable, reliable distributed applications. it provides support for a flexible set of message operations as well as process creation and management. it has been successfully used in developing distributed monte carlo, disjunctive programming and integer goal programming codes.it differs from pvm and similar libraries in its support for lightweight, unreliable messages, as well as asynchronous delivery of interrupt-generating messages. in addition, dp supports the development of long-running distributed applications tolerant to the failure or loss of a subset of its processors.
application-controlled file caching policies. we consider how to improve the performance of file caching by allowing user-level control over file cache replacement decisions. we use two-level cache management: the kernel allocates physical pages to individual applications (allocation), and each application is responsible for deciding how to use its physical pages (replacement). previous work on two-level memory management has focused on replacement, largely ignoring allocation. the main contribution of this paper is our solution to the allocation problem. our solution allows processes to manage their own cache blocks, while at the same time maintains the dynamic allocation of cache blocks among processes. our solution makes sure that good user-level policies can improve the file cache hit ratios of the entire system over the existing replacement approach. we evaluate our scheme by trace-based simulation, demonstrating that it leads to significant improvements in hit ratios for a variety of applications.
efficient support for p-http in cluster-based web servers. this paper studies mechanisms and policies for supporting http/1.1 persistent connections in cluster-based web servers that employ content-based request distribution. we present two mechanisms for the efficient, content-based distribution of http/1.1 requests among the back-end nodes of a cluster server. a trace-driven simulation shows that these mechanisms, combined with an extension of the locality-aware request distribution (lard) policy, are effective in yielding scalable performance for http/1.1 requests. we implemented the simpler of these two mechanisms, back-end forwarding. measurements of this mechanism in connection with extended lard on a prototype cluster, driven with traces from actual web servers, confirm the simulation results. the throughput of the prototype is up to four times better than that achieved by conventional weighted round-robin request distribution. in addition, throughput with persistent connections is up to 26% better than without.
scalable content-aware request distribution in cluster-based network servers. we present a scalable architecture for content-aware request distribution in web server clusters. in this architecture, a level-4 switch acts as the point of contact for the server on the inernet and distributes the incoming requests to a number of back-end nodes. the switch does not perform any contect-based distribution. this function is performed by each of the back-end nodes, which may forward the incoming request to another back-end based on the requested contect. in terms of scalability, this architecture compares favorably to existing approaches where a front-end node performs contect-based distribution. in our architecture, the expensive operations of tcp connection estabilishment and handoff are distributes among the back-ends, rather than being centralized in the front-end node. only a minimal additional latency penatly is paid for much improved scalability. we have implemented this new architecture, and we demonstrate its superior scalability by comparing it to a system that performs contect-aware distribution in the front-end, both under synthetic and trace-drive workloads.
u-p2p: a peer-to-peer framework for universal resource sharing and discovery. we present u-p2p, an open source framework for developing, deploying and discovering file-sharing communities. we address the problem of search in peer-to-peer file sharing by allowing the end user to add meta-data to shared documents. each file-sharing community allows the sharing of a particular structured document type. communities are themselves modeled as structured documents, thus enabling their sharing and discovery just like any other document. the creator of a particular community specifies, among other properties, the document type that it shares and the deployment model. u-p2p's extensible architecture allows developers to create new properties or extend existing ones. for example, developers can provide new deployment models or custom privacy and authentication features. u-p2p makes use of other open source projects such as jakarta tomcat and exist, an xml database system.
jemacs-the java/scheme-based emacs. jemacs is a re-implementation of the emacs programmable text editor. it is written in java, and uses the swing gui toolkit. emacs is based on the extension language emacs lisp (elisp), which is a dynamically-scoped member of the lisp family. jemacs supports elisp, as well as the use of scheme, a more modern statically-scoped lisp dialect. both languages get compiled to java bytecodes, either in advance or on-the-fly, using the kawa compilation framework.
implementation of ipv6 in 4.4 bsd. the widespread availability of the tcp/ip protocols in early versions of bsd unix fostered the currently widespread use of those protocols in commercial products. rapid depletion of the ipv4 address space has caused the internet engineering task force to design version 6 of the internet protocol (ipv6). ipv6 has some similiarities with ipv4, but it also has many differences, most notably in address size. this paper describes our experience creating a freely distributable implementation of ipv6 inside 4.4 bsd, with focus on the areas that have changed between the ipv4 and ipv6 implementations.
concert/c: a language for distributed programming. concert/c is a new language for distributed c programming that extends ansi c to support distribution and process dynamics. concert/c provides the ability to create and terminate processes, connect them together, and communicate among them. it supports transparent remote function calls (rpc) and asynchronous messages. interprocess communications interfaces are typed in concert/c, and type correctness is checked at compile time wherever possible, otherwise at runtime. all c data types, including complex data structures containing pointers and aliases, can be transmitted in rpcs. concert/c programs run on a heterogeneous set of machine architectures and operating systems and communicate over multiple rpc and messaging protocols. the current concert/c implementation runs on aix 3.2, sunos 4.1, solaris 2.2 and os/2 2.1, and communicates over sun rpc, osf/dce and udp multicast. several groups inside and outside ibm are actively using concert/c, and it is available via anonymous ftp from software.watson.ibm.com:/pub/concert.
techniques for the design of java operating systems. language-based extensible systems, such as java virtual machines and spin, use type safety to provide memory safety in a single address space. by using software to provide safety, they can support more efficient ipc. memory safety alone, however, is not sufficient to protect different applications from each other. such systems need to support a process model that enables the control and management of computational resources. in particular, language-based extensible systems should support resource control mechanisms analogous to those in standard operating systems. they need to support the separation of processes and limit their use of resources, but still support safe and efficient ipc. we demonstrate how this challenge is being addressed in several java-based systems. first, we lay out the design choices when implementing a process model in java. second, we compare the solutions that have been explored in several projects: alta, k0, and the j-kernel. alta closely models the fluke operating system; k0 is similar to a traditional monolithic kernel; and the j-kernel resembles a microkernel-based system. we compare how these systems support resource control, and explore the tradeoffs between the various designs.
c-jdbc: flexible database clustering middleware. large web or e-commerce sites are frequently hosted on clusters. successful open-source tools exist for clustering the front tiers of such sites (web servers and application servers). no comparable success has been achieved for scaling the backend databases. an expensive smp machine is required if the database tier becomes the bottleneck. the few tools that exist for clustering databases are often database-specific and/or proprietary. clustered jdbc (c-jdbc) addresses this problem. it is a freely available, open-source, flexible and efficient middleware for database clustering. c-jdbc presents a single virtual database to the application through the jdbc interface. it does not require any modification to jdbc-based applications. it furthermore works with any database engine that provides a jdbc driver, without modification to the database engine. the flexible architecture of c-jdbc supports large and complex database cluster architectures offering various performance, fault tolerance and availability tradeoffs. we present the design and the implementation of c-jdbc, as well as some uses of the system in various scenarios. finally, performance measurements using a clustered implementation of the tpc-w benchmark show the efficiency and scalability of c-jdbc.
improving application performance through swap compression. the performance of large applications tends to be poor due to the high overhead added by the swapping mechanism. the same problem may be found in highly-loaded multi-programmed systems where many of the running applications have to use the swap space in order to be able to execute at the same time. furthermore, those large applications might not be able to run on laptop or home computers as their resources are usually smaller than the ones found in an office system. in this paper, we present a solution to both problems that we have implemented in the linux kernel. the idea consists of compressing the swapped pages and keeping them in a swap cache whenever possible. we have tested this new mechanism with a set of real applications obtaining a significant performance improvement.
supporting mobility in mosquitonet. the goal of the mosquitonet project is to provide continuous internet connectivity to mobile hosts. mobile hosts must be able to take advantage of the best network connectivity available in any location, whether wired or wireless. we have implemented a mobile ip system that supports seamless switching between different networks and communication devices. in contrast to previous approaches to mobile ip, we believe mobile hosts should not assume any explicit mobility support from the networks they visit, aside from basic internet connectivity. this decision places extra responsibilities on the mobile hosts themselves. in this paper, we describe the design and implications of such a system. measurements of our implementation show that the inherent overhead to switch networks (below 10ms) is insignificant compared to the time required to bring up a new communication device.
porting the coda file system to windows. we first describe how the coda distributed filesystem was ported to windows 95 and 98. coda consists of user level cache managers and servers and kernel level code for filesystem support. severe reentrancy difficulties in the win32 environment on this platform were overcome by extending the djgpp dos c compiler package with kernel level support for sockets and more flexible memory management. with this support library and kernel modules for windows 9x filesystems in place, the coda file system client could be ported with very little patching and will likely soon run as well on windows 9x as on linux. we ported coda file servers to windows nt. for fileservers the cygwin32 kit was used. we will not report here on the port of the coda client to windows nt, which is in an early stage. in both cases cross compilation from a linux environment was most helpful to get a good development environment.
the globe distribution network. the goal of the globe project is to design and build a middleware platform that facilitates the development of large-scale distributed applications, such as those found on the internet. to demonstrate the feasibility of our design and to test our ideas, we are currently building a new internet application: the globe distribution network. the globe distribution network, or gdn, is an application for the efficient, worldwide distribution of free software and other free data. the gdn can be seen as an improvement to anonymous ftp and the world wide web due to its flexibility and extensive support for replication. this paper describes the design of the gdn. we start by explaining how the replication facilities of the globe middleware are used to make the gdn efficient, and how these facilities are implemented. next, we present the architecture of the gdn and discuss how the domain name system can be used as a first approach towards a worldwide service for naming software packages and other entities. this is followed by an analysis of the security requirements for the gdn and measures taken to satisfy these requirements. we hope to make globe and gdn itself available for free under the bsd license by 2001.
a hierarchical internet object cache. this paper discusses the design and performance of a hierarchical proxy-cache designed to make internet information systems scale better. the design was motivated by our earlier trace-driven simulation study of internet traffic. we challenge the conventional wisdom that the benefits of hierarchical file caching do not merit the costs, and believe the issue merits reconsideration in the internet environment. the cache implementation supports a highly concurrent stream of requests. we present performance measurements that show that our cache outperforms other popular internet cache implementations by an order of magnitude under concurrent load. these measurements indicate that hierarchy does not measurably increase access latency. our software can also be configured as a web-server accelerator; we present data that our httpd-accelerator is ten times faster than netscape's netsite and ncsa 1.4 servers. finally, we relate our experience fitting the cache into the increasingly complex and operational world of internet information systems, including issues related to security, transparency to cache-unaware clients, and the role of file systems in support of ubiquitous wide-area information systems.
auto-diagnosis of field problems in an appliance operating system. the use of network appliances, i.e., computer systems specialized to perform a single function, is becoming increasingly widespread. network appliances have many advantages over traditional general-purpose systems such as higher performance/cost metrics, easier configuration and lower costs of management. unfortunately, while the complexity of configuration and management of network appliances in normal usage is much lower than that of general-purpose systems, this is not always true in problem situations. the debugging of configuration and performance problems with appliance computers is a task similar to the debugging of such problems with general-purpose systems, and requires substantial expertise. this paper examines the issues of appliance-like management and performance debugging. we present a number of techniques that enable appliance-like problem diagnosis. these include continuous monitoring for abnormal conditions, diagnosis of configuration problems of network protocols via protocol augmentation, path-based problem isolation via cross-layer analysis, and automatic configuration change tracking. we also describe the use of these techniques in a problem autodiagnosis subsystem that we have built for the data ontap operating system. our experience with this system indicates a significant reduction in the cost of problem debugging and a much simpler user experience.
a scalable and explicit event delivery mechanism for unix. unix applications not wishing to block when doing i/o often use the select() system call, to wait for events on multiple file descriptors. the select() mechanism works well for small-scale applications, but scales poorly as the number of file descriptors increases. many modern applications, such as internet servers, use hundreds or thousands of file descriptors, and suffer greatly from the poor scalability of select(). previous work has shown that while the traditional implementation of select() can be improved, the poor scalability is inherent in the design. we present a new event-delivery mechanism, which allows the application to register interest in one or more sources of events, and to efficiently dequeue new events. we show that this mechanism, which requires only minor changes to applications, performs independently of the number of file descriptors.
transparent run-time defense against stack-smashing attacks. the exploitation of buffer overflow vulnerabilities in process stacks constitutes a significant portion of security attacks. we present two new methods to detect and handle such attacks. in contrast to previous work, the new methods work with any existing pre-compiled executable and can be used transparently per-process as well as on a system-wide basis. the first method intercepts all calls to library functions known to be vulnerable. a substitute version of the corresponding function implements the original functionality, but in a manner that ensures that any buffer overflows are contained within the current stack frame. the second method uses binary modification of the process memory to force verification of critical elements of stacks before use. we have implemented both methods on linux as dynamically loadable libraries and shown that both libraries detect several known attacks. the performance overhead of these libraries range from negligible to 15%.
monkey see, monkey do: a tool for tcp tracing and replaying. the performance of popular internet web services is governed by a complex combination of server behavior, network characteristics and client workload-all interacting through the actions of the underlying transport control protocol (tcp). consequently, even small changes to tcp or to the network infrastructure can have significant impact on end-to-end performance, yet at the same time it is challenging for service administrators to predict what that impact will be. in this paper we describe the implementation of a tool called monkey that is designed to help address such questions. monkey collects live tcp trace data near a server, distills key aspects of each connection (e.g., network delay, bottleneck bandwidth, server delays, etc.) and then is able to faithfully replay the client workload in a new setting. using monkey, one can easily evaluate the effects of different network implementations or protocol optimizations in a controlled fashion, without the limitations of synthetic workloads or the lack of reproducibility of live user traffic. using realistic network traces from the google search site, we show that monkey is able to replay traces with a high degree of accuracy and can be used to predict the impact of changes to the tcp stack.
creating a portable programming language using open source software. on a first glance, the field of compiler construction and programming language design may not seem to have experienced major innovations over the last decade. by now, it is almost common knowledge how a lexer works, how parsing is done, but not many have yet realized how open source software-and in particular the gnu compiler collection-have silently offered language implementors new and better ways to do their job. therefore, this paper describes the novel advantages open source software provides and, furthermore, it illustrates these with practical examples showing how the presented concepts can be put into practice. another important contribution of this paper is to give an overview over the existing limitations and the technical problems that can occur when creating an open source based programming language implementation.
limitations of the kerberos authentication system. the kerberos authentication system, a part of mit's project athena, has been adopted by other organizations. despite kerberos's many strengths, it has a number of limitations and some weaknesses. some are due to specifics of the mit environment; others represent deficiencies in the protocol design. we discuss a number of such problems, and present solutions to some of them. we also demonstrate how special-purpose cryptographic hardware may be needed in some cases.
mapping and visualizing the internet. we have been collecting and recording routing paths from a test host to each of over 90,000 registered networks on the internet since august 1998. the resulting database contains intersting routing and reachability information, and is available to the public for research purpose. the daily scans cover approximately a tenth of the networks on the internet, with a full scan run roughly once a month. we have also been collecting lucent's intranet data, and applied these tools to understanding its size and connectivity. we have also detected the sloss of power to routers in yugoslavia as the result of nato bombing. a simulated spring-force algorithm lays out the graphs thgat results from these database. this algorithm is well known, but has never been applied to such a large problem. the internet graph, with around 88,000 nodes and 100,000 edges, is larger than those previsouly considered tractable by the data visualization community. the resulting internet layouts are pleasent, though rather cluttered. on smaller networks, like lucent's intranet, the layouts present the data in a useful way. for the internet data, we have also tried plotting a minimum distance spanning tree; by throwing away edges, the remaining graph can be made more accessible. once a layout is chosen, it can be colored in various ways to show network-relevant data, such as ip address, domain information, location, isps, and result of scan (completed, filtered, loop, etc). this paper expands and updates the description of the project given in an ieee computer article [1].
managing traffic with altq. altq is a package for traffic management. altq includes a queueing framework and several advanced queueing disciplines such as cbq, red, wfq and rio. altq also supports rsvp and diffserv. altq can be configured in a variety of ways for both research and operation. however, it requires understanding of the technologies to set up things correctly. in this paper, i summarize the design trade-offs, the available technologies and their limitations, and how they can be applied to typical network settings.