waashk/dblp huggingface dataset:
compression proxy server: design and implementation. automatic data compression in the web proxy server is an important mechanism that can potentially reduce network bandwidth consumption and web access latency significantly. however, unlike traditional data compression, web protocols and data have unique characteristics that make compression challenging. these include data block streaming, wide range of data object sizes and types, and real-time response. in this paper, we focus on automatic web data compression in the http proxy server. a new classification of web data compression based on system complexity and http requirements is proposed: stream, block and file compression. then, the concept of hybrid web data compression is introduced. to understand the potentials of web data compression better, an implementation of the proposed hybrid compression in the squid proxy server is described. the result is very promising, as about 30% of the bandwidth can be saved easily. furthermore, even with a low end pentium 266 mhz pc as the proxy machine, the compression overhead is less than 1% of the transfer time.
why do internet services fail, and what can be done about it? in 1986 jim gray published his landmark study of the causes of failures of tandem systems and the techniques tandem used to prevent such failures [6]. seventeen years later, internet services have replaced fault-tolerant servers as the new kid on the 24×7-availability block. using data from three large-scale internet services, we analyzed the causes of their failures and the (potential) effectiveness of various techniques for preventing and mitigating service failure. we find that (1) operator error is the largest cause of failures in two of the three services, (2) operator errors is the largest contributor to time to repair in two of the three services, (3) configuration errors are the largest category of operator errors, (4) failures in custom-written front-end software are significant, and (5) more extensive online testing and more thoroughly exposing and detecting component failures would reduce failure rates in at least one service. qualitatively we find that improvement in the maintenance tools and systems used by service operations staff would decrease time to diagnose and repair problems.
a highly scalable electronic mail service using open systems. email is one of the most important of the internet services. as a very large, fast growing, internet service provider, earthlink requires a robust and powerful email architecture that will support rapid expansion. this paper describes such an architecture, its motivations, its future, and the difficulties in implementing a service on this scale.
on the performance of tcp splicing for url-aware redirection. this paper describes the design, implementation and performance of a layer-7 switch which supports url-aware redirection of http traffic. currently, there are several vendors who are beginning to announce the availability of such switches in the market, but little or no implementation and performance information is available. we discuss design issues pertaining to such switches through a prototype implementation of a url-aware switch in the linux kernel, and analyze the performance of our implementation. we investigate the use of tcp splicing as a mechanism for improving the performance of the switch; we explore whether tcp splicing will benefit url-aware redirection even though http connections, on average, are short-lived and transfer small amounts of data. results from our implementation show that tcp splicing does improve the performance of url-aware switches that handle short-lived http connections. our results also re-affirm earlier findings that tcp splicing substantially improves the performance of any application-layer proxy when large amounts of data are transferred through the splice.
connection scheduling in web servers. under high loads, a web server may be servicing many hundreds of connections concurrently. in traditional web servers, the question of the order in which concurrent connections are serviced has been left to the operating system. in this paper we ask whether servers might provide better service by using non-traditional service ordering. in particular, for the case when a web server is serving static files, we examine the costs and benefits of a policy that gives preferential service to short connections. we start by assessing the scheduling behavior of a commonly used server (apache running on linux) with respect to connection size and show that it does not appear to provide preferential service to short connections. we then examine the potential performance improvements of a policy that does favor short connections (shortest-connection-first). we show that mean response time can be improved by factors of four or five under shortest-connection-first, as compared to an (apache-like) size-independent policy. finally we assess the costs of shortest-connection-first scheduling in terms of unfairness (i.e., the degree to which long connections suffer). we show that under shortest-connection-first scheduling, long connections pay very little penalty. this surprising result can be understood as a consequence of heavy-tailed web server workloads, in which most connections are small, but most server load is due to the few large connections. we support this explanation using analysis.
using the structure of html documents to improve retrieval. the world wide web (www) is a gigantic information resource, which is growing daily. as more and more data are added to the www, it is becoming increasingly difficult to effectively locate useful information from this environment. in this paper, we propose a method for making use of the structures and hyperlinks of html documents to improve the effectiveness of retrieving html documents. our study assigns the occurrences of terms in a document collection into six classes according to the tags in which a particular term appears (such as title, h1-h6, and anchor). based on the assignment, we extend the weighting schemes in traditional information retrieval by incorporating different importance factors to terms in different classes. the rationale is that terms appearing in different places of a document may have different significance in identifying the document. for this research we have built a web based search tool, webor, created a testbed, and conducted extensive experiments to determine an optimal class importance factor combination. our study indicates that substantial improvement of retrieval effectiveness can be achieved using this technique.
rate of change and other metrics: a live study of the world wide web. caching in the world wide web is based on two critical assumptions: that a significant fraction of requests reaccess resources that have already been retrieved; and that those resources do not change between accesses. we tested the validity of these assumptions, and their dependence on characteristics of web resources, including access rate, age at time of reference, content type, resource size, and internet top-level domain. we also measured the rate at which resources change, and the prevalence of duplicate copies in the web. we quantified the potential benefit of a shared proxy-caching server in a large environment by using traces that were collected at the internet connection points for two large corporations, representing significant numbers of references. only 22% of the resources referenced in the traces we analyzed were accessed more than once, but about half of the references were to those multiply-referenced resources. of this half, 13% were to a resource that had been modified since the previous traced reference to it. we found that the content type and rate of access have a strong influence on these metrics, the domain has a moderate influence, and size has little effect. in addition, we studied other aspects of the rate of change, including semantic differences such as the insertion or deletion of anchors, phone numbers, and email addresses.
hpp: html macro-preprocessing to support dynamic document caching. a number of techniques are available for reducing latency and bandwidth requirements for resources on the world wide web, including caching, compression, and delta-encoding [12]. these approaches are limited: much data on the web is dynamic, for which traditional caching is of limited use, and delta-encoding requires both a common version base against which to apply a delta and the complete generation of the resource prior to encoding it. in contrast to these approaches, we take an application-specific view, in which we separate the static and dynamic portions of a resource. the static portions (called the template) can then be cached, with (presumably small) dynamic portions obtained on each access. our html extension, which we refer to as hpp (for html pre-processing) supports resources that contain variable number of static and dynamic elements, such as query responses. results with macro-encoding of query response resources from local cgi scripts and two popular search engines indicate that our approach promises a substantial reduction of network traffic, server load, and access latency for dynamic documents. the size of network transfers using hpp are comparable to delta-encoding (factors of 2-8 smaller than the original resource), while the data generated by content providers is simpler, and the load on the end-servers is slightly lower.
model-based resource provisioning in a web service utility. internet service utilities host multiple server applications on a shared server cluster. a key challenge for these systems is to provision shared resources on demand to meet service quality targets at least cost. this paper presents a new approach to utility resource management focusing on coordinated provisioning of memory and storage resources. our approach is model-based: it incorporates internal models of service behavior to predict the value of candidate resource allotments under changing load. the model-based approach enables the system to achieve important resource management goals, including differentiated service quality, performance isolation, storage-aware caching, and proactive allocation of surplus resources to meet performance goals. experimental results with a prototype demonstrate model-based dynamic provisioning under web workloads with static content.
prefetching hyperlinks. this paper develops a new method for prefetching web pages into the client cache. clients send reference information to web servers, which aggregate the reference information in near-real-time and then disperse the aggregated information to all clients, piggybacked on get responses. the information indicates how often hyperlink urls embedded in pages have been previously accessed relative to the embedding page. based on knowledge about which hyperlinks are generally popular, clients initiate prefetching of the hyperlinks and their embedded images according to any algorithm they prefer. both client and server may cap the prefetching mechanism's space overhead and waste of network resources due to speculation. the result of these differences is improved prefetching: lower client latency (by 52.3%) and less wasted network bandwidth (24.0%).
the measured access characteristics of world-wide-web client proxy caches. the growing popularity of the world wide web is placing tremendous demands on the internet. a key strategy for scaling the internet to meet these increasing demands is to cache data near clients and thus improve access latency and reduce network and server load. unfortunately, research in this area has been hampered by a poor understanding of the locality and sharing characteristics of web-client accesses. the recent popularity of web proxy servers provides a unique opportunity to improve this understanding, because a small number of proxy servers see accesses from thousands of clients. this paper presents an analysis of access traces collected from seven proxy servers deployed in various locations throughout the internet. the traces record a total of 47.4 million requests made by 23,700 clients over a twenty-one day period. we use a combination of static analysis and trace-driven cache simulation to characterize the locality and sharing properties of these accesses. our analysis shows that a 2- to 10-gb second-level cache yields hit rates between 24% and 45% with 85% of these hits due to sharing among different clients. caches with more clients exhibit more sharing and thus higher hit rates. between 2% and 7% of accesses are consistency misses to unmodified objects, using the squid and cern proxy cache coherence protocols. sharing is bimodal. requests for shared objects are divided evenly between objects that are narrowly shared and those that are shared by many clients; widely shared objects also tend to be shared by clients from unrelated traces.
energy conservation policies for web servers. energy management for servers is now necessary for technical, financial, and environmental reasons. this paper describes three policies designed to reduce energy consumption in web servers. the policies employ two power management mechanisms: dynamic voltage scaling (dvs), an existing mechanism, and request batching, a new mechanism introduced in this paper. the first policy uses dvs in isolation, except that we extend recently introduced task-based dvs policies for use in server environments with many concurrent tasks. the second policy uses request batching to conserve energy during periods of low workload intensity. the third policy uses both dvs and request batching mechanisms to reduce processor energy usage over a wide range of workload intensities. all the policies trade off system responsiveness to save energy. however, the policies employ the mechanisms in a feedback-driven control framework in order to conserve energy while maintaining a given quality of service level, as defined by a percentile-level response time. we evaluate the policies using salsa, a web server simulator that has been extensively validated for both energy and response time against measurements from a commodity web server. three daylong static web workloads from real web server systems are used to quantify the energy savings: the nagano olympics98 web server, a financial services company web site, and a disk intensive web workload. our results show that when required to maintain a 90th-percentile response time of 50ms, the dvs and request batching policies save from 8.7% to 38% and from 3.1% to 27% respectively of the cpu energy used by the base system. the two polices provide these savings for complementary workload intensities. the combined policy is effective for all three workloads across a broad range of intensities, saving from 17% to 42% of the cpu energy.
the ninja jukebox. we present the design and implementation of the "ninja jukebox", an infrastructural service that allows a community of users to build a distributed, collaborative music repository that delivers digital music to internet clients, and that performs simple collaborative filtering based on users' song preferences inferred by the service. the jukebox, implemented in java, was designed to allow rapid service evolution and reconfiguration, simplicity in participation, and extensibility. we demonstrate that our careful use of a distributed component architecture enabled rapid prototyping of the service, and that our use of carefully designed, strongly typed interfaces enabled the smooth evolution of the service from a simple prototype to a more complex, mature system.
going beyond the sandbox: an overview of the new security architecture in the java development kit 1.2. this paper describes the new security architecture that has been implemented aspart of jdk1.2, the forthcoming javatm development kit. in going beyond the sandbox security model in the original release of java, jdk1.2 provides fine-grained access control via an easily configurable security policy. moreover, jdk1.2 introduces the concept of protection domain and a few related security primitives that help to make the underlying protection mechanism more robust.
system design issues for internet middleware services: deductions from a large client trace. in this paper, we present the analysis of a large client-side web trace gathered from the home ip service at the university of california at berkeley. specifically, we demonstrate the heterogeneity of web clients, the existence of a strong and very predictable diurnal cycle in the clients' web activity, the burstiness of clients' requests at small time scales (but not large time scales, implying a lack of self-similarity), the presence of locality of reference in the clients' requests that is a strong function of the client population size, and the high latency that services encounter when delivering data to clients, implying that services will need to maintain a very large number of simultaneously active requests. we then present system design issues for internet middleware services that were drawn both from our trace analysis and our implementation experience of the transend transformation proxy.
skipnet: a scalable overlay network with practical locality properties. scalable overlay networks such as chord, can, pastry, and tapestry have recently emerged as flexible infrastructure for building large peer-to-peer systems. in practice, such systems have two disadvantages: they provide no control over where data is stored and no guarantee that routing paths remain within an administrative domain whenever possible. skipnet is a scalable overlay network that provides controlled data placement and guaranteed routing locality by organizing data primarily by string names. skipnet allows for both fine-grained and coarse-grained control over data placement: content can be placed either on a pre-determined node or distributed uniformly across the nodes of a hierarchical naming sub-tree. an additional useful consequence of skipnet's locality properties is that partition failures, in which an entire organization disconnects from the rest of the system, can result in two disjoint, but well-connected overlay networks.
a document-based framework for internet application control. this paper motivates and details a document-based framework for manipulating the components that comprise distributed internet applications. in the framework, xml documents are used to describe both server-side functionality and the mapping between a client's applications and the servers it accesses. our system model contrasts with explicitly context-aware application designs, where location information must be explicitly manipulated by the application to affect change; instead, a middleware layer is interposed between client applications and services so that invocations between the two can be transparently remapped. this approach is useful for a subset of application domains, including our example domain of "remote control" of local resources (e.g., lights, stereo components, etc.). we illustrate how the framework allows for 1) remapping of a portion of an existing user interface to a new service, 2) viewing of arbitrary subsets and combinations of the available functionality, and 3) mixing dynamically-generated user interfaces with existing user interfaces. the use of a document-based framework in addition to a conventional object-oriented programming language provides a number of key features. one of the most useful is that it exposes the mappings between programs/uis and the objects to which they refer, thereby providing a standard location for manipulation of this indirection.
network-sensitive service discovery. we consider the problem of network-sensitive service selection (nsss): finding services that match a particular set of functional and network properties. current solutions handle this problem using a two-step process. first, a user obtains a list of candidates through service discovery. then, the user applies a network-sensitive server selection technique to find the best service. such approaches are complex and expensive since each user has to solve the nsss problem independently. in this paper, we present a simple alternative: network-sensitive service discovery (nssd). by integrating network-sensitivity into the service discovery process, nssd allows users who are looking for services to specify both the desired functional and network properties at the same time. users benefit since they only have to solve a restricted version of the server selection problem. moreover, nssd can solve the nsss problem more efficiently by amortizing the overhead over many users. we present the design of nssd, a prototype implementation, and experimental results that illustrate how nssd can be utilized for different applications.
improving web server performance by caching dynamic data. dynamic web pages can seriously reduce the performance of web servers. one technique for improving performance is to cache dynamic web pages. we have developed the dynamic web cache which is particularly well-suited for dynamic pages. our cache has improved performance significantly at several commercial web sites. this paper analyzes the design and performance of the dynamicweb cache. it also presents a model for analyzing overall system performance in the presence of caching. our cache can satisfy several hundred requests per second. on systems which invoke server programs via cgi, the dynamicweb cache results in near-optimal performance, where optimal performance is that which would be achieved by a hypothetical cache which consumed no cpu cycles. on a system we tested which invoked server programs via icapi which has significantly less overhead than cgi, the dynamicweb cache resulted in near-optimal performance for many cases and 58% of optimal performance in the worst case. the dynamicweb cache achieved a hit rate of around 80% when it was deployed to support the official internet web site for the 1996 atlanta olympic games.
a study of the performance potential of dht-based overlays. we use simulation to study whether overlays based on the recent distributed hash tables (dhts) have the potential to deliver performance comparable to that of overlays based on measurements. our work is motivated by the use of dhts for services such as multicast, which is already targeted by measurement-based overlays; there is currently little understanding of how the two approaches compare at scales where both are viable. we compare three dht-based overlays (can, chord and pastry) with two measurement-based overlays (narada and nice), as well as power-law random graphs (plrgs) that represent gnutella. to enable comparisons, we configure the overlays with the same average out-degree and focus on moderate scale. to gauge potential, we look at current and idealized dht algorithms. we find that basic versions of dhts have a latency stretch that is at least twice that of nice and narada, but similar performance in terms of bandwidth hotspots. however, dht performance can be improved considerably with routing heuristics and topology-aware overlay construction, which have the potential to bring dht performance at par with nice. we also report on performance of overlays with power-law structure and the impact of hierarchy on performance.