waashk/dblp huggingface dataset:
porting kernel code to four bsds and linux. the u.s. naval research laboratory develops and maintains a freely available ipv6 and ip security distribution. all of the software builds and runs on bsd/os, freebsd, netbsd, and openbsd, and a growing portion of the software builds and runs on linux. each of the four bsds has evolved significantly from their original 4.4bsd-lite ancestor, and increasingly more of that evolution is along divergent paths. linux shares no significant ancestry with the bsds, but is still a posix system, which means that many of the same high-level facilities are available even though their implementation might be completely different. this paper discusses many of the differences and many of the similarities we encountered in the internals of these systems. it also discusses the techniques and glue software that we developed for isolating and abstracting the differences so that we could build a significant base of system code that is portable between all five systems.
tcl: an embeddable command language. tel is an interpreter for a tool command language. it consists of a library package that is embedded in tools (such as editors, debuggers, etc.) as the basic command interpreter. tel provides (a) a parser for a simple textual command language, (b) a collection of built-in utility commands, and a c interface that tools use to augment the built-in commands with tool-specific commands. tel is particularly attractive when integrated with the widget library of a window system; it increases the programmability of the widgets by providing mechanisms for variables, procedures, expressions, etc, it allows users to program both the appearance and the actions of widgets; and it offers a simple but powerful communication mechanism between interactive programs.
managing volunteer activity in free software projects. during the last few years, thousands of volunteers have created a large body of free software. even though this accomplishment shows that the free software development model works, there are some drawbacks associated with this model. due to the volunteer nature of most free software projects, it is impossible to fully rely on participants. volunteers may become busy and neglect their duties. this may lead to a steady decrease of quality as work is not being carried out. the problem of inactive volunteers is intensified by the fact that most free software projects are distributed, which makes it hard to quickly identify volunteers who neglect their duties. this paper shows debian's approach to inactive volunteers. insights presented here can be applied to other free software projects in order to implement effective quality assurance strategies.
efficiently scheduling x clients. the x server is charged with providing window system services to many applications simultaneously, and needs a scheduling mechanism to distribute it's limited resources among these applications. the original scheduling mechanism was simplistic and caused graphics-intensive applications to starve interactive applications. a new scheduling mechanism has been designed which fairly distributes time among the requesting applications while at the same time increasing performance by a small amount. descriptions of the original and new scheduling mechanism and empirical measurements demonstrating the effects of scheduling within the x server are included along with a discussion on how the design was arrived at.
an analysis of file migration in a unix supercomputing environment. the supercomputer center at the national center for atmospheric research (ncar) migrates large numbers of files to and from its mass storage system (mss) because there is insufficient space to store them on the cray supercomputer''s local disks. this paper presents an analysis of file migration data collected over two years. the analysis shows that requests to the mss are periodic, with one day and one week periods. read requests to the mss account for the majority of the periodicity; as write requests are relatively constant over the course of a week. additionally, reads show a far greater fluctuation than writes over a day and week since reads are driven by human users while writes are machine-driven.
a new rendering model for x. x version 11 [sg92] was originally designed and implemented in 1987. in the intervening 13 years, there have been advancements in both applications and hardware, but the core of the x window system has remained largely unchanged. the last major x server architecture changes were included in x11r4. the last wide-spread functional enhancement exported by the x server might well be the shape extension [pac89], designed (in the hot tub) at the 1989 winter usenix in san diego. the rise of inexpensive unix desktop systems in the last couple of years has led to the development of new user-interface libraries, which are not well served by the existing x rendering model. a new 2d rendering model is being developed to serve this new community of applications. the problem space and proposed solutions are discussed.
rama: easy access to a high-bandwidth massively parallel file system. massively parallel file systems must provide high bandwidth file access to programs running on their machines. most accomplish this goal by striping files across arrays of disks attached to a few specialized i/o nodes in the massively parallel processor (mpp). this arrangement requires programmers to give the file system many hints on how their data is to be laid out on disk if they want to achieve good performance. additionally, the custom interface makes massively parallel file systems hard for programmers to use and difficult to seamlessly integrate into an environment with workstations and tertiary storage. the rama file system addresses these problems by providing a massively parallel file system that does not need user hints to provide good performance. rama takes advantage of the recent decrease in physical disk size by assuming that each processor in an mpp has one or more disks attached to it. hashing is then used to pseudo-randomly distribute data to all of these disks, insuring high bandwidth regardless of access pattern. since mpp programs often have many nodes accessing a single file in parallel, the file system must allow access to different parts of the file without relying on a particular node. in rama, a file request involves only two nodes -- the node making the request and the node on whose disk the data is stored. thus, rama scales well to hundreds of processors. since rama needs no layout hints from applications, it fits well into systems where users cannot (or will not) provide such hints. fortunately, this flexibility does not cause a large loss of performance. rama's simulated performance is within 10-15% of the optimum performance of a similarly-sized striped file system, and is a factor of 4 or more better than a striped file system with poorly laid out data.
integrating a command shell into a web browser. the transition from command-line interfaces to graphical interfaces has resulted in programs that are easier to learn and use, but harder to automate and reuse. another transition is now underway, to html interfaces hosted by a web browser. to help users automate html interfaces, we propose the browser-shell, a web browser that integrates a command interpreter into the browser's location box. the browser-shell's command language is designed for extracting and manipulating html and text, and commands can also invoke local programs. command input is drawn from the current browser page, and command output is displayed as a new page. the browser-shell brings to web browsing many advantages of the unix shell, including scripting web services and creating pipelines of web services and local programs. a browser-shell also allows legacy command-line programs to be wrapped with an html/cgi interface that is graphical but still scriptable, and offers a new shell interaction model, different from the conventional typescript model, which may improve usability in some respects.
lightweight structured text processing. text is a popular storage and distribution format for information, partly due to generic text-processing tools like unix grep and sort. unfortunately, existing generic tools make assumptions about text format (e.g., each line is a record) that limit their applicability. custom-built tools are one alternative, but they require substantial time investment and programming expertise. we describe a new approach, lightweight structured text processing, which overcomes these difficulties by enabling users to define text structure interactively and manipulate the structure with generic tools. our prototype system, lapis, is a web browser that can highlight, filter, and sort text regions described by the user. lapis has several advantages over other systems: (1) the ability to define custom structure with a simple, intuitive pattern language; (2) interactive specification, showing pattern matches in context and letting users choose the most convenient combination of manual selection and pattern matching; and (3) external parsers for standard text formats. the pattern language in lapis, text constraints, describes text structure in high-level terms, with region relationships like before, after, in, and contains. we describe an implementation of text constraints using a novel, compact representation of region sets as collections of rectangles, or region intervals. we also illustrate some examples of applying lapis to web pages, text files, and source code.
ask: active spam killer. we present active spam killer (ask), a program that attempts to validate unknown senders before allowing delivery of their message. validation occurs by means of a challenge reply sent to senders who are not yet known to the system. messages are kept in a queue pending confirmation until the sender replies to the challenge. further messages coming from confirmed senders are delivered immediately. in a sample of 1000 spam mails, ask was 99.7% effective at blocking spam, resulting in only 3 spam messages being delivered. other programs' best ratios were 97.8% or as many as 22 spam messages delivered.
strlcpy and strlcat - consistent, safe, string copy and concatenation. as the prevalence of buffer overflow attacks has increased, more and more programmers are using size or length-bounded string functions such as strncpy() and strncat(). while this is certainly an encouraging trend, the standard c string functions generally used were not really designed for the task. this paper describes an alternate, intuitive, and consistent api designed with safe string copies in mind. there are several problems encountered when strncpy() and strncat() are used as safe versions of strcpy() and strcat(). both functions deal with nul-termination and the length parameter in different and non-intuitive ways that confuse even experienced programmers. they also provide no easy way to detect when truncation occurs. finally, strncpy() zero-fills the remainder of the destination string, incurring a performance penalty. of all these issues, the confusion caused by the length parameters and the related issue of nul-termination are most important. when we audited the openbsd source tree for potential security holes we found rampant misuse of strncpy() and strncat(). while not all of these resulted in exploitable security holes, they made it clear that the rules for using strncpy() and strncat() in safe string operations are widely misunderstood. the proposed replacement functions, strlcpy() and strlcat(), address these problems by presenting an api designed for safe string copies (see figure 1 for function prototypes). both functions guarantee nul-termination, take as a length parameter the size of the string in bytes, and provide an easy way to detect truncation. neither function zero-fills unused bytes in the destination.
flash: an efficient and portable web server. this paper presents the design of a new web server architecture called the asymmetric multi-process event-driven (amped) architecture, and evaluates the performance of an implementation of this architecture, the flash web server. the flash web server combines the high performance of single-process event-driven servers on cached workloads with the performance of multiprocess and multi-threaded servers on disk-bound workloads. furthermore, the flash web server is easily portable since it achieves these results using facilities available in all modern operating systems. the performance of different web server architectures is evaluated in the context of a single implementation in order to quantify the impact of a server's concurrency architecture on its performance. furthermore, the performance of flash is compared with two widely-used web servers, apache and zeus. results indicate that flash can match or exceed the performance of existing web servers by up to 50% across a wide range of real workloads. we also present results that show the contribution of various optimizations embedded in flash.
a better update policy. some file systems can delay writing modified data to disk, in order to reduce disk traffic and overhead. prudence dictates that such delays be bounded, in case the system crashes. we refer to an algorithm used to decide when to write delayed data back to disk as an update policy. traditional unix® systems use a periodic update policy, writing back all delayed-write data once every 30 seconds. periodic update is easy to implement but performs quite badly in some cases. this paper describes an approximate implementation of an interval periodic update policy, in which each individual delayed-write block is written when its age reaches a threshold. interval periodic update adds little code to the kernel and can perform much better than periodic update. in particular, interval periodic update can avoid the huge variances in read response time caused by using periodic update with a large buffer cache.
pandora: a flexible network monitoring platform. this paper presents pandora, a network monitoring platform that captures packets using purely passive techniques. pandora addresses current needs for improving internet middleware and infrastructure by providing both in-depth understanding of network usage and metrics to compare existing protocols. pandora is flexible and easy to use and deploy. the elementary monitoring tasks are encapsulated as independent entities we call monitoring components. the actual packet analysis is performed by stacking the appropriate components. pandora also preserves user privacy by allowing control of the "anonymization" policy. finally, the evaluation we conducted shows that overheads due to pandora's flexibility do not significantly affect performance. pandora is fully functional and has already been used to collect web traffic traces at inria rocquencourt.
union mounts in 4.4bsd-lite. this paper describes the design and rationale behind union mounts, a new filesystem-namespace management tool available in 4.4bsd-lite. unlike a traditional mount that hides the contents of the directory on which it is placed, a union mount presents a view of a merger of the two directories. although only the filesystem at the top of the union stack can be modified, the union filesystem gives the appearance of allowing anything to be deleted or modified. files in the lower layer may be deleted with whiteout in the top layer. files to be modified are automatically copied to the top layer. this new functionality makes possible several new applications including the ability to apply patches to a cd-rom and eliminate symbolic links generated by an automounter. also possible is the provision of per-user views of the filesystem, allowing private views of a shared work area, or local builds from a centrally shared read-only source tree.
implementing lottery scheduling: matching the specializations in traditional schedulers. we describe extensions to lottery scheduling, a proportional-share resource management algorithm, to provide the performance assurances present in traditional non-real time process schedulers. lottery scheduling enables flexible control over relative process execution rates with a ticket abstraction and provides load insulation among groups of processes using currencies. we first show that a straightforward implementation of lottery scheduling does not provide the responsiveness for a mixed interactive and cpu-bound workload offered by the decay usage priority scheduler of the freebsd operating system. moreover, standard lottery scheduling ignores kernel priorities used in the freebsd scheduler to reduce kernel lock contention. in this paper, we show how to use dynamic ticket adjustments to incorporate into a lottery scheduler the specializations present in the freebsd scheduler to improve interactive response time and reduce kernel lock contention. we achieve this while maintaining lottery scheduling's flexible control over relative execution rates and load insulation. in spite of added scheduling overhead, the throughput of cpu-bound workloads under our scheduler is within one percent of the freebsd scheduler for all but one test. we describe our design, evaluate our implementation, and relate our experience in deploying our hybrid lottery scheduler on production machines.
lexical file names in plan 9, or, getting dot-dot right. symbolic links make the unix file system non-hierarchical, resulting in multiple valid path names for a given file. this ambiguity is a source of confusion, especially since some shells work overtime to present a consistent view from programs such as pwd, while other programs and the kernel itself do nothing about the problem. plan 9 has no symbolic links but it does have other mechanisms that produce the same difficulty. moreover, plan 9 is founded on the ability to control a program's environment by manipulating its name space. ambiguous names muddle the result of operations such as copying a name space across the network. to address these problems, the plan 9 kernel has been modified to maintain an accurate path name for every active file (open file, working directory, mount table entry) in the system. the definition of 'accurate' is that the path name for a file is guaranteed to be the rooted, absolute name the program used to acquire it. these names are maintained by an efficient method that combines lexical processing--such as evaluating .. by just removing the last path name element of a directory--with local operations within the file system to maintain a consistently, easily understood view of the name system. ambiguous situations are resolved by examining the lexically maintained names themselves. a new kernel call, fd2path, returns the file name associated with an open file, permitting the use of reliable names to improve system services ranging from pwd to debugging. although this work was done in plan 9, unix systems could also benefit from the addition of a method to recover the accurate name of an open file or the current directory.
plumbing and other utilities. plumbing is a new mechanism for interprocess communication in plan 9, specifically the passing of messages between interactive programs as part of the user interface. although plumbing shares some properties with familiar notions such as cut and paste, it offers a more general data exchange mechanism without imposing a particular user interface. the core of the plumbing system is a program called the plumber, which handles all messages and dispatches and reformats them according to configuration rules written in a special-purpose language. this approach allows the contents and context of a piece of data to define how it is handled. unlike with drag and drop or cut and paste, the user doesn't need to deliver the data; the contents of a plumbing message, as interpreted by the plumbing rules, determine its destination. the plumber has an unusual architecture: it is a language-driven file server. this design has distinct advantages. it makes plumbing easy to add to an existing, unix-like command environment; it guarantees uniform handling of inter-application messages; it offloads from those applications most of the work of extracting and dispatching messages; and it works transparently across a network.
acme: a user interface for programmers. a hybrid of window system, shell, and editor, acme gives text-oriented applications a clean, expressive, and consistent style of interaction. traditional window systems support interactive client programs and offer libraries of pre-defined operations such as pop-up menus and buttons to promote a consistent user interface among the clients. acme instead provides its clients with a fixed user interface and simple conventions to encourage its uniform use. clients access the facilities of acme through a file system interface; acme is in part a file server that exports device-like files that may be manipulated to access and control the contents of its windows. written in a concurrent programming language, acme is structured as a set of communicating processes that neatly subdivide the various aspects of its tasks: display management, input, file server, and so on. acme attaches distinct functions to the three mouse buttons: the left selects text; the middle executes textual commands; and the right combines context search and file opening functions to integrate the various applications and files in the system. acme works well enough to have developed a community that uses it exclusively. although acme discourages the traditional style of interaction based on typescript windows-- teletypes--its users find acme's other services render typescripts obsolete.
libckpt: transparent checkpointing under unix. checkpointing is a simple technique for rollback recovery: the state of an executing program is periodically saved to a disk file from which it can be recovered after a failure. while recent research has developed a collection of powerful techniques for minimizing the overhead of writing checkpoint files, checkpointing remains unavailable to most application developers. in this paper we describe libckpt, a portable checkpointing tool for unix that implements all applicable performance optimizations which are reported in the literature. while libckpt can be used in a mode which is almost totally transparent to the programmer, it also supports the incorporation of user directives into the creation of checkpoints. this user-directed checkpointing is an innovation which is unique to our work.
alternatives for detecting redundancy in storage systems data. storage systems frequently maintain identical copies of data. identifying such data can assist in the design of solutions in which data storage, transmission, and management are optimised. in this paper we evaluate three methods used to discover identical portions of data: whole file content hashing, fixed size blocking, and a chunking strategy that uses rabin fingerprints to delimit content-defined data chunks. we assess how effective each of these strategies is in finding identical sections of data. in our experiments, we analysed diverse data sets from a variety of different types of storage systems including a mirrored section of sunsite.org.uk, different data profiles in the file system infrastructure of the cambridge university computer laboratory, source code distribution trees, compressed data, and packed files. we report our experimental results and present a comparative analysis of these techniques. this study also shows how levels of similarity differ between data sets and file types. finally, we discuss the advantages and disadvantages in the application of these methods in the light of our experimental results.