Optimistic compiler optimizations for network systems

This dissertation describes techniques that can optimize the performance of modernday network systems. They are applied through the analysis and transformation of programs that implement network protocols. The first of these techniques involves the use of Program Specialization, a well-established code-optimization approach, to optimize network protocol stacks. The second, Remote Specialization makes specialization amenable to resource-limited embedded systems by deferring it over the network to a more capable system. The third technique revolves around a novel memory manager introduced in this thesis and optimizes a network server's use of the underlying hardware caches. Finally, the fourth technique uses static analysis to integrate the proposed memory manager with an existing network server. Improving the performance of network systems is one of the most researched areas in the domain of Operating Systems and Networking. It has led to the publication of hundreds of research papers and over a score of PhD theses over the past decade. In spite of the large body of available literature, it is still the focus of intense study. The author's intuition of the cause of this phenomenon is as follows. In the course of the past two decades, network technology has mutated at every level from the transmission medium to the manipulation of application-level protocol messages. Each mutation combined with parallel mutations in computing hardware and hanging trends in Internet commerce, has altered the problematic of network-system optimization, consistently raising the need for further research. Given the highly dynamic nature of this domain, any optimization that involves removing pathological inefficiencies in network software is likely to be outdated by the next evolution in network technology and computer architecture. In contrast, the goal of this thesis is to study the performance of network systems and build optimizations based on a set of tenets that we hope will outlive a sizable number of evolutions and mutations in computer and network systems. These tenets are inspired by two principle properties of network software: their active manipulation of memory objects and their layered structure. The first tenet is based on the mismatch between the memory latency and compute latency of modern-day processors. The computational activities carried out in most network applications are lightweight and do not involve complex mathematical operations, making the overhead of memory latency all the more significant. We will demonstrate that by strategically managing the behaviour of a server with respect to the underlying hardware caches, the number of memory accesses can be reduced drastically, boosting the performance of a network server. The main mechanism that drives these changes is implemented through the co-design of a memory manager and a scheduling algorithm. The memory manager is novel, and a contribution of this thesis. The scheduling algorithm is a variant of one recently introduced in the literature. The combination of the scheduling strategy and memory manager is integrated into an existing server using a set of program analysis tools, developed in the context of this thesis. The second tenet involves the layered complexity of network software. Implementations of protocol stacks are highly generic owing to their development as components of generic operating systems. These implementations often evolve over several years through which they gain features, performance and stability, but at the cost of added complexity. The natural solution to optimizing such protocol stacks has been to redesign them in a way that allows a programmer to collapse them at the time they are used, removing unnecessary functionalities and layers. While this approach has been shown to yield compelling results in terms of the performance and the reduced size of the resulting code, it is impractical to apply to existing, mature implementations. In this thesis, we use program specialization to transform generic implementations of protocol stacks into efficient, specialized implementations. This process is supported by a novel remote specialization infrastructure that allows the resource-intensive activity of program specialization to be carried out on a remote specialization server. The individual solutions developed in this thesis share a common theme: they are implemented as compiler optimizations. Coupling them with traditional approaches such as profile-guided optimization and efficient OS primitives that have issued from the industry and research will yield network applications whose performance approaches the theoretical upper limit.