Architecture Sensitive Database Design: Examples from the Columbia Group
In this article, we discuss how different aspects of modern CPU architecture can be effectively used by database systems. While we shall try to cover most important aspects of modern architectures, we shall not aim to provide an exhaustive survey of the literature. Rather, we will use examples from our own recent work to illustrate the principles involved. While there is related work on using other components of modern machines such as graphics cards and intelligent disk controllers for database work, we limit our discussion here to just a single CPU and the memory hierarchy (excluding disks). In recent years the primary focus of database performance research has fundamentally shifted from disks to main memory. Systems with a very large main memory allow a database’s working set to fit in RAM. At the same time processors have continued to double in performance roughly every eighteen months in line with “Moore’s Law,” while memory speed improvements have been marginal. The CPU now spends many hundreds of cycles waiting for data to arrive from main memory, and database workloads are often memory-bound. Because of this bottleneck, database research at Columbia and elsewhere has focused on effectively using the cache hierarchy to improve database performance. Changes in processor technology have also influenced database research. Modern processors feature a very long pipeline, which helps to provide better performance. It also results in a higher branch misprediction penalty, because the pipeline must be flushed and restarted when a branch is mispredicted. Modern CPUs provide the database programmer with a rich instruction set. Using specialized instructions, such as SIMD instructions, database performance can be improved by allowing a degree of parallelism and eliminating some conditional branch instructions, thus reducing the impact of branch mispredictions.

title: A multi-resolution block storage model for database design
abstract: We propose a new storage model called MBSM (multiresolution block storage model) for laying out tables on disks. MBSM is intended to speed up operations such as scans that are typical of data warehouse workloads. Disk blocks are grouped into "super-blocks," with a single record stored in a partitioned fashion among the blocks in a superblock. The intention is that a scan operation that needs to consult only a small number of attributes can access just those blocks of each super-block that contain the desired attributes. To achieve good performance given the physical characteristics of modern disks, we organize super-blocks on the disk into fixed-size "mega-blocks." Within a megablock, blocks of the same type (from various super-blocks) are stored contiguously. We describe the changes needed in a conventional database system to manage tables using such a disk organization. We demonstrate experimentally that MBSM outperforms competing approaches such as NSM (N-ary storage model), DSM (decomposition storage model) and PAX (partition attributes across), for I/O bound decision-support workloads consisting of scans in which not all attributes are required. This improved performance comes at the expense of single-record insert and delete performance; we quantify the trade-offs involved. Unlike DSM, the cost of reconstructing a record from its partitions is small. MBSM stores attributes in a vertically partitioned manner similar to PAX, and thus shares PAX's good CPU cache behavior. We describe methods for mapping attributes to blocks within super-blocks in order to optimize overall performance, and show how to tune the super-block and mega-block sizes.

title: Implementing database operations using SIMD instructions
abstract: Modern CPUs have instructions that allow basic operations to be performed on several data elements in parallel. These instructions are called SIMD instructions, since they apply a single instruction to multiple data elements. SIMD technology was initially built into commodity processors in order to accelerate the performance of multimedia applications. SIMD instructions provide new opportunities for database engine design and implementation. We study various kinds of operations in a database context, and show how the inner loop of the operations can be accelerated using SIMD instructions. The use of SIMD instructions has two immediate performance benefits: It allows a degree of parallelism, so that many operands can be processed at once. It also often leads to the elimination of conditional branch instructions, reducing branch mispredictions.We consider the most important database operations, including sequential scans, aggregation, index operations, and joins. We present techniques for implementing these using SIMD instructions. We show that there are significant benefits in redesigning traditional query processing algorithms so that they can make better use of SIMD technology. Our study shows that using a SIMD parallelism of four, the CPU time for the new algorithms is from 10% to more than four times less than for the traditional algorithms. Superlinear speedups are obtained as a result of the elimination of branch misprediction effects.

title: Improving Database Performance on Simultaneous Multithreading Processors
abstract: Simultaneous multithreading (SMT) allows multiple threads to supply instructions to the instruction pipeline of a superscalar processor. Because threads share processor resources, an SMT system is inherently different from a multiprocessor system and, therefore, utilizing multiple threads on an SMT processor creates new challenges for database implementers.We investigate three thread-based techniques to exploit SMT architectures on memory-resident data. First, we consider running independent operations in separate threads, a technique applied to conventional multi-processor systems. Second, we describe a novel implementation strategy in which individual operators are implemented in a multi-threaded fashion. Finally, we introduce a new data-structure called a work-ahead set that allows us to use one of the threads to aggressively preload data into the cache.We evaluate each method with respect to its performance, implementation complexity, and other measures. We also provide guidance regarding when and how to best utilize the various threading techniques. Our experimental results show that by taking advantage of SMT technology we achieve a 30% to 70% improvement in throughput over single threaded implementations on in-memory database operations.

title: TWINE: A Hybrid Emulation Testbed for Wireless Networks and Applications
abstract: In this paper, we present TWINE, a high fidelity, efficient emulation framework that combines the accuracy and realism of emulated and physical networks and the scalability and repeatability of simulation in an integrated testbed, for evaluation of real protocols and applications. Our measurements show that the TWINE emulation kernel has a memory footprint of less than 100KB, and occupies no more than 3.5% CPU cycles. Thanks to such small overhead and the accurate modelling of physical layer events(at microseconds level), application throughput measured in TWINE is within 5% of the measured throughput from an equivalent physical wireless LAN. A single commodity PC in TWINE can emulate at least four wireless hosts or simulate sixty nodes in real time at microseconds granularity. This paper also illustrates TWINE’s novel capabilities via two case studies: a protocol to maintain fairness in mesh networks and an adaptive streaming media application operating in heterogeneous wireless networks. The results from the case studies clearly show the benefit of the TWINE evaluation methodology, by identifying a mismatch between the performance of the protocol or application based on actual user experience versus its performance as measured using traditional network performance metrics such as application throughput.

title: An intensive survey of fair non-repudiation protocols
abstract: With the phenomenal growth of the Internet and open networks in general, security services, such as non-repudiation, become crucial to many applications. Non-repudiation services must ensure that when Alice sends some information to Bob over a network, neither Alice nor Bob can deny having participated in a part or the whole of this communication. Therefore a fair non-repudiation protocol has to generate non-repudiation of origin evidences intended to Bob, and non-repudiation of receipt evidences destined to Alice. In this paper, we clearly define the properties a fair non-repudiation protocol must respect, and give a survey of the most important non-repudiation protocols without and with trusted third party (TTP). For the later ones we discuss the evolution of the TTP's involvement and, between others, describe the most recent protocol using a transparent TTP. We also discuss some ad-hoc problems related to the management of non-repudiation evidences.

title: An Efficient Public-Key Framework
abstract: Public-key certificates play an important role in binding the public key with the identity of the owner of the corresponding private key. A certificate might be revoked before its scheduled expiry date by the issuing CA. Efficient and timely distribution of certificate revocation information is a big challenge facing the PKI providers. Existing certificate revocation schemes place a considerable processing, communication, and storage overheads on the CA as well as the relying parties. To improve the current situation, we propose a revocation-free public-key framework, in which the maximum lifetime of a certificate is divided into short periods and the certificate could expire at the end of any period under the control of the certificate owner (or his manager in a corporate environment). The verifier can check the status of such a certificate without retrieving the revocation information from the CA. The new framework is especially useful for applications on wireless devices that are unable to make simultaneous connections. The new framework could be easily integrated into existing PKI products that support X.509-based certificates.

title: Toward automatic annotation of in situ mRNA expression patterns of drosophila embryos
abstract: The in situ mRNA hybridization gene expression pattern images of Drosophila melanogaster in the course of embryogenesis provide spatial-temporal information of the expression patterns of a target gene. This information is critical for understanding the roles of genes during the development of embryos. Currently, the annotation of these images is often done by manually assigning a subset of image ontology terms to the images. This approach is time consuming and depends heavily on the consistency of the experts. Alternatively, if the annotation process can be automated or semi-automated, efficiency and consistency are likely to be greatly enhanced. We formulate this task as a pattern classification problem, and present preliminary results. We consider both the template-based global matching and the multi-objective classification based on neural networks (e.g. multi-layer perceptron). We develop a method to combine them to optimize efficiency and accuracy. This method has been applied to the gene expression pattern image database generated by the Berkeley Drosophila Genome Project.

title: Optimizing parallel execution of detailed wireless network simulation
abstract: With parallel and discrete event simulation (PDES) techniques, the runtime performance of detailed wireless network simulation can be improved significantly without compromising fidelity of the simulation results. However, modelling characteristics of wireless communications such as signal propagation and interference may severely hinder the potential speedup yielded by PDES. This paper proposes various optimization techniques to address three major concerns in achieving efficient parallel execution of wireless network simulation: i.e., (1) reducing communication and computation overhead of simulating signal propagation across multiple logical processes; (2) reducing synchronization overhead among logical processes; (3) minimizing event scheduling overhead within individual logical processes. These techniques have been implemented in a parallel version of GloMoSim and QualNet. The experimental results with mobile ad hoc networking scenarios demonstrate that the proposed optimization techniques can improve the performance of parallel wireless network simulation by up to an order of magnitude.