Online Transaction Processing (OLTP) databases include a suite of features -disk-resident B-trees and heap files, locking-based concurrency control, support for multi-threading -that were optimized for computer technology of the late 1970's. Advances in modern processors, memories, and networks mean that today's computers are vastly different from those of 30 years ago, such that many OLTP databases will now fit in main memory, and most OLTP transactions can be processed in milliseconds or less. Yet database architecture has changed little. Based on this observation, we look at some interesting variants of conventional database systems that one might build that exploit recent hardware trends, and speculate on their performance through a detailed instruction-level breakdown of the major components involved in a transaction processing database system (Shore) running a subset of TPC-C. Rather than simply profiling Shore, we progressively modified it so that after every feature removal or optimization, we had a (faster) working system that fully ran our workload. Overall, we identify overheads and optimizations that explain a total difference of about a factor of 20x in raw performance. We also show that there is no single "high pole in the tent" in modern (memory resident) database systems, but that substantial time is spent in logging, latching, locking, Btree, and buffer management operations.
Harizopoulos et al. REF performed a detailed performance study on the components in a memory-resident transaction processing system, and showed that only a small portion of the total time is spent on the useful work.
OLTP through the looking glass, and what we found there
SIGMOD Conference
Computer Science