The absence of benchmarks for Web sites with dynamic content has been a major impediment to research in this area. We describe three benchmarks for evaluating the performance of Web sites with dynamic content. The benchmarks model three common types of dynamic content Web sites with widely varying application characteristics: an online bookstore, an auction site, and a bulletin hoard. For the online bookstore, we use the TPC-W specification. For the auction site and the bulletin hoard, we provide OUT own specification, modeled after ebay.com and slahdot.org, respectively. For each benchmark we describe the design of the database and the interactions provided by the Web server. We have implemented these three benchmarks with a variety of methods for building dynamic-content applications, including PHP, Java servlets and ETB (Enterprise Java Beans). In all cases, we use commonly used open-source software. We also provide a client emulator that allows a dynamic content Web server to he driven with various workloads. Our implementations are available freely from our Web site for other researchers to use. These benchmarks can he used for research in dynamic Web and application server design. In this paper, we provide one example of such possible use, namely discovering the bottlenecks for applications in a particular server configuration. Other possible uses include studies of clustering and caching for dynamic content, comparison of different application implementation methods, and studying the effect of different workload characteristics on the performance of servers. With these benchmarks we hope to provide a common reference point for studies in these areas.
RUBiS and RUBBoS REF characterize the workload of a dynamic-content auction site and bulletin board, respectively.
Specification and implementation of dynamic Web site benchmarks
2002 IEEE International Workshop on Workload Characterization
2002 IEEE International Workshop on Workload Characterization
Computer Science