Benchmarking a Site with Realistic Workload (2002)

The rapid growing of Web llSeTS number, and the consequent importance of Capacity Planning, have leaded to the development of Web benchmarking tools largely available in the market. One of the most common critics to this approach, as that synthetic workload produced by web stressing tools is far t o be realistic. This paper deals with a benchmarking methodology based on a workload characterization generated from the log files. Customer Behavior Model Graph (CBMG) has been proposed in 1131 as a workload characterization of an e-Commerce site. Ifere we discuss how CBMG methodology has a wider field of application and how to use this model to eficiently improve a fully integrated Web Stressing Tool. We also evaluate the differences between OUT approach and other models based on different characterizations. Since 1995, web servers have draniatically illcreased the number of connections served per seconds. Throughput strongly depends on network capacity. However, network capacity is improving fa..ter than server capacity, and enhancements of the network infrastructure (Gigabytes wide-area network, ISDN, xDSL lines, cable modems, and so on) rcduce network latency. Latency, measured by RTT (Round 'hip time), is one of the key element of Web Performance, and is dependent on network conditions and web server capacity. A high error rate is the most immediate perception of a web server unreliability. Server errors are mainly generated for two reasons: bad implementation or inadequate capacity. Avoiding and/or detecting implementation errors is easier to obtain than adequate capacity. If a server has a low capacity (i.e., a real-time streaming server which can onlv serve 200 rcquests per second). manv clients I . 1 T-L-..A..-&:-receive a refused message: service is not I l , , L l U U U L . C I U I . The World Wide Web is one of the most used interfaces to access remote data and services (both commercial and not commercial). The number of actors involved in these transactions is growing very quickly. As a consequence users may experience very slow connection to popular web sites during rush hours [5 ] . On average, users leave a site after 8 seconds of busy wait. Despite the growing development of servers and network technologies, Web performance problems are still quite difficult to solve: the Web is a composite saucerful of multimedia sources, static and dynamic objects, local and distributed architectures, and so on. available and clients may be lost. Since the main bottleneck is at the server side, expected workload is difficult t o define (see [7] for a survey on workload characterization problems). The continuing web framework evolution, from a simple ClientlServer architecture to a complex distributed system, has caused a series of consequences: a client request can be fulfilled by many servers; routing strategies can be defined at different levels (the client may originate requests to the primary server as well as to one of its mirrors); DNS servers can resolve the address at different hierarchical levels; Web switches are often used to disoatcli reouests in a DOOI of servers Drovidine load bal~~ a ~~~ . ~ ~ ~ ~ . ~~~~~~ ~ ~ ~ Four metrics are coonb' used to the performance of a Web Site: (1) Requests Served Per secand; (2) TllroughPut in bytes Per second; (3) trip time (RTT); (4) Errors. In order t o improve the overall performance, all these Parameters shodd be considered as well. ancing [SI. Moreover, workload characterization deals with clients, servers and proxies [16]. In this context, http request/response messages between clients and Server are also definitely influenced by cache activities. Different monitoring strategies can be cllosen to describe web workload in order to capture desired metrics (we only need information traced at server-side). To evaluate web server performances, either commer*V. Russo has been granted by CSP during the preparation of this paper. 0-7803-768 1-1/02/$17.0002002 EEE 14 cia1 or opensource web stressing tools are largely available (OpenSTA [15] and Load Runner [14]). They are both used to perform load and stress test on a replicated web site. During the testing period, monitoring agents collect a set of system resources parameters (CPU and RAM utilization rate, I j O Disks accesses, network traffic, and so on). Generated workload is usually based on a predefined user session, which is replicated many times by a number of Virtual Users (see 4). In the rest of the paper we will present a methodology to evaluate the performance of a web site to avoid “lack of planning” problems. This methodology is based on the usage of a stressing tool, which imposes a synthetic realistic workload on the web site under test. In Section 2, we will briefly review related works about benchmarking and workload characterization. In Section 3, a quick survey of a workload characterization model called Customer Behavior Model Graph (CBMG 3) will be given as well. After intrcducing the approach in detail (Section [13]), we will propose an integration between existing benchmarking tools and the CBMG workload characterization model, in order t o test the web site in a more realistic scenario than using random navigational patterns. Before concluding, we will outline future directions of this research, giving an experimental evaluation of it in Section 6.