Performance characteristics, such as response time, throughput and scalability, are key quality attributes of distributed applications. Current practice, however, rarely applies systematic techniques to evaluate performance characteristics. We argue that evaluation of performance is particularly crucial in early development stages, when important architectural choices are made. At first glance, this contradicts the use of testing techniques, which are usually applied towards the end of a project. In this paper, we assume that many distributed systems are built with middleware technologies, such as the Java 2 Enterprise Edition (J2EE) or the Common Object Request Broker Architecture (CORBA). These provide services and facilities whose implementations are available when architectures are defined. We also note that it is the middleware functionality, such as transaction and persistence services, remote communication primitives and threading policy primitives, that dominate distributed system performance• Drawing on these observations, this paper presents a novel approach to performance testing of distributed applications. We propose to derive applicationspecific test cases from architecture designs so that performance of a distributed application can be tested using the middleware software at early stages of a development process. We report empirical results that support the viability of the approach. Middleware, Software Testing, Distributed Software Architecture, Software Performance, Performance Analysis Models tThe research was conducted while this author was on study leave at UCL. *This work has been partially supported by the European Union through the Research Training Network Segra Vis. Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, to republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee.  INTRODUCTION Various commercial trends have lead to an increasing demand for distributed applications. Firstly, the number of mergers between companies is increasing. The different divisions of a newly merged company have to deliver unified services to their customers and this usually demands an integration of their IT systems. The time available for delivery of such an integration is often so short that building a new system is not an option and therefore existing system components have to be integrated into a distributed system that appears as an integrating computing facility. Secondly, the time available for providing new services are decreasing. Often this can only be achieved if components are procured off-the-shelf and then integrated into a system rather than built from scratch. Components to be integrated may have incompatible requirements for their hardware and operating system platforms; they have to be deployed on different hosts, forcing the resulting system to be distributed. Finally, the Internet provides new opportunities to offer products and services to a vast number of potential customers. The required scalability of e-commerce or e-government sites cammt usually be achieved by centralised or client-server architectures but demand the use of distributed software architectures. Given the growing importance of distributed systems, we are interested in devising systematic ways to ascertain that a given distributed software architecture meets performance requirements of their users. Performance can be characterised in several different ways. Latency typically describes the delay between request and completion of an operation. Throughput denotes the number of operations that can be completed in a given period of time. Scalability identifies the dependency between the number of distributed system resources that can be used by a distributed application (typically number of hosts or processors) and latency or throughput. Despite the practical significance of these various aspects it is still not adequately understood how to test the performance of distributed applications. Weyuker and Vokolos reported on the weakness of the published scientific literature on software performance testing in [26] . To this date no significant scientific advances have been made on performance testing. Furthermore the set of tools available for software performance testing is fairly limited. The most widely used tools are workload generators and performance profilers that provide support for test execution and debugging, but they do not solve many 94
In 2004, Denaro, Polini and Emmerich presented an approach to conducting early performance testing of distributed software applications REF .
Early performance testing of distributed software applications
WOSP '04
Computer Science