Performance Characterization, Modeling, and Experimentation. Typical short-term goals are to help write and tune programs that run fast for current problems on current computer systems, and to establish procurement criteria. A complementary long-term goal is to help design computer systems that will effectively support future applications. All of these goals require determining the performance characteristics of application programs and computer systems. However, currently available tools and techniques provide only a subset of the capabilities needed to meet these goals.
Current tools are usually specialized.
These tools are generally restricted to empirical measurements of a particular combination of code, system, and sample problems. Little or no support is provided for developing predictive models of application performance, or even for acquiring the information needed to construct such models. Additional work is needed characterize massively parallel systems and extract application requirements, as well as to construct and validate models. These gaps can be filled, in part by applying existing techniques to a wider range of applications and systems, and in part by further research collaborations between computer science researchers, application developers, and vendors. Our major suggestion is that each major MPP class and application community should be supported by at least one research group chartered specifically to characterize the performance of important applications on that MPP class, both now and with an eye toward future scalability, and to develop the tools and techniques necessary to support such characterization.