Economic Aspects of QoS in Grid Computing Research Thesis Submitted in Partial Fulfillment of the Requirements for the Degree of Master of Science in Computer Science

Grids are becoming mission-critical components in research and industry, offering quite sophisticated solutions for the exploitation of virtually unlimited computing and storage resources. The grid resources are usually shared among multiple organizations and typically managed in a “best effort” manner. However, many real-world supercomputing applications, such as computational fluid dynamics (CFD), weather forecasting, and complex system simulations, rely on co-allocation of large numbers of reliable resources as well as on a static and stable execution environment, making such “best effort” Quality of Service inadequate. In our work we aim at implementing a new concept of quasi-opportunistic supercomputing that will enable execution of demanding parallel applications on massive non-dedicated resources in grid environments. In this paper we identify a set of requirements that must be satisfied by a grid resource management system in order for grids to become an affordable alternative to supercomputing. The European-funded project QosCosGrid is currently developing such a system. Usability and efficiency of our implementation will be tested using a set of benchmark applications from the Complex Systems domain. 1. The challenges of supercomputing on top of a grid The term “supercomputer” typically refers to dedicated special-purpose multiprocessor computing systems that provide close to best achievable performance for demanding parallel workloads [5]. Indeed, supercomputers have several characteristics that make them indispensable to efficient execution of such workloads. First, all the high-end hardware components such as CPUs, memories, interconnects and storage are distinguished not only by high capacity but also by high reliability and performance predictability. Second, supercomputer middleware provides a convenient abstraction of a homogeneous computational and networking environment, automatically allocating resources according to the underlying networking topology [22]. Third, the resources are managed exclusively by a single centralized system, which enforces global resource utilization policies, thus maximizing hardware utilization while minimizing the turnaround time of individual applications. All the above contribute to the perception that supercomputers are a platform with unprecedented performance, stability, convenience, and dependability. The vision of grids as huge virtual supercomputers is unattainable without bridging the existing gap in their performance and reliability. However, grids differ radically from supercomputers in a number of characteristics, making the realization of this vision even more challenging. Non-dedicated resources. Real-world grids typically comprise many distributively owned clusters of resources, each of which often serves the community of local users while executing external jobs arriving from the grid. In such a setup, local users are typically prioritized. This policy results in unpredictable performance degradation of the grid-originating jobs. The fluctuations in resource availability would be prevented through application of the mechanisms for negotiation and enforcement of global resource sharing policies (e.g. advanced reservation), coupled with the incentive for the resource providers to maintain these policies. Note that such an incentive, if introduced, should be taken into account during the scheduling process [11], [12], [13], [14]. Network heterogeneity. The network topology of a typical grid can be presented as a graph with multiple cliques (clusters) with high capacity links inside cliques and low capacity links among them. High performance cannot be attained without the grid middleware’s T ec hn io n C om pu te r Sc ie nc e D ep ar tm en t M . S c. T he si s M SC -2 00 818 2 00 8