Distributed computations in a dynamic, heterogeneous grid environment

In order to face the rapidly increasing need for computational resources of various scientific and engineering applications one has to think of new ways to make more efficient use of the worlds current computational resources. In this respect, the growing speed of wide area networks made a new kind of distributed computing possible: Metacomputing or (distributed) Grid computing. This is a rather new and uncharted field in computational science. The rapidly increasing speed of networks even outperforms the average increase of processor speed: Processor speeds double on average each 18 month whereas network bandwidths double every 9 months. Due to this development of local and wide area networks Grid computing will certainly play a key role in the future of parallel computing. This type of distributed computing, however, distinguishes from the traditional parallel computing in many ways since it has to deal with many problems not occurring in classical parallel computing. Those problems are for example heterogeneity, authentication and slow networks to mention only a few. Some of those problems, e.g. the allocation of distributed resources along with the providing of information about these resources to the application have been already attacked by the Globus software. Unfortunately, as far as we know, hardly any application or middle-ware software takes advantage of this information, since most parallelizing algorithms for finite differencing codes are implicitly designed for single supercomputer or cluster execution. We show that although it is possible to apply classical parallelizing algorithms in a Grid environment, in most cases the observed efficiency of the executed code is very poor. In this work we are closing this gap. In our thesis, we will o show that an execution of classical parallel codes in Grid environments is possible but very slow o analyze this situation of bad performance, nail down bottlenecks in communication, remove unnecessary overhead and other reasons for low performance o develop new and advanced algorithms for parallelisation that are aware of a Grid environment in order to generelize the traditional parallelization schemes o implement and test these new methods, replace and compare with the classical ones o introduce dynamic strategies that automatically adapt the running code to the nature of the underlying Grid environment The higher the performance one can achieve for a single application by manual tuning for a Grid environment, the lower the chance that those changes are widely applicable to other programs. In our analysis as well as in our implementation we tried to keep the balance between high performance and generality. None of our changes directly affect code on the application level which makes our algorithms applicable to a whole class of real world applications. The implementation of our work is done within the Cactus framework using the Globus toolkit, since we think that these are the most reliable and advanced programming frameworks for supporting computations in Grid environments. On the other hand, however, we tried to be as general as possible, i.e. all methods and algorithms discussed in this thesis are independent of Cactus or Globus. We start with a short introduction to Grid computing and explain its benefits for science and economics. Furthermore, basic terms and notions that are relevant for this thesis are explained and extended from standard parallel computing to the usage in Grid environments. In Chapter 2 we give an overview about what we call “traditional parallel computing”, in Chapter 3 we analyze the Grid awareness of MPI implementations, present some smaller scaled test runs and end with a description of general problems. Chapter 4 deals with a simple model to predict the performance of a distributed run in a Grid environment while in Chapter 5 we present a first large scaled distributed run in a real Grid environment where we manually applied novel techniques to improve the efficiency of the running code. These experiences can also be found in the award winning paper Supporting Efficient Execution in Heterogeneous Distributed Computing Environments with Cactus and Globus (presented at the Supercomuting Conference 2001). Since we have learned much from those runs we will analyze and develop in Chapter 6 algorithms that automatically achieve the configuration we adjusted manually in Chapter 5. Since some environment characteristics (e.g. bandwidth or the used application code algorithm) may change in time we also developed dynamic strategies to adapt to the current Grid environment, which we are investigating in Chapter 7, including adaptive compression, adaptive selection of ghost zone sizes and dynamical load balancing. These two Chapters form the content of the paper Grid Aware Paralellizing Algorithms. In the following Chapter 8 we discuss briefly the implementation of previously analyzed algorithms within the Cactus framework using the Globus toolkit, followed by a presentation of the achieved results in Capter 9, a brief presentation of related work in this area in Capter 10 and a discussion of possible future directions in this field in the last Chapter 11. The achievements in this thesis, in particular those in Chapter 7 were awarded the Gordon Bell Prize in the special category. Furthermore, our experiments in Chapter 5 significantly supported the building and funding of the TeraGrid.