Programming Abstractions for Large-scale Distributed Applications

The number of coupled distributed computational infrastructures has recently increased. Some prominent examples are the US TeraGrid and OSG, the UK National Grid Service, and the European DEISA and EGEE. However, the number of applications that can utilize such a distributed infrastructure in novel and effective ways is limited. Most applications on such infrastructures just use a single parallel computer. [? , ? ] Even though individual computers are becoming more powerful, there remains and will remain a need for aggregating distributed resources for scientific computing. In the simplest case, the required amount of computing power at an instant of time may exceed the capacity of individual supercomputers, thus requiring the coupling of physically distributed resources. Additional reasons are the need for greater “peak” capacity arising from time varying resource requirements, the need for higher throughput, the requirement to use specialized hardware in conjunction with general purpose computing units, the need to do computing near data that is too large to be efficiently transferred, the desire to use distinct computer architectures on different parts of a single application, and collaborations between physically separated groups who each have to their own computing. It is generally accepted that the ability to write distributed applications has lagged seriously behind the ability to develop and deploy infrastructure. Writing applications for distributed infrastructure is notoriously hard; acceptable models and abstractions are not yet available. There are many ad-hoc solutions employed to write distributed applications, which results in lack of generality and the inability of programs to be extensible and independent of infrastructure details. In most cases, though, the difficultly simply leads to potential distributed applications not being written. The complexity of writing large-scale, distributed applications stems from combining the fields of highperformance computing and large-scale, distributed systems. No single solution from either field appears to be suitable for large-scale distributed applications. In high-performance computing today, performance is paramount, and systems are typically homogeneous, failure-free over the lifetime of the computation, and operated within a single administrative domain. In large-scale, distributed systems, scalability, heterogeneity, fault-tolerance and security issues prevail; performance as measured by peak utilisation is often considered a secondary concern; and arguably, the most complicating factor is the requirement to compute across virtual organizations and administrative domains. In general, other than performance, the constraints in distributed programming are in many ways much harder to satisfy than in parallel programming, although as parallel machines grow in size and complexity, particularly at the highest end, fault tolerance is becoming increasing important there as well. Thus, there exists a particular interest in large-scale distributed applications, as these reinforce the need for distributed computing, as well as serving as a reminder of the extreme challenges involved in designing and programming such large-scale, distributed applications. The scope of this survey paper is set by the application classes defined in Section ? ; specific applications