Patent ID: 8370493

Claim:
A computer-implemented method for managing distributed execution of programs, the method comprising: receiving, by one or more configured computing systems via a provided application programming interface (API), a request from a first user to execute multiple execution jobs of an indicated program on multiple computing nodes using indicated input data, the request including configuration information for use in execution of the indicated program that includes information about how to select subsets of the indicated input data to be used by the multiple execution jobs; tracking, by the one or more configured computing systems, information about a state of execution of the multiple execution jobs of the indicated program on the multiple computing nodes, the tracking including identifying that performance of at least one operation of the multiple execution jobs is complete, and the tracking further including identifying intermediate results that are produced from the performance of the at least one operation and are stored on the multiple computing nodes; after determining to terminate execution at a first time of at least one of the multiple execution jobs, initiating, by the one or more configured computing systems, persistent storage of the identified intermediate results produced from the performance of the at least one operation, the at least one execution jobs further having at least one other operation whose performance is not complete; at a later second time after the first time, initiating, by the one or more configured computing systems, a resumed execution of the at least one execution jobs on at least one computing node to complete the performance of the at least one other operations without repeating the completed performance of the at least one operations, the resumed execution being performed in a manner based at least in part on the persistently stored intermediate results; and after the execution of the multiple execution jobs of the indicated program is completed, providing, by the one or more configured computing systems, final results from the execution to a first user.