Patent ID: 8631279

Claim:
At a distributed data processing system, the distributed data processing system including a client, a distributed data set, and a compute node cluster, the distributed data set partitioned into a plurality of data partitions, the compute node cluster including a plurality of compute nodes, a method for propagating unhandled exceptions from the plurality of compute nodes to the client, the method comprising: an act of receiving a data processing plan from the client, the data processing plan defining how a data access request for specified data from within the distributed data set is to be executed across the plurality of compute nodes within the compute node cluster, the data processing plan including a plurality of different stages, each different stage defining a type of processing that is to be performed on data within the distributed data set; an act of orchestrating the data processing plan on the compute node cluster by executing one or more of the plurality of different stages of the data processing plan, including for each stage: an act of instructing one or more compute nodes to: (a) execute instructions against data within or derived from one or more data partitions of the distributed data set, (b) catch any encountered unhandled exceptions occurring during execution of the instructions against the data within or derived from one more data partitions, each unhandled exception indicating at least one of: a failure at a particular node or a failure at a particular stage of the data processing plan, and (c) store encountered unhandled exceptions along with corresponding auxiliary data in a designated storage location, the corresponding auxiliary data associating each of the unhandled exceptions with the data partition, from among the one or more data partitions, that lead to the unhandled exception; an act of harvesting stored unhandled exceptions along with corresponding auxiliary data; an act of aggregating the harvested unhandled exceptions along with corresponding auxiliary data into one or more container objects, the one or more container objects conveying details of compute node and stage failures that happened during orchestration of the data processing plan; and an act of sending the one or more container objects to the client.