Patent ID: 9396036
Filing Date: 2016-07-19
CPC Classification: G06F

Claim Text:
1. A method of performing a large-scale data processing job, comprising: executing a plurality of processes on a plurality of interconnected processors, the plurality of processes including a master process for coordinating a data processing job for processing a set of input data, a plurality of map processes, and a plurality of reduce processes; in the master process, assigning input data blocks of a set of input data to respective map processes of the plurality of map processes, assigning partitions of intermediate data to respective reduce processes of the plurality of reduce processes, and determining when the data processing job has reached a predefined level of completion; in each of the plurality of map processes: in each of the plurality of reduce processes: in accordance with a determination that processing of an identified partition of the intermediate data, identified using predefined criteria, is likely to delay the data processing job, taking a remedial action with respect to the identified partition; and in accordance with the master process determining that the data processing job has reached the predefined level of completion,