Patent ID: 8381015

Claim:
A map/reduce data processing system comprising: a computer with at least one processor and memory communicatively coupled to a computing cluster of nodes over a computer communications network; a map/reduce module hosted in the computer, the map/reduce module subdividing a computational problem into a set of sub-problems, mapping each of the sub-problems in the set to respective ones of the nodes, directing processing of the sub-problems in the nodes and collecting results from completion of processing of the sub-problems in the nodes; and, fault tolerance logic coupled to the map/reduce module, the logic comprising program code enabled to determine a computation time to complete computation of all of the sub-problems; establish a polling interval at which time the health of each of the nodes computing a corresponding sub-problem is assessed; upon detecting failed nodes during a time period prior to a final polling interval before completion of the computation of all of the sub-problems, restart the respective sub-problems; upon determining that the final polling interval has commenced, identify nodes still processing respective sub-problems and replicate sub-problems in the identified nodes; remove duplicate results collected for the replicated sub-problems and reduce the results into a result set for the problem.