Patent ID: 8239847

Claim:
A machine implemented method for distributed parallel processing, comprising: receiving an expression from a sequential program that is executing at a first machine; analyzing the expression for one or more patterns indicative of suitable map and reduction processing; automatically generating an execution plan including a map phase and a reduction phase for executing the expression in parallel at nodes of a compute cluster in response to a positive identification of the one or more patterns, the reduction phase includes a reduce function and the map phase includes a data partitioning operation; providing the execution plan to an execution engine that controls parallel execution of the expression in the computer cluster; and optimizing the execution plan prior to providing the execution plan to the execution engine, the optimizing includes determining that the reduce function is combiner-enabled and in response to the determining, moving a grouping function from the reduce phase to before the data partitioning operation of the map phase to effect a partial data reduction prior to the data partitioning operation.