Patent ID: 7689582

Claim:
A computer implemented method for generating an execution plan graph from a data flow, the computer implemented method comprising: configuring one or more computer processors to perform an operation comprising: receiving the data flow, wherein the data flow includes operators and execution dependencies between the operators, each execution dependency specifying: (i) a first operator; (ii) and a second operator; and (iii) that the second operator is to execute only after the first operator has finished executing; and wherein the operators are supported by a plurality of runtime engine types; generating, by operation of the one or more computer processors and in response to receiving the data flow, a metadata representation of the data flow to produce a logical operator graph, wherein generating the metadata representation comprises: classifying the operators into groups according to the execution dependencies and the plurality of runtime engine types, wherein at least one of the groups comprises a sequence of operators in the data flow that is supported by a single runtime engine type, and wherein the operators are classified to preclude any circular execution dependency among the groups; determining, based on the groups and the execution dependencies, an order in which to execute the groups; and optimizing at least one of the groups based on the runtime engine type of the at least one of the groups; generating a set of code units from the metadata representation, wherein each code unit in the set of code units is executable on the plurality of runtime engine types; and processing the set of code units to produce the execution plan graph.