Patent ID: 8537160

Claim:
One or more computer-storage memory having computer-executable instructions embodied thereon that, when executed, perform a method for generating distributed dataflow graphs, the method comprising: receiving a plurality of data elements; reading and parsing the plurality of data elements to form a sequential dataflow graph having a plurality of vertices; determining at least one of the plurality of vertices that is capable of being performed multiple times in parallel by applying at least one heuristic to the sequential dataflow; determining at least two of the plurality of vertices that must be sequentially performed; basing a number of times to replicate the at least one vertex on a number of processors available to perform data processing; replicating the at least one vertex the number of times; dividing the sequential dataflow graph into a plurality of subgraphs based upon the determined at least one vertex that is capable of being performed multiple times in parallel, wherein one subgraph comprises the at least two of the plurality of vertices that must be sequentially performed; determining a quantity of data each of a plurality of available processors can simultaneously process; using sizes of the data elements and the quantity of data each of the plurality of available processors can simultaneously process to determine a number of times to replicate each of the plurality of subgraphs; replicating the plurality of subgraphs that are determined to be capable of being performed multiple times in parallel the number of times; and generating a distributed dataflow graph by connecting the replicated subgraphs.