Patent ID: 8875145

Claim:
A method for processing data in a data processing system using compiled dataflow graphs stored in a data storage system, the method including: receiving multiple units of work that each include one or more work elements; and processing a first unit of work using a first compiled dataflow graph loaded into the data processing system in response to receiving the first unit of work, the first compiled dataflow graph having been compiled into data structures representing a first dataflow graph that includes nodes representing data processing components connected by links representing flows of work elements between data processing components, the processing including analyzing the first unit of work to determine a characteristic of the first unit of work; identifying one or more compiled dataflow graphs from multiple compiled dataflow graphs stored in the data storage system that include at least some compiled dataflow graphs that were compiled, before the first unit of work was received, for processing a unit of work having the determined characteristic of the first unit of work; loading one of the identified one or more compiled dataflow graphs into a second data flow graph being executed by the data processing system since before the first unit of work was received, as the first compiled dataflow graph; and generating one or more output work elements from at least one work element in the first unit of work using the first dataflow graph.