Patent ID: 8954981
Filing Date: 2015-02-10
Classification: G06F

Abstract:
1. A method of optimizing resources for a parallel data integration job, the method comprising: receiving a job request specifying a parallel data integration job to deploy in a grid wherein the job request includes operators specifying parallel integration operations performed when the parallel data integration job is run; predicting grid resource utilizations for hypothetical runs of the specified job on respective hypothetical grid resource configurations responsive to a model based on performance data from a plurality of actual runs of previously deployed, parallel data jobs; selecting a grid resource configuration for running the parallel data integration job, including automatically selecting a grid resource configuration responsive to the predicted grid resource utilizations and an optimization criterion based on at least one resource utilization index for the job; and generating the at least one resource utilization index for the job, comprising: generate resource utilization indices for each respective operator responsive to the predicted grid resource utilizations on resource portions; generating a respective operator index maximum for each respective operator; generating, for each of a respective group of the operators, a respective maximum of the operator index maxima among the operators of the respective group; selecting a first maximum of resource utilization indices for a first and second subset of data source and sink operator groups; selecting a second maximum of resource utilization indices for a first and second subset of processing and scratch operator groups; and generating the at least one resource utilization index for the job responsive to a ratio of the first and second maxima.