Patent ID: 8335758

Claim:
An improved data ETL (Extraction-Transformation-Loading) dispatching method comprising: a. extracting data from distributed and heterogeneous data sources; b. placing the extracted data into an interim intermediate layer for cleaning, transformation and integration; and c. loading the extracted data to a data warehouse or data mart used for online analysis; wherein, in the execution of the steps a, b and c, key dispatching information lscycle (latest successful data cycle), curcycle (current processing data cycle), and endcycle (processing end data cycle) are memorized and transferred to called programs, and after the executions of the called programs, a dispatching engine records and keeps updated key dispatching information; in the re-extraction of the steps a, b and c, the updated key dispatching information lscycle (latest successful data cycle), curcycle (current processing data cycle), and endcycle (processing end data cycle) are adopted for automatic re-extractions of some tasks and some cycles therein; wherein, the improved data ETL dispatching method further comprises: a fixed cycle data ETL dispatching method wherein a fixed cycle data represents static data within an analysis cycle; and a cyclic closed-loop data ETL dispatching method wherein cyclic closed-loop data represents the recordings of the last change within the certain extraction cycle which are variable, where the cycle is not fixed and ranges from the execution time of last extraction to present one; wherein in the improved data ETL dispatching method: if a dispatching flow is represented by Wf1, and a task B1 εWf1, then State of Wf1={wstate—state, lscycle—latest successful data cycle, endcycle—processing end data cycle, curcycle—current processing data cycle} State of B1={bstate—state, succnum—time of successful running, succtotal—total hours of successful running, succavg—average hours of successful running}; and the improved data ETL dispatching method further comprises: determining if a task has been suspended or falsely executed for some reason, and lags behind a preset plan, restoring the task to normal operation, and automatically accelerating the task according to the characteristics of the time cycle.