Patent ID: 7739547

Claim:
A method comprising: loading data from a plurality of source files into an information warehouse, the data loading being assigned a state of data chosen from the group consisting of: beginning, completing, aborting, undoing, syncing, sync-aborting, sync-completing, and sync-undoing; maintaining dependencies among multiple database tables affected by the data loading in said information warehouse by tracking a state of database table transition, said state of database table transition being selected as one of: empty, add pending, added synced and sync-pending, wherein the state of database table transition transitions from one state to another state in response to a condition of said data loading; performing checkpointing and versioning concurrently with said data loading, the checkpointing ensuring consistency among said multiple database tables, wherein checkpoints are maintained in a checkpoint log file and said versioning comprises: assigning a current version number to each of the plurality of source files; and keeping track of a maximum version number for each of the plurality of source files that is a maximum of the assigned current version numbers for the source file; tracking a number of records associated with each data stored in each of the multiple database tables; recovering from a failure in the data loading using the checkpointing by restoring the information warehouse to at least one of said checkpoints, wherein said restoring comprises; avoiding duplicate data loading by determining a data item having a consistent number of records in each of said multiple database tables from data associated with said at least one checkpoint, removing data from said multiple database tables having an inconsistency in their corresponding number of records; and resuming the data loading from a point to which the information warehouse has been restored; and processing an undo operation of the loaded data from a current version to a previous version in response to an error in data loading using said versioning wherein said undo operation comprises: updating the current version number of one of said plurality of source files to a chosen version number less than the maximum version number for the source file; and leaving the maximum version number for the source file unchanged.