Patent ID: 7984019
Filing Date: 2011-07-19
Classification: G06F

Abstract:
1. A method of loading data into a data-warehouse system for use in a data-warehouse system wherein there is to be data loaded or restored for use, said system comprising at least one computing device communicatively coupled with a large dataset, said large dataset having at least one storage device, said at least one storage device containing a full-backup data file and at least one partial-backup data file, said data files containing real-world entity data, said computing device having a Extraction Load Program (ETL) program to subject the data intended to be loaded to a transform stage according to data-warehouse-specific rules, the method comprising the step of: loading a first partial-backup data file, wherein the loading of said first partial-backup data file makes the data in said first partial-backup data file available for use, at a time when no other backup data files have been loaded, wherein said first partial-backup data file is one partial-backup data file within a series of historical partial-backup files for said data-warehouse system, and wherein said first partial-backup data file is not a full-backup data file; thereby loading data into the data-warehousing system, further comprising the step of: loading at least one additional partial-backup data file, wherein the loading of each said at least one additional partial-backup data file makes the data in each at least one additional partial-backup data file available for use, at a time when other backup data files are available, but have not been loaded, wherein said at least one additional partial-backup data file is any partial-backup data file within a series of historical partial-backup files for said data-warehouse system, wherein said at least one additional partial-backup data file is not a full-backup data file, and wherein each record to be extracted from said at least one additional partial-backup data file for loading into said data-warehouse system is only allowed to be written to said data-warehouse system if either the record to be written to said data-warehouse system is not already represented in said data-warehouse system or the record to be written to said data-warehouse system is designated as newer than its corresponding existing record, further comprising the steps of: defining a data-containing unit for use in constructing, extracting, and validating each record to be written in said data-warehouse system from said at least one additional partial-backup file; defining a dependent query based on database constraints or based on extrapolated relationships between data fields, wherein said dependent query is designed to populate said data-containing unit; performing said dependent query for each record in said at least one additional partial-backup data file, wherein said data-containing unit is populated and pushed into said ETL program, wherein referential integrity is maintained for all extracted data that is pushed to said ETL program, and wherein additional queries are made for all related entity data and said related entity data is pushed into said ETL program; validating and transforming each said dependent-query data set with said ETL program; and pushing only each said validated and transformed data set into said data-warehouse system.