Patent Publication Number: US-6665654-B2

Title: Changing table records in a database management system

Description:
BACKGROUND OF THE INVENTION 
     1. Technical Field 
     The present invention relates to a method and system for updating and deleting records in tables generally in a database management system, and particularly in a business data warehouse, due to changes in source files which are logically associated with the tables. 
     2. Related Art 
     Updating and deleting records in a large table in a business data warehouse can be prohibitively time consuming. Thus, there is a need for a method and system for efficiently updating and deleting records in a large table in a business data warehouse. 
     SUMMARY OF THE INVENTION 
     The present invention provides a system for updating a tableset T of a family, in relation to 
     N source files denoted as S 1 , S 2 , . . . , S N , said N at least 1, said tableset T having M tables T 1 , T 2 , . . . , T M , said M at least 1, said N source files logically associated with said M tables, said family having a common family key K F  across the M tables and across the N source files, 
     a fileset S A  comprising files S 1A , S 2A , . . . , S NA  respectively embodying S 1 , S 2 , . . . , S N  at a time t A  such that T 1 , T 2 , . . . , T M  at the time t A  accurately reflects S A,  and 
     a fileset S B  comprising files S 1B , S 2B , . . . , S NB  respectively embodying S 1 , S 2 , . . . , S N  at a time t B , said t B  and t A  being different times, 
     the system comprises: 
     a Delta program adapted to generate a dataset D of non-redundant family keyvalues embodying K F , each said non-redundant family keyvalue drawn from selected Update records of S 1B  relative to S 1A , said selected Update records not having any redundant family keyvalues, for i=1, 2, . . . , N; 
     a Trigger program adapted, for i=1, 2, . . . , N: to step through all records of S 1B , including to compare the family keyvalue K FVR  within each record of S 1B  against the family keyvalues in the dataset D and to save in a file ΔS i  each said record of S iB  for which K FVR  equals one of the family keyvalues K FV  in the dataset D; 
     a Bridge program adapted to generate files ΔT 1 , ΔT 2 , . . . , ΔT M  that includes the data in ΔS 1 , ΔS 2 , . . . , ΔS N  in a form that is compatible with the M tables T 1 , T 2 , . . . , T M , respectively; and 
     at least one Load program adapted to update the tables T 1 , T 2 , . . . , T M  with the data in ΔT 1 , ΔT 2 , . . . , ΔT M , respectively. 
     The present invention provides a method for updating a tableset T of a family, in relation to 
     N source files denoted as S 1 , S 2 , . . . , S N , said N at least 1, said tableset T having M tables T 1 , T 2 , . . . , T M , said M at least 1, said N source files logically associated with said M tables, said family having a common family key K F  across the M tables and across the N source files, 
     a fileset S A  comprising files S 1A , S 2A , . . . , S NA  respectively embodying S 1 , S 2 , . . . , S N  at a time t A  such that T 1 , T 2 , . . . , T M  at the time t A  accurately reflects S A , and 
     a fileset S B  comprising files S 1B , S 2B , . . . , S NB  respectively embodying S 1 , S 2 , . . . , S N  at a time t B , said t B  and t A  being different times, 
     the method comprises: 
     executing a Delta program, including generating a dataset D of non-redundant family keyvalues embodying K F , each said non-redundant family keyvalue drawn from selected Update records of S iB  relative to S iA , said selected Update records not having any redundant family keyvalues, for i=1, 2, . . . , N; 
     executing a Trigger program, including for i=1, 2, . . . , N: stepping through all records of S iB , including comparing the family keyvalue K FVR  within each record of S iB  against the family keyvalues in the dataset D and saving in a file ΔS i  each said record of S iB  for which K FVR  equals one of the family keyvalues K FV  in the dataset D; 
     executing a Bridge program, including generating files ΔT 1 , ΔT 2 , . . . , ΔT M  that includes the data in ΔS 1 , ΔS 2 , . . . , ΔS N  in a form that is compatible with the M tables T 1 , T 2 , . . . , T M , respectively; and 
     executing at least one Load program, including updating the tables T 1 , T 2 , . . . , T M  with the data in ΔT 1 , ΔT 2 , . . . , ΔT M , respectively. 
     The present invention provides a system for deleting deletion-targeted records in a tableset T of a family, in relation to 
     N source files denoted as S 1 , S 2 , . . . , S N , said N at least 1, said tableset T having M tables T 1 , T 2 , . . . , T M , said M at least 1, said N source files logically associated with said M tables, said family having a common family key K F  across the M tables and across the N source files, 
     a fileset S A  comprising files S 1A , S 2A , . . . , S NA  respectively embodying S 1 , S 2 , . . . , S N  at a time t A  such that T 1 , T 2 , . . . , T M  at the time t A  accurately reflects S A , and 
     a fileset S B  comprising files S 1B , S 2B , . . . , S NB  respectively embodying S 1 , S 2 , . . . , S N  at a time t B , said t B  and t A  being different times, 
     the system comprises: 
     a Delta program adapted to generate a dataset D DEL  of non-redundant family keyvalues embodying K F , each said non-redundant family keyvalue drawn from selected Delete records of S iB  relative to S iA , said selected Delete records not having any redundant family keyvalues, for i=1,2, . . . , N; and 
     at least one Load program adapted to delete the deletion-targeted records in the tables T 1 , T 2 , . . . , T M , each said deletion-targeted record having one of the non-redundant family keyvalues that exists in the dataset D DEL . 
     The present invention provides a method for deleting deletion-targeted records in a tableset T of a family, in relation to 
     N source files denoted as S 1 , S 2 , . . . , S N , said N at least 1, said tableset T having M tables T 1 , T 2 , . . . , T M , said M at least 1, said N source files logically associated with said M tables, said family having a common family key K F  across the M tables and across the N source files, 
     a fileset S A  comprising files S 1A , S 2A , . . . , S NA  respectively embodying S 1 , S 2 , . . . , S N  at a time t A  such that T 1 , T 2 , . . . , T M  at the time t A  accurately reflects S A , and 
     a fileset S B  comprising files S 1B , S 2B , . . . , S NB  respectively embodying S 1 , S 2 , . . . , S N  at a time t B , said t B  and t A  being different times, 
     the method comprises: 
     executing a Delta program, including generating a dataset D DEL  of non-redundant family keyvalues embodying K F , each said non-redundant family keyvalue drawn from selected Delete records of S iB  relative to S iA , said selected Delete records not having any redundant family keyvalues, for i=1, 2, . . . , N; and 
     executing at least one Load program, including deleting the deletion-targeted records in the tables T 1 , T 2 , . . . , T M , each said deletion-targeted record having one of the non-redundant family keyvalues that exists in the dataset D DEL . 
     The present invention provides a method and associated system for efficiently updating and deleting records in a large table in a database of a database management system (DBMS) such as in a business data warehouse of a DBMS. 
    
    
     BRIEF DESCRIPTION OF THE DRAWINGS 
     FIG. 1 depicts a family of 5 tables, in accordance with embodiments of the present invention. 
     FIG. 2 depicts 8 source files logically associated with the 5 tables of FIG. 1 with a common family key across the 5 tables and 8 source files, in accordance with embodiments of the present invention. 
     FIG. 3 is a block diagram of a system for updating and deleting records a family of tables based on differences between source filesets S A  and S B  at times t A  and t B , respectively, said system adapted to execute programs including a Delta program, a Trigger Program, a Bridge program, a Sourcegen macroprogram, and an Objectload program, in accordance with embodiments of the present invention. 
     FIG. 4 depicts an example of source files S 1A  and S 1B  of filesets S A  and S B , respectively, of FIG. 3, in accordance with embodiments of the present invention. 
     FIG. 5 depicts an example of a source files S 2A  and S 2B  of filesets S A  and S B , respectively, of FIG. 3, in accordance with embodiments of the present invention. 
     FIG. 6 depicts an example of source files S 3A  and S 3B  of filesets S A  and S B , respectively, of FIG. 3, with indicated Update records and Delete records, in accordance with embodiments of the present invention. 
     FIG. 7 is a flow chart of an embodiment of the Delta program of FIG. 3, in accordance with embodiments of the present invention. 
     FIG. 8 depicts the result of applying the embodiment of the Delta program shown in FIG. 7 for Update records of the source files S 1A , S 1B , S 2A , S 2B , S 3A , and S 3B  depicted in FIGS. 4-6, in accordance with embodiments of the present invention. 
     FIG. 9 depicts the result of applying the embodiment of the Delta program shown in FIG. 7 for Delete records of the source files S 1A , S 1B , S 2A , S 2B , S 3A , and S 3B  depicted in FIGS. 4-6, in accordance with embodiments of the present invention. 
     FIG. 10 depicts a flow chart of an embodiment of the Trigger program of FIG. 3 for generating an output fileset ΔS, in accordance with embodiments of the present invention. 
     FIG. 11 depicts the structure of each file that is generated in accordance with FIG. 10, in accordance with embodiments of the present invention. 
     FIG. 12 is a block diagram of a first computer configuration for the system of FIG. 3, in accordance with embodiments of the present invention. 
     FIG. 13 is a block diagram of a second computer configuration for the system of FIG. 3, in accordance with embodiments of the present invention. 
    
    
     DETAILED DESCRIPTION OF THE INVENTION 
     FIG. 1 depicts a family of 5 tables denoted individually as T 1 , T 2 , T 3 , T 4 , and T 5 , and collectively as a tableset T, in accordance with embodiments of the present invention. The tableset T exemplifies a set of tables that may exist in a business data warehouse, or more generally in a database management system such as DB2, ORACLE, etc. A table is generally a two-dimensional data structure characterized by rows and columns, or records and fields, respectively. The terms “row” and “record” are considered herein to have the same meaning. The terms “column” and “field” are likewise considered herein to have the same meaning. Rows and columns are each one-dimensional arrays. 
     Each table is characterized by a primary key, namely one or more columns (or fields) that make each record of the table unique. The tables T 1 , T 2 , T 3 , T 4 , and T 5  are grouped together as a “family” because the primary key of each table comprises a common key component called a “family key” K F . For example, if the tableset T comprises data relating to invoices, then the tables T 1 , T 2 , T 3 , T 4 , and T 5  may have a family key consisting of an Invoice Number, and thus each of T 1 , T 2 , T 3 , T 4 , and T 5  would include an Invoice Number column (or field). 
     Each of tables T 1 , T 2 , T 3 , T 4 , and T 5  is a “family member” of the family. FIG. 1 shows the 5 tables in a family arrangement that is hierarchical, with T 1  as a parent, T 2 , T 3 , and T 4  each a child of T 1 , and T 5  a child of T 3  or a grandchild of T 1 . The hierarchy of the family is governed by hierarchical relationship of the primary keys of the family members. As the parent, T 1  may have the family key K F  as its primary key, while T 2 , T 3 , T 4 , and T 5  may have primary keys represented symbolically as {K F , K 2 }, {K F , K 3 }, {K F , K 4 }, and {K F , K 3 , K 5 }, respectively. For the present invention, the only key of interest is the family key K F . The scope of the present invention includes the special case in which the family members are in a parallel arrangement with each having the same primary key, namely the family key K F . The family members in such a parallel arrangement may be thought of as brothers and sisters. 
     The present invention comprises an efficient method and system for updating and deleting data in the tables of a family based on data in source files that are logically associated with the tables. To illustrate, FIG. 2 depicts a fileset S of 8 source files logically associated with the 5 tables of FIG.  1 . The 8 source files of FIG. 2 each comprise records, each record having the same family key K F  as the 5 tables of FIG.  1 . The source files may each have its records organized into a table structure with each record being organized identically, such that each record has the same number of fields and the fields are located in the same byte locations in each record. Alternatively, the source files may have a more complex structure. While each record must include the family key K F  in an identifiable portion of the record, it is not required that K F  be located in corresponding bytes or fields along the length of each record. For example, the records of a source file may be structured such that the first 16 bytes of each record include an identification of the byte positions that include the family key. In the latter example, each record of the source file may include the family key in different byte positions within the record. Additionally, the records of the source file may be of either fixed length or of variable length. 
     The fileset S of source files and the associated tableset T of tables are related in several ways which include the following ways. A first way that S and T are related is that the fileset S and tableset T have the same family key K F  as discussed supra. Accordingly, the family is said to have a common family key, namely K F , across the tables of T and the source files of S. A second way that S and T are related is that the source files of S collectively include data that could be used to generate records in the tables of T. Accordingly, the source files of S are said to be logically associated with the tables of T such that data in the records of the source files of S could be transferred into records of the tables of T. It is not required that the logical association between S and T be simple. An example of such a logical association between S and T in FIGS. 1-2 is: S 1  is associated with T 1 ; S 2 , S 4 , and S 5  are collectively associated with T 2 ; S 6  and a portion of S 3  are respectively associated with T 5  and T 3 ; and S 7 , S 8 , and a remaining portion of S 3  are collectively associated with T 4 . Another example of such a logical association between S and T in FIGS. 1-2 is that a portion of each of files S 1 , S 2 , . . . , S 8  includes data that logically belongs in each of tables T 1 , T 2 , . . . , T 8 . The scope of the present invention includes any logical association between S and T such that one of ordinary skill in the art could map data of S into records of the tables of T. 
     The present invention addresses the issue of how to update and delete records in the tableset T to reflect changes in the fileset S. In practice, both the tables of T and the files of S may be very large. Additionally, there be many such filesets S distributed in many locations, such as around the globe, that are changed frequently (e.g., daily). Such frequent changes in many large filesets is very time consuming using existing software for current database management systems, because the current database management systems do not implement changes to database tables efficiently. The present invention comprises a novel and efficient way of extracting data changes in S for subsequent insertion of the data changes into T by updating and deleting records of T, and further comprises a novel way of loading records (or of deleting records) associated with said extracted data changes into T, as discussed next. The following discussion presents an overview of the present invention with reference to FIG.  3 . 
     FIG. 3 is a block diagram of a system  10  for updating and deleting records in a tableset T of a family of tables T 1 , T 2 , . . . , T M  (M≧1) based on updated or deleted data in N source files S 1 , S 2 , . . . , S N , (N≧1), in accordance with embodiments of the present invention. The family has a common family key K F  across the M tables and across the N source files. A fileset S A  comprising files S 1A , S 2A , . . . , S NA  respectively embodies S 1 , S 2 , . . . , S N  at a time t A  such that T 1 , T 2 , . . . , T M  at time t A  accurately reflects S A ; i.e., at the time t A , the data in T 1 , T 2 , . . . , T M  is up-to-date and thus consistent with the data in S 1 , S 2 , . . . , S N . Additionally, a fileset S B  comprising files S 1B , S 2B , . . . , S NB  respectively embodies S 1 , S 2 , . . . , S N  at a time t B , said t B  and t A  being different times. FIG. 3 illustrates the system and method of the present invention for updating T to reflect the data in S at the time t B . Although t B  is typically later than t A , the scope of the present invention includes the situation in which t B  is earlier than t A . For example, T may have been erroneously updated from an earlier time to a later time, and now such erroneous updating needs to be reversed necessitating updating the database from the later time to the earlier time in order to restore the database to its state that existed at the earlier time. 
     The present invention considers two types of “Changes” in the source files S 1 , S 2 , . . . , S N  and associates tables T 1 , T 2 , . . . , T M , namely “Updates” and “Deletes.” An “Update” may be either a “Modify” Update (change in data of one or more fields of a record) or an “Add” Update (addition of a new record of data). A “Delete” pertains to deletion of an entire record of data. Definitionally, a “Change” in a source file is either an “Update” or a Delete”, and a Change record is either an Update record or a Delete record. 
     The system  10  in FIG. 3 uses a Delta program  12  (for both Update and Delete changes) and a Trigger program  14  (for Update changes only) in sequence, to generate a fileset ΔS comprising files ΔS 1 , ΔS 2 , . . . , ΔS N  which respectively include records of S 1B , S 2B , . . . , S NB  that have data not included in the tableset T at time t A . Next for Update changes, a Bridge program  16  accepts ΔS 1 , ΔS 2 , . . . , ΔS N  as input and generates a fileset ΔT (i.e., ΔT 1 , ΔT 2 , . . . , ΔT M ) as output. ΔT includes the data in ΔS, and ΔT is in a form that is compatible with the M tables of the tableset T. The files ΔT 1 , ΔT 2 , . . . , ΔT M  include data for respectively updating the tables T 1 , T 2 , . . . , T M . The Bridge program  16  receives ΔS from the Trigger program  14  over a data path  15 . The Bridge program  16  transforms ΔS to ΔT by accounting for the logical association between S and T, discussed supra, and also by reconciling differences in data format between S and T (e.g., reconciling differences in the number of bytes allocated for storing data, converting from integer representations to character representations of the same data, packing distributed data into one data word or vice versa, reducing the number of significant figures in a floating point number, etc.). FIG. 3 also shows a data path  25  for transmission of a dataset D DEL  (to be described infra) from the Delta Program  12  to the Objectload Program  26 . 
     Given ΔT (i.e., ΔT 1 , ΔT 2 , . . . , ΔT M ) as output from the Bridge program  16 , the present invention uses each ΔT m  to update the table T m , for m=1, 2, . . . , M. The updating of T m  in its database  30  is accomplished by an Objectload program (O m )  26 , which is an object-level program. As shown in FIG. 3, the Objectload program (O m )  26  is dynamically generated by utilizing a Sourcegen macroprogram  18  to generate source code  20  that is compiled by a compiler  22  into object code  24  that serves as the Objectload program (O m )  26 . 
     In an embodiment of the present invention, the Bridge program  16 , the Objectload program  26  (as well as the associated Sourcegen macroprogram  18  and the compiler  22 ), and the database  30  may be located at a central site. The tableset T in the database  30  may be frequently changed by data drawn from one or more remote sites such that each such remote site has a source fileset that periodically causes updating and deletion of records in T. The Delta program  12  and the Trigger program  14  would exist at each such remote site. Thus, the data path  15  between the Trigger program  14  and the Bridge program  16 , as well as the data path  25  between the Delta Program  12  and the Objectload Program  26 , might represent a data communications network between the central site and the remote site. Definitionally, a data communications network comprises communication lines over which data is transmitted from one node to another, and each said node may include, inter alia, a computer, a terminal, a communication control unit, etc. This embodiment of the present invention is further discussed infra in conjunction with FIG.  12 . 
     In another embodiment of the present invention, the Delta program  12 , the Trigger program  14 , the Bridge program  16 , the Objectload program  26  (as well as the associated Sourcegen macroprogram  18  and the compiler  22 ), and the database  30  would all be located at one site rather than being distributed. This embodiment of the present invention is further discussed infra in conjunction with FIG.  13 . 
     The Delta program  12 , the Trigger program  14 , the Bridge program  16 , the Sourcegen macroprogram  18 , and the Objectload program  26  have been referred to supra as programs. The term “program” is used herein to denote a computer program, a program module or subprogram of a software system, etc. Thus some or all of the aforementioned programs may be isolated computer programs or may be interconnected as modules or subprograms of a larger computer program or of a computer software package. 
     The following discussion presents aspects of the present invention in greater detail. These aspects include matters relating to the Delta program  12 , the Trigger Program  14 , and the Sourcegen macroprogram  18 . 
     FIGS. 4-9 describe an example using the Delta program  12  in conjunction with 3 source files: S 1 , S 2 , and S 3  (N=3). For this example, FIGS. 4,  5 , and  6  depict files S 1A , S 2A , S 3A  respectively embodying S 1 , S 2 , S 3  at times t A , in accordance with embodiments of the present invention. FIGS. 4,  5  and  6  additionally depict files S 1B , S 2B , S 3B  respectively embodying S 1 , S 2 , S 3  at times t B , in accordance with embodiments of the present invention. The tableset T is assumed to initially be consistent with S 1A , S 2A , S 3A  at time t A , and in need of being updated to reflect S 1B , S 2B , S 3B  at time t B . In FIGS. 4-6, the family relates to purchase orders with a family key K F  of Purchase Order Number. Accordingly, all source files depicted in FIGS. 4-6 have the family key of Purchase Order Number in field 1. The source files S 1A  and S 1B  in FIG. 4 are general purchase order files having fields of Purchase Order Number, Purchase Order Date, and Purchase Order Point Location (i.e., the location from which the purchase order was placed). The source files S 2A  and S 2B  in FIG. 5 are purchase order—vendor files having fields of Purchase Order Number, Vendor Identification, Vendor Region (i.e., region of USA where the vendor is located: NE, NW, SE, SW), and Vendor Phone Number. The source files S 2A  and S 2B  in FIG. 5 are purchase order—vendor files having fields of Purchase Order Number, Vendor Identification, Vendor Region (i.e., region of USA where the vendor is located: NE, NW, SE, SW), and Vendor Phone Number. The source files S 3A  and S 3B  in FIG. 6 are purchase order—buyer files having fields of Purchase Order Number, Buyer Identification, Buyer Initials, and Buyer Phone Number. 
     In FIGS. 4-6, the source files of S 1B , S 2B , and S 3B  at time t B  show changes (i.e., “Updates” and “Deletes”) relative to the source files of S 1A , S 2A , and S 3A  at time t A , as highlighted by an asterisk (*), a double asterisk (**), or a triple asterisk (***) to the right of the rows or records having data to be updated or deleted in the tableset T. As stated supra, “Update” records of S 1B  relative to S 1A  and are either of two types. A “Modify” Update record (denoted by the asterisk *) reflects a change of data in one or more fields of a record already existing in S 1A , S 2A , S 3A , while an “Add” Update record (denoted by the double asterisk **) is a new record of data that did not exist in S 1A , S 2A , S 3A . Thus, the Update record includes two types of Update records, namely “Modify” Update records and “Add” Updates records, and the present invention may be implemented for either or both of Modify Update records and Add Update records. FIG. 6 illustrates a special type of Modify Update for Purchase Order Number  247  which had missing Buyer Phone Number at the time t A  but included an explicit Buyer Phone Number at the time t B . A “Delete” record of S iB  relative to S iA  (denoted by the triple asterisk ***) is a record that appears in S iA  but not in S iB , and is thus a record that has been deleted from the source file. 
     Given the source file data of FIGS. 4-6, the Delta program  12  of FIG. 1 generates a dataset D UP  of non-redundant family keyvalues embodying K F  (i.e., Purchase Order Number) in the Update records of S 1B , S 2B , and S 3B . Similarly, the Delta program  12  generates a dataset D DEL  of non-redundant family keyvalues embodying K F  in the Delete records of S 1B , S 2B , and S 3B . A dataset is a collection a data such as a file or a table. A family keyvalue in a record is the value of the family key in the record. For example, the files S 1B , S 2B , and S 3B  in FIGS. 4-6 have family key values of 256, 204, 223, 203, 201, 247, 267, 219, 284, 255, 234, 246, 291, 292, and 293. Of the preceding keyvalues, only non-redundant keyvalues in the Update records of S 1B , S 2B , and S 3B  (considered collectively) are placed by the Delta program  12  in the output dataset D up . Similarly, only non-redundant keyvalues in the Delete records of S 1A , S 2A , and S 3A  (considered collectively) are placed by the Delta program  12  in the output dataset D DEL . 
     FIG. 7 illustrates a flow chart of the Delta program  12  (see FIG. 1) that implements selection and placement of said non-redundant keyvalues in the dataset D UP  or D DEL , in accordance with embodiments of the present invention. In particular, FIG. 7 shows a three-step process of Compare  37 , Sort  38 , and Select  39 . 
     FIG. 8 shows the result of how the Delta program  12  applies the aforementioned Compare  37 , Sort  38 , and Select  39  steps of FIG. 7 to Update records of the source file data of FIGS. 4-6, in accordance with embodiments of the present invention. In FIG. 8 the Compare array  33  shows the family keyvalues extracted from sequentially processing the source files S 1B , S 2B , and S 3B  in sequential order. The Compare  37  operation scans each of S 1B , S 2B , and S 3B  source files and performs comparisons with the S 1A , S 2A , and S 3A  source files to identify records in S 1B , S 2B , and S 3B  which have changed relative to S 1A , S 2A , and S 3A  (i.e., Modify Update records denoted by * in FIGS. 4-6) and to further identify new records in S 1B , S 2B , and S 3B  relative to S 1A , S 2A , and S 3A  (i.e., Add Update records denoted by ** in FIGS.  4 - 6 ). As each such Update record is so identified, the family keyvalue of the record is extracted and stored until the full Compare array  33  of FIG. 8 has been generated. Next, the Compare array  33  is sorted, resulting in the Sort array  34  shown in FIG.  8 . Since the Sort array  34  has redundant keyvalues of 234, 247, 291, 292, and 293, said redundant keyvalues are removed from the Sort array  34 , or equivalently the non-redundant keyvalues are selected from the Sort array  34 , resulting in the Select array  35  of non-redundant keyvalues. The non-redundant keyvalues of the Select array  35  are next stored in the dataset D UP  of FIG.  3 . 
     Note that the Sort  38  step is a practical step in the flow chart of FIG.  7 . Nonetheless, a person of ordinary skill in the art of computer programming would be able to generate the Select array  35  directly from the Compare array  33  without sorting and without undue experimentation. Thus, the Sort  38  step may be used but is not required. If the Sort  38  step is omitted, then the keyvalues in the dataset D UP  will not necessarily be in sorted order, which is acceptable since the present invention requires that the keyvalues in the dataset D UP  be non-redundant and does not require that the keyvalues in the dataset D UP  be sorted. Accordingly, any departure of the Delta program  12  from the flow chart in FIG. 7 or from the description of the Delta program  12  herein, as would be known or obvious to one of ordinary skill in the art of computer programming, is within the scope of the present invention. 
     FIG. 9 shows the result of how the Delta program  12  applies the aforementioned Compare  37 , Sort  38 , and Select  39  steps of FIG. 7 to Delete records of the source file data of FIGS. 4-6, in accordance with embodiments of the present invention. In FIG. 9 the Compare array  63  shows the family keyvalues extracted from sequentially processing the source files S 1B , S 2B , and S 3B  in sequential order. The Compare  37  operation scans each of S 1A , S 2A , and S 3A  source files and performs comparisons with the S 1B , S 2B , and S 3B  source files to identify records in S 1A , S AB , and S 3A  which do not appear in S 1B , S 2B , and S 3B  (i.e., Delete records denoted by *** in FIGS.  4 - 6 ). As each such Delete record is so identified, the family keyvalue of the record is extracted and stored until the full Compare array  63  of FIG. 9 has been generated. Next, the Compare array  63  is sorted, resulting in the Sort array  64  shown in FIG.  9 . Since the Sort array  64  has redundant keyvalues of 204 and 255, said redundant keyvalues are removed from the Sort array  64 , or equivalently the non-redundant keyvalues are selected from the Sort array  64 , resulting in the Select array  65  of non-redundant keyvalues. The non-redundant keyvalues of the Select array  65  are next stored in the dataset D DEL  of FIG.  3 . 
     Note that the Sort  38  step is a practical step in the flow chart of FIG.  7 . Nonetheless, a person of ordinary skill in the art of computer programming would be able to generate the Select array  65  directly from the Compare array  63  without sorting and without undue experimentation. Thus, the Sort  38  step may be used but is not required. If the Sort  38  step is omitted, then the keyvalues in the dataset D DEL  will not necessarily be in sorted order, which is acceptable since the present invention requires that the keyvalues in the dataset D DEL  be non-redundant and does not require that the keyvalues in the dataset D DEL  be sorted. Accordingly, any departure of the Delta program  12  from the flow chart in FIG. 7 or from the description of the Delta program  12  herein, as would be known or obvious to one of ordinary skill in the art of computer programming, is within the scope of the present invention. 
     As shown in FIG. 3, the dataset D UP  as generated by the Delta program  12  is processed by the Trigger program  14  to generate the fileset ΔS, which is then transformed by the Bridge Program  16  into the fileset ΔS for use in changing the tables T 1 , T 2 , . . . , T M  through Updates, in accordance with the Objectload program  26 . In contrast, the dataset D DEL  as generated by the Delta program  12  bypasses the Trigger program  14  and the Bridge Program  16 , and is used more directly to change the tables T 1 , T 2 , . . . , T M  through Deletes, in accordance with the Objectload program  26 . 
     FIG. 10 depicts a flow chart of an embodiment of the Trigger program  14  for generating the fileset ΔS, in accordance with embodiments of the present invention. The Trigger program  14  steps through all records of S iB , (for i=1, 2, . . . , N) to compare the family keyvalue K FVR  of each record of S iB  against the family keyvalues K FV  in the dataset D. What is saved in each file ΔS i  (i=1, 2, . . . , N) is every record of SB for which K FVR  equals one of the family keyvalues K FV  in the dataset D. 
     In FIG.  10  and as shown in block  40 , the Trigger program  14  initially loads control information into memory through, inter alia, a control file  11  shown in FIG.  3 . The control file  11  includes data such as, inter alia, an identification of where the N source files S 1 , S 2 , . . . , S N  are located, an identification of the family, an identification of the family key, etc. 
     Block  41  of FIG. 10 indicates that the family keyvalues from the dataset D are to be loaded into memory in order to minimize access time to said family keyvalues during subsequent processing by the Trigger program  14 . Block  42  sets a counter i to 1, wherein the counter i points to the source file S iB . Decision block  43  asks whether i&lt;N is satisfied in order to determine whether all N source files have been processed. If i=N, then the Trigger program  14  ENDs as shown. If i&lt;N, then the source file S iB  is processed beginning with the decision block  44  which asks whether the source file S iB  exists. If S iB  doesn&#39;t exist then block  45  steps the counter i by 1 and the decision block  43  is again entered to determine whether i&lt;N. If S iB  exists then the source file S iB  is opened (block  46 ) and the output file ΔS i  is opened (block  47 ). Next, a header record may be written to the output file ΔS i  (block  48 ). Said header record may include data such as, inter alia, the current date, an identification of the family, a sequence number, etc. The sequence number is an update number or revision number that is incremented by a fixed amount (e.g., 1) each time that a ΔS fileset is generated, so that a receiving site of the ΔS fileset could check to determine that the received ΔS fileset has an expected sequence number. Block  49  reads the next record R of the source file S iB . Decision block  50  asks whether the source file S iB  is experiencing an End-Of-File (EOF) indication. If the source file S iB  is experiencing an EOF indication, then at least one trailer record may be written to the output file ΔS i  (block  51 ), the counter i is stepped by 1 (block  45 ), and the decision block  43  is again entered to determine whether i&lt;N (i.e., to determine whether all such S iB  files have been processed). If the source file S iB  is not experiencing an EOF indication, then the family keyvalue K FVR  of the current S iB  record R is ascertained (block  52 ). Next, the decision block  53  asks whether said K FVR  equals any K FV  in memory. If K FVR  equals any K FV  in memory, then the current record R of the source file S iB  is written to the output file ΔS i  (block  54 ). Next, there is a return to the block  49  for reading the next record of the source file S iB . FIG. 11 depicts the structure of each file ΔS i  that is generated in accordance with FIG.  10 . 
     The trailer record in FIG. 10 (see block  51 ) may include any number of data items that could be generated through implementing the algorithm of FIG.  10 . For example, the trailer record of ΔS i  may include the total number of records of ΔS i . As another example, the trailer record may include a cumulative dollar amount computed as a summation of corresponding individual dollar amounts in a given field of ΔS i , wherein the summation is performed over all records of ΔS i . For the preceding examples, the data in the trailer record could serve as a consistency check. For example, the Bridge program  16  (see FIG.  3 ), or another program at the site where the Bridge program  16  is located, could independently determine the total number of records of ΔS i  and the cumulative dollar amount, and compare said independently determined parameters with the corresponding parameters on the trailer record of ΔS i . 
     Any departure of the Trigger program  14  from the flow chart in FIG. 10 or from the description of the Trigger program  14  herein, as would be obvious to one of ordinary skill in the art of computer programming, is within the scope of the present invention. 
     The preceding discussion of FIGS. 4-10, in relation to the Delta program  12  and the Trigger program  14 , demonstrates a highly efficient method of assembling data into the fileset ΔS from the source dataset S in order to subsequently change the tableset T through Updates and Deletes. First, only source records having changed data (i.e., Updated or Deleted data) are included in the dataset D UP , the dataset D DEL , and the fileset ΔS. Second, the family keyvalues, rather than source data records, are manipulated in the Delta program  12  and the Trigger program  14 , which substantially reduces the amount of data that the Delta program  12  and the Trigger program  14  process. Third, elimination of redundant keyvalues prevents redundant source data records from being stored in the dataset D UP , the dataset D DEL , and the fileset ΔS. 
     The Objectload Program  26  is used to change the tableset T with Updates as dictated by the fileset ΔS, and with Deletes as dictated by the dataset D DEL . The Objectload Program  26  may used to accomplished either Updates or Deletes, or both Updates and the Deletes. Additionally, the Updates and Deletes may be accomplished within one such Objectload Program  26 , or alternatively the Updates and Deletes may each be accomplished by distinct and independent Objectload Programs  26 . 
     Accomplishment of Deletes by the Objectload Program  26  is straightforward. The Objectload Program  26  deletes all records in the tables T 1 , T 2 , . . . , T M  that have a non-redundant family keyvalue of the dataset D DEL . The records so deleted in the tables T 1 , T 2 , . . . , T M  are said to be “deletion-targeted.” Thus any record in the tables T 1 , T 2 , . . . , T M  that has a non-redundant family keyvalue that exists in the dataset D DEL  is deletion-targeted. 
     FIG. 3 shows how the Objectload Program  26  is generated. As stated supra, the Objectload program  26  is dynamically generated by utilizing the Sourcegen macroprogram  18  to generate source code  20  that is compiled by the compiler  22  into object code  24 . The object code  24  serves as the Objectload program  26 . The source code  20  and the compiler  22  may be in conjunction with any computer language such as PL−1, C, C++, etc. The Sourcegen macroprogram  18  includes macros, wherein a macro is an instruction written as part of a source language that will expand into multiple source language instructions when compiled. Thus, the Sourcegen macroprogram  18  is itself compiled so as to generate the source code  20 . As shown in FIG. 3, the Sourcegen macroprogram  18  reads an input file  17  as input. The input file  17  comprises whatever control information is needed for enabling the Sourcegen macroprogram  18  to generate the source code  20  (and thus the Objectload program  26  downstream) for updating T m  based on data in ΔT m , for m=1, 2, . . . , M. Accordingly, the input file  17  includes parameters comprising at least one of: an identification of ΔT m , a data format description of ΔT m , an identification of T m , a data format description of T m , and a “tweak” flag. A tweak flag informs the Sourcegen macroprogram  18 , and thus also the Objectload program  26 , to implement special/tailored coding for selective updating of targeted fields in a record without processing the whole record in order to reduce computational effort. The input file  17  also includes control information for performing Deletes on the tableset T in accordance with the dataset D DEL . 
     The Objectload program  26  may function without being subject to either of two types of read lockout. If the first type of read lockout were to occur, then an updating or deleting of records of the family tables by the Objectload program  26  would lock out other users who are reading the family tables until the updating ends. If the second type of read lockout were to occur, then read operations on the family tables by other users would prevent the Objectload program  26  from updating or deleting records of the family tables while the operations by other users are occurring. Fortunately, neither type of read lockout occurs with the Objectload program  26 . The term “read lockout,” as used herein, includes both the first type and the second type of read lockout. In contrast with the Objectload program  26 , a database management system (DBMS) may implement a read lockout for a data transaction (i.e., a data read or data write) on any table T A  in the DBMS if said data transaction on the table T A  is executed through a transaction statement of the DBMS (e.g., a SELECT statement of a DBMS such as ORACLE that is based on Structured Query Language). Thus, the Objectload program  26  could coexist with a DBMS that implements such a read lockout without the Objectload program  26  being subject to the read lockout, because data reading and writing by the Objectload program  26  does not occur through transaction statements of the DBMS. 
     The preceding feature of avoiding read lockout may be applied selectively with respect to Updates and Deletes on the tableset T, as illustrated in the following examples. As a first example, read lockout may be avoided on Updates but not on Deletes, or vice versa. As a second example, read lockout may be avoided on both Updates and Deletes. As a third example, read lockout may applied to both Updates and Deletes. 
     The preceding discussion demonstrates that the methodology of the present invention for loading data into the tableset T is highly efficient. First, the present invention uses object code (i.e., the Objectload program  26  of FIG.  3 ), rather than less inherently efficient interpreted code, to directly load data into T. Second, transaction statements of a DBMS, which are highly inefficient for reading and writing onto database tables, are avoided. Third, the present invention loads the tableset T while avoiding read lockout, which maximizes use of a database for both users of the present invention and other users of the DBMS. 
     While the preceding discussion of updating the tableset T of the database  30  was based on processing each table T m  (m=1, 2, . . . , M) separately, and on generating each of the M Objectload programs  26  separately, the system  10  could be modified to have the Sourcegen Macroprogram  18  generate the source code  20  such that the all M tables T 1 , T 2 , . . . , T M  are updated by one source code  20  and by one Objectload program (O)  26 , instead of by the M source codes  20  and M Objectload programs (O 1 , O 2 , . . . , O M )  26 . Thus the present invention requires at least one source code  20  and at least one Objectload program  26  to update the M tables T 1 , T 2 , . . . , T M . 
     Any departure of the flow chart in FIG. 3 of the path from the Sourcegen Macroprogram  18  to the Objectload program  26  or from the description thereof herein, as would be obvious to one of ordinary skill in the art of computer programming, is within the scope of the present invention. 
     Although the present invention beneficially uses the Objectload program  26  for changing the tableset T by Updates and Deletes, the present invention may alternatively use a less efficient program of source statements (e.g., SQL statements) that are interpreted instead of being compiled, since the use of a family key, and other techniques discussed herein, in the Delta Program  12  and the Trigger Program  14  adds substantial efficiency regardless of whether the Objectload program  26  is used. The present invention may also use the Objectload program  26  selectively with respect to Updates and Deletes applied to the fileset T. For example, the Objectload program  26  may be used for Updates, while Deletes are implemented by an interpreter of source code such as SQL code, and vice versa. 
     FIG. 12 is a block diagram of a first computer configuration for the system of FIG. 3, in accordance with embodiments of the present invention. FIG. 12 illustrates a computer system  70  for performing Updates and Deletes on the M tables T 1 , T 2 , . . . , T m  of a family as described supra, in accordance with embodiments of the present invention. The computer system  70  comprises a centralized site  85  and a remote site  75 . The centralized site  85  and the remote site  75  communicate over a data communications network  88  such was described supra in conjunction with the data paths  15  and  25  of FIG.  3 . 
     The centralized site  85  includes a processor  81 , an input device  82  (representing at least one input device) coupled to the processor  81 , an output device  83  (representing at least one output device) coupled to the processor  81 , and a memory device  84  (representing at least one memory device) coupled to the processor  81 . The input device  82  may be, inter alia, a keyboard, a mouse, etc. The output device  83  may be, inter alia, a printer, a plotter, a computer screen, a magnetic tape, a removable hard disk, a floppy disk, etc. The memory device  84  may be, inter alia, a hard disk, a dynamic random access memory (DRAM), a read-only memory (ROM), etc. The memory device  84  stores the Bridge program  16  and the Sourcegen macroprogram  18 . The processor  81  executes the Bridge program  16  and the Sourcegen macroprogram  18 . The memory device  84  includes input data for the Bridge program  16  and the Sourcegen macroprogram  18 . The output device  83  displays output from Bridge program  16  and the Sourcegen macroprogram  18 . Additionally, the output device  83  may be used to display output, source code, graphics, etc. 
     The remote site  75  includes a processor  71 , an input device  72  (representing at least one input device) coupled to the processor  71 , an output device  73  (representing at least one output device) coupled to the processor  71 , and a memory device  74  (representing at least one memory device) coupled to the processor  71 . The input device  72  may be, inter alia, a keyboard, a mouse, etc. The output device  73  may be, inter alia, a printer, a plotter, a computer screen, a magnetic tape, a removable hard disk, a floppy disk, etc. The memory device  74  may be, inter alia, a hard disk, a dynamic random access memory (DRAM), a read-only memory (ROM), etc. The memory device  74  stores the Delta program  12 , the Trigger program  14 , and the family keyvalues  78 . The processor  71  executes the Delta program  12  and the Trigger program  14 . The memory device  74  includes input data for the Delta program  12  and the Trigger  14  program. The output device  73  displays output from Delta program  12  and the Trigger program  14 . Additionally, the output device  73  may be used to display output, source code, graphics, etc. 
     While FIG. 12 shows the computer system  70  as a particular configuration of hardware and software, any configuration of hardware and software, as would be known to a person of ordinary skill in the art, may be utilized for the purposes stated supra in conjunction with the particular computer system  70  of FIG.  12 . For example, the Bridge program  16  and the Sourcegen macroprogram  18  may be in memory devices and may be executed by different processors. 
     FIG. 13 is a block diagram of a second computer configuration for the system of FIG. 3, in accordance with embodiments of the present invention. FIG. 13 illustrates a computer system  90  for performing Updates and Deletes on the M tables T 1 , T 2 , . . . , T m  of a family as described supra, in accordance with embodiments of the present invention. The computer system  90  comprises a processor  91 , an input device  92  coupled to the processor  91 , an output device  93  coupled to the processor  91 , and memory devices  94  and  95  each coupled to the processor  91 . The input device  92  may be, inter alia, a keyboard, a mouse, etc. The output device  93  may be, inter alia, a printer, a plotter, a computer screen, a magnetic tape, a removable hard disk, a floppy disk, etc. The memory devices  94  and  95  may be, inter alia, a hard disk, a dynamic random access memory (DRAM), a read-only memory (ROM), etc. The memory device  95  includes a computer programs  97  (i.e., the Delta program  12 , the Trigger program  14 , the Bridge program  16 , and the Sourcegen macroprogram  18  (see FIG. 3) and the family keyvalues  98 . The processor  91  executes the computer programs  97 . The memory device  94  includes input data  96 . The input data  96  includes input required by the computer programs  97 . The output device  93  displays output from the computer programs  97 . 
     While FIG. 13 shows the computer system  90  as a particular configuration of hardware and software, any configuration of hardware and software, as would be known to a person of ordinary skill in the art, may be utilized for the purposes stated supra in conjunction with the particular computer system  90  of FIG.  13 . For example, the memory devices  94  and  95  may be portions of a single memory device rather than separate memory devices. While embodiments of the present invention have been described herein for purposes of illustration, many modifications and changes will become apparent to those skilled in the art. Accordingly, the appended claims are intended to encompass all such modifications and changes as fall within the true spirit and scope of this invention.