Abstract:
Presented is a method by which large amounts of data may be sorted and downloaded. In the presented invention, two keys are selected which may be associated with all records in a designated distributed, relational, object-oriented, graph or other database, and the records are sorted first by means of the first and second keys to generate a first file, and subsequently by means of the second and first keys to generate a second file. It is determined which of the resultant files contains a larger number of groups, and the file containing the larger number of groups is selected for loading, preferably by means of a plurality of data load streams. Further presented is means by which the data load streams may be executed in parallel with one another whilst minimizing contention for shared resources. The method additionally helps to increase the amount of parallelism that may be deployed in support of database search operations.

Description:
FIELD OF THE INVENTION 
       [0001]    The present invention relates to information technology systems and architectures for database loading and searching. More particularly, the present invention relates to dynamic database management system architectures wherein the distribution of linked records within a system is arranged or altered. 
       BACKGROUND OF THE INVENTION 
       [0002]    The subject matter discussed in the background section should not be assumed to be prior art merely as a result of its mention in the background section. Similarly, a problem mentioned in the background section or associated with the subject matter of the background section should not be assumed to have been previously recognized in the prior art. The subject matter in the background section merely represents different approaches, which in and of themselves may also be inventions. 
         [0003]    Information technology systems that store data and enable searching thereof are widely available in all economic sectors. Many systems have high volumes of data, wherein potentially terabytes of data must be searched within milliseconds or microseconds. In large database systems, to include distributed database systems, the arrangement of data within a plurality of resources can be a determination of the effectiveness of high-volume and broad searching. 
         [0004]    The prior art, however, fails to optimally, dynamically configure information technology systems or networks that adapt and distribute data among resources to enable rapid searching of large amounts of data, to include terabytes and more. Means to distribute and load volumes of data among a plurality of storage resources in a method that significantly improves likelihood of distributed the data more evenly about the data storage resources are currently non-optimal in the prior art. 
         [0005]    There is therefore a long-felt need to provide a system and method for the configuration of distributed data of systems or networks among resources to allow rapid loading and searching of large amounts of data. 
       OBJECTS OF THE INVENTION 
       [0006]    It is an object of the method of the present invention to provide a novel and non-obvious ability to distribute and load volumes of data among a plurality of storage resources in a method that significantly improves likelihood of distributed the data more evenly about the data storage resources and thereby better improving the operation of an information technology system or database management network by better solving the technological challenge of efficiently storing different collections of data of many varieties. 
         [0007]    It is an additional object of the method of the present invention to better solve the technological problem of information technology systems in the challenges of applying resources in distributing data to better enable efficient searching of the storage of data. 
         [0008]    It is an additional and optional object of the method of the present invention to enable information systems and information technology networks to better solve the technological problems of efficiently storing of large volumes of rapidly acquired data, e.g., receiving terabytes or more of data received generated within seconds of time, and to enable efficient searching of these large volumes of data as have been efficiently and distributively stored in accordance with the method of the present invention. 
       SUMMARY OF THE INVENTION 
       [0009]    Towards these objects and other objects that will be made obvious in light of the present disclosure a process is presented by which a database may contain an extremely large number of linked data records, and by which a first and second key are selected by a computer processor, and both the first key and the second key may be associated with any plurality of data records within the database. The database may use any database technology known in the art, including but not limited to, graph, object-oriented, relational, key-value, column store, structured query language (“SQL”), not only structured query language (“NoSQL”), or document database management system (hereinafter “DBMS”). The DBMS may be centralized or distributed. The database technology type is largely immaterial to the effective functioning of the invention, and thus may be or comprise any suitable current or future database technology known in the art. The first key and/or the second key may optionally, but are not limited to, be an element directly taken from the data record (e.g. the final two digits of a plurality of phone numbers), or may optionally be an algorithmically generated hash of an element of the data record, or of the entirety of the data record. 
         [0010]    In the preferred embodiment of the invention, the processor may subsequently sort the plurality of records using the first key to generate a first intermediate resultant file, i.e. sorting the plurality of records in order by the first key as a major key and the second key as a minor key to generate a first candidate file. The processor may count the number of “groups” of records corresponding to the key pairs within the first candidate file. The processor may subsequently sort the same plurality of records by the second key as the major key and the first key as the minor key to generate a second candidate file and count the number of groups of records corresponding to the key pairs within the second candidate file. The processor, determining the candidate file having the greater number of groups, chooses the candidate file having the greater number of groups, and loads the candidate file into the receiving database of a database management system (“DBMS”), wherein the DBMS is capable of searching the designated file upon receipt. It is not necessary that the data records be homogenous, but each data record must preferably have similar data elements such that the first key and the second key may be applied thereto. 
         [0011]    It is understood that in certain alternate preferred embodiments of the method of the present invention that a key group comprising three or more keys selected from a plurality of software records may be formed and applied to designate a plurality of resources and/or to selectively associate software records of the plurality of software records with the plurality resources for the purpose of selectively distributing and loading the plurality of software records into the plurality of resources. 
         [0012]    The resources to which the output files may be assigned may optionally be a plurality of physically separate computing systems, or may optionally be one or more DBMS systems made available by a designated receiving database. Keys are mapped to corresponding resources, whereby the resources are designed to receive the keys to which the resources correspond, and to create a sorted, easily searchable store of the keys. The speed and efficiency with which the data load resources are supplied with keys and/or key pairs are preferably restricted only by the physical limitations of the computing power of the receiving database, rather than by technological or methodological means—in other words, given that the software operations are ultimately occurring on a physical medium, the limitations of physical media apply. The hosting resource, or the DBMS resource hosted by a computing system, however, is made more efficient by the application of resources toward the distribution of data, such that the data is distributed more evenly about data storage resources. 
         [0013]    In a further aspect of the preferred embodiment of the invention, the computer processor or a human may determine an arbitrary, fixed number of output files to be generated, and a number of records to be assigned to each output file. The computer processor may further preferably assign a substantively equal number of groups to each of the arbitrary, fixed number of output files, and may subsequently upload the designated number of output files to the DBMS in a parallel manner. 
         [0014]    In a yet further aspect of the preferred embodiment of the invention, the candidate file is preferably split into at least two, but optionally more than two, output files, and each group is included in exactly one output file, without being split or divided. The two or more output files may subsequently be loaded in a parallel state to the DBMS, according to a plurality of unique key pairs associated with unique records within the output files. A list of all possible key pairs among the first key and the second key is preferably generated, and records having the same key as either component (key) of a key pair are preferably not uploaded in parallel to one another when they are contained within different output files. 
         [0015]    Despite trying to achieve an even spread of groups of data to be loaded in parallel, it is possible that some groups could be larger than others. This may cause one or more streams to hold shared resources for longer than an average time. An enhanced stream key list algorithm may mitigate this problem by (1) retaining a frequency count for each key pair; and (2) checking the size of each batch inserted into the load files during a pass across each of the data load streams and adding one or more batches to data load streams that are likely to finish processing a batch before others. 
         [0016]    This Summary is provided to introduce a selection of concepts in a simplified form that are further described below in the Detailed Description. This Summary is not intended to identify key features or essential features of the claimed subject matter, nor is it intended to be used to limit the scope of the claimed subject matter. 
     
    
     
       BRIEF DESCRIPTION OF THE FIGURES 
         [0017]    These, and further features of the invention, may be better understood with reference to the accompanying specification and drawings depicting the preferred embodiment, in which: 
           [0018]      FIG. 1  is a flowchart of an aspect of the invented method, wherein a processor of a computing system selects, orders and loads a plurality of keys by means of one or more data load streams; 
           [0019]      FIG. 2  is a flowchart of an additional aspect of the invented method, wherein a plurality of key pairs are loaded in parallel processes; 
           [0020]      FIG. 3  is a flowchart of a further aspect of the invented method, wherein the computing system sorts keys, assigns non-conflicting key pairs to one or more data load streams, and loads data in parallel; 
           [0021]      FIG. 4  is a block diagram of a communications network, wherein the network includes a computing system, and a database bidirectionally communicatively coupled by means of one or more network coupling means known in the art, including but not limited to the Internet; 
           [0022]      FIG. 5  is a block diagram of the computing system of  FIG. 4 ; 
           [0023]      FIG. 6  is a block diagram of the database of  FIG. 4 ; 
           [0024]      FIG. 7A  is a block diagram of an exemplary first key having a flag indicating the key&#39;s availability 
           [0025]      FIG. 7B  is a block diagram of the exemplary first key having a flag indicating that the key is committed to a resource; 
           [0026]      FIG. 8A  is a block diagram of a key list, wherein the key list contains keys which are available to be loaded by resources in key pairs, and wherein each key is mapped to a single loading resource; 
           [0027]      FIG. 8B  is a block diagram of a key list, wherein the key list contains keys which are available to be loaded by resources in key pairs, and wherein a plurality of available keys are mapped to a single loading resource; and 
           [0028]      FIG. 9  is a networked block diagram of the above-described processes. 
       
    
    
     DETAILED DESCRIPTION 
       [0029]    Referring now generally to the Figures and particularly to  FIG. 1 ,  FIG. 1  is a flowchart of an aspect of the invention whereby a processor  2  of a computing system  4  directs the computing system  4  to select, order and load a plurality of keys KEY.001-KEY.N to access a plurality of data records REC.001-REC.N by means of one or more data load streams  6 . In step  1 . 02  the processor  2  selects a plurality of data records REC.001-REC.N from a data load stream  6 , input file  8  or a database  10 , which database  10  may use any database maintenance technology known in the art, including but not limited to, graph, object-oriented, relational, key-value, column store, structured query language (“SQL”), not only structured query language (“NoSQL”), or document database management system (hereinafter “DBMS”). The DBMS may be centralized or distributed. The particular database technology type chosen for application of the method of the invention is largely immaterial to the effective functioning of the method of the invention, and thus may be or comprise any suitable current or future database maintenance technology known in the art. The data records REC.001-REC.N may optionally be homogenous, but homogeneity is not necessary for the functioning of the invention; the data records REC.001-REC.N need only have similar enough data elements  12  such that a first key KEY 1  and a second key KEY 2  may be utilized. 
         [0030]    In step  1 . 04  the processor  2  selects the data elements  12  for incorporation into the first key KEY 1  and the second key KEY 2 , and determines how to generate the first key KEY 1  and the second key KEY 2 . The first key KEY 1  and the second key KEY 2  may optionally be selected as direct elements of the data records REC.001-REC.N, for example as the last two numbers of a telephone number, or the first key KEY 1  and the second key KEY 2  may optionally be an algorithmically determined hash of a data element of the data records REC.001-REC.N by any convenient algorithm that preserves uniqueness. Additionally, a plurality of different data records REC.001-REC.N may optionally generate an identical first key KEY 1  and/or second key KEY 2 , as in the telephone number example: thousands of phone numbers may share the identical terminating digits, but may otherwise be distinct. 
         [0031]    In step  1 . 06  the processor  2  sorts the selected plurality of data records REC.001-REC.N by the first key KEY 1  as a major key and the second key KEY 2  as a minor key, generating a first candidate file  14 A. The processor  2  may additionally count the number of “groups” 16 of data records REC.001-REC.N generated within the first candidate file  14 A by the sorting of the first key KEY 1  through the second key KEY 2 . In step  1 . 08 , the processor  2  sorts the selected plurality of data records REC.001-REC.N by the second key KEY 2  as the major key and the first key KEY 1  as the minor key, generating a second candidate file  14 B. The processor  2  may additionally count the number of “groups” 16 of data records REC.001-REC.N generated within the second candidate file  14 B by the sorting of the second key KEY 2  through the first key KEY 1 . The two separate sorting processes preferably generate differing numbers of groups  16  within the first candidate file  14 A and the second candidate file  14 B, and the processor  2  in step  1 . 10  selects the candidate file  16 A or  16 B having the greater number of groups  16  therein. The greater number of groups  16  allows for greater evenness of groups  16  among a plurality of data load streams  6 . 
         [0032]    In step  1 . 12  the processor  2  generates a key pair list  18  from the selected candidate file  16 A or  16 B (hereinafter “selected file  16 A or  16 B”), wherein the key pair list  18  contains every detected combination of the first key KEY 1  and the second key KEY 2 . In step  1 . 14  the processor  2  generates a key list  20 , wherein the key list  20  contains every one of the first key KEY 1  and of the second key KEY 2  generated from the data records REC.001-REC.N. In step  1 . 16  the plurality of first keys KEY 1  and second keys KEY 2  are associated with access and loading resources  22  on a many-to-one basis, such that each of the plurality of first keys KEY 1  and second keys KEY 2  corresponds with a single access and loading resource  22 . The loading resources  22  may optionally be physically separate computing devices, or may optionally be one or more DBMS systems within a single computing device. The speed and power of loading resources  22  are limited only by the capabilities of the computing devices on which they function. The allocation of the plurality of first keys KEY 1  and the plurality of second keys KEY 2  is important for at least two reasons: first, the access and loading resource  22  is potentially shareable among users, and thus the goal of the allocation process is to only allow one user to write to the data and loading resource  22  at a time, minimizing bottlenecks; second, distributing the data to known places increases the efficiency of parallel search operations. 
         [0033]    It is understood that in certain alternate preferred embodiments of the method of the present invention that a key group comprising three or more keys KEY 1 -KEYN may be formed and applied to designate individual resources  22  of the plurality of resources  22  and/or to selectively associate the plurality of records REC.001-REC.N with the plurality resources  22  for the purpose of selectively distributing and loading the plurality of records REC.001-rec.N into the plurality of resources  22 . 
         [0034]    In step  1 . 18  the processor  2  selects a key pair  24  from the list of key pairs  24  generated above. In step  1 . 20  the processor  2  determines whether the keys KEY.001-KEY.N in the selected key pair  24  are available. To determine availability, the processor  2  determines whether one or both of the keys KEY.001-KEY.N in the selected key pair  24  are already being allocated to another access or loading process  6  (also represented herein as a “stream  6 ”). When the processor  2  determines in step  1 . 20  that the keys KEY.001-KEY.N in the selected key pair  24  are not available, the processor  2  returns to step  1 . 18 , wherein the processor  2  selects an alternative key pair  24 , and repeats steps  1 . 18  through  1 . 20  until it is determined that the selected keys KEY.001-KEY.N are available. When it is determined in step  1 . 20  that the keys KEY.001-KEY.N in the selected key pair  24  are available, the processor  2  associates the selected key pair  24  to a data load stream  6 . In step  1 . 24  the processor  2  determines whether the data load stream  6  is available, i.e. that the data load stream  6  is not already associated to a maximum number of key pairs  24 . When the processor  2  determines that the data load stream  6  is unavailable, the processor  2  proceeds to step  1 . 26 , wherein the processor  2  waits for the data load stream  6  to become available. The processor  2  subsequently proceeds to step  1 . 22 , and associates the selected key pair  24  to the data load stream  6 . Alternately, when the processor  2  determines in step  1 . 24  that the data load stream  6  is available, the processor  2  proceeds to step  1 . 28 , wherein the processor  2  commits the key pair  24  to the available data load stream  6 . A committed key pair  24  cannot be accessed for the duration of the key pair&#39;s  24  load process. One or more keys KEY.001-KEY.N may be indicated to be committed to a key pair  24  load process by the triggering within the one or more keys KEY.001-KEY.N a “committed” flag FLAG.CMT.001-FLAG.CMT.N. 
         [0035]    In step  1 . 30  the processor  2  loads an input file  8  from the committed keys KEY.001-KEY.N. In step  1 . 32  the input file  8  is transferred to an associated computing resource  22 . The processor  2  releases the keys KEY.001-KEY.N back to the key list  20 , wherein the released keys KEY.001-KEY.N on the key list  20  may be used for alternate load processes in alternate key pairs  24  in step  1 . 34 . When the keys KEY.001-KEY.N are released to the key list  20 , an “available” flag FLAG.AVAIL.001-FLAG.AVAIL.N is triggered within the one or more keys KEY.001-KEY.N, indicating that the keys KEY.001-KEY.N are available to be paired and used. In step  1 . 35  any shared resource  22  which has been obtained for exclusive write/update usage at the start of the loading of a new batch of key pair  24  records REC.001-REC.N is released at the end of loading that batch of key pair  24  records REC.001-REC.N. In step  1 . 36  the key pair  24  is excluded from the key pair list  18 , as it is no longer necessary to access that particular key pair  24 . 
         [0036]    The invention is intended for use, but is not limited to use, in highly parallel environments, such as Apache Hadoop MapReduce, authored by Apache Software Foundation of Delaware, US, or Apache Spark, authored by Apache Software Foundation of Delaware, US. The invention may be used for batches of “Big Data” (as defined below) or for micro-batches of “Fast Data” (as defined below). Although the invention is most effective when applied to the loading of graph, or network, data, it could also be applied in any situation where clustering of data or use of a computing resource could overload a particular device, such as a disk drive, communications switch, or Central Processing Unit (CPU). 
         [0037]    “Big Data” is a broad term for data sets so large or complex that traditional data processing applications are inadequate. The challenges facing those who work with Big Data are analysis, capture, data curation, search, sharing, storage, transfer, visualization, and information privacy. 
         [0038]    “Fast Data” is a term coined by the computer/social science expert John Furrier. Fast Data is a “cousin” of Big Data and implies the ability to make near real-time decisions and enable orders of magnitude improvements in elapsed time to decisions for businesses. 
         [0039]    In step  1 . 38  the processor  2  determines whether a key pair  24  remains which has not yet been accessed. When the processor  2  determines in step  1 . 38  that one or more additional key pairs  24  remain unaccessed, the processor  2  returns to step  1 . 18 , selects a new key pair  24 , and executes the loop of steps  1 . 18  through  1 . 38  until it is determined in step  1 . 38  that no key pairs  24  remain. When the processor  2  determines that no key pairs  24  remain, the processor  2  proceeds to step  1 . 40 , wherein the processor  2  continues to alternate operations. 
         [0040]    Referring now generally to the Figures, and particularly to  FIG. 2 ,  FIG. 2  is flowchart of an additional aspect of the invention, wherein data records REC.001-REC.N are loaded to the database  10  in parallel processes. The processor  2  proceeds from step  1 . 22  of the method of  FIG. 1 , and determines in step  2 . 00  whether a download stream is available. When the processor  2  determines that a download stream is not available, the processor  2  waits for the stream to become available in step  2 . 02 , and subsequently returns to step  2 . 00  and determines whether the download stream is available. When the processor  2  determines in step  2 . 00  that the download stream is available, the processor  2  assigns a new key pair to the download stream in step  2 . 04 . In step  2 . 06  the download stream waits for the key pair  24 . In step  2 . 08 , it is determined whether the download stream has accessed the key pair  24 . When it is determined that the download stream has not accessed the key pair  24 , the download stream continues to wait for the key pair  24  in step  2 . 06 . When it is determined that the download stream has accessed the key pair  24  in step  2 . 08 , the data is loaded from the key pair  24  in step  2 . 10 . Steps  2 . 04  through  2 . 10  are subsequently repeated as necessary. 
         [0041]    Referring now generally to the Figures, and particularly to  FIG. 3 ,  FIG. 3  is a flowchart of a further aspect of the invented method, wherein the processor  2  sorts keys KEY.001-KEY.N, assigns key pairs  24  to one or more data load streams  6 , and loads data records REC.001-REC.N in parallel. In step  3 . 02  the processor  2  selects the first key KEY 1  and the second key KEY 2  from the data elements  12  of the data records REC.001-REC.N. In step  3 . 04  the processor  2 , or a user, determines whether it is necessary to shorten or compress the first key KEY 1  and the second key KEY 2 . The primary criteria by which the need for compression or shortening may be judged are (a.) the overall length of the data element of the data record; and (b.) the nature of the data element of the data record—whether it is alphanumeric, or contains other symbols which may interrupt the load processes. When the processor  2  determines that it is necessary for the first key KEY 1  and the second key KEY 2  to be compressed, hashed, or shortened, the processor  2  proceeds to step  3 . 06 , wherein the first key KEY 1  and the second key KEY 2  are compressed, hashed, or shortened as necessary. When it is determined that the first key KEY 1  and the second key KEY 2  do not need to be shortened, compressed, or hashed, the processor  2  advances to step  3 . 08 , wherein the data records REC.001-REC.N are sorted first by the first key KEY 1  as major key and by the second KEY 2  as the minor key, generating a first candidate file  14 A, containing a plurality of groups  16  of data records REC.001-REC.N. In step  3 . 10  the data records REC.001-REC.N are sorted first by the second key KEY 2  as the major key and by the first key KEY 1 , as the minor key, generating a second candidate file  14 B, containing a second plurality of groups  16  of data records REC.001-REC.N. The number of groups  16  of data records REC.001-REC.N contained within the first candidate file  14 A and the second candidate file  14 B are compared in step  3 . 14 , and the candidate file containing the greater number of groups  16  of data records REC.001-REC.N is selected. 
         [0042]    In step  3 . 16  the total number of data load streams  6  to use in the data load process is chosen either by the processor  2  or by a user. The number of data load streams  6  may be limited by processing power of a designated computing system  4 , or by any other data limiting method known in the art. In step  3 . 18  the processor  2  creates a “can-use key list,”  26  composed of all of the keys KEY.001-KEY.N in the set of data records REC.001-REC.N, without allowing for repetition of keys KEY.001-KEY.N. In step  3 . 20  a key pair  24  is randomly designated from the keys KEY.001-KEY.N in the can-use key list  26 , and designated for a specific data load stream  6 . In step  3 . 22  the processor  2  removes the keys KEY.001-KEY.N composing the newly created key pair  24  from the can-use key list  26 , granting exclusive access to the keys KEY.001-KEY.N in the key pair  24  to the data load stream  6  in which the key pair  24  has been placed. The keys KEY.001-KEY.N comprising the key pair  24  may be indicated to be committed to a load process by the triggering within the designated keys KEY.001-KEY.N a “committed” flag FLAG.CMT.001-FLAG.CMT.N. 
         [0043]    In step  3 . 24  the data records REC.001-REC.N associated with the key pair  24  in the data load stream  6  are placed into a stream file  28 . A next data load stream  6  is selected in step  3 . 26 . In step  3 . 28  the processor  2  determines whether more data load streams  6  and/or keys KEY.001-KEY.N and key pairs  24  are available for access. When the processor  2  determines in step  3 . 28  that more data load streams  6  and/or keys KEY.001-KEY.N and key pairs  24  are available for access, the processor  2  returns to step  3 . 20  and repeats the loop of steps  3 . 20  through  3 . 28  until it is determined in step  3 . 28  that no more data load streams  6  and/or keys KEY.001-KEY.N and key pairs  24  are available for access. When the processor  2  determines that no more data load streams  6  and/or keys KEY.001-KEY.N and key pairs  24  are available, the processor  2  proceeds to step  3 . 30 , wherein the data records REC.001-REC.N are loaded in parallel processes. In step  3 . 32 , the processor  2  terminates the process. 
         [0044]    Referring now generally to the Figures and particularly to  FIG. 4 ,  FIG. 4  is a block diagram of an electronic communications network  30  (hereinafter “the network  30 ”), wherein the network includes the computing system  4 , and a database  10  bidirectionally communicatively coupled by means of one or more network coupling means known in the art, including but not limited to the Internet. The computing system  4  and the database  10  each preferably comprise or are bi-directionally communicatively coupled with a separate database management system software, respectively a computing system DBMS  4 A and a database DBMS  10 A. 
         [0045]    The computing system DBMS  4 A and/or the database DBMS  10 A may be or comprise a graph database management system, an object oriented database management system (“OODBMS”) and/or a relational database management system (“RDBMS”), and one or more databases DBS  41  and/or DBS  10 I, may be or comprise a graph database, an object oriented database and/or a relational database. More particularly, the computing system DBMS  4 A and/or the database DBMS  10 A may be, may be comprised within, or may comprise one or more prior art database management systems including, but not limited to, an OBJECTIVITY/DB ™ object-oriented database management system marketed by Objectivity, Inc. of San Jose, Calif., an INFINTEGRAPH™ graph database management system marketed by Objectivity, Inc. of San Jose, Calif., an ORACLE DATABASE™ database management system marketed by Oracle Corporation, of Redwood City, Calif.; a Database 2™, also known as DB2™, relational database management system as marketed by IBM Corporation of Armonk, N.Y.; a Microsoft SQL Server™ relational database management system as marketed by Microsoft Corporation of Redmond, Wash.; MySQL™ as marketed by Oracle Corporation of Redwood City, Calif.; and a MONGODB™ as marketed by MongoDB, Inc. of New York City, USA; and the POSTGRESQL™ open source object-relational database management system, or other suitable database management system or systems known in the art. 
         [0046]    The database  10  may bi-directionally communicate and transfer data with the computing system  4  via the network  30  by suitable electronic communications messaging protocols and methods known in the art including, but not limited to, Simple Object Access Protocol, Representational State Transfer, and/or a web service adapted to conform with the architecture and structure of the World Wide Web. 
         [0047]    It is understood that the computing system  4  comprises a computing system software program SW.CMP comprised within, hosted by and/or enabled by a bundled computer software and hardware product such as, but not limited to, a.) a network-communications enabled THINKSTATION WORKSTATION ™ notebook computer marketed by Lenovo, Inc. of Morrisville, N.C.; (b.) a NIVEUS 5200 computer workstation marketed by Penguin Computing of Fremont, Calif. and running a LINUX ™ operating system or a UNIX ™ operating system; (c.) a network-communications enabled personal computer configured for running WINDOWS XP™, or WINDOWS 8™ operating system marketed by Microsoft Corporation of Redmond, Wash.; or (d.) other suitable computational system or electronic communications device known in the art capable of providing or enabling a electronic communications and database management operations known in the art. 
         [0048]    It is understood that the database  10  additionally comprises a database system software program SWDB comprised within, hosted by and/or enabled by a bundled computer software and hardware product, such as, but not limited to, a.) a network-communications enabled THINKSTATION WORKSTATION™ notebook computer marketed by Lenovo, Inc. of Morrisville, NC; (b.) a NIVEUS 5200 computer workstation marketed by Penguin Computing of Fremont, Calif. and running a LINUX ™ operating system or a UNIX ™ operating system; (c.) a network-communications enabled personal computer configured for running WINDOWS XP™, or WINDOWS 8™ operating system marketed by Microsoft Corporation of Redmond, Wash.; or (d.) other suitable computational system or electronic communications device known in the art capable of providing or enabling a electronic communications and database management operations known in the art. 
         [0049]    Referring now generally to the Figures, and particularly to  FIG. 5 ,  FIG. 5  is a block diagram of the computing system  4  of  FIG. 4 , wherein the computing system  4  may comprise: a central processing unit (“CPU”)  4 B; a user input module  4 D; a display module  4 E; a software bus  4 C bi-directionally communicatively coupled with the CPU  4 B, the user input module  4 D, the display module  4 E; the software bus  4 C is further bi-directionally coupled with a network interface  4 F, enabling communication with alternate computing devices by means of the network  30 ; and a memory  4 G. The computing system software bus  4 C facilitates communications between the above-mentioned components of the computing system  4 . 
         [0050]    The memory  4 G of the computing system  4  includes a computing system software operating system OP.SYS  4 H. The computing system  4  software OP.SYS  4 H of the computing system  4  may be selected from freely available, open source and/or commercially available operating system software, to include but not limited to a LINUX™ or UNIX™ or derivative operating system, such as the DEBIAN™ operating system software as provided by Software in the Public Interest, Inc. of Indianapolis, Ind.; a WINDOWS XP™, or WINDOWS 8™ operating system as marketed by Microsoft Corporation of Redmond, Wash.; or the MAC OS X operating system or iPhone G4 OS™ as marketed by Apple, Inc. of Cupertino, Calif. The computing system memory  4 G further includes the computing system software program SW.CMP, a computing system user input driver UDRV.CMP, a computing system display driver DIS.CMP, and a computing system network interface drive NIF.CMP. 
         [0051]    The exemplary computing system software program SW.CMP is optionally adapted to enable the computing system  4  to manage the computing system DBMS  4 A to perform, execute and instantiate all elements, aspects and steps as required of the computing system  4  to practice the invented method in its various preferred embodiments interaction with the database  10 . 
         [0052]    Referring now generally to the Figures, and particularly to  FIG. 6 ,  FIG. 6  is a block diagram of the server of  FIG. 4 , wherein the server may comprise: a central processing unit (“CPU”)  10 B; a user input module  10 D; a display module  10 E; a software bus  10 C bi-directionally communicatively coupled with the CPU  10 B, the user input module  10 D, the display module  10 E; the software bus  10 C is further bi-directionally coupled with a network interface  10 F, enabling communication with alternate computing devices by means of the network  30 ; and a memory  10 G. The server software bus  10 C facilitates communications between the above-mentioned components of the server. 
         [0053]    The memory  10 G of the server includes a server software operating system OP.SYS  10 H. The server software OP.SYS  10 H of the server may be selected from freely available, open source and/or commercially available operating system software, to include but not limited to a LINUX™ or UNIX™ or derivative operating system, such as the DEBIAN™ operating system software as provided by Software in the Public Interest, Inc. of Indianapolis, IN; a WINDOWS XP™, or WINDOWS 8™ operating system as marketed by Microsoft Corporation of Redmond, Wash.; or the MAC OS X operating system or iPhone G4 OS™ as marketed by Apple, Inc. of Cupertino, Calif. The server memory  10 G further includes the server software program SW.SRV, a server user input driver UDRV.SRV, a server display driver DIS.SRV, and a server network interface drive NIF.SRV. 
         [0054]    The exemplary server software program SW.SRV is optionally adapted to enable the server to manage the server DBMS  10 A to perform, execute and instantiate all elements, aspects and steps as required of the server to practice the invented method in its various preferred embodiments interaction with the database  10 . 
         [0055]    Referring now generally to the Figures, and particularly to  FIG. 7A ,  FIG. 7A  is a block diagram of a first exemplary software key KEY.001 when the exemplary first key KEY.001 is displaying a flag FLAG.AVAIL.001 indicating the availability of the first key KEY.001. The first key KEY.001 further includes a key identifier KEY.ID.001, by which the first key KEY.001 may be identified within a key list  20 , and by the processor  2  of the computing system  4 , and by the database  10 ; and a record identifier REC.ID.001 , by which the data record REC.001 that the first key KEY.001 represents may be identified. 
         [0056]    Referring now generally to the Figures, and particularly to  FIG. 7B ,  FIG. 7B  is a block diagram of a first exemplary software key KEY.001 when the exemplary first key KEY.001 is displaying a flag FLAG.CMT.001 indicating that first key KEY.001 has been committed to a loading resource  22 . The first key KEY.001 further includes a key identifier KEY.ID.001 , by which the first key KEY.001 may be identified within a key list  20 , and by the processor  2  of the computing system  4 , and by the database  10 ; a record identifier REC.ID.001, by which the data record REC.001 that the first key KEY.001 represents may be identified; and a resource identifier RSC.ID.001 by which the loading resource  22  to which the key has been committed may be identified. 
         [0057]    Referring now generally to the Figures, and particularly to  FIG. 8A ,  FIG. 8A  is a block diagram of a key list  20 , wherein the key list  20  contains keys KEY.001-KEY.N which are available to be loaded by loading resources  22  in key pairs  24 , and wherein each key KEY.001-KEY.N is mapped to a single loading resource  22 . Each key KEY.001-KEY.N within the key list  20  comprises a key identifier KEY.ID.001-KEY.ID.N, a record identifier REC.ID.001-REC.ID.N, and an “available” flag FLAG.AVAIL.001-FLAG.AVAIL.N, indicating that the key is not yet committed to a data load process. 
         [0058]    Referring now generally to the Figures, and particularly to  FIG. 8B ,  FIG. 8B  is a block diagram of the key list  20 , wherein the key list  20  contains keys KEY.001-KEY.N which are available to be loaded by loading resources  22  in key pairs  24 , and wherein a plurality of keys KEY.001-KEY.N are mapped to each available loading resource  22 . Such an occurrence may be necessary when the physical or software restraints of a computing system create a situation in which insufficient loading resources  22  are available for the necessary data load process. 
         [0059]    Referring now generally to the Figures, and particularly to  FIG. 9 ,  FIG. 9  is a block diagram of a networked process of the invention. The Figure shows the computing system  4 , having a database management system therein, and the processor  2 , which enables the execution of the steps the invented process as described in  FIG. 1  through  FIG. 3 . The Figure further shows a plurality of data load streams  6 , each data load stream  6  containing a plurality of key pairs  24 , wherein all but one of key pairs  24  of each data load stream  6  is queued for access. The key pairs  24  at the “front” of a data load stream  6  each correspond to a download resource  22 , contained within a database  10 , such that there is optimally no delay in the load and/or access of any particular key. 
         [0060]    The foregoing description of the embodiments of the invention has been presented for the purpose of illustration; it is not intended to be exhaustive or to limit the invention to the precise forms disclosed. Persons skilled in the relevant art can appreciate that many modifications and variations are possible in light of the above disclosure. 
         [0061]    Some portions of this description describe the embodiments of the invention in terms of algorithms and symbolic representations of operations on information. These algorithmic descriptions and representations are commonly used by those skilled in the data processing arts to convey the substance of their work effectively to others skilled in the art. These operations, while described functionally, computationally, or logically, are understood to be implemented by computer programs or equivalent electrical circuits, microcode, or the like. Furthermore, it has also proven convenient at times, to refer to these arrangements of operations as modules, without loss of generality. The described operations and their associated modules may be embodied in software, firmware, hardware, or any combinations thereof. 
         [0062]    Any of the steps, operations, or processes described herein may be performed or implemented with one or more hardware or software modules, alone or in combination with other devices. In one embodiment, a software module is implemented with a computer program product comprising a non-transitory computer-readable medium containing computer program code, which can be executed by a computer processor for performing any or all of the steps, operations, or processes described. 
         [0063]    Embodiments of the invention may also relate to an apparatus for performing the operations herein. This apparatus may be specially constructed for the required purposes, and/or it may comprise a general-purpose computing device selectively activated or reconfigured by a computer program stored in the computer. Such a computer program may be stored in a non-transitory, tangible computer readable storage medium, or any type of media suitable for storing electronic instructions, which may be coupled to a computer system bus. Furthermore, any computing systems referred to in the specification may include a single processor or may be architectures employing multiple processor designs for increased computing capability. 
         [0064]    Embodiments of the invention may also relate to a product that is produced by a computing process described herein. Such a product may comprise information resulting from a computing process, where the information is stored on a non-transitory, tangible computer readable storage medium and may include any embodiment of a computer program product or other data combination described herein. 
         [0065]    Finally, the language used in the specification has been principally selected for readability and instructional purposes, and it may not have been selected to delineate or circumscribe the inventive subject matter. It is therefore intended that the scope of the invention be limited not by this detailed description, but rather by any claims that issue on an application based herein. Accordingly, the disclosure of the embodiments of the invention is intended to be illustrative, but not limiting, of the scope of the invention, which is set forth in the following claims.