Abstract:
A data processing system (10) comprising a general purpose computer (20) for carrying out database processing operations. An aggregation system (30) is in communication with the general purpose computer (20). The aggregation system (30) includes a memory (32) for storing, for each of a plurality of data values, a base value and one or more incremental updates. An aggregation processor (34) forming part of the aggregation system (30) receives the incremental updates from the general purpose computer (20), writes the incremental updates into the memory (32), and controls the aggregation of base values and incremental updates stored in said memory (32) to read the data values. In this manner data values are updated and read without the use of locking operations.

Description:
This is a continuation of application Ser. No. 07/630,557 filed Dec. 20, 1990 now abandoned. 
    
    
     RELATED APPLICATIONS 
     The following U.S. patent-applications are assigned to the assignee hereof and contain subject matter related to the subject matter of the present application: 
     1. U.S. patent application Ser. No. 07/363,298 entitled &#34;Architecture For A Storage Access System For High Performance Computer Systems&#34; filed for E. J. Gausman, K. S. Grewal, and G. E. Herman on Jun. 6, 1989, now U.S. Pat. No. 5,230,073 issued Jul. 20, 1993, which is a continuation of U.S. application Ser. No. 06/888,733 filed on Jul. 21, 1986, now abandoned; and 
     2. U.S. patent application Ser. No. 07/253,240, entitled &#34;A High Performance VLSI Data Filter&#34; filed for Gary Herman, K. C. Lee, and Takako Matoba on Oct. 4, 1988 now U.S. Pat. No. 5,050,075, issued Sep. 17, 1991. 
     3. U.S. patent application Ser. No. 07/630,556 now U.S. Pat. No. 5,263,156, issued Nov. 16, 1993, entitled &#34;Optimistic Concurrency Control Certification Using Hardware Filtering&#34; filed for William H. Mansfield and Thomas F. Bowen on even date herewith. 
     The contents of the above-identified related applications are incorporated herein by reference. 
     FIELD OF THE INVENTION 
     The present invention relates to processing of database transactions and specifically to a system and method for reading and updating data values. 
     BACKGROUND OF THE INVENTION 
     A database is defined as a collection of data values, organized according to a data model and accessed by carrying out a transaction which comprises one or more database operations. While the inventive system and method applies to a variety of data models, the invention is best illustrated using a relational database model. 
     In a relational database, which is also known as a relation, data is organized in columns. Each column comprises one attribute of the relation. Each column or attribute of a relation has a domain which comprises the data values in that column. One row of a relation which comprises one value from each attribute is known as a tuple or a record. 
     Numeric data is stored in a database as integer values or floating point numbers. A series of updates of a numeric data item are commutative if the order of application of updates does not alter the final result. That is 
     
         A+B+C+D=A+D+C+B 
    
     To access a relational database, a select operation may be utilized. In the case of a relational database, a select operation generally refers to selecting those tuples of a relation R whose attribute A is equal to a specified constant C. This select operation is denoted R.A=C or S R .A=C. Operators other than equal such as &#34;greater than&#34;, &#34;less than&#34; or &#34;not equal&#34; are allowed, as are combinations of operations including AND and OR operations to build up complex selections. 
     A transaction is defined as a program that includes database operations such as the above-described select operation. If the database operations in a transaction only retrieve data, the transaction is called a read-only transaction. The collection of tuples obtained by applying a select operation against a database is known as a transaction read-set. The set of tuples which is modified, added, or deleted in a transaction is known as a transaction write-set. A transaction is an atomic unit of work that is either completed in its entirety or not done at all. A begin operation marks the beginning of transaction execution. An end operation marks the end-execution point in the transaction and indicates that database changes (if any) are ready for permanent application to the database. A commit operation propagates a transaction write-set to the database. A correct execution of a transaction must take the database from one consistent state to another. 
     Concurrency control is a mechanism that guarantees that the database state is always consistent. It also guarantees that an atomic transaction receives data from and applies data to a consistent database from the transaction&#39;s perspective. There are several techniques, for performing concurrency control operations including locking, time-stamp ordering, multiversion and optimistic schemes. 
     When a locking scheme is used for concurrency control, a specific transaction involving certain tuples or records locks these records against access by other transactions until the specific transaction is completed by propagating the specific transaction&#39;s write-set into the database. In this manner, it is insured that the specific transaction takes the database from one consistent state to another consistent state, as no other transaction can gain access to the locked records while the specific transaction is performed. However, a shortcoming of the locking scheme is that there are serialization delays as transactions queue up to gain access to certain records. Such serialization delays are especially severe when certain records are &#34;hot-spots&#34;, i.e., records which are utilized very frequently in transactions. Standard locking methods for concurrency control require the locking of records from the time of the first read operation of a transaction until the commit operation for changes in the database is complete. 
     Prior work as referenced below has been done to try to reduce the time that locks are held for individual records or other data items during a transaction. This prior work relates to aggregate records. Aggregate records are records which are semantically dependent on other records in the database. Therefore, aggregate data records are read and written more frequently than other records because they are modified every time one of their dependent records is modified. The prior art techniques for reducing locking time take advantage of the fact that most applications need to first read the aggregate record to verify that a particular data value in the record is within a specific range, and then if so, to modify the particular data value in the aggregate record. 
     In IBM&#39;s Fast Path Method (see, e.g., D. Gawlick &#34;Processing Hot Spots in High Performance Systems&#34;, Proceedings of Spring COMP CON &#39;85 - 30th IEEE Computer Society International Conference 1985) two special primitives are provided, verify and modify. Transactions verify a particular data value in an aggregate record by issuing verify operations which specify an aggregate record and a predetermined range for the particular data value as arguments. Transactions change a particular data value in an aggregate record by issuing modify operations which specify a particular data value in an aggregate record to be modified and a delta function. The delta function describes a change in the particular data value that can be applied atomically. 
     Verifies and modifies are saved in a work to do list. When a transaction commits, its work to do list is processed. During the first phase of commit processing, records accessed by verifies and modifies in the work to do list are locked. For each verify, a particular data value in an aggregate record is read and compared to the range contained in the argument of the verify operation. If the particular data value in the aggregate record is within the range, the verify succeeds, otherwise it fails. Any transaction having a failing verify is aborted. If all verifies succeed, the transaction moves to the second commit phase in which the modifies are applied to aggregate records in the database. After all the modifies are completed the accessed records are unlocked. 
     The above described Fast Path concurrency control method for aggregate records is an improvement over the standard concurrency control method used for non-aggregate records in that the period for which locks are held is shortened. As indicated above, standard methods require the locking of records from the time of the first read operation until the end of the commit operation. The Fast Path concurrency control method described above requires locking only during commit operations, which as indicated above comprise verify and modify operations. Shortening the locking duration improves concurrency control performance because other concurrent transactions needing to access the same aggregate records are less likely to have to wait for them to be unlocked. 
     An alternative technique for shortening the locking time of aggregate records is known as Reuter&#39;s Method (see A. Reuter &#34;Concurrency on High Traffic Data Elements&#34; ACM Symposium on Principles of Data Base Systems, March 1982). Whereas the Fast Path method does not make use of the verify and modify operations until commit time, Reuter&#39;s method makes use of verify and modify operations when they are issued. In accordance with Reuter&#39;s method, for each aggregate record in the database, the database management system stores two values to indicate the possible high and low values a particular data value in an aggregate record could obtain. The high and low values are based on all possible combinations of successes or failures of the current uncommitted transactions which have issued modifies on the aggregate record. 
     Consider the case where a new transaction comes along and issues a verify on a specific aggregate record. The previously stored high and low values for the specific aggregate record are used to determine whether or not a particular data value in the specific aggregate record falls within the acceptable range contained in the argument of the verify operation issued by the new transaction. If the particular data value of the specific record is not within the acceptable range as indicated by the high and low values, the new transaction can abort immediately, rather than waiting until commit time, thereby avoiding unproductive work. If the particular data value of the specific aggregate record is acceptable to the verify issued by the new transaction, the new transaction can continue. Thus, the new transaction issues a modify and the database management system checks to see if accepting the modification would change the high and low value of the specific aggregate record in such a way that the result of a verify operation of a previous transaction which is not yet committed would change from true to false. If so, the modification proposed by the new transaction is rejected and the new transaction can abort immediately. Otherwise, the new transaction can continue and the aggregate record to be modified is locked. The foregoing combination of verify and modify actions insures that transactions whose verify or modify operations fail can abort early, thus avoiding the unnecessary locking of aggregate records. 
     Both the Fast Path technique and the Reuter&#39;s technique discussed above reduce the locking time of records but do not eliminate locking. 
     In view of the foregoing, it is an object of the present invention to provide a system and method for avoiding locking altogether while allowing concurrent updates to and reads of aggregate records or other aggregate data items such as numeric summary data items. 
     SUMMARY OF THE INVENTION 
     The present invention relates to a system and method for maintaining the consistency of data values such as aggregate numeric data values while allowing concurrent updates without the use of locking operations. To avoid database locking operations, an update of a data value in a database is transformed from an update operation that changes a unique value in the database to a logging operation that records incremental changes in a data value. A database read operation is correspondingly transformed from a read of a single value, to a selection of a base value and the aggregation of all the incremental changes in the log associated with the selected base value. Since logging operations avoid the locking of the database records for extended periods of time, processing serialization delays caused by lock contention during updates are avoided and greater levels of concurrency are achieved. 
     In accordance with an illustrative embodiment of the invention, a data processing system comprises a general purpose computer including a database management system for executing transactions on a database. Associated with the general purpose computer is an aggregation system. The aggregation system enables the general purpose computer to update and read certain data values such as numeric summary data values which are updated frequently (i.e. hotspot data values) with minimal or no use of locking operations. 
     The aggregation system comprises memory means which stores two relations. The first relation comprises for each of a plurality of data values a base value and a time-stamp indicating when the base value was computed. The second relation is a log of the incremental updates to the base values in the first relation. Thus, for each data value, the second relation comprises one or more incremental updates and a time-stamp for each incremental update. 
     The aggregation system also comprises a processor. The processor is in communication with the general purpose computer. The aggregation system processor performs update operations by receiving the incremental updates from the general purpose computer and writing them into the log relation maintained in the memory means. 
     The aggregation system processor also receives requests to read data values from the general purpose computer and returns the read data values to the general purpose computer. To read a data value such as a numeric summary data value, the aggregation system includes selection means for selecting from the first relation a base value and for selecting from the second relation the incremental updates associated with the base value. The incremental updates which are selected have a time-stamp which is later than the time-stamp of the selected base value. The read operation then involves aggregating the selected base value and incremental updates. The aggregated value is then returned to the computer to respond to the read request. In addition, periodically the base values in the first relation stored in the aggregation system memory means are updated by aggregating the base values with the associated incremental updates to form new base values. A new base value and a time-stamp replace a corresponding old base value and its time-stamp in the first relation. Incremental updates incorporated into new base values may be deleted from the log operation at any time as the use of time-stamps prevents these incremental updates from impacting subsequent aggregation operations. 
     In an illustrative embodiment of the invention, the selection means comprises a transmitter for cyclically broadcasting the first and second relations in the form of a bit stream over a communication channel and a filter subsystem for selecting from the broadcast bit stream the base value and appropriate incremental updates. 
     The selected base value and incremental updates are aggregated either locally at the filter subsystem or by the aggregation system processor and then transmitted to the general purpose computer. As indicated above, the aggregation system memory may also be updated by substituting for the original base value a new base value which is equal to the recently computed aggregate value and by deleting all of the incremental updates included in the new base value. 
     Through use of the aggregation system described above, a data value is updated and read in a consistent fashion without the use of locking operations. 
    
    
     BRIEF DESCRIPTION OF THE DRAWING 
     FIG. 1 schematically illustrates a data processing system including an aggregation system for performing database updates and reads without locking operations in accordance with the present invention. 
     FIG. 2 illustrates one embodiment of an aggregation system for use in the data processing system of FIG. 1. 
     FIG. 3 and FIG. 4 illustrate relations utilized by the aggregation system of FIG. 1 to perform database updates. 
    
    
     DETAILED DESCRIPTION OF THE INVENTION 
     A database transaction processing system 10 which performs a variety of database transactions including consistently updating and reading data values such as aggregate numeric data values is schematically illustrated in FIG. 1. The transaction processing system 10 comprises a general purpose computer 20 for executing database transactions including database read and write operations. The general purpose computer 20 is in communication with the aggregation system 30 which performs updates and reads of data values using aggregation techniques in accordance with the present invention. In particular, the aggregation system 30 consistently performs updates and reads of summary numeric data values without the use of locking operations. To perform data value update operations, the aggregation system 30 receives and stores incremental updates of data values from the general purpose computer 20. The incremental updates represent incremental changes in the data values. Therefore, to compute the current value of a numeric summary value, incremental updates associated with the numeric summary value and having a time stamp later than the time-stamp of the base value associated with the numeric summary value are aggregated with the base value. The aggregate system 30 also aggregates incremental updates for various data values to read these data values in response to read requests from the general purpose computer 20. In other words, the aggregation system arithmetically sums the incremental updates having a time-stamp which is later than the time-stamp of the selected base value with the selected base value to generate an aggregated value in response to the read request. 
     FIG. 2 illustrates the aggregation system 30 in greater detail. As shown in FIG. 2, the aggregation system 30 comprises a memory 32, a processor 34, and a selection system 40. 
     A numeric data value N i  which is consistently maintained by the aggregation system 30 is split into a static base value base i  and a series of incremental updates δ ij . The base value base i  is associated with a time-stamp t i  which is the time at which the base value was computed. A time-stamp t ij  is also associated with each incremental update δ ij  which indicates when the incremental update δ ij  was received in the aggregation system 30 from the general purpose computer 20 and logged in the memory 32. Illustratively, the time-stamps t i  and t ij  are computed by a clock 35 associated with the processor 34. 
     Thus, a numeric data value N i  may be represented as 
     
         N.sub.i =base.sub.i +Σδ.sub.ij 
    
     where the δ ij  represent all increments associated with N i  since the time t i  that base i  was computed. 
     In a relational database implementation of the present invention, two relations are stored in the memory 32 of the aggregation system 30. The first relation is a base relation which stores the base values base i  of the data values N i  and the associated time-stamps t i . The second relation is a log relation which stores the incremental updates δ ij  and the associated time-stamps t ij . 
     FIG. 3 schematically illustrates the base relation stored in the memory 32. This base relation includes three attributes: value identification (i), base value (base i ) and time-stamp (t i ). 
     FIG. 4 schematically illustrates the log relations stored in the memory 32. This relation includes three attributes: item identification (i), delta value (δ ij ) and time-stamp (t ij ). 
     To read a numeric data value N i , the following algorithm may be utilized. 
     1. select the base value base i  and the time-stamp t i  from the base relation; 
     2. select from the log relation the increments δ ij  having a time-stamp t ij  which is later than t i  ; and 
     3. compute N i  =base i  +Σδ ij . 
     Thus, to read a data value such as N 3  the base value base 3  is selected from the base relation of FIG. 3 and the incremental updates δ 31 , δ 32  and δ 33  are selected from the log relation of FIG. 4 assuming that the time-stamps t 31 , t 32  and t 33  of the incremental updates are later than the time-stamp t 3  of the base value. 
     Typically, a value N i  is computed in response to a request received at the aggregation system processor 34 from the general purpose computer 20. Thus, once N i  is computed it is returned to the general purpose computer 20 from the aggregation system processor 34. In addition, periodically the original value base i  and time-stamp t i  in the memory 32 is replaced by the computed value N i  which becomes a new base value and a new time-stamp t i . All increments δ ij  aggregated into the new base value may be deleted from the log relation. The periodic updates of the base values may or may not be in response to a request issued by the general purpose computer 20. 
     The aggregation system 30 of FIG. 2 may be utilized to carry out the above-described aggregation algorithm as follows. The selection system 40 of the aggregation system 30 makes use of a unique memory access system for gaining access to the log and base relations stored in the memory 32. This memory access system is described in detail in the above-identified U.S. patent application Ser. No. 07/363,298, now U.S. Pat. No. 5,230,073 issued Jul. 20, 1993. In particular, the contents of the base and log relations stored in the memory 32 are cyclically broadcast by the transmitter unit 41 over the downstream communications channel 42. Located along the channel 42 are a plurality of filter subsystems 43-1, 43-2, . . . , 43-N. The filter subsystems 43 serve as &#34;listening processors&#34;. Specifically, the filter subsystems listen to the bit stream broadcast via the channel 42 and filter out certain selected information in response to instructions received from the aggregation processor 34. One embodiment of a filter subsystem 43 is disclosed in the above-identified U.S. patent application Ser. No. 07/363,298, now U.S. Pat. No. 5,230,073 issued Jul. 20, 1993. A higher speed embodiment of a filter subsystem is disclosed in the above-identified U.S. patent application Ser. No. 07/253,240, now U.S. Pat. No. 5,050,075 issued Sep. 17, 1991. Generally, each filter subsystem 43 comprises a high speed comparator as well as some local processing capability. 
     To execute a read request, the aggregation processor 34 transmits via the channel 44 to one of the filter subsystems 43 instructions to select from the cyclically broadcast data stream a certain base value base i  and corresponding incremental updates δ ij . In response to these instructions, the filter subsystem 43 selects the appropriate data values from the broadcast data stream. For example, to retrieve the tuple containing a base value base i  from the base relation, an instruction containing the value identification i is sent to a specific filter subsystem 43. The specific filter system 43 then performs a select operation by comparing the value identification i with all of the tuples in the base relation as they are broadcast over the channel 42 to pick out the tuple containing the value identification i and thus retrieve base i  and its associated time-stamp t i . Similarly, to retrieve the incremental updates δ ij  from the log relation, the filter system 43 looks at the broadcast bit stream to select tuples having the value identification i and a time-stamp t ij  which is later than t i . The updated data value N i  may be computed locally at a filter subsystem 43, in which case N i  is transmitted back to the aggregation processor 34 via channel 44. Alternatively, the selected base value and incremental updates are transmitted back to the aggregation processor 34 for aggregation. Once N i  is computed, it is sent to the general purpose computer 20. In addition, periodically, the base values in the base relation are updated and the incremental updates incorporated into the updated base values are removed from the log relation. 
     Finally, the above-described embodiments of the invention are intended to be illustrative only. Numerous alternative embodiments may be devised by those skilled in the art without departing from the spirit and scope of the following claims.