High speed method for flushing data buffers and updating database structure control information

A method is provided in a multi-processing system where multiple user programs can operate concurrently and simultaneously to access a series of database access methods which hold multiple buffer units. Thus, there can operate simultaneously a series of Write operations to the buffer units which copy the buffer information into specialized file structures followed by a parallel series of Input/Output completion operations which test the results of the copy operations followed by parallel and simultaneous writing to the file structures of the storage control information necessary to maintain their physical integrity.

FIELD OF THE INVENTION

This disclosure relates to multi-processing and multi-programming environments which require the flushing of data buffers and updating of the database structure control information.

BACKGROUND OF THE INVENTION

It is recognized that modern computer and communication systems continuously utilize huge mounts of data. Often as a result of this, the data management and data storage operations have been seen to become significant technical issues. Multi-programing and multi-processing operations which are typical of complex systems have large data requirements which require the use of massive amounts of storage.

In such multi-processing and multi-programing type systems, the data is generally stored on a storage media, such as magnetic disk, as blocks of data. Subsequently, the data is often read from the storage media into a temporary memory such as a cache or buffer which might consist of Random Access Memory. After this, it can be accessed by user application programs. One of the chief problems involved has to do with the updating of the database structure control information and flushing the data buffers of stale information. It is desirable that each of these updates will occur utilizing the minimal or least amount of time by taking advantage of the computer systems ability to run asynchronous processes and to overlap the Input/Output operations.

The optimum situation for the taking place of Input/Output operations is that they take place while no user applications are actively accessing that particular data within the database. Copies of the data will exist both within the applications and within the database. This is done to maintain both physical and referential data integrity. It is quite consistently necessary to provide updates to the database's buffers, and rather than maintaining control as a single process, or initiating multiple new independent processes to perform the updates, it is possible to use local environments that can access shared data and be so used to take advantage of the user application infrastructures that are already present. These types of procedures can, in effect, run on top of the user programs.

Prior systems which flushed data buffers and updated the database structures operated on a relatively slow serial basis. This type of mechanism was responsible for determining which of the structures had data buffers to be flushed, then writing to the disk, then testing for Input/Output completion, then writing the control information, then restarting the other applications. Due to the fact that this was a serialized process, it was not only inefficient, but did not take advantage of the performance that is inherently possible in multi-processor technology.

The presently described method operates to eliminate the relatively slow serial process and provide a fast high-speed flushing of data buffers and operations for updating the database structure control information. The described method and system is completely asynchronous. Rather than simply waiting, the user tasks are utilized concurrently in time as workers and as such, user tasks participate in the process of independently writing the data buffers, then testing for I/O completions, and finally updating the structure control information.

In the presently-described method, various tasks may enter the process and engage at any particular phase or change roles at any time. These processes are First-In, First-Out (FIFO) in nature. As an example, the task initiating the WRITES for a given set of data buffers is not necessarily the task that assures their completion.

Further, coordination is achieved by selecting a single process to perform only those housekeeping functions that absolutely require serialization. The use of shared data is limited only to those instances where it is required to drive the process forward. Asynchrony is further assured by only restricting access, via a software data lock, to those instances where the shared variables require alteration. This mechanism limits serialization to the absolute minimum necessary to ensure integrity. The previously used systems and methods for a given database configuration, did not significantly benefit from an increase in performance beyond that which could be afforded only by the addition of a second processor. Thus, the operations did not appropriately scale in proportion to the number of processors added. However, the presently described system and method has the advantage of being about 30% faster for two and three processor configurations, and of continuing to provide increased throughput even as the workload increases and additional processors are added.

SUMMARY OF THE INVENTION

The present method describes an asynchronous mechanism for distributing the operating workload of flushing data buffers over N number of processes. One distinct advantage involved is that of making use of the user tasks already running on the computer system, so that the time required to accomplish a given task is minimized in two specialized ways.

Firstly, all of the write operations are initiated: this indicates that there is a high probability that the first set of Write operations will have finished even before the last set of Write operations have been initiated. Then, in a similar fashion, the requisite testing for the correct completion of each of these operations is handled. That is to say, by using the same orderly sequence as in the initiation sequence then, the majority, if not all, of the Writes will have completed by the time the completion information has been examined. As a result the probability of having to wait for some operation to complete, is extremely low.

Secondly, the workload is spread across the active update users of the database. This allows the multi-processor system to also take advantage of its ability to perform multiple tasks simultaneously. A concomitant advantage is that the use of the existing application tasks does not burden the system with the overhead of initiation, and the management thereof with specialized tasks to perform these functions. Thus, the method takes full advantage of the parallelism inherent in multi-processor systems.

DESCRIPTION OF PREFERRED EMBODIMENT

As was previously discussed, earlier systems and methods used a serial or sequential step-by-step process for determining which structures had data buffers to flush, after which there was a writing of the buffers and then testing for Input/Output completion, then writing the control information and then re-starting other applications. This type of serialized process was rather inefficient and did not take advantage of the performance available inherently in multi-processor technology.

The present method is completely asynchronous and rather than waiting for serial tasks to be concatenated in sequence, the present system allows the user tasks to be concurrently utilized as worker and to participate in the process of independently writing data buffers, testing for their I/O completions, and updating the structure control information in the database files.

Now referring to FIG. 1 , there is seen a multi-processor system indicating, for example, three central processing units, designated 11 , 12 and 13 , which are all connected to a common memory system 16 .

Then, working through the memory system, it is seen that the first processor 11 is connected to the database engines 21 , 22 , 23 which reside in the Master Control Program 20 . The database engine 21 then communicates back and forth to the data file structures indicated therein as 31 a and 31 b.

Likewise, the processor 12 communicates through the common memory 16 to the database engines 22 , 21 , 23 which then intercommunicates with the data file structures 32 a and 32 b.

The processor 13 communicates through the common memory to the database engines 23 , 21 , 22 of the Master Control Program and the database engine 23 then interconnects and intercommunicates with the database files 33 a and 33 b.

Referring to FIG. 2 , there is seen the Master Control Program 20 (MCP) which interacts with, and controls, a number of individual units some of which are shown herein as the database engine 2 N which interconnects with various user-application programs shown as 41 , 42 , and 43 .

The database engine actually includes a set of access and control routines which enable communication between the user application programs and the database engine, and also from there to a set of the database file structures indicated as 30 a , 30 b and 30 c.

The database engine 2 N is seen having a buffer pool 50 where multiple buffer units are available for the uses of the various user application programs and which can be used to copy data from specialized chosen buffers into specialized records on the database file structures.

The present system can be elucidated by the Unisys Corporation Data Management System II (DMS II), which is described in Unisys Corporation Document No. 8807 6625-000, and dated September 1997, as published by the Unisys Corporation Publications Division in Plymouth, Mich.

The following description is based on the items which have been defined in the previously-listed Glossary items which define the various concepts and software programs which are involved.

The procedure CONTROLPOINTDUMP had been used to both initiate and monitor completion of all control point Input/Output (I/O) activity. This mechanism was designed to run entirely on the stack of the last User program to leave the transaction state. As was previously indicated, the earlier control point mechanism actually created a bottleneck because it was single-threaded , while the present system operates on a parallel operating basis which can be considered multiple-threaded . The CONTROLPOINT, as used in the Data Management System of Unisys, is a feature that limits the amount of audit information that must be scanned by Halt/Load and the Abort/Recovery processes when recovering the database. The CONTROLPOINT is performed when the last user task leaves the transaction state, given that the requisite number of transactions have occurred or that other specialized activities are about to take place. The transaction state is the period in a user-language program, which occurs between a Begin transaction operation and an End transaction operation For all audited databases, the DMS II software allows an application program to logically group the update operations and to process this group as a single transaction. The start of the group transaction in signaled by a Begin transaction operation. The end of the group transaction is identified by a End transaction operation. However, while the program is between the Begin and the End transaction operation, the program and the database are considered to be in the transaction state .

A transaction involves one or more of the following: (i) the transfer of one message from a terminal or a host program to a receiving host program, where the processing is carried out by the receiving host program and the return of an answer to the sender; and (ii) a complete unit of work (X/Open). This can comprise many computational tasks involving data retrieval and communication. A typical transaction modifies the shared resources (iii) In data management, a sequence of operations grouped by a user program because the operations constitute a single logical change to the database. (iv) In the Screen Design Facility Plus (SDF Plus), it is the structure that performs the transfer of the message.

The efficiency and capabilities have been greatly improved by the present system because the CONTROLPOINT mechanism has been designed to take full advantage of the inherent capabilities of both multi-programming and multi-processing environments.

FIG. 3 is an illustration of a set of parallel arrays used for Data Control Block control words. A Data Control Block represents control information that pertains to a particular buffer. In order to utilize the system memory efficiently, Data Control Blocks are implemented as a series of parallel arrays, an seen in FIG. 3 . Each of these arrays is unique, in that it contains information of a particular type, for example, the Input/Output (I/O) results for all buffers are contained in an array called IORESULTS. A Data Control Block (DCB) is a variable (data item) that represents one word out of each of these parallel arrays: DCB N thus consists of USERCOUNTS (N), plus BUFFDESCS (N), IORESULTS (N), and so on. Active Data Control Blocks are organized in one-to-one correspondence with active buffers.

An seen in FIG. 3 , a particular DCB has a DCB number or cell number which is the index value of that particular DCB. Thus, the DCB at cell number 1 is seen to encompass three areas of the parallel arrays, and it is seen that the number 1 inside the upper half of the upper array indicates that a modification has taken place, while the lower part of the upper half gives the structure number. Here, the structure is a term that is analogous to a file. Each file that is directly managed by the DMS II Database Management System is assigned a number that is internal to the Database Management System. Note that the number three used above is for illustrative purposes and that a DCB is comprised of a larger number of parallel arrays and, further, that number does increase from time to time.

Referring to FIG. 4A , there is seen a set of steps which are called from the from local procedures which refers to the user application program steps just prior to entering into the database engine steps that are used in order to flush the buffers; which means that the data of the buffer has now been written to a disk file. This may only involve copying the data from the buffer into the disk file and leaving the data still residing and accessible in the buffer. Thus, the space occupied in the buffer by whatever has been flushed, may or may not become available for other uses. Thus, in step A, the first operation involves the general housekeeping duties and tasks such as setting the process control variables to their initial states in preparation for writing the buffers.

Then at step B, the system will build a list of buffers and structures involved. The various internal substeps of this operation are later shown in FIG. 4 B. At step C, the system will set-up pointers for the list.

At step D, the system will wake-up the helpers , which are the other user application programs that are waiting in database code for the control point to complete in order that they may resume their own work.

At step E, the system will enter into procedures that write the buffers and update specialized information that is used to maintain the physical integrity of the data (this is known as storage control information).

At step F, a decision block is operated to check whether the work has been finished. If the answer is NO , then the system must wait for the finished event. If the answer is YES , then the system will continue to finish the CONTROLPOINT which is performed when the last process leaves the transaction state.

Referring to FIG. 4 B 1 , which is a more detailed set of steps of section B, which involves building a list of buffers and structures. Thus, in FIG. 4 B 1 , the first step involves step Bo, which involves building a CONTROLPOINTDUMP list.

Then at step B 1 , the system will initialize an array containing the starting indexes of the buffer list (CP_DUMP_HEAD).

At step B 2 , the system will search the array of DCB control words ( FIG. 3 ) to locate the DCB's marked as having been modified; then return the indexes of the DCBs.

At step B 3 , a decision point is reached to ask whether this is the end of the array. If the answer is YES , the system will exit. If the answer is NO , then the system goes to step B 4 to see if any other DCB's have been found, after which the system returns back to step B 2 .

At step B 4 , if another modified DCB has been found, then at step B 5 , the system will extract the structure number. Then at step B 6 , the system will query whether this structure has been seen before. If the answer is NO , then the system will increment the structure pointers. However, if the structure number has been seen before YES , then the procedure will go onto step B 8 of FIG. 4 B 2 .

It should be noted that the storage disks or database files of FIG. 1 ( 31 , 32 , 33 ) constitute a set of data structures which may be a series of files which follow a sequence of 1, 2, 3, . . . , N. Each of the structures shown in FIG. 1 , such as 31 a, b , and 32 a, b , etc., can be considered as a logical file. However, each of these logical files will consist of physical files which may be considered as files 1, 2, 3, . . . , N physical files.

As will be described later in FIGS. 4D , 4 G, step zero is designated as a sequence of operations where there is a writing to the buffers for all the structures or sections involved. Then step 1 , FIG. 4F , is a set of operations where there is a checkout for successful I/O completion for all the structures and sections, that is to say, to see that the buffers were copied into the appropriate file structures.

It should be noted that there is a parallel operation here in that step zero, (0) FIGS. 4D , 4 G, there are many write operations going to different buffers and operating in parallel. Likewise, in step 1 FIGS. 4F , 4 G, there is a multiple set of operations to check for successful I/O completion all operating in parallel.

Then step 2 , FIG. 4G , is the performance of the storage control Writes for all of the structures. These also occur as a multiple set of asynchronous operations.

Thus, step zero (0) FIGS. 4D , 4 E, and one (1) FIG. 4F involves user data while step 2 FIG. 4G , involves the storage control information used to manage the user data.

Now referring to FIG. 41 there is seen a flow chart of the step 1 operation where at step F 1 there is a decision block to query whether or not this is still step zero (this limits the execution of steps F 1 thru F 7 to being performed once by a single task) and whether the flag is true. If the answer here is yes then at step F 2 the next procedure step number is set to step number 1 . After this at step F 3 , there is a resetting of the scan flag. And subsequently at step F 4 the procedure fills the array CP_DUMP_CURSECT (Current Section) with the value of hexadecimal F (all bits on) to indicate that the array cells are in their initialized state. At step F 5 the procedure is to set to minus one ( 1) for the value of MY_STR.

After this then step C 2 will use the ALGOL pointer expression to set up the beginning of the list of structures. This Pointer is designated CP_DUMP_HEAD_PTR.

Then at step C 3 the program will calculate the size and words of the CP_DUMP_HEAD (designated as CP_DUMP_HEAD_LOCATOR).

In step C 4 there will be provided a duplicate count of the sections per each structure; this sequence is designated CP_SECTION_CTR. By using a duplicate the original values can be preserved while the duplicate is consumed by this process.

At step C 5 the program will set the control point in progress flag to true.

Then upon completion of step C 5 the program will progress to step D, which involves the waking up of the helper tasks.

This progresses to step E where there is an entering of the procedure CP_DUMP_BUFFS with a parameter of true to indicate that this system is in the controller stack. The helper tasks enter with a value of false .

Then the sequence of E continues by reference to FIG. 4 D. Now referring to FIG. 4D , which shows the steps of the E sequence. Here at step E 1 the program will get the control point lock (CPT_LOCK), after which at step E 2 there is a decision block inquiring if there is work to be done by testing whether CP_STRSTO_PROCESS is greater than zero. If the answer is no then the procedure relinquishes the control point lock, exits, and resumes its previous state of waiting for the control point activities to complete.

At step E 2 if the work to be done decision signifies yes then step E 3 will occur where there is a decrementation of the total structures to be processed. The combination of steps E 2 and E 3 limits the number of workers to the number of files that must be written to: this reduces contention for the software lock CPT_LOCK. At step E 4 another decision block inquires as to whether the stack is the controlling stack. If the answer is Know then there is an incrementation of the worker count (step E 9 ). The logic sequence then proceeds to step E 5 , which is designated as step zero .

At step E 4 , if the answer is yes , then this is the controlling stack and the worker count (step E 9 ) is not incremented. Next, a sequence of operations will occur which is designated as step zero.

This step zero will be seen to encompass steps E 5 , E 6 , E 7 , and E 8 thru E 11 .

Thus at step E 5 , a query is done to see whether this is step zero, and if the answer is yes then at step E 6 there is executed a procedure called CONTROLPOINTDUMPSKAN resulting in the acquisition of either the structure number or the combination of the structure and section numbers. After this at step E 7 there is the giving up of the control point lock which leads to step E 8 which continues on FIG. 4 E. Giving up the lock allows other stacks traversing this area of code to obtain their own unique value of structure or structure and section numbers by calling CONTROLPOINTDUMPSKAN.

In FIG. 4E there is shown a continuation of step zero where step E 8 leads to the decision block E 9 designated undefined. This is where the information returned from CONTROLPOINTDUMPSKAN is examined and determined to be either valid structure information or a value ( UNDEFINED ) that signifies that the list of structures has been exhausted.

If the answer is no then the procedure goes to step E 10 which is another decision block inquiring whether the structure is in use by the offline dump (software for backing up database structures). If the answer here is no then at step E 11 , there will be a calling of the procedure that will write the buffer for this particular data structure. The procedure is designated DUMPBUFFERS STR . Upon completion of DUMPBUFFERS, the CPT_LOCK is reacquired and the task returns to step E 5 (FIG. 4 D). Returning to step E 9 on the yes leg, it is seen that the next sequence of steps is defined by the step F which is shown subsequently in FIG. 4 F.

Returning to FIG. 4E , there is seen an indication of the helper stack. The helper stack relates to participation in step E but first starting at step H 1 there is a waiting for the control point to occur during and after which at step H 2 a decision block occurs as to whether the control point is still in progress. If the answer is no then step H 3 will return the system back to its normal work routines either without any need for buffer writes or after they have been completed.

If step H 2 is answered in the affirmative yes , then the sequence continues on to step E which was shown in FIG. 4 D.

Now referring to FIG. 4F there is seen a flow chart of the step 1 operation where at step F 1 there is a decision block to query whether or not this is still step zero (this limits the execution of steps F 1 thru F 7 to being performed once by a single task) and whether the flag is true. If the answer here is yes then at step F 2 the next procedure step number is set to number 1. After this at step F 3 , there is a resetting of the scan flag. And subsequently at step F 4 the procedure fills the array CP_DUMP_CURSECT (Current Section) with the value of hexadecimal F (all bits on) to indicate that the array cells are in their initialized state. At step F 5 the procedure is to set to minus one ( 1) the value of MY_STR.

At step F 6 there is a resetting of the scan pointer and at step F 7 there is a resetting DUMP_HEADLOCATOR. The above operations are performed in preparation to entering into the next phase. From this the procedure goes to step G which is seen in FIG. 4 G.

Referring back to step F 1 , if the answer is no then steps just described for the yes branch have been completed (they must execute once and only once for any given control point) and the procedure goes to step G which is seen in FIG. 4 G.

Referring to FIG. 4G the bracket G 1 is placed to indicate how a series of steps are done in sequence such as step zero, step 1 , step 2 , and then the setting made to step 3 . The logic is essentially the same as already described for FIGS. 4D , 42 , and 4 F. The difference lies in the parameters passed to DUMPBUFFERS at E 11 . There are three types of calls into DUMPBUFFERS: the first will cause the data buffers to be written, the second will cause the I/O completion activities to occur, and the third causes the Storage Control information to be written. From the setting of CP_STEP to 3 the next sequence item is designated as G 2 which is a decision block to query am I with the controlling stack If the answer is yes then the program proceeds to step J which is seen in FIG. H.

At step G 2 if the answer is non then the sequence proceeds to step G 3 which involves decrementing the worker count after which there is decision block G 4 to question whether this is the last worker. If the answer is yes then at step G 6 another decision block inquires - - - has a finish event occurred - - - and if this is the case yes , the program will exit and return to waiting (the same as for G 5 described below) for the controlling stack to finish the remainder of the Control Point activities. Now returning back to step G 4 querying whether this is the last worker, if the answer is no then the sequence proceeds to step G 5 which is a return to its state of waiting for all of the control point activities to finish.

Then returning to step G 6 on the decision as to whether the finish event has occurred, if the answer is no then at step G 7 the program will set the worker active flag to false. After this at step G 8 , there will be caused the finish event and at step G 9 there is a return to Control Point waiting as just described. For all intents and purposes these are returns to H 1 on FIG. 4 E.

Referring to FIG. 4H there is seen the J sequence which derives from step G 2 in FIG. 4 G.

Now that J sequence in FIG. 43 starts at step J 1 where the system will get a control point lock (CP_LOCK). After this at step J 2 there in a decision block with the inquiry if the workers are still active and whether or not the workers finished event has not yet occurred.

If the answer to step J 2 is yes then at step J 3 the system will give up the control point lock and at step J 4 it will wait for the workers to finish.

Returning to step J 2 if the answer is no then at step J 5 the system will give up the control point lock and at step J 6 it will set the control point in progress flag to false.

Then at step J 7 the controlling stack will continue on to finish the remainder of the Control Point work and, just prior to finishing will awaken the tasks that had gone back to waiting after helping with the buffer flushing. At this point, all tasks are able to proceed with normal ordinary processing operations other than that of flushing the buffers and writing structure control information associated with Control Points.

Now referring to FIG. 4 I 1 , there is seen the procedure called CONTROLPOINTDUMPSCAN . This procedure comes from and is referenced to the step E 6 of FIG. 4 D and illustrates the step procedures involved obtaining structure and section numbers from the previously built lists.

In FIG. 4 I 1 , starting at step K 1 , there is seen a decision block which has the query of CP_DUMP_HEAD_LOCATOR as to whether this is greater than zero. This tests whether there are entries remaining in the list. If the answer is YES , then the sequence proceeds to step K 2 which is another decision block which inquires whether the structure number is greater than zero. If the answer is YES , then this is not the very first scan and the procedure goes to step K 3 to another decision block which has the query of CP_DUMP_CURSECT, which involves seeking to know whether this structure is one that is sectioned (made up of multiple physical files).

If the answer at K 3 is yes, then the sequence proceeds to K 4 in order to increment the local value of the section number for this structure, this variable being designated CP_MYSECT.

The next step is step K 6 which involves a decision block to inquire whether CP_SECT is less than the maximum for this structure. If the answer here is YES , then the sequence proceeds to step K 7 where it will save the section number for use by the next worker when it reaches step K 3 . This involves the array designated CP_DUMP_CURSECT which is indexed by the structure number.

After this at step K 8 , the program will exit and return the value of the structure number and the section number, and then return to step E 6 of FIG. 4 D.

The present sequence of events involved a series of YES , starting from step E 6 in order to arrive back to the finality of E 6 . However, it is necessary to look at the other choice involving NO .

At step K 1 , if the answer is NO , then the sequence proceeds to exit and return the value of undefined at step K 12 , which then returns to step E 6 at FIG. 4 D.

At step K 1 , if the answer is NO , then the sequence proceeds to step K 9 where there is a scan for structures with buffers to write. This then proceeds to step K 10 which is a decision block to question whether the list of structures have been exhausted or not. If the answer is NO , then the sequence proceeds to step K c on FIG. 4 I 2 .

At step K 10 , if the answer is YES , that is to say the list of structures are exhausted, then at step K 11 there is a setting of the scan flag to TRUE , after which there is an exit and return value of undefined at step K 12 , which then returns to step E 6 at FIG. 4 D.

Now returning to step K 6 of FIG. 4 I 1 , where the decision block has a NO response, then the sequence proceeds to step K 13 in order to advance to the next cell of the pointer associated with CP_DUMP_HEAD so that the next scan operation will proceed to the next structure. After this at step K 14 , the sequence will convert the scanner index to the structure number and get the value of the index into the array designated CP_DUMP_HEAD.

From step K 14 to step K 15 there appears a decision block to query said index at to whether the list is exhausted. If the answer is NO , then the sequence returns back to step K 9 . If the answer at step K 15 is YES (list exhausted), then the procedure goes to step K 16 to set the scan flag to TRUE .

Now, referring to FIG. 4 I 2 , which continues from the step K c from FIG. 4 I 1 .

At FIG. 4 I 2 , the first step is designated K c1 which involves converting the scanner index to the structure number. This then proceeds to step K c2 which involves a decision block as to whether the structure is sectioned or not. If this is a YES , then at step K c3 the procedure will take the initialized section number and set it to zero, after which at step K c4 there will be a return of the structure number and the section number during which the sequence proceeds to step E 6 of FIG. 4 D.

Returning to step K c2 of FIG. 4 I 2 , there is the query of whether this structure is sectioned or not Here, if the answer is NO , the sequence proceeds to step K c5 to advance the CP_DUMP_HEAD pointer. After step K c5 , the next step K c6 is a decision block to query whether the structure list is exhausted or not. If the answer is NO , then at step K c8 there will be a return of the structure number to step E 6 . However, if the answer is YES , then the procedure goes to step K c7 and will set the scan flag to TRUE .

In the earlier version of the Unisys DMS II system, the CP_SOC_BUFF array was a shared buffer used to handle storage control rights by a procedure designated STORAGEOPENCLOSE during the CONTROLPOINTS. In the earlier implementation, this was declared at the D 2 level. The designation D 2 , D 3 . . . D 8 level involves a lexical level, or lax level, which involves a number that indicates the relative level of an addressing space within the stack of an executing program. Here, (i) The lexical levels range from 0 through either 15 or up to 31 depending on the type of computer family. A lower lexical level indicates a more global addressing space. (ii) It is also a measure of the number of other blocks that a given block is nested within. The outer block of a program has a lex level of 2 or 3, depending on whether the program has a procedure heading. Each block has a lex level which is one higher than the block it is nested within.

The present system rather now declares an array in what is called the COMMONDBS in order that a private CP_SOC_BUFF exists for each individual structure or section (no need for buffer sharing). This allows concurrent Input/Output (I/O) activity for multiple structures and sections, during the time of performing storage control updates during a CONTROLPOINT. In the earlier operational system of the Unisys DMSII, the CONTROLPOINTDUMP procedure consisted of three modules, each of which was controlled by bits located in the parameter passed in when it was called (the OP word). This procedure was called twice when executing a CONTROLPOINT . The first call here resulted in executing modules one and two, while the second call executed modules two and three. Modules two and three then call ALL_DUMPBUFFERS, a total of three times: once to initiate the I/O's, once to wait for the I/O completions, and once to clean-up any remaining storage control information.

Now, quite contrarily, the new system and method divides this into three separate procedures, the first one, BUILDCOTROLDUMPLIST, and this is only executed by the controlling stack, that is to say, the last one out of the transaction state. This builds the list of structures having buffers that must be flushed during this particular control point. In addition, the first procedure counts the number of structures and the sections that will be involved in this particular CONTROLPOINT.

A section count is obtained from the array designated CP_DUMP_SECTINFO, which is indexed by structure number, and filled-in for sectioned structures by STROPENSTRUCTURE of SECTION_SB when the structure is first opened for use. This only applies to data sets since index sequential is the only set type currently allowed to be defined as XE , and they are actually a single physical file.

The second procedure, designated CONTROLPTDUMPSCAN, is called by all CONTROLPOINT participants in order to obtain the next available structure, and where appropriate, section numbers from the list that was built in the previous step. Logically, this occurs twice, once to write the buffers, and once to wait for the I/O completions. In actual practice, each of these activities occurs as many times as there are structures and sections to process.

Once the structure number (and the section number) is obtained, then the procedure ALL_DUMPBUFFERS can be called. The outer block DUMPBUFFERS for sectioned structures (data sets) will call specific sections when a CONTROLPOINT is in progress.

Actual test results have indicated that this configuration of the new method resulted in considerably better performance than having each caller issue a task and then have to wait for its own I/O completions.

The final step involved is to perform the storage control I/O's on the remaining structures and sections. The requisite structure/section information is then obtained by calling the program CPGETALLSTRNUMS.

The initial housekeeping (step A of FIG. 4A ) and control of the list building occurs in the procedure AUDITCONTROLPOINT. The actual CONTROLPOINT work as described above, is coordinated with a new procedure designated: CP_DUMPBUFFS. It's various stages are coordinated by CP_SETP; in turn, each stage signals its completion by returning the value UNDEFINED, rather than a structure-number/section-number pair.

The controlling stack enters CP_DUMPBUFFS with a parameter value of TRUE, and all helper stacks enter with FALSE. The parameter, called CTLSTACK, is used upon entry and exit to control information associated only with the helper stacks.

The procedure involves a new lock, CPT_LOCK, which coordinates access to the structure and section numbers. Further, in order to minimize contention, the number of stacks allowed to participate is limited to the number of structures and sections (CP_STRSTOPROCESS and CP_TOTAL_SECTS) which is obtained from the BUILDCONTROLPOINTDUMPLIST.

The helper stacks are awakened by executing a DMSCAUSE ( 0 ). Now, given that CPINPROGRESS in TRUE, then the helper stacks enter CP_DUMPBUFFS which then follows the steps described above. After the helper stacks exit, they go back to sleep and wait for the CONTROLPOINT to complete in the standard procedural fashion.

Described herein is an asynchronous mechanism which distributes the workload of flushing data buffers of the file structures by operating concurrently over a number of processes. A singular advantage is one of taking and using user tasks which are already running on a single or multiple processor computer system. Thus, the time required to accomplish the task of writing buffers and then flushing or copying them to structured data files is minimized in basically two ways.

Initially, all of the Write operations are initiated which means there is an exceedingly high probability that the first started Write operations will have finished even before the last set of Write operations have been initiated. Likewise, there is a series of tests in order to determine the correct completion of the operations as having been copied from buffers to the final repository locations in the structured files. Thus, likewise in a parallel sequential manner the I/O completions are handled similar to the sequence of initiations done by the Write operation. This reduces the probability of any necessity to wait for some operation to complete.

As a second advantage, the workload is spread across the active update users of the system, so this permits a multi-processor system to take advantage of the ability to perform multiple tasks simultaneously. Another advantage of the present system is that the use of existing application tasks does not burden the system with the overhead of initiation and management of specialized tasks. Thus, the present method enables full advantage of the parallelism of operations inherent in multi-processor computer systems.

While the preferred embodiment of the method has been described in specific detail, the invention may also be implemented in other similar ways and should be considered to be defined by the attached claims.