Method and apparatus for storing transactional information in persistent memory

A method and apparatus for storing transactional information in persistent memory. In one embodiment, the invention features a persistent volatile memory and an intermediary program in communication with the persistent volatile memory. The intermediary program receives transactional information and stores the information in the persistent volatile memory. A computer uses the intermediary program to enable the contents of the persistent volatile memory to remain unaltered during a failure of the computer. Additionally, the intermediary program may determine whether the transactional information meets a predetermined criteria before storing the information in the persistent volatile memory.

FIELD OF THE INVENTION

The invention relates generally to storing information and more specifically to storing transactional information in persistent memory on a computer.

BACKGROUND OF THE INVENTION

Information accessed by a computer system is often preserved for later retrieval by the computer system. In some circumstances, a computer system prevents access to a particular file stored on a disk when writing information (e.g., transactional information) to that file until such writing completes. This occurs to avoid corrupted transactions. For example, a database management system (DBMS) generally requires that all updates to files stored in non-volatile storage (e.g., a disk) are completed before that file is made available for access by an application.

In particular, a “database commit” is the final step in the successful completion of an update to a file made as a part of handling a transaction. For example, when a single transaction includes several steps, then all of the steps must be completed before the transaction is deemed successful and the database file is actually changed to reflect the transaction. When a transaction completes successfully, the changes to the file are said to be “committed”.

Since a read or write transaction requires a read from or a write to non-volatile storage (e.g., disk) and access to the disk is limited until the previous transaction is committed, access to the database file is slowed by the delays in committing the data. Consequently, the performance of the DBMS decreases.

One prior art solution to this problem is to store the information in a temporary log file in volatile cache memory so that the DBMS can commit the information to disk at a later time. Further, the performance decrease of the computer system when writing the information is typically reduced when the information is first written to the log file rather than non-volatile storage (e.g., disk). However, if the computer system failures (i.e., crashes) before the computer system updates the disk with the data updates stored in the volatile cache memory, all update information stored in the volatile cache memory is deleted and typically irretrievable.

Thus, there remains a need to store transactional information in a manner that allows the information to be available following a computer system failure without having to perform the same transaction again. Additionally, there remains a need to have the transactional information available after a system failure in a more efficient manner than currently available.

SUMMARY OF THE INVENTION

An object of the invention is to ensure availability of transactional information following a failure of a computer system without having to perform the same transaction again. Additionally, another object of the invention is to provide availability to the transactional information following a system failure in a more efficient manner than currently available. The invention features a persistent volatile memory and an intermediary program in communication with the persistent volatile memory. The computer uses the intermediary program to enable the contents of the persistent volatile memory to remain unaltered during a failure of the computer.

In one aspect, the invention features a method for storing transactional information in a computer. The method comprises the steps of: (a) receiving transactional information; (b) storing the particular transactional information in a persistent volatile memory on the computer; and (c) retrieving the transactional information after a computer failure by accessing the transactional information stored in the persistent volatile memory on the computer.

Additionally, the method may also comprise flushing the persistent volatile memory to a persistent mass storage device. In one embodiment, the flushing occurs when the transactional information stored in the persistent volatile memory exceeds some predetermined threshold. In another embodiment, the flushing occurs when a predefined amount of time has elapsed since the storage of the transactional information in the persistent volatile memory. In yet another embodiment, the flushing occurs when a program, such as the operating system of the computer, is not busy. Alternatively, the flushing occurs when a file is closed or when the computer is shut down.

In another aspect, the invention features a method for providing persistent mass storage of transactional information. The method comprises the steps of: (a) receiving transactional information; (b) determining whether the transactional information meets a predetermined criteria; and (c) storing the transactional information that meets the predetermined criteria in a persistent cache. In one embodiment, the transactional information comprises unbuffered writes to disk, which are writes requested by an application and in which notification to the application of the completion of the write is necessary. For example, an unbuffered write can include copying a file from one directory to another directory, backing up and/or updating a file, and initializing a file.

In still another aspect, the invention features a persistent volatile memory and an intermediary program. The intermediary program receives transactional information and stores the transactional information in the persistent volatile memory. The contents of the persistent volatile memory remain unaltered through a system failure. In one embodiment, the intermediary program is a filter driver module that identifies particular transactional information to store in the persistent volatile memory. In another embodiment, the invention includes a flushing thread to flush the contents of the persistent volatile memory to a persistent non-volatile memory.

DETAILED DESCRIPTION OF THE INVENTION

Referring toFIG. 1A, an embodiment of a computer4constructed in accordance with the invention is depicted. The computer4can be any personal computer (e.g.,286,386,486, Pentium, Pentium II, Macintosh computer), Windows™-based terminal (developed by Microsoft Corporation of Redmond, Wash.), Network Computer, wireless device, information appliance, RISC Power PC, X-device, workstation, mini computer, main frame computer, personal digital assistant, or other computing device that has a Windows™-based desktop and sufficient persistent mass storage.

The computer4includes a microprocessor6, a memory8for storing programs and/or data, an input/output (I/O) controller10, and a communications bus12allowing communication among these components. In one embodiment, the microprocessor6is a Pentium Classic/MMX CPU, developed by Intel Corporation of Austin, Tex., an AMD-K6 CPU, developed by AMD of Sunnyvale, Calif., and the like.

The computer4(i.e., the I/O controller10) is additionally in communication with a persistent mass storage22, such as a magnetic disk or magneto-optical drive. In one embodiment, the persistent mass storage22is an internal component of the computer4. In another embodiment, a persistent mass storage22′ (not shown) is an external component of the computer4. In particular, some computers4have redundant arrays of independent disks (RAID arrays) used as failure-tolerant persistent mass storage22. The computer4can also be in communication with a peripheral device (not shown), such as a mouse, printer, alphanumeric keyboard, and display. In some embodiments, the computer4also includes a network connection.

The memory8in such a computer4typically includes random-access memory (RAM)14, read-only memory (ROM)16, and non-volatile random-access memory (NVRAM)20. The RAM14typically contains one or more application programs18and an operating system24. Examples of the OS include, but are not limited to, Windows NT developed by Microsoft Corporation of Redmond, Wash., OS/2 developed by IBM Corporation of Armonk, N.Y., Netware developed by Novell, Incorporated of San Jose, Calif., and the like. In addition, the RAM14includes one or more intermediary programs28. In one embodiment and as described further below, an intermediary program28is a filter driver module that stores certain types of I/O transactional information in a persistent memory to ensure stability after a computer failure (e.g., from a computer crash).

In one embodiment, the RAM14is additionally partitioned into a volatile memory32and a persistent volatile memory36. The volatile memory32is directly accessible to the operating system24and is typically initialized or modified during a boot cycle of the computer4. The intermediary program28handles requests such as read request(s)38and/or write request(s)39from the operating system24which are directed to the persistent volatile memory36. In one embodiment, the persistent volatile memory36is a persistent cache memory. In a further embodiment, the persistent volatile memory36includes a log file in persistent cache memory (i.e., log cache).

The ROM16includes a modified basic input-output system (BIOS)40that handles the boot process of the computer4. The modified BIOS40prevents the operating system24from directly accessing the contents of the persistent volatile memory36. The persistent volatile memory36is not directly accessible to the operating system24and therefore is not modified or initialized by the operating system24during a boot cycle. In one embodiment, configuration information44regarding the location and size of the persistent volatile memory component42is stored in an entry in NVRAM20.

In general and during a normal boot operation of a typical computer system, a computer usually invokes a BIOS that provides low-level access to peripheral devices; identifies RAM available to the processor of the computer; initializes this RAM, typically destroying its contents; and then installs the operating system into RAM, giving the operating system access to the entire RAM to move information into and out of memory as necessary. If the computer is started after having been powered down, all of its memory will have been initialized.

In contrast and referring again toFIG. 1A, during a normal boot operation the computer4invokes the modified BIOS40. The modified BIOS40retrieves configuration information44from NVRAM20. This configuration information44includes the start address and the size of persistent volatile memory36. The modified BIOS40then separates the RAM14into the volatile memory32and the persistent volatile memory36. The BIOS40then initializes the volatile memory32. The modified BIOS40provides low-level access to peripherals (not shown), installs the operating system24into the volatile memory32of RAM14, and prevents the operating system24from directly accessing the persistent volatile memory36during the boot cycle and normal computer operation. The operating system24is, in effect, unaware of the persistent volatile memory36. The operating system24then typically initializes or installs its own programs into the volatile memory32, often modifying the contents of the volatile memory32, but does not modify the contents of the persistent volatile memory36. This renders the contents of the persistent volatile memory36constant through a boot cycle.

In one embodiment of the invention, the intermediary program28is aware of the persistent volatile memory36and is able to access its contents. After reading the configuration information44, the intermediary program28serves as a link between the operating system24and the persistent volatile memory36. The intermediary program28receives a read request38from the operating system24to access the persistent volatile memory36and returns information to the operating system24from the appropriate location in the persistent volatile memory36. Similarly, the intermediary program28receives a write request39from the operating system24and stores information at the appropriate location in the persistent volatile memory36.

For example, in one embodiment of the invention the operating system24is the Windows 2000 operating system. Under Windows 2000, the persistent volatile memory36accessible through the intermediary program28appears to the operating system24as a RAM disk, in contrast to the invention, although the contents of a normal RAM disk do not survive a boot cycle. A Windows 2000 read request38or write request39includes an offset value (in bytes) from the start of the persistent volatile memory36and a length value (in bytes) of the data to read or the data to write. The intermediary program28computes the appropriate location in the persistent volatile memory36by adding the offset value in the request to the start address of the persistent volatile memory36. In one embodiment, the persistent volatile memory36includes 1 MB of configuration information at the beginning of the persistent volatile memory36, so the appropriate location is actually the sum of the offset value, the start address of the persistent volatile memory36, and 1 MB.

For a read request38, the intermediary program28copies a number of bytes equal in size to the length value from the computed location in the persistent volatile memory36to the user's buffer. For a write request39, the intermediary program28copies a number of bytes equal in size to the length value passed by the operating system24from the user's buffer to the computed location in the persistent volatile memory36. This interaction permits the operating system24to indirectly access the persistent volatile memory36without threatening the integrity of the contents of the persistent volatile memory36during a boot cycle. In another embodiment where Windows 2000 is the operating system24, the intermediary program28invokes the functionality of the operating system24to map the computed location onto the virtual address space of the operating system24for the copy operation. Other operating system24functionality completes the copy operation and unmaps the computed location from the virtual address space of the operating system24.

It is possible for the operating system24to crash while a write request39to persistent volatile memory36is being executed. In that case, an incomplete version of the request would be stored in the persistent volatile memory36. This can cause problems during subsequent operation, because a computer application may attempt to restore its state based on this incomplete information, potentially crashing the application and necessitating time-consuming reconstruction of the information lost during the crash.

To prevent this problem, the invention in one embodiment creates a table in the persistent volatile memory36and uses the table to describe the transactional information stored in the persistent volatile memory36. In particular, the intermediary program28creates and maintains the table. The intermediary program28creates and initializes the table when the computer4loads the intermediary program28. Examples of parameters that the table includes are, without limitation, a file descriptor, an offset into the file (i.e., a file byte location), a length value which indicates the number of bytes in the I/O request, a pointer that points to the location in the persistent volatile memory36at which the unbuffered write is stored, and a status field. The status field may include a free state that denotes that the location in the table is available to store information. The status field may also include a reserved state to denote that the intermediary program28is copying the transactional information to the persistent volatile memory36. In yet another embodiment, the status field may include an in-use state to denote that the table entry contains valid information and is not available for storage of new information. Although several examples of the parameters included in the table are described above, it should be noted that the table may also include additional parameters not described.

Alternatively, the invention creates look-aside buffer in the persistent volatile memory36and uses it for the atomic update and storage of transactional information; only when the write request39has been buffered and completed is it transferred out of the look-aside buffer. The intermediary program28may use the look-aside buffer to complete an unfinished write to the persistent volatile memory36(i.e., a computer failure before the write to the persistent volatile memory36completes). In greater detail, a look-aside buffer includes a set of bits that describe its state.FIG. 1Bshows how the state of the look-aside buffer changes to reflect various stages in the processing of a write request39.

When no information is in the buffer, for example at the creation and initialization of the buffer, the buffer state is 0 (Step10). When a write request39is received by the intermediary program28, the intermediary program28stores the computed location and the length (in bytes) of the request in the look-aside buffer and the state of the buffer becomes 1 (Step12). At this point, the actual contents of the write request are copied into the buffer (Step14). If the copy is successfully completed, the buffer state becomes 2 (Step16). If the copy fails because of, for example, a system crash, the buffer state remains at 1. Once the buffer state is set to 2, the contents of the write request are copied out of the look-aside buffer to their computed location in the persistent volatile memory36(Step18). When this is successfully completed, the buffer state returns to 0 (Step10).

The effect of the value of the state of the look-aside buffer on the subsequent boot process is depicted in FIG.1C. At system reboot (Step20), the intermediary program28locates all the look-aside buffers in the persistent volatile memory36. If there are no more look-aside buffers to check (Step22), the system boot process continues (Step24). If there are more look-aside buffers (Step22), the intermediary program28proceeds to examine the state of each look-aside buffer, one at a time, in the persistent volatile memory36(Step26). If the state of the buffer presently under examination is 0, the intermediary program28knows that there is no information stored in the look-aside buffer and the intermediary program28checks the next look-aside buffer (Step22). If the buffer state is 1, the intermediary program28knows that the information in the look-aside buffer is the result of an incomplete transaction and should not be moved into the persistent volatile memory36for recovery by a computer application. The intermediary program28sets the state of this buffer to 0 (Step20) and checks the next look-aside buffer (Step22). If the buffer under examination is in state2, then the intermediary program28knows that the contents of the look-aside buffer are the result of a completed transaction that did not get copied into the persistent volatile memory36. The intermediary program28copies the contents of the look-aside buffer to the computed location in the persistent volatile memory36(Step28). When the copy is completed, the buffer state is set to 0 (Step20) and the intermediary program28checks the next look-aside buffer (Step22). Eventually the intermediary program28will have checked the state of all the look-aside buffers, and the system boot will continue (Step24).

Although described above and below as a table or a buffer, it should be noted that any data structure can be used to provide information about the state of the transactional information that is stored in the persistent volatile memory36.

Referring toFIG. 1D, during a boot cycle the computer4loads the programs implementing the invention into the memory8at Step40. In one embodiment, the programs are the intermediary program28and the modified BIOS40. The programs28,40divide the RAM14into two portions: the volatile memory32directly accessible to the operating system24in Step42and the persistent volatile memory36that is not directly accessible to the operating system24in Step44. This is accomplished through modifications to the BIOS40. The inaccessibility to the operating system24renders the contents of the persistent volatile memory36resistant to initialization or modification during a boot cycle. Again, one skilled in the art will recognize that the invention permits multiple persistent and non-persistent memory regions, but for the sake of simplicity of discussion and depiction, the present discussion assumes one volatile memory32and one persistent volatile memory36.

Once the memory partitioning has been achieved, the intermediary program28provides indirect access to the persistent volatile memory36to the operating system24. In step46, the intermediary program28waits for a read request38or a write request39from the operating system24. The intermediary program28decides (Step48) whether a read request has been received, and if one has, then the intermediary program reads (Step50) from the appropriate location in the persistent volatile memory36and returns the result to the operating system24. Similarly, if the intermediary program28decides (Step52) that a write request39has been received, then the intermediary program28stores (Step54) information at the appropriate location in the persistent volatile memory36. If neither type of request has been received, then the intermediary program28returns to step46and continues to wait for requests. Typically, read and write requests from the operating system24to the volatile memory32operate as they would have before the installation of the invention.

An example of the intermediary program28is a filter driver module. The filter driver module28stores certain types of I/O transactional information in the persistent volatile memory36so that the information does not get erased during a computer crash. Thus, following a crash of the computer4, the application program18still recognizes what the application program18had done just prior to the computer crash.

In one embodiment, the transactional information are unbuffered writes (i.e., writes requested by the application18and in which notification to the application18of the completion of the write is necessary) to the persistent mass storage22. Examples of an unbuffered write include, without limitation, copying a file from one directory to another directory, backing up and/or updating a file, initializing a file (e.g., writing zeros to the file), and the like. Although described above and below with transactional information, any information can be used within the scope of the invention.

For example, the application18can be a database management system (DBMS) that verifies all updates to files before that file can be made available again to the application18. It should be noted that a request to access the file for a read or write transaction can be from a different query from the same application or can be from a different application altogether.

In general and during a normal unbuffered write of transactional information, an application program executing on a computer typically writes transactional information to disk and the operating system of the computer transmits a confirmation message to the application when the write completes. If the application program does not receive a confirmation message after a predetermined time, the application often performs the previous write to disk again and subsequently waits for another confirmation message. This process is generally wasteful and decreases the performance of the application program.

In the invention described above and below, the application18generates a write to the persistent mass storage22(e.g., disk). The operating system24instead uses the filter driver module28and writes the information to the persistent volatile memory36. Following the completion of this write to the persistent volatile memory36, the operating system24transmits the confirmation to the application18. The application18receives the confirmation soon after the generation of the write, as a write to memory (e.g., persistent volatile memory36) is faster than a write to the persistent mass storage22. Therefore, in one embodiment the application18receives the confirmation from the operating system24before the transactional information is stored in the persistent mass storage22.

The filter driver module28may be a passive filter or an active filter. A passive filter is a filter that monitors the information that the filter driver module28stores in the persistent volatile memory36. For example, the computer4configures the passive filter driver module28to monitor all unbuffered writes requested by a particular application18, such as by a DBMS. This may be used to help determine performance decreases associated with multiple unbuffered writes by a particular application18.

As an active filter, the filter driver module28receives an instruction to store in the persistent volatile memory36and performs some modification on the instruction before storing the instruction. For example, the active filter driver module28may receive a certain type of transactional information, such as an unbuffered write to initialize a file by writing zeros to the file. The active filter driver module28may alter the unbuffered write to write ones to the file if the operating system24determines the writing of ones to be necessary for initialization. In a further embodiment, the filter driver module28is created by a file systems filter driver kit (FDDK), developed by Open Systems Resources, Incorporated of Amherst, N.H.

Referring toFIG. 2A, a logical flow chart depicts the operation of the computer4on unbuffered writes. The application18generates (step205) an unbuffered write and the filter driver module28detects (step210) the unbuffered write. After detecting the unbuffered writes, the filter driver module28updates (step212) the table described above with information associated with the unbuffered write. For example, the filter driver module28updates the status field to a reserved state to denote that the detected unbuffered write is about to be copied to the persistent volatile memory36.

After updating the table, the filter driver module28stores (step215) the transactional information in the persistent volatile memory36. In another embodiment, all writes (unbuffered writes and buffered writes) are stored in the persistent volatile memory36. In yet another embodiment, the filter driver module28stores the transactional information in volatile memory32, makes a copy of the transactional information stored in the volatile memory32, and then transfers the copy into the persistent volatile memory36.

In one embodiment and as further described below, the operating system24additionally starts a timer to enable future recordation of the time elapsed from the transferring of the transactional information to the persistent volatile memory36. In another embodiment, the operating system24stores the time read from a predetermined register located in the computer4.

The filter driver module28then updates (step220) the table to denote that the transfer of the transactional information to the persistent volatile memory36is complete. In particular and in one embodiment, the filter driver module28updates the status field associated with the particular transactional information to an in-use state. Thus, if a failure of the computer4occurs prior to the completion of a transmittal of transactional information to the persistent volatile memory36, the filter driver module28can determine that the transmittal of particular session information did not complete prior to the computer failure (i.e., the status field associated with the transactional information will not be set to an in-use state). If the filter driver module28determines that the transactional information was not transmitted to the persistent volatile memory36, then the filter driver module28repeats step215to store the transactional information in the persistent volatile memory.

The operating system24then notifies (step223) the application18that the transactional information has been stored in the persistent volatile memory36. In one embodiment, the notification occurs as a confirmation message to the application. The operating system24then determines (step225) whether the operating system24should flush, or transfer, the persistent volatile memory36to the persistent mass storage22. In one embodiment, the filter driver module28includes a thread responsible for flushing the persistent volatile memory36to the persistent mass storage22. As described further below, the thread can flush the persistent volatile memory36when a particular event occurs, such as when the operating system24transmits a message to the filter driver module28instructing the filter driver module28to flush the persistent volatile memory36. In another embodiment, the thread may poll the persistent volatile memory36to determine whether the data stored in the persistent volatile memory should be flushed.

FIG. 2Billustrates an embodiment of the steps performed by the operating system24to determine (step225) whether the operating system24should flush the persistent volatile memory36to the persistent mass storage22. In one embodiment, the operating system24determines (step230) whether the data stored in the persistent volatile memory36exceeds or is about to exceed some predetermined threshold (e.g., the allotted size of the persistent volatile memory36). If so, the operating system24flushes (step233) the persistent volatile memory36to the persistent mass storage22. If not, the operating system24does not flush (step234) the persistent volatile memory36. The operating system24can also flush (step235) the persistent volatile memory36if the operating system24determines (step235) that a predefined amount of time since the transferring of the transactional information has elapsed. In some embodiments, the operating system24uses the timer described above to make this determination. In another embodiment, the operating system24records the current time stored in the register described above. Using this recorded time and the previously recorded time, the operating system24determines the amount of time that has elapsed since the transferring the transactional information to the persistent volatile memory36. If the elapsed time is greater than a predefined amount of time, the operating system24flushes (step233) the persistent volatile memory36.

In another embodiment, the operating system24determines (step240) to flush the persistent volatile memory36when the operating system24is not servicing the application18(i.e., the operating system is not busy). For example, the operating system24flushes the persistent volatile memory36when the application18is idle.

Alternatively, the operating system24determines (step245) whether the application18requests to write the transactional information to a file that has previously been closed. In one embodiment, the application28transmits a message to the operating system24when the application28closes. If so, the operating system24flushes (step233) the persistent volatile memory36. The operating system24additionally flushes (step233) the persistent volatile memory233when the computer4is in the process of being (step250) shut down. In one embodiment, the filter driver module28additionally updates (step255) the table to denote that the transactional information has been stored in the persistent mass storage22.

By storing the transactional information in a persistent volatile memory36, the information is accessible to the computer4at any instant in time. Therefore, a retrieval of such information does not significantly hamper the performance of the computer4. Additionally, the persistent volatile memory36(and the persistent mass storage22) enable the session information to be accessible to the computer4after a computer failure, which would ordinarily erase the information from a volatile memory.

As an example and referring again toFIG. 2A, suppose the transaction is a catalog merchandise order phoned in by a customer and entered into the computer4by a customer representative. The customer enters the order into an application18associated with the catalog and the application18generates (step205) the writes to a database file that should occur for the order. In particular, the order transaction involves checking an inventory database file, confirming that the item is available, placing the order and confirming that the order has been placed. Considering these steps as a single transaction, then all of the steps are to be completed before the transaction is successful and the inventory database file is actually changed to reflect the new order.

In greater detail, the application18associated with the catalog checks the inventory database file and confirms that the item is available. If the item is available, the application18places the order. In some prior art computer systems, the application18stores the transactional information in RAM14so that the application18(e.g., the DBMS) can commit the information to persistent mass storage22at a later time. If the computer4failures (e.g., crashes) at this point in time, the order would have been placed, but the information previously stored in RAM14is erased. Consequently, the transactional information has not been reflected in the inventory database file and, therefore, to retrieve this information to update the inventory database file, the application18frequently has to repeat the transaction again.

Unlike the above scenario, the invention uses the filter driver module28to determine (step210) that the transaction involves an unbuffered write to the inventory database file. The filter driver module28then stores (step215) the transactional information to the persistent volatile memory36so that the transactional information can survive a crash of the computer4. Thus, if the failure of the computer4occurs, the transactional information has already been stored in the persistent volatile memory36. The operating system24notifies (step223) the application18of the completion of the write and consequently determines (step225) whether to flush the persistent volatile memory36, as described above.

Having described certain embodiments of the invention, it will now become apparent to one of skill in the art that other embodiments incorporating the concepts of the invention may be used. Therefore, the invention should not be limited to certain embodiments, but rather should be limited only by the spirit and scope of the following claims.