Procedure to reduce copy time for data backup from short-term to long-term memory

Systems and methods for backup of data in redundant data storage systems. In this regard, one embodiment can be broadly summarized by a representative system that copies a block of data from a primary storage unit to a primary backup storage unit using a primary addressing sequence that begins with a first start address; and substantially concurrently copies a second block of data from a secondary storage unit to a secondary backup storage unit using a secondary addressing sequence that begins with a second start address.Another embodiment can be described as a method wherein the first start address is the same as the second finish address; the primary addressing sequence uses an incrementing count, and the secondary addressing sequence uses a decrementing count. Other systems and methods are also provided.

FIELD OF THE INVENTION

The present invention generally relates to the field of redundant data storage systems, and in particular to preventing data loss in disk storage subsystems.

DESCRIPTION OF THE RELATED ART

External memory storage systems often combine multiple inexpensive disk drives into an integrated array, to obtain a cost-effective system whose performance, capacity, cost savings, and reliability exceeds that of a single large drive. An array of such drives appears to a host computer as a single logical drive. If the array is configured to operate in a non-redundant manner, the mean time between failures (MTBF) is relatively low, as it equals the MTBF of an individual drive divided by the number of drives in the array. To improve MTBF, an array of drives may be configured as a redundant array of inexpensive disks (RAID). A RAID array incorporates fault tolerance by using a redundant architecture that protects against data loss created by hardware failure.

The disk controller in a RAID system is termed a RAID controller. RAID controllers have been defined at different “levels” to provide different sets of features. User data in a RAID system is moved between different RAID levels by a data storage management system that may utilize a “data map” to keep track of data movement. This data map is updated periodically, and its preservation is critical for system operational integrity. If the data map is lost, then access to the user's data is also lost. In some systems this data map is stored in non-volatile RAM (NVRAM), which acts as a short-term memory storage mechanism that is powered from an on-board battery in the event the main system power fails.

The NVRAM also performs another important task. Data access latency for the host computer is reduced by the NVRAM operating as a “write cache” to temporarily store data before the data is transferred to the RAID memory disks. Once the data is written by the host computer into the NVRAM, the transaction is acknowledged and the host computer moves on to its next operation. At this point, as far as the host computer is concerned the data has been successfully transferred to a hard disk drive. Therefore for a period of time, data destined for RAID memory disk storage is stored only in the NVRAM. If main system power fails at this time, the on-board battery is used to keep the NVRAM data from being lost, while a data backup process is initiated to copy the contents of the NVRAM into a pre-defined form of long-term non-volatile storage that is generally referred to as a “dump disk.” Once the data is copied to the dump disk, which may be, for example, a magnetic disk storage device, stored data is retained even if the power supply to the dump disk is subsequently disconnected.

The dump disk is generally not used as a “primary” mechanism for storing a data map because the read/write performance of this disk is much lower than that of the NVRAM.

Upon main system power failure, it is desirable that a dump disk remain powered-up at least long enough to allow the disk controller to copy the data map and write cache data, which are jointly referred to henceforth in this disclosure as “valid” data, from the NVRAM to one or more dump disks. One way to do this is to use an external battery backup system (in addition to the NVRAM batteries) that has the capability under main system power failure conditions to operate the dump disk until the NVRAM data is safely copied.

One of the biggest challenges with this architecture is to ensure that the external battery backup system has enough capacity to complete the backup, but is not too large as to unnecessarily increase cost, complexity, etc.

As disk arrays become larger, and system requirements increase, there is a corresponding demand for larger NVRAM size. The duration of time that data stored in the NVRAM is retained upon main system power failure, is directly related to battery capacity and NVRAM size. With finite improvements in battery technology and the limitations placed upon battery capacity due to cost factors and physical size constraints, the ongoing demand for larger NVRAM size leads to a situation where the duration of time that NVRAM data can be retained upon main system power failure, is being reduced. This leads to a serious problem during a data backup process from NVRAM to dump disk after a power failure.

To support a fault tolerant system, such as a high-availability RAID system, at least two separate RAID controllers are used, with one of the two controllers being designated to operate as a “master” controller. Each of the two RAID controllers has its own corresponding NVRAM, and dump disk, together with appropriate power sources. The NVRAMs in the two controllers generally contain identical data. Under a system power failure condition, the master controller carries out a data backup by copying the valid data contained in its NVRAM, into the two dump disks. The valid data is generally copied from NVRAM to each of the two dump disks in a sequential manner. For example, if the NVRAM contained 64 Kbytes of valid data, the copy sequence may begin at address 0000 and end at address FFFF (Hex).

In systems that contain much larger amounts of valid NVRAM data, it is likely that this copy operation, which needs to be replicated for the two dump disks, may take several minutes. During this time, there may be a catastrophic failure in the battery systems that power the dump disk or the charge in the battery system may not be sufficient to permit the entire contents of the NVRAM to be copied to the dump disks. If this happens, and the copying operation is not completed, the data remaining in the NVRAM may be lost subsequently, because this data can be retained only as long as the NVRAM battery can power the NVRAM. The NVRAM battery may last only a few hours or a few days depending on its capacity, and the size of the NVRAM.

It would therefore be desirable to provide a solution that minimizes the time required to copy data stored in short term non-volatile memories such as NVRAM, into longer-term non-volatile storage media, such as magnetic dump disk, during a main system power failure. Protecting valid data becomes especially important in redundant data storage systems, where fault tolerance is a critical parameter.

SUMMARY OF THE INVENTION

The present invention provides methods, systems and means for backup of data in redundant data storage systems. In this regard, one such data storage system can be broadly summarized by a representative system that copies a block of data from a primary storage unit to a primary backup storage unit using a primary addressing sequence that begins with a first start address; and substantially concurrently copies a second block of data from a secondary storage unit to a secondary backup storage unit using a secondary addressing sequence that begins with a second start address.

Another embodiment can be described as a method wherein the second start address is the same as a first finish address, the primary addressing sequence uses an incrementing count, and the secondary addressing sequence uses a decrementing count. It can be seen that this method creates a crossover point in time where the data transferred to the two backup storage units complement each other and jointly provide a complete image of the data that had been stored individually in the primary and the secondary storage units. At this crossover point the information has been protected in a joint manner in as little as half the time it would have taken in a conventional method that stores two identical redundant copies in two backup storage units.

DETAILED DESCRIPTION

The present invention will now be described with particular reference to a system incorporating a disk array architecture, such as used in a redundant array of inexpensive disks (RAID) system. It should be apparent, however, to persons having ordinary skill in the art that the explanation can be extended to other applications that utilize a redundant architecture incorporating a combination of short-term as well as long-term memory storage of data.

FIG. 1is a schematic representation of an array of disks that is controlled by two disk controllers in a redundant disk controller system.

The host computer135may be a personal computer (PC), a microprocessor system, a network server, a mainframe computer, or a dedicated system controller unit. In this particular illustration, host computer135is shown connected via a bus interface140eto a redundant disk controller system160, which comprises of two disk controllers130and145that are located external to the host computer135. Disk controller145is generally selected to be identical to disk controller130, so that it may provide equivalent operating capabilities in case of failure of disk controller130.

Bus interface140emay utilize a variety of mechanisms, such as a small computer system interface (SCSI) or a high-speed Fibre-channel interface. It may also use an AT Attachment (ATA) interface that most desktop PCs use to access IDE (Integrated Drive Electronics) disks.

The disks110,115,120, and125are configured as a redundant array of disks having a primary serial interconnection and a secondary serial interconnection. The primary serial interconnection, which is formed by bi-directional communication links140a,140b,140c, and140dis associated with disk controller130. The secondary serial interconnection, which is formed by bi-directional communication links150a,150b,150c, and150dis associated with disk controller145.

The redundant disk controller system160provides protection against failure of any one of the two disk controllers130and145. For example, if disk controller130were to fail, host computer135may still access the array of disks using disk controller145, and retrieve data stored in the array of disks.

For ease of explanation, the circuitry associated with disk controller130is labeled Side A, while the circuitry associated with disk controller145is labeled Side B. In a typical application, one of the two sides assumes control in a role that is referred to by various names, such as “master,” “active,” or “primary.” The other side is consequently referred to by corresponding names such as “slave,” “standby,” or “secondary.” This “master-slave” arbitration is used to avoid operational conflicts between two devices when they interact with common elements. As an example, such a situation may arise when both disk controllers130and145attempt to simultaneously transmit identical information back to the host computer135, over the single bi-directional communication link140ethat is shared by both disk controllers. This type of problem may occur in response to a “read disk memory” request initiated by the host computer135. Under this condition, only the designated “master” disk controller is permitted to transmit data back to the host computer135. If the designated master disk controller were to fail, the slave disk controller takes over the role of master.

The bi-directional communication link132provides a mechanism to convey messages between the two redundant disk controllers130and145. These messages may include information necessary to ensure that both disk controllers contain identical data. This process of ensuring identical data is termed “synchronization” or “sync,” and is used to provide uninterrupted operation of the system, after a switchover has taken place from the master to the slave controller due to a failure in the master controller.

While the illustration ofFIG. 1indicates two distinct disk controllers, and two distinct serial interconnections in the disk array, it will be obvious to one of ordinary skill in the art, that the physical implementation of these configurations may comprise a single controller unit that incorporates both disk controllers inside (shown as the dashed box inFIG. 1). For example, the disk controllers130and145may be part of a dual-ported Fibre Channel drive with the two serial interconnections to the array of disks physically incorporated within a single cable structure. Where found more convenient, single port drive structures such as those used for ATA type drives may be utilized to provide two independent serial interconnections to the array of disks. The two disk controllers may also be physically integrated into the host computer135in certain systems.

It can also be appreciated that the redundant disk controller system shown inFIG. 1may be referred to specifically as a RAID system, and the disk controllers130and145may be individually referred to as RAID controller130, and RAID controller145. Furthermore, if at any particular instance, RAID controller130was operating in the “master” mode it may be referred to as the “primary RAID controller” and the RAID controller145may be correspondingly referred to as the “Secondary RAID controller.”

The terms “primary RAID controller” and “secondary RAID controller” may also be referred to broadly in connection with redundant storage systems in general, as a “primary controller” and a “secondary controller.”

FIG. 2is a schematic representation ofFIG. 1together with dump disks that are used for backing up data during main system power failure. The disk controller130is connected by communication link260ato a non-volatile disk storage system that is defined as Side A dump disk255, and by a communication link260bto a non-volatile disk storage system that is defined as Side B dump disk265. Disk controller145is connected by a communication link260cto the Side A dump disk255, and by a communication link260dto the Side B dump disk265. The communication links260aand260ballow disk controller130to read/write data to either dump disk255or to dump disk265. Similarly, communication links260cand260dallow disk controller145to read/write data from either dump disk255or dump disk265. Dump disk265is typically selected to be identical to dump disk255.

While the dump disks255and265are shown inFIG. 2to exist outside the disk controllers for example purposes, these dump disks may generally exist internally within the disk controllers as mentioned earlier in this disclosure. In certain other applications, dump disks255and265may also be incorporated as a part of the disk array comprised of disks110,115,120, and125.

The communication links260a,260b,260c, and260dare shown distinctly separate from one another inFIG. 2, for explanation purposes. But it can be appreciated, that the physical implementation of these communication links may take various forms, which in certain cases may utilize an integrated medium such as a single cable connection that may simultaneously carry multiple individual communication signals.

FIG. 3is a schematic representation of the functional blocks in a disk controller system160. Disk controller160, may incorporate a control processor340, such as a microprocessor that is coupled to the host interface circuit320and the disk interface circuit355. The control processor340is also coupled to a short term memory such as NVRAM350, and a main memory335. Buses350a,350b,350c, and350dprovide the connections for control processor140and may be physically distinct to one another or can take the form of a common bus. Those of ordinary skill in the art will recognize several types of bus architectures such as a star, a common bus, or a daisy-chain may be utilized for this purpose.

As described earlier with reference toFIG. 1, disk memories110,115,120, and125may be configured as an array of disks. Such an array of disks may be connected to the disk controller160via disk interface circuit355.

InFIG. 3, data transfers between the host computer135and the array of disks are routed by control processor340, through the host interface circuit320to cache memory345. Cache memory345may comprise a “write cache” and/or a “read cache.” The write-cache is sometimes referred to as a “buffer cache.” Though shown for example purposes to be contained in NVRAM350, cache memory345may comprise of non-volatile and/or volatile memories, depending upon the system requirements for redundancy in case of main system power failure.

When the host computer135initiates a write operation, the data is first written to the cache memory345and then transferred via disk interface circuit355, to the array of disks. Similarly when the host computer135requests for a data fetch, which constitutes a read operation from the array of disks, disk controller160first determines if the requested data is already stored in the cache memory345. If the requested data is stored in cache345, the disk controller160provides this “cached” data to the host computer135, thereby avoiding the necessity of accessing the array of disks which is slower than the cache memory345. If the requested data is not stored in cache memory345, then the data is retrieved from the array of disks, stored in cache memory345and then provided to the host computer135. With this technique, the disk controller160maintains in the cache memory345a copy of the data transferred between host computer135and the array of disks during a read or a write operation.

The main memory335ofFIG. 3may include several types of memory devices such as read only memories as well as random access memories that permit read as well as write operations. The read only memories may contain programming information in the form of software and/or firmware. An example of such information is shown in the data backup manager500. The random access memories contained in main memory335may be used to store information associated with the application that is being run on host computer135. An example of such information is a cache management table that is used to operate cache memory345.

FIG. 4illustrates an example system that incorporates NVRAM together with battery backed dump disk(s) in a redundant disk controller system. NVRAM410is located inside disk controller130, and due to the provision of an independent power source, such as a battery415, retains its data contents when main system power fails. The Side A dump disk255is also provided with an independent power source, such as a battery420. Battery420is designed with adequate capacity to allow the dump disk255to operate during main system power failure at least long enough to permit the contents of the NVRAM410to be copied to the dump disk255. The battery420is also typically used to operate other circuitry on the disk controller130. Such circuitry may be needed to carry out backup operations in the event of a main system power failure.

NVRAM425is located inside disk controller145, and similar to NVRAM410, is a non-volatile random access memory that retains its data contents when main system power fails. This data retention capability is provided by an independent power source, such as a battery430, that is connected to NVRAM425. Side B dump disk265is similar to dump disk255. A power source, such as battery435, allows the dump disk265to operate during main system power failure. Battery435is designed with adequate capacity to allow the dump disk265to operate during main system power failure at least long enough to permit the contents of the NVRAM425to be transferred to the dump disk265. Similar to the usage of battery420, the battery435, or an equivalent power source, may also be typically used to operate other circuitry on the disk controller145.

Under normal system operating conditions, the data contents of NVRAM410and NVRAM425are identical, having been “synched” together by the master-slave operational configuration. This action is facilitated by the communication link132.

The valid data contained in the NVRAM located in the master controller is also copied to both the dump disk255and dump disk265in a periodical manner. For example, if disk controller130is the master controller at any particular instant in time, the valid data contained in NVRAM410is copied by master controller130into Side A dump disk255using link260a, and also into Side B dump disk265using link260b.

This data copy operation is generally carried out during normal conditions when the system power has not failed, and the disk controller130is not actively involved in servicing commands from host computer135.

Therefore, in general, when there is a system level power failure, the dump disks255and265already contain a part of the valid NVRAM data. The only NVRAM data that needs to be transferred into the dump disks at this stage, would consist of data that had not been transferred prior to the power failure. It can be appreciated that this “incremental” valid data may reside in a memory address sector that does not necessarily start at address location 0000. For example, if valid NVRAM data exists from 0000 to FAAA (Hex), and data contents from addresses 0000 through AAAA (Hex) have been already been transferred to the dump disks during normal operating conditions, under a system level power failure condition at this time, only the incremental valid data from AAAB (Hex) to FAAA (Hex) may require transfer into the dump disks. It may be unnecessary to transfer the entire NVRAM data from 0000 to FAAA (Hex) under this power failure condition. Certain systems may utilize this type of “incremental backup” procedure so as to minimize the time needed to backup NVRAM data after a power failure.

FIG. 5is a flow-chart that illustrates a representative process for copying the contents of NVRAM located on each of two disk controllers to their corresponding dump disks. It is to be understood that any process steps or blocks shown inFIG. 5represent modules, segments, or portions of code that include one or more executable instructions for implementing specific logical functions or steps in the process. It will be appreciated that, although particular example process steps are described, alternative implementations are feasible. Moreover, steps may be executed out of order from that shown or discussed, including substantially concurrently or in reverse order, depending on the functionality involved. Code may also be contained in several devices of the invention, and may not be necessarily confined to any particular device. The explanation below, while possibly implying code residency and functionality in certain devices, does so solely for the purposes of explaining the concept behind the invention, and the description should not be construed as a limiting parameter for the invention's various implementations in other applications.

The system determines in steps515and520if the main system power has failed. If the main system power has failed, as determined by the system in step520, the process of transferring NVRAM data to dump disk proceeds via steps535and540.

Steps535and540may be generally implemented to run concurrently, but in certain applications may be allowed to operate with a small time delay between the Side A and the Side B transfers. In step535, the side A disk controller130begins to copy valid data from NVRAM410into the side A dump disk255, and in a fairly simultaneous fashion, in step540, the side B disk controller145begins to copy valid data from its NVRAM425into the side B dump disk265.

As this copy process explained in steps535and540, operates concurrently on both side A and side B of the redundant disk controller system, the time required to backup data into the dump disks is minimized. This is especially significant when compared to a system that may back up data into the dump disks in a non-simultaneous fashion using a sequential or time-multiplexed backup.

FIG. 6is another embodiment ofFIG. 5that includes addressing counters. The system determines in steps615and620if the main system power has failed. If main system power has not failed, the system sets a start address required for Side A of the redundant disk controller system. This process of setting the start address, which is carried out in step625, may be a static process whereby the start address is pre-determined and is set accordingly. For example, it may be set at FFFF (Hex).

Alternatively, the process of setting the start address may be a dynamic process, whereby the start address is determined based upon the size of valid data bytes contained in the NVRAM. For purposes of illustration, let us assume that the incremental valid data that has not yet been written into the dump disk, exists from NVRAM address 0FA2 (Hex) to address FFAE (Hex). Under this condition, in the dynamic process for setting the start address, the Side A start address may be set at 0FA2 (Hex) in step625.

Step625ofFIG. 5is followed by Step630wherein the system sets a start address for Side B. This is done in a manner compatible to either the static or the dynamic addressing process carried out in step625. If address FFFF (Hex) were used in a static process in step625, the system may correspondingly set the Side B start address to 0000 in step630. If the dynamic process was used in step625, wherein the Side A start address was set to 0FA2 (Hex), the Side B start address may be correspondingly set to FFAE (Hex) in step630.

With further reference toFIG. 6, if the main system power has failed, as determined by the system in step620, the process of transferring NVRAM data to dump disk proceeds via steps635and640. Steps635and640may be generally implemented to run concurrently, but in certain applications may be allowed to operate with a small time delay between the Side A and the Side B transfers. Using the example cited above for the dynamic process, step635ofFIG. 6is referenced to Side A, with a start address of 0FA2 (Hex). In step635, disk controller130(refer FIG.4) begins to transfer data bytes from NVRAM410to Side A dump disk255, using an incrementing sequence counter that increments from 0FA2 (Hex) towards FFAE (Hex).

In step640, disk controller145(ofFIG. 4) begins to transfer data bytes, using the Side B start address of FFAE (Hex) in this example application, from NVRAM425to Side B dump disk265, using a decrementing sequence counter that decrements from FFAE (Hex) towards 0FA2 (Hex).

It can be seen that at some time subsequent to the start of the data transfer process indicated in steps635and640, a crossover point is reached where the data transferred to the two dump disks complement each other and jointly provide a complete image of the NVRAM valid data that had been stored in each NVRAM individually. At this point the NVRAM information has been protected in a joint manner in as little as half the time it would have taken in the conventional method, which may use two counters with an identical incrementing counting scheme in both counters. The novel data backup process outlined inFIG. 6, therefore creates less exposure to NVRAM data loss during a main system power failure. Once the crossover point is reached, each controller can continue copying data onto the hard disks until full redundancy is obtained.

Though the example above utilized two sides, Side A and Side B, it can be appreciated that more than two sides may be implemented in a system that requires such a configuration. In such a case, the start addresses of these multiple sides can be selected in a suitable manner, providing for sufficient address ranges in each of the multiple sides.

While the example above uses an incrementing sequence counter and a decrementing sequence counter, other types of counters using suitable sequence counting schemes may also be used. The purpose behind utilizing count sequences is to accomplish storage of NVRAM data across multiple dump disks in an efficient manner.

An example of a variation in sequence counting schemes can be illustrated using a system wherein the NVRAM has valid data from address 0000 through address FFFF (Hex), and three dump disks, designated Side A, Side B, and Side C, are used.

In this example, Side A may be designated by a start address of 0000, while Side B may start at 5555 (Hex) and Side C may start at AAAA (Hex). If all three counters associated with these three sides are set to increment, Side A will increment from 0000 towards 5555 (Hex), while Side B will increment from 5555 (Hex) towards AAAA (Hex), and Side C increments from AAAA (Hex) towards FFFF (Hex). A complementary copy of the NVRAM valid data will be available across these three dump disks at the moment when Side A counter reaches 5555 (Hex). At this time, Side B counter would read AAAA (Hex), and Side C counter would read FFFF (Hex).

While the software and/or firmware necessary to implement the flowcharts described inFIG. 5andFIG. 6, may typically reside in the disk controllers, as is shown by the data backup manager500inFIG. 3, this software and/or firmware may also be designed to be resident in the host computer135. The advantage in doing this may lie in overcoming any delays or ambiguities that may occur in the master-slave arbitration process between two or more disk controllers operating amongst themselves to make the determination of who is master.

In most systems, devices which incorporate data backup software are typically designed to operate from a back-up power source in the event of a main system power failure. In certain embodiments, external memory storage subsystems may be designed with more efficient backup system power available to the host computer135, than that available to the disk controller160. In such a situation, the backup software may be placed in the host computer135, rather than in the disk controller160. In another embodiment, the disk controller160may also contain a sub-set of the data backup software present in the host computer135.

It should be emphasized that the above-described embodiments of the present invention are merely possible examples of implementations and merely set forth for a clear understanding of the principles of the invention. Many variations and modifications may be made to the above-described embodiment of the invention without departing substantially from the spirit and principles of the invention. For example, it will be appreciated by those skilled in the art that the particular format of the information and medium on which it is communicated could be chosen from any means capable of doing so. All such modifications and variations are intended to be included herein within the scope of the present invention and protected by the following claims.