Apparatus and method in a cached raid controller utilizing a solid state backup device for improving data availability time

An apparatus for reducing data unavailability time after a loss of main power in a storage controller is described. The controller backs up its volatile memory containing posted-write data to a non-volatile memory upon detecting a loss of main power. The controller continues to provide battery power to the volatile memory to sustain the posted-write data. If the battery is able to supply power to the volatile memory until main power is restored, the controller foregoes restoring the posted-write data to the volatile memory from the non-volatile memory. By not incurring the restore time, which may be substantial if the volatile memory is large since read rates from volatile memories are typically slow, the data unavailability time is reduced. The selective restore feature is user-disableable and also includes a brown-out timer for allowing a user to specify how long to battery-power the volatile memory if the feature is enabled.

FIELD OF THE INVENTION

The present invention relates in general to the field of mass storage controllers, and particularly to write-caching controllers that use a non-volatile backup device to avoid loss of cached user data.

BACKGROUND OF THE INVENTION

Redundant Array of Inexpensive Disk (RAID) systems have become the predominant form of mass storage systems in most computer systems today that are used in applications that require high performance, large amounts of storage, and/or high data availability, such as transaction processing, banking, medical applications, database servers, internet servers, mail servers, scientific computing, and a host of other applications. A RAID controller controls a group of multiple physical disk drives in such a manner as to present a single logical disk drive (or multiple logical disk drives) to a computer operating system. RAID controllers employ the techniques of data striping and data redundancy to increase performance and data availability.

An important characteristic of RAID controllers, particularly in certain applications such as transaction processing or real-time data capture of large data streams, is to provide fast write performance. In particular, the overall performance of the computer system may be greatly improved if the write latency of the RAID controller is relatively small. The write latency is the time the RAID controller takes to complete a write request from the computer system.

Many RAID controllers include a relatively large cache memory for caching user data from the disk drives. Caching the data enables the RAID controller to quickly return data to the computer system if the requested data is in the cache memory since the RAID controller does not have to perform the lengthy operation of reading the data from the disk drives. The cache memory may also be employed to reduce write request latency by enabling what is commonly referred to as posted-write operations. In a posted-write operation, the RAID controller reads the data specified by the computer system from the computer system into the RAID controller's cache memory and then immediately notifies the computer system that the write request is complete, even though the RAID controller has not yet written the data to the disk drives. Posted-writes are particularly useful in RAID controllers, since in some redundant RAID levels a read-modify-write operation to the disk drives must be performed in order to accomplish the system write request, i.e., not only must the specified system data be written to the disk drives, but some of the disk drives may also have to be read before the user data and redundant data can be written to the disks, which may make the write latency of a RAID controller even longer than a non-RAID controller.

However, posted-write operations make the system vulnerable to data loss in the event of a power failure. This is because the cache memory is a volatile memory that loses the user data when power is lost and the data has not yet been written to the disk drives.

To solve this problem, some RAID controllers include a battery to continue to provide power to the cache memory in the event of a loss of main power. Typically, the system automatically notifies a system administrator who attempts to restore power to the system. Although the battery greatly reduces the likelihood that user data will be lost, because the charge on the battery is finite, the possibility still exists that the battery power will run out before main power can be restored, in which case the user data will be lost. To avoid this possibility of user data loss, other RAID controllers include some form of non-volatile memory, such as a FLASH memory or small disk drive. When main power is lost, while the battery supplies power, the RAID controller copies the cache memory contents to the FLASH memory and then disables battery power. When main power is restored, the RAID controller restores the contents of the cache memory prior to the main power outage from the FLASH memory so that the posted-writes can be completed and the user data can be made available again.

However, the time required to restore the cache memory contents from the FLASH memory may be relatively lengthy, particularly, on the order of minutes. Assume, for example, a RAID controller that has 512 MB of cache memory and current FLASH memories that provide a sustained read rate of approximately 9 MB/second. In this example, the time required to restore the cache memory from FLASH memory is approximately one minute; that is, one minute more is required to boot the RAID controller after main power is restored. This is one minute more that the user data is not available to the host computer system, which in some user applications may translate to thousands of dollars of lost income. Furthermore, the additional time spent restoring the cache memory from FLASH may cause the predetermined timeout values of some server applications to be exceeded, thereby causing the application to fail. Finally, the restore time—and therefore user data unavailability time—is even greater for RAID controllers with larger cache memories than the example; and, the problem will be exacerbated even further as RAID controller cache memory sizes increase, which appears to be a definite trend.

Therefore, what is needed is an apparatus and method for reducing the data unavailability time after a loss of main power in a cached RAID controller with a non-volatile device for backing up the volatile cache memory.

BRIEF SUMMARY OF INVENTION

The present invention provides an apparatus and method for reducing the data unavailability time after a loss of main power in a cached RAID controller by continuing to supply battery power to the volatile cache memory even after the volatile memory has been backed up to the non-volatile memory, and foregoing performing a restore from the non-volatile memory to the volatile memory if it determines that the battery was able to continue supplying power to the volatile memory until the main power was restored.

In one aspect, the present invention provides a mass storage controller for providing improved data availability. The controller includes a volatile memory, for storing posted-write data, and a non-volatile memory, coupled to the volatile memory, for backing up the posted-write data from the volatile memory, in response to a loss of main power. The controller also includes a battery, coupled to supply power to the volatile memory in response to the loss of main power, and an indicator, for indicating whether the battery continued to supply power to the volatile memory until the main power was restored. The controller also includes control logic, coupled to the indicator, configured to forego restoring the posted-write data to the volatile memory from the non-volatile memory in response to the main power being restored, if the indicator indicates the battery continued to supply power to the volatile memory until the main power was restored.

In another aspect, the present invention provides a method for improving data availability in a redundant array of disks (RAID) controller having a volatile memory for storing posted-write data, a non-volatile memory, and a battery for providing power to the controller in response to loss of main power to the controller. The method includes backing up the posted-write data from the volatile memory to the non-volatile memory, in response to the loss of main power. The method also includes determining whether the battery sustained the posted-write data in the volatile memory until the main power is restored, and restoring the posted-write data to the volatile memory from the non-volatile memory, in response to the main power being restored, only if the battery did not sustain the posted-write data in the volatile memory until the main power is restored.

In another aspect, the present invention provides an apparatus for improving data availability in a write-caching controller having a volatile memory for caching posted-write data, a non-volatile memory to which a back up of the posted-write data is performed after a main power failure, and a battery for providing power while the backup is being performed. The apparatus includes the battery attempting to continue to provide power to the volatile memory after the backup is complete and until the main power is restored. The apparatus further includes a storage element that stores an indication of whether the battery successfully continued to provide power to the volatile memory until the main power was restored. The apparatus also includes control logic, coupled to the storage element, which restores the posted-write data from the non-volatile memory to the volatile memory when the main is restored, but only if the indication indicates the battery did not continue to provide power to the volatile memory until the main power was restored.

DETAILED DESCRIPTION

Referring now toFIG. 1, a block diagram illustrating a mass storage controller100, such as a RAID controller, according to the present invention is shown. In one embodiment, the controller100may be one of a pair of active-active redundant fault-tolerant RAID controllers for providing high data availability.

The controller100includes a disk interface128for interfacing to disk drives or other mass storage devices, including but not limited to, tape drives and optical storage devices, such as CDROM or DVD drives. The disk drives store the user data. The disk interface128may include, but is not limited to, the following interfaces: Fibre Channel, Small Computer Systems Interface (SCSI), Advanced Technology Attachment (ATA), Serial Attached SCSI (SAS), Serial Advanced Technology Attachment (SATA), Ethernet, Infiniband, HIPPI, ESCON, or FICON. The controller100reads and writes data from or to the disk drives in response to I/O requests received from host computers.

The controller100also includes a host interface126for interfacing with host computers. In one embodiment, the controller100is a local bus-based controller, such as a controller that plugs into, or is integrated into, a local I/O bus of the host computer system, such as a PCI, PCI-X, CompactPCI, PCI-Express, PCI-X2, EISA, VESA, VME, RapidIO, AGP, ISA, 3GIO, HyperTransport, Futurebus, MultiBus, or any other local bus. In this type of embodiment, the host interface126comprises a local bus interface of the local bus type. In another embodiment, the controller100is a standalone controller in a separate enclosure from the host computers that issue I/O requests to the controller100. For example, the controller100may be part of a storage area network (SAN). In this type of embodiment, the host interface126may comprise various interfaces such as Fibre Channel, Ethernet, Infiniband, SCSI, HIPPI, Token Ring, Arcnet, FDDI, LocalTalk, ESCON, FICON, ATM, SAS, SATA, and the like.

The controller100also includes a battery112for supplying power to the controller100in the event of a loss of main power, as described in more detail below. The battery is re-charged via a re-charge signal132from the main power supply. The battery112may be disabled by the backup/restore manager124(discussed below) via a disable signal136or by a micro-CPLD134(also discussed below) via a second disable signal138.

The controller100also includes a volatile memory102, or cache memory102. The volatile memory102is volatile because it ceases to store its data when it is not powered. In one embodiment, the volatile memory102comprises dynamic random access memory (DRAM), which includes a self-refresh mode. When the DRAM is placed into self-refresh mode, the DRAM consumes less power than when not operating in self-refresh mode. In other embodiments, the volatile memory102may include other types of volatile memory, such as static random access memory (SRAM).

The volatile memory102is employed by the controller100to buffer data transferred between the hosts and disks. When a host requests data be written to the disks, the controller100transfers the data from the host via the host interface126into the volatile memory102and subsequently transfers the data from the volatile memory102via the disk interface128to the disks. Conversely, when a host requests data to be read from the disks, the controller100transfers the data from the disks via the disk interface128to the volatile memory102and subsequently transfers the data from the volatile memory102via the host interface126to the host. In particular, the volatile memory102is used by the controller100to perform write-caching of data.

As mentioned above, when a host requests data be written to the disks, the controller100transfers the data from the host via the host interface126into the volatile memory102and subsequently transfers the data from the volatile memory102via the disk interface128to the disks. Normally, the controller100does not indicate to the host that the write request is complete until the data is actually written to disk. However, if configured to do so, the controller100may cache the data in the volatile memory102and indicate to the host that the write request is complete before the data is actually written to the disk, and subsequently write, or flush, the data to disk. This operation is referred to as write-caching, or may also be referred to as a posted-write operation. The data associated with a posted-write operation is referred to as posted-write data. That is, posted-write data is data stored in the volatile memory102that has not yet been written to disk but concerning which the controller100has told the host that the write operation has completed. Additionally, the posted-write data as referred to herein may refer to control information required to write the data to disk, including but not limited to, the logical block addresses and disk drive unit numbers to which the data must be written, and information specifying whether the data is part of a RAID array with a RAID level requiring redundant data that also must be written to disk to be generated based on the posted-write data.

The volatile memory102may also be used by the controller100to perform read-caching of data, i.e., to provide requested data to the hosts from the volatile memory102, rather than from the disks, if the data is already present in the volatile memory102because of a previous read request of the same data. Finally, the controller100may use the volatile memory102for buffering redundant RAID data generated for writing to the disks.

The controller100also includes a non-volatile memory104. The non-volatile memory104is non-volatile because it continues to store its data when it is not powered. In one embodiment, the non-volatile memory104comprises a Compact FLASH memory. In one embodiment, the non-volatile memory104comprises a micro-disk drive. The non-volatile memory104is used by the controller100to backup the contents, particularly the posted-write data, of the volatile memory102in response to a loss of main power so that when main power returns the posted-write data may be restored from the non-volatile memory104to the volatile memory102. However, as discussed above, the restore operation may be lengthy, and until the data is restored to the volatile memory102the user data is not available to the hosts. Advantageously, the present invention provides an apparatus and method for selectively foregoing the restore operation if it is determined that the battery was able to maintain the integrity of the posted-write data in the volatile memory102until main power returned, thereby making the user data available to the hosts sooner than if the restore operation were performed, as described in detail below.

The controller100also includes a processor108for executing programs to control the transfer of data between the disk drives and the hosts. The processor108receives commands from the hosts and responsively issues commands to the disk interface128to accomplish data transfers with the disk drives. The processor108may also perform storage controller functions such as RAID control, logical block translation, buffer management, and data caching. As discussed below in more detail, when main power is restored to the controller100after a loss of main power, the processor108selectively instructs the backup/restore manager124to perform or not to perform a restore of the volatile memory102to the non-volatile memory104based on whether the battery112was able to continue supplying power to the volatile memory102throughout the main power outage, referred to herein as a selective restore feature.

The controller100also includes a selective restore feature enable flag118. In one embodiment, the user may provide input to enable or disable the selective restore feature. The user selection is stored in the selective restore feature enable flag118. In one embodiment, the selective restore feature enable flag118is stored in a memory coupled to the processor108used for storing program instructions executed by the processor108; however, other embodiments may store the selective restore feature enable flag118in other storage locations, such as a discrete register or in the volatile memory102.

One advantage of the ability for the user to disable the selective restore feature is that it may reduce the likelihood of user data loss in the event of two main power losses within a period of time that is short relative to the battery112recharge time as follows. When a main power outage occurs, a considerable amount of the battery112power is typically used to perform the backup to the non-volatile memory104. If the selective restore feature is enabled, more of the battery112power will be consumed by powering the volatile memory102with battery112power. Once main power is restored, the battery112may be recharged, but the recharge may require a relatively long length of time—perhaps on the order of hours if the battery112has been fully discharged. If the battery112has not been recharged sufficiently to perform a backup to the non-volatile memory104when a second main power outage occurs, user data may be lost if posted-writes are pending. Thus, by enabling the user to selectively disable the selective restore feature, the user is given the choice between potentially faster data availability time or lower risk of data loss in the event of two or more main power outages within a period of time that is short relative to the battery112recharge time. The brown-out timer116, described below, gives the user even further control over this question by enabling the user to specify a length of time to consume more battery112power by continuing to power the volatile memory102after it has been backed up to the non-volatile memory104.

The controller100also includes a micro-CPLD134. The micro-CPLD134comprises a complex programmable logic device (CPLD) that consumes very low power. Although an embodiment is described employing a micro-CPLD, other circuits may be employed to perform the functions described herein that are performed by the micro-CPLD134, such as custom integrated circuits or discrete components.

The micro-CPLD134includes a data good indicator122. The data good indicator122may be read by the processor108and written by a backup/restore manager circuit124which is described below. The micro-CPLD134is configured such that when it is powered up, the data good indicator122resets to a predetermined value. In one embodiment, the data good indicator122power-up resets to a Boolean value of zero. Furthermore, the power-off threshold of the micro-CPLD134is at least as high as the power threshold at which the volatile memory102begins to lose its data. As described below, when main power is lost, the backup/restore manager124writes a value into the data good indicator122different from the power-up reset value. Thus, if the battery112fails to supply power to the volatile memory102(and therefore also fails to supply power to the data good indicator122), when main power is restored, the processor108will read the power-up reset value from the data good indicator122rather than the value written by the backup/restore manager124. Consequently, the processor108will determine that the volatile memory102must be restored from the non-volatile memory104. However, if the processor108reads from the data good indicator122the value written by the backup/restore manager124, then the processor108will determine that it can forego restoring the volatile memory102thereby making the user data available to the hosts sooner, as described in detail below.

The micro-CPLD134also includes a brown-out timer116. The brown-out timer116is a timer that may be started running by the backup/restore manager124. In one embodiment, the user may specify the expiration time of the brown-out timer116. If the selective restore feature flag118is enabled, then the battery112will be used to continue to power the volatile memory102until the brown-out timer116expires. Thus, the brown-out timer116may be used to reduce the likelihood that posted-write data will be lost in the event of back-to-back main power losses, as described above with respect to the selective restore feature flag118.

The controller100also includes a backup/restore manager and bus bridge circuit124, coupled to the processor108, micro-CPLD134, volatile memory102, non-volatile memory104, host interface126, and disk interface128. The backup/restore manager124receives a main power present signal114that indicates whether the main power supply is supplying power to the controller100. In one embodiment, the backup/restore manager and bus bridge circuit124is a custom large scale integrated circuit. The bus bridge124bridges the volatile memory102bus, the non-volatile memory104, the processor108bus (which in one embodiment is a Pentium processor bus), and the host interface126and disk interface128buses (which in one embodiment are PCI-X buses). The bus bridge124includes a memory controller for controlling the volatile memory102and the non-volatile memory104. In one embodiment, the backup/restore manager124includes a direct memory access controller (DMAC) used to copy the data from the volatile memory102to the non-volatile memory104during the backup operation. The operation of the backup/restore manager124in conjunction with the other circuit elements of the controller100will now be described with respect toFIG. 2.

Referring now toFIG. 2, a flowchart illustrating operation of the controller100ofFIG. 1is shown. Flow begins at block202.

At block202, the backup/restore manager124detects a loss of main power via the main power present signal114. In response, the backup/restore manager124causes the battery112to provide power to the other circuits of the controller100, and in particular to the backup/restore manager124, the volatile memory102, the non-volatile memory104, and the micro-CPLD134. Flow proceeds to block204.

At block204, the backup/restore manager124sets the data good indicator122to a predetermined value. The predetermined value is different from the value the data good indicator122is cleared to in response to a power-on reset. In one embodiment, the data good indicator122is a single bit in a register of the micro-CPLD134which power-on resets to a Boolean zero value, and the predetermined value to which the backup/restore manager124writes the data good indicator122is a Boolean one value. In another embodiment, the register stores a longer data word which power-on resets to a Boolean zero value, and the predetermined value to which the backup/restore manager124writes the data good indicator122is a predetermined signature value known by the software executed by the processor108. Flow proceeds to block206.

At block206, the backup/restore manager124starts the brown-out timer116running. In one embodiment, the controller100receives input from the user specifying the expiration time of the brown-out timer116, prior to the loss of main power. Flow proceeds to block208.

At block208, the backup/restore manager124backs up the volatile memory102contents to the non-volatile memory104. In one embodiment, the backup/restore manager124backs up the volatile memory102only if there is posted-write data in the volatile memory102, i.e., only if the volatile memory102is dirty. In one embodiment, the backup/restore manager124copies only the posted-write data to the non-volatile memory104. In one embodiment, the backup/restore manager124simply copies an image of the volatile memory102to the non-volatile memory104. Flow proceeds to decision block212.

At decision block212, the backup/restore manager124determines whether the selective restore feature is enabled by examining the selective restore feature enable flag118. In one embodiment, the controller100receives input from the user enabling or disabling the selective restore feature, prior to the loss of main power. If the feature is disabled, flow proceeds to block234; otherwise, flow proceeds to block214.

At block214, the backup/restore manager124places the volatile memory102into self-refresh mode via signal106in order to reduce the drain on the battery112. Flow proceeds to block216.

At block216, the backup/restore manager124disables battery power to all circuits except the volatile memory102and the micro-CPLD134to further reduce drain on the battery112. Flow proceeds to decision block218.

At decision block218, the micro-CPLD134determines whether the brown-out timer116expired. If so, flow proceeds to block236; otherwise, flow proceeds to decision block222.

At decision block222, if the battery112power runs out, flow proceeds to block238; otherwise, flow proceeds to decision block224.

At decision block224, as long as the main power is not restored, flow returns to decision block218; however, when the main power is restored, flow proceeds to decision block226.

At decision block226, the processor108boots up in response to the main power being restored, and determines whether the data good indicator122is set to the predetermined value. If so, flow proceeds to block232, thereby foregoing the restore operation; otherwise, flow proceeds to block228to perform the restore operation.

In one embodiment, the processor108makes additional determinations at decision block226to decide whether to proceed to block228to perform the restore operation, such as whether the non-volatile memory104is present; whether a backup operation to the non-volatile memory104(such as the backup started at block208) is in progress, and if not, whether the backup operation completed successfully; and whether the data backed up into the non-volatile memory104is valid. The controller100does not perform a restore from non-volatile memory104unless a non-volatile memory104is present, a backup was successfully completed, and the information backed up into the non-volatile memory104is valid. In one embodiment, the non-volatile memory104may be a field-replaceable unit; consequently, when main power is restored and the processor108reboots, if the processor108determines the non-volatile memory104is not present, no restore is performed. In one embodiment, the controller100maintains a flag in a separate small non-volatile memory (in one embodiment a CMOS NVRAM) indicating whether or not a backup of the volatile memory102to the non-volatile memory104was performed. One reason the controller100may not have performed a backup is because at the time main power was lost, the volatile memory102was not dirty with posted-write data. In one embodiment, the backup/restore manager124writes signature information into the non-volatile memory104after successful completion of a backup operation. When main power is restored and the processor108reboots, if the processor108determines the signature information in the non-volatile memory104is not good, then it is assume the posted-write data backed up to the non-volatile memory104is not valid, and does not perform a restore operation.

At block228, the processor108instructs the backup/restore manager124to restore the volatile memory102from the non-volatile memory104. That is, the backup/restore manager124copies from the non-volatile memory104to the volatile memory102the data that was backed up at block208. At this point, the user data is now available to the host computers. In one embodiment, when the processor108reboots, it takes the volatile memory102out of self-refresh mode. Flow proceeds to block232.

At block232, once the processor108has booted up, it writes the posted-write data to disk. If block232was arrived at because the data good indicator122was set to the predetermined value, i.e., if the battery112maintained the integrity of the volatile memory102such that the restore operation at block228was not performed, then the user data is available as soon as the processor108boots up. Thus, advantageously, the controller100of the present invention may potentially make the user data available to the hosts sooner than in a conventional controller that does not have the selective restore feature of the present invention by an amount equal to the restore operation time. As discussed above, the restore operation time may be significant. Flow ends at block232.

At block234, the backup/restore manager124disables the battery112power to all circuits of the controller100via disable signal136. Flow proceeds to block238.

At block236, the micro-CPLD134disables the battery112power to all circuits of the controller100via disable signal138. Flow proceeds to block238.

At block238, the loss of battery power causes the data good indicator122to be cleared. In particular, the data good indicator122no longer holds the predetermined value to which it was set at block204. Rather, when main power is restored and the micro-CPLD134experiences a power-on reset, the data good indicator122will be storing a value other than the predetermined value to which it was set at block204, thereby indicating that the battery112failed to continue to supply power to the volatile memory102, and therefore the data is no longer valid in the volatile memory102. Flow proceeds to decision block242.

At decision block242, as long as the main power is not restored, flow returns to decision block242; however, when the main power is restored, flow proceeds to decision block226.

As may be seen from the description above, unlike conventional storage controllers, the controller100of the present invention requires no additional time attributed to restoring the volatile memory102after a main power loss if the battery112power survives until main power is restored since the posted-write data in the volatile memory102is maintained by powering the volatile memory102in self-refresh mode via the battery112. Furthermore, by placing the volatile memory102in self-refresh mode, the battery112power time is increased, further increasing the likelihood that a restore from the non-volatile memory104will be unnecessary. Consequently, the present invention has both the advantage of extremely low likelihood of user data loss because it has a non-volatile memory104for backing up the posted-write data, and the advantage of fast data availability once main power is restored because it maintains battery112power to the volatile memory102during the main power outage as long as possible, and can therefore in most cases have faster boot times because no restore of the volatile memory102is required.

As used herein, the term control logic may be used to refer to the processor108, the backup/restore manager124, micro-CPLD134, individually or any combination thereof.

Although the present invention and its objects, features, and advantages have been described in detail, other embodiments are encompassed by the invention. For example, although embodiments have been described in which the storage controller is a RAID controller, the selective restore apparatus and method described herein may also be employed in any storage controller (i.e., a non-RAID controller) that uses a cache memory to post write operations to disk drives or other storage devices. Furthermore, the invention is not limited to storage controllers; rather, the selective restore feature described herein may be employed in any controller that includes a battery and a non-volatile memory for backing up data that must be maintained through a power loss, but which requires its data to be available as soon as possible after main power is restored. Finally, although an embodiment has been described including the backup/restore manager124, in other embodiments the processor108may perform the functions described herein that are performed by the backup/restore manager124; however, the processor108will likely consume more battery112power than the backup/restore manager124.

Finally, those skilled in the art should appreciate that they can readily use the disclosed conception and specific embodiments as a basis for designing or modifying other structures for carrying out the same purposes of the present invention without departing from the spirit and scope of the invention as defined by the appended claims.