Abstract:
A system and method providing interrupted write protection to a stand-alone commodity storage array utilized within a database system. The method identifies writes of data from the database system to the storage array requiring interrupt protection, and for each write, generates an entry for an intent log maintained within the database system. The log entries include a write identifier, storage device information associated with a write, and a write statues indication which are used to identify failed writes following a database system failure.

Description:
CROSS REFERENCE TO RELATED APPLICATIONS 
     This application claims priority under 35 U.S.C. §119(e) to the following and commonly-assigned patent application, which is incorporated herein by reference: 
     Provisional Patent Application Ser. No. 61/922,544, entitled “TORN WRITE PROTECTION WITH GENERIC STORAGE,” filed on Dec. 31, 2013, by Gary Lee Boggs. 
    
    
     FIELD OF THE INVENTION 
     The present invention relates to a database system and stand-alone storage array including a device driver and daemon that provide interrupted write protection for database data. 
     BACKGROUND OF THE INVENTION 
     Within a database system, device and software failures, system resets, and power losses can result in unsuccessful write operations. During multi-sector writes, it is possible for a data transfer to be interrupted by software failures such as a node software “panic” or hardware failures such as node processor or memory errors, adapter failures, cable failures or loss of power to the system. This can result in partially written data, with some new sectors and some old sectors in the area that was to be written. This failure is known as an interrupted, a subset of unsuccessful writes. 
       FIG. 1  illustrates an interrupted database write operation. Referring to  FIG. 1 , a database write operation, WRITE DATA  101 , is issued by database system  100  to storage subsystem  103  to write data to data block  107  within data storage device  105 . A hardware or software failure, or other event, results in a successful write of only a portion  107 A of data block  107 , while the write to a portion  107 B of the data block is unsuccessful. 
     Teradata Corporation database systems have employed HDD and SSD disk array systems from vendors that incorporate interrupted write protection into their products. In these implementations, when data in the interrupted write area is read, a special pattern is returned instead of the data. The database system can detect this pattern and optimize its recovery. That is, it can distinguish the special case of an incomplete write from that of general corruption, which is what the interrupted write would have looked like had the actual data been returned instead of the special pattern. This interrupted write protection is not available when using commodity storage such as direct attached disks or software RAID. These direct attached disks or software RAID devices are generic products which usually do not include interrupted write detection. 
     A system and method for providing interrupted write protection to a stand-alone commodity storage array is described below. 
    
    
     
       BRIEF DESCRIPTION OF THE DRAWINGS 
         FIG. 1  is a simple block diagram illustrating a interrupted write operation. 
         FIG. 2  is a block diagram of a node of a database system. 
         FIG. 3  is a block diagram of a parsing engine. 
         FIG. 4  is a flow chart of a parser. 
         FIG. 5  is a flow chart illustrating the operation of a device driver and daemon to provide interrupted write protection to a stand-alone commodity storage array utilized within a database system, in accordance with the present invention. 
         FIG. 6  is a block diagram of a database node and commodity storage subsystem including a device driver and daemon above the commodity storage subsystem providing interrupted write protection for database data in accordance with the present invention. 
     
    
    
     DETAILED DESCRIPTION OF THE INVENTION 
     In the following description, reference is made to the accompanying drawings that form a part hereof, and in which is shown by way of illustration specific embodiments in which the invention may be practiced. These embodiments are described in sufficient detail to enable one of ordinary skill in the art to practice the invention, and it is to be understood that other embodiments may be utilized and that structural, logical, optical, and electrical changes may be made without departing from the scope of the present invention. The following description is, therefore, not to be taken in a limited sense, and the scope of the present invention is defined by the appended claims. 
     Environment 
     The technique for providing interrupted write protection to a commodity storage array disclosed herein has particular application, but is not limited, to large databases that might contain many millions or billions of records managed by a database system (“DBS”)  200 , such as a Teradata Active Data Warehousing System available from Teradata Corporation.  FIG. 2  shows a sample architecture for one node  205   1  of the DBS  200 . The DBS node  205   1  includes one or more processing modules  201   1 . . . N , connected by a network  215 , that manage the storage and retrieval of data in data-storage facilities  220   1 . . . N . Each of the processing modules  210   1 . . . N  may be one or more physical processors or each may be a virtual processor, with one or more virtual processors running on one or more physical processors. 
     For the case in which one or more virtual processors are running on a single physical processor, the single physical processor swaps between the set of N virtual processors. 
     For the case in which N virtual processors are running on an M-processor node, the node&#39;s operating system schedules the N virtual processors to run on its set of M physical processors. If there are 4 virtual processors and 4 physical processors, then typically each virtual processor would run on its own physical processor. If there are 8 virtual processors and 4 physical processors, the operating system would schedule the 8 virtual processors against the 4 physical processors, in which case swapping of the virtual processors would occur. 
     Each of the processing modules  210   1 . . . N  manages a portion of a database that is stored in a corresponding one of the data-storage facilities  220   1 . . . N . Each of the data-storage facilities  220   1 . . . N  includes one or more disk drives. The DBS may include multiple nodes  205   2 . . . P  in addition to the illustrated node  205   1 , connected by extending the network  215 . 
     The system stores data in one or more tables in the data-storage facilities  22   1 . . . N . The rows  225   1 . . . Z  of the tables are stored across multiple data-storage facilities  220   1 . . . N  to ensure that the system workload is distributed evenly across the processing modules  210   1 . . . N . A parsing engine  230  organizes the storage of data and the distribution of table rows  225   1 . . . Z  among the processing modules  210   1 . . . N . The parsing engine  230  also coordinates the retrieval of data from the data-storage facilities  220   1 . . . N  in response to queries received from a user at a mainframe  235  or a client computer  240 . The DBS  200  usually receives queries and commands to build tables in a standard format, such as SQL. 
     In one implementation, the rows  225   1 . . . Z  are distributed across the data-storage facilities  220   1 . . . N  by the parsing engine  230  in accordance with their primary index. The primary index defines the columns of the rows that are used for calculating a hash value. The function that produces the hash value from the values in the columns specified by the primary index is called the hash function. Some portion, possibly the entirety, of the hash value is designated a “hash bucket”. The hash buckets are assigned to data-storage facilities  220   1 . . . N  and associated processing modules  210   1 . . . N  by a hash bucket map. The characteristics of the columns chosen for the primary index determine how evenly the rows are distributed. 
     In one example system, the parsing engine  230  is made up of three components: a session control  300 , a parser  305 , and a dispatcher  310 , as shown in  FIG. 3 . The session control  300  provides the logon and logoff function. It accepts a request for authorization to access the database, verifies it, and then either allows or disallows the access. 
     Once the session control  300  allows a session to begin, a user may submit a SQL request, which is routed to the parser  305 . As illustrated in  FIG. 4 , the parser  305  interprets the SQL request (block  400 ), checks it for proper SQL syntax (block  405 ), evaluates it semantically (block  410 ), and consults a data dictionary to ensure that all of the objects specified in the SQL request actually exist and that the user has the authority to perform the request (block  415 ). Finally, the parser  305  runs an optimizer (block  420 ), which develops the least expensive plan to perform the request. 
     Interrupted Write Protection for Commodity Storage Array 
     The present invention is a design for an IWIL device driver that is inserted into the storage device stack—the chain of attached device objects that represent a device&#39;s storage device drivers—and a user-space process, an IWIL daemon, that handles recovery and remote requests. For example, on Linux, the IWIL driver could be a separate block device driver or a module that provides an interface to an existing driver, such as the Teradata Virtual Storage Extent Driver, depending on the requirements of the implementation. 
     When a write needing interrupted write protection, as determined by the application layers above, occurs, it is routed through the IWIL driver. The IWIL driver and daemon perform the following actions, as illustrated in  FIG. 5 : 
     Step  501 : The driver generates an intent log entry representing the write. 
     Minimally, this entry contains:
         a. The device identifier of the destination device, such as the Linux dev_t or the TVS global device id.   b. The block offset on the device of the write.   c. The length of the write.   d. A checksum of the data to be written. For implementations supporting T-10 DIF/DIX, the T10 checksum or IP checksum can be used. For other implementations, a checksum can be calculated using, for example, the CRC32 instruction of the Intel CPU or a software checksum implementation.   e. An identifying stamp of the form (node id, sequence number) which is used to determine the order of the entries when reading the intent logs following a crash.   f. A status indication (active).
 
Step  502 : The driver writes the intent log entry to local non-volatile storage.
       

     Ideally, this is non-volatile memory in the node, such as a flash card, but may also be a locally attached solid state disk (SSD) or hard disk (HDD). 
     In parallel with the write to local non-volatile storage, the driver sends the intent log entry to at least one other node in the system via a network connection such as Infiniband, Ethernet or bynet. This is done in case the original node does not come back after a crash and the application, e.g., Teradata AMP, moves to another node.
         a. For implementations that support RDMA and use non-volatile memory, the driver can initiate a transfer directly into the non-volatile memory of the remote node to an area reserved for use by the initiating node.   b. For other implementations, a message is sent to the IWIL daemon on the target node which will perform the write.
 
Step  503 : Once step  502  completes, the original data write is allowed to continue.
 
Step  504 : Once the data write completes, step  502  is repeated with the same device, offset, and length, but a new, higher, sequence number and a status of “inactive”. An optimization is to allow the write completion to be communicated to the application prior to the intent log writes completing, and to batch these completions together to minimize network and I/O traffic.
 
Step  505 : Following a crash, the IWIL daemon obtains the intent logs from the local node&#39;s non-volatile storage and from any of the other nodes with which it can communicate. The IWIL daemon groups the log entries by device and offset. For each device and offset, the daemon determines the entry with the highest sequence number for each sector range, assuming that only one node will be writing to a given device and sector range, and from this entry the daemon determines the status of the write.
 
Step  506 : Any writes identified as having active entries in the log are considered potential interrupted writes. The daemon reads the sectors covered by a potential interrupted write and computes a checksum. If the checksum matches that in the log entry, the write is identified as completed and no further action need be taken. If the checksum does not match, the write is considered to be an interrupted write and this status will be communicated to the application on the next read of the affected sectors. This can be done in a couple of different ways, depending on requirements:
   a. The daemon could write the special interrupted write pattern over the affected sector range.   b. The device and block range could be uploaded to another driver which monitors reads and that driver could return the interrupted write pattern when a matching read is detected.       

       FIG. 6  shows a block diagram of a database system node and commodity storage subsystem, including a device driver and daemon above the commodity storage subsystem providing interrupted write protection for database data. 
     Referring to  FIG. 6 , a Teradata Trusted Parallel Application (TPA) node  601  and commodity storage subsystem  603  and storage drives  605  are shown. A TPA node is a database node that manages a Teradata database. 
     TPA node  601  includes existing software components TVSAEXT  607 , a the Teradata Virtual Storage Extent Driver; a Linux Block &amp; SCSI driver  609 ; and Storage Interconnect  611 . Linux Block &amp; SCSI driver  609  and Storage Interconnect  611  are parts of the Linux operating system and are the drivers used to access disk storage. Generic interrupted write protection if provided by the IWIL driver  613 . The arrow between IWIL driver  613  and TVSAEXT  607  is intended to show a call interface, where the TVSAEXT driver  607  tells IWIL driver  613  that a particular write needs interrupted write protection. For non-Teradata implementations, a block driver could be constructed to replace TVSAEXT driver  607  in  FIG. 6 . 
     Non-volatile Random Access Memory (NVRAM) or Local SSD  615  provides physical storage for the intent log, either non-volatile memory in the node or on a solid state drive (SSD) attached to the node. The intent log entries written to NVRAM or Local SSD  615  are copied to another node over a network, such as infiniband, so that the log entries are not lost if the node  601  crashes. 
     Storage Subsystem  603  and disks  605  represent a generic disk storage subsystem such as a disk array, a plurality of raw disks (JBOD for Just a Bunch of Disks, or a software RAID. A software RAID would be a mirroring or other RAID (redundant array of inexpensive disks) implementation in software that does not use a disk array controller. 
     The figures and specification illustrate and describe a method for providing interrupted write protection to a stand-alone commodity storage array utilized within a database system. 
     The foregoing description of the invention has been presented for purposes of illustration and description, it is not intended to be exhaustive or to limit the invention to the precise form disclosed. Accordingly, this invention is intended to embrace all alternatives, modifications, equivalents, and variations that fall within the spirit and broad scope of the attached claims