Abstract:
Disclosed is a method for creating a large-scale storage array by combining multiple mid-range storage arrays via a host based aggregation engine software application. Each mid-range storage array, also call a storage building block, consists of one or more RAID volumes. Each mid-range storage array has equivalent configuration and property settings including number of drives, RAID level, volume segment sizes, and volume cache settings, but not including the volume label. The complex combination of mid-range storage arrays appears as a single storage system to a data management application of a host computer system. Once the mid-range storage arrays are aggregated into a large-scale storage array, or storage complex array, common features may be modified as a collection of items so that a common modification need only be entered one time for all items in the collection. The storage complex array also permits a management application to interact with the storage complex array as a virtual volume without the need to handle the complexities of the individual mid-range storage arrays. A separate graphical user interface application permits a system administrator to configure the aggregation engine without putting the burden of graphics and user interaction into the operation of the aggregation engine. The host based aggregation engine provides cost savings by creating a high end storage system without the need for costly specialized hardware. The aggregation engine is also scalable, permitting the addition or subtraction of mid-range storage arrays.

Description:
BACKGROUND OF THE INVENTION  
       [0001]     a. Field of the Invention  
         [0002]     The present invention generally pertains to storage systems and more particularly to a combination of Redundant Array of Independent Disks (RAID) data storage volumes.  
         [0003]     b. Description of the Background  
         [0004]     RAID storage is common high end data storage for corporate and personal computing. RAID storage permits various strategies to optimize a storage system for redundancy and/or speed, as well as minimizing trade offs between redundancy and speed. To further expand the abilities of RAID storage, a Storage Building Block (SBB) may be created as a combination of individual conventional RAID storage volumes. By combining the individual RAID volumes into a larger system, a SBB storage system may be created that is larger than individual RAID storage volumes alone.  
         [0005]     RAID storage and SBB systems utilize many technical specifications to create standard products that are capable of interoperating with other devices. Communication standards are one of the primary areas of standardization for RAID storage and SBB storage systems. Common communication standards used for RAID storage and SBB systems include: Fibre Channel, Small Computer System Interface (SCSI), Serial Attached SCSI (SAS), Serial Advanced Technology Attachment (SATA), and others. The Fibre Channel, SCSI, SAS, SATA, and many other technical specifications are kept by the American National Standards Institute (ANSI). ANSI is located at 11 West 42nd Street, 13th Floor, New York, N.Y. 10036, telephone number 212-642-4900, and web site www.ansl.org.  
       SUMMARY OF THE INVENTION  
       [0006]     An embodiment of the present invention may therefore comprise a method for creating a large-scale storage array comprising the steps of: combining multiple storage building blocks into a storage complex, the storage building blocks being a sub-group made up of at least one RAID storage volume, the RAID storage volume being a RAID storage volume made up of at least one physical hard drive, each of the storage building blocks having equivalent configuration and property settings such as number of drives, RAID level, volume segment sizes, and volume cache settings, but not including a volume label; physically connecting the storage complex to a host computer system; and managing the storage complex using aggregation engine software running on the host computer system such that the storage complex appears as a single storage system to a management application on the host computer system.  
         [0007]     An embodiment of the present invention may further comprise a large-scale storage array system comprising: a storage complex, the storage complex being a combination of multiple storage building blocks, the combination of multiple storage building blocks being a sub-group made up of at least one RAID storage volume, the RAID storage volume being a RAID storage volume made up of at least one drive, the combination of multiple storage building blocks and the RAID storage volume being physically connected to a host computer system, each of the combination of multiple storage building blocks having equivalent configuration and property settings such as number of drives, RAID level, volume segment sizes, and volume cache settings, but not including a volume label; and an aggregation engine software application that manages the storage complex such that the storage complex appears as a single storage system to a management application on the host computer system.  
         [0008]     An embodiment of the present invention may further comprise a large-scale storage array system comprising: means for combining multiple storage building blocks into a storage complex; means for physically connecting the storage complex to a host computer system; and means for managing the storage complex array using aggregation engine software running on the host computer system. 
     
    
     BRIEF DESCRIPTION OF THE DRAWINGS  
       [0009]     In the drawings,  
         [0010]      FIG. 1  is a schematic illustration of the system architecture for a large-scale storage array.  
         [0011]      FIG. 2  is a schematic illustration of the architecture of a Storage Building Block (SBB) mid-range storage array.  
         [0012]      FIG. 3  is a schematic illustration of the architecture of a Redundant Array of Independent Disks (RAID) data storage volume.  
         [0013]      FIG. 4  is a schematic illustration of the topology for a storage complex array with two storage building blocks (SBB&#39;s), eight volumes per SBB, two controllers per SBB, and two drives per volume.  
         [0014]      FIG. 5  is a table of the relationship between drive clusters and SBB volumes.  
         [0015]      FIG. 6  is a schematic illustration of the concept of a Logical Unit Number (LUN) cluster.  
         [0016]      FIG. 7  is a table of the relationship between LUN cluster numbers and LUN numbers.  
         [0017]      FIG. 8  is a state diagram of the possible operational states for a storage complex array volume. 
     
    
     DETAILED DESCRIPTION OF THE INVENTION  
       [0018]      FIG. 1  is a schematic illustration of the system architecture  100  for a large-scale storage array  120 . The large-scale storage array  120 , or storage complex array  120 , is a combination of multiple mid-range arrays  118 . A mid-range storage array  118 , or Storage Building Block (SBB)  118 , is made up of one or more conventional RAID storage volumes. The SBB  118  is an aggregation of conventional RAID storage volumes, and the storage complex array  120  is an aggregation of SBB&#39;s  118 . Hence, the storage complex array  120  is an aggregation  120  of aggregations  118  of conventional RAID storage volumes. The aggregation of aggregations permits a large-scale storage system that appears to the host  102  data application  104  as a single virtual storage volume  120  for ease of use in data storage and system management.  
         [0019]     The host computer system  102  runs the data application  104  that views the storage complex array  120  as a single virtual storage system. The aggregation engine  108  and the volume manager  106  running on the host  102  provide the ability for the system to interact with the storage complex array  120  as a single storage system. The volume manager  106  is a standard software application that interacts with the operating system of the host  102  to control the communication path aggregation for the storage complex  120 . The volume manager is a software application that may be provided by a number of volume manager software providers such as Veritas Software. Veritas Software is located at 350 Ellis Street, Mountain View, Calif. 94043, telephone number 650-527-8000, and web site www.veritas.com.  
         [0020]     The SBB&#39;s  118  communicate with the host computer system  102  using standard storage system communication protocol I/O channels  116 . The standard communication protocols include, but are not limited to: Fibre Channel, Small Computer System Interface (SCSI), Serial Attached SCSI (SAS), and Serial Advanced Technology Attachment (SATA). All SBB&#39;s  118  in a storage complex array  120  must have equivalent configuration and property settings including number of drives, RAID level, volume segment sizes, and volume cache settings, but not including the volume label.  
         [0021]     While the volume manager  102  handles the communication path aggregation, the aggregation engine provides the management and control of the storage complex array  120 . The aggregation engine  108  is the application which combines the SBB&#39;s  118  into a virtual storage complex array  120  for interaction with the array management application  114 . The aggregation engine  108  is a multi-tasking application capable of managing multiple instances of storage complex arrays  120 . The aggregation engine  108  is also capable of communicating to the multiple storage complex arrays  120  in a parallel fashion. That is, if multiple storage complex arrays  120  require the same communication message, the aggregation engine is capable of sending the communication message to multiple storage complex arrays  120  simultaneously.  
         [0022]     The number n of SBB&#39;s  118  in a storage complex array  120  is a configurable parameter of the aggregation engine  108 . Since the number of SBB&#39;s  118  is configurable, the storage complex array  120  is scalable because one may add or subtract SBB&#39;s  118  to the storage complex array  120 . Configuration of the storage complex array  120  is performed via a software array management application  114  and does not require new hardware. The array management application  114  is a separate software application from the aggregation engine  108 . The aggregation engine  108  may be written in the Java programming language and does not include any graphical interface features. The array management software  114  provides the graphical user interface to configure and manage the aggregation engine  108 . The array management application  114  may be run on the host computer system  102 , but to avoid the security and system inefficiency problems associated with graphical user interfaces, the array management application  114  is typically run on a separate management station computer  112 . The management station  112  communicates with the host computer  110  over a standard computer network connection  110 . The array management application  114  sends all management commands  110  over the network  110  to the aggregation engine  108  running on the host computer system  102 .  
         [0023]      FIG. 2  is a schematic illustration of the architecture  200  of a Storage Building Block (SBB) mid-range storage array  210 . Each SBB  210  consists of one or more RAID volumes  208 . The drives comprising the RAID volumes  208  communicate with the controller hardware  206  over controller to drive I/O communication channels  206  as specified by the controller manufacturer. The number x of controllers  204 , in combination with the number of I/O channels per controller  202 , determines the number m of RAID volumes  208  per SBB  210 . In  FIG. 2  each controller  204  supports 4 RAID volumes  208  per each controller  204 . The controllers  204  communicate with the host computer system using standard storage communication I/O channels  202  including, but not limited to: Fibre Channel, SCSI, SAS, and SATA.  
         [0024]      FIG. 3  is a schematic illustration of the architecture  300  of a Redundant Array of Independent Disks (RAID) data storage volume  306 . A RAID volume  306  consists of multiple data storage drives  304  connected in one of many RAID configurations. The RAID configuration is not important to the storage complex array. The numbery of RAID data drives determines how many drive clusters comprise a storage complex array volume. A storage complex array volume is an aggregation of RAID volumes that may consist of RAID volumes belonging to one or more SBB&#39;s. The drives comprising the RAID volumes  304  communicate  302  with the SBB controllers over controller to drive I/O channels  302  defined by the RAID system  306  manufacturers.  
         [0025]      FIG. 4  is a schematic illustration of the topology  400  for a storage complex array with two storage building blocks (SBB&#39;s)  420 ,  422 , eight volumes  412 ,  414 ,  416 ,  418  per SBB  420 ,  422 , two controllers  404  per SBB  420 ,  422 , and two drives  408 ,  410  per SBB RAID volume  412 ,  414 ,  416 ,  418 . An embodiment may utilize the SYMbol Application Programming Interface (API) when creating the aggregation engine and the array management applications. The SYMbol API is a development tool created by Engenio Information Technologies, Inc. for assisting programmers to communicate with and manage RAID storage volumes and SBB&#39;s  420 ,  422 . For information on the SYMbol API contact Engenio Information Technologies, Inc., located at 670 N. McCarthy Boulevard, Milpitas, Calif. 95035, telephone number 408-935-6300, and web site www.engenio.com. The Storage Management Initiative Specification (SMI-S) is another programming tool that may be used as an alternative to, or in addition to, the SYMbol API. For information on the Storage Management Initiative Specification (SMI-S) contact the Storage Networking Industry Association (SNIA), located at 500 Sansome Street, Suite #504, San Francisco, Calif. 94111, telephone number 415-402-0006, and website www.snia.org. Any programming tool intended to assist developers in creating storage system applications may be utilized to create an embodiment of the invention.  
         [0026]     In the system  400  shown in  FIG. 4 , the SYMbol API permits addressing a number of different attributes including volumes  412 ,  414 ,  416 ,  418 , SBB&#39;s  420 ,  422 , controllers  404 , and drive clusters  408 ,  410 . Each SBB  420 ,  422  is made up of eight RAID volumes  412 ,  414 ,  416 ,  418 . The RAID volumes each contain two data drives  408 ,  410 . Each SBB  420 ,  422  has equivalent configuration and property settings including number of drives, RAID level, volume segment sizes, and volume cache settings, but not including the volume label.  
         [0027]     SBB  1  ( 420 ) contains eight RAID volumes  412 ,  414 . The first four RAID volumes  412  are connected to controller  1  ( 424 ) of SBB  1  ( 420 ). The second four RAID volumes  414  of SBB  1  ( 420 ) are connected to controller  2  ( 426 ) of SBB  1  ( 420 ). The connection  406  from the drives comprising the RAID volumes  412 ,  414  to the controllers  404  is implemented using communication channels  406  specified by the RAID volume  412 ,  414  and controller  404  manufacturers. The controllers  404  provide I/O channels  402  to the host computer system using standard storage system communication protocols including, but not limited to: Fibre Channel, SCSI, SAS, and SATA.  
         [0028]     SBB  2  ( 422 ) contains eight RAID volumes  416 ,  418 . The first four RAID volumes  416  are connected to controller  1  ( 428 ) of SBB  2  ( 422 ). The second four RAID volumes  418  of SBB  2  ( 422 ) are connected to controller  2  ( 430 ) of SBB  2  ( 422 ). The connection  406  from the RAID volumes  416 ,  418  to the controllers  404  is implemented using communication channels  406  specified by the RAID volume  416 ,  418  and controller  404  manufacturers. The controllers  404  provide I/O channels  402  to the host computer system using standard storage system communication protocols including, but not limited to: Fibre Channel, SCSI, SAS, and SATA.  
         [0029]     A unique concept for the SYMbol API is the drive cluster  408 ,  410 . A drive cluster  408 ,  410  is an addressing mechanism that permits a developer to address drive  1 &#39;s ( 408 ) for every SBB RAID volume  412 ,  414 ,  416 ,  418  in the entire storage complex volume  400 . In  FIG. 2  the RAID volumes  412 ,  414 ,  416 ,  418  consist of two drives  408 ,  410 . Drive  1 &#39;s ( 408 ) are addressed via drive cluster  1  ( 408 ) and Drive  2 &#39;s ( 410 ) are addressed via drive cluster  2  ( 410 ).  
         [0030]     The number of volumes  412 ,  414 ,  416 ,  418  per SBB  420 ,  422 , the number of drives  408 ,  410  per volume  412 ,  414 ,  416 ,  418 , and the number of SBB&#39;s  420 ,  422  per storage complex array  400  are all configurable values and are not limited to the values shown in  FIG. 2 .  
         [0031]      FIG. 5  is a table  500  of the relationship between drive clusters  502 ,  504  and SBB RAID volumes  512 . The table  500  is a reflection of the system described with respect to  FIG. 4 . Each SBB  506 ,  508  has two controllers  510  and eight SBB RAID volumes  512 . Each SBB RAID volume  512  has two data storage drives  502 ,  504 . Drive cluster  1  ( 502 ) addresses the drive  1 &#39;s ( 502 ) for the SBB RAID volumes  512  of both SBB  1  ( 506 ) and SBB  2  ( 508 ). Drive cluster  2  ( 504 ) addresses the drive  2 &#39;s for the SBB RAID volumes  512  of both SBB 1  ( 506 ) and SBB  2  ( 508 ).  
         [0032]      FIG. 6  is a schematic illustration  600  of the concept of a Logical Unit Number (LUN) cluster. The LUN cluster ( 606 ) is another unique addressing mechanism of the SYMbol API for a storage complex. Typically there is a separate LUN for each individually addressable component of a SBB  614 ,  616 . The LUN cluster number  606  permits addressing all objects within a single LUN cluster  606  with one number  606 . In  FIG. 6  RAID volumes  1 - 8  ( 608 ) of SBB  1  ( 614 ) have LUN  0 - 7  respectively. Similarly, RAID volumes  1 - 8  ( 610 ) of SBB  2  ( 616 ) have LUN  0 - 7  respectively. The LUN cluster  606  permits addressing RAID volumes  1  to  8  ( 608 ,  610 ) of both SBB  1  ( 614 ) and SBB  2  ( 616 ) using a single LUN cluster number  606 . The LUN and LUN cluster are mapped  604  into the addressing scheme of the host computer system  602 .  
         [0033]      FIG. 7  is a table  700  of the relationship between LUN cluster numbers  702  and LUN numbers  704 . The number of LUN numbers  704  per LUN cluster  702  is equal to the number of RAID volumes per SBB. For a system with eight RAID volumes per SBB, the first eight LUN numbers  708  of each SBB are associated with LUN cluster  0  ( 706 ). The next eight LUN numbers  712  of each SBB are associated with LUN cluster  1  ( 710 ). The progression continues until the maximum number of LUN clusters is reached. The maximum number of LUN clusters is a function of the number m of RAID volumes per SBB. The maximum number of LUN clusters is equal to (256/m)−1. For example, if m is eight, then the maximum number of LUN clusters is 31. The reason for subtracting one from the number is to leave a LUN cluster number available for the Universal Transport Mechanism (UTM) LUN used with the SYMbol API.  
         [0034]      FIG. 8  is a state diagram of the possible operational states  800  for a storage complex array volume. At the beginning  802  the state initially moves to the optimal state  808 . The state of the storage complex volume as a whole is dependent on the individual state of each SBB RAID volume contained in the storage complex array volume. As long as any SBB RAID volumes do not fail or become degraded, then the state of the storage complex array volume stays optimal  808 . If a SBB RAID volume fails  804 , the state of the storage complex array volume becomes failed  810 . As long as the number of failed SBB RAID volumes is greater than or equal to one, the state of the storage complex array volume remains failed  810 . If additional SBB RAID volumes fail  812 , the state of the storage complex array remains failed  810 . If an SBB RAID volume becomes degraded  806  while other SBB RAID volumes are failed  810 , the state of the storage complex array volume remains failed  810 . If the failed SBB RAID volumes become optimal  816 , the state of the system is returned to the historical state  824  that represents the prior degraded or optimal states of the SBB RAID volumes. If the system is returned to an optimal state  808  and a SBB RAID volume becomes degraded  818 , then the state of the storage complex array volume becomes degraded  822 . In the degraded state  822  there are not any SBB RAID volumes that have a failed state and there are one or more SBB RAID volumes that have a degraded state. If additional SBB RAID volumes become degraded  826 , the state of the storage complex volume remains degraded  822 . If a SBB RAID volume fails  820 , the storage complex volume state changes from degraded  822  to failed  810  and follows the logic associated with the failed state  810  discussed previously. If there is not a failed SBB RAID volume and all degraded SBB RAID volumes become optimal  814 , the state of the storage complex volume is returned to optimal  808 .  
         [0035]     Various embodiments therefore provide the ability to create a high end storage system by providing a host based software aggregation engine that permits a user to avoid the cost of specialized hardware. The aggregation engine further permits the system to be scalable by adding or removing mid-range storage arrays. The aggregation engine will typically be integrated with a volume manager application of an operating system to provide greater functionality than the volume manager or aggregation engine provide alone. The array management application provides a familiar graphical user interface for the aggregation engine. The array management application may be run remotely, thus, permitting a host to operate the aggregation engine without the burden of handling the graphics and user interaction associated with a graphical user interface.  
         [0036]     The foregoing description of the invention has been presented for purposes of illustration and description. It is not intended to be exhaustive or to limit the invention to the precise form disclosed, and other modifications and variations may be possible in light of the above teachings. The embodiment was chosen and described in order to best explain the principles of the invention and its practical application to thereby enable others skilled in the art to best utilize the invention in various embodiments and various modifications as are suited to the particular use contemplated. It is intended that the appended claims be construed to include other alternative embodiments of the invention except insofar as limited by the prior art.