Patent Publication Number: US-6993625-B2

Title: Load balancing storage system

Description:
The present application is a continuation of application Ser. No. 09/836,458, filed Apr. 18, 2001, now U.S. Pat. No. 6,763,438, the contents of which are incorporated herein by reference. 
    
    
     BACKGROUND OF THE INVENTION 
     The present invention relates to a storage system operating in communication with a host unit; and, more particularly, the invention relates to the connecting configuration of recording media having the structure of a loop connecting plural recording media. 
     Recently, with the increased interest in the concept of a Storage Area Network (SAN), the loop connection using an optical fiber channel is rapidly being adopted as an interface to connect a host unit with various devices. The structure of the connections between components inside a disk array unit, which does not use a bus connection, but uses a loop connection having a fiber optic channel, is becoming popular (for example, a loop connection between a controller and recording media using a fiber optic channel). For instance, Japanese patent publication No. Kokai 11-338646 discloses a disk array unit which does not use a bus, but the switching network structure for the connection between components. 
     SUMMARY OF THE INVENTION 
     The above-described patent publication is directed to the connection between the components with a single network. In a storage system connecting components with plural loops, such as this network, the following considerations will be important for mounting and setting up the recording media. 
     1. In case the storage system is a disk array system, the data from the host unit is distributed, generated and relocated to the data by the Redundant Array of Inexpensive (Independent) Disks (RAID) method. The data from the host unit is also recorded dispersedly into plural recording media which are set as a RAID group. Therefore, it is important to optimize the access paths and the load balancing to each recording medium. 2. There are plural loops of the recording media. Therefore, it is important to optimize the access path and the load balancing to these loops. 
     However, the above-described patent publication discloses nothing about the optimization of the access paths and the load balancing in connecting the recording media with multipath loops for the recording media. 
     As a method of load balancing optimally, it is desirable to mount the recording media at an optimum location, considering the loop for the recording medium, at the time of mounting the recording medium. In addition, at the time of forming a RAID group with the mounted recording media, it is desirable to select the recording media connected to the optimum loop for the recording media in order to build the RAID group. However, the selection of the mounting location or the selection of the recording media while taking into consideration the problem of load balancing is difficult for a person other than those who are familiar with the internal structure of the controller, such as the connecting location of the loop. Even when an operator who is in charge of the mounting and the setting executes such operation, there is a high possibility that such person will fail in distributing the recording media with a uniform load. 
     The object of the present invention is to provide a recording media mounting system that enables the adequate setting of access paths and load balancing without specifically considering the mounting location. 
     As a configuration to attain the above-described object, the storage system of the present invention consists of two or more recording media storing data received from a host unit, a controller controlling the data transfer between the recording media and the host unit, and plural loops, located between the controller and the recording media, connecting the different recording media. A recording medium is connected to an adjacent different loop than that of another recording medium, and the number of the recording media connected to each loop can be equalized by simply connecting the recording media to successive loops in turn. 
    
    
     
       BRIEF DESCRIPTION OF THE DRAWINGS 
         FIG. 1  is a block diagram of the total structure of the present invention. 
         FIG. 2  is a block diagram of the configuration inside a disk controller. 
         FIG. 3  is a schematic diagram illustrating an example of the connection of hard disk drives with-loops. 
         FIG. 4  is a schematic diagram illustrating an example of the connection of hard disk drives with loops. 
         FIG. 5  is a schematic diagram illustrating an example of the connection of hard disk drives with loops. 
         FIG. 6  is a schematic diagram illustrating an example of the connection of the loops with a multiplicity of disk drives. 
         FIG. 7  is a perspective view of a disk storage subsystem with the front panel removed. 
     
    
    
     DETAILED DESCRIPTION OF THE INVENTION 
     A preferred embodiment of the present invention will be described with reference to the drawings. 
       FIG. 1  is an example of a configuration of the storage system, which uses hard disk-drives as the recording medium, in accordance with an embodiment of the present invention. Hereafter, in a disk array system where both the connections with a host unit and the connections with a group of recording media are optical fiber channel connections, a loop for performing communication and data transfer by connecting the controller with the group of recording media will be referred to as a back-end loop, and a loop for connecting with a host unit(s) will be referred to as a front-end loop for purposes of description. 
     In  FIG. 1 , a disk controller  112  is provided with a port  111 , which is an interface to connect the disk controller with host units  100 - 1 ,  100 - 2  and  100 - 3 , that operate as host units or central processing units, through a Storage Area Network (SAN), and ports  150  and  151 , that serve as interfaces to connect the disk controller  112  to back-end loops  140  and  141 , respectively, through which the disk controller is connected with the hard disk drives  120  that operate as the recording media. 
     The disk controller  112  controls the transfer of data written from the host units  100  to the hard disk drives  120 , or the transfer of data read from the hard disk drives  120  to the host units  100 . In a word, the disk controller  112  controls the whole disk storage subsystem  110 . 
     This disk storage subsystem  110  consists of the hard disk drives  120 - 0  through  120 - 9 ; connecting bays  130 - 0  through  130 - 9 , which function as connectors connecting the hard disk drives detachably with the back-end loops that are formed by the optical fiber channels; the back-end loops  140  and  141 , which connect the hard disk drives to the disk controller; and the disk controller  112  itself for controlling the hard disk drives. The back-end loops- 140  and  141  are electrically connected through the connecting bays  130 - 0  through  130 - 9  with the hard disk drives  120 - 0  through  120 - 9 . It is possible to connect the hard disk drives  120 - 0  through  120 - 9  to the connecting bays  130 - 0  through  130 - 9  independently, and it is not necessary to mount connecting drives in all of the connecting bays. 
     The disk storage subsystem  110  uses the loop interfaces of the optical fiber channels for all connections of the back-end loops  140  and  141 , the connecting bays  130 - 0  through  130 - 9 , the hard disk drives  120 - 0  through  120 - 9 , and the interfaces  150  and − 151 . It is needless to say that the back-end loops  140  and  141 , that operate as communication paths, are not limited to optical fiber channels. 
     The back-end loop is illustrated in the figure as an ellipse and with the respective lines from the ellipse extending to respective ones of the connecting bays  130 . In the figure, though each connection is shown by a line for ease of description, the connection from the ellipse to the connecting bay  130  also forms a loop which is electrically detachable with respect to the loop indicated by the ellipse. In the following description, each of the loops is shown by one line, except for the loops in FIG.  3 .  FIG. 3  shows the loop as a pair of lines. 
     Here, the port  150  is connected with the back-end loop  140 , and the port  151  is connected with the back-end loop  141 . The connecting bays  130 - 0 ,  130 - 2 ,  130 - 4 ,  130 - 6  and  130 - 8  are connected with the back-end loop  140 , and the connecting bays  130 - 1 ,  130 - 3 ,  130 - 5 ,  130 - 7  and  130 - 9  are connected with the back-end loop  141 . The mounting position of each connecting bay alternately connected with each back-end loop is, for example, mounted from the left side sequentially in the order of the consecutive connecting bay number, as shown in the figure. Therefore, in case of mounting the hard disk drives  120  to each connecting bay, regardless of the number of hard disk drives, it is easy to alternately allocate the hard disk drives  120  to the back-end loops  140  and  141  by mounting the hard disk drives  120  in the ascending order from the location of the connecting bay  130 - 0  or by mounting the hard disk drives  120  in the descending order from the location of the connecting bay  130 - 9 . Therefore, the number of hard disk drives connected to each back-end loop can be shared equally. Connecting the hard disk drives  120  to each of the back-end loops  140  and  141  alternately is a method which results in optimal distribution of the load to the throughput of the back-end loop. 
       FIG. 2  shows the outline of the controller  112  of the disk storage subsystem  110 . The controller  112  comprises a port  111  connected in communication with a host unit  100  to control protocol processing and data transfer with the host unit  100 ; ports  150  and  151  connected to the hard disk drive group  120  to control protocol processing and data transfer with the hard disk drive  120 ; a cache memory  202  for storing the transferred data temporarily; the CPU  203  that operate as a processor for controlling each of these elements and the entire disk controller  112 ; and a bus  201  which provides data transfer between the ports and the cache memory and command transfer between the ports and the CPU. 
     When the CPU  203  of the disk storage subsystem  110  receives a data write instruction from the host unit  100 , the CPU  203  temporarily stores the data sent from the host unit  100  via the port  111  in the cache memory  202  through the bus  201 . The CPU  203  reports the end of the write operation to the host unit when the CPU  203  completes the storage of the data in the cache memory  202 . Then, the processor in the port  150  or  151  writes the data stored in the cache memory  202  into the hard disk drive group  120  while being controlled with RAID technology. 
     When the CPU receives the data read instruction from the host unit  100 , the CPU refers to the information managing the cache memory and decides whether the instructed data is stored in the cache memory  202 . If the instructed data is stored in the cache memory  202 , the CPU transfers the data to the host unit  100  through the port  111 . If the data is not stored in the cache memory  202 , the CPU copies the data from the hard disk drives  120  to the cache memory  202 , and then transfers the data from the cache memory  202  to the host unit  100  via the port  111 . 
     For ease of description, an example of a single host unit  100  is shown; however, it is possible to connect the port  111  to the SAN as shown in  FIG. 1 , and it is also possible to connect the ports  111  to two or more host units provided with two or more ports  111 . The disk controller  112  can be not only bus connected, but it can also connect the cache memory to each interface through an independent path. 
     Next, with reference to  FIG. 3 , a system comprising two RAID groups  160  and  161  will be explained, one of which is assigned four hard disk drives and the other of which is assigned two hard disk drives. 
     First, the mounting method will be explained. In a disk storage subsystem  110  shown in  FIG. 3 , the connecting bays  130 - 0 ,  130 - 3 ,  130 - 4 ,  130 - 7  and  130 - 8  are connected to the back-end loop  140 , and the connecting bays  130 - 1 ,  130 - 2 ,  130 - 5 ,  130 - 6  and  130 - 9  are connected to the back-end loop  141 . Forming pairs of hard disk drives  120 , except for other parts, can decrease the number of loops from the back-end loop to the connecting bay. 
     With such a configuration of the loops and the connecting bays, mounting the hard disk drives  120  from the connecting bay  130 - 0  in the ascending order or from the connecting bay  130 - 9  in the descending order allows the fiber channel loops  140  and  141  to have the same number of the hard disk drives  120 , when the total number is even, and to have the numbers of the hard disk drives  120  different by one drive, when the total number is odd. The load is balanced among the back-end loops. It is needless to say that the configuration of pairs of the hard disk drives  120 , including the other parts, can provide an effectively balanced load. 
     Similarly to  FIG. 1 , the load can be balanced with the connection of the connecting bays  130 - 0 ,  130 - 2 ,  130 - 4 ,  130 - 6  and  130 - 8  to the back-end loop  140  and the connection of the connecting bays  130 - 1 ,  130 - 3 ,  130 - 5 ,  130 - 7  and  130 - 9  to the back-end loop  141 . 
     Next, the configuration of the RAID groups  160  and  161 , will be described. Four hard disk drives required by the RAID group  160  are allocated from the connecting bay  130 - 0  in the ascending order. Here, the hard disk drives are allocated alternately to the back-end loops  140  and  141 , respectively. Similarly, a pair of required hard disk drives are allocated from the connecting bay  130 - 4  in the ascending order to the RAID group  161 , and then the load is balanced among the back-end loops. 
     In this preferred embodiment of the present invention, a simple operation of allocating the required number of the hard disk drives from the connecting bay  130 - 0  in the ascending order can realize a mounting of the hard disk drives and a load balancing among the back-end loops of the hard disk drives inside the RAID group. 
     Next, the preferred embodiment of the expansion of the back-end loops, the connecting bays, and the hard disk drives will be explained with reference to FIG.  4 . The components of the disk storage subsystem  110  are the components used in FIG.  1  and the disk storage subsystem  110  is connected with an extension unit  1110  containing back-end loops  142  and  143 , connecting bays  131 - 0  through  131 - 9 , and hard disk drives  121 - 0  through  121 - 9 . The extension unit  1110  is connected with the disk storage subsystem  110  through the connection of the back-end loop  142  with the back-end loop  140  and the back-end loop  143  with the back-end loop  141 . For the ease of description, the figure shows the connection between the back-end loop in the disk storage subsystem  110  and the back-end loop in the extension unit  1110  as the connection of a single line; however, another loop is formed with the back-end loop in the disk storage subsystem  110  and the back-end loop in the extension unit  1110 . 
     In this case, the back-end loop and the connecting bay are connected as follows: the connecting bays  131 - 0 ,  131 - 2 ,  131 - 4 ,  131 - 6  and  131 - 8  connect to the back-end loop  142 , and the connecting bays  131 - 1 ,  131 - 3 ,  131 - 5 ,  131 - 7  and  131 - 9  connect to the back-end loop  143 . In this case, the above-described connection makes it possible to count or set up the hard disk drives  120  to each connecting bay. 
     The following describes an example of a configuration of two RAID groups. One RAID group is allocated with twelve of the hard disk drives  120  and the other RAID group is allocated with four of the hard disk drives  120 . 
     The RAID group is mounted with the necessary twelve hard disk drives from the connection bay  130 - 0  in the ascending order. In a case where the hard disk drives  120  cannot be connected due to the failure of the connecting bay  130 - 0  or some other reason, the mounting of the hard disk drives  120  does not have to start from the connecting bay  130 - 0 , but it can start from any other of the connecting bays  130  sequentially. The following describes the configuration of the RAID groups  162  and  163 . The twelve hard disk drives  120  used for the RAID group  162  are sequentially allocated from the connecting bay  130 - 0  or any mounted connecting bay. 
     The sequential allocation from the connecting bays allocates the hard disk drives  120  alternately to the connected group of the back-end loops  140  and  142  and the connected group of the back-end loops  141  and  143 ; therefore, the load can be balanced between the back-end loops. As in the case of the RAID group  162 , the RAID group  163  can be allocated with the four necessary hard disk drives  120  from the connecting bay  131 - 2  sequentially to balance the load between the back-end loops. In this preferred embodiment of the present invention, a simple rule of sequentially allocating the necessary number of hard disk drives  120  from the connecting bay  130  can realize a mounting of a hard disk drives  120  and the load balancing among the loops of the hard disk drives inside the RAID group. 
       FIG. 5  shows the preferred embodiment of the present invention for a disk storage subsystem provided with three back-end loops. The components of the disk storage subsystem  110  are similar to the components of the disk storage subsystem used in FIG.  1 . The main difference from the preferred embodiment of  FIG. 1  is that there are three systems in the connection between ports and back-end loops, and between back-end loops and connecting bays. Each of the ports  150 ,  151  and  152  is connected with the back-end loops  140 ,  141  and  144 , respectively, the back-end loop  140  is connected with the connecting bays  130 - 0 ,  130 - 3 ,  130 - 6  and  130 - 9 , the back-end loop  141  is connected with the connecting bays  130 - 1 ,  130 - 4  and  130 - 7  and the back-end loop  144  is connected with the connecting bays  130 - 2 ,  130 - 5  and  130 - 8 . 
     Mounting the hard disk drives  120  to each of the connecting bays  130  connects the adjacent hard disk drives  120  to a different back-end loop and port. In this preferred embodiment, the theory of the load balancing is similar to the case of the two back-end loops. Three back-end loops will reduce the amount of the load per one back-end loop as compared with two back-end loops. 
       FIG. 6  shows an example of a storage system provided with plural controllers and loops. The data is generally transferred between the back-end loop  140  or  141  and each of the connecting bays  130  through the optical fiber channel shown with a solid line in FIG.  6 . That is, the data transfer under normal condition is as in the case of FIG.  1 . In a case where a failure has occurred in the optical fiber channel between the back-end loop and each of the connecting bays  130 , the data is transferred through the optical fiber channel shown with a dotted line using a different back-end loop from the normal one. 
     To be more specific, the normal connection route for the hard disk drive  120 - 1  is from the host unit  100  through the port  111  and the port  151  and via the back-end loop  141  to the connecting bay  130 - 1 ; however, in case of a failure between the back-end loop  141  and the connecting bay  130 - 1 , the connection route for the hard disk drive  120 - 1  will be changed to another route from the host unit  100  through the port  111 , the port  150 , and the back-end loop  140  to the connecting bay  130 - 1 , so that the data can be transferred without giving the user any reduction of the failure. 
     In case of a failure in one port or in one back-end loop, the other port and the other back-end loop make data transfer possible between the host unit  100  and the hard disk drive  120 . 
     Though the above-described preferred embodiments of the present invention have been explained with reference to a hard disk drive as a recording medium, the recording medium can be an optical disk or a magneto-optical disk, or can be either of a tape unit or a card type memory, such as a flash memory. 
     The above-described disk storage subsystem  110  has, for instance, the external appearance as shown in FIG.  7 . The disk storage subsystem  110  has a rectangular frame  301 , a front panel  302  with ventilating holes, which is located in front of the disk storage subsystem  110  and covers the front surface, and the connecting bays  130 - 0  through  130 - 9  which are arranged in numerical order to mount the hard disk drives  120  on the upper side of the front panel  302 . Mounting into these connecting bays  130  allows the hard disk drives to be located in a row consisting of ten independent drives in sequence from  120 - 0  through  120 - 9 . 
     The extension unit  1110  explained with reference to  FIG. 4  has a shape for accommodating the connecting bays  131  for mounting the hard disk drives  121  on the upper side of the frame  301 , and the back-end loop on the disk array unit  110  can be connected with the back-end loop on the extension unit  1110  on the rear panel shown in FIG.  7 . 
     Therefore, the hard disk drives can be easily expanded by stacking the extension unit  1110  and the disk storage subsystem  110 . In stacking the units, it is possible to mount the disk storage subsystem  110  and the extension unit  1110  into a rack, or to cover the disk storage subsystem  110  and the extension units  1110  with a frame having a size according to the number of the hard disk drives the user desires. 
     According to the present invention, as described above, in the disk storage subsystem provided with two or more back-end loops for connecting the hard disk drives, the load balancing in each loop and the setting work for mounting can be facilitated by connecting the back-end loops to the connecting bays which mount the hard disk drives with an adequate pattern so that the connecting bays conform to the mounting order of the hard disk drives.