Storage system for back-end communications with other storage system

In a first and second storage system, first and second switching apparatuses are interposed between a first and second controller and a storage device. At least one of the first and second switching apparatuses comprises a shared memory. The constitution is such that the first and second controllers communicate with one another via this shared memory.

CROSS-REFERENCE TO PRIOR APPLICATION

This application relates to and claims the benefit of priority from Japanese Patent Application No. 2006-278566 filed on Oct. 12, 2006, the entire disclosure of which is incorporated herein by reference. This application relates to Japanese Patent Application No. 2006-278237 filed on Oct. 12, 2006, the entire disclosure of which is incorporated herein by reference.

BACKGROUND

The present invention relates to a storage system that is connected to another storage system.

A first storage system is connected to a second storage system, for example, when data stored in a first storage system is migrated to a second storage system. The technology disclosed in Patent Literature 1 (Japanese Laid-open Patent No. 2004-102374) is an example of data migration technology.

For example, as a first storage system, a storage system comprising a first controller, a plurality of (or one) first storage devices, and a first switching apparatus interposed between the first controller and the first storage devices can be considered. In this case, the constitution can be such that the first controller receives an input/output request (write request/read request) sent from a first upper-level device (for example, a host computer), and, in accordance with the received input/output request, carries out a data write and/or read to any one of the above-mentioned plurality of first storage devices via the first switching apparatus. Further, as a second storage system, a storage system comprising the same constitution as the first storage system can be considered (For components that are the same as those of the first storage system, the term “first” is re-read as “second.”)

For example, technology, which directly links first and second controllers via a leased line (for example, cable) for inter-controller communications, can be considered as one inter-storage system communications technology. There are times when another, different communications technology is desirable. Further, the interconnection of storage systems is not limited to instances of data migration; other types of cases are also possible.

SUMMARY

Therefore, an object of the present invention is to provide a novel communications technology between storage systems.

Other objects of the present invention will become clear from the explanations that follow.

A shared memory is provided in at least one of the first and second switching apparatuses. The constitution is such that the first and second controllers communicate with one another via this shared memory.

More specifically, for example, the switching apparatus comprising this shared memory comprises a switching controller (for example a microprocessor) for receiving an update command and/or a reference command for updating and/or referencing the shared memory from a plurality of command issuing units (for example, SCSI initiators). When a controller sends a message to the other controller, it sends an update command together with this message to the switching controller, thereby enabling the message to be written to the shared memory. Similarly, when a controller receives a message from the other controller, the controller sends a reference command to the switching controller, thereby enabling the message to be read from the shared memory.

DESCRIPTION OF THE PREFERRED EMBODIMENT

The concept of an embodiment of the present invention will be explained.

In a first and second storage system, a first and second switching apparatus are interposed between a first and second controller and a storage device. A shared memory is provided in at least one of the first and second switching apparatuses. The constitution is such that the first and second controllers communicate with one another via this shared memory.

FIG. 20Ashows a specific example. The switching apparatus, for example, is a SAS (Serial Attached SCSI) Expander. A shared memory is provided in the Expander. The Expander is constituted having an SES target so as to be able to receive an SES (SCSI Enclosure Services) command, which conforms to SCSI specifications, from a plurality of initiators A and B (for example, a first and second controller). As SES commands, there is a command for referencing the internal state of a SES target, and a command for updating this internal state. Using this internal state as a shared memory enables a message to be exchanged between the plurality of initiators A and B via this shared memory.

Further, as shown inFIG. 20B, dedicated data migration ports independent of the host connection ports (ports to which a host computer is connected) are provided in the switching apparatuses of the respective storage systems. The dedicated data migration ports are connected by a leased line, and data migration is executed via this leased line. Further, the migration-destination controller is able to process an input/output request from a host computer during data migration, thereby enabling data to be read out from a LU of the first storage system by way of the leased line (e.g. cable) and provided to the host computer, and enabling data to be written to the LU via the leased line.

A number of working examples of the present invention will be explained hereinbelow by referring to the figures. In can be supposed at this time that a storage device mounted to a storage system is a hard disk drive (hereinafter referred to as HDD), for example, but a variety of storage devices, such as a DVD (digital versatile disk) drive, magnetic tape drive, and flash memory can be used instead of this HDD.

First Working Example

FIG. 1shows a block diagram of a computer system related to a first working example of the present invention. Furthermore, inFIG. 1, there are first and second storage systems, and the same numerals with different reference letters are assigned to the same elements of these storage systems. In this case, the reference letter “A” is assigned to the first storage system, and the reference letter “B” is assigned to the second storage system. Hereinbelow, when the same elements are not to be distinguished from one another, they will be explained using only a numeral, and when they are to be distinguished from one another, a combination of a numeral followed by a reference letter will be used.

A plurality (or one) of host computers192, and first and second storage systems920A and920B are connected to a SAN (Storage Area Network)190. A different type of communications network can be used instead of the SAN190.

A management terminal198and the first and second storage systems920A and920B are connected to a LAN (Local Area Network)191. Instead of the LAN191, another type of communications network can be used.

The host computer192sends an input/output request (write request/read request) to a storage system920, thereby either writing or reading data to or from the storage system920A.

The management terminal198is a computer for managing the first and second storage systems920A and920B. The management terminal198, for example, can provide a user interface, such as a GUI (Graphical User Interface) or CLI (Common Line Interface), required for work related to creating a logical unit (also called a logical volume), setting security, or otherwise changing configuration information, which will be explained hereinbelow. The processing carried out by the management terminal198will be described in detail below.Since the hardware configurations of the first and second storage systems920A and920B are approximately the same, an explanation will be given using the first storage system920A as a representative example.

The first storage system920A comprises a controller120A, and either one or a plurality of drives160A. In this subsystem920A, the controller120A and either one or a plurality of drives160A are cascadingly connected. If the flow of data from the host computer192to a HDD12A during a data write is taken as an example, this connection configuration is constituted such that the controller120A is the furthest upstream, and thereafter, the connections continue downstream as drive1, drive2, . . . drive p (where p is an integer of greater than 1 (here an integer of greater than 3)). Hereinbelow, when indicating this kind of connection configuration in this working example, the term “cascade connection configuration” will be used. A cascade connection configuration, for example, is not limited to a straight line as shown in the figure, and, for example, can also be a tree-shaped configuration.

The controller120A is an apparatus for controlling the operations of the first storage system920A, and differs from a switching apparatus like the first drive controller303A comprised in the drive enclosure160A. The controller120A, for example, is made redundant (for example, duplexed), and each controller120A is connected to another controller120A via an inter-controller communication interface apparatus (hereinafter abbreviated as interface “I/F”)118A. Thus, it is possible for a failure that occurred in one controller120A to be detected by another controller120A, and for the other controller120A to carry out a degenerate operation. Further, in this working example, the constitution is such that the access routes to the respective HDD12A have also been made redundant, and even if a failure should occur on one access route, the controller120A can access a HDD12A via another access route.

The controller120A comprises either one or a plurality of host I/F114A, a management I/F119A, a processor112A, a memory105A, and a drive I/F115A.

The respective host I/F114A, for example, are communication ports, and receive an input/output request from the host computer192.

The management I/F119A, for example, is a communication port, and receives a command from the management terminal198.

The memory105A stores a plurality of computer programs and data. The plurality of computer programs, for example, include an input/output processing program132A, a configuration management program134A, and a data migration program136A. Further, a cache area150A is provided in the memory105A, and data to be written to or read from a HDD12A by the host computer192is temporarily stored in this cache area150A.

The drive I/F115A is an interface apparatus for communicating with a HDD12A via the first drive controller303A of the drive enclosure160A.

The processor112A, for example, is a CPU (Central Processing Unit), and executes the reading in of a computer program stored in the memory105A. Hereinbelow, when a computer program is the subject, the explanation will describe the processing performed by the processor that actually executes this computer program.

The input/output processing program132A processes an input/output request received by the host I/F114A. When the input/output request is a write request, the input/output processing program132A temporarily writes data that complies with this write request to the cache area150A, reads this data out from the cache area150A and writes it to a HDD12A. Conversely, when the input/output request is a read request, the input/output processing program132A reads the data out from the HDD12A, writes this data temporarily to the cache area150A, reads this data out from the cache area150A, and sends it to the host computer192that is the source of this read request.

The configuration management program134A is a program for managing configuration information, which will be explained below. The configuration management program134A, for example, can update configuration information in response to a command from the management terminal198.

The data migration program136A executes data migration between logical units (LU). The processing carried out by the data migration program136A will be explained in detail below.

A drive enclosure160A comprises redundant first drive controllers303A and data migration ports601A. Either one or a plurality of HDD12A are connected to the respective first drive controllers303A. A data migration port601A is connected to a first drive controller303A, and is not connected to the controller120A. A data migration port601A is connected via a leased line (hereinafter, data migration cable)930to a data migration port601B of the second storage system920B. A communications network can be used instead of the data migration cable930. But in either case, in this working example, communications between the first drive controllers303A and303B, in other words, communications between the storage systems920A and920B is carried out via this data migration cable930, which comprises the back-end, and is not carried out via the SAN190to which the controllers120A and120B are connected. However, if this back-end communications should become impossible, communications can also be carried out between the controllers120A and120B by way of the SAN190.

The preceding is an overview of a computer system related to this working example. Furthermore, the data migration cable930can be redundant, or it can be a single cable. Further, controller120A, first drive controller303A, and data migration port601A can be implemented on a single circuit board. More specifically, for example, memory105A and a memory185A described hereinbelow (a memory185A that belongs to the first drive controller303A) can be a single memory, and the processor301A is a processor that is independent of the processor112A that processes an input/output request from a host computer192. This independent processor301A can write a message between processors112A and112B to memory, or read from memory and provide a message between processors112A and112B.

Next, the first drive controller303A will be explained in detail.

FIG. 2shows a block diagram of the first drive controller303A.

The first drive controller303A is a switching apparatus. This switching apparatus, for example, can be an Expander of an SAS (Serial Attached SCSI), or it can be a Fibre Channel switching apparatus. A drive controller303comprises a device manager300A, a memory185A, a processor301A, a crossbar switch311A, and a plurality of I/F309A,315A,313A, and317A.

The device manager300A is a device for acquiring information from a prescribed component inside the first storage system920A. For example, the device manager300A can acquire from a fan (for example, a fan for cooling the HDD12A), or a thermometer information (for example, a measured temperature from the thermometer) required for managing the first storage system920A.

The processor301A, for example, is a CPU, and can read out and execute a computer program from memory185A.

The memory185A stores a device management program304A and an inter-storage communications program305A. The device management program304A is a program for managing a prescribed component of the first storage system920A, and, for example, based on information acquired by the device manager300A, can perform HDD12A blockage processing, and control the power unit and fan. The inter-storage communications program305A is a program for carrying out back-end communications between the storage systems920A and920B. Inter-storage communications program305A can communicate with partner inter-storage communications program305B via a data migration port601A.Further, a communications area307A, which is a storage area for communications between a plurality of initiators, is provided in memory185A. This communications area307A is a storage area that is shared by a plurality of initiators. Furthermore, initiator as used here refers to a processor that constitutes the controllers120A and120B of this working example, and more specifically, the command issuing units of the controllers120A and120B.

The crossbar switch311A is an apparatus for switching the mutual connections between the processor112A and the plurality of I/F309A,315A,313A and317A. Instead of a crossbar switch311A, another type of switching unit (for example, a data transmission circuit as one type of LSI (Large-Scale Integration)) can be used.

Each of the plurality of I/F309A,315A,313A and317A, for example, is a communication port (more specifically, the “Phy” in a SAS Expander). Of the plurality of I/F309A,315A,313A and317A, I/F315A can be a different type of I/F than the other I/F309A,313A,317A. These I/F can take a switch type (star type) topology, and are also applicable to SAS and Fibre Channel.

In this working example, for example, the configuration is the above-mentioned cascade connection configuration comprising controller120A, first drive controller303A of drive1, first drive controller303A of drive2, . . . , and first drive controller303A of drive p. An upstream I/F (either the drive I/F115A of the controller120A or the I/F131A of an upstream first drive controller303A) is connected to I/F309A. A downstream I/F (I/F309A of the downstream first drive controller303A, and when this first drive controller303A is the furthest downstream, either nothing can be connected, or the HDD12A can be connected) is connected to I/F313A. The HDD12A is connected to I/F317A.

FIG. 3is a schematic diagram of one example of allocation of the storage space of a plurality of HDD.

The storage space of a plurality of HDD12A can be broadly divided into a user volume area501A and a system area190A.

The user volume area501A is an area for storing data (user data) used by an application program executed by the host computer192. More specifically, for example, it is a group of LU created using the storage space of a RAID (Redundant Array of Independent (or Inexpensive) Disks) constituted from two or more HDD12A.

The system area190A is either a part or all of the area of one or more HDD12A, and is an area for storing information (hereinafter, configuration information)172A related to the configuration of the first storage system920A. Configuration information172A, for example, comprises a LU configuration management table1100A, a port management table1200A, and a RAID group management table1300A. The configuration information172A, for example, can be stored in a prescribed address of a prescribed HDD12A in a prescribed format so as to be accessible from a second storage system A connected to the data migration port601A. Or, for example, the locations of the configuration information172A and172B can be recognized by each of the controllers120A and120B by exchanging information indicating the location where the configuration information172is stored in accordance with sending and receiving messages via the communications area307of a drive controller303.

FIG. 7Ashows an example of the constitution of a LU configuration management table1100A.

The LU configuration management table1100A is a table for managing LU, and, for example, an internal LUN, allocated host ID, host LUN, capacity, physical location, RAID group ID, and pair status are recorded in this table1100A for each LU. The internal LUN is a LUN for the first storage system920A to identify a LU (hereinafter, target LU). By contrast, the host LUN is a LUN for the host computer192to identify a target LU. The allocated host ID is an identifier of the host computer that is the allocation destination of a target LU. The input/output processing program132A allows a host with an allocated host ID of “1” to access a LU having an internal LUN of 0 and 1, but prohibits access to LU of internal LUN other therethan. The capacity is the capacity of a target LU. The physical location expresses the location of a target LU, and, for example, is a storage system identifier. By virtue of the fact that this physical location column is a single point, and can record a physical location, the first storage system920A can have a LU that is on an HDD12B of the second storage system920B. The RAID group ID is an identifier of the RAID group to which the target LU belongs. The pair status is the status of the LU pair comprising the target LU. A pair status of “simple” refers to the fact that the partner LU, which, together with the target LU, constitutes a LU pair, does not exist.

FIG. 7Bshows an example of the constitution of a port management table1200A.

The port management table1200A is a table for managing information related to respective host I/F114A. For example, a host I/F number (identification number of a host I/F114A), an IP address, a subnet mask, a default gateway, and an allocated internal LUN are recorded in this table1200A for each host I/F114A. The allocated internal LUN is the internal LUN of a LU allocated to a host I/F (a LU accessible via this host I/F). In this working example, it is supposed that a host I/F114A is an iSCSI port. Therefore, a setting required for an iSCSI-based IP is recorded in this port management table1200A.

FIG. 7Cshows an example of the constitution of a RAID group management table1300A.

The RAID group management table1300A is a table for managing information related to respective RAID groups. For example, a RAID group ID, a RAID level, a HDD configuration, and a LU allocated area are recorded in this table1300A for each RAID group. The RAID group ID is the identifier of a RAID group. The HDD configuration is the identification number of the respective HDD12A constituting a RAID group. The LU allocated area indicates a part of the RAID group-provided storage space that has been allocated as a LU. A free (non-allocated) storage space part can be specified from this LU allocated area, making it possible to use this storage space part to create a new LU.

FIG. 4shows an example of the constitution of a communications area307A.

The communications area307A comprises a first channel area951A, a second channel area953A, and a locking information area955A. In this working example, two channels are provided, but three or more channels can be provided. Further, a plurality of locking information areas can also be provided.

Each channel area951is an area utilized as a channel for sending a message. The first channel area951A is the area of the channel in the direction from the first storage system920A to the second storage system920B. The second channel area953A is the area of the channel in the direction from the second storage system920B to the first storage system920A.

The following information areas are included in each channel area951. That is, an area in which a validation flag is written, an area in which a sequence number is written, an area in which a message size is written, and an area in which the body of a message is written. The validation flag is a flag, which indicates whether or not a message was sent to the intended communication partner. The sequence number is used to check whether or not a new message, which differs from that checked by the receiver the previous time, has been written (a message can be determined to be new if the number is different from the previous time). The sequence number can be incremented by the message sender. The message size is the data size of the body of a message. The body of a message is the message itself.

The locking information area955A is an area utilized for the exclusive control of an update to the configuration information172A. More specifically, for example, a locking variable, which can be constituted from one bit of either a 0 or a 1, is written in the locking information area955A. A locking variable of 1 indicates lock acquisition is complete, and a locking variable of 0 indicates that lock has not been acquired. Here, when lock has been acquired, updating of the configuration information172A inside the first storage system920A is prohibited (in other words, exclusive control is achieved). Furthermore, the exclusive control range can be changed, and a locking variable can be provided for each exclusive control target (for example, LU, and configuration information172A).

The flows of the various processes carried out in this working example will be explained hereinbelow.

FIG. 5Ashows a flowchart of the processing of the controller120A (processor112A) when the first storage system920A sends a message to the second storage system920B.FIG. 5Bshows a flowchart of the processing of the controller120B (processor112B) when the first storage system920A sends a message to the second storage system920B.FIG. 5Cshows a flowchart of the processing of the drive controller303(processor301) when the first storage system920A sends a message to the second storage system920B. Furthermore, a step is abbreviated as “S” in the figures.

The drive controller303to be used in communications between the storage systems920A and920B, for example, is set in advance by the controllers120A and120B and the drive controller303, and the following processing is carried out in accordance with this setting. This setting, for example, can be recorded in the respective memories105A,105B and185of the controllers120A,120B and drive controller303. For the below processing, it is supposed that the drive controller303utilized in communications is drive controller303A.

As shown inFIG. 5A, controller120A (processor112A) sends an update command for updating the communications area307A to first drive controller303A (S101).

This processing can be executed by a dedicated program (not shown in the figure), it can be executed by any of the programs132A,134A and136A described hereinabove, and it can be executed by a prescribed program from thereamong (for example, the data migration program136A).

As shown inFIG. 5B, controller120B (processor112B) sets a timeout time, and starts a timer operation (time operation) (S111). The timeout time can be set to a value corresponding to a situation, or it can be a predetermined fixed value. Further, instead of a timeout time, a number of times (a polling frequency) for looping S112and beyond can be set, and instead of a timer operation, the polling frequency can be counted.

The processor112B, after standing by for a fixed time (S112), sends a reference command for referencing the communications area307A to the first drive controller303A (S113), and in response thereto receives various information recorded in the first channel area951A (S114).

The processor112B analyzes this information, and sets a validation flag (validation flag=1), and determines if the sequence number is different from the previous reference (S114). If the sequence number is different (S114: YES), the processor112B ends processing as message reception successful, and if the sequence number is not different (S114: NO), checks if the timeout time subsequent to the commencement of the timer operation has elapsed (S115), and it this timeout time has not elapsed (S115: NO), returns to S112, and if the timeout time has elapsed (S115: YES), executes a timeout.

The above processing can be executed by a dedicated program (not shown in the figure), it can be executed by any of the programs132B,134B and136B described hereinabove, and it can be executed by a prescribed program from thereamong (for example, the data migration program136B).

Each step shown inFIG. 5C, for example, can be executed by the inter-storage communications program305A of the first device controller303A. When a command is received in wait to receive a command (S121), the inter-storage communications program305A determines the command type (S122).

In S122, if the command is an update command for updating the communications area307A, the inter-storage communications program305A updates the first channel area951A in accordance with this update command (S124), and sends a response relative to this update command (for example, updating complete) to the processor112A (S126). The constitution can also be such that the channel area to be updated (first channel area951A) is specified in the update command, and the inter-storage communications program305A updates the information of the specified channel area, and if a channel area is not specified in the update command, the inter-storage communications program305A selects the channel area corresponding to the source of the update command, and updates the information recorded in the selected channel area. This can be made the same for a reference in accordance with a reference command.

In S122, if the command is a reference command for referencing the communications area307A, the inter-storage communications program305A acquires information from the first channel area951A in accordance with this reference command, and sends this information to the command issuer (processor112B) (S125). Then, the inter-storage communications program305A sends a response relative to this reference command (for example, referencing complete) to the processor112B (S126).

In S122, if the command is another command besides an update command or a reference command for updating/referencing the communications area307A, the processor301A executes processing in accordance with the received command (S123), and returns a response relative to this command to the command source (S126). As this other command, for example, there are a command for acquiring the temperature inside the enclosure, a lock acquisition command, which will be explained below, and a lock release command.

According to the processing ofFIG. 5A,FIG. 5BandFIG. 5Cabove, the controllers120A and120B can send and receive messages to and from one another. Furthermore, in this sending and receiving, a message and reference command from controller120B, and a message provided to controller120B go by way of the prescribed drive controller303B, data migration port601B, data migration cable930and data migration port601A. More specifically, for example, information indicating that drive controller303B, which is being used to send and receive messages, is drive controller303A rather than itself is stored in the memory185B of the prescribed drive controller303B, and the inter-storage communications program305B of this drive controller303B recognizes this information. The controller120B sends a communications area307A reference command to the prescribed drive controller303B. In this case, the inter-storage communications program305B makes the above-mentioned recognition, and thereby transmits this reference command to the first data controller303A via the data migration port601B and data migration cable930. Thus, the first drive controller303A can receive the reference command sent by controller120B. The inter-storage communications program305A responds to this reference command, and can send a message recorded in the first channel area951A to the drive controller303B by way of the data migration port601A and data migration cable930. The inter-storage communications program305B can transmit this received message to the controller120B. Or, based on the information that the drive controller being used to send and receive a message is drive controller303A, controller120B can also communicate directly to drive controller303A (More specifically, the controller120B can issue a command, such as a reference command, to drive controller303A by way of drive controller303B.).

Further, according to the processing ofFIG. 5A,FIG. 5BandFIG. 5Cabove, it is possible to achieve synchronization between controllers120A and120B. That is, for example, when a message having sequence number 1 is sent from controller120A, controller120B can operate without processing this sequence number 1 message a plurality of times. Further, as another method of achieving synchronization, for example, the message receiving side can set a flag indicating whether or not the message was referenced in the channel area in which this message is recorded, and the message sending side writes the next message if this flag is on (for example, if the flag is 1) but does not write the next message if this flag is off (for example, if the flag is 0).

With regard to a command between the controllers120A and120B and drive controller303A, for example, there is a method that utilizes the Mode Select (update command)/Mode Sense (reference command) of SCSI (More specifically, for example, there is a method, which uses a command that conforms to SES). By defining a mode page corresponding to the respective channel areas951and953of the communications area307, it is possible to send and receive the contents of the channel areas951and953as mode parameters of these commands. It is possible to apply this method when using a standard that is compatible with SCSI (for example, Fibre Channel, SAS, iSCSI) as the I/F of a drive enclosure160.

In the example described hereinabove, the working example is explained using an example of when the first storage system920A sends a message to the second storage system920B, but when the second storage system920B sends a message to the first storage system920A, the second channel area953A is used.

There are a plurality of drive controllers303inside the respective storage systems920A and920B, and the drive controller for use in communications between storage systems920A and920B can be preset as explained hereinabove. Further, a communications area307need not be provided in all the drive controllers303, but instead can be provided in at least one drive controller303. A communications area307can be provided in any drive controller303(For example, a communications area307can be in the drive controller of the drive that is furthest upstream of a plurality of drives.). The above holds true for the data migration ports601as well.

The processes executed when the first drive controller303A (processor301A) receives a lock acquisition command and a lock release command, respectively, will be explained below by referring toFIGS. 6A and 6B. In this explanation, it is supposed that controller120B sends a lock acquisition command and a lock release command to prohibit the updating of configuration information172A by controller120A, and to release this prohibition.

FIG. 6Ashows a flowchart of processing executed in S123ofFIG. 5Cwhen the first drive controller303A (processor301A) receives a lock acquisition command.

Upon receiving a lock acquisition command from controller120A, the processor301A references the first channel area951A, and checks the locking variable (S131) As a result, if the locking variable is 1, the processor301A responds to controller120A that lock acquisition failed (S132). Conversely, if the locking variable is 0, the processor301A changes this locking variable to 1, and responds to controller120A that lock acquisition succeeded (S133).

FIG. 6Bshows a flowchart of processing executed in S123ofFIG. 5Cwhen the first drive controller303A (processor301A) receives a lock release command.

Upon receiving a lock release command from controller120A, the processor301A references the first channel area951A, and checks the locking variable (S141). As a result, if the locking variable is 1, the processor301A changes this locking variable to 0, and responds to controller120A that lock acquisition succeeded (S142). Conversely, if the locking variable is 0, the processor301A responds to controller120A that lock release failed (S143).

The condition determinations (S131and S141), and the changing of the locking variables (S133and S143) inFIGS. 6A and 6Bmust be executed atomically (inseparably) to prevent a conflict, such as a lock being acquired simultaneously by two controllers120.

Now then, in this working example, in order to replace a storage system, for example, the data of the old (existing) first storage system920A is migrated to a new (unused) second storage system920B. For this purpose, the second storage system920B is connected to the first storage system920A.

FIG. 8shows a flowchart of processing executed when first drive controllers303A and303B are connected via a data migration cable930.

The first storage system920A, which is in operation, is waiting for an event to occur (S151).

In S152, a person (for example, a maintenance person) connects the data migration cable930, which is connected to data migration port601A of the first storage system920A, to data migration port601B of the second storage system920B, and thereafter, the power to the second storage system.920B is turned ON. Thereafter, the second storage system920B waits to receive a notification (S153). More specifically, for example, the controller120B polls (regularly issues reference commands) to the first channel area951A of the communications area307.

According to S152, for example, a prescribed electric signal from the data migration port601A is inputted to the device manager300A, and the device manager300A detects that the second storage system920B is connected via the data migration port601A. The device management program304A reports the fact that the device manager300A made this kind of detection (that is, the fact that the second storage system920B is connected) to controller120A.

In S154, controller120A responds to this report by sending a message indicating the status (hereinafter, a status message) to the second storage system920B. At this time, controller120A can write a message indicating, the status to the first channel area951A of the communications area307A.

In S155, controller120B determines whether or not the first storage system920A is operating normally. Here, for example, controller120B can determine that operation is normal when it acquires the status message, and this status message indicates normal (S155: YES). Conversely, when the status message does not indicate normal, or when timeout occurs without being able to reference the status message, controller120B can make the determination that operation is not normal (S155: NO). When S155is YES, processing ends, and when S155is NO, processing proceeds to S156.

In S156, controller120B executes HDD discovery in the first storage system920A. More specifically, for example, controller120B sends a prescribed reference command for checking the HDD12A that exist in the first storage system920A to the first drive controller303A. Accordingly, all the HDD12A subordinate to the first storage system920A become subordinate to the second storage system920B. In other words, ownership of all the HDD12A transfers from the first storage system920A to the second storage system920A.

In S157, configuration information172A is read out from the system area190A (refer toFIG. 3), and is merged (added) to configuration information172B. More specifically, for example, the configuration management program134B of controller120B sends a read command for referencing configuration information172A to the first drive controller303A. For example, the location of the configuration information172A (for example, the location specified by the HDD discovery of S156) is specified in this read command. The inter-storage communications program305A of the first drive controller303A responds to this read command, acquires the configuration information172A from the HDD12A, and sends the acquired configuration information172A to controller120B. The configuration management program134B adds the information recorded in configuration information172A to configuration information172B. Configuration information172A can be deleted from the system area190A at this time.

According to the above-described process, when the first drive controllers303A and303B are connected at the back-end via the data migration cable930, controller120B can determine the status of controller120A, and when controller120A is determined not to be normal, controller120B automatically acquires ownership of the HDD and LU of the first storage system920A.

FIG. 9is a flowchart of processing for transferring LU ownership from the first storage system920A to the second storage system920B.

This process, for example, is executed when controller120A is normal, but ownership of a portion of the LU of the plurality of LU in the first storage system920A is transferred. The advantage of this process, for example, is that the load of controller120A is distributed to controller120B. Further, this process can also be used at the data migration of a target LU. In this process, it is supposed that ownership of LU of internal LUN4in the first storage system920A is transferred. Making the LU of internal LUN4the target can be specified, for example, from the management terminal198.

Controller120B sends a lock acquisition command to the first drive controller303A (S161). The inter-storage communications program305A acquires a lock (makes the locking variable 1) (S162), thereby creating a state wherein updating of configuration information172A is prohibited. More specifically, even if controller120A requests the updating of configuration information172A, in response to this request, the inter-storage communications program305A returns a message to controller120A indicating that configuration information172A is under exclusive control, thereby making it impossible for configuration information172A to be updated. Subsequent to S162, controller120A waits to receive a notification, for example, transitioning to a state wherein it regularly sends reference commands to the communications area307A (S163)

Controller120B writes a message to the communications area307A requesting that configuration information172A be written to an HDD12A, and notifying the target LU (the LU of LUN4, which is the target of LU ownership migration) (S164). It is requested that configuration information172A be written out to a HDD12A here because there are cases in which the configuration information172A is read out to the memory105A of controller120A, and referenced or updated in this memory105A. Therefore, this kind of request is not necessary when the configuration information172A is not capable of being updated in memory105A. Subsequent to S164, controller120B waits to receive a notification, for example, transitioning to a state wherein it regularly sends reference commands to the communications area307A (S165).

In S166, controller120A responds to the message written in S164, writes the configuration information172A to a HDD12A (writes it to the system area190A), and destages/offlines the target LU. Destage here refers to writing to the target LU from the cache area150A data that is not reflected in the target LU from among the data that exists in the cache area150A. Further, offline here refers to cutting the connection between the target LU and the host computer that is allowed to access this target LU (the host computer identified from the allocated host ID, hereinafter, the target host). Once the writing of the configuration information172A to the HDD12A is complete, controller120A writes a message indicating the completion thereof to the communications area307A (S167).

In S168, controller120B responds to the message written in S167, and sends a read command to the HDD12A for reading the configuration information172A.

In S169, controller120B transfers ownership of the target LU by updating configuration information172A and172B. More specifically, as shown inFIG. 10A, controller120B deletes the row comprising internal LUN4from LU configuration management table1100A, and, as shown inFIG. 10B, adds this deleted row to LU configuration management table1100B.

In S170, controller120B carries out path switching, and brings the target LU online. More specifically, for example, controller120B sets the IP address and so forth of the host I/F114A, to which internal LUN4is allocated, in an offline host I/F114B in which an IP address and so forth has not been set, and brings this host I/F114B online. As path switching methods for the target LU, there is, in addition to IP address takeover, a method for changing the IP address for accessing the target LU by updating the iSNS (Internet Storage Name Server) or changing the setting inside the host192.

In S171, controller120B sends a lock release command to the first drive controller303A. The inter-storage communications program305A cancels the prohibition on updating configuration information172A by releasing the lock (making the locking variable 0) (S172).

The preceding is the process for transferring LU ownership. Prior to carrying out S170, when the target host specifies an IP address for accessing the target LU and sends an input/output request, controller120A receives this input/output request, but subsequent to carrying out S170, controller120B receives this input/output request. Accordingly, it is possible to change the destination of an input/output request for the target LU from the first storage system920A to the second storage system920B without stopping an input/output request from the target host. Furthermore, when the host I/F114A and114B are Fibre Channel, instead of carrying out IP address takeover, controller120A can stop receiving input/output requests for the target LU, and controller120B can start receiving input/output requests for the target LU.

Further, subsequent to carrying out S170, when controller120B receives an input/output request for the target LU inside the first storage system920A, for example, this input/output request can be transmitted to the first storage system920A in accordance with the following processing. That is, controller120B specifies the internal LUN4corresponding to the host LUN specified by this input/output request, and references the physical location corresponding to this internal LUN4. When this physical location represents a subsystem1, controller120B sends a write command/read command for the HDD12A corresponding to this internal LUN4to this HDD12A by way of the first drive controller303A. Accordingly, controller120B can write or read data in accordance with this input/output request to the target LU of the first storage system920A.

FIG. 11is a flowchart of processing for the second storage system920B to set a LU subordinate to itself in a free portion of user volume area501A of the first storage system920A.

This process, for example, can be started in accordance with the management terminal198instructing controller120B to create a new LU. Required information, such as whether or not the LU will be created in an existing RAID group (RG), and the RAID group at which RAID level is to be created, can set in the new LU creation instruction.

Approximately the same processing as S161to S167is carried out (S181to S187). Unlike the transfer of LU ownership, there is no need to carry out a target LU notification, or to destage/offline the target LU.

In S188, controller120B sends a read command for reading out configuration information172A to the HDD12A by way of the first drive controller303A, thereby acquiring the configuration information172A.

When controller120B creates a LU in an existing RAID group (RG), processing proceeds to S190, and when controller120B creates a LU by creating a new RAID group, processing proceeds to S191.

In S190, a LU is created in an existing RAID group inside the first storage system920A by updating configuration information172A and172B, and ownership of this LU is set in the second storage system920B. More specifically, as shown inFIG. 12A, RAID group2is selected, a LU of internal LUN0is created in this RAID group2, and information relevant to this LU is added to LU configuration management table1100B. Furthermore, the reason for selecting RAID group2is because it was the RAID level specified in the above-mentioned new LU creation instruction, and because it has free capacity of greater than the capacity specified in this new LU creation instruction. The free capacity of the respective RAID groups can be specified by referencing the LU allocated area. Further, a required location of the configuration information172A is updated in line with this processing. More specifically, for example, the LU allocated area corresponding to RAID group ID2of the RAID group management table1300A is updated.

In S191, a new RAID group and LU are created in the first storage system920A by updating configuration information172A and172B, and ownership of this LU is set in the second storage system920B. More specifically, as shown inFIG. 12B, RAID group5is newly created, a LU of internal LUN0is created in this RAID group5, and information relevant to this LU is added to LU configuration management table1100B. Furthermore, RAID group5, for example, can be created when the number of unused HDD12A corresponds to the specified RAID level. An unused HDD12A, for example, can be specified from the HDD configuration of RAID group management table1300A. For example, when there is a missing HDD number in a prescribed range of HDD numbers, a determination can be made that this HDD number is not being used. In this S191, a required location of the configuration information172A is once again updated. More specifically, for example, a new row corresponding to RAID group ID5is added to the RAID group management table1300A.

FIG. 13is a flowchart of processing executed when all the data inside the first storage system920A migrates to the second storage system920B.

This process, for example, can be automatically carried out subsequent to the back-end connection being made between first drive controllers303A and303B, and can also be carried out in accordance with an instruction from the management terminal198. The management terminal198can send a total data migration instruction to both controllers120A and120B.

In S201, controller120A determines whether or not there is authorization for a data migration from a user. When authorization exists, processing proceeds to S204. Furthermore, “user data migration authorization” can be carried out from the management terminal198. The purpose of performing this step, for example, is as a security measure to prevent data leakage resulting from an unauthorized data migration. If security is not a problem, then this step can be eliminated.

In S204, controller120A writes a message to the communication area307A notifying of authorization to migrate data (hereinafter, migration authorization message). Thereafter, controller120A waits for a request to takeover all LU (S205).

S201to S204above need not be executed when controller120A is not operating normally.

In S202, controller120B determines the status of controller920A. More specifically, for example, controller120B writes a message to the communications area307A inquiring as to the status of controller120A, and thereafter, polls the communications area307A (waits for a status message). When a status message is acquired, the status of controller120A is determined from this status message. When controller120A is determined to be normal (S202: YES), processing proceeds to S203(execution of polling to wait for a migration authorization message), and if a migration authorization message is acquired in S203, processing proceeds to S206. When controller120A is determined not to be normal, or when a timeout occurs (S202: NO), processing proceeds to S206.

In S206, controller120B determines if there is the HDD capacity needed for data migration. More specifically, for example, controller120B acquires configuration information172A, and determines from the configuration information172A whether or not the storage capacity of more than the total capacity of all LU in the first storage system920A is greater than the unused storage capacity of the second storage system920B (the storage capacity capable of being specified from configuration information172B). If it is not, processing proceeds to S207and data migration is cancelled, and if it is, processing proceeds to S206A.

In S206A, controller120B determines whether or not controller120A is normal. The same processing as that of S202can be carried out once again here, and the determination results of S202can be inherited. If the result of S206A is YES, processing proceeds to S208, and if the result of S206A is NO, processing proceeds to S212.

In S208, controller120B writes a message to the communications area307A requesting takeover of all LU. Then, controller120B waits to receive a notification (for example, the polling execution status of communications area307A) (S209).

In S210, controller120A, for example, acquires from the communications area307A the message requesting the takeover of all LU by performing polling in S205. In response thereto, controller120A sets all host I/F114A offline (for example, severs communications), destages/offlines all LU (for example, makes all LU exclusive), executes a write to the HDD12A of configuration information172A, and writes a message indicating the completion of this processing (hereinafter, S210-complete message) to the communications area307A.

In S211, the operation of controller120A stops, either automatically subsequent to S210, or as a result of the below-described stop processing in S212.

In S212, controller120B can execute the following processing thanks, for example, to acquiring the S210-complete message via the polling in S209, and to the fact that S206A resulted in NO. That is, controller120B specifies the IP addresses of the respective host I/F114A from the port management table1200A in the configuration information172A, and sets the respective IP addresses in host I/F114B. In other words, the IP addresses of the respective host I/F114A are taken over by the respective host I/F114B. Furthermore, in this S212, for example, when the result of S206A was NO, processing for completely stopping controller120A (stop processing) can be executed. For example, controller120B can send a stop command for stopping controller120A to the first drive controller303A, and the processor301A of the first drive controller303A can stop the processor112A of controller120A by virtue of an interrupt.

In S213, controller120B notifies the host computer192of the IP addresses of the respective host I/F114B.

In S214, controller120B starts receiving the input/output requests from the host computer192for all the migration-source LU in the first storage system920A. Thereafter, controller120B executes data copying in S220.

FIG. 14is a flowchart of the data copying of S220ofFIG. 13.

In S221, controller120B, based on configuration information172A, creates a user volume area501B of the same configuration as the user volume area501A of the first storage system920A.

More specifically, for example, controller120B copies all the rows constituting LU configuration management table1100A to LU configuration management table1100B as shown in the example inFIG. 15. Accordingly, LU ownership of internal LUN0to4(migration-sources LU0to4) is transferred to the second storage system920B. Thus, controller120B can receive and process input/output requests for migration-source LU0to4.

Furthermore, controller120B adds the respective rows corresponding to the copied rows to LU configuration management table1100B. More specifically, rows respectively comprising internal LUN5through9corresponding respectively to internal LUN0through4are added. The capacity and RAID group ID are respectively copied in each added row. Accordingly, migration-destination LU5to9corresponding respectively to all the migration-source LU0to4in the first storage system920A are created in the second storage system920B. Furthermore, columns called migration-source LUN and migration status are added to the LU configuration management table1100B at data migration (These columns can also be provided in advance.). A migration-source LUN is an internal LUN of a migration-source LU. The fact that the migration-source LUN corresponding respectively to internal LUN5to9are numbered 0 to 4 makes it clear that the migration-source LU corresponding respectively to the migration-destination LU5to9are numbered 0 to 4. Further, migration status represents the status related to migration. This status is one of three types: “incomplete” (migration has not been carried out yet), “migrating” (in the process of copying the data), and “complete” (data copying is over).

Refer toFIG. 14once again. In S222, controller120B determines if all LU copying was completed. This determination can be made, for example, based on whether or not all the migration statuses of the LU configuration management table1100B are “complete”. If all the migration statuses are complete, processing moves to S223, and if all the migration statuses are not complete, processing moves to S224.

In S223, controller120B destroys the data of all the LU (migration-source LU0to4) of the first storage system920A. Accordingly, all data migration is complete. That is, in this working example, for a data migration between LU (migration), it is possible to end data migration by destroying the data inside a migration-source (copy source) LU subsequent to copying the data between the LU.

In S224, controller120B selects one migration-source LU from a group of migration-source LU for which copying is not yet complete, and commences copying to a migration-destination LU corresponding to this selected migration-source LU. More specifically, as shown in the example inFIG. 16, for example, controller120B selects migration-source LU2, changes the migration status corresponding to the migration-source LU2from “incomplete” to “migrating”, and executes data copying to migration-destination LU7from migration-source LU2. Once this data copying has ended, controller120B changes this migration status from “migrating” to “complete”.

In S225, subsequent to completion of this copying, controller120B changes the host access destination to the migration-destination LU. More specifically, as shown in the example inFIG. 17, for example, controller120B makes the allocated host ID and host LUN, which correspond to migration-source LU2, correspondent to migration-destination LU7, and deletes this allocated host ID and host LUN from the row of migration-source LU2. Further, controller120B also deletes migration-source LUN2corresponding to migration-destination LU7. However, changing the host access destination to the migration-destination LU can also be done subsequent to completing the copying of data to all the LU (after S222). In this case, a write from a host to a LU to which copying was completed is reflected in both the migration-source and migration-destination LU prior to data copying being completed for all LU. Opting for the latter system is less advantageous than the former system from the standpoint of performance, but it enables the latest data to be saved in the first storage system920A, making it possible to continue working by returning the access path to the first storage system920A when it becomes impossible to continue migration due to a failure or the like during migration.

Repeating S224and S225completes data copying from all migration-source LU to all migration-destination LU, and LU configuration management table1100B becomes like the example shown inFIG. 17.

The preceding is an explanation of total data migration processing. Furthermore, the data copying in S224, for example, can be carried out by a flow of processes such as the following. For example, the data migration program136B reads data from the selected migration-source LU, writes this data temporarily to the cache area150B, and then reads this data from the cache area150B, and writes this data to the migration-destination LU. Instead of this method, for example, the data migration program136B can also carry out data copying by setting the physical migration-source location (for example, the HDD number and address range), where the migration-source LU data exists, in first drive controller303A, setting the physical address (for example, the HDD number and address range) corresponding to the migration-destination LU in first drive controller303A, having first drive controller303A read the data from the migration-source location and transmit same to first drive controller303B, and having first drive controller303B store the transmitted data in the migration-destination location. In other words, in a data migration, data targeted for migration can be written to a migration-destination LU subsequent to being temporarily written to the cache area150of controller120B, and data migration can also be carried out without writing the data to the cache area150.

The preceding is an explanation of the first working example. Furthermore, the operation of controller120B, which was explained by referring toFIGS. 13 and 14, is carried out by the data migration program136B. Further, in total data migration processing, data copying can commence after a lock is acquired by controller120B. Furthermore, as for a back-end connection (connection between the data migration ports601A and601B), back-end data migration, and host access path switching, for example, the processing disclosed in Patent Application No. 2006-278237 (This application was not laid open to the public at the time of application.) can be cited as needed.

According to this first working example, data migration ports601A and601B are connected respectively to first drive controllers303A and303B, and these data migration ports601A and601B are connected by a data migration cable930. In a data migration, data flows from the migration-source LU to first drive controller303A, data migration port601A, data migration cable930, data migration port601B, drive controller303B and the migration-destination LU, without going through controller120A of the migration-source storage system920A. Accordingly, the migration of data is possible even when controller120A is inoperable.

Further, in this first working example, a communications area307, which is a shared memory, is provided in the drive controller303by way of which data flows in a data migration, and an inter-storage communications program305is installed in the drive controller303. This inter-storage communications program305is configured so as to be able to receive and process commands from controllers120A and120B for referencing and updating this communications area307. Accordingly, it becomes possible to send and receive messages between controllers120A and120B, which make use of the storage resources of the back-end drive controllers303.

Further, in this first working example, when locking information for exclusive control is recorded in the communications area307A, and a lock is acquired, the updating of a prescribed resource (for example, configuration information172A or a LU) inside the first storage system920A is prohibited. Accordingly, for example, when data migration and/or LU ownership transfer is executed online (while receiving input/output commands from a host computer192), this execution can be carried out safely without destroying the configuration of the migration-source. This mechanism is valid when controller120A is normal in particular.

Further, in this first working example, since each controller120can communicate with the drive controller303in the other storage system even when the other controller120is not operating normally, configuration information172in the other storage system920can be referenced and updated. Accordingly, ownership of the LU of the other storage system920can be acquired even if the other controller120is inoperable.

Second Working Example

A second working example of the present invention will be explained below. The points of difference with the first working example will mainly be explained at this time, and explanations of the points in common with the first working example will either be simplified or omitted.

In the first working example, during data migration, controller120B receives an input/output request, and the data conforming to this input/output request goes through the data migration cable930until access from the host is switched to the migration-destination LU. Since data also goes by way of this cable930during a data migration, the bandwidth of the cable930becomes constricted.

Accordingly, in the second working example, an input/output request is received and processed by controller120A. In accordance therewith, data conforming to the input/output request does not go by way of the data migration cable930, thereby making it possible to relieve the communication burden placed on the cable930.

More specifically, when controller120A is determined to be normal, the total data migration processing ofFIG. 18is executed instead of that ofFIG. 13. This will be explained in detail hereinbelow. Furthermore, the same holds true for the first working example, but the data migration program136can execute the operation executed by the controller120involved in a data migration.

Controller120B waits to receive a request, for example, it polls the communications area307A (S231).

When there is data migration authorization from the user (S232: YES), controller120A writes a message to the communications area307A requesting ownership of all free HDD (S233). Then, controller120A waits to receive a notification, for example, it polls the communications area307A (S234).

Upon acquiring this message, controller120B responds to this message, writes configuration information172B to system area190B (HDD12B), and writes a message to the communications area307A indicating the completion thereof (S235). Then, controller120B waits to receive a communication (S238).

Upon acquiring this message by waiting in S234, controller120A executes data copying (S220). The details of the processing of S220are as shown inFIG. 14. In this processing, either controller120A or120B can carry out data migration.

Subsequent to the end of S220, the same processing as that of S210to S214ofFIG. 13is executed (S237to S241).

The preceding is an explanation of the second working example.

Third Working Example

FIG. 19shows a block diagram of a computer system related to a third working example of the present invention.

Controllers120A and120B in the respective storage systems920A and920B are not redundant. Therefore, when a failure occurs in controller120A, there is no element to operate in its place, but nevertheless, convenience is maintained since the above-described data migration is possible if controller120B and the first drive controllers303A and303B are normal.

Furthermore, in this third working example, the data migration cable930and data migration ports601A and601B are not redundant, but they can be made redundant. By so doing, even if a failure occurs in the one cable930, and either data migration port601A or data migration port601B, data migration can be carried out via the other cable930, and either data migration port601A or data migration port601B.

The embodiment and a number of working examples of the present invention have been explained hereinabove, but this embodiment and these working examples are merely illustrations for explaining the present invention, and the scope of the present invention is not restricted to this embodiment and these working examples alone. The present invention can be put into practice in a variety of other modes without departing from the gist thereof.

For example, data copying in LU units and the transfer of LU ownership can be executed alternately.

Further, the decision to make one storage system the migration-source and to make the other storage system the migration-destination can be made manually by a user, or it can be decided automatically. A method for deciding automatically, for example, is as follows. That is, because storage system920A is operating alone at first, there is nothing connected to the other end of the data migration port601A, and as a result, even if controller120A detects drive controller303A, it does not detect any device at the other end of the data migration port601A. Thus, when only one drive controller303A can be detected, controller120A can make the determination that this storage system920A is the storage system of the migration-source. Conversely, it is supposed that storage system920B will start up in a state in which drive controller303A is connected to data migration port601B via a data migration cable. This being the case, controller120B can detect drive controller303A in addition to drive controller303B. Thus, when both drive controllers303A and303B are detected, controller120B can determine that this storage system920B is the storage system of the migration-destination, and can determine that the other storage system920A is the storage system of the migration-source. Thus, when there is a difference in the startup times of the storage systems920A and920B, a determination can be made automatically from the above-mentioned flow. Furthermore, when there is no difference in the startup times of the storage systems920A and920B, the migration-source and migration-destination can be specified to the respective controllers120A and120B from the management terminal.

Further, for example, the drive controller used in communications can be switched automatically. More specifically, for example, a setting, which determines in advance that “303A of the drive controller A, to which the controller (1)120A of the migration-source storage system920A is connected, will be used”, is made in the respective controllers120A and120B. If a failure occurs in the predetermined drive controller303A, a communication is attempted using the next candidate drive controller based on a predetermined candidate search order (for example, drive controller A of the migration-source subsystem→drive controller A of the migration-destination subsystem→drive controller B of the migration-source subsystem→drive controller B of the migration-destination subsystem), and once a drive controller capable of communication is detected, it is possible to continue communications between the controllers120A and120B via this drive controller.