Remote copy system

Even when a host does not give a write time to write data, consistency can be kept among data stored in secondary storage systems. The present system has plural primary storage systems each having a source volume and plural secondary storage systems each having a target volume. Once data is received from a host, each of the plural storage systems creates write-data management information having sequential numbers and reference information and sends, to one of the primary storage systems, the data, sequential number and reference information. Each of the secondary storage systems records reference information corresponding to the largest sequential number among serial sequential numbers and stores, in a target volume in an order of sequential numbers, data corresponding to reference information having a value smaller than the reference information based on the smallest value reference information among reference information recorded in each of the plural secondary storage systems.

CROSS-REFERENCES TO RELATED APPLICATIONS

This application relates to and claims priority from Japanese Patent Application No. 2004-219483, filed on Jul. 28, 2004, the entire disclosure of which is incorporated herein by reference.

BACKGROUND

The present invention relates to a storage system for storing data to be used by a computer and having data updated by a computer and more particularly to remote copy processing in which copies of data are held among plural storage systems.

EP 0672985 discloses a remote copy system for storing data to be used by a computer in a primary storage system and storing a copy of the data in a remote secondary storage system. In EP 0672985, copies of data are stored in the secondary storage system in the same writing order as that of storing the data in the primary storage system.

In EP 0672985, the primary storage system having received write data from a primary host computer reports the completion of reception of the write data to the primary host computer immediately after the reception of the write data. Then, the primary host computer reads a copy of the write data from the primary storage system. A write time is given to the write data. In this case, the write time is a time when a write request is issued for the write data. When the write data is loaded to the primary host computer, the write time is sent to the primary host computer. Furthermore, the primary host computer transfers the write data and the write time to a secondary host computer. The secondary host computer having received the write data and the write time writes the information including the write time in a control volume in the secondary storage system and writes respective write data in the secondary storage system in an order of write times with reference to write times given to the write data. By writing write data in the secondary storage system in an order of write times, the secondary storage system can always hold consistent data.

When write data is stored in the storage system in defiance of writing order, the end of a transaction cannot be identified in database processing, for example, which is a problem. In database processing, log data is first written in a storage system before data is written in the storage system. The log data includes information indicating the end of the transaction. Thus, when an error occurs before one transaction completes, a host computer can recover data to the state at the completion of the transaction with reference to the log data written in the storage system. However, when write data is stored in a storage system in defiance of writing order, data may be written in the storage system before the log data is written in the storage system. When an error occurs in the middle of a transaction under this condition, the host computer cannot process data having written until the middle of the transaction since the storage system does not hold the log data corresponding to the data. Thus, an in-progress result of the transaction remains in the storage system. Data having such an in-progress result of the transaction cannot be continuously used for sequent database processing.

As an example that an in-progress result of a transaction remains in a storage system, processing for transferring a deposit from Bank Account A to Bank Account B will be considered below. In this case, decreasing an amount in Account A and increasing an amount in Account B must be executed as one transaction. However, when an error occurs after increasing the amount of Account B (that is, after writing the balance after the increasing the amount of Account B) and before decreasing the amount of Account A (that is, before writing the balance after decreasing the amount of Account A), a period occurs in which the amount of the deposit in Account B is increased before the amount of deposit of Account A is decreased. In this case, when the log data is stored in a storage system thereof in advance, the host computer can recover data with reference to the log data. On the other hand, when the log data is not stored in a storage system thereof, the database processing cannot be recovered and inconsistent data remains in the storage system.

Similarly in a remote copy system, a failure may occur in a primary storage system and the primary storage system cannot be used after increasing the amount of Account B and before decreasing the deposit amount of Account A, an in-progress result of the transaction remains in a secondary storage system. In this case, when log data is not stored in the secondary storage system, improper processing is executed even though a secondary host computer takes over the transaction after that. Therefore, by holding consistent data in writing order, the validity of an order of related operations can be assured for related data thereof like the relationship between log data and data in database processing.

U.S. Pat. No. 6,092,066 discloses a technology by which data to be used by a computer is stored in a storage system and the storage system copies the data to another remote storage system so that the data can be held even when one storage system is unavailable due to a natural disaster or a fire.

SUMMARY

According to EP 0672985, when write data received by a primary storage system from a host computer is reflected to a secondary storage system, the write operations are performed in an order of write times given to the write data by the host computer. Therefore, when the host computer does not give write times to write data, the consistency of the write data stored in the secondary storage system cannot be kept. While a so-called mainframe host computer can give a write time to a write request, a so-called open-system host computer does not give a write time to a write request. Therefore, consistency cannot be kept for the write data from an open-system host computer according to the technology disclosed in EP 0672985.

U.S. Pat. No. 6,092,066 does not disclose the case that a host computer does not give a write time to write data, either.

Accordingly, a remote copy system implementing a technology for keeping consistency of data stored in a secondary storage system even when a host computer does not give a write time to write data is provided.

There is provided a remote copy system having plural primary storage systems each having a source logical volume in which data is written and plural secondary storage systems each having a target logical volume in which a copy of data stored in a respective source logical volume is written.

Once write data is received from a host computer, each of the plural storage systems creates write-data management information having sequential numbers based on an order of receiving write data for the received write data and reference information and sends, to one of the secondary storage systems, write-data, write-data information having write data and sequential numbers and reference information included in the write-data management information corresponding to the write data.

Each of the secondary storage systems having received write data and write-data information from the primary storage system records reference information included in write-data information having the largest sequential number among write-data information having serial sequential numbers. Each of the secondary storage systems stores, in the target logical volume in an order of sequential numbers, write data with write-data information including reference information having a value smaller than the reference information based on the smallest value reference information among reference information recorded in each of the secondary storage systems.

Thus, in the remote copy system, even when a host computer does not give a write time to write data, consistency can be kept among data stored in secondary storage systems.

DESCRIPTION OF THE PREFERRED EMBODIMENT

An embodiment of the invention will be described below. It should be understood that the invention is not limited by the embodiment described below.

FIG. 1is a diagram showing a configuration example of a computer system according to an embodiment.

The computer system includes plural primary storage systems (also called primary storage systems) A-100, a main frame host (MF) A-600, an open system host A-700, plural secondary storage systems (also called secondary storage systems) B-190, an MF B-690and an open system host B-790.

The plural primary storage systems A-100, the MF A-600and the open system host A-700are connected through an I/O path900. The plural secondary storage systems B-190, the MF B-690and the open system host B-790are similarly connected through an I/O path900. The MF B-690and the open system host B-790are normally at a standby state. The MF A-600, MF B-690, the open system host A-700and the open system host B-790are also connected to a network920.

Each of memories in the MF A-600and MF B-690stores an operating system (OS)610and application software (APP)620, which are to be executed by processors of the MF A-600and MF B-690. Each of memories in the open system host A-700and open system host B-790stores an OS710and APP720, which are to be executed by processors of the open system host A-700and open system host B-790. An I/O request issued from APP of the MF A-600, open system host A-700, MF B-690or open system host B-790through the OS is processed by the primary storage system A-100or the secondary storage system B-190through the I/O path900. Here, the APP620and the APP720include software such as DBMS.

Each of the plural primary storage systems A-100includes a control unit200, a control memory300, a cache400and one or plural disk devices. The control unit200includes a write-data receiving unit A-210, a write-data transfer unit A-220, a section-number creating unit A-230and a marker creating unit A-250. Management software A-800operates in the control unit200of one of the plural primary storage systems A-100. The management software A-800is software stored in the control memory300and is executable by the control unit200.

The control memory300stores group management information310, the other volume information320, write-data management information330, section-number information360, and operation-mode information380, which are to be used in processing to be performed by the control unit200. The processing will be described later.

The cache400is a high-speed memory mainly storing read data and/or write data. By using the cache400, a high I/O processing performance can be achieved.

Desirably, the control unit200, the control memory300and the cache memory400are duplexed for resistance to failures and usability, and a backup power source is prepared therefor.

The plural primary storage systems A-100are connected to each other through a transfer path910. Each of the primary storage systems A-100is similarly connected to one or plural secondary storage systems B-190through a transfer path930. Furthermore, each of the primary storage systems A-100is connected to the network920.

Each of the secondary storage systems B-190also includes a control unit200, a control memory300, a cache400and one or plural disk devices. The control unit200includes a write-data receiving unit B-211and a write-data reflecting unit240. Management software B-890operates in the control unit200of one of the plural secondary storage systems B-190. The management software B-890is software stored in the control memory300and is executable by the control unit200.

The control memory300stores group management information310, the other logical volume information320, write-data management information330, received write-data information370and operation-mode information380, which are to be used by the control unit200.

The cache400plays the same role as that of the primary storage system A-100.

The plural secondary storage systems B-190are connected to each other through a transfer path910. The secondary storage systems B-190are also connected to the network920.

Each of the primary storage systems A-100and the secondary storage systems B-190provides a logical volume500as a data storage area to the MF A-600, the open system host A-700, the MF B-690or the open system host B-790. In this case, the logical volume500may be one or plural disk devices. One of the logical volumes500does not have to be physically one disk device and may be a set of storage areas distributed to plural magnetic disk devices, for example. The logical volume500may have a redundant construction such as a mirror construction and a RAID construction having parity data.

The logical volume500provided by the primary storage system A-100to the MF A-600is a different kind from that of the logical volume500provided to the open system host A-700. Logical and/or physical interfaces of the I/O path900for sending I/o requests from the host computer to the logical volume500are different between the MF A-600and the open system host A-700. The same is true for the secondary storage system B-190, the MF B-690, and the open system host B-790. Especially, a write request from the MF A-600or the MF B-690includes a time of issuing the write request as a write time while a write request from the open system host A-700or the open system host B-790does not include a write time.

The plural primary storage systems A-100and the plural secondary storage systems B-190are connected through the transfer path930as described above. A copy of data stored in a logical volume of one of the primary storage system A-100and the secondary storage system B-190can be held in another logical volume of the other as described later. According to this embodiment, a copy of data stored in the logical volume500of the primary storage system A-100is held in the logical volume500of the secondary storage system B-190. Information on data update performed on the logical volume500of the primary storage system A-100is sent to the secondary storage system B-190through the transfer path930and is also reflected to the corresponding logical volume500of the secondary storage system B-190. The primary storage system A-100and the secondary storage system B-190have the other logical volume information320indicating a correspondence between the logical volumes as described later and execute processing of copying data as described above by using the other logical volume information320. Here, a logical volume of the primary storage system A-100and a logical volume of the secondary storage system B-190storing a copy of data stored in the logical volume of the primary storage system A-100are paired. The other logical volume information320is defined in the primary storage system A-100and the secondary storage system B-190by a management terminal (not shown) connecting to the network920based on an input by a user.

According to this embodiment, plural pairs exist between the primary storage system A100and the secondary storage system B-190, and the plural pairs are grouped.FIG. 2shows a conceptual diagram showing an example of the grouped pairs.

InFIG. 2, the dashed-lines indicate correspondences between the logical volumes500or the groups. According to this embodiment, an writing order of write data received by the primary storage system A-100, transfer of write data from the primary storage system A-100to the secondary storage system B-190and processing of reflecting write data in the secondary storage system B-190are managed for each of the groups, and a resource required for these kinds of processing are assigned to each of the groups.

When these kinds of processing are performed for each of logical volumes, many targets must be managed, which may be complicated. Furthermore, because of many targets to be processed, resources required for those kinds of processing may increase. On the other hand, the entire plural logical volumes500in the primary storage system A-100are grouped to one group, fine-grained management can not be performed. Especially, since requirements for performances of the logical volumes500are significantly different between a mainframe host and an open system host, the logical volumes500for a mainframe host and open system must be separately grouped, and processing must be performed thereon separately such that the separate groups can be manipulated separately by a user for processing or a tuning condition can be defined separately therefor. Therefore, by grouping as described above, flexible copy processing and management can be achieved as required by users and/or operations.

WhileFIG. 2schematically shows the case having one primary storage system A-100and one primary storage system B-190, the same is true for a case having plural primary storage systems A-100and plural secondary storage systems B-190. Each of the primary storage systems A-100and the secondary storage systems B-190holds group management information310in the control memory300for managing the above-described groups. The group management information310is defined in each storage system based on an input by a user through a management terminal (not shown) connecting to the network920, like the above-described other logical volume information320.

Next, write processing, transfer processing and reflection processing will be described. In the write processing, write data sent from the MF A-600or the open system host A-700to the logical volume500of the primary storage system A-100is written in the primary storage system A-100. In the transfer processing, write data is transferred to the secondary storage system B-190. In the reflection processing, write data received by the secondary storage system B-190is reflected in the secondary storage system B-190. Even when a host computer issues a write request to the primary storage system A-100without giving a write time to the write request, these kinds of processing can hold consistency in write data in the plural secondary storage systems B-190.

FIG. 3is a diagram showing an example of processing for a case where the primary storage system A-100receives a write request for a logical volume500(called source logical volume500) having data to be copied by remote copy processing from the MF A-600or the open system host A-700.

The write-data receiving unit A-210of the primary storage system A-100receives a write request from the MF A-600or the open system host A-700(step1000). Next, the write-data receiving unit A-210refers to the operation-mode information380stored in the control memory300(step1001) and checks the defined operation mode.

FIG. 4is a diagram showing an example of the operation-mode information380.

One of two operation modes of a section-number mode and a write-time mode is to be recorded as the operation mode information380. A user or manager can define one of the operation modes in the primary storage system A-100and the primary storage system B-190through a management terminal (not shown) connecting to the network920or a management terminal included in the primary storage system A-100or the primary storage system B-190.

The section-number mode is an operation mode suitable for a case where a write time may not be given to a write request issued from the host computer. Like this embodiment, the section-number mode is preferably defined as an operation mode in a system including the open system host A-700, which does not give a write time to a write request. The write-time mode is an operation mode suitable for a case where a write time is given to a write request issued from a host computer and is suitable for a case where the MF A-600giving a write time to a write request only writes write data in the primary storage system A-100.

Referring back toFIG. 3, when the operation mode is the section-number mode, the write-data receiving unit A-210stores write data in the cache400(step1005) and creates write-data management information330by giving a section number and a sequential number to the write data (step1006).

A section number is a number to be given with reference to the section-number information360stored in the control memory300, the details of which will be described later. In order to give a sequential number, the write-data receiving unit A-210refers to a sequential number registered with the group management information310of a group to which a logical volume subject to write processing belongs, records a value resulting from the addition of 1 to the registered sequential number as a sequential number of write data and records the new sequential number in the group management information310to update the sequential number recorded in the group management information310.

FIG. 5is a diagram showing an example of the group management information310of each group.

A group ID is an ID for identifying a group in the primary storage system A-100(or secondary storage system B-190). Sequential numbers are numbers to be serially given to write data to be written in the logical volume500belonging to this group, which is received by the primary storage system A-100from a host computer. The initial value of the sequential number is zero (0), for example, and is sequentially incremented by one (1). At the step1006inFIG. 3, the write-data receiving unit A-210gives a value resulting from the addition of one (1) to the sequential number to the write-data management information330. The number of logical volumes is a number of logical volumes belonging to this group. The logical volume number is an ID for identifying a logical volume belonging to this group in the storage system A-100. The other storage system ID is an ID (such as a serial number) for identifying the copying secondary storage system B-190paired with the primary storage system A-100. The other group ID is an ID for identifying a group to which the copying logical volume (called target logical volume)500belongs in the other storage system (destination secondary storage system B-190).

FIG. 6is a diagram showing an example of the write-data management information330of each write data created by the step1006inFIG. 3.

The logical volume ID is an ID of the logical volume500in which write data is to be written. The write address is a write starting address of write data in a logical volume indicated by the given logical volume ID. A write data length is a length of the write data. The write-data pointer is a storage starting address of the write data in the cache400.

The sequential numbers are numbers serially given to the write data in a group to which the logical volume500belongs in which the write data is to be written. The sequential number is a value resulting from the addition of one (1) to a value obtained by the write-data receiving unit A-210from the group management information310at the step1006inFIG. 3as described above. The transfer requirement bit is a bit indicating whether the write data must be transferred to the secondary storage system B-190or not. The write-data receiving unit A-210sets the transfer requiring bit to ON when the write-data receiving unit A210receives the write data and creates write-data management information330(at the step1006and step1003inFIG. 3).

The write time given to a write request or a section number obtained from the section-number information360is recorded in the write time/section number field. In other words, when an operation mode is the section-number mode at the step1001inFIG. 3, the write time/section number field records a section number obtained by the write-data receiving unit A-210based on the section-number information360at the step1006. The write time/section number field records a write time received from a host computer along with write data, as described later, when the operation mode is the write-time mode at the step1001inFIG. 3. Adopting the same format for section numbers and write times can reduce the storage area for the write data management information330. Furthermore, as described later, partially common processing of reflecting write data in the secondary storage system B-190can be performed between the two operation modes. The write-data management information330is managed in a list structure for each group, for example.

Referring back toFIG. 3, if the operation mode is the write-time mode at the step1001, the write-data receiving unit A-210stores write data in the cache400(step1002), obtains a sequential number thereof in the same manner as that of the step1006, and records a write time received from a host computer along with the obtained sequential number and the write data to create the write-data management information330(step1003).

Finally, at the step1004, the write-data receiving unit A-210reports the write completion to the MF A-600or the open system host A-700(step1004).

The processing shown inFIG. 3excludes processing requiring time such as processing of writing write data stored in the cache400into a physical medium such as a magnetic disk device for the logical volume500and processing of transferring write data to the secondary storage system B-190. Those kinds of processing are performed later asynchronously in proper timing. Thus, the primary storage system A-100can receive a write request and send a write completion report within a short time, and the primary storage system A-100can quickly respond to the MF A-600or the open system host A-700therefore.

FIG. 7is a diagram showing an example of processing in a case where the management software A-800operated in the control unit200commands the write-data receiving unit A-210of the primary storage system A-100to put the processing of a write request on hold and to update section numbers.

As described later, by implementing the processing shown inFIG. 7, write data having already undergone the write processing shown inFIG. 3into the source logical volume500of the primary storage system A-100when the processing is implemented can be synchronously reflected to the target logical volume500in the secondary storage system B-190. Thus, the consistency can be established among write data stored in the target logical volume600in the plural secondary storage systems B-190.

The management software A-800in one of the plural primary storage systems A-100first commands all of the primary storage systems A-100to put processing of a write request on hold (step1100). In response to the command, the write-data receiving unit A-210of each of the primary storage systems A-100puts the processing of write requests received from a host computer after the reception of the command by the step1100on hold (step1101) and reports the fact that holding write requests has started to the management software A-800(step1102). Each of the primary storage systems A-100may report to the management software A-800not after putting processing of write requests on hold but upon reception of the command issued by the step1100.

The management software A-800checks whether all of the primary storage systems A-100having received the command at the step1100have reported or not and then goes to next processing (steps1103and1104).

Next, the management software A-800commands all of the primary storage systems A-100to update section numbers (step1105). The command includes, as a parameter, a new section number, which is a value resulting from the addition of one (1) to a section number selected when the management software A-800issues the command previously.

In response to the command issued by the management software A-800at the step1105, the section-number creating unit A-230of each of the primary storage systems A-100records the received section number in the section-number information360shown inFIG. 8(step1106). Then, the section-number creating unit A-230reports the completion of the section-number update to the management software A-800(step1107).

FIG. 8is a diagram showing an example of the section-number information360stored in the control memory of the primary-storage system A-100. Each of the primary storage systems A-100stores the section number included in the section-number update command received from the management software A-800at the step1105inFIG. 7in the control memory as the section-number information360. Every time the section-number creating unit A-230receives the section-number update command, the section-number creating unit A-230update the section number stored as the section-number information360(step1106inFIG. 7).

Referring back toFIG. 7, the management software A-800checks whether all of the primary storage systems A-100having received the section-number update command at the step1005have reported or not and goes to the next processing (steps1108and1109).

The management software A-800commands all of the primary storage systems A-100to cancel the holding of processing of write requests (step1110). In response to the command, the write data receiving unit A-210of each of the primary storage systems A-100cancels the holding of processing of write requests (step1111) and reports the fact that the holding has been cancelled to the management software A-800(step1112).

FIG. 13is a diagram showing an example of correspondences of section numbers and write data.

The horizontal axis indicates time, and every time the processing of updating section numbers shown inFIG. 7is implemented, the value of the section number stored in each of the storage systems A-100is incremented by one (1). As described with reference toFIG. 3, when the operation mode is the section-number mode, the section number held by each of the primary storage systems A-100at the time of the reception of a write request (indicated by “O” inFIG. 13) received by each of the primary storage systems A-100is given to write-data management information relating to write data subject to the write request.

Notably, when the operation mode is the write-time mode, the holding of processing of write requests and the processing of updating section numbers shown inFIG. 7are not required. The two operation modes of the section-number mode and the write-time mode allow the implementation of processing as required only.

FIG. 9is a diagram showing an example of processing of transferring write data from the primary storage system A-100to the secondary storage system B-190. The write data transfer unit A-220refers to a list of the write data management information330and obtains write data to transfer (that is, write data with write-data management information330having the transfer-requirement bit at ON). Then, the write-data transfer unit A-220refers to the write-data management information330, group management information310and the other logical volume information320and creates write-data information (step1200).

The write-data information includes a write address, write data length, sequential number and section number or write time referred from the write-data management information330, the other storage system ID and the other logical volume ID referred from the other logical volume information320and the other group ID referred from the group management information310by using a logical volume ID recorded in the write-data management information330.

FIG. 10is a diagram showing an example of the other logical volume information320of each logical volume.

The logical volume ID is an ID for identifying a source logical volume of the primary storage system A-100, and the other storage system ID is an ID (such as serial number) for identifying the secondary storage system B-190having a target logical volume paired with the source logical volume. The other logical volume ID is an ID for identifying a target logical volume in the secondary storage system B-190.

Referring back toFIG. 9, the write-data transfer unit A-220next transfers the write data identified at the step1200and the write-data information prepared at the step1200to the secondary storage system B-190(step1201).

The write-data receiving unit B-211of the secondary storage system B-190stores the received write data and write-data information in the cache400(step1202) and creates write-data management information330from write-data information (step1203). Items of the write-data management information330of the secondary storage system B-190are the same as those of the write-data management information330of the primary storage system A-100shown inFIG. 6. However, the write-data management information330stored in the control memory300of the secondary storage system B-190is different from the write-data management information of the primary storage system A-100in that a logical volume ID is an ID of the target logical volume500, the write-data pointer is a storage starting address of write data in the cache400of the secondary storage system B-190, and the transfer requirement bit is always OFF. The other is the same.

The secondary storage system B-190has the group management information310having the same items as those of the group management information310of the primary storage system A-100. However, the group ID of the group management information310of the secondary storage system B-190is an ID for identifying a group to which the target logical volume500belongs. The other storage system ID is an ID of the primary storage system A-100having the source logical volume500paired with the target logical volume500. The other group ID is an ID for identifying a group to which the source logical volume500belongs in the primary storage system A-100indicated by the other storage system ID.

Furthermore, the secondary storage system B-190has the other logical volume information320having the same items as those of the primary storage system A-100shown inFIG. 10. However, the logical volume ID is an ID for identifying the target logical volume500. The other storage system ID is an ID for identifying a primary storage system to which the source logical volume500paired with the target logical volume500belongs. The other logical volume ID is an ID for identifying the source logical volume500in the primary storage system A-100indicated by the other storage system ID.

Referring back toFIG. 9, the write-data receiving unit B211next checks whether the section number or write time included in the received write-data information is the latest or not by a method, which will be described later, next (step1204). If so, the write-data receiving unit B-211records the section number or write time in the received write-data information370(step1205).

FIG. 11is a diagram showing an example of the received write-data information370for each group.

The group ID is an ID for identifying a group in the secondary storage system B-190. The received latest section number or latest write time is the largest value among section numbers or write times of write-data information given to write data for the target logical volume500belonging to a group indicated by the group ID, which have been received by the write-data receiving unit B-211of the secondary storage system B-190up to that point. However, when some sequential numbers are missing in write-data information, the section number or write time included in write-data information having the largest sequential number among write-data information having serial sequential numbers is recorded as the received latest section number or latest write time.

Referring back toFIG. 9, at the step1205, the write-data receiving unit B-211compares the section number or write time held as the received write-data information370and the section number or write time included in the received write-data information. If the section number or write time included in the received write data information has a larger value, the write-data receiving unit B-211updates the received write-data information370to the value.

However, when a part of write data has not been received with reference to sequential numbers included in write-data information (that is, when some sequential numbers are missing in the received write data information and a part of write data is missing), the section number or write time included in the write-data information received along with the last write data (write data immediately before missing write data) among write data having serial sequential numbers (that is, the section number or write time included in write-data information having the largest sequential number among write-data information having serial sequential numbers) and the value of the received write-data information370are compared. Then, the processing at the step1205is performed.

Notably, plural write data may be transferred in parallel between the write-data transfer unit A-220and the write-data receiving unit B-211by using plural links set between the primary storage system A-100and the secondary storage system B-190. When the parallel transfer is performed, the write-data receiving unit B-211may not always receive write data in an order of sequential numbers. However, since write data is reflected to each group in the secondary storage system B-190in an order of sequential numbers as described later, write data is also written in the target volume500in the same order as the write order for the source volume.

Finally, the write-data receiving unit B-211reports the completion of write-data reception to the write-data transfer unit A-220(step1206). The write-data transfer unit A-220having received the report of the write-data reception sets the transfer requirement bit of the write-data management information330at OFF for the write data corresponding to the completion report. When the transfer requirement bit of the write-data management information330is set to OFF, the write data can be discarded from the cache400in the primary storage system A-100(that is, the storage area on the cache400storing the write data can be opened) on the condition that the source logical volume500of the primary storage system A-100holds the write data.

FIG. 12is a diagram showing an example of processing of reflecting write data to a target logical volume500in the secondary storage system B-190.

The management software B-890in one of the plural secondary storage systems B-190checks the operation mode first with reference to the operation mode information380stored in the control memory300of the secondary storage system B-190(step1301).

If the defined operation mode is the section-number mode (step1302), the section numbers recorded in the received write-data information370for all groups are obtained from all of the secondary storage systems B-190. Then, the smallest section number among the obtained section numbers is obtained (step1305). Next, the management software B-890commands all of the secondary storage systems B-190to reflect write data having the section numbers smaller than the section number obtained at the step1305in the write-data management information330to the target logical volume500(step1306).

The details of the operation mode information380of the secondary storage system B-190are basically the same as those of the operation mode information380of the primary storage system A-100. One of the two operation modes including the section-number mode and write-time mode is to be stored and is selected and defined by a user or an administrator.

If the operation mode is the write-time mode at the step1302, the management software B-890obtains write times recorded in the received write-data information370from all of the secondary storage systems B-190for all groups therein. The management software B-890obtains the smallest (that is, the oldest) write time among the obtained write times (step1303). Next, the management software B-890commands the secondary storage systems B-190to reflect write data up to the write time obtained at the step1303to the target logical volume500(step1304).

The write-data reflecting unit240of each of the secondary storage systems B-190having received a command issued from the management software B-890at the step1304or step1306refers to the write-data management information330and group management information310based on the command and writes the write data in the target logical volume500in an order of sequential numbers in each group (step1307).

Then, once the write-data reflecting unit B-240have completed the processing of writing the write data corresponding to the command issued at the step1304or step1306to the target logical volume500, the write-data reflecting unit B-240reports the completion of the reflection to the management software B-890(step1308).

The management software B-890determines all of the secondary storage systems B-190have reported the completion of normal reflection for the command issued at the step1304or step1306(step1309or step1310) and finishes the reflection processing.

Since, in the processing shown inFIG. 12, the management software B-890obtains the smallest section number or write tine at the step1305or step1303, all of the write data corresponding to the command issued at the step1306or step1304have already arrived at the corresponding plural secondary storage systems B-190. Therefore, the write data corresponding to the command issued at the step1306or step1304can be always written into the target logical volume500. Therefore, when the operation mode is the section-number mode, write data having the section numbers up to the section number specified by the command at the step1306are always reflected to the target logical volumes500in the secondary storage systems B-190. When the operation mode is the write-time mode, write data having the write time up to the write time specified by the reflection command at the step1304are always reflected to the logical volumes in the secondary storage systems B-190. As a result, write data written in the primary storage systems A-100up to a certain instant are reflected to the plural secondary storage systems B-190, and the consistency can be achieved among data stored in the target logical volumes.

As described above, when the format of the section numbers is the same as that of the write times, a common parameter format can be adopted for commands to refer to section numbers or write times, for example, and for commands to reflect. Furthermore, partially common reflection processing shown inFIG. 12can be performed.

The management software A-800and the management software B-890repeat the above-described processing. Thus, an update to the source logical volume500in the primary storage system A-100is regularly reflected to the target logical volume500in the secondary storage system B-190. Because of the above-described processing, the consistency among copies created in the plural primary storage systems A-100and plural secondary storage systems B-190can be always kept even when a host computer is included which does not give write times to write data. Therefore, the MF-B690or open system host B-790can take over the task by using consistent data stored in the secondary storage systems B-190.

The primary storage systems A-100and the secondary storage systems B-190do not have to be connected in a one-to-one relationship. When the logical volumes500and groups are paired and mapped as sources and targets, the same number of primary storage systems A-100and secondary storage systems B-190do not have to be provided.

In addition to the processing, based on a command from a user, for example, the marker creating unit A-250of each of the primary storage systems A-100may uniquely create special write data (called marker) only for write-data information excluding actual write data and give the updated section number or write time (that is, the section number or write time resulting from the addition of 1) to the marker. Then, after the primary storage system A-100transfers a marker to the secondary storage system B-190, the transfer of write data may be terminated. In this case, the secondary storage system B-190reflects write data before the received marker to the target logical volume500and terminates the reflection of write data. Thus, the remote copy processing by the primary storage system A-100and secondary storage system B-190can be terminated (which is called suspend state).

In this case, all of the write data before the suspend state are reflected to the target logical volumes in the secondary storage systems B-190. In the suspend state, the operation mode of the primary storage systems A-100and/or secondary storage systems B-190can be switched to same mode. The section-number information360and/or received write time information370are initialized when the operation mode is switched.

While, in the above-described processing, the transfer path910is used to exchange various commands and reports and/or information between the management software A-800or management software B-890and the primary storage system A-100or secondary storage system B-190, the network920may be used. Furthermore, the logical volume500, which is not subject to the holding of processing of write requests, may be provided in the secondary storage system A-100, and a command to update a section number from the management software A-800may be executed by sending a write request to the logical volume500.

While, in the write data transfer processing described above, the write-data transfer unit A-220transfers write data to the write-data receiving unit B-211first, the write-data transfer unit B-211may issue a request for transferring write data to the write-data transfer unit220first. Then, the write-data transfer unit A-220having received the request may transfer the write data to the write-data transfer unit B-211. By using a write-data transfer request, the pace of write-data transfer can be adjusted in accordance with a state and/or load of processing and/or amount of stored write data in the secondary storage system B-190.

While, in the above-described processing, write data is stored in the cache400, another logical volume500for storing write data may be prepared and write data may be stored in the logical volume500once. Since the logical volume500generally has a larger capacity than that of the cache400, a more amount of write data can be stored.

The write time does not have to be in years, months, dates, hours, minutes, seconds, milliseconds, microseconds or nano seconds but may be sequential numbers.

While, in the description above, the management software A-800exists in the storage system A-100and the management software B-890exists in the storage system B-190, the management software A-800and the management software B-890may be operated in one of the MF A-600, MF B-690, open system host A-700, open system host B-790, storage system A-100and storage system B-190when commands, reports and/or information can be exchanged through the transfer path910or the network920. The management software A-800and/or the management software B-890may exist in another computer (such as a management terminal on the network920), not shown, connecting to the storage system A-100and the storage system B-190. When the storage system A-100and the storage system B-190controls the processing for achieving the consistency among target logical volumes as described above, the consistency among copies can be achieved only by processing by the storage system A-100and the storage system B-190, which does not need to use resources on a host. A user or an administrator can select the storage system A-100and/or the storage system B-190in which the management software A-800and/or the management software B-890operates from the management terminal (not shown) connecting to the network920or a management terminal of each of the primary storage systems A-100and secondary storage systems B-190.

A user or an administrator can further select or delete from the subjects the storage system A-100, storage system B-190and/or a group therein to be processed by the management software A-800and/or management software B-890from a management terminal (not shown) connecting to the network920or a management terminal of each of the primary storage systems A-100and secondary storage systems B-190.

The MF-A600, MF B-690, open system host A-700, open system host B-790and other computers, not shown, can read and check section numbers recorded in the section-number information360of the storage systems A-100, section numbers or write times recorded in the received write-data information370of the storage systems B-190.