Shared-bandwidth multiple target remote copy

In one embodiment, a method is provided for sharing bandwidth in a data processing system having a plurality of locations. The method includes splitting data into a plurality of data elements. The method also includes sending, from a sender, each one of the plurality of data elements to a different location selected from the plurality of locations. The method further includes sending a message from the sender to each of the locations receiving one of the data elements, where the message is sent from the sender separate from the data elements. The message includes information about the data elements, and for each location receiving one of the data elements, an address of the location. The message includes all of the addresses of all of the locations of the plurality of locations receiving one of the data elements of the data.

BACKGROUND

The present invention relates to copying data between storage devices. In particular, the invention relates to a method for sharing bandwidth between multiple remote copy targets.

Peer to Peer Remote Copy (PPRC) is a method to replicate a storage volume to another storage unit over extended distances to a remote site. PPRC is used to provide business continuity and disaster recovery capabilities. A PPRC may be performed synchronously or asynchronously. In Synchronous PPRC, each write to a local site is performed to the remote site as well. The Synchronous PPRC is complete when the write completes to both sites. An example of a Synchronous PPRC implementation is IBM® Metro Mirror (IBM is a registered trademark of International Business Machines Corporation in the United States, other countries, or both.) In Asynchronous PPRC, each write is made to the local site, and then copied to the remote site when time permits. The Asynchronous PPRC is complete when the write completes to the local site. Different copy functions may be combined to provide remote copy functionality. For example, a Point in Time copy may be made at the local site between a source volume and a target volume, and then a remote copy may be made from the target volume to a remote site. An example of a combinatorial implementation is IBM Global Mirror. PPRC may be used to provide very fast data recovery due to failure of the primary site.

PPRC may also be extended to more than one remote site to improve business continuity and disaster recovery capabilities.

If an organisation has three sites, A, B, and C, and wishes to replicate data from a storage system at A onto similar systems at B and C using a PPRC method, a high-bandwidth network connection is required from A to B, and from A to C. An example of a suitable network connection protocol is Fibre Channel (FC). However, FC links are expensive as they are priced by distance and bandwidth.

BRIEF SUMMARY

In one embodiment, a method is provided for sharing bandwidth in a data processing system having a plurality of locations. The method includes splitting data into a plurality of data elements. The method also includes sending, from a sender, each one of the plurality of data elements to a different location selected from the plurality of locations. Each data element is different. The method further includes sending a message from the sender to each of the locations receiving one of the data elements, where the message is sent from the sender separate from the data elements. The message includes information about the data elements, and for each location receiving one of the data elements, an address of the location. The message includes all of the addresses of all of the locations of the plurality of locations receiving one of the data elements of the data. Also, the data is split into the plurality of data elements at a disk level, such that a storage system copying a plurality of disks splits a first half of the disks into a first one of the data elements, and splits a second half of the disks into a second one of the data elements.

In another embodiment, a method is provided for sharing bandwidth in a data processing system having a plurality of locations. The plurality of locations comprises a first location and plurality of further locations. The method includes receiving a first data element of first data from the first location and other data elements of the first data from the further locations. Each data element of the first data is different. Also, the method includes receiving, after receiving the first data element of the first data from the first location, a message from the first location. The message is sent from the first location separate from the first data element received from the first location. The message comprises an address of each of the locations of the plurality of locations having one of the data elements of the first data, such that the message includes all of the addresses of all of the locations of the plurality of locations receiving one of the data elements of the first data. Also, the first data has been split into the data elements of the first data at a disk level, such that a storage system copying a plurality of disks has split a first half of the disks into a first one of the data elements, and has split a second half of the disks into a second one of the data elements.

Computer code for implementing the methodology presented herein may be embodied on a computer program product.

DETAILED DESCRIPTION

The following description discloses several preferred embodiments of systems, methods and computer program products for sharing bandwidth in a data processing system.

One general embodiment of the present invention provides a computer management apparatus for sharing bandwidth in a data processing system, where the data processing system comprises a plurality of locations, the apparatus comprising: a split component configured to split data into a plurality of data elements; a send component configured to send each one of the plurality of data elements to a different location selected from the plurality of locations in response to the split component splitting the data, where each data element is different, responsive to the split component splitting the data; and a message component configured to send a message to each of the locations.

Advantageously, various embodiments of the present invention allow bandwidth requirements of a remote copy environment to be reduced, without reducing the amount of data that may be copied.

Another general embodiment of the present invention provides a computer management apparatus for sharing bandwidth in a data processing system, where the data processing system includes a plurality of locations typically in communication with a host, where the plurality of locations includes a first location and plurality of further locations. The apparatus includes: a receive component configured to receive a first data element; an analyze component configured to receive a message, where the message comprises an address of each of the further locations; and a send component configured to send the first data element to each of the further locations of the plurality of further locations in response to the analyze component determining the address of each of the further locations.

Advantageously, various embodiments of the present invention enable an apparatus on a remote location to forward on data to the other remote locations, and also to receive data from the other remote locations. Received data may be combined to replicate the data at the local location.

In yet another general embodiment, a method for sharing bandwidth in a data processing system that includes a plurality of locations includes splitting data into a plurality of data elements; sending each one of the plurality of data elements to a different location selected from the plurality of locations, wherein each data element is different; and sending a message to each of the locations.

In a further general embodiment of the present invention, a method for sharing bandwidth in a data processing system, wherein the data processing system comprises a plurality of locations, typically in communication with a host, and where the plurality of locations comprises a first location and plurality of further locations, the method comprising the steps of: receiving a first data element; receiving a message, wherein the message comprises an address of each of the further locations; and sending the first data element to each of the further locations of the plurality of further locations, responsive to determining the address of each of the further locations.

In yet another general embodiment, a system for sharing bandwidth in a data processing system having at least one local location and a plurality of remote locations is provided. The system includes: a local apparatus operable on the at least one local location, wherein the local apparatus comprises: a split component operable for splitting data into a plurality of data elements; a first send component, operable for sending each one of the plurality of data elements to a different remote location selected from the plurality of remote locations, wherein each data element is different, responsive to the split component splitting the data; and a message component, operable for sending a message to each of the remote locations; and a remote apparatus, operable on each remote location, wherein the remote apparatus comprises: a receive component, operable for receiving a first data element selected from the plurality of data elements; an analyze component, operable for receiving the message, wherein the message comprises an address of each of the other remote locations; and a second send component, operable for sending the first data element to each of the other remote locations, responsive to the analyze component determining the address of each of the other remote locations.

Any combination of one or more computer readable medium(s) may be utilized. The computer readable medium may be a computer readable signal medium or a non-transitory computer readable storage medium. A non-transitory computer readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples (a non-exhaustive list) of the non-transitory computer readable storage medium include the following: a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), a portable compact disc read-only memory (e.g., CD-ROM), a Blu-ray disc read-only memory (BD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the context of this document, a non-transitory computer readable storage medium may be any tangible medium that is capable of containing, or storing a program or application for use by or in connection with an instruction execution system, apparatus, or device.

Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fibre cable, RF, etc., or any suitable combination of the foregoing.

FIG. 1is a block diagram depicting a data processing system10, in accordance with the prior art, and which may be modified and improved to include an embodiment of the present invention may be implemented.

The illustrated data processing system10comprises a host server node subsystem15having a set of server nodes20, which are connectable through a network30to a back-end storage subsystem90. A network30typically comprises network devices31, for example switches, and cabling that connect a server node subsystem15to a hardware back-end storage subsystem90. The storage subsystem90may comprise a variety of physical storage devices having, for example, stand-alone a Just a Bunch of Disks (JBOD) device50, and a Redundant Array of Independent Disks (RAID) array42. The RAID array42comprises a plurality of storage devices60. Devices42,50may be presented to the server node subsystem15as a set of physical or logical storage volumes (not depicted). Typically the system10is managed by a management subsystem70comprising management servers75, connectable to the server node subsystem15, the storage subsystem90, and the network devices31through the network30or through a separate Local Area Network (LAN)95. Typically, a RAID Controller40controls the functionality of the RAID array42, including data accesses and power controls to the individual storage devices60. Read and write commands may be sent to the storage subsystem90by a requester (not depicted) that may be an application program operable in the data processing system10. A further storage subsystem92may be present at a remote site.

FIG. 2is also a block diagram depicting a data processing system200, in accordance with the prior art, and which may be modified and improved to include an embodiment of the present invention.FIG. 2depicts an exemplary data processing system200with a local site210, and two remote sites215,220. A host server node subsystem205is connectable through a network250to a local back-end storage subsystem A225at local site210. The local site210is connectable to remote site B215through output network255and input network (not depicted). Local site210is connectable to remote site C220through output network260and input network (not depicted). There are also network connections275,285between remote site B215and remote site C220. Remote site B215comprises a storage subsystem B230, and remote site C220comprises a storage subsystem C235.

As an example, according to the prior art, if replication is required at rate of 10 Mbps, the network connections from the local site210to remote site B215(A→B), and from local site210to remote site C220(A→C) each need to have 10 Mbps of bandwidth. The same stream of data is sent from local site210to remote site B215(A→B) as is sent from local site210to remote site C220(A→C). For symmetry and redundancy, there is a link275,285from remote site B215to remote site C220, also with a 10 Mbps bandwidth. The host server node subsystem205writes data on network250to the local site210at 10 Mbps. To replicate writes to remote site B215and remote site C220, network255,260with 10 Mbps of bandwidth are required. The remaining networks275,285are idle, except for non-data traffic.

Data may be sent from A→B, and then remote site B215may forward the data to remote site C220. This still requires the links to be able to cope with 10 Mbps of bandwidth.

FIG. 3, which may be read in conjunction withFIGS. 4 and 5, is a high-level exemplary schematic flow diagram300depicting typical operation method steps performed for copying data in a data processing system, in accordance with a preferred embodiment of the present invention.FIG. 4, is an exemplary block diagram depicting a computer management apparatus in which various embodiments of the present invention may be embodied.FIG. 5is also a block diagram depicting a data processing system500, in which a preferred embodiment of the present invention may be implemented.FIG. 5depicts an exemplary data processing system500with a local site510and two remote sites515,515.

Referring toFIGS. 4-5, respectively, in a preferred embodiment of the present invention, the data is split at local site445,510into portions. One portion is sent on network555to remote site B,450,515and the other portion is sent on network560to remote site C455,520. Remote site B450,515comprises a storage subsystem B530and remote site C455,520comprises a storage subsystem C535. The remote sites450,515,455,515send each other the portion that they were sent by the local site445,510. In a preferred embodiment the portions are of equal size. Advantageously, each network555,560need only provide half the bandwidth of the prior art, e.g., as depicted inFIG. 2, but could provide faster connections.

To illustrate a preferred embodiment of the invention, an exemplary bandwidth of 10 Mbps will be used, as the required replication rate.

Referring toFIG. 3, and also with reference to the components ofFIGS. 4 and 5, the method starts at step301. Steps305,310,315and320are operable at the local site445,510. At step310, a receive component410of a computer management apparatus400at local site445,510receives data from a host server node subsystem460,505over the network550at 10 Mbps. At step310, a split component420of a computer management apparatus400operable at the local site445,510splits the received data into different data elements: portion B and portion C. In this example, the data is split into two equal sized elements, referred to below as portions.

At step315, a send component415at the local site445,510sends portion B to remote site B450,515, and portion C to remote site C455,520over the respective output networks555,560. The bandwidth required for both connections is 5 Mbps.

At step320, a message component430at the local site445,510sends a message to each of the remote sites450,515,455,520. The message comprises an address of each of the locations. In one approach, the message comprises, for example, information about both portions, and the address of the remote sites450,515,455,520.

Steps325,330,335,340and345are operable at each of the remote sites. As an example, remote site B450,515, is used to illustrate the steps. However, an equivalent set of steps is operable at remote site C455,520. At step325, a receive component at remote site B450,515receives a first data element, e.g., portion B. At step330, an analyze component425at remote site B450,515receives the message sent at step320by the message component430at the local site445,510. The analyze component425at remote site B450,515also analyzes the message to determine information about portion C, and the address of remote site C455,520. At step335, a send component415at remote site B450,515sends portion B to remote site C455,520over a network575at 5 Mbps.

At step340, the receive component at remote site B450,515receives a second data element, e.g., portion C from remote site C455,520over a network585at 5 Mbps. At step345, the analyze component determines whether all portions have been received. If further portions are required, the method returns to step340. However, if all portions have been received, at step350, a combine component405at remote site B450,515combines the portions, e.g., portion B and portion C and stores the resulting data. The method ends at step399.

FIG. 3is set forth as a logical flow chart diagram. As such, the depicted order and labelled steps are indicative of one embodiment of the presented method. Other steps and methods may be conceived that are equivalent in function, logic, or effect of one or more steps or portions thereof, of the illustrated method. Additionally the format and symbols employed are provided to explain the logical steps of the method and are understood not to limit the scope of the method.

In an alternative embodiment, the portions are of different size. Moreover, multiple data split algorithms may be used, for example by disk, or by batch.

The split of data may split at the disk level, so a storage system copying ten disks may split the first five disks into portion B, and the second five disks into portion C. In this case, the replication protocol is unaware of the split as this is a static configuration. It is limited, though, as it required the input/output (IO) workload to the ten disks to be spread such that a similar amount of data is sent to each disk. IO to any given disk may never reach the full performance of the inter-site links.

In an alternative embodiment, data is split at a batch level. Batch IO operations on local site445,510may be applied on remote sites450,515,455,520. Each batch IO completing all writes from one batch before starting the next. Even batches may be sent (A→B→C). Odd batches may be sent (A→C→B). The combine component405may then combine the batches in the correct order.

In an alternative embodiment data is split at the write level. Writes may be sent in any order to the remote sites450,515,220,455,520, if the local site210,445,510has not yet completed the write back to the host server node subsystem460,505. In this case, writes may be sent to each location alternately, e.g., (A→B→C), and (A→C→B). Messages regarding completion of IO may either come back the same path or shortcut directly back to local site445,510.

In a preferred embodiment, data recovery from remote site B450,515, to local site445,510use the method ofFIG. 3to send portion B (B→A), and portion C (B→C→A). In an alternative embodiment, portion B is sent (B→A), and portion C (C→A), using the portion stored on remote site C455,520.

The method ofFIG. 3may also be applied to further remote sites. An example is depicted inFIG. 6.

FIG. 6is also block diagram depicting a data processing system600in which a preferred embodiment of the present invention may be implemented.FIG. 6depicts an exemplary data processing system600similar to that500shown inFIG. 5, but with an additional remote site D622(see also458ofFIG. 4), comprising a storage subsystem D640. Remote site D622is connectable through network connections696,698to remote site C520(see also455ofFIG. 4), and through network connections565,585to remote site B515(see also450ofFIG. 4). Data received at the local site510at 10 Mbps is split into three portions, and sent to the remote sites515,520,622at about a third of 10 Mbps (e.g., 3.3 Mbps). Each remote site515,520,622forwards on the portion that it received to the other remote sites515,520,622. The portions are combined and stored when all portions have been received. In this example connections are made at 3.3 Mbps between remote sites515,520,622.

It will be clear to one of ordinary skill in the art that all or part of the method of the preferred embodiments of the present invention may suitably and usefully be embodied in a logic apparatus, or a plurality of logic apparatus, comprising logic elements arranged to perform the steps of the method and that such logic elements may comprise hardware components, firmware components or a combination thereof.

It will be appreciated that the method and arrangement described above may also suitably be performed fully or partially in software running on one or more processors (not depicted in the Figures), and that the software may be provided in the form of one or more computer program elements carried on any suitable data-carrier (also not depicted in the Figures) such as a magnetic or optical storage device or the like.

For the avoidance of doubt, the term “comprising”, as used herein throughout the description and claims is not to be construed as meaning “consisting only of”. Also for the avoidance of doubt, copying one location to another, as used herein throughout the description and claims, is to be construed as meaning copy the data contents of one location to the other location.