Merging an out of synchronization indicator and a change recording indicator in response to a failure in consistency group formation

A first data structure stores indications of storage locations that need to be copied for forming a consistency group. A second data structure stores indications of new host writes subsequent to starting a point in time copy operation to form the consistency group. Read access is secured to a metadata storage area and a determination is made as to whether the second data structure indicates that there are any new host writes. In response to determining that the second data structure indicates that there are new host writes, write access is secured to the metadata storage area, the first data structure is updated with contents of the second data structure to determine which additional storage locations need to be copied for formation of a next consistency group, and the second data structure is updated to indicate that that the second data structure is in an initialized state.

BACKGROUND

The disclosure relates to a method, system, and article of manufacture for merging an out of synchronization indicator and a change recording indicator in response to a failure in consistency group formation.

Certain mirroring mechanisms provide data replication over extended distances between two sites for disaster recovery. If adequate bandwidth exists, such mirroring mechanisms may provide a recovery point objective of as low as 3-5 seconds or less between the two sites at extended distances, with little or no performance impact on the application at the primary site. Certain mirroring mechanisms may copy the data asynchronously and also form a consistency group at a regular interval, allowing a clean recovery of data.

Certain mirroring mechanisms may pause updates of the primary volumes and then use a bitmap to drain updates from the primary volumes to the secondary volumes at the remote site. After all primary updates have been drained, the secondary volumes are used as the source for a point in time copy to tertiary volumes at the recovery site. This ensures that the tertiary copy of the volumes has a point-in-time consistency. By grouping many volumes into a single session, multiple volumes may be copied to the recovery site simultaneously, while maintaining point-in-time consistency across those volumes.

Such mirroring mechanisms may control the formation of consistency groups for data consistency. A consistency group is a collection of volumes across multiple storage units that are managed together when creating consistent copies of data. The order of dependent writes is preserved in consistency groups. The formation of these consistency groups may be controlled by a primary storage unit, which sends commands over remote mirror and copy volumes to remote storage units.

SUMMARY OF THE PREFERRED EMBODIMENTS

Provided are a method, a system, and a computer program product in which a first data structure stores indications of storage locations that need to be copied for forming a consistency group. A second data structure stores indications of new host writes subsequent to starting a point in time copy operation to form the consistency group, where the first data structure and the second data structure are maintained in one or more metadata storage areas. Read access is secured to a metadata storage area and a determination is made as to whether the second data structure indicates that there are any new host writes. In response to determining that the second data structure indicates that there are new host writes, write access is secured to the metadata storage area, the first data structure is updated with contents of the second data structure to determine which additional storage locations need to be copied for formation of a next consistency group, and the second data structure is updated to indicate that that the second data structure is in an initialized state.

In further embodiments, the first data structure is an out of synchronization bitmap, and the second data structure is a change recording bitmap.

In additional embodiments, the metadata storage area is a metadata track, and the write access to the metadata track is necessary if any bit the change recording bitmap is not zero.

In additional embodiments, the updating of the out of synchronization bitmap comprises performing an OR operation of corresponding bits of the change recording bitmap and the out of synchronization bitmap.

In yet further embodiments, a remote copy operation of primary volumes of a primary computational device to secondary volumes of a remote computational device is performed. The point in time copy operation is performed to generate point in time copies of the secondary volumes of the remote computational device to tertiary volumes of the remote computational device.

DETAILED DESCRIPTION

Exemplary Embodiments

FIG. 1illustrates a block diagram of a computing environment100that includes a primary computational device102and a remote computational device104, in accordance with certain embodiments.

The primary computational device102and the remote computational device104may comprise any suitable computation device, such as, a server, a personal computer, a storage controller, a controller, a blade computer, a telephony device, a mainframe computer, etc. The primary computational device102and the remote computational device104may be coupled over a network, such as the Internet, an intranet, a storage area network, etc. The primary computational device102and the remote computational device104may be coupled to one or more storage devices via an interface.

The primary computational device102and the remote computational device104may receive commands from one or more host computational devices that desire access to data controlled by the primary computational device102and the remote computational device104.

The primary computational device102may include a remote mirroring application106, bitmap metadata tracks108, and non volatile residual information metadata tracks110(also referred to as metadata tracks). In certain embodiments, under the control of the remote mirroring application106, asynchronous remote copy operations110are performed to asynchronously copy exemplary volume A112(i.e., a storage volume) maintained by the primary computational device102to the exemplary volume B114maintained by the remote computational device104. The exemplary volume. A112may also be referred to as an asynchronous remote copy primary volume, and a plurality of such volumes may be controlled via the primary computational device102. The exemplary volume B114may also be referred to as an asynchronous remote copy secondary volume, and a plurality of such volumes may be controlled via the remote computational device104.

In certain embodiments, point in time copy operations116are performed to create point in time copies (i.e., a consistent copy at an instant of time) of volume B114to volume C118. Volume B114which is an asynchronous remote copy secondary volume may also be a point in time copy source volume, and volume C118may comprise a point in time copy target volume. A plurality of volumes similar to volume B114or volume C118may be controlled by the remote computational device104.

When a volume is created, bitmap metadata tracks108are allocated for copy services use for the remote mirroring application106. The number of bitmap metadata tracks allocated depends on the number of customer tracks on the volume. In certain embodiments, each bitmap metadata track includes a plurality of data fields that may be indexed sequentially. The same data field on every bitmap metadata track may represent the same bitmap. In other words, that one bitmap is scattered across all bitmap metadata tracks occupying the same data field on each track. When a bitmap (such as the change recording bitmap120or the out of synchronization bitmap122) is allocated, a unique field/bitmap index that corresponds to the bitmap is assigned. The information about the bitmap index that is currently used by each process is kept in the non-volatile residual information metadata track110. The information may be maintained via a change recording bitmap pointer124and an out of synchronization bitmap pointer126.

Therefore,FIG. 1illustrates ac environment100in which asynchronous remote copy and point in time copy operations are performed by using change recording bitmaps120and out of synchronization bitmaps122to generate consistency groups.

FIG. 2illustrates a block diagram200that shows failure in consistency group formation, in accordance with certain embodiments. The operations shown in block diagram200may be performed by the computational devices102and104in the computing environment100.

In certain embodiments, attempts for consistency group formation in the computing environment100that performs remote mirroring is comprised of the following operations as shown in block200:

2. Drain out of synchronization bitmap (block204);

3. Perform logical point in time copy (block206);

4. Perform physical point in time copy (Hock208); and

When all operations202,204,206,208,210have completed successfully for all the volumes in a session, this is considered a successful consistency group formation. If there is a failure on remote copy or point in time copy operations or network error during the operations202,204,206,208or other operations, there may be failure in consistency group formation (as shown via reference numeral212)

In case of a failure in consistency group formation, both the change recording bitmap and the out of synchronization bitmap have vital data and are merged for all volumes in the session, to generate an updated out of synchronization bitmap for the next consistency group. This merge operation may be time consuming since in some cases it has to modify almost all bitmap metadata tracks allocated for a volume. Additionally, new consistency group formation cannot be started until the merge operation has completed. Furthermore, this may generate many modified bitmap metadata tracks in cache that may create more contention.

In certain embodiments, a failover may comprise an event in which a system in an exemplary cluster may automatically switch over to one or more backup nodes in the event of a failure of a node. In certain embodiments, a central electronics complex (CEC) failover may occur, in which if one CEC fails, then the system performs a failover to another CEC. Having a lot of modified data in cache affects failover time since every lost metadata track may have to be recovered by setting all the corresponding bits in the out of synchronization bitmap. These issues may become worse with increased volume size.

In the start increment phase (block202), change recording is started for all volumes in the session. New host writes are recorded in the change recording bitmap120. All tracks marked for asynchronous remote copy in the out of synchronization bitmap122are transferred (at block204) to the remote site104. Once the out of synchronization bitmap122has been drained (i.e., all bits that were set to 1 to indicate that the corresponding track has to be consistently copied to the remote computational device104have been set to 0), the logical and physical point in time copies can be made (at blocks206,208).

If the point in time copies are successfully generated, then this may result in the formation of a successful consistency group. If at any time one of the steps is not completed successfully, the consistency group is considered failed and the point in time copy relationship may have to be reverted. Certain embodiments provided in this disclosure address the situation when consistency group formation fails.

In case of a failure in the formation of consistency group, certain operations are performed during increment complete (at block210). In certain embodiments the change recording is stopped, and the out of synchronization bitmap is updated by performing OR operations of the corresponding bits of the change recording bitmap into the out of synchronization bitmap. The bits of the change recording bitmap are changed to zeros via a background task. In such a situation, control returns to block202once again and new writes are recorded in the change recording bitmap and the updated out of synchronization bitmap is used for draining.

To speed up the process of merging the out of synchronization bitmap and change recording bitmap, in certain embodiments prior to acquiring “write” access to bitmap metadata tracks, “read” access is acquired (the merging of change recording bitmap into the out of synchronization bitmap may also be referred to as a merger of bitmaps). If on read access the bits of the change recording bitmap are all zero, then no merge is needed. Therefore, in certain embodiments, no “actual” merge is required on bitmap metadata tracks with “clean” change recording bitmap fields which means that there are just a few number of modified bitmap metadata tracks. In some embodiments where host “writes” are not spread across the volume, the embodiments may speed up consistency group formation which may result in a lower recovery point objective.

FIG. 3illustrates a block diagram300that shows states of certain bitmaps, in accordance with certain embodiments. An exemplary out of synchronization bitmap302is shown in an initial state in which all bits are set to 0, i.e., no tracks remain to be consistently copied from the primary computational device102to the remote computational device104.

Block304shows the state of an exemplary out of synchronization bitmap306and an exemplary change recording bitmap308at the beginning of the start increment phase202ofFIG. 2. In such a state, the exemplary change recording bitmap308is in an initialized state and has all bits set to 0 and the exemplary out of synchronization bitmap has at least some bits set to 1 to indicate tracks that have yet to be asynchronously copied to the remote computational device104from the primary computational device102.

FIG. 4illustrates a block diagram400that shows states of certain bitmaps at the start of the increment complete phase, in accordance with certain embodiments. The state of bitmaps shown inFIG. 4may be caused by unsuccessful conclusion of the point in time copy operations206,208to generate consistency groups. Unsuccessful consistency group formation may also be caused because of a failure in remote copy, a network error, or for other reasons.

In the state shown inFIG. 4, the bits of the exemplary out of synchronization bitmap402may or may not be all set to all 0 depending on whether all tracks have copied over to the remote computational device104, i.e., the out of synchronization bitmap402has been drained. As a track is transferred, the corresponding bit in the out of synchronization bitmap402is changed from 1 to 0. In the particular example shown inFIG. 4, not all tracks have been copied over to remote computational device104when there is an unsuccessful formation of consistency group and hence certain bits of the exemplary out of synchronization bitmap402are shown to be one.

InFIG. 4, the exemplary change recording bitmap404has at least some bits set to 1, to indicate tracks modified by host writes during the point in time copy operations (blocks206,208ofFIG. 2show the point in time copy operations).

FIG. 4also shows an exemplary bitmap metadata track406, in which at location408the out of synchronization bitmap402is stored. Furthermore, at location410the Change recording bitmap404is stored.

FIG. 4also shows that the out of synchronization bitmap pointer412points at location408of the exemplary bitmap metadata track406and the change recording bitmap pointer414points to location410of the bitmap metadata track406.

FIG. 5illustrates a block diagram500that shows how the two bitmaps402and404(shown inFIG. 4) are merged in response to a failure in consistency group formation, in accordance with certain embodiments.

Corresponding bits of the out of synchronization bitmap402and the change recording bitmaps undergo “OR” operations to generate the exemplary out of synchronization bitmap504. The exemplary change recording bitmap506has all the bits set to zero after the merger, i.e., the exemplary change recording bitmap506is in an initialized state. The two bitmaps504,506generated after the merger are shown in block502.

Therefore, in certain embodiments, when a remote mirroring session is unsuccessful in forming a consistency group, the out of synchronization bitmap is updated by merging with the change recording bitmap. The merging of change recording bitmap into the out of synchronization bitmap is performed by OR operations on corresponding bits.

A new round of consistency group formation may start with the out of synchronization bitmap504and the change recording bitmap506shown in block502.

FIG. 6illustrates a first flowchart600that shows operations performed in the computing environment100, in accordance with certain embodiments. The operations shown inFIG. 6may be performed via the primary computational device102in the increment complete phase210. Certain embodiments provide improved time and space efficient mechanisms with operations that are different from those shown inFIG. 6.

InFIG. 6, for each bitmap metadata track, write access is obtained to the metadata track, and operations are performed to OR the bits in the change recording bitmap into the out of synchronization bitmap (at block602). A background process clears the bits of the change recording bitmap to make the bits all zero. Such operations require write access of the metadata track, which may be time and resource inefficient.

FIG. 7illustrates a second flowchart700that shows operations performed in the computing environment100, in accordance with certain embodiments. The operations shown inFIG. 7may be performed via the primary computational device102. The operations shown inFIG. 7provide improvements in processing time and lesser usage of cache storage space in comparison to operations shown inFIG. 6.

Control starts at block702where the start increment complete process for a failed consistency group starts and metadata track processing starts. A determination is made at block704as to whether any bitmap metadata track is left to be processed, and if so, then the metadata track is selected (at block708). Read access is secured (at block710) to the metadata track and a determination is made (at block712) as to whether all bits in the change recording bitmap are zero.

If a determination is made that all bits of the change recording bitmap are zero (at block712) then significant time is saved over the operations shown inFIG. 6, as no merger needs to be performed for the track, and read access to the metadata track is released (at block720). In such embodiments, the state of the bitmap metadata track (irrespective of whether or not the bitmap metadata track has been modified) remains unchanged.

If at block712, a determination is made that not all bits of the change recording bitmap are zero, then control proceeds to block714, and the access of the metadata track is changed to “write” access. Thus, unlike inFIG. 6where “write” access was acquired indiscriminately for metadata tracks, inFIG. 7“write” access which is more time and resource inefficient is acquired only when “read” access is inadequate and the change recording bitmap is not all zero.

Control proceeds to block716where an operation is performed to OR the bits in the change recording bitmap to the our of synchronization bitmap, and then all bits of the change recording bitmap are set (at block718) to zero. Write access to the metadata track is released (at block720) and control returns to block704.

If at block704, a determination is made that no metadata track is left to be processed then all bits of the change recording bitmap are already zero, and at block706, an indicator indicating that the change recording bitmap is all zeros is set. Unlike in operations shown inFIG. 6, no background task to change the bits of the change recording bitmap to zero is needed.

FIG. 8illustrates a third flowchart800that shows operations performed in the computing environment100, in accordance with certain embodiments. The operations shown inFIG. 8may be performed via the primary computational device102. The operations shown inFIG. 8provide improvements in processing time and lesser usage of storage space in comparison to operations shown inFIG. 6.

Control starts at block802in which a first data structure (e.g., an out of synchronization bitmap) stores indications of storage locations (e.g., tracks) that need to be copied for forming a consistency group. A second data structure (e.g. a change recording bitmap) stores indications of new host writes subsequent to starting a point in time copy operation to form the consistency group, where the first data structure and the second data structure are maintained in one or more metadata storage areas.

Control proceeds to block804, in which read access is secured to a metadata storage area (e.g., a metadata track110) and a determination is made as to whether the second data structure indicates that there are any new host writes.

In response to determining (at block804) that the second data structure indicates that there are new host writes, write access is secured to the metadata storage area, the first data structure is updated with contents of the second data structure to determine which additional storage locations need to be copied for formation of a next consistency group, and the second data structure is updated to indicate that the second data structure is in an initialized state. While in the initialized state, the second data structure indicates that no host writes have been recorded in the second data structure.

Therefore,FIGS. 1-8illustrate certain embodiments in which in case of a failure of consistency group formation, the merging of the out of synchronization bitmap and the change recording bitmap are performed by first performing a read access to the metadata track, and only if necessary write access to the metadata track is acquired. Improved space utilization and less processing time are achieved in such embodiments (described at least inFIGS. 7 and 8) in comparison to mechanisms that indiscriminately acquire write access to metadata tracks (as shown inFIG. 6).

Additional Embodiment Details

The described operations may be implemented as a method, apparatus or computer program product using standard programming and/or engineering techniques to produce software, firmware, hardware, or any combination thereof. Accordingly, aspects of the embodiments may take the form of an entirely hardware embodiment, an entirely software embodiment (including firmware, resident software, micro-code, etc.) or an embodiment combining software and hardware aspects that may all generally be referred to herein as a “circuit,” “module” or “system.” Furthermore, aspects of the embodiments may take the form of a computer program product embodied in one or more computer readable medium(s) having computer readable program code embodied there.

FIG. 9illustrates a block diagram that shows certain elements that may be included in the computational devices102,104in accordance with certain embodiments. The system900may comprise the computational devices102,104and may include a circuitry902that may in certain embodiments include at least a processor904. The system900may also include a memory906(e.g., a volatile memory device), and storage908. The storage908may include a non-volatile memory device (e.g., EEPROM, ROM, PROM, RAM, DRAM, SRAM, flash, firmware, programmable logic, etc.), magnetic disk drive, optical disk drive, tape drive, etc. The storage908may comprise an internal storage device, an attached storage device and/or a network accessible storage device. The system900may include a program logic910including code912that may be loaded into the memory906and executed by the processor904or circuitry902. In certain embodiments, the program logic910including code912may be stored in the storage908. In certain other embodiments, the program logic910may be implemented in the circuitry902. Therefore, whileFIG. 9shows the program logic910separately from the other elements, the program logic910may be implemented in the memory906and/or the circuitry902.