Replication of volumes on demands using absent allocation

A method, non-transitory computer readable medium and programmed apparatus that receives a request to replicate a volume from a source to a destination. The volume includes data and metadata including information descriptive of the data. The method includes determining a first set of blocks and a second set of blocks associated with the source, where the first set of blocks is associated with the metadata, and where the second set of blocks is associated with the data. The method includes initiating, based on the first set of blocks, replication of the volume from the source to the destination to generate a replicated volume at the destination. The replicated volume includes replicated metadata generated based on the replicated first set of blocks and includes absent allocated data corresponding to the data included in the volume storage at the source storage system.

FIELD

This technology is generally related to replication of and relocation of data volumes using absent allocation.

BACKGROUND

Use of storage systems (e.g., storage area networks (SANs), network appliances, etc.) to store data have become widespread, and may store large amounts of data (e.g., fifty (50) terabytes or more) using a plurality storage devices (e.g., hard disk drives, solid state storage devices, optical disk drives, etc.). Data stored at such storage systems may include a data (e.g., user files, applications, application data, etc.) and metadata that includes information descriptive of the data. The data may be organized into containers (e.g., directories, volumes, virtual volumes, partitions, etc.) by a file system.

Additionally, the storage systems may span across geographically disparate areas. For example, a SAN may include a first storage system located at a first location and a second set of storage system located at a second location that is remote to the first location. Data stored at the SAN may be distributed among the first storage system and the second storage system. The SAN may be coupled to client devices (e.g., personal computing devices, laptop computing devices, tablets, mobile communication devices, servers, etc.) located at the first and second locations via a network (e.g., a local area network (LAN), a wide area network (WAN), etc.) and may provide the client devices with access to the data stored at the first storage system and the second storage system.

As the amount of data stored in the data storage systems increases, management of the data becomes more difficult. For example, a portion of the data may be replicated (e.g., as part of a clone operation, a data relocation operation, a backup operation, etc.) from the first storage system to the second storage system. During replication of the data, access to the first storage system by client devices may be locked to prevent modification of the portion of the data until the replication is complete. Alternatively, an image (e.g., a snapshot) of the data stored at the first storage system may be captured and the portion of the data may be replicated using the image (i.e., so that client devices may continue to access the portion of the data).

Regardless of whether the portion of the data is replicated using the image or by locking access to the portion of the data, the portion of the data is replicated by performing a block-by-block copy of the data from a source storage system (e.g., the first storage system) to a destination storage system (e.g., the second storage system). This may take several minutes or hours which may introduce undesirable delays.

Additionally, when the portion of the data is replicated based on the snapshot, the client devices may modify the portion of the data prior to completing the replication. Thus, the replicated portion of the data may be inaccurate or out of date upon completion of the replication, requiring additional replication using block-by-block copying, introducing further delays.

SUMMARY

Disclosed herein are methods, systems, apparatuses, and non-transitory computer-readable media that enable a data volume to be replicated (e.g., moved, cloned, backed up, etc.) virtually on demand. A volume may include data stored at one or more storage devices of a storage system (e.g., a source storage apparatus). A file system associated with the storage system may manage the data and may generate metadata that includes information descriptive of the data included in the volume. In an aspect, the volume may be replicated from the source storage system to a destination storage system to create a replicated volume. The replicated volume may be created using absent allocation. Replicating the volume using absent allocation may include replicating the metadata associated with the volume. For example, the metadata associated with the volume may be replicated from the source storage system to the destination storage system. The data included in the volume may be replicated to the destination storage system using absent allocation, such that the replicated volume created at the destination storage system includes replicated metadata (e.g., a copy of the metadata associated with the source volume) and absent allocated data (e.g., information indicating that the data included in the volume has not been replicated to the destination storage system). After the replicated volume is created, a client device may access the data via the destination volume based on the absent allocated data.

To illustrate, absent allocating the data at the destination storage system may include associating absent allocation states (AA states) with the replicated volume (or the replicated metadata). The replicated metadata may include a buffer tree and a container map (also known as a container file). The buffer tree and the container map may include information that indicates whether the data is present at the destination storage system. In an aspect, the information may include suggestive indicators (e.g., indicators that identify data in a first AA state) that provides a non-determinative indication that data is absent from the replicated volume. In an additional aspect, the information may include determinative indicators (e.g., indicators that identify data a second AA state) that provides a determinative indication that the data is absent from the replicated volume. In an aspect, the suggestive indicators may be stored in the buffer tree, and the determinative indicators may be stored in the container map.

Subsequently, the data may be replicated from the source storage system to the destination storage system using various techniques. For example, the data may be replicated as part of a background replication process (e.g., as system resources permit), or may be replicated in response to a request received at the destination storage system (e.g., a request to access particular data). As the data is replicated from the source storage system to the destination storage system, the AA state indictors (e.g., the suggestive indicators and the determinative indicators) may be updated.

By replicating metadata from a source storage system to a destination storage system and absent allocating data from the source storage system to the destination storage system, the replicated volume at the destination storage system may provide access to the data in less time than traditional replication techniques. Additionally, a volume replicated according to one or more aspects of this technology may provide increased volume accessibility to client devices because the data is accessible via the destination storage system prior to replicating the data to the destination storage system. This may reduce an amount of delay introduced by replication of the volume as compared to other replication techniques and may reduce an amount of storage system resources required to perform the replication.

This technology provides a number of advantages including providing methods, systems, apparatuses, and non-transitory computer-readable media that effectively enable replication of volumes on demand using absent allocation. This technology may reduce an amount of time required to replicate a volume (e.g., an amount of time between receiving the volume replication request112and completing the cutover process). Additionally, by using the suggestive indicators and determinative indicators to indicate whether absent allocated data is present or absent from a replicated volume, this technology is able to reduce an amount of time required to retrieve the absent allocated data. Further, one or more aspects of this technology may enable access to data that is absent from a replicated volume

DETAILED DESCRIPTION

Referring toFIG. 1, an illustration of a storage system for replicating volumes using absent allocation according to an aspect of this technology is shown and designated100. As shown inFIG. 1, the environment100includes a first storage apparatus102and a second storage apparatus104. In an aspect, the environment100may be a storage area network (SAN) that provides a persistent common view of data stored at the first storage apparatus102and the second storage apparatus104. The first storage apparatus102and the second storage apparatus104may be communicatively coupled via a network110. In an aspect the network110may be a local area network (LAN), a wide area network (WAN), a wireless local area network (WLAN), an internet, a combination of these networks, or other types of networks. This technology provides a number of advantages including providing methods, systems, apparatuses, and non-transitory computer-readable media that effectively enable replication of volumes on demand using absent allocation.

The environment100may provide client devices106,108and114with access to data stored at the first storage apparatus102and the second storage apparatus104. For example, as shown inFIG. 1, a first client device106may be communicatively coupled to the first storage apparatus102and a second client device108may be coupled to the second storage apparatus104. In another aspect, a third client device114may be communicatively coupled to the environment100via the network110. The client devices106,108, and114may access (e.g., locally or via the network110) the data stored at the first storage apparatus102and the second storage apparatus104. The client devices106,108, and114may modify, delete, create, move, and/or copy data stored at the first storage apparatus102and the second storage apparatus104.

As shown inFIG. 1, the first storage apparatus102includes a controller120and a data storage device130. The controller120may include a processor122, a memory124, and a block replication engine (BRE)128, although the controller may comprise other types and/or numbers of other system, devices, components and/or other elements in other configurations. Additionally, the first storage apparatus102may include other system, devices, components and/or other elements in other configurations which have not been illustrated inFIG. 1in order to simplifyFIG. 1. For example, for ease of illustration a network interface to couple the first storage apparatus102to the network110and/or the client device106is not illustrated since these are well known to be used by one of ordinary skill in the art. As another example, the first storage apparatus102may include a module or other programmed instructions that process data received from or provided to the network110according to one or more communication protocols (e.g., a TCP/IP protocol and/or other communication protocols), and may include another module that processes data to be stored at or retrieved from the data storage device130in accordance with one or more data storage protocols (e.g., a SAN protocol, a small computer system interface (SCSI) protocol, a fiber channel protocol (FPC), etc.). The memory124may store instructions126that, when executed by the processor122, cause the processor122to perform operations in accordance with one or more aspects of this technology, as illustrated and described in the examples herein with reference toFIGS. 1-3. It is noted that although the BRE engine128is illustrated inFIG. 1as being separate from the processor122, in another aspect, the BRE engine128may be integrated with the processor122(e.g., in hardware) or may be implemented by the processor122during execution of the instructions126.

The data storage device130may include one or more storage devices (e.g., hard disk drives (HDDs), solid state drives (SSDs), FLASH memory devices, optical disk drives, and/or other computer-readable storage devices) configured to store data. By way of example only, inFIG. 1, the data storage device130includes a volume140and a container map146, although the data storage device may have other system, devices, components and/or other elements in other configurations. In this example, the volume140may include data142and metadata144, although the volume may have other contents. The data142may include user data (e.g., files), applications, application data, etc. The metadata144may include information descriptive of the data142. Additional aspects of this example of the technology with respect to the metadata144and the container map146are described with reference toFIG. 2. Although not shown inFIG. 1, the second storage apparatus104may also include a controller and other components as described and illustrated with respect to the first storage apparatus102. The second storage apparatus104also includes a data storage device150. The data storage device150may include one or more storage devices (e.g., HDDs, SSDs, FLASH memory devices, optical disk drives, and/or other computer-readable storage devices) configured to store data.

The data storage devices130,150may each comprise a plurality of data blocks that may be used to provide various logical and/or physical storage containers, such as the volume140. The plurality of data blocks may also be used to provide other containers (e.g., files, volumes, aggregates, virtual disks, etc.). The logical and physical storage containers may be defined using an array of blocks indexed or mapped either logically or physically by a file system using various types of block numbers. For example, a file may be indexed by file block numbers (FBNs), a container file (e.g., a volume) by virtual block numbers (VBNs), an aggregate by physical block numbers (PBNs), and disks by disk block numbers (DBNs). To translate an FBN to a disk block, a filesystem (e.g., a WAFL filesystem) may use several steps, such as translating the FBN to a VBN, translating the VBN to a PBN, and then to translating the PBN to a DBN. Storage containers having various attributes may be defined and utilized using such logical and physical mapping techniques.

Data structures may be used to map the various types of block numbers to locations on disks of a storage device. For example, an aggregate may be a physical volume (PVOL) that includes one or more storage devices (e.g., the storage device130). A container, such as the volume140, may be a virtual volume (VVOL) that is created within the PVOL of the aggregate. The one or more storage devices included in aggregate may include a plurality of disk blocks indexed by DBNs. For example, the plurality of disk blocks may include disk blocks 0-N that may be indexed using DBNs DBN-0 to DBN-N.

When the container (e.g., the volume140) is created on the aggregate, a subset of the plurality of disk blocks (e.g., DBN-0 to DBN-M, where M<N) may be allocated to the container. As explained above, the container may be indexed by VBNs that map to PBNs, and the PBNs may be mapped to DBNs (e.g., the DBNs DBN-0 to DBN-M). Thus, when the container is created, data structures to provide mapping of the VBNs and the PBNs of the VVOL to the DBNs of the PVOL may be created.

As an example, a buffer tree may be created and stored within the container (e.g., as metadata) and may enable mapping of VBNs and PBNs used to index the container to the DBNs of the aggregate. The buffer tree may include information (e.g., pointers) that references locations (e.g., DBNs) on the aggregate where data (e.g., the data142) included in the VVOL are stored. The pointers may reference indirect blocks (e.g., “L1” blocks) that, in turn, reference the DBNs (e.g., “L0” blocks). In an aspect, the metadata144may include a buffer tree associated with the volume140. An illustrative embodiment of a buffer tree according to one or more aspects of this technology is described with reference toFIG. 2. As another example, a container map, such as the container map146ofFIG. 1, may be created and may also enable mapping of the VBNs and PBNs of the container to the DBNs of the aggregate. In an aspect, the container map may be stored external to the container. An illustrative embodiment of a container map according to one or more aspects of this technology is illustrated and described herein with reference toFIG. 2.

Referring toFIG. 2, an illustration of data structures that may be used to facilitate replication of volumes using absent allocation according to aspects of this technology is shown and designated200. InFIG. 2, the container map146ofFIG. 1and the replicated container map166ofFIG. 1are shown. Additionally, a buffer tree220and a buffer tree250are shown. In an aspect, the buffer tree220may be included in the metadata144ofFIG. 1and the buffer tree250may be included in the replicated metadata164ofFIG. 1.

As described with reference toFIG. 1, the container maps146,166and the buffer trees220,250may be used to map virtual block numbers (VBNs) to physical block numbers (PBNs), and the PBNs may be mapped to disk block numbers (DBNs) corresponding to disk blocks of an aggregate that have been allocated to a container (e.g., the volume140ofFIG. 1or the replicated volume160ofFIG. 1) and that store data (e.g., the data142ofFIG. 1). As the container map146and the buffer tree220are replicated (indicated by the arrows210and220, respectively) from the first storage apparatus102to the second storage apparatus104to create the replicated volume160ofFIG. 1, absent allocation states (AA states) associated with the data (e.g., the data162ofFIG. 1) may be applied to the information included in the container map166and the buffer tree250to indicate that the data is absent from the replicated volume160ofFIG. 1at the destination (e.g., the second storage apparatus104).

The AA states may include suggestive absent allocated information (e.g., suggestive indicators) and determinative absent allocated information (e.g., determinative indicators), as described with reference toFIG. 1. The suggestive absent allocated information may be included within the buffer tree250which may be stored within the replicated volume160(e.g., as part of the replicated metadata164), and the determinative absent allocated information may be included in the replicated container map166, which may be stored external to the replicated volume160. The suggestive absent allocated information may provide a non-authoritative or non-determinative indication (i.e., a rebuttable suggestion) that the data of a particular data block may not be present at the destination. The determinative absent allocated information may provide an authoritative or determinative indication (e.g., a final determination) that the data of a particular data block is not present on the destination.

As shown inFIG. 2, the container map146may include a plurality of entries212-216that identify a plurality of VBNs (e.g., VBN-1 to VBN-M) that have been allocated to the volume140. The plurality of VBNs of the container map146may include a first entry212having a first VBN index VBN-1 that may be associated with (or mapped to) a first PBN having an index of PBN-1. The plurality of VBNs may include a second entry214having a second VBN index VBN-2 that may be associated with (or mapped to) a second PBN index PBN-2. The plurality of VBNs may include an Mthentry216having an MthVBN index VBN-M that may be associated with (or mapped to) a MthPBN index PBN-M. The plurality of entries of the container map146may be used to map VBNs and PBNs (associated with requests for access to the data142ofFIG. 1) to DBNs (e.g., to disk blocks where the data142resides), as described with reference toFIG. 1. Information stored in the container map146may also be stored in the buffer tee220.

For example, as shown inFIG. 2, the buffer tree220includes a plurality of “L1” blocks222to224. The “L1” block222may include the first VBN index VBN-1 that may be associated with (or mapped to) the first PBN index PBN-1, and the Mth“L1” block224may include the MthVBN index VBN-M that may be associated with (or mapped to) the MthPBN index PBN-M. The plurality of entries of the buffer tree220may also be used to map the VBNs and the PBNs to DBNs (e.g., disk blocks232,234,236), as described with reference toFIG. 1.

During replication of the volume140, the processor122or the BRE engine128in the controller120ofFIG. 1may apply absent allocation indicators (e.g., by applying the reserved PBNs to the absent allocated data blocks) to the metadata144to generate the replicated metadata164. For example, the processor122or the BRE engine128in the controller120may scan the first snapshot and, for each “L1” block (e.g., the “L1” blocks222-224), may insert reserved PBNs into each of the entries of the buffer tree220. The reserved PBNs inserted in to the replicated buffer tree250may correspond to suggestive indicators and may be invalid PBNs. Additionally, the processor122or the BRE engine128in the controller120may replicate the container map146by applying the same or a different reserved PBNs to the plurality of entries of the container map146to generate the replicated container map166. The reserved PBNs inserted in to the replicated container map166may correspond to determinative indicators and may be invalid PBNs.

To illustrate, as shown inFIG. 2, the replicated container map166may include a plurality of entries242-246that identify a plurality of VBNs (e.g., VBN-1′ to VBN-M′) that have been allocated to the replicated volume160. The plurality of VBNs of the container map166may include a first entry242having a first VBN index VBN-1′ that may be associated with (or mapped to) a first PBN having an index of PBN-1′. The plurality of VBNs may include a second entry244having a second VBN index VBN-2′ that may be associated with (or mapped to) a second PBN index PBN-2′. The plurality of VBNs may include an Mthentry246having an MthVBN index VBN-M′ that may be associated with (or mapped to) a MthPBN index PBN-M′. In an aspect, the PBN indices PBN-1′ to PBN-M′ may correspond to a set of one or more reserved PBNs used to provide a determinative indication that data is absent (e.g., an indication as to whether a particular data block is in an absent AA state or a present AA state).

As shown inFIG. 2, the buffer tree250includes a plurality of “L1” blocks252to254. The “L1” block252may include the first VBN index VBN-1′ that may be associated with (or mapped to) a PBN index PBN-1″, and the Mth“L1” block254may include the MthVBN index VBN-M′ that may be associated with (or mapped to) the MthPBN index PBN-M″. The plurality of entries of the buffer tree220may also be used to map the VBNs and the PBNs to DBNs (e.g., disk blocks262,264,266), as described with reference toFIG. 1. In an aspect, the PBN indices PBN-1″ to PBN-M″ may correspond to a set of one or more reserved PBNs used to provide a suggestive indication that data (e.g., the data262,264,266) is absent allocated (e.g., an indication as to whether a particular data block is in an absent allocated AA state).

In an aspect, the suggestive indicators and the determinative indicators may be used to determine whether the requested data is present in the replicated volume160, and, when a determination is made that requested data is absent, may also be used to retrieve (or fetch and fill) the requested data from a source (e.g., the volume140ofFIG. 1). To illustrate, when a client device (e.g., one of the client devices106,108, and114ofFIG. 1) attempts to access data associated with the absent allocated data block266included in the replicated volume160, a controller (not shown inFIG. 1) of the second storage apparatus104may access the buffer tree250to determine a suggestive indication as to the AA state of the absent allocated data block266. The determination may be made, for example, based on the PBN index PBN-M″ of the “L1” block254of the replicated buffer tree250, which may suggestively indicate that the AA state for the absent allocated data block266is absent allocated. This may suggest to the controller that the data for the absent allocated data block266is not present in the replicated volume160.

In response to a determination that the AA state for the absent allocated block266is absent allocated, the controller may access the replicated container map166to determine a determinative indication as to the AA state of the absent allocated data block266. The determination may be made, for example, based on the PBN index PBN-M′ of the entry246of the replicated container map166, which may determinatively indicate that the AA state for the absent allocated data block266is absent. In response to authoritatively determining, based on the determinative indicator of the entry246of the replicated container map166, that the AA state for the absent allocated data block266is absent, the controller may initiate a fetch and fill operation.

The fetch and fill operation may include requesting the data corresponding to the absent allocated data block266from a source (e.g., the first storage apparatus102). The fetch request may be transmitted from the second storage apparatus104to the first storage apparatus102via the network110. The first storage apparatus102may retrieve the data corresponding to the absent allocated data block266and may provide the data to the second storage apparatus104via the network110. Upon receiving the data from the first storage apparatus102, the second storage apparatus104may fill (e.g., write) the data to the replicated volume160. The controller of the second storage apparatus104may then update the suggestive indicator and the determinative indicator to indicate that the data associated with the absent allocated data block266is now present (e.g., the AA state of the absent allocated data block266is present) at the replicated volume160.

As another example, assume that the client device requests to access the absent allocated data blocks262,264of the volume160. Further, assume that the data corresponding to the absent allocated data blocks262,264have been filled at the replicated volume160(e.g., using a background process or another technique). When the data was filled, the determinative indicator (e.g., the PBN index PBN-1′ of the entry242of the replicated container map166) associated with the absent allocated data blocks262,264may have been updated (e.g., changed from a reserved PBN index to a valid PBN index) to indicate that the data associated with the absent allocated data blocks262,264is present (e.g., a present AA state) in the replicated volume160, but the suggestive indicator (e.g., the PBN index PBN-1′ of the “L1” block252of the replicated buffer tree252) may not have been updated.

The controller may access the buffer tree250to determine a suggestive indication as to the AA state of the absent allocated data blocks262,264. The determination may be made, for example, based on the PBN index PBN-1″ of the “L1” block252of the replicated buffer tree250, which may suggestively indicate that the AA state for the absent allocated data blocks262,264is absent allocated. This may suggest to the controller that the data for the absent allocated data blocks262,264are not present in the replicated volume160.

In response to a determination that the AA state for the absent allocated blocks262,264is absent allocated, the controller may access the replicated container map166to determine a determinative indication as to the AA state of the absent allocated data blocks262,264. The determination may be made, for example, based on PBN index PBN-1′ of the entry242of the replicated container map166, which may determinatively indicate that the AA state for the absent allocated data blocks262,264is present. In response to authoritatively determining, based on the determinative indicator of the entry242of the replicated container map166, that the AA state for the absent allocated data blocks262,264is present, the controller may retrieve the data based on the PBN index PBN-1′.

Thus, replication of volumes according to one or more aspects of this technology may reduce an amount of time required to replicate a volume (e.g., an amount of time between receiving the volume replication request112and completing the cutover process). Additionally, by using the suggestive indicators and determinative indicators to indicate whether absent allocated data is present or absent from a replicated volume, an amount of time required to retrieve the absent allocated data may be reduce. Further, one or more aspects of this technology may enable access to data that is absent from a replicated volume.

Referring toFIGS. 1-3, an example of a method300for performing volume replication on demand using absent allocation according to an aspect of this technology will now be illustrated and described below.

In step310during operation of the environment100, the processor122of the controller120in the first storage apparatus102may by way of example only receive a volume replication request112from the first client device106. The volume replication request112may include a request to replicate (e.g., move, clone/copy, back up, etc.) the volume140from the first storage apparatus102to the second storage apparatus104.

In step320, in response to receiving the volume replication request112, the processor122of the controller120may determine a first set of blocks of the source (e.g., the first storage apparatus102) and a second set of blocks of the source (e.g., the first storage apparatus102). In an aspect, the first set of blocks may be associated with the metadata144, and the second set of blocks may be associated with the data142. In another aspect, the data142and the metadata144may be stored on data blocks of one or more storage devices at the data storage device130. Identifiers may be used to identify data blocks storing particular types of data. For example, data blocks storing the data142may be associated with a first identifier (e.g., an “L0” block) and data blocks storing the metadata144may be associated with a second identifier (e.g., an “L1” block). The identifiers may be stored in association with, or included in the metadata144. In an aspect, a file (e.g., a kireedi file) may be included in the volume140and the file may include the identifiers associated with each of the data blocks included in the volume. In an aspect, the controller120may determine the first and second sets of blocks using the BRE engine128. For example, the BRE engine128may scan a plurality of data blocks (or the kireeti file) associated with the volume140to identify a first set of data blocks associated with the second identifier (e.g., a set of “L1” blocks) and may determine a second set of blocks associated with the first identifier (e.g., a set of “L0” blocks). The first set of blocks may include information that may enable the controller to map VBNs and PBNs to DBNs responsive to requests to access portions of the data142, as described above.

At step322, the processor122of the controller120in this example may generate (or in another example instruct the BRE engine128to generate) a first snapshot of the volume140. The first snapshot of the volume140may be a read only copy or image of the volume140and may establish a baseline for the volume140. The baseline may be subsequently compared to additional snapshots to determine an incremental difference between the baseline and the subsequent snapshot, as described below. The processor122or the BRE engine128in the controller120may determine the first set of blocks based on the first snapshot. In an additional aspect, the replicated metadata164may be generated based on the first snapshot. For example, because the first snapshot is created as a read only file, the first snapshot may provide a stable image of the volume140that may be used to replicate the metadata144from the first storage apparatus102to the second storage apparatus104. The processor122or the BRE engine122may replicate the metadata142from the first storage apparatus102to the second storage apparatus104by reading information associated with the first set of blocks from the first snapshot and transmitting the information to the second storage apparatus104via the network110to create the replicated volume160. In an aspect, the processor122or the BRE engine128in the controller120may modify (or mutate) the information read from the first snapshot to indicate an absent allocated state (AA state) for each data block (e.g., the second set of blocks) corresponding to the data142. The AA states may be provided by the suggestive indicators and the determinative indicators, as described with reference toFIG. 2. By replicating the metadata144using the first snapshot, access to the data142included in the volume140by the client devices106,108, and114may continue without interruption during the replication process. However, such access by the client devices106,108, and114may modify the metadata144, such that when the replication of the metadata144based on the snapshot completes, the replicated metadata164varies from the metadata144.

In step330, the processor122of controller120may initiate, based on the first set of blocks, replication of the volume140from the first storage apparatus102(e.g., a source) to the second storage apparatus104(e.g., a destination) to generate a replicated volume160at the second storage apparatus104. The replicated volume160may include replicated metadata164generated based on the first set of blocks and absent allocated data162associated with the second set of blocks (e.g., the data142included in the volume140. The absent allocated data162may be absent allocated (e.g., not present) in the replicated volume160. In an aspect, replication of the volume140may also include replication of the container map146. For example, as shown inFIG. 1, the replicated volume160may be associated with a replicated container map166.

According to an aspect of this technology, absent allocation of the data162may be provided by modifying the replicated metadata164and the replicated container map164to provide a suggestive indicators and determinative indicators associated with the absent allocated data162. In an aspect, a particular suggestive indicator provides a suggestive indication that a particular portion of the data142included in the volume140is absent (or absent allocated) from the replicated volume160, and a particular determinative indicator provides a determinative indication that the particular portion of the data142included in the volume140is absent from the replicated volume160. The replicated volume160may provide access to the particular portion of the data142based at least in part on the particular suggestive indicator and the particular determinative indicator. In an aspect, the suggestive indicator may be included in the replicated metadata164(e.g., in the buffer tree) and the determinative indicator may be included in the replicated container map166. Illustrative aspects of providing access to the particular portion of the data142based at least in part on the particular suggestive indicator and the particular determinative indicator are described with reference toFIG. 2.

In step340, the processor122or the BRE engine128in the controller120may generate a second snapshot of the volume140. The second snapshot may be generated subsequent to the first snapshot and may reflect the changes to the metadata144caused by the access of the data142by one of the client devices106,108, and114by way of example only as the metadata144was replicated to the second storage apparatus104based on the first snapshot.

In step350, the processor122or the BRE engine128in the controller120may determine an incremental difference between the first set of blocks as indicated in the first snapshot and the first set of blocks as indicated in the second snapshot. The incremental difference may identify changes to the metadata144included in the volume140, changes to the data142included in the volume140, or both that occurred during a time period between generation of the first snapshot and generation of the second snapshot.

In step352, the processor122or the BRE engine128in the controller120may estimate an amount of time to update the replicated metadata164based on the incremental difference. The estimated amount of time of time may correspond to an amount of time required to replicate the incremental difference to the destination (e.g., incorporate the changes to the metadata144since the first snapshot was generated into the replicated metadata164).

In step354, the processor122in the controller120may determine whether the determined amount of time is below a threshold amount of time. The threshold amount of time may correspond to an amount of time that the volume may be locked (e.g., no access to the volume by one of the client devices106,108, or114) at the source without introducing an unacceptable amount of delay. In an aspect, the threshold amount of time may be between thirty (30) and forty five (45) seconds.

If in step354, the processor122in the controller120determines the amount of time is below the threshold amount of time, then the Yes branch is taken to step360. In step360, the processor122or the BRE engine128in the controller120may update the replicated metadata164at the second storage apparatus104based on the incremental difference.

If back in step354, the processor122in the controller120determines the amount of time is not below the threshold amount of time, then the No branch is taken to step370. In step370, the processor122in the controller120may lock access to the volume140until the updating of the replicated metadata is complete. Locking access to the volume140may enable synchronization of the metadata144(and the container map146) with the replicated metadata164(and the replicated container map166).

After updating of the replicated metadata is complete (e.g., after the metadata144and the replicated metadata164are synchronized and/or the container map146and the replicated container map166are synchronized), the processor122may initiate a cutover process. The cutover processor may cause access to the data142included in the volume140to be provided via the second storage apparatus104based on the absent allocation indicators included in the replicated metadata164and/or the replicated container map166.

In an additional aspect, after initiation of the cutover process, the data142, or a portion thereof, may be subsequently replicated from the first storage apparatus102to the second storage apparatus104using various techniques or processes. For example, the data142may be treated as a single virtual file that may replicated from the first storage apparatus102to the second storage apparatus104using a background process as resources of the environment100(e.g., resources of the first storage apparatus102, the second storage apparatus104, and the network110) are available.

As another example, the processor122in controller120may initiate replication of the data142using single-file batch processes in which each file (e.g., user data files, applications, application data file, etc.) included in the data142is replicated individually (and potentially simultaneously) from the first storage apparatus102to the second storage apparatus104.

As yet another example, the data142may be replicated from the first storage apparatus102to the second storage apparatus104using a fault-on-demand process where a portion of the data142(e.g., individual or select groups of files, applications, application data files, etc.) is replicated from the first storage apparatus102to the second storage apparatus104in response to a request, received at the second storage apparatus104, to access the portion of the data142. Irrespective of the particular technique or process used to replicate the data142the first storage apparatus102to the second storage apparatus104, one or more of the absent allocation indicators included in the replicated metadata164and the replicated container map166may be updated to indicate that the data142is present (i.e., physically stored) in the replicated volume160at the data storage device150.

In another aspect, the processor122or the BRE engine128in controller120may replicate the data142(or a portion thereof) based on snapshots of the volume140in a manner similar to replication of the metadata144, as described above. Additionally, the processor122and/or the BRE engine128in controller120may determine incremental differences between the data142(or the portion thereof) based on successive snapshots until replication of the data142(or the portion thereof) is complete.

In yet another aspect, replication of the data142may include identifying, by the processor122and/or by the BRE engine128in controller120, the second set of blocks (e.g., the “L0” blocks) associated with the data142(or the portion thereof) and replicating the second set of blocks from the first storage apparatus102to the second storage apparatus104. As the second set of blocks is replicated from the first storage apparatus102to the second storage apparatus104, the absent allocated data162may be updated or modified to indicate that the data142has been replicated to the replicated volume160and is physically present at the data storage device150.

Replication of volumes according to one or more aspects of this technology may reduce an amount of time required to replicate the volume140(e.g., an amount of time between receiving the volume replication request112and completing the cutover process). In an aspect, the reduced amount of time may be proportional to a size of the metadata relative to a size of the volume. For example, let P represent a total storage capacity of a volume represented as a percentage (e.g., P=one hundred percent (100%)). Now let D represent a percentage of the volume associated with the data142and let M represent a percentage of the volume associated with the metadata144, where P=D+M. A speed increase (T) may realized by replicating the volume140using one or more aspects of this technology and may be determined according to the following equation:
T=P/(P−D)  Equation 1
As shown in Equation 1, when ninety five percent (95%) of the volume140is used to store the data142(e.g., D=95) and five percent (5%) of the volume140is used to store the metadata144(e.g., M=5), replication of the volume140according to one or more aspects of this technology may complete up to twenty (20) times faster (e.g., T=(100/(100−95))=20). Stated another way, the amount of time between receiving the volume replication request112and initiating the cutover process using aspects of this technology may be reduced two (2) orders of magnitude when compared to other replication techniques. Thus, one or more aspects of this technology may improve throughput and performance of a storage apparatus, such as the environment100ofFIG. 1.

Additionally, replication of the volume140using one or more aspects of this technology may increase the modularity of the environment100, the first storage apparatus102, and the second storage apparatus104. To illustrate, the metadata144(and/or the container map146) may be used by the controller120(or other applications executing on the processor122) to reason about the layout of the file system (e.g., the volume140). By replicating the metadata144to the second storage apparatus104and absent allocating the data142at the second storage apparatus104, a controller (not shown inFIG. 1) of the second storage apparatus104may be capable of mapping the absent allocated data to the data142located at the first storage apparatus102in a manner that is transparent to the applications, to the client devices106,108, and114, to end users of the environment100, and to other processes of the environment100, the first storage apparatus102, and the second storage apparatus104.

It is noted that, although replication of the volume140has been described with reference toFIG. 1as including replicating the entire volume, aspects of this technology may be used to replicate portions of the volume140(e.g., the replicated volume160may include only a portion of the data142, a portion of the metadata144, and/or a portion of the container map146), to replicate multiple volumes (e.g., supporting virtual server migration), or replicate a combination of entire volumes and portions of volumes (e.g., to consolidate a first volume with a portion of a second volume). Additionally, although aspects of this technology are described as transferring all of the metadata, volumes may be replicated according to one or more aspects of this technology by replicating only a portion (or subset) of the metadata144(e.g., only replicating portions of the metadata144required for mapping block numbers, such as VBNs and PBNs, to DBs). Additionally, aspects of this technology may be used to clone the volume140at the second storage apparatus104, to back up the volume140at the second storage apparatus104, and to implement other data management techniques depending on a particular system design, configuration, implementation, and performance considerations.

In yet another aspect, disk blocks of the data storage device130allocated to the volume140may be released (e.g., made available for allocation to other containers or volumes) upon completing replication of the volume140(e.g., completing replication of the data142, the metadata144, and the container map146) from the first storage apparatus102to the second storage apparatus104. Thus, when replication of the volume140corresponds to a volume move operation (e.g., moving the volume from the first storage apparatus102to the second storage apparatus104), the resources (e.g., the disk blocks) allocated to the volume140at the first storage apparatus102may be reclaimed.

Additionally, although the first storage apparatus102and the second storage apparatus104are illustrated as being remotely connected via the network110, the first storage apparatus102and the second storage apparatus104may be located at a single location (e.g., a single building or a data center).

Further, although the environment100ofFIG. 1is illustrated as including two storage apparatuses102and104, aspects of this technology may be utilized in apparatuses or other systems having more than two or fewer than two storage systems. Further, although aspects ofFIG. 1have been described in terms of replicating the volume140from the first storage apparatus102to the second storage apparatus104, aspects of this technology may also be used for restoring volumes (e.g., from a backup copy of the volume) or replicating volumes within a single storage system. An amount of time to restore a volume from a backup may be reduced according to one or more aspects of the embodiments disclosed herein.

Accordingly, as described and illustrated by the examples herein, replication of volumes according to one or more aspects of the this technology may reduce an amount of time required to replicate a volume (e.g., an amount of time between receiving the volume replication request112and completing the cutover process). Additionally, by using the suggestive indicators and determinative indicators to indicate whether absent allocated data is present or absent from a replicated volume, an amount of time required to retrieve the absent allocated data may be reduce. Further, one or more aspects of this technology may enable access to data that is absent from a replicated volume.