Patent Publication Number: US-11042504-B2

Title: Managing overwrites when archiving data in cloud/object storage

Description:
CROSS-REFERENCES TO RELATED APPLICATIONS 
     The present application is related to commonly-owned U.S. patent application Ser. No. 16/035,439, entitled “Archiving Data in Cloud/Object Storage Using Local Metadata Staging,” which is filed concurrently herewith. The entire contents of this application are incorporated herein by reference for all purposes. 
     BACKGROUND 
     In computing, “object storage” is a data storage model that manages data in the form of containers referred to as objects, rather than in the form of files (as in file storage) or in the form of blocks (as in block storage). “Cloud/object storage” is an implementation of object storage that maintains these objects on servers that are accessible via the Internet. Examples of commercially-available cloud/object storage services include Amazon&#39;s Simple Storage Service (S3) and Google Cloud Storage. 
     Cloud/object storage generally offers high scalability, high durability, and relatively low cost per unit of storage capacity, which makes it an attractive solution for organizations seeking to archive large volumes of data for long-term backup and recovery purposes. However, there are a number of complexities that make it difficult to use existing cloud/object storage services as a backup target. For example, many existing cloud/object storage services can only guarantee eventual consistency to clients, which means that if an update is made to an object, all subsequent client accesses to that object will eventually, but not necessarily immediately, return the object&#39;s updated value. Some cloud/object storage services mitigate this by guaranteeing read-after-write consistency for newly created objects. But, without a stronger consistency model that also guarantees read-after-write consistency for modified objects, it is difficult to build a data backup/restore system that ensures clients have a consistent view of the archived data. 
     Further, the network bandwidth between an organization&#39;s on-premises (i.e., local) site and cloud/object storage is usually limited due to the need to traverse the Internet. Similarly, the latency from on-premises equipment to cloud/object storage is relatively high, and network timeouts or other network issues can be prevalent. These factors increase the costs of writing a large number of objects per backup task and can cause write throttling to occur. 
    
    
     
       BRIEF DESCRIPTION OF THE DRAWINGS 
         FIG. 1  depicts a system environment according to an embodiment. 
         FIG. 2A  depicts an initial snapshot upload workflow according to an embodiment. 
         FIG. 2B  depicts an example structure of a cloud archive after the initial snapshot workflow of  FIG. 2A  according to an embodiment. 
         FIG. 3  depicts a workflow for staging snapshot metadata using an arbitrary mapping approach according to an embodiment. 
         FIG. 4  depicts a delta snapshot upload workflow according to an embodiment. 
         FIGS. 5A and 5B  depict workflows for managing overwrites to the superblock chunk of a cloud archive according to an embodiment. 
         FIG. 5C  depicts an example structure of a cloud archive after the creation of one or more. ARCHIVE files for the superblock chunk according to an embodiment. 
     
    
    
     DETAILED DESCRIPTION 
     In the following description, for purposes of explanation, numerous examples and details are set forth in order to provide an understanding of various embodiments. It will be evident, however, to one skilled in the art that certain embodiments can be practiced without some of these details, or can be practiced with modifications or equivalents thereof 
     1. Overview 
     Embodiments of the present disclosure describe techniques that can be performed by a client system running at an organization&#39;s on-premises site for backing up (i.e., archiving) data from the on-premises site to cloud/object storage using a mechanism referred to as local metadata staging. According to one set of embodiments, the client system can (1) receive an initial snapshot of a source dataset (e.g., file) to be archived, (2) package the data blocks of the snapshot into fixed-sized chunks, and (3) upload each chunk, as it is filled with snapshot data, to the cloud/object storage. The uploaded chunks can be appended/added to a data structure maintained on the cloud/object storage for the source dataset, referred to as the dataset&#39;s cloud archive. 
     Simultaneously with (2) and (3), the client system can locally stage (e.g., create and update) metadata describing the structure of the snapshot (as it is stored in cloud/object storage) in on-premises storage. This metadata, which is staged in a data structure on the on-premises storage referred to as the dataset&#39;s resident archive, can take the form of a B+ tree. The leaf nodes of the B+ tree can identify cloud physical block addresses (CBPAs) of the cloud archive where the data blocks of the snapshot are uploaded. 
     Finally, once all of the snapshot data has been uploaded and the locally-staged snapshot metadata has been fully updated, the client system can upload the snapshot metadata (as well as archive metadata) in the form of chunks to the cloud archive residing in cloud/object storage, thereby completing the archival/upload workflow for the snapshot. The client system can subsequently repeat this workflow for further snapshots of the dataset by calculating a delta between a given snapshot and the previous snapshot and uploading the data and modified metadata for the delta. 
     The foregoing and other aspects of the present disclosure are described in further detail in the sections that follow. 
     2. System Environment 
       FIG. 1  is a simplified block diagram of a system environment  100  in which embodiments of the present disclosure may be implemented. As shown, system environment  100  includes an on-premises client system  102  at a customer (i.e., on-premises) site  104  that is connected via the Internet  106  to a cloud/object storage service/system  108 . Client system  102  may be, e.g., a physical computer system or a virtual machine (VM). Cloud/object storage  108  may be any such storage service/system known in the art, such as Amazon&#39;s S3. 
     Although an exhaustive discussion of cloud/object storage  108  is beyond the scope of this disclosure, the following are a few salient characteristics that may be exhibited by cloud/object storage  108  in certain embodiments:
         Each object in cloud/object storage  108  (also referred to herein as a “chunk”) can be maintained in a flat address space and can include the data for the object itself (i.e., the object&#39;s data payload), a variable amount of object metadata, and a globally unique identifier (i.e., key).   Cloud/object storage  108  can expose a relatively simple data access API (application programming interface) to client system  102  that includes (1) a GET(k) function for retrieving an object identified by specified key k; (2) a PUT(o, k) function for creating or updating specified object o identified by specified key k; and (3) a DELETE(k) function for deleting an object identified by specified key k.       

     Typically, cloud/object storage  108  will be owned and maintained by a storage service provider, such as Amazon, that is distinct from the entity that owns customer site  104 . However, in some embodiments, cloud/object storage  108  can be part of a private cloud that is owned/maintained by the same entity as customer site  104 . 
     In addition to being connected to cloud/object storage  108 , client system  102  is also connected to an on-premises storage system  110  that includes a dataset  112 . Dataset  112  may be, e.g., virtual disk data for one or more VMs, a document repository, or any other type of dataset that is modified on an ongoing basis at customer site  104 . In this environment, the goal of client system  102  is to periodically archive dataset  112  from on-premises storage  110  to cloud/object storage  108  for data protection, such that the most recently backed-up copy of dataset  112  can be restored from cloud/object storage  108  if a disaster or failure occurs that causes the on-premises copy of the dataset to be lost. However, as mentioned previously, there are a number of challenges that make it difficult to accomplish this in an efficient and performant manner (e.g., weak consistency guarantees offered by cloud/object storage  108 , low bandwidth and high latency between customer site  104  and cloud/object storage  108 , etc.). 
     To address the foregoing and other related issues, client system  102  of  FIG. 1  is enhanced to include a novel archive management agent  114 . In various embodiments, archive management agent  114  may be implemented in software, in hardware, or a combination thereof. In a particular embodiment, archive management agent  114  may be implemented as a user-mode application and thus can make use of certain network security protocol libraries for communicating with cloud/object storage  108 , such as Transport Layer Security (TLS), that are only available in user space. 
     As detailed in the sections that follow, archive management agent  114  can employ techniques for archiving point-in-time copies (i.e., snapshots) of dataset  112  to cloud/object storage  108  in a manner that streams the new/modified data for each snapshot (in the form of fixed-size chunks) to a “cloud archive”  116  in cloud/object storage  108 , but stages metadata for the snapshot locally on client system  102  in a “resident archive”  118  while the snapshot data is being uploaded. This metadata can comprise a B+ tree structure whose leaf nodes point to cloud physical block addresses (CPBAs) in cloud archive  116  where each data block of the snapshot is uploaded, and whose intermediate nodes guide traversal down the tree (based on logical block addresses of dataset  112 ). 
     Then, when all of the new/modified snapshot data has been uploaded and the locally-staged snapshot metadata has been fully updated, archive management agent  114  can upload the snapshot metadata in the form of chunks to cloud archive  116 . Archive management agent  114  can also upload archive metadata comprising information regarding the snapshot (e.g., an association between the snapshot ID and a pointer to the root node of the snapshot&#39;s B+ tree, the snapshot&#39;s range of data chunks, the snapshot&#39;s range of metadata chunks, checksums, etc.). Once this metadata upload is done, the archival/upload workflow for the snapshot is complete. Archive management agent  114  can subsequently repeat this workflow for delta changes to dataset  112  captured in further snapshots, thereby archiving those further snapshots in cloud archive  116 . 
     With the high-level approach described above, a number of advantages can be realized. First, because the metadata for the snapshot upload is staged locally and updated/finalized in on-premises storage  110  before being sent to cloud/object storage  108 , there is no need to overwrite snapshot metadata in the cloud; this metadata is uploaded exactly once for each snapshot, at the end of the archival/upload workflow (note that there will typically be a large amount of metadata “churn” during this workflow as snapshot data chunks are processed and uploaded due to the creation and splitting of B+ tree nodes). Similarly, snapshot data is always appended to (rather than overwritten in) cloud archive  116 . These aspects avoid the problems raised by the eventual consistency model employed by existing cloud/object storage systems. 
     Second, by batching and uploading snapshot data and metadata in fixed-sized chunks (i.e., objects) rather than on a per-block basis, archive management agent  114  can more efficiently use the available bandwidth between customer site  104  and cloud/object storage  108 . 
     Third, in certain embodiments the locally-staged metadata in resident archive  118  can be leveraged by client system  102  to accelerate various archive operations, such as delete and restore. 
     It should be noted that two different approaches as possible for allocating local and cloud PBAs to snapshot metadata as the metadata is staged during the archival/upload workflow. According to a first approach (referred to herein as the “one-to-one mapping” approach), a particular predefined range of LPBAs may be reserved for snapshot metadata in resident archive  118  of on-premises storage  110  and an identical predefined range of cloud physical block addresses (CPBAs) may be reserved for snapshot metadata in cloud archive  116  of cloud/object storage  108 . For example, a range of zero to 2 terabytes may be reserved in the LPBA space of resident archive  118  and the CPBA space of cloud archive  116  respectively. Note that the CPBA of a given block in cloud archive  116  is determined by its chunk ID, the chunk size, and offset within that chunk; for instance, if agent  114  uploads metadata to cloud archive  116  in 1 MB chunks, the CPBA of a metadata block stored at chunk  4 , offset 4K will be (4×1 MB+4K)=4100K. 
     Then, at the time of creating/staging metadata locally in resident archive  118  during a snapshot upload, archive management agent  114  can allocate data blocks sequentially from the reserved LPBA range in resident archive  118  for holding the metadata, and at the time of uploading the locally staged metadata, archive management agent  114  can pack those metadata blocks according to the same sequence into chunks having sequential chunk IDs within the reserved CPBA range and upload the chunks to cloud archive  116 . This effectively results in a one-to-one mapping between the LBPAs of the metadata blocks in resident archive  118  and the CPBAs of those metadata blocks in cloud archive  116 , which avoids the need to perform any address translations at the time the metadata blocks are uploaded to cloud archive  116 . This approach is explained in further detail in Section 3 below. 
     According to a second approach (referred to herein as the “arbitrary mapping” approach), there is no correspondence between the LPBAs used to store metadata blocks on-premises and CPBAs used to store those same metadata blocks in cloud/object storage; rather, agent  114  uses any available blocks in the LPBA range of resident archive  118  to hold metadata during the local staging. As a result, once all of the metadata blocks for a given snapshot have been full updated in on-premises storage and are ready to be uploaded to cloud/object storage, agent  114  needs to identify the pointers in the B+ tree structure created for the snapshot (i.e., the pointers pointing to nodes within the B+ tree) and update those pointers to properly point to the CPBAs where those nodes will reside in the CPBA range of cloud archive  116 . This approach is explained in further detail in Section 4 below. 
     It should be appreciated that system environment  100  of  FIG. 1  is illustrative and not intended to limit embodiments of the present disclosure. For example, although only a single on-premises client system  102  is shown, any number of client systems may be configured to interact with cloud/object storage  108  for the purpose of backing up or restoring data set  112 , potentially on a concurrent basis. Further, the various entities depicted in  FIG. 1  may be organized according to alternative configurations or arrangements and/or may include components or functions that are not specifically described. One of ordinary skill in the art will recognize other variations, modifications, and alternatives. 
     3. Initial Snapshot Upload Workflow 
       FIG. 2A  depicts a workflow  200  that may be executed by archive management agent  114  for uploading/archiving an initial (i.e., first) snapshot of dataset  112  to cloud/object storage  108  using local metadata staging according to an embodiment. This workflow assumes that the metadata for the snapshot will be mapped in a one-to-one manner from the LPBA of resident archive  118  to the CPBA of cloud archive  116 . 
     Starting with step  202 , an initial snapshot (e.g., snapshot S 0 ) of dataset  112  can be taken on on-premises storage  110  and made available to archive management agent  114 . Since this is the first snapshot of dataset  112 , the snapshot will contain the entirety of the data of dataset  112 . 
     At step  204 , archive management agent  114  can allocate space on on-premises storage  110  for the resident archive of dataset  112  (i.e., resident archive  118 ), which will be used to locally stage metadata for the snapshots of dataset  112  that will be uploaded to cloud/object storage  108 . The physical block address range that is allocated to resident archive  118  here is referred to as the local physical block address (LPBA) range of archive  118 . As part of this step, archive management agent  114  can reserve a portion of the LPBA range for a “superblock,” which is a segment of resident archive  118  that stores metadata about the archive itself (e.g., snapshots in the archive, checksums, etc.). This superblock will typically be allocated one chunk, where “chunks” are the units of data that are uploaded by agent  114  to cloud/object storage  108 . In various embodiments, one chunk may have a fixed-size, such as 1 MB, 2 MB, 4 MB, etc. Archive management agent  114  can also reserve a portion of the LPBA range of resident archive  118  for storing snapshot metadata (e.g., a range of zero of 2 TB within the LPBA range). 
     Once archive management agent  114  has allocated space for resident archive  118  in on-premises storage  110 , agent  114  can also initialize a “bucket” in cloud/object storage  108  corresponding to the cloud archive for dataset  112  (i.e., cloud archive  116 ) (step  206 ). This bucket is essentially a named container that is configured to hold cloud objects (i.e., chunks) representing the snapshot data/metadata for dataset  112  that is uploaded by agent  114 . The cloud physical address space (CPBA) of cloud archive  116  starts at zero and is extended each time a chunk is written to archive  116 . Thus, since an unlimited number of chunks may generally be uploaded to cloud/object storage  108 , the CPBA of cloud archive  116  can potentially extend to infinity. The CPBA of a given block of data/metadata within cloud archive  116  can be calculated as chunk ID (i.e., ID of chunk in which the block resides)×chunk size+offset (i.e., offset of block within chunk). 
     In various embodiments, as part of step  206 , archive management agent  114  can create a superblock chunk in cloud archive  116  that corresponds to the superblock allocated in resident archive  118  at step  204 . In addition, archive management agent  114  can reserve a range of CPBAs (i.e., range of chunk IDs) in cloud archive  116  for snapshot metadata that is identical to the reserved metadata LPBA range in resident archive  118 . 
     At step  208 , archive management agent  114  can initialize a “data chunk ID” variable to some starting value X that corresponds to the chunk ID/location in the CPBA of cloud archive  116  where data chunks should begin being written to (this may be, e.g., the first chunk ID after the reserved metadata range). Archive management agent  114  can then begin reading the data in the initial snapshot of dataset  112 , on a block-by-block basis in increasing logical block address order (step  210 ). 
     At steps  212  and  214 , for each data block read from the initial snapshot, archive management agent  114  can place the data block into a memory buffer of fixed size that corresponds to the fixed-size chunks that will be uploaded to cloud/object storage  108 . For example, if agent  114  is configured to upload 4 MB chunks to cloud/object storage  108 , the memory buffer will be 4 MB in size. Archive management agent  114  can assign a chunk ID to this memory buffer corresponding to the current value of the data chunk ID variable (step  216 ). 
     Further, at step  218 , archive management agent  114  can build/update metadata (i.e., a B+ tree) for the initial snapshot based on the read data block and locally write this metadata to sequential blocks within the reserved metadata LPBA range of resident archive  118 . The internal nodes of the B+ tree are nodes that guide tree traversal down to the leaf nodes. The leaf nodes, in turn, are configured to point to the CPBAs (i.e. chunk IDs and offsets) in cloud archive  116  where the data blocks of the snapshot will be archived. The keys of the internal nodes reflect the logical block address space of the snapshot file. 
     For instance, assume a new data block of the initial snapshot is placed into the memory buffer at step  214  (for upload to cloud/object storage  108 ). In this case, a new leaf node of the snapshot&#39;s B+ tree can be created at step  218  that includes a pointer to the CPBA of the data block (i.e., chunk ID of memory buffer×chunk size+offset) and this leaf node will be written to the next free block within the reserved metadata LPBA range of resident archive  118 . Further, if the creation of the leaf node necessitates the creation of one or more parent (i.e., intermediate) nodes in the B+ tree per standard B+ tree node split criteria, such parent nodes will also be created and written sequentially into blocks in the reserved LPBA range of resident archive  118 . 
     At step  220 , archive management agent  114  can check whether the memory buffer used to hold data blocks from the snapshot has become full; if not, agent  114  can return to the start of the loop (step  212 ) to process the next data block. On the other hand, if the memory buffer has become full at step  220 , archive management agent  114  can package the contents of the memory buffer into a data chunk, upload the data chunk (with its assigned chunk ID) to cloud archive  116  of cloud/object storage  108 , and increment the data chunk ID variable (step  222 ) before reaching the end of the current loop iteration (step  224 ) returning to the start of the loop. Although not explicitly shown, if the current data block is the last data block in the snapshot, archive management agent  114  can package and upload the contents of the memory buffer to cloud/object storage  108  even if it has not reached capacity. 
     Once all of the data blocks from the initial snapshot have been read and processed, archive management agent  114  can sequentially read the metadata blocks that have been written to the reserved metadata LPBA range of resident archive  118  (step  226 ), package the metadata blocks into fixed-size chunks in a manner similar to the data blocks (step  228 ), and then sequentially upload these metadata chunks to the reserved CPBA range of cloud archive  116  (step  230 ). These metadata chunks are assigned chunk IDs that result in the LPBAs of the metadata blocks in resident archive  118  matching one-to-one with the CPBAs of the metadata blocks as they are stored in cloud archive  116 . Among other things, this one-to-one mapping ensures that the internal pointers in the B+ tree represented by the metadata (i.e., pointers pointing to internal nodes in the tree) are still valid once uploaded to cloud/object storage  108 , and thus the tree can be properly traversed using the cloud-archived metadata. 
     Finally, at step  232 , archive management agent  114  can upload archive metadata to the superblock chunk in cloud archive  116  that includes, e.g., an association between the ID of the current snapshot (e.g., S 0 ) and the PBA of the root node of the B+ tree for the snapshot (thereby allowing the metadata for the snapshot to be found and traversed), as well as potentially other archive metadata (e.g., range of metadata chunks for snapshot, range of data chunks for snapshot, checksums, etc.). Once this is completed, the archival/upload process for the snapshot is done and the workflow can end. 
       FIG. 2B  is a diagram  250  that illustrates the contents of cloud archive  116  at the conclusion of upload workflow  200  according to an embodiment. As shown in diagram  250 , cloud archive  116  includes a superblock chunk  252  (associated with chunk ID  0 ), a number of metadata chunks  254 ( 1 )-(M) for the uploaded snapshot (associated with chunk IDs  1  to M within reserved metadata range  256 ), and a number of data chunks  258 ( 1 )-(N) (associated with chunk IDs X to X+N, where X is the first chunk ID after the end of reserved metadata range  256 ). In this example, the CPBA of cloud archive extends from zero to (X+N)×S, where S is the fixed size of each metadata/data chunk. This CPBA will be extended further as new chunks are uploaded to cloud archive  116  for subsequent snapshots of dataset  112 . 
     4. Alternative Metadata Mapping (Arbitrary) 
     As mentioned previously, as an alternative to performing one-to-one mapping of metadata between the LPBA of resident archive  118  and the CPBA of cloud archive  116 , archive management agent  114  can instead arbitrarily allocate blocks for metadata from the LPBA during local metadata staging. With this alternative approach, there is no reserved address range for metadata in the LPBA or CPBA; instead, as agent  114  is building the B+ tree for the snapshot, the agent can allocate blocks from anywhere in the LPBA and use those allocated blocks to hold the B+ tree data (i.e., node information). Then, when all data chunks have been sent to the cloud, archive management agent  114  can perform a process for uploading the metadata to cloud/object storage  108  that includes translating metadata pointers that point to LPBAs (i.e., pointers to internal tree nodes) to instead point to appropriate CPBAs where the metadata will be uploaded.  FIG. 3  depicts a workflow  300  of this metadata upload process according to an embodiment. 
     Starting with step  302 , archive management agent  114  can walk through the B+ tree created/built during the data upload phase of archival workflow  200 , from the lowest to highest level in tree. 
     For each encountered tree node (step  304 ), archive management agent  114  can place the node into a fixed-size memory buffer corresponding to the size of a single chunk (step  306 ) and can assign a chunk ID to this buffer (step  308 ). Agent  114  can start this chunk ID at the last value of the data chunk ID variable described earlier, such that metadata chunks are written to the CPBA immediately following the data chunks for the snapshot. 
     At step  310 , archive management agent  114  can record the current chunk ID and offset for the node within the chunk in a temporary mapping table. This mapping table can associate the cloud chunk ID/offset for the node with the node&#39;s LPBA in resident archive  118 . 
     Then, if the node includes a pointer to a LPBA for a child node in the B+ tree (step  312 ), archive management agent  114  can determine the cloud chunk ID/offset for that child node from the temporary mapping table based on its LBPA (step  314 ) and can replace the LPBA with the chunk ID/offset in the node, thereby translating the LPBA to a CPBA (i.e., chunk ID/offset) (step  316 ). 
     Finally, if the memory buffer is now full (step  318 ), archive management agent  114  can upload the contents of the memory buffer as a chunk (with its assigned chunk ID) to cloud archive  116  in cloud/object storage  108 , thereby archiving it there (step  320 ). The current loop iteration can then end (step  322 ) and archive management agent  114  can return to the top of the loop (step  302 ) and repeat this process until all tree nodes have been processed. 
     With workflow  300 , the structure of cloud archive  116  shown in  FIG. 2B  will be slightly different since there is no reserved metadata range  256 ; instead, the metadata chunks for the uploaded snapshot ( 254 ( 1 )-(M)) will appear in the CPBA after data chunks  258 ( 1 )-(N). 
     5. Delta Snapshot Upload Workflow 
       FIG. 4  depicts a workflow  400  that may be executed by archive management agent  114  for uploading/archiving a delta (e.g., second or later) snapshot of dataset  112  to cloud/object storage  108  using local metadata staging according to an embodiment. This workflow assumes that at least one snapshot of dataset  112  has already been uploaded per workflow  200  of  FIG. 2A  and now a second snapshot needs to be uploaded that captures changes to dataset  112  since the first snapshot. 
     The steps of workflow  400  are largely similar to workflow  200 ; however, rather than starting with an initial snapshot of dataset  112 , a new snapshot of the dataset is taken at block  402  and a delta between the new snapshot and the immediately previous snapshot (i.e., the data blocks that have changed between the two snapshots) is determined at block  404 . This delta is then read by archive management agent  114  and processed at subsequent blocks  406 - 428  in a manner that is analogous to blocks  210 - 232  of workflow  200 . 
     It should be noted that, as part of building the B+ tree for the delta snapshot data, archive management agent  114  can reuse the nodes of B+ trees of previous snapshot (in other words, point to existing tree nodes of previous snapshot(s) for portions of the tree that have not changed). For portions of the B+ tree that do need to be modified for the delta snapshot data, archive management agent  114  can employ copy-on-write to create new copies of those specific nodes. 
     In addition, it should be noted that at step  428  archive management agent  114  overwrites the existing superblock chunk in cloud archive  116  in order to update it with the metadata for the current snapshot (e.g., snapshot ID and pointer to the root node of the snapshot&#39;s B+ tree). As mentioned previously, performing such overwrites in cloud/object storage  108  can raise consistency issues since most cloud/object storage systems only guarantee eventual consistency. One mechanism for managing this issue is addressed in the next section below. 
     6. Managing Overwrites to Superblock Chunk 
     Per block  428  of workflow  400 , archive management agent  114  overwrites the superblock chunk in cloud archive  116  at the conclusion of the snapshot archival/upload process in order to update the superblock with metadata regarding the uploaded snapshot (e.g., snapshot ID and pointer to snapshot&#39;s B+ tree root node). Since overwrites are only eventually consistent in most cloud/object storage systems, this can cause numerous problems when the superblock needs to be accessed again for various archive operations. For example, consider a scenario where a client wishes to restore the most recently archived snapshot of dataset  112  (e.g., snapshot S 100 ). In this case, the client will read the superblock chunk of cloud archive  116 , which was updated with information regarding S 100  during the last upload workflow. However, assuming cloud/object storage  108  is only eventually consistent, the read (i.e., GET) operation requested by the client may return an older version of the superblock that identifies a snapshot that is older than the most recent snapshot (e.g., snapshot S 90 ). Thus, the client may begin restoring from older snapshot S 90  under the erroneous belief that it is restoring the latest version of the data. 
     To address this,  FIG. 5A  depicts a workflow  500  that can be performed by archive management agent  114  at the time of overwriting the superblock chunk in cloud archive  116  and  FIG. 5B  depicts a complementary workflow  500  that can be performed by a client at the time of accessing the superblock in order to identify the most recently uploaded snapshot. Taken together, these two workflows can ensure that the client can always correctly determine the most recent snapshot in cloud archive  116 , despite the eventual consistency property of cloud/object storage  108  (this solution assumes that cloud/object storage  108  supports read-after-write consistency for newly created objects). 
     Starting with step  502  of workflow  500 , archive management agent  114  can overwrite (i.e., update) the superblock chunk of cloud archive  116  with archive metadata for the most recently uploaded snapshot. This archive metadata can include an identifier of the snapshot and a pointer (e.g., chunk ID and offset) to the root node of the snapshot&#39;s B+ tree. This step is substantially similar to step  428  of workflow  400 . 
     However, rather than simply overwriting the superblock chunk, archive management agent  114  can also create a new instance of a special file in cloud archive  116  (referred to as a “.ARCHIVE” file) that has a version number corresponding to the snapshot ID number (step  504 ). For example, if the most recently uploaded snapshot is SX, the .ARCHIVE file created at block  504  will have a version number X (e.g., .ARCHIVE.X). This newly created file version will be readable by all clients immediately after its creation under the property of read-after-write consistency. This is illustrated in diagram  570  of  FIG. 5C , which shows cloud archive  116  with .ARCHIVE files ARCHIVE. 0  to ARCHIVE.X (one file for each uploaded snapshot S 0  to SX). In various embodiments, these .ARCHIVE files do not contain any data content of substance; instead, the reason for creating these files is to simply track the ID of the most recently uploaded/archived snapshot by virtue of the .ARCHIVE file version numbers. 
     Turning now to workflow  550 , at the time a client wishes to determine the most recently archived snapshot for dataset  112 , the client can first read the superblock chunk in cloud archive  116  and determine the latest snapshot ID recorded there (step  552 ). For example, the client may determine that the latest snapshot ID in the superblock is SY, where Y is some number. The client can then check whether a .ARCHIVE file exists in cloud archive file  116  with a version number corresponding to Y+1 (step  554 ). If not, the client can conclude that Y is the latest snapshot archived for dataset  112  (step  556 ). 
     However, if the client determines at step  554  that a .ARCHIVE file does exist with a version number corresponding Y+1, the client can set Y=Y+1 (step  558 ) and then return to step  554  to continue checking whether a .ARCHIVE file exists with a further incremented version number. This process can repeat for increasing values of Y until the latest version of the .ARCHIVE file is found at step  556 , which identifies the most recently archived snapshot of dataset  112 . 
     Finally, once the latest .ARCHIVE file (and thus latest snapshot) is found, the client can take an appropriate action based on this information (step  560 ). For example, if the client is attempting to restore the latest snapshot and determines that the latest snapshot differs from what is found in the superblock at step  552 , the client may wait until the superblock properly reflects the archive metadata for the latest snapshot. Alternatively, the client may simply decide to begin restoring from the older snapshot found in the superblock. 
     Certain embodiments described herein can employ various computer-implemented operations involving data stored in computer systems. For example, these operations can require physical manipulation of physical quantities—usually, though not necessarily, these quantities take the form of electrical or magnetic signals, where they (or representations of them) are capable of being stored, transferred, combined, compared, or otherwise manipulated. Such manipulations are often referred to in terms such as producing, identifying, determining, comparing, etc. Any operations described herein that form part of one or more embodiments can be useful machine operations. 
     Further, one or more embodiments can relate to a device or an apparatus for performing the foregoing operations. The apparatus can be specially constructed for specific required purposes, or it can be a general purpose computer system selectively activated or configured by program code stored in the computer system. In particular, various general purpose machines may be used with computer programs written in accordance with the teachings herein, or it may be more convenient to construct a more specialized apparatus to perform the required operations. The various embodiments described herein can be practiced with other computer system configurations including handheld devices, microprocessor systems, microprocessor-based or programmable consumer electronics, minicomputers, mainframe computers, and the like. 
     Yet further, one or more embodiments can be implemented as one or more computer programs or as one or more computer program modules embodied in one or more non-transitory computer readable storage media. The term non-transitory computer readable storage medium refers to any data storage device that can store data which can thereafter be input to a computer system. The non-transitory computer readable media may be based on any existing or subsequently developed technology for embodying computer programs in a manner that enables them to be read by a computer system. Examples of non-transitory computer readable media include a hard drive, network attached storage (NAS), read-only memory, random-access memory, flash-based nonvolatile memory (e.g., a flash memory card or a solid state disk), a CD (Compact Disc) (e.g., CD-ROM, CD-R, CD-RW, etc.), a DVD (Digital Versatile Disc), a magnetic tape, and other optical and non-optical data storage devices. The non-transitory computer readable media can also be distributed over a network coupled computer system so that the computer readable code is stored and executed in a distributed fashion. 
     Finally, boundaries between various components, operations, and data stores are somewhat arbitrary, and particular operations are illustrated in the context of specific illustrative configurations. Other allocations of functionality are envisioned and may fall within the scope of the invention(s). In general, structures and functionality presented as separate components in exemplary configurations can be implemented as a combined structure or component. Similarly, structures and functionality presented as a single component can be implemented as separate components. 
     As used in the description herein and throughout the claims that follow, “a,” “an,” and “the” includes plural references unless the context clearly dictates otherwise. Also, as used in the description herein and throughout the claims that follow, the meaning of “in” includes “in” and “on” unless the context clearly dictates otherwise. 
     The above description illustrates various embodiments along with examples of how aspects of particular embodiments may be implemented. These examples and embodiments should not be deemed to be the only embodiments, and are presented to illustrate the flexibility and advantages of particular embodiments as defined by the following claims. Other arrangements, embodiments, implementations and equivalents can be employed without departing from the scope hereof as defined by the claims.