Method and apparatus for increasing the accuracy of predicting future IO operations on a storage system

A method of increasing the accuracy of predicting future IO operations on a storage system includes creating a snapshot of a production volume, linking the snapshot to a thin device, mounting the thin device in a cloud tethering subsystem, and tagging the thin device to identify the thin device as being used by the cloud tethering subsystem. When data read operations are issued by the cloud tethering subsystem on the tagged thin device, the data read operations are executed by a front-end adapter of the storage system to forward data associated with the data read operations to a cloud repository. The cache manager, however, does not use information about data read operations on tagged thin devices in connection with predicting future IO operations on the cache, so that movement of snapshots to the cloud repository do not skew the algorithms being used by the cache manager to perform cache management.

FIELD

This disclosure relates to computing systems and related devices and methods, and, more particularly, to a method and apparatus for increasing the accuracy of predicting future IO operations on a storage system.

SUMMARY

The following Summary and the Abstract set forth at the end of this document are provided herein to introduce some concepts discussed in the Detailed Description below. The Summary and Abstract sections are not comprehensive and are not intended to delineate the scope of protectable subject matter, which is set forth by the claims presented below.

A method of increasing the accuracy of predicting future IO operations on a storage system includes creating a snapshot of a production volume, linking the snapshot to a thin device, mounting the thin device in a cloud tethering subsystem, and tagging the thin device to identify the thin device as being used by the cloud tethering subsystem. When data read operations are issued by the cloud tethering subsystem on the tagged thin device, the data read operations are executed by a front-end adapter of the storage system to forward data associated with the data read operations to a cloud repository. The cache manager, however, does not use information about data read operations on tagged thin devices in connection with predicting future IO operations on the cache, so that movement of snapshots to the cloud repository do not skew the algorithms being used by the cache manager to perform cache management. Likewise, the compression subsystem does not include data associated with read operations on tagged thin devices in the uncompressed data pool, to prevent movement of snapshots to a cloud repository from consuming space in the uncompressed data pool.

DETAILED DESCRIPTION

Aspects of the inventive concepts will be described as being implemented in a storage system100connected to a host computer102. Such implementations should not be viewed as limiting. Those of ordinary skill in the art will recognize that there are a wide variety of implementations of the inventive concepts in view of the teachings of the present disclosure.

Some aspects, features and implementations described herein may include machines such as computers, electronic components, optical components, and processes such as computer-implemented procedures and steps. It will be apparent to those of ordinary skill in the art that the computer-implemented procedures and steps may be stored as computer-executable instructions on a non-transitory tangible computer-readable medium. Furthermore, it will be understood by those of ordinary skill in the art that the computer-executable instructions may be executed on a variety of tangible processor devices, i.e., physical hardware. For ease of exposition, not every step, device or component that may be part of a computer or data storage system is described herein. Those of ordinary skill in the art will recognize such steps, devices and components in view of the teachings of the present disclosure and the knowledge generally available to those of ordinary skill in the art. The corresponding machines and processes are therefore enabled and within the scope of the disclosure.

The terminology used in this disclosure is intended to be interpreted broadly within the limits of subject matter eligibility. The terms “logical” and “virtual” are used to refer to features that are abstractions of other features, e.g. and without limitation, abstractions of tangible features. The term “physical” is used to refer to tangible features, including but not limited to electronic hardware. For example, multiple virtual computing devices could operate simultaneously on one physical computing device. The term “logic” is used to refer to special purpose physical circuit elements, firmware, and/or software implemented by computer instructions that are stored on a non-transitory tangible computer-readable medium and implemented by multi-purpose tangible processors, and any combinations thereof.

FIG. 1illustrates a storage system100and an associated host computer102, of which there may be many. The storage system100provides data storage services for a host application104, of which there may be more than one instance and type running on the host computer102. In the illustrated example, the host computer102is a server with host volatile memory106, persistent storage108, one or more tangible processors110, and a hypervisor or OS (Operating System)112. The processors110may include one or more multi-core processors that include multiple CPUs (Central Processing Units), GPUs (Graphics Processing Units), and combinations thereof. The host volatile memory106may include RAM (Random Access Memory) of any type. The persistent storage108may include tangible persistent storage components of one or more technology types, for example and without limitation SSDs (Solid State Drives) and HDDs (Hard Disk Drives) of any type, including but not limited to SCM (Storage Class Memory), EFDs (Enterprise Flash Drives), SATA (Serial Advanced Technology Attachment) drives, and FC (Fibre Channel) drives. The host computer102might support multiple virtual hosts running on virtual machines or containers. Although an external host computer102is illustrated inFIG. 1, in some embodiments host computer102may be implemented as a virtual machine within storage system100.

The storage system100includes a plurality of compute nodes1161-1164, possibly including but not limited to storage servers and specially designed compute engines or storage directors for providing data storage services. In some embodiments, pairs of the compute nodes, e.g. (1161-1162) and (1163-1164), are organized as storage engines1181and1182, respectively, for purposes of facilitating failover between compute nodes116within storage system100. In some embodiments, the paired compute nodes116of each storage engine118are directly interconnected by communication links120. As used herein, the term “storage engine” will refer to a storage engine, such as storage engines1181and1182, which has a pair of (two independent) compute nodes, e.g. (1161-1162) or (1163-1164). A given storage engine118is implemented using a single physical enclosure and provides a logical separation between itself and other storage engines118of the storage system100. A given storage system100may include one storage engine118or multiple storage engines118.

Each compute node,1161,1162,1163,1164, includes processors122and a local volatile memory124. The processors122may include a plurality of multi-core processors of one or more types, e.g. including multiple CPUs, GPUs, and combinations thereof. The local volatile memory124may include, for example and without limitation, any type of RAM. Each compute node116may also include one or more front end adapters126for communicating with the host computer102. Each compute node1161-1164may also include one or more back-end adapters128for communicating with respective associated back-end drive arrays1301-1304, thereby enabling access to managed drives132. A given storage system100may include one back-end drive array130or multiple back-end drive arrays130.

In some embodiments, managed drives132are storage resources dedicated to providing data storage to storage system100or are shared between a set of storage systems100. Managed drives132may be implemented using numerous types of memory technologies for example and without limitation any of the SSDs and HDDs mentioned above. In some embodiments the managed drives132are implemented using NVM (Non-Volatile Memory) media technologies, such as NAND-based flash, or higher-performing SCM (Storage Class Memory) media technologies such as 3D XPoint and ReRAM (Resistive RAM). Managed drives132may be directly connected to the compute nodes1161-1164, using a PCIe (Peripheral Component Interconnect Express) bus or may be connected to the compute nodes1161-1164, for example, by an IB (InfiniBand) bus or fabric.

In some embodiments, each compute node116also includes one or more channel adapters134for communicating with other compute nodes116directly or via an interconnecting fabric136. An example interconnecting fabric136may be implemented using InfiniBand. Each compute node116may allocate a portion or partition of its respective local volatile memory124to a virtual shared “global” memory138that can be accessed by other compute nodes116, e.g. via DMA (Direct Memory Access) or RDMA (Remote Direct Memory Access). Shared global memory138will also be referred to herein as the cache of the storage system100.

The storage system100maintains data for the host applications104running on the host computer102. For example, host application104may write data of host application104to the storage system100and read data of host application104from the storage system100in order to perform various functions. Examples of host applications104may include but are not limited to file servers, email servers, block servers, and databases.

Logical storage devices are created and presented to the host application104for storage of the host application104data. For example, as shown inFIG. 1, a production device140and a corresponding host device142are created to enable the storage system100to provide storage services to the host application104.

The host device142is a local (to host computer102) representation of the production device140. Multiple host devices142, associated with different host computers102, may be local representations of the same production device140. The host device142and the production device140are abstraction layers between the managed drives132and the host application104. From the perspective of the host application104, the host device142is a single data storage device having a set of contiguous fixed-size LBAs (Logical Block Addresses) on which data used by the host application104resides and can be stored. However, the data used by the host application104and the storage resources available for use by the host application104may actually be maintained by the compute nodes1161-1164at non-contiguous addresses (tracks) on various different managed drives132on storage system100.

In some embodiments, the storage system100maintains metadata that indicates, among various things, mappings between the production device140and the locations of extents of host application data in the virtual shared global memory138and the managed drives132. In response to an IO (Input/Output command)146from the host application104to the host device142, the hypervisor/OS112determines whether the IO146can be serviced by accessing the host volatile memory106. If that is not possible then the IO146is sent to one of the compute nodes116to be serviced by the storage system100.

There may be multiple paths between the host computer102and the storage system100, e.g. one path per front end adapter126. The paths may be selected based on a wide variety of techniques and algorithms including, for context and without limitation, performance and load balancing. In the case where IO146is a read command, the storage system100uses metadata to locate the commanded data, e.g. in the virtual shared global memory138or on managed drives132. If the commanded data is not in the virtual shared global memory138, then the data is temporarily copied into the virtual shared global memory138from the managed drives132and sent to the host application104by the front end adapter126of one of the compute nodes1161-1164. In the case where the IO146is a write command, in some embodiments the storage system100copies a block being written into the virtual shared global memory138, marks the data as dirty, and creates new metadata that maps the address of the data on the production device140to a location to which the block is written on the managed drives132. The virtual shared global memory138may enable the production device140to be reachable via all of the compute nodes1161-1164and paths, although the storage system100can be configured to limit use of certain paths to certain production devices140(zoning).

Not all volumes of data on the storage system are accessible to host computer104. When a volume of data is to be made available to the host computer, a logical storage volume, also referred to herein as a TDev (Thin Device), is linked to the volume of data, and presented to the host computer104as a host device142. For example, to protect the production device against loss of data, a snapshot (point in time) copy of the production device may be created and maintained by the storage system. If the host needs to obtain access to the snapshot copy, for example for data recovery, the snapshot copy may be linked to a logical storage volume and presented to the host computer104as a host device142that the host computer102can use to access the data of the snapshot copy.

As shown inFIG. 1, in some embodiments the storage system100has an operating system150, and one or more system applications, such as a cache manager, cloud tethering subsystem154, snapshot subsystem156, data services subsystem158, and compression subsystem160. In some embodiments, the system applications are implemented as applications executing within virtual machines on storage system100.

One goal of the cache manager152is to attempt to place data into the cache (implemented as shared memory138) that is likely to be required by one or more of the applications104. By intelligently selecting data to be pre-fetched to the cache or to be retained in the cache138, it is possible to increase the cache hit ratio. Because the storage system100can service an IO faster if the requested data is in the cache, increasing the cache hit ratio can thereby decrease overall latency of the storage system100. Example adjustments to cache policies might include changing an amount of cache allocated to each application, adjusting how long data is retained in the cache, adjusting cache lookahead parameters, adjusting how much cache space is allocated to content that is requested once vs content that is requested two or more times, and other cache adjustments.

In some embodiments, the cache manager152executes intelligent algorithms designed to predict future IO operations on the storage system, to determine what data should be stored in the cache. Since the storage system is able to serve read IOs faster if the data is stored in the cache, correctly determining what data should be pre-loaded to the cache or retained in the cache can reduce the overall latency of the storage system. In some embodiments, the cache manager152uses data read IO operation and write IO operation patterns, in connection with the intelligent algorithms, to determine which data should be stored in the cache138.

The cache manager152may implement multiple cache policies to try to optimize what data is stored in the cache. For example, one policy that the cache manager152may implement is a cache look-ahead policy. A cache look-ahead policy causes multiple consecutive blocks to be loaded to the cache each time a read IO or read IO pattern occurs on the storage system. For example, if a storage system is experiencing a high volume of consecutive data read operations, changing the cache look-ahead policy to increase the number of blocks that are pre-fetched to the cache may cause the cache hit ratio to increase. Likewise, if the storage system is experiencing a high volume of repeat read operations (reading the same data), increasing the proportion of the cache allocated to store repeatedly read data may increase the cache hit ratio.

Unfortunately, movement of snapshots by the cloud tethering subsystem involves long sequential read operations, which can skew the intelligent algorithms used by the cache manager152, and reduce performance of the storage system. As discussed in greater detail herein, in some embodiments read operations by the cloud tethering subsystem154that are associated with movement of snapshots to the cloud repository162are not considered by the intelligent algorithms, to prevent read operations by the cloud tethering subsystem154from negatively affecting the ability of the cache manager152to accurately predict future IO operations on the storage system100.

Compression subsystem160, in some embodiments, is designed to reduce the amount of storage space required to store a given data volume, by running a compression algorithm on the data. Since decompression takes a finite amount of time, to minimize latency of the storage system, the compression subsystem160keeps at least a minimum amount of data, such as the most recent 20% of the data, in an uncompressed storage pool. Unfortunately, movement of data by the cloud tethering subsystem to cloud repository162can use a portion of the uncompressed storage pool, thus reducing the amount of space available for use by other active data. In some embodiments, the compression subsystem160does not include data associated with movement of snapshots by the cloud tethering subsystem in the uncompressed storage pool, to thereby free up room in the uncompressed storage pool for other more important data.

Snapshot subsystem156, in some embodiments, is configured to create “snapshots” of a volume of data. A “snapshot,” as that term is used herein, is a copy of a volume of data as that volume existed at a particular point in time. A snapshot of a production device140, accordingly, is a copy of the data stored on the production device140as the data existed at the point in time when the snapshot was created. A snapshot can be either target-less (not linked to a thin device) or may be linked to a target thin device when created. When a snapshot of a production volume is created, the snapshot may include all of the data of the production volume, or only the changes to the production volume that have occurred since the previous snapshot was taken.

In some embodiments, a user will set policies on a group of LUNs referred to as a storage group. These policies define the frequency of the snapshot, the retention period of the snapshot, and the cloud provider where the object repository is to be hosted. The frequency tells the snapshot subsystem156in the storage array130to create a snapshot against all the LUNs in a storage group at a regular cadence, as defined by the user. The set of snapshots taken against a storage group are referred to as snapsets. The retention period defines the age of the snapshot when it should be deleted. The cloud provider tells the storage array the object repository where the snapshots need to be shipped.

The cloud tethering subsystem154is responsible for managing transmission of snapshots from the storage system100to an external cloud repository162. For example, it may be desirable to move at least some of the snapshot copies from the storage system100to a cloud repository162to free up space in the back-end drive arrays130, or for many other reasons. In some embodiments, the cloud tethering subsystem154is implemented as an application104executing in a container in an emulation on storage system100. To access storage resources of the storage system100, the cloud tethering subsystem154issues read and write IO operations146, which are received by front end adapter126of the storage system, and processed by the front end adapter126in the same manner as other read/write operations.

According to some embodiments, the cloud tethering subsystem uses a set of thin devices that are tagged. The tag identifies the thin devices as being used by the cloud tethering subsystem to move snapshots to a cloud repository. After the snapshots are moved to the cloud repository, the snapshots are deleted on the storage system. Since the read operations associated with movement of snapshots to the cloud repository involve data that is being removed from the storage system, the data associated with the data read operations by the cloud tethering subsystem will not recur. By tagging the thin devices, the cache manager152and compression subsystem160can treat the read operations differently than other normal read operations. Specifically, when the cache manager detects that the read operation was on a tagged thin device, the cache manager does not use the read operation in connection with updating its intelligent algorithms. Similarly, when the compression subsystem detects that the read operation was on a tagged thin device, the compression subsystem does not include the data associated with the read operation in the uncompressed data pool.

FIG. 2is a functional block diagram of a storage system100connected to a cloud repository162, according to some embodiments. As shown inFIG. 2, in some embodiments the snapshot subsystem156periodically creates snapshot copies164of a production volume140. If the cloud tethering subsystem154determines that the snapshot copy164should be moved from the storage system100to a cloud repository162, a TDev (Thin Device) is linked to the snapshot copy and presented to the cloud tethering subsystem154. The TDev that is linked to the snapshot copy for use by the cloud tethering subsystem will be referred herein as a “donor TDev”168. The cloud tethering subsystem154executes read operations on the donor TDev168to access the data of the snapshot copy164, and then transmits the data to the cloud repository162. Once a snapshot replica166has been created in the cloud repository162, the snapshot copy164on the storage system100can be erased.

If it is later necessary to retrieve the snapshot replica166from the cloud repository162, the cloud tethering subsystem154is assigned a thin device to be used to store the snapshot164. The cloud tethering subsystem then retrieves the snapshot replica166from the cloud repository162, and executes write operations on the thin device168to cause the snapshot164to be restored to the storage system100.

In some embodiments, the cloud tethering subsystem154implements a block LUN based snapshot shipping application, that ships sets of snapshots taken on a timeline to a heterogenous cloud repository162in object format. In some embodiments, the cloud tethering subsystem154is implemented as a Linux-based in-build container that is responsible for shipping the snapshots164to the cloud repository162. The cloud snapshot shipping process generates cloud read traffic end to end within the storage system100where the cloud tethering subsystem154container is hosted, when the snapshot164is shipped to the cloud repository162.

Once the snapshot is shipped, all traces of the snapshot164are removed from the embedded operating system150running on the storage system100, although the cloud tethering subsystem154container continues to maintain a full catalogue of snapshots in the cloud repository162. The user can see a full list of the snapshots in the cloud repository162via storage manager155, and can use the storage manager155to instruct the cloud tethering subsystem154to recover the particular snapshots from the cloud repository162to the storage system100. The recovery operation generates write IO traffic into the storage system100.

The cloud tethering subsystem154is the primary data mover that is responsible for moving IO traffic between the back-end drive array130in the storage system100and the remote cloud repository162. The storage system100has high value machine learning techniques used by the cache manager152to increase the cache read hits, which involves keeping the most active data uncompressed and ready in the cache138. This is also referred to as activity-based compression for red hot data. In some embodiments, 20% of the red-hot data is always kept uncompressed in the cache138. Unfortunately, shipment of the snapshot copies164from the storage system100to the cloud repository162by the cloud tethering subsystem154can create a skew in these red-hot data calculations. Specifically, when the cloud tethering subsystem154executes a large set of sequential read operations, these operations by the cloud tethering subsystem154can cause the cache manager152to determine that traffic on the cache is more sequential than it actually is. This can make it difficult for the cache manager152to accurately predict future IO operations on the storage system, which can result in a lower cache hit rate and, accordingly, increased storage system latency.

According to some embodiments, movement of data by the cloud tethering subsystem154between the storage system100and cloud repository162is treated different than other data access operations in the storage system100, to thereby increase the accuracy of predicting future IO operations on the storage system100. By making the storage system aware of IO operations associated with movement of data to the cloud repository162, the storage system100can handle those conditions intelligently without impacting the overall system performance and in a fully automated way.

In some embodiments, read IO traffic by the cloud tethering subsystem154is not used by the cache manager152in connection with predicting future IO operations on the storage subsystem. Specifically, since the read IO operations by the cloud tethering subsystem154are associated with removing data (snapshot164) from the storage system, those data associated with the read operations by the cloud tethering subsystem154will only occur once, and will not occur again after the snapshot has been moved to the cloud repository162. Accordingly, the cache manager152should not use the data read operations by the cloud tethering subsystem154in connection with the intelligent algorithms. Likewise, the compression subsystem160should not keep any of the data associated with read operations by the cloud tethering subsystem154in uncompressed data pool. Cache write operations by the cloud tethering subsystem154, however, involve movement of data back to the storage system100at the request of a user, and accordingly, that data is likely to be accessed at a future point in time. Accordingly, in some embodiments, cache write operations by the cloud tethering subsystem154are included in the intelligent algorithms implemented by the cache manager152to predict future IO operations on the storage system.

FIG. 3is a flow chart of a method of increasing the accuracy of predicting future IO operations on a storage system, such as in the storage system ofFIG. 1, in connection with movement of a snapshot to a cloud repository, according to some embodiments. The method may be implemented in connection with read operations associated with transmission of snapshots to a cloud repository, according to some embodiments. AlthoughFIG. 3will discuss movement of volumes of data by the cloud tethering subsystem154in connection with movement of snapshots164to a cloud repository, the cloud tethering subsystem154could also be used to move other volumes of data from the storage system100to the cloud repository162as well. Likewise, although some embodiments are discussed in connection with movement of data off the storage system100by cloud tethering subsystem154, the method ofFIG. 3can be extended to movement of storage volumes by other system applications where the other system applications are removing the storage volumes from the storage system100.

As shown inFIG. 3, in some embodiments the cloud tethering subsystem154is moving snapshots164that are created by the snapshot subsystem156(block300). In some embodiments, some snapshots164are kept on the storage system100, and some snapshots164are shipped by the cloud tethering subsystem154to the cloud repository162. Those snapshots164that are to be moved to the cloud repository162are tagged as being designated to be moved to the cloud repository162(block305). A tag such as “Cloud_Donor_Snap” may be used to designate a snapshot as being intended to be moved to the cloud repository162. The snapshots164may be tagged as designated to be moved to the cloud repository162immediately upon creation by the snapshot subsystem156, or may be tagged at some later point in time after creation of the snapshot164.

In some embodiments, a designated set of TDevs (Thin Devices) are created for use by the cloud tethering subsystem154in connection with shipping data volumes to the cloud repository162. These thin devices are tagged, for example, using a “CLOUD_DONOR_ACCESS_THIN_DEV” tag. As used herein, the term “donor TDev” will be used to refer to a thin device that has been tagged to identify it as being used by the cloud tethering subsystem154to move snapshots from the storage system100to the cloud repository162.

When the cloud tethering subsystem154wants to ship the snapshot164to the cloud repository162, it needs to access the snapshot164. In some embodiments, the cloud tethering subsystem154sends a prepare call to the snapshot subsystem156. As part of the prepare call, the cloud-tagged snapshot164is linked in nocopy mode to a donor TDev (block310). In no-copy mode, the data of the snapshot is not actually copied to the donor TDev. The donor TDev is tagged to indicate that it is linked to cloud data (block315). The donor TDev world-wide number (WWN) is also shared with the cloud tethering subsystem154, and the donor TDev is masked and mapped to the container of the cloud tethering subsystem154. With the WWN of the donor TDev, the cloud tethering subsystem154can issue read operations on the donor TDev to cause the data of the snapshot to be shipped to the cloud repository162(block320).

The donor TDev may be defined or undefined. As used herein, the term “define” is used to refer to a process that changes the pointers of tracks of a target linked volume to share the same backend data as the source volume. When a target volume is linked, a define process scans the entire target device. The define process changes the pointers of each track on the target device to identify the appropriate backend data of the source volume.

Thus, if the donor TDev is defined, the define process is used to cause the tracks of the donor TDev to point to the backend tracks of the snapshot164before the cloud tethering subsystem154issues read operations on the donor TDev. If the donor TDev is undefined, by contrast, the define process is not used, and accordingly the tracks of the donor TDev do not point to the backend tracks of the snapshot164before the cloud tethering subsystem154issues read operations on the donor TDev.

The manner in which the storage system100handles read operations by the cloud tethering subsystem154will depend on whether the donor TDev is defined or undefined. Specifically, as shown inFIG. 3, in some embodiments, a determination is made as to whether the donor TDev is defined (block325).

If the donor TDev is defined (a determination of YES at block325), the cloud tethering subsystem154issues a read operation to the front-end adapter126for the data referenced in the defined track of the donor TDev (block330). The front-end adapter moves the requested data to a selected cache slot in shared memory138, and completes the read request by sending out the data from the cache slot address to the cloud repository162(block335).

If the donor TDev is not defined (a determination of NO at block325), the storage system uses the process shown inFIGS. 4 and 5to implement the read operations on the donor TDev.

FIG. 4is a functional block diagram illustrating operation of the storage system100, when the donor TDev168that is linked to the snapshot164is undefined, according to some embodiments.FIG. 5is a flow chart of a method of implanting the read operations shown inFIG. 4when the donor TDev168is undefined, according to some embodiments.

As shown inFIGS. 4 and 5, if the donor TDev is undefined, a read IO operation on a track of the donor TDev168, issued by the cloud tethering subsystem154(FIG. 4, arrow1;FIG. 5, block500), will not contain metadata identifying the location of the requested snapshot data. Accordingly, when the front-end adapter126receives the read IO, the front-end adapter126sends a read miss to the data services subsystem158(FIG. 4, arrow2;FIG. 5, block505). The data services subsystem158sends the request to the snapshot subsystem156to ask the snapshot subsystem156to resolve the data address of the snapshot data (FIG. 4, arrow3;FIG. 5, block510). In response, the snapshot subsystem156does a one time on-demand definition of the requested donor TDev track (FIG. 5, block515) to identify the location within managed drives132where the requested data is stored.

The snapshot subsystem156sends the data track number of one of the managed drives132where the data resides to the data services subsystem158(FIG. 4, arrow4;FIG. 5, block520). The data services subsystem158allocates a slot in cache138(FIG. 5, block525), and the data services subsystem158fills the allocated cache slot with the data from the address provided at the previous step (FIG. 5, block530). The data services subsystem158then responds to the front-end adapter126with the identity of the cache slot holding the data (FIG. 4, arrow5;FIG. 5, block535). The process then returns toFIG. 3, where the front-end adapter completes the read request from the cloud tethering subsystem154by sending out the data from the cache slot address to the cloud repository162(FIG. 3, block335).

FIG. 6is a flow chart of a method of updating the storage system operational statistics to increase the accuracy of predicting future IO operations in connection with read operations associated with movement of a snapshot to a cloud repository, according to some embodiments. As shown inFIG. 6, in some embodiments the cache manager152uses machine learning algorithms to collect read IO metrics and write IO metrics, and uses these statistics to predict future IO operations on the storage system. These algorithms attempt to keep data in cache that is likely to be requested again, and to pre-load data that is likely to be requested in the future, to increase the percentage of read operations that can be fulfilled from the cache138. Increasing the percentage of read operations that can be fulfilled from the cache138(cache hit ratio), reduces the latency of the storage system100, to thereby improve overall performance of the storage system.

In some embodiments, when a read operation occurs on the cache, and is reported by the front-end adapter126to the cache manager152(block600), the cache manager152checks to determine if the read operation was from a tagged donor TDev168(block605). In some embodiments, this determination is implemented by checking for the presence of the donor tag on the thin device associated with the read operation. If the thin device does not have a donor tag (a determination of NO at block605) the read operation was not associated with movement of data to the cloud by the cloud tethering subsystem154. Accordingly, the cache manager152uses the read IO operation to update its read IO operation metrics. Likewise, if the thin device is not tagged as a donor TDev, the compression subsystem160treats the read IO operation in a normal manner to selectively compress or not compress the data in a normal manner.

If the read operation was from a thin device that was tagged using the donor tag as a donor TDev (a determination of YES at block605), the cache slot associated with the read operation is returned to the pool of available cache slots (block615). This immediately makes the cache slot available for use with other IO operations on the cache138to increase the number of times a given cache slot can be reused during an operational period. Increasing the reuse frequency of cache slots makes it easier for other operations to be implemented on the cache, thus reducing cache contention within the storage system100.

The read operation on the tagged donor TDev also does not result in changing the heuristics used by the cache manager152to determine placement of data in the cache (block620). By ignoring the read operations associated with movement of data by the cloud tethering subsystem154from the storage array130to the cloud repository162, it is possible to increase the accuracy of predicting future IO operations on the storage subsystem100by the cache manager152. Specifically, the read operations by the cache tethering subsystem may have exhibit high sequentiality, which could skew the prediction algorithms used by cache manager152toward implementing cache policies optimized for highly sequential data access patterns. By ignoring the read operations by the cloud tethering subsystem154, the cache manager152is able to base its cache management policies on the other traffic access patterns that are currently being experienced by the storage system100. This enables the cache manager152to more accurately adjust the cache management policies, to optimize placement of data in the cache and thereby increase the overall cache hit ratio.

Similarly, the activity-based compression system160checks to determine if a read operation was on a tagged donor TDev. If the read IO was on a tagged donor TDev, the compression system160does not store the data associated with the read IO operation in the uncompressed data pool that is reserved for recently read data. By not including the data from the donor TDev in the uncompressed data pool, additional space is available in the uncompressed data pool for other data that might potentially be requested in future IO requests.

FIG. 7is a flow chart of a method of updating primary storage system operational metrics in connection with data write operations associated with bringing snapshot data from the cloud repository162to the storage system100, according to some embodiments. Movement of a snapshot from the cloud repository162to the storage system100is typically done at the initiative of a user, because the user needs access to the data. Accordingly, when a snapshot is moved from the cloud repository162to the storage system100, it is likely that at least some of the data will be accessed at some point.

As shown inFIG. 7, when the cloud tethering subsystem154brings a snapshot from the cloud repository162to the storage system100, the cloud tethering subsystem154reads the requested data from the cloud and generates write IO requests to the front end adapter126on the designated thin device (LUN) that has been assigned to hold the snapshot (block700). The front-end adapter126allocates a cache slot in cache138and accepts the write IO from the cloud tethering subsystem154(block705). The data will subsequently be destaged to one or more of the managed drives132of storage array130.

After acknowledging the write IO, the front-end adapter126and the data services subsystem158coordinate to capture IO statistics on the designated LUN to build write heuristics to predict future IO activity on this device (block710). Example IO statistics that may be collected on the designated LUN may include IO age, IO timestamp, and the extent size of the IO, although other or additional statistics may be collected as well depending on the implementation.

Additionally, the cache manager152uses the write IOs from the cloud tethering subsystem154in connection with determining overall traffic patterns on the storage system100(block715). Accordingly, the write IOs from the cloud tethering subsystem are considered part of the overall IO activity of the storage system100, and are considered in connection with predicting future IO operations on the storage system.

Some embodiments have been described in connection with movement of snapshots of data by the cloud tethering subsystem to a cloud repository. Other instances where data is permanently being moved off the storage system, such as data migration, may also be implemented in a similar manner. Specifically, by using tagged donor TDevs and not using read operations on the tagged donor TDevs in connection with updating the intelligent algorithms used to implement cache management, it is possible to prevent movement of data off of the storage system from impacting the intelligent algorithms used to predict future IO operations on the storage system100. Likewise, by not including data that is being moved off the storage system in the uncompressed data pool, it is possible to maintain a greater portion of active data in an uncompressed form on the storage system.

The methods described herein may be implemented as software configured to be executed in control logic such as contained in a CPU (Central Processing Unit) or GPU (Graphics Processing Unit) of an electronic device such as a computer. In particular, the functions described herein may be implemented as sets of program instructions stored on a non-transitory tangible computer readable storage medium. The program instructions may be implemented utilizing programming techniques known to those of ordinary skill in the art. Program instructions may be stored in a computer readable memory within the computer or loaded onto the computer and executed on computer's microprocessor. However, it will be apparent to a skilled artisan that all logic described herein can be embodied using discrete components, integrated circuitry, programmable logic used in conjunction with a programmable logic device such as a FPGA (Field Programmable Gate Array) or microprocessor, or any other device including any combination thereof. Programmable logic can be fixed temporarily or permanently in a tangible computer readable medium such as random-access memory, a computer memory, a disk drive, or other storage medium. All such embodiments are intended to fall within the scope of the present invention.