Use of flash cache to improve tiered migration performance

For data processing in a computing storage environment by a processor device, the computing storage environment incorporating at least high-speed and lower-speed caches, and tiered levels of storage, and at a time in which at least one data segment is to be migrated from one level to another level of the tiered levels of storage, a data migration mechanism is initiated by copying data resident in the lower-speed cache corresponding to the at least one data segment to be migrated to a target on the another level, and reading remaining data, not previously copied from the lower-speed cache, from a source on the one level, and writing the remaining data to the target.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates generally to computers, and more particularly, to mechanisms for migrating data segments in a computing storage environment.

2. Description of the Related Art

In today's society, computer systems are commonplace. Computer systems may be In the field of computing, a “cache” typically refers to a small, fast memory or storage device used to store data or instructions that were accessed recently, are accessed frequently, or are likely to be accessed in the future. Reading from or writing to a cache is typically cheaper (in terms of access time and/or resource utilization) than accessing other memory or storage devices. Once data is stored in cache, it can be accessed in cache instead of re-fetching and/or re-computing the data, saving both time and resources.

SUMMARY OF THE DESCRIBED EMBODIMENTS

Caches may be provided as multi-level caches. For example, a caching system may include both a “primary” and “secondary” cache. When reading data, a computing system or device may first look for data in the primary cache and, if the data is absent, look for the data in the secondary cache. If the data is not in either cache, the computing system or device may retrieve the data from disk drives or other storage devices. When writing data, a computing system or device may write data to the primary cache. This data may eventually be destaged to the secondary cache or a storage device to make room in the primary cache.

Storage environments in computer systems may include so-called tiered storage architectures, which may include a variety of storage mediums such as as enterprise hard disk drives (HDD), serial advanced technology attachment (SATA) disk drives, solid state drives (SSD), tape drives, and other devices. Data may be migrated between such devices. Placement of certain kinds of data in the appropriate medium may greatly enhance overall performance of the storage environment.

In an existing tiered migration mechanism, data is read from a source location and destaged to a target. During this step, data can still be destaged to the source, which if occurs is recorded in a bitmap. In a subsequent step, any new destages are withheld to the source while tracks are read that were previously destaged to the source and destaged to the target. This mechanism results in additional time being expended while new destages are withheld, and while the previously destaged tracks are re-read and destaged from the source to the target. A need exists for a mechanism that effectively reduces this additional time, among other factors, to increase storage performance.

Accordingly, and in view of the foregoing, various exemplary method, system, and computer program product embodiments for data processing in a computing storage environment by a processor device, the computing storage environment incorporating at least high-speed and lower-speed caches, and tiered levels of storage, are provided. At a time in which at least one data segment is to be migrated from one level to another level of the tiered levels of storage, a data migration mechanism is initiated by copying data resident in the lower-speed cache corresponding to the at least one data segment to be migrated to a target on the another level, and reading remaining data, not previously copied from the lower-speed cache, from a source on the one level, and writing the remaining data to the target. In addition to the foregoing exemplary embodiment, various other system and computer program product embodiments are provided and supply related advantages.

DETAILED DESCRIPTION OF CERTAIN EMBODIMENTS

As previously mentioned, in an existing tiered migration mechanism, data is read from a source location and destaged to a target. During this step, data can still be destaged to the source, which if occurs is recorded in a bitmap. In a subsequent step, any new destages are withheld to the source while tracks are read that were previously destaged to the source and destaged to the target. This mechanism results in additional time being expended while new destages are withheld. Any new writes from the host to these data segments will need to wait for the destages to finish, which may take a significant amount of time, since it may take such time to copy data from a source to target location (e.g., while the previously destaged tracks are re-read and destaged from the source to the target). In addition, this current mechanism does not examine data in cache when migration operations are performed. Such mechanism starts at a beginning of a data segment (e.g., an extent) and proceeds to the end in order. If tracks are being maintained in cache, alternatively, these tracks may be copied first, before the tracks get aged out of cache.

In view of this scenario, the mechanisms of the present invention utilize primary and secondary cache architectures to allow for more direct involvement of such caches in the data migration process. As a result, for example, destages (i.e., resources in the computing storage environment) are not held while other storage operations (such as the aforementioned reads and writes of the data being migrated) occur. For example, destage conflicts which may occur during data segment migration operations are in progress may be significantly reduced. Accordingly, overall storage performance and efficiencies are enhanced as will be further described.

For the purposes of this disclosure, the phrase “secondary cache” is used to refer to any cache (including, for example, L2 or L3 cache) that resides between a primary cache and a storage device, such as a disk drive, tape drive, or the like.

Turning now toFIG. 1, a block diagram of a tiered computing storage environment100in accordance with certain embodiments is illustrated. The computing storage environment100includes a first computational device, such as, a storage system102, coupled to one or more computational devices, such as, clients104. In certain embodiments, the storage system102and the clients104may comprise any suitable computational device, including those presently known in the art, such as, a personal computer, a workstation, a mainframe, a midrange computer, a network appliance, a palm top computer, a telephony device, a blade computer, a hand held computer, etc.

In some embodiments, a storage manager106, such as, the Tivoli® Storage Manager® (TSM) product marketed by International Business Machines (IBM®) Corporation, may be used for securely storing and managing data segments according to aspects of the present invention. The storage manager106may execute in a storage management server, such as a TSM server102or elsewhere. In one embodiment, the storage manager is operable by and/or in conjunction with processor device105as shown. One of ordinary skill in the art will appreciate that various other configurations of the processor105, storage manager106, and related additional processing and/or memory components are contemplated. IBM, Tivoli, and Tivoli Storage Manager are trademarks or registered trademarks of IBM Corporation.

The TSM may provide data storage services to clients, such as TSM clients104a, for management of data. The TSM server102may store files sent to the TSM server102by one or more TSM clients104a,104b. The storage manager106and/or processor device105may allow a system administrator to configure storage pools, where a storage pool comprises a group of devices used for storing data received from the TSM clients104a,104b. Storage pools are used as targets for store operations from the TSM clients104a,104b, and are referenced in TSM server policies and other constructs for processing.

As shown, a variety of storage devices may be organized into a storage hierarchy. Storage media within the storage hierarchy may thus be grouped into data structures referred to herein as storage pools. The storage hierarchy may be organized to correspond with one or more metrics, such as a performance metric including write or read speeds. The storage hierarchy108as shown may be organized such that the top of the hierarchy may include a cache pool110having a highest amount or quality of a particular performance metric. Below the cache pool110, a number of solid state drive (SSD) class devices may be organized into SSD pools by the same, similar, or other metrics (e.g., SSD pools112and114).

Below the SSD pools112and114, a first tier of disk pools (e.g., disk pools116,118, and120) may be then organized. As one of ordinary skill in the art will appreciate, disk pools116,118, and120may include a variety of disk devices such as pools of enterprise disk drives, SATA disk drives, disk devices configured in a particular redundant array of independent disks (RAID) configuration, and the like.

The first tier of disk pools may be located above a second tier of disk pools (e.g., pools122,124, and126) by virtue of exhibiting a greater amount, stronger attribute or attributes, or quality of the performance metric. Below the second tier of disk pools, an additional tier of tape pools (e.g., tape pools128,130, and132) may then be organized. Various considerations for the organization of such storage hierarchies108may be apparent to one of ordinary skill in the art. In one embodiment, the system administrator may assist in performing such configurations in the storage hierarchy108by inputs to the TSM administrative client104bor another mechanism. While tape pools128,130, and132are shown within the storage hierarchy108as shown, it should be noted that generally such tape pools are found in a storage subsystem external to those pools ranking higher in the hierarchy.

Referring toFIG. 2, one embodiment of a storage system102containing an array of hard-disk drives204and/or solid-state drives204is illustrated. The internal components of the storage system102are shown since the caching system may, in certain embodiments, be implemented within such a storage system102, although the caching system may also be applicable to other storage systems102. As shown, the storage system102includes a storage controller200, one or more switches202, and one or more storage devices204such as hard disk drives204or solid-state drives204(such as flash-memory-based drives204). The storage controller200may enable one or more clients104(e.g., open system and/or mainframe servers104) to access data in the one or more storage devices204. The clients104(e.g.,FIG. 1) may be accessible through a Storage Area Network (SAN)220as shown.

In selected embodiments, the storage controller200includes one or more servers206. The storage controller200may also include host adapters208and device adapters210to connect the storage controller200to host devices106and storage devices203,204, respectively. Multiple servers206a,206bmay provide redundancy to ensure that data is always available to connected hosts106. Thus, when one server206afails, the other server206bmay remain functional to ensure that I/O is able to continue between the clients104and the storage devices204. This process may be referred to as a “failover.”

One example of a storage system102having an architecture similar to that illustrated inFIG. 2is the IBM® DS8000™ enterprise storage system. The DS8000™ is a high-performance, high-capacity storage controller providing disk storage that is designed to support continuous operations. The DS8000™ series models may use IBM's POWER5™ servers206a,206b, which may be integrated with IBM's virtualization engine technology. Nevertheless, the caching system disclosed herein is not limited to the IBM® DS8000™ enterprise storage system, but may be implemented in any comparable or analogous storage system110, regardless of the manufacturer, product name, or components or component names associated with the system110. Furthermore, any system that could benefit from one or more embodiments of the invention is deemed to fall within the scope of the invention. Thus, the IBM 4 DS8000™ is presented only by way of example and is not intended to be limiting.

In selected embodiments, each server206may include one or more processors212(e.g., n-way symmetric multiprocessors) and memory214. The memory214may include volatile memory (e.g., RAM) as well as non-volatile memory (e.g., ROM, EPROM, EEPROM, hard disks, flash memory, etc.). The volatile memory and non-volatile memory may, in certain embodiments, store software modules that run on the processor(s)212and are used to access data in the storage devices204. The servers206may host at least one instance of these software modules. These software modules may manage all read and write requests to logical volumes in the storage devices204.

In selected embodiments, the memory214may include a cache218. Whenever a client104(e.g., an open system or mainframe server104) performs a read operation, the server206that performs the read may fetch data from the storages devices204and save it in its cache218in the event it is required again. If the data is requested again by a client104, the server206may fetch the data from the cache218instead of fetching it from the storage devices204, saving both time and resources. Similarly, when a client104performs a write, the server106that receives the write request may store the write in its cache218, and destage the write to the storage devices204at a later time. When a write is stored in cache218, the write may also be stored in non-volatile storage (NVS)220of the opposite server206so that the write can be recovered by the opposite server206in the event the first server206fails.

Referring toFIG. 3, while continuing to refer generally toFIG. 2, as previously mentioned, a storage system102may include both hard disk drives204and solid-state drives (SSDs)204, such as flash-memory-based drives204. The I/O performance of SSDs204or other types of solid-state memory is typically far higher than the I/O performance of hard disk drives204. Because of the higher I/O performance, the solid-state drives204may, in certain embodiments, be used to provide a large secondary cache300between the primary cache218and the hard disk drives204. This large secondary cache300may significantly improve the I/O performance of the storage system102, and may be referred to herein as “Flash Cache.” Herein, the primary cache may be referred to as a high-speed or higher-speed cache (as it typically has access to the fastest Dynamic Read Only Memory or DRAM architectures), and the secondary, Flash Cache may be referred to as a low-speed or lower-speed cache (in comparison to the primary, DRAM cache), although this terminology is not intended to be limiting in any way.

Using the secondary cache300, if a read request is received by a server206(e.g.,FIG. 2), the server206may initially look for data in the primary cache218and, if the data is not present, look for the data in the secondary cache300(residing in the solid-state drives204). If the data is not available in either cache, the server206may retrieve the data from the disk drives204. Similarly, when writing data, a server206may initially write the modified data to the primary cache218. This modified data may eventually be destaged to the secondary cache300to make room in the primary cache218. This data may then be destaged to the disk drives204to make space in the secondary cache300, as needed.

In certain embodiments, the secondary cache300may be sized to provide about one to twenty percent, or in other embodiments about five percent of the total storage capacity of the storage system102. Thus, for a storage system102that contains about ten terabytes (TB) of storage (from both hard disk drives204and solid state drives204), about 0.5 TB of this storage space may be used as a secondary, “Flash” cache300. Such a large amount of secondary cache300may allow data to be destaged from the secondary cache300far less frequently than conventional primary or secondary caches. As an example, a very large secondary cache300could store writes for an entire day without having to destage the writes to the disk drives204. The writes could then be destaged at night or during a period of relative inactivity. Cache management algorithms may be redesigned to efficiently utilize the additional space in the secondary cache300and take advantage of the performance improvements that are possible using a large secondary cache300.

As shown inFIG. 3, each cache218,300may store data302a,302band metadata304a,304b. The data302a,302bmay be stored in the form of tracks. Each track in the secondary cache300may have a secondary track control block (STCB) associated therewith. The STCB may also be referred to herein as Cache Flash Control Block (CFCB). Along with other information, the STCB for each track may include a pointer to the next track in the chain, information indicating whether the track is free or in-use, as well as information indicating which sectors in the track have been modified. In certain embodiments, the STCBs for all the tracks may be stored in an STCB table306stored in the secondary cache300as shown, or elsewhere.

In addition, each track in the secondary cache300may have a secondary stride control block (SSCB) associated therewith. The SSCB, like the STCB may include diagnostic and/or statistical information, but instead relating to strides (groups of tracks) stored in the secondary cache300. The SSCB may also be referred to herein as Cache Flash Element (CFE). In certain embodiments, the SSCBs for all the strides may be stored in an SSCB table308stored in the secondary cache300as shown, or elsewhere.

Similarly, the primary cache218may also store metadata304aassociated with the secondary cache300. For example, the primary cache218may store a secondary cache index table (SCIT)308that provides a directory for tracks in the secondary cache300. In certain embodiments, the SCIT308is essentially a hash table with a constant hash function. To locate a specific track in the SCIT308, the hash function may convert a track identifier (e.g., a track number) to a hash value. This hash value may then be looked up in the SCIT308to find the STCB for the track. Alternatively, the SCIT308could be incorporated into a cache directory of the primary cache218, thereby providing a single hash table that stores tracks for both the primary and secondary caches218,300. In selected embodiments, the SCIT308is kept exclusively in the primary cache218. The SCIT308may be built or rebuilt (in the event of a failover, failback, or initial microcode load (IML)) by reading the STCB table306in the secondary cache300.

In certain embodiments, the primary cache218may also store a list of free tracks (LOFT)310that indicates which tracks in the secondary cache300are free (i.e., unoccupied). This list310may be used to locate free space in the secondary cache300in order to destage data from the primary cache218to the secondary cache300. In selected embodiments, inserting or removing tracks from the LOFT310may be performed in a log structured manner. For example, tracks may be inserted at the end of the LOFT310and deleted from the front of the LOFT310. The LOFT310may be kept exclusively in the primary cache218and may be built or rebuilt by reading the STCB table306in the secondary cache300.

The primary cache218may also store a sorted tree of tracks (STOT)312that sorts the tracks by “trackid” or some other indicator. The STOT312may be used to minimize seek time (on the disk drives204) when destaging tracks from the secondary cache300to the disk drives204. The STOT312may be kept exclusively in the primary cache218and may be built or rebuilt by reading the STCB table306in the secondary cache300.

The primary cache218may also store statistics per stride (STATS)314for each stride having one or more tracks in the secondary cache300. A “stride’ refers to a set of logically sequential data that might be segmented across multiple disks combined with additional parity information as is for example used in a RAID-5 (redundant array of inexpensive disks) configuration. In general, the STATS314may be used to determine which tracks require the least number of disk operations (“disk ops”) to destage from the secondary cache300to the disk drives204. In general, the destage penalty for a track will be less where more tracks are present in a stride. When selecting tracks to destage, tracks requiring the least number of disk ops may be destaged first to minimize resource utilization. In selected embodiments, the STATS314may store information such as the number of tracks that are present in the secondary cache300for each stride, and the number of disk ops required to destage a track in a stride. In certain embodiments, the STATS314may store a “recency” bit for each stride. The recency bit may be incremented each time an eviction process passes through a stride. The recency bit may be reset each time a track is added to a stride. The recency bit may be used to keep strides in the secondary cache300that are actively being written to. The STATS314may be kept exclusively in the primary cache218and may be built rebuilt by reading the STCB table306in the secondary cache300.

The metadata304a,304bdescribed above may be structured and stored in various different ways and is not limited to the illustrated structure or organization. The metadata304a,304bis provided by way of example to show one technique for storing and structuring the metadata304a,304b. For example, in certain embodiments, the data and metadata may be stored together in the secondary cache300in a circular log-structured array. Other methods for structuring and storing metadata304a,304bmay be used and are encompassed within the scope of the invention.

As previously mentioned, one advantage of a large secondary cache300is that data can be destaged from the secondary cache300far less frequently than conventional secondary caches. This may enable more data to accumulate in the secondary cache300before it is destaged to the disk drives204. Accordingly, in selected embodiments, an apparatus and method in accordance with the invention may be configured to wait for full strides of data to accumulate and coalesce in the secondary cache300before the data is destaged to the disk drives204. As explained above, this may minimize the number of disk ops required to destage data from the secondary cache300to the disk drives204, thereby improving overall system performance.

Referring toFIG. 4, a first exemplary method for migration of data segments using secondary cache (Flash Cache), in a computing environment having dual lower and higher speed levels of cache, and tiered levels of storage, is illustrated. In the illustrated embodiment, the secondary cache is represented as the lower speed level of cache, and the higher speed cache may be implemented in the storage controller as DRAM cache as in a previous exemplary illustration. Method400begins (step402) as a data movement mechanism is initiated. The data movement mechanism is adapted for, implementing Flash Cache (for example, in 1-128 blocks) to perform data copy/migration (for example in 10 MB and 1 GB segments). Flash Cache is implemented to perform data migration/copy functionality since the Flash Cache is physically and logically prioritized higher than other tiered levels of storage and can accommodate all Input/Output (I/O).

To avoid destage conflicts that may result in a DRAM demote to Flash Cache if a data segment (e.g., stride) is being destaged in progress, an out of order migration operation (similar to a RAID rebuild) for Read Miss, and Write (Flash Cache) may be incorporated. Accordingly, in step404, data resident in the lower-speed (Flash Cache) is copied to the target, and in step406, the remaining data, not previously copied, is read from the source and copied to the target (step408). The method400then ends (step408).

The data migration mechanism implemented using Flash Cache may be further adapted for implementing a preference of data movement of the partial data segments to the lower speed cache (again, e.g., Flash Cache) based on several metrics. Two possible such metrics are the amount of holes and data “hotness,” or a data heat metric. One objective of the preference of data movement previously described is to free up more space in the higher speed cache with a subsequent destage operation (more data being destaged) and coalesce into a single write to the secondary, lower speed cache.

FIG. 5, following, illustrates an exemplary method500of operation of the data movement mechanism as previously implemented inFIG. 4. Method500begins (step502) with a determination of a need to migrate a data segment (e.g., extent) by the tiered storage (step504). In a first pass, the tiered storage passes through the lower-speed cache and copies data resident in the cache to the target extent (step506). The data that has been copied is marked (for example with a bit or recorded in a bitmap) (step508).

In a subsequent step, a second pass is completed. In this pass, the tiered storage system copies that data which has not been copied in the first pass (step510). The source file is read and then the target file is written to. In this pass, if it is determined that data must be destaged from DRAM cache (higher-speed) to make space for new writes (step512), then the data segments are destaged to Flash Cache (lower-speed) (step514). This significantly reduces the destage conflicts since destages are not held off for the entire pass.

Once the second pass is complete, the logical extent is switched to point to the new physical extent (or whichever data segment is undergoing migration) (step516), and the method500ends (step518).

In some cases, for example if the Flash Cache is currently full of modified data, or otherwise inaccessible, it may not be possible at a particular point in time to destage to Flash Cache. In this case, the tiered storage system may switch to an alternative method of destaging in which Flash Cache is not implemented (and destages are held, for example, until data is migrated).

While one or more embodiments of the present invention have been illustrated in detail, one of ordinary skill in the art will appreciate that modifications and adaptations to those embodiments may be made without departing from the scope of the present invention as set forth in the following claims.