Multi-tier file storage management using file access and cache profile information

In one embodiment, a method for managing data includes determining that a cache access count for a given data block is greater than an average cache access count, receiving a list of active applications accessing the given data block with an anticipated access count for each active application, receiving a list of applications that are anticipated to access the given data block within a time window with an anticipated future access count for each anticipated application, determining that a block application access weight is greater than a block application access threshold, determining that a cache profile weight for the given data block is greater than zero, and sending the cache profile weight to a file system. Other systems, methods, and computer program products are described according to more embodiments.

BACKGROUND

The present invention relates to management of a multi-tier storage environment, and more specifically, this invention relates to efficient management of high performance tiers in a multi-tier storage environment.

A file system defines how files are named and manages how they are placed for storage and retrieval. File system functionality may be divided into two components: a user component and a storage component. The user component is responsible for managing files within directories, file path traversals, and user access to files. The storage component of the file system determines how files are stored physically on the storage device.

File blocks are mapped to logical blocks, which are then mapped onto actual physical blocks on storage media. A logical to physical mapping layer is used to make file management independent of storage management. A file system102is shown inFIG. 1, where File1has two file blocks112: FBlock0(FB0) and Fblock1(FB1). FB0and FBI for File1are mapped to two logical blocks110: LBlock0and LBlock1. LBlock0and LBlock1are mapped to actual physical blocks108(Block0and Block10) on the storage medium104. For File2, file blocks112FB0, FB1, and FB2are mapped to LBlock2, LBlock3, and LBlock4, which are mapped to actual physical blocks108(Block30, Block50, and Block60) on the storage medium104. Since storage medium104(such as hard disk drive (HDD), magnetic tape, etc.) accesses are slower, data blocks are stored to the in-memory cache106for quicker access. On a first read operation, data is copied from the storage medium104to the in-memory cache106, in an action referred to as a “Cache Miss.” Subsequent accesses on the block are performed from the in-memory cache106once the desired data is stored therein. Blocks from the in-memory cache106are written to the storage medium104in either of two scenarios: 1) in-memory cache106space is limited, so when new blocks are to be stored to the in-memory cache106, old blocks are evicted from the in-memory cache106and stored on the storage medium104in an action referred to as a “Cache Eviction:” and 2) when an application explicitly commands the in-memory cache106to flush data to the storage medium104.

Multi-tiered storage is a storage method where data is stored on various types of storage devices primarily based on criteria of the access, frequency of use, security, and/or data recovery requirements. For example, data that is frequently accessed by an application that is response time sensitive might be stored on a solid state drive (SSD). Other data that is infrequently accessed and for which a higher response time is more tolerable might be stored on high capacity 7200 RPM HDDs. The cost per Gigabyte of storage is much higher for SSDs than it is for the 7200 RPM HDDs. One challenge in effectively using multi-tiered storage is identifying the data that benefits from the higher cost/higher performance storage tiers. Over time, the optimal tier for a given piece of data may change; thus, the identification and movement of data to an appropriate tier is an ongoing and evolving process.

Since SSDs are costlier than HDDs, preferred solutions allow for dynamic relocation of data across tiers based on the data usage by placing “hot” data with high I/O density and low response time requirements on SSDs while targeting HDDs or other slower-responding data storage devices for “cooler” data that is accessed more sequentially and/or at lower rates.

BRIEF SUMMARY

In one embodiment, a method for managing data includes determining that a cache access count for a given data block is greater than an average cache access count for a plurality of data blocks, receiving a list of active applications accessing the given data block with an anticipated access count for each active application contingent upon the cache access count being greater than the average cache access count, receiving a list of applications that are anticipated to access the given data block within a time window with an anticipated future access count for each anticipated application contingent upon the cache access count being greater than the average cache access count, determining that a block application access weight is greater than a block application access threshold contingent upon the cache access count being greater than the average cache access count, determining that a cache profile weight for the given data block is greater than zero contingent upon the block application access weight being greater than the block application access threshold, and sending the cache profile weight to a file system contingent upon the cache profile weight being greater than zero.

In another embodiment, a system includes a processor and logic integrated with and/or executable by the processor, the logic being configured to determine a cache profile weight for a given data block upon receiving a request to evict the given data block from cache, and determine a storage tier to store the given data block based on at least one of: an extent cache profile weight of an extent including the given data block, and a heat count for the given data block.

In yet another embodiment, a computer program product for managing data includes a computer readable storage medium having program code embodied therewith, the program code being readable and/or executable by a processor to cause the processor to determine, by the processor, that a cache access count for a given data block is greater than an average cache access count for a plurality of data blocks, receive, by the processor, a list of active applications accessing the given data block with an anticipated access count for each active application contingent upon the cache access count being greater than the average cache access count, receive, by the processor, a list of applications that are anticipated to access the given data block within a time window with an anticipated future access count for each anticipated application contingent upon the cache access count being greater than the average cache access count, determine, by the processor, that a block application access weight is greater than a block application access threshold contingent upon the cache access count being greater than the average cache access count, determine, by the processor, that a cache profile weight for the given data block is greater than zero contingent upon the block application access weight being greater than the block application access threshold, and send, by the processor, the cache profile weight to a file system contingent upon the cache profile weight being greater than zero.

DETAILED DESCRIPTION

The following description discloses several preferred embodiments of systems, methods, and computer program products for efficient management of higher tiers in a multi-tiered file system, particularly with regard to identifying and managing high access files.

In one general embodiment, a method for managing data includes determining that a cache access count for a given data block is greater than an average cache access count for a plurality of data blocks, receiving a list of active applications accessing the given data block with an anticipated access count for each active application contingent upon the cache access count being greater than the average cache access count, receiving a list of applications that are anticipated to access the given data block within a time window with an anticipated future access count for each anticipated application contingent upon the cache access count being greater than the average cache access count, determining that a block application access weight is greater than a block application access threshold contingent upon the cache access count being greater than the average cache access count, determining that a cache profile weight for the given data block is greater than zero contingent upon the block application access weight being greater than the block application access threshold, and sending the cache profile weight to a file system contingent upon the cache profile weight being greater than zero.

In another general embodiment, a system includes a processor and logic integrated with and/or executable by the processor, the logic being configured to determine a cache profile weight for a given data block upon receiving a request to evict the given data block from cache, and determine a storage tier to store the given data block based on at least one of: an extent cache profile weight of an extent including the given data block, and a heat count for the given data block.

In yet another general embodiment, a computer program product for managing data includes a computer readable storage medium having program code embodied therewith, the program code being readable and/or executable by a processor to cause the processor to determine, by the processor, that a cache access count for a given data block is greater than an average cache access count for a plurality of data blocks, receive, by the processor, a list of active applications accessing the given data block with an anticipated access count for each active application contingent upon the cache access count being greater than the average cache access count, receive, by the processor, a list of applications that are anticipated to access the given data block within a time window with an anticipated future access count for each anticipated application contingent upon the cache access count being greater than the average cache access count, determine, by the processor, that a block application access weight is greater than a block application access threshold contingent upon the cache access count being greater than the average cache access count, determine, by the processor, that a cache profile weight for the given data block is greater than zero contingent upon the block application access weight being greater than the block application access threshold, and send, by the processor, the cache profile weight to a file system contingent upon the cache profile weight being greater than zero.

FIG. 2illustrates an architecture200, in accordance with one embodiment. As shown inFIG. 2, a plurality of remote networks202are provided including a first remote network204and a second remote network206. A gateway201may be coupled between the remote networks202and a proximate network208. In the context of the present architecture200, the networks204,206may each take any form including, but not limited to a LAN, a WAN such as the Internet, public switched telephone network (PSTN), internal telephone network, etc.

In use, the gateway201serves as an entrance point from the remote networks202to the proximate network208. As such, the gateway201may function as a router, which is capable of directing a given packet of data that arrives at the gateway201, and a switch, which furnishes the actual path in and out of the gateway201for a given packet.

Further included is at least one data server214coupled to the proximate network208, and which is accessible from the remote networks202via the gateway201. It should be noted that the data server(s)214may include any type of computing device/groupware. Coupled to each data server214is a plurality of user devices216. Such user devices216may include a desktop computer, lap-top computer, hand-held computer, printer or any other type of logic. It should be noted that a user device211may also be directly coupled to any of the networks, in one embodiment.

A peripheral220or series of peripherals220, e.g., facsimile machines, printers, networked and/or local storage units or systems, etc., may be coupled to one or more of the networks204,206,208. It should be noted that databases and/or additional components may be utilized with, or integrated into, any type of network element coupled to the networks204,206,208. In the context of the present description, a network element may refer to any component of a network.

FIG. 3shows a representative hardware environment associated with a user device216and/or server214ofFIG. 2, in accordance with one embodiment. Such figure illustrates a typical hardware configuration of a workstation having a central processing unit310, such as a microprocessor, and a number of other units interconnected via a system bus312.

The workstation shown inFIG. 3includes a Random Access Memory (RAM)314, Read Only Memory (ROM)316, an I/O adapter318for connecting peripheral devices such as disk storage units320to the bus312, a user interface adapter322for connecting a keyboard324, a mouse326, a speaker328, a microphone332, and/or other user interface devices such as a touch screen and a digital camera (not shown) to the bus312, communication adapter334for connecting the workstation to a communication network335(e.g., a data processing network) and a display adapter336for connecting the bus312to a display device338.

As shown inFIG. 4, a multi-tier monitoring module402, which is configured to monitor for I/Os performed on storage media (such as hard disk drives (HDDs), optical disk drives, magnetic tape, etc.), may be provided with a multi-tier storage system400. This multi-tier storage system400may be based an IBM Easy-Tier architecture or some other multi-tier system known in the art. The multi-tier monitoring module402may receive data and/or instructions and commands from a host410connected thereto. Based on data access frequency, the multi-tier monitoring module402is configured to identify data that is heavily accessed (“hot”), possibly relying on a number of access counts for the data being above a predetermined threshold.

A data relocator404is configured to store this hot data on a higher tier406, which may have SSDs or some other suitable storage media, for high throughput, low response times, and I/O operations per second (IOPS)-energy-efficient characteristics. As a result, the amount of expensive storage required to meet a given level of performance is minimized compared to other methods of data placement for tiered storage. This hot data may be moved from a lower tier408, and after the data is no longer considered to be hot, it may be moved back to the lower tier408, in various approaches.

One of the problems with multi-tiered data storage is described with regard toFIG. 5. A situation is shown inFIG. 5where a host502(which may comprise an application), similar to a storage system, may comprise a host cache504. For a first access of a particular block of data, when the data block is not stored in the host cache504, a “Cache Miss” is reported which triggers a block read operation to be issued to the storage system via the multi-tier monitoring module506. After this first block read from the storage system outside of the host502, subsequent accesses to the same data block are resolved using the host cache504until the data block is evicted from the host cache504.

A block write operation may be triggered when the data block is modified, according to a host cache eviction policy, during block synchronization, and/or in response to or during some other event known in the art. In the case of a block write operation being triggered on the storage system, any data block read/write outside of the host502does not actually map directly to the actual application read/write; instead, they are reported as a “Cache Miss” and cache evictions or block syncs.

As shown inFIG. 5, the storage system has performed six accesses on Block A. The first access caused a “Cache Miss” for a Block A read, the next four accesses were performed using the host cache504. Then, assuming that Block A is modified, based on an eviction policy or block synchronization requirement, Block A is moved from the host cache504. This is seen as a Block A write to the storage system.

Therefore, for six application accesses of Block A, the multi-tier monitoring module506will receive either two accesses (a block read and a block write when Block A has been modified by the application) or one access (for the first block read prior to the block being stored to the host cache504). Thus, there is mismatch that may affect how high access frequency data is determined.

Now referring toFIG. 6, a storage system600is shown according to one embodiment. Note that some of the elements shown inFIG. 6may be implemented as hardware and/or software, according to various embodiments. The storage system600may include a storage system manager612for communicating with a plurality of media on a higher storage tier602and a lower storage tier606. The higher storage tier602preferably may include one or more random access and/or direct access media604, such as hard disks in hard disk drives (HDDs), nonvolatile memory (NVM), solid state memory in solid state drives (SSDs), etc., and/or others noted herein. The lower storage tier606may preferably include one or more sequential access media608, such as magnetic tape in tape drives, optical media, etc., and/or others noted herein. Additional storage tiers616may include any combination of storage memory media. The storage system manager612may communicate with the storage media604,608on the higher and lower storage tiers602,606through a network610, such as a storage area network (SAN), as shown inFIG. 6. The storage system manager612may also communicate with one or more host systems (not shown) through a host interface614, which may or may not be a part of the storage system manager612. The storage system manager612and/or any other component of the storage system600may be implemented in hardware and/or software, and may make use of a processor (not shown) for executing commands of a type known in the art, such as a central processing unit (CPU), a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), etc. Of course, any arrangement of a storage system may be used, as will be apparent to those of skill in the art upon reading the present description.

According to some embodiments, the storage system (such as600) may include logic adapted to receive a request to open a data set, logic adapted to determine if the requested data set is stored to a lower storage tier606of a tiered data storage system600in multiple associated portions, logic adapted to move each associated portion of the requested data set to a higher storage tier602of the tiered data storage system600, and logic adapted to assemble the requested data set on the higher storage tier602of the tiered data storage system600from the associated portions.

Now referring toFIG. 7, one problem with traditional methods of determining the access frequency of data in a file system is shown, according to one example. In order to make the determination that a given data block is “hot,” the number of times that the data block is accessed on the I/O line708is tracked. However, from a point of view of the multi-tier monitoring module706, this traffic is due to Cache Misses (Reads) and Cache Evictions (Writes), and not every data block access is visible. Therefore, observed traffic may not be directly mapped to application access count (the number of times that the data block is accessed by the application). This leads to a mismatch between which data blocks the application considers “hot” and which data blocks the multi-tier monitoring module706considers “hot,” which ultimately impacts multi-tier placement policy and decisions.

It is noted that hot blocks that are stored on the host702within the host cache704should not also be placed on the higher tier of the data storage system, as this will lead to unnecessary wastage of higher tier space. Instead, it is pointed out that as long as a data block is in host cache704, there is no need to move it to a higher storage tier. Furthermore, when the data is evicted from the host cache704, the cache hit profile (which includes the number of accesses while the data block was in the host cache) for that data block should be considered in determining whether the data block is “hot” or “cold” for placement in one of the tiers of the multi-tier storage system. If this cache hit profile is not considered, it results in higher latencies for the “hot” data blocks during access on one of the storage tiers.

As an example, as shown inFIG. 7, Block A is considered “hot” by the application with ten accesses; however, there are only two Cache Miss/Evictions for Block A which are seen by the multi-tier monitoring module706since most of the accesses are performed within the host cache704. Meanwhile, Block B is considered “cold” by the application with only three accesses, but as part of LRU cache policy or some other scheme or policy, Block B was evicted many times which caused more Cache Misses, resulting in five accesses visible to the multi-tier monitoring module706. Thus, the multi-tier monitoring module706might consider Block B as the “hot” block compared to Block A, which may be considered “cold.” During placement on a higher storage tier, Block B will be preferred and placed in a higher storage tier when compared to Block A. However, this conclusion is lacking the visibility of how many accesses occurred on the host cache704, which is referred to as host access count, which leads to a wrong interpretation of application hot data and access latencies when hot application blocks are evicted from the host cache704.

Thus, application hot Block A, when evicted from the host cache704, might be placed directly to a lower storage tier. Missing this consideration results in higher latencies for the really hot application blocks during storage tier accesses, e.g., a file system might be syncing a cache copy for this block but the sync operation might take a lot of time as the data block is not on the higher storage tier and accesses will be performed only on the higher storage tier, which means the data block must be promoted prior to access.

In order to provide efficient management of high performance tiers in a multi-tier architecture, data block access count monitoring, referred to as a “heatmap information,” may be influenced by introducing information based on a cache access profile for each data block such that during disk I/O operation, when the cache hit count of a given block is significantly more than an average cache hit count, the file system may determine whether this given data block would benefit from high performance accesses for disk I/Os, and assign CacheProfileWeight to the given data block. The multi-tier monitoring module may consider CacheProfileWeight to influence heatmap information maintained for the given data block.

The file system may assign CacheProfileWeight to the given data block based on any of the following conditions: 1) average cache hits per data block at the file system level, 2) state of the application accesses, and 3) purpose of the access request (e.g., disk I/O).

The average cache hits per data block at the file system level may be based on a number of cache hits on this data block compared with an average value for all data blocks. The state of the application accesses may be based on how many applications are currently accessing the given data block and an access frequency for each application, including a number of applications that have the possibility of accessing the data block in the near future (next 10 minutes, next 5 minutes, next minute, next 30 seconds, next 10 seconds, next 5 seconds, next 1 second, etc., depending on the granularity of information provided on access frequency) and access frequency. The purpose of the access request may be for several different reasons. One reason is for storage medium synchronization, where CacheProfileWeight value will be higher as I/Os are needed to be performed immediately. When the purpose of the access request is cache eviction due to LRU or some other policy, CacheProfileWeight will be less than in the case of disk sync.

At the multi-tier monitoring module, CacheProfileWeight will be considered apart from monitored access counts (in the heatmap) to conclude whether a given block is hot and eligible for placement on a higher performance storage tier, in this embodiment. In implementation, existing file system caches maintain in-memory data structures for each data block that is present in the in-memory cache. It includes fields, such as last access time and access counts, which are used by the cache eviction policy mechanism. This mechanism may be extended to have overall cache hit count and average cache hit count values as well.

During an access request operation (e.g., a disk I/O), when the cache hit count of the given block is significantly more than the average cache hit count for all data blocks, the file system may determine whether this data block would benefit from high performance accesses for accesses based on any of: cache hit counts, live and future applications possibly requesting this block, and respective access frequency and purpose of eviction. The CacheProfileWeight value is determined for the given data block. The CacheProfileWeight value is passed to the multi-tier monitoring module, where the heatmap table stored therein is updated to consider this value. A multi-tier relocation module then considers this CacheProfileWeight value as another criteria apart from observed disk accesses during heatmap calculation of the given data block and placement on higher performance storage tiers.

When a non-dirty block is evicted, there is no access request (e.g., disk I/O) involved, but if that data block has high CacheProfileWeight and the application(s) might use it in the near future, a special cache profile command (e.g., CACHE_PROFILE_CMD) may be used to communicate the CacheProfileWeight value to the host (appliance).

For communication between the file system and the multi-tier appliance, a separate out-of-band protocol may be used, or reserved fields in the I/O command descriptor block (CDB) (e.g., SCSI CDB) may be used. This is to ensure that in remote copy scenarios, when a fail over occurs, the right data is desired to be in the higher performance storage tier. This mechanism may be used for communication of percentage heat transfer value from the host (application) to the multi-tier monitoring module in the multi-tier file system.

Now referring toFIG. 8, a flowchart of a method800for managing data is shown according to one embodiment. Method800may be executed in any desired environment, including those shown inFIGS. 1-7, among others. Furthermore, more or less operations than those specifically described inFIG. 8may be included in method800.

In operation802, it is determined whether a cache access count (CacheAcc), or cache hit count, for a given data block is greater than an average cache access count (AveCacheAcc). This determination may be made during, directly after, immediately following, close in proximity to, and/or triggered by a request for or actual cache block eviction and/or a request for and/or an actual medium synchronization (e.g., disk sync, etc.) for the given data block.

In one embodiment, the AveCacheAcc may be multiplied by an access count threshold (AccThreshold) which may be adjusted to account for conditions of the system, as would be understood by one of skill in the art. By default, the AccThreshold may be set to 1, but any positive value may be used to tweak the desired behavior of the system, such as 1.25, 1.5, 2.0, 0.8, 0.75, 0.6, 0.5, etc., or more or less.

When the CacheAcc is not greater than the AveCacheAcc (with and without the AccThreshold used in the calculation according to alternate embodiments), method800continues to operation822where the given data block is processed normally (e.g., without using a calculated CacheProfileWeight in the placement decision).

In operation804, a list of active applications that are currently accessing the given data block is received. This list may also include an anticipated access count for each application for the given data block. This anticipated access count may be based on any desired factor, and may rely on historical and/or learned information about the applications, the given data block, etc.

In operation806, a list of applications that are anticipated to access the given data block within a time window is received. This time window may include the near future, a predetermined amount of time in which the anticipated given data block accesses would be affected by the placement of the given data block on lower or higher storage tiers within the file system, as would be understood by one of skill in the art.

This list may also include an anticipated future access count for each anticipated application for the given data block. This anticipated future access count may be based on any desired factor, and may rely on historical and/or learned information about the applications, the given data block, etc.

In operation808, a block application access weight (BlockAppAccWeight) is determined for the given data block. Any desired formula may be used to calculate the BlockAppAccWeight known in the art. The BlockAppAccWeight may be calculated using Formula 1, according to one embodiment.

Changes, manipulations, and substitutions may be made to this formula, as would be understood by one of skill in the art, without violating the inherent purpose of the calculation. When SCA (sum of current accesses) is the sum of the anticipated access counts for current active application(s) in the near future for the given block, SFA (sum of future accesses) is the sum of anticipated access count(s) for the applications that might start in the near future for the given block, then Formula 1 may be as shown below.
BlockAppAccWeight=A*SCA+B*SFA  Formula 1

In Formula 1, A and B are system defined parameters, which may range in value from 0.1 to 10, or more or less, depending on a desired effect on the system's performance. Either or both of A and B may be defaulted to 1, in one approach.

In operation810, it is determined whether the BlockAppAccWeight is greater than an application access threshold (AppAccThreshold). The AppAccThreshold may be a system variable which may be set to cause certain system performance, and may be set to any positive value, such as 1, 10, 100, 200, 500, 1000, 10,000, 50,000, 100,000, etc.

When BlockAppAccWeight is greater than AppAccThreshold, method800continues to operation812; otherwise, method800returns to operation822to perform normal processing of the given data block.

In operation812, a variable referred to as “Purpose” is initialized and set to zero when the access and/or access request for the given data block is performed for a cache eviction; otherwise, Purpose is set to 1 when the access and/or access request for the given data block is performed for medium synchronization (e.g., disk sync, etc.). Furthermore, in operation812, the CacheProfileWeight value for the given data block is calculated.

Any suitable formula may be used to calculate the CacheProfileWeight value, and in one embodiment, Formula 2, described later, may be used to calculate the CacheProfileWeight value.

In order to calculate the CacheProfileWeight value, Formula 2 shown below may be used, in one approach. Changes, manipulations, and substitutions may be made to this formula, as would be understood by one of skill in the art, without violating the inherent purpose of the calculation. When AveCacheAcc is the average cache access count at the file system in-memory cache before the given data block is evicted, CacheAcc is the cache access count for the given data block, BlockAppAccWeight is the application block weight value determined from the previous formula, purpose is 0 for eviction and 1 for file sync, LastAccDur is a difference between a last access time when the given data block is accessed and a current time, which is set when the purpose is eviction (i.e., purpose=0), and DataSyncFreq is a frequency of storage medium data synchronization for the given data block, which is set when the purpose is medium synchronization (i.e., purpose=1).

There are two conditions that, in one embodiment, would cause the CacheProfileWeight value to be reported as 0. These two conditions are when AveCacheAcc*AccThreshold>CacheAcc, and when AppAccWeight>AppAccThreshold. Otherwise, the CacheProfileWeight may be calculated as:
CacheProfileWeight=X*CacheAcc/AveCacheAcc+Y*AppAccWeight−(Z*(1−Purpose)*LastAccDur)+(W*Purpose*DataSyncFreq)  Formula 2

In Formula 2, X, Y, Z and W are system defined parameters, which may range in value from 0.1 to 10, or more or less, depending on a desired effect on the system's performance. Any of X, Y, Z, and/or W may be defaulted to 1, in one approach.

Then, in operation814, it is determined whether CacheProfileWeight is greater than zero. When CacheProfileWeight >0, method800continues to operation816; otherwise, method800returns to operation822to perform normal processing of the given data block.

In operation816, it is determined whether the Purpose is for cache eviction (e.g., equal to 0) or for medium synchronization (e.g., equal to 1). When the Purpose is for cache eviction and the data block is not dirty (e.g., is in an invalid, valid, or reserved state, but is not the only copy of the data in an incoherent state which needs to be updated), method800continues to operation818; otherwise, method800continues to operation820.

In operation818, the CacheProfileWeight is sent to the file system (e.g., via the data relocator module, the multi-tier monitoring module, etc.) in a cache profile command (e.g., a CACHE_PROFILE_CMD that includes the CacheProfileWeight).

In operation820, the CacheProfileWeight is sent to the file system (e.g., via the data relocator module, the multi-tier monitoring module, etc.) using one or more reserved bits in an I/O CDB, such as a SCSI CDB, which is explained in more detail herein.

In this embodiment, the file system CacheProfileWeight calculation may be performed during cache eviction or medium synchronization and reported using the following exemplary SCSI Read/Write CDB, as would be understood by one of skill in the art. In this SCSI CDB, the top row indicates the bit position, while the first column indicates the byte location.

Bit2from Byte 1 and Bits5and6from Byte 14 are reserved (Res.). These three bits may be used to transfer the CacheProfileWeight value in this I/O (e.g., SCSI) CDB, according to one embodiment. Because three bits are available, this provides for 8 values in a range from 0-7 for the CacheProfileWeight value. In one embodiment, 0 indicates a small weight value, while 7 indicates a maximum weight value, with the range of numbers therebetween indicating varying other degrees of weight to be assigned to the data block in consideration for storage location and access frequency.

In an alternate embodiment, 7 is small while 0 is large. For less granularity, only two of the reserved bits may be used, resulting in four possible values, 0-3, which may distributed and used in the same fashion.

After both of operations818and820, the given data block is processed using the CacheProfileWeight to determine storage tier placement, when appropriate, such as by using the data relocator module and/or the multi-tier monitoring module, in various approaches.

In one embodiment, a heatmap monitoring table may be maintained on the multi-tier monitoring module. One exemplary heatmap monitoring table is shown in Table 1, below. The multi-tier monitoring module increments the access count values when any access to blocks in the given range is accessed and/or request to one of these blocks is received. In a further embodiment, entries in this heatmap monitoring table may include a column that stores ExtentCacheProfileWeight values. The ExtentCacheProfileWeight value is a sum of the CacheProfileWeight for all blocks in the given extent, as per the inputs provided by the file system. The data relocator module may refer to these entries in the heatmap monitoring when deciding whether a given block and/or a given extent is hot and should be placed on the higher storage tier in various embodiments.

To handle the CacheProfileWeight value at the data relocator module, the following steps may be taken. For any medium I/O command on the given data block, the data relocator module may determine whether an associated SCSI CDB has a CacheProfileWeight therein, possibly stored to reserved fields thereof, in one embodiment. When the CDB has a CacheProfileWeight, the data relocator module may refer to the heatmap monitoring table in order to determine an extent number for the given data block (the extent which contains the given data block). Next, the ExtentCacheProfileWeight reported in the column associated with the given data block's extent is added to the CacheProfileWeight value included in the CDB.

In another embodiment, when a CACHE_PROFILE_CMD is received, the data relocator may also utilize the heatmap monitoring table in order to determine the extent number for the given data block (the extent which contains the given data block). Next, the ExtentCacheProfileWeight reported in the column associated with the given data block's extent is added to the CacheProfileWeight value included in the CACHE_PROFILE_CMD.

Referring now toFIG. 9, a method900for determining a location for one or more extents is shown according to another embodiment. Method900may be executed in any desired environment, including those shown inFIGS. 1-7, among others. Furthermore, more or less operations than those specifically described inFIG. 9may be included in method900.

In operation902, a heat count (HeatCount) for a given extent in a plurality of extents is determined. Once the HeatCount is determined, it is added to a heatmap list. In one embodiment, it may be included in the form of a tuple <extent #, HeatCount>. Of course, any format for reporting the extent and HeatCount may be used, as would be known to one of skill in the art.

In one embodiment, in order to determine the HeatCount for an extent, Formula 3 may be used. Of course, any desired formula may be used, as would be understood by one of skill in the art, in various other approaches. In Formula 3, AccessCount is a value from the heatmap monitoring table which reflects a number of data block accesses for a given extent, CacheProfileWeight is a value from the heatmap monitoring table which reflects the CacheProfileWeight for a given extent, which may be calculated according to Formula 2, above.
HeatCount=P*AccessCount+Q*CacheProfileWeight  Formula 3

In Formula 3, P and Q are system defined parameters, which may range in value from 0.1 to 10, or more or less, depending on a desired effect on the system's performance. Any of P and/or Q may be defaulted to 1, in one approach.

In operation904, it is determined whether a HeatCount has been determined for all extents in the plurality of extents. When all HeatCounts have been determined and stored to the heatmap list, method900continues to operation906; otherwise, method900returns to operation902to determine a next extent's HeatCount. This is repeated until HeatCounts have been determined for all extents in the plurality of extents.

In operation906, the heatmap list is shortened (if necessary) to only include a number of extents which is less than a maximum number of extents that may be stored in a higher storage tier. The higher storage tier may have limited space, and therefore only a predetermined number of extents may be stored to the higher storage tier. Of course, this maximum is dictated by the actual storage capacity of the higher storage tier.

In one embodiment, the heatmap list may be sorted in either increasing order by HeatCount or decreasing order by HeatCount, and then all extents which have the lowest HeatCount values are removed from the heatmap list in order to shorten the heatmap list to the maximum number of extents that may be stored in the higher storage tier.

In operation908, it is determined whether the given extent from the heatmap list is stored to the higher storage tier already. When the given extent from the heatmap list is stored to the higher storage tier, method900continues to operation912; otherwise, method900continues to operation910.

In operation910, a second extent in the higher storage tier is searched for and/or located which does not appear in the heatmap list. Once such a second extent is determined, the second extent is copied (or moved) to a lower storage tier, and the second extent is replaced with the given extent on the higher storage tier.

In operation912, it is determined whether a storage location has been determined for every extent in the heatmap list (and each extent in the heatmap list has been stored to the higher storage tier). When the storage location has been determined for every extent in the heatmap list, method900ends; otherwise, method900returns to operation908to determine a storage location for a next given extent. This is repeated until the storage location has been determined for every extent in the heatmap list.

In one embodiment, in order to determine the HeatCount for an extent, Formula 3 may be used. Of course, any desired formula may be used, as would be understood by one of skill in the art, in various other approaches. In Formula 3, AccessCount is a value from the heatmap monitoring table which reflects a number of data block accesses for a given extent, CacheProfileWeight is a value from the heatmap monitoring table which reflects the CacheProfileWeight for a given extent, which may be calculated according to Formula 2, above. Furthermore, P and Q are system defined parameters, which may range in value from 0.1 to 10, or more or less, depending on a desired effect on the system's performance. Any of P and/or Q may be defaulted to 1, in one approach.
HeatCount=P*AccessCount+Q*CacheProfileWeight  Formula 3

According to one embodiment, the data relocator module and/or the multi-tier monitoring module may perform method900periodically to perform relocation of blocks across storage tiers based on heatmap information.

The methods described above may be executed individually or in combination in a system, device, apparatus, and/or computer program product utilizing a computer readable storage medium.

The system may include logic (hard and/or soft) that is implemented in a processor, of any type known in the art. The logic may be encompassed by the processor, accessible to the processor, and/or stored to memory that the processor accesses to perform the functionality dictated by the logic, according to various embodiments.

The file system solutions presented herein in various embodiments are applicable to storage systems, hybrid storage systems, and storage clouds, among other systems.

To more fully describe the methods presented herein according to various embodiments, consider the following example. Assume that an application is accessing blocks in a sequence of fourteen steps as shown inFIG. 10. This sequence represents an iteration, which may be repeated as many times as necessary to accomplish the desired result. An Application is going to perform multiple iterations of the same steps using different data. Also, assume for this example that the cache is capable of storing only two data blocks, and a LRU policy is implemented on the cache.FIG. 10shows observations and actions at the cache and multi-tier monitoring module.

At the start and at the end of the fourteen exemplary steps of the iteration, Block A, Block B, and Block C are not present in the cache. When AccThreshold is 30, Block B and Block C will be located on a higher storage tier, such as a SSD tier, a Flash memory Tier, etc., while Block A will be located on a lower storage tier, such as a HDD tier, optical drive tier, magnetic tape tier, etc. Thus, even when Block A is “hot” in regard to the particular application, the multi-tier monitoring module will determine Block A to be “cold.” It will place it on the lower storage tier rather than the higher storage tier which will cause access latency at Step1when it is accessed from the storage medium.

Currently, multi-tier monitoring modules try to monitor I/Os performed by observing I/O traffic between host and storage devices. The conclusion whether a given block is hot is determined based on how many times block data is accessed across the I/O line. However, in general from an application point of view, this traffic is due to cache misses (reads) and cache evictions (writes).

Table 2 shows an application access sequence for three data blocks, Block A, Block B, and Block C, with application access counts (requests) and medium accesses observed at the multi-tier monitoring module for each data block. The step at which each medium access is observed is indicated in the parenthesis after the number of observed accesses.

Table 3 shows the application access sequence for Block A, Block B, and Block C after 10 iterations, with application access counts (requests) and medium accesses observed at the multi-tier monitoring module for each data block.

TABLE 3ApplicationMedium Accesses Observed at theBlockAccessesMulti-Tier Monitoring ModuleA8020B2040C2040

This observed traffic might not be directly mapped to application access count. As shown in Tables 2 and 3, application access counts (requests) on Block A are actually 80 after 10 iterations, but the multi-tier monitoring module may observe access counts as only 20. Meanwhile, for Block B and Block C, application accesses are actually 20 but the multi-tier monitoring monitor may observe access counts as 40. Thus, Block A may not be considered as hot compared to Block B and Block C from the multi-tier monitoring module point of view. But from the application view, Block A is really hotter than Block B and Block C, and thus this block, when evicted, may be considered hotter and preferred to be placed on the higher storage tier over Block B and Block C when the application point of view is used.