Conflict management for application directed data placement in storage environments

A storage controller receives hints provided by one or more applications over a period of time, wherein the hints are used by the storage controller for organizing data in storage managed by the storage controller. Data on conflicts caused by the provided hints are collected over the period of time. Based on the collected data on the conflicts, one or more conflict avoidance rules are executed to reduce possibility of future conflicts.

BACKGROUND

Embodiments relate to conflict management for application directed data placement in storage environments.

A storage system may comprise a storage controller that is coupled to a plurality of storage devices. A storage management application that executes in the storage controller may control the plurality of storage devices, and provide access to data stored in the plurality of storage devices to one or more hosts that are coupled to the storage controller. In certain situations, applications that access storage volumes stored in the plurality of storage devices may execute on the storage controller or in the one or more hosts that are coupled to the storage controller.

In an application directed shared storage environment, applications that execute in hosts or the storage controller may provide hints in real-time to guide or dictate storage system functionality such as data placement. Such hints may allow the applications to dynamically configure the storage system to serve the needs of the applications, based on the application's awareness of the end-user performance expectancies, policies, workloads, system events, etc.

The storage management application that executes in the storage controller may act not only based on the hints provided by the applications but also based on the internal system knowledge and policies employed by the storage management application.

In certain situations, a single application may provide conflicting hints for the same data elements (e.g., extents of storage volumes). In other situations, a plurality of applications may provide conflicting hints for the same data elements. As a result there may be poor resource utilization. There may also be wrong placement of the data for some workloads and an inability to meet performance expectations. In operational conflicts, different hints may attempt to move the same extent to different locations to cause a conflict. In intent conflicts, an application may specify different intents for the same extent. It should be noted that intents expressed by applications are converted to operations by the storage controller, and this may cause operational conflicts.

In certain situations, such conflicts may be resolved by certain administrator generated pre-defined policies that are used to break ties and resolve the conflicts. For example, a pre-defined policy may be to prefer Application A over Application B. In conflict resolution mechanisms, the loser in the conflict may have relatively degraded performance and system utilization may suffer.

SUMMARY OF THE PREFERRED EMBODIMENTS

Provided are a method, a system, and a computer program product in which a storage controller receives hints provided by one or more applications over a period of time, wherein the hints are used by the storage controller for organizing data in storage managed by the storage controller. Data on conflicts caused by the provided hints are collected over the period of time. Based on the collected data on the conflicts, one or more conflict avoidance rules are executed to reduce possibility of future conflicts.

In additional embodiments, a determination is made based on the collected data, that thrashing exceeds a threshold level. In response to determining that the thrashing exceeds the threshold level, the storage controller disables hints that correspond to extents that are affected by the thrashing. An automated storage management mechanism of the storage management application is used to manage the selected extents without using the hints corresponding to the selected extents.

In further embodiments, the hints corresponding to the selected extents are disabled for a predetermined period of time and the hints corresponding to the selected extents are enabled after an expiry of the predetermined period of time.

In additional embodiments, a determination is made that hint spread is below a first threshold level and that a target tier in which extents with hints are placed has a utilization level that is below a second threshold level. Caching is used for segments of the extents that have hints indicating that the segments are likely to have relatively high utilization, rather than tiering an entirety of the extents.

In yet additional embodiments, the segments of the extents that have hints indicating that the segments are likely to have relatively high utilization are placed in the target tier without moving the entirety of the extents to the target tier.

In additional embodiments, hints of the extents are disabled, in response to failure of the caching and a failure of the placing of the segments of the extents in the target tier to increase the utilization level to at least the second threshold level.

In certain embodiments, data is reorganized, in response to determining that the conflicts exceed a first threshold, and a number of conflicting applications exceeds a second threshold.

In further embodiments, a reduction is made in a lease window of data if all the conflicting application have same priority. Additionally, higher priority is provided to an application with the highest rejection rate.

DETAILED DESCRIPTION

Certain embodiments provide mechanisms to avoid conflicts caused by conflicting hints. By adopting mechanisms to avoid conflict resolution, the likelihood of actually being forced to perform conflict resolution is decreased and as a result system efficiency may be increased.

Exemplary Embodiments

FIG. 1illustrates a block diagram of a first computing environment100, in accordance with certain embodiments. InFIG. 1, a storage controller102controls a storage environment106comprising a plurality of storage devices108a. . .108n. The storage controller102may comprise any suitable computational device known in the art, such as a mainframe computer, a midrange computer, a personal computer, a tablet device, a handheld device, a telephony device, a storage blade, a server, a processing element, etc.

In certain embodiments, the storage environment106may comprise a tiered storage environment and may also include a cached storage environment. In a tiered storage environment, some of the storage devices may be grouped in a plurality of tiers, for example a first storage tier, a second storage tier, and a third storage tier where the first storage tier is comprised of solid state disks, the second storage tier is comprised of enterprise drives, such as fibre channel (FC) or Serial Attached SCSI (SAS) drives in the 10-15K RPM (rotations per minute) range, and the third storage tier is comprises nearline drives with higher capacity but slower RPM (e.g., 7200 RPM) drives in comparison to the enterprise drives. It should be noted that the types of storage devices stored in the different storage tiers may be different in alternative embodiments. In certain embodiments, the first storage tier has a higher performance but lower storage capacity than the second storage tier, and the second storage tier has a higher performance but lower storage capacity than the third storage tier.

The storage maintained in the storage environment may be represented as volumes by a storage management application110that executes in the storage controller102. The unit of storage may comprise an extent in certain embodiments and a volume may have a plurality of extents. Extents may be moved between tiers, and certain data stored in extents may be staged and destaged from cache storage.

The storage management application110may provide access to the storage environment106to a plurality of applications112a. . .112p,114a. . .114qthat execute in a plurality of hosts116a. . .116rand the storage controller102. The application112. . .112p,114a. . .114pmay also be termed hint providing applications as they provide hints that are used by the storage management application110for placement of data and organization of data in the storage environment106. In certain embodiments, in the process of interpreting hints the storage management application110performs conflict avoidance.

Therefore,FIG. 1illustrates certain embodiments in which a storage management application110that executes in the storage controller102receives hints from a plurality of applications112a. . .112p,114a. . .114q, and performs conflict avoidance in the placement and organization of data in the storage environment106.

FIG. 2illustrates a first flowchart200for conflict avoidance, in accordance with certain embodiments. In certain embodiments, the operations shown inFIG. 2may be performed by the storage management application110that executes in the storage controller110.

Control starts at block202in which the storage management application110is programmed to identify metrics that may be tracked to detect poor utilization of the storage environment106because of data layout issues, intent conflicts or operational conflicts.

Control proceeds to block204, in which the metrics are used by the storage management application110to adjust parameters such as a lease window for an intent (i.e., hint), where a lease window for an intent is the time period for which the intent is applicable. For example, an intent provided by an application may request placement of certain data of the application on solid state disks on Monday to Friday during office hours when the data is likely to be used most.

In block204, the storage management application110may also apply data placement strategies. For example, caching may be used for certain data, whereas tiering may be used for other data.

In block204, the storage management application110may also use data layouts and reorganization of data that address the avoidance of conflicts caused by conflicting hints provided by the hint providing applications.

FIG. 3illustrates a block diagram300that shows exemplary conflict data collection, in accordance with certain embodiments. The collection of exemplary conflict data may be performed over a period of time and may be regarded as a learning mechanism for reducing future likelihood of conflicts.

Various representative conflict data collection mechanisms are shown inFIG. 3. They include collection of data to identify potential conflict resolution strategies. Data collection window based on tiering may also be used. Data may be collected in certain embodiments at the granularity of an extent, and various measures may be aggregated.

Such measures may include hint counts on the total number of hints received for an extent and operational conflict counts on the number of hints rejected due to conflict. Such measures may also include intent conflict counts that include the number of hints rejected due to intent conflicts. Track may also be kept of the number of conflicting applications, i.e., the number of unique application whose hints have been rejected. Average time between migrations may also be kept track of to measure thrashing. The rate of rejection per application may be kept track of. In certain embodiment, the percentage of an extent affected by hints (which comprises the spread of hints) may be measured. If a hint affects 1 kilobyte when an extent is 1 megabyte, then only 0.1% of the extent is affected by the hint and this is referred to as the spread of the hint. In certain embodiments, the relative heat (i.e., frequency of use) of the extent on a target tier (e.g. a solid state disk tier) may be measured.

Once the various statistics shown inFIG. 3are collected by the storage management application110, then the statistics are interpreted for conflict avoidance.

FIG. 4illustrates a second flowchart400for conflict avoidance, in accordance with certain embodiments. In certain embodiments, the operations shown inFIG. 4may be performed by the storage management application110that executes in the storage controller110.

Control starts at block402in which a storage controller102receives hints provided by one or more applications112a. . .112p,114a. . .114qover a period of time, where the hints are used by the storage controller102for organizing data in storage106managed by the storage controller102. Data on conflicts caused by the provided hints are collected (at block404) over the period of time. Based on the collected data on the conflicts, one or more conflict avoidance rules are executed (at block406) to reduce the possibility of future conflicts. The conflict avoidance rules may be based on the amount of thrashing, hint spreads, the number of conflicting applications or other factors.

FIG. 5illustrates a third flowchart500for conflict avoidance based on a level of thrashing, in accordance with certain embodiments. In certain embodiments, the operations shown inFIG. 5may be performed by the storage management application110that executes in the storage controller110.

Control starts at block502, in which a determination is made based on the collected data, that thrashing (i.e., movement of extents) exceeds a threshold level. In response to determining that the thrashing exceeds the threshold level, the storage controller disables (at block504) hints that correspond to extents that are affected by the thrashing.

Control proceeds to block506, in which an automated storage management mechanism of the storage management application110is used to manage the selected extents without using the hints corresponding to the selected extents. The hints corresponding to the selected extents are disabled (at block508) for a predetermined period of time and the hints corresponding to the selected extents are enabled after an expiry of the predetermined period of time.

FIG. 6illustrates a fourth flowchart600for conflict avoidance based on hint spread and utilization level of tiers, in accordance with certain embodiments. In certain embodiments, the operations shown inFIG. 6may be performed by the storage management application110that executes in the storage controller110.

Control starts at block602in which a determination is made that that hint spread is below a first threshold level and that a target tier in which extents with hints are placed has a utilization level that is below a second threshold level. Caching is used (at block604) for segments of the extents that has hints indicating that the segments are likely to have relatively high utilization, rather than tiering an entirety of the extents.

Control proceeds to block606, in which the segments of the extents that have hints indicating that the segments are likely to have relatively high utilization are placed in the target tier (e.g., a solid state disk tier) without moving the entirety of the extents to the target tier. Hints of the extents are disabled (at block608), in response to failure of the caching and a failure of the placing of the segments of the extents in the target tier to increase the utilization level to at least the second threshold level.

FIG. 7illustrates a fifth flowchart700for conflict avoidance based at least on the number of applications, in accordance with certain embodiments. In certain embodiments, the operations shown inFIG. 7may be performed by the storage management application110that executes in the storage controller110.

Control starts at block702in which data is reorganized, in response to determining that the conflicts exceed a first threshold, and a number of conflicting applications exceeds a second threshold. Control proceeds to block704, in which a reduction is made in a lease window of data if all the conflicting application have same priority and a round robin scheduling mechanism for hints of the applications may be used for fairness. Additionally, higher priority is provided (at block706) to an application with highest rejection rate or a combination of round-robin scheduling and scheduling based on rejection rate may be used.

In certain additional embodiments an alternate location for a selected applications data may be proposed. If an application can provide data entity and logical address via the hints, then the storage management application110may perform the reorganization and provide the new logical address along with the data entity for easier lookup.

Therefore,FIGS. 1-7provide certain mechanisms by which a storage management application avoids conflict in managing storage, when conflicting hints are provided by one or more applications.

Cloud Computing Environment

Cloud computing is a model for enabling convenient, on-demand network access to a shared pool of configurable computing resources (e.g., networks, servers, storage, applications, and services) that can be rapidly provisioned and released with minimal management effort or service provider interaction.

Additional Embodiment Details

The described operations may be implemented as a method, apparatus or computer program product using standard programming and/or engineering techniques to produce software, firmware, hardware, or any combination thereof. Accordingly, aspects of the embodiments may take the form of an entirely hardware embodiment, an entirely software embodiment (including firmware, resident software, micro-code, etc.) or an embodiment combining software and hardware aspects that may all generally be referred to herein as a “circuit,” “module” or “system.” Furthermore, aspects of the embodiments may take the form of a computer program product embodied in one or more computer readable medium(s) having computer readable program code embodied there.

FIG. 8illustrates a block diagram that shows certain elements that may be included in the storage controller102in accordance with certain embodiments. The system800(e.g., a computational device, such as the storage controller202and/or the host220) may include a circuitry802that may in certain embodiments include at least a processor804. The system800may also include a memory806(e.g., a volatile memory device), and storage808. The storage808may include a non-volatile memory device (e.g., EEPROM, ROM, PROM, RAM, DRAM, SRAM, flash, firmware, programmable logic, etc.), magnetic disk drive, optical disk drive, tape drive, etc. The storage808may comprise an internal storage device, an attached storage device and/or a network accessible storage device. The system800may include a program logic810including code812that may be loaded into the memory806and executed by the processor804or circuitry802. In certain embodiments, the program logic810including code812may be stored in the storage808. In certain other embodiments, the program logic810may be implemented in the circuitry802. Therefore, whileFIG. 8shows the program logic810separately from the other elements, the program logic810may be implemented in the memory806and/or the circuitry802.