System and method for QoS-based storage tiering and migration technique

The present invention is directed to a method for providing Quality Of Service (QoS)-based storage tiering and migration in a storage system. The method allows for configurable application data latency thresholds to be set on a per user basis and/or a per application basis so that a storage tiering mechanism and/or a storage migrating mechanism may be triggered for moving application data to a different class of storage.

FIELD OF THE INVENTION

The present invention relates to the field of storage resource and data management and particularly to a system and method for Quality of Service (QoS)-based storage tiering and migration technique.

BACKGROUND OF THE INVENTION

Currently available methods for providing storage resource and data management in data storage systems may not provide a desired level of performance.

Therefore, it may be desirable to provide system(s) and method(s) for providing storage resource and data management in a data storage system which addresses the above-referenced shortcomings of currently available solutions.

SUMMARY OF THE INVENTION

Accordingly, an embodiment of the present invention is directed to a method for providing Quality Of Service (QoS)-based storage tiering and migration in a storage system, said method including the steps of: monitoring an application process when the application process is accessing a first storage tier of the storage system; based on said monitoring, determining a QoS factor measurement of the application process; comparing the QoS factor measurement to at least one of: a first pre-determined QoS factor threshold (ex.—a tier threshold) and a second pre-determined QoS factor threshold (ex.—a migrate threshold); when said comparing indicates that the QoS factor measurement is greater than the first pre-determined QoS factor threshold, creating a Point-in-Time (PiT) copy and establishing an access permission of the PiT copy as Read Only, wherein the PiT copy is based upon the first storage tier; when said comparing indicates that the QoS factor measurement is: less than the first pre-determined QoS factor threshold; and greater than the second pre-determined QoS factor threshold, migrating data from the PiT copy to a second storage tier; after the migrating of data from the PiT copy to the second storage tier has completed, setting establishing access permissions of the second storage tier as Read and Write and directing I/O commands to the second storage tier; when said comparing indicates that the QoS factor measurement is: less than the first pre-determined QoS factor threshold; and less than the second pre-determined QoS factor threshold, mirroring second storage tier data to the first storage tier; when mirroring of data of the second storage tier data to the first storage tier is synchronized, directing read commands to the second storage tier; when mirroring of the second storage tier data to the first storage tier is synchronized, synchronizing write commands between the first storage tier and the second storage tier.

A further embodiment of the present invention is directed to a computer program product comprising: a signal bearing medium bearing: computer-usable code configured for monitoring an application process when the application process is accessing a first storage tier of the storage system; computer-usable code configured for based on said monitoring, determining a QoS factor measurement of the application process; computer-usable code configured for comparing the QoS factor measurement to at least one of: a first pre-determined QoS factor threshold and a second pre-determined QoS factor threshold; computer-usable code configured for, when said comparing indicates that the QoS factor measurement is greater than the first pre-determined QoS factor threshold, creating a Point-in-Time (PiT) copy and establishing an access permission of the PiT copy as Read Only, wherein the PiT copy is based upon the first storage tier; computer-usable code configured for, when said comparing indicates that the QoS factor measurement is: less than the first pre-determined QoS factor threshold; and greater than the second pre-determined QoS factor threshold, migrating data from the PiT copy to a second storage tier; computer-usable code configured for, after the migrating of data from the PiT copy to the second storage tier has completed, setting establishing access permissions of the second storage tier as Read and Write and directing I/O commands to the second storage tier; computer-usable code configured for, when said comparing indicates that the QoS factor measurement is: less than the first pre-determined QoS factor threshold; and less than the second pre-determined QoS factor threshold, mirroring second storage tier data to the first storage tier; computer-usable code configured for, when minoring of data of the second storage tier data to the first storage tier is synchronized, directing read commands to the second storage tier; computer-usable code configured for, when mirroring of the second storage tier data to the first storage tier is synchronized, synchronizing write commands between the first storage tier and the second storage tier.

A still further embodiment of the present invention is directed to a storage system, including: a processor; a memory, said memory being connected to the processor; and control programming for executing on the processor, wherein the control programming is configured for: monitoring an application process when the application process is accessing a first storage tier of the storage system; based on said monitoring, determining a QoS factor measurement of the application process; comparing the QoS factor measurement to at least one of: a first pre-determined QoS factor threshold and a second pre-determined QoS factor threshold; and when said comparing indicates that the QoS factor measurement is greater than the first pre-determined QoS factor threshold, creating a Point-in-Time (PiT) copy and establishing an access permission of the PiT copy as Read Only, wherein the PiT copy is based upon the first storage tier, wherein said control programming is further configured for: when said comparing indicates that the QoS factor measurement is: less than the first pre-determined QoS factor threshold; and greater than the second pre-determined QoS factor threshold, migrating data from the PiT copy to a second storage tier; after the migrating of data from the PiT copy to the second storage tier has completed, setting establishing access permissions of the second storage tier as Read and Write and directing I/O commands to the second storage tier; when said comparing indicates that the QoS factor measurement is: less than the first pre-determined QoS factor threshold; and less than the second pre-determined QoS factor threshold, mirroring second storage tier data to the first storage tier; when mirroring of data of the second storage tier data to the first storage tier is synchronized, directing read commands to the second storage tier; when minoring of the second storage tier data to the first storage tier is synchronized, synchronizing write commands between the first storage tier and the second storage tier.

DETAILED DESCRIPTION OF THE INVENTION

A number of current commercially available storage tiering mechanisms may be based upon the following: 1) cost per byte of storage space; 2) data access patterns; and 3) migration and placement of less frequently accessed data to a lower class storage device. However, none of these current commercially available storage tiering mechanisms address the quality aspects pertaining to the data that a user is accessing on a day-to-day basis. Further, in these current commercially available storage tiering mechanisms no configurable Quality Of Service (QoS) parameters (Note: Quality of Service may also be abbreviated as (QoS)) are available to set the data access latency thresholds for triggering the tiering/migrating process based on application process and user data.

The system(s) and method(s) of the present invention, such as those disclosed herein, allow for configurable application data latency thresholds to be set on a per user basis and/or on a per application basis, such that a storage tiering mechanism (ex.—a storage migrating mechanism) is triggered for moving application data to a different class of storage. Further, the system(s) and method(s) of the present invention, such as those disclosed herein, may further provide the following features: 1) promoting of significant improvement in QoS levels associated with applications hosted by a storage subsystem; 2) predictable latency associated with storage access; 3) tunability, for ensuring complete utilization of storage resources which deliver the highest QoS, thereby delivering a full Return On Investment (ROI) on high performance tiers; 4) on-the-fly manual administration for improving QoS for business critical and/or time critical application needs; 5) automatic retiring of lower performing storage based upon QoS thresholds; 6) automatic migration of user and application data to a different storage tier based on QoS thresholds; 7) significant improvements in sustaining predictable application performance on shared storage; 8) significant improvement in latency associated with application specific data access; 9) automatic policy-based tiering and migration without requiring human intervention; and 10) predictive modeling and projections to recommend needed capacity of high performance tiers and lower performing tiers based on historical use. Further, the system(s) and method(s) of the present invention, such as those disclosed herein, may involve changing enterprise level configurable QoS parameters on a per user, per application basis by an administrator. Still further, the system(s) and method(s) of the present invention provide for spontaneous storage tiering and migration based on a QoS concept applied at a user level and/or an application data level.

Referring generally toFIGS. 1A,1B and3A through3C, a storage system100upon/within which the method(s) for providing Quality of Service (QoS)-based storage tiering and migration may be implemented is shown in accordance with exemplary embodiments of the present invention. In exemplary embodiments, the storage system100may include a processor102. In further embodiments, the storage system100may further include a memory104(ex.—Random Access Memory (RAM)). In still further embodiments, the storage system100may further include a bus106. For instance, the processor102and the memory104may be connected to the bus106. In current embodiments of the present invention, the storage system100may include a storage subsystem, which may further include a plurality of storage tiers, such as a first storage tier108and a second storage tier112. Each storage tier may include one or more storage devices which make up a storage pool (ex.—a base volume108) for that storage tier. Further, the first storage tier may be a different class of storage than the second storage tier, the two storage tiers being distinguished by the performance of the storage devices belonging to the first tier compared to the performance of the storage devices belonging to the second tier (ex.—one tier's storage devices may be higher performing storage devices than the storage devices of the other tier). In further embodiments of the present invention, the storage subsystem of the storage system100may be connected to the processor102and the memory104via the bus106. In still further embodiments of the present invention, the system100may be configured (ex—via control programming executing on the processor102) for allowing method(s) for providing Quality Of Service (QoS)-based storage tiering and migration to be performed via and/or within the storage system100, as will be set forth below.

Referring toFIG. 4, a flowchart is provided which illustrates a method for providing Quality of Service (QoS)-based storage tiering and migration in a storage system, in accordance with an exemplary embodiment of the present invention. The method400may include the step of: monitoring an application process when the application process is accessing a first storage tier of the storage system402. For example, in the storage system100of the present invention, one or more QoS factors160of/corresponding to one or more application processes150may be monitored (ex.—continuously monitored and/or sampled) as the application processes are accessing a storage pool (ex.—a base volume108) of a first storage tier (as shown inFIGS. 1A and 1B). In exemplary embodiments of the present invention, the QoS factors160may be real-time QoS factors associated with/pertaining to/corresponding to the application processes150.FIG. 1Aillustrates a plurality of application processes150accessing the storage pool108of the first storage tier when QoS factors160are at an optimum level during runtime. Monitoring (ex.—sampling) may be done in real-time in order to keep the monitoring window open for as long as the application processes150are active. In current embodiments of the present invention, some of the QoS factors160which may govern the application processes150in real-time may include but are not limited to the following: read latency; write latency; storage media response time; idle time (ex.—time when no active reads/writes are performed); pervasiveness of data (ex.—data being written and read are in close proximity within an application share hosted in a storage device of the storage subsystem); data localization (ex.—degree to which particular data location(s) are being accessed compared to others); age of data; and age of application files.

In further embodiments, the method400may further include, based upon said monitoring, determining a QoS factor measurement of the application process404. For instance, when QoS factor(s)160of (ex.—associated with) application processes150are being monitored, reading(s) or measurement(s) for the QoS factor(s) may be determined. These measurements (ex.—QoS factor measurements) may be continuously determined for the duration of monitoring. After a period of time, one or more QoS factor(s)160associated with a particular application process150may begin to deteriorate (as shown via the designation “Critical QoS” inFIG. 1B), thereby impacting overall QoS thresholds for that particular application process. In exemplary embodiments of the present invention, the method400may further include the step of: comparing the QoS factor measurement to at least one of: a first pre-determined QoS threshold (ex.—a Tier Threshold (TTH)) and a second pre-determined QoS factor threshold (ex.—a Migrate Threshold (MH))406. In current embodiments, these thresholds may serve as trigger points for latency aspects of the system100. For example, based upon said comparison, the Tier Threshold and the Migrate Threshold may serve as trigger points which may trigger the system100to execute a sequence of steps for: a) storage tiering; or b) migrating application data to a different class of storage respectively, as will be described in further detail below. The graphical representation (ex.—graph)200shown inFIG. 2illustrates a QoS level(s)/a measurement of a QoS factor (ex.—storage latency associated with application data) plotted against sampling time (ex.—monitoring time). A first region of the graph202illustrates a point in time when the application processes150are running at optimum QoS levels. Solid line204indicates actual non-tiered storage latency. Dashed line206indicates predicted non-tiered storage latency. Dotted line208indicates actual latency measured after application data is tiered to a different class (ex.—tier) of storage. Within the sampling period (SP), the QoS levels may be measured and compared against the trigger thresholds (ex.—the Tier Threshold (TTH) and the Migrate Threshold (MTH).

In exemplary embodiments of the present invention, the method400may further include, when said comparing indicates that the QoS factor measurement is greater than the first pre-determined QoS factor threshold, creating a Point-in-Time (PiT) copy and establishing an access permission of the PiT copy as Read Only, wherein the PiT copy is based upon the first storage tier408. For example, as shown inFIG. 3A, when the comparison of the QoS factor measurement to the Tier Threshold (TTH) indicates that the QoS factor measurement is greater than the TTH, one or more PiT copies110(ex.—PiT snapshots or PiTs) may be created based upon the first storage tier (ex.—may be created off of the base volume108).

In further embodiments of the present invention, the method400may further include migrating data from the PiT copy to a second storage tier when said comparing indicates that the QoS factor measurement is: less than the first pre-determined QoS factor threshold; and greater than the second pre-determined QoS factor threshold410. For instance, as shown inFIG. 3B, when the comparison of the QoS factor measurement to the Tier Threshold (TTH) and the Migrate Threshold (MTH) indicates that the QoS factor measurement is less than the TTH, but greater than the MTH, data may be migrated from the one or more PiT copies (ex.—PiTs)110to a second (ex.—next) storage tier112. A heuristic approach may be implemented when migrating said data for promoting prevention of thrashing in cases when the QoS factor measurement(s) (ex.—the QoS) may fluctuate quickly between thresholds.

In current embodiments of the present invention, the method400may further include, after the migrating of data from the PiT copy to the second storage tier has completed, setting establishing access permissions of the second storage tier as Read and Write and directing I/O commands to the second storage tier412. For example, once all the data from the one or more PiT copies110has been migrated to the second storage tier412, incoming I/Os may be re-directed to and serviced by the second storage tier112, with the access permissions of the second storage tier112being set to Read and Write (R/W) and PiT copies being continuously generated off of the base volume108.

In exemplary embodiments of the present invention, the method400may further include, when said comparing indicates that the QoS factor measurement is: less than the first pre-determined QoS factor threshold; and less than the second pre-determined QoS factor threshold, mirroring second storage tier data to the first storage tier414. For instance, as shown inFIG. 3C, when the comparison of the QoS factor measurement to the Tier Threshold (TTH) and the Migrate Threshold (MTH) indicates that the QoS factor measurement is less than the TTH and less than the MTH, data in the second storage tier112may be mirrored to the first storage tier108. A heuristic approach may be implemented when mirroring said data in order to prevent thrashing between thresholds.

In further embodiments of the present invention, the method400may further include, when mirroring of data of the second storage tier data to the first storage tier is synchronized, directing read commands to the second storage tier416and synchronizing write commands between the first storage tier and the second storage tier418. For example, when mirroring (ex.—mirror) is synchronized, all reads may be serviced by the second tier112and all writes may be synchronized between the base volume108and the second tier112.

It is to be noted that the foregoing described embodiments according to the present invention may be conveniently implemented using conventional general purpose digital computers programmed according to the teachings of the present specification, as will be apparent to those skilled in the computer art. Appropriate software coding may readily be prepared by skilled programmers based on the teachings of the present disclosure, as will be apparent to those skilled in the software art.

It is to be understood that the present invention may be conveniently implemented in forms of a software package. Such a software package may be a computer program product which employs a computer-readable storage medium including stored computer code which is used to program a computer to perform the disclosed function and process of the present invention. The computer-readable medium/computer-readable storage medium may include, but is not limited to, any type of conventional floppy disk, optical disk, CD-ROM, magnetic disk, hard disk drive, magneto-optical disk, ROM, RAM, EPROM, EEPROM, magnetic or optical card, or any other suitable media for storing electronic instructions.

It is understood that the specific order or hierarchy of steps in the foregoing disclosed methods are examples of exemplary approaches. Based upon design preferences, it is understood that the specific order or hierarchy of steps in the method can be rearranged while remaining within the scope of the present invention. The accompanying method claims present elements of the various steps in a sample order, and are not meant to be limited to the specific order or hierarchy presented.