Migration decision window selection based on hotspot characteristics

Methods and arrangements for selecting a migration decision window for hotspots in a multi-tier enterprise storage system. Aspects include collecting usage statistics for data stored in the multi-tier enterprise storage system, identifying hotspots from data stored in the multi-tier enterprise storage system based on the usage statistics, and determining one or more characteristics of the identified hotspots. Aspects further include calculating an average lifespan of the identified hotspots based on the one or more characteristics of the identified hotspots and selecting the migration decision window based on the average lifespan of the identified hotspots and the one or more characteristics of the identified hotspots.

BACKGROUND

The present invention relates to storage of data in a multi-tier storage system, and more specifically, to selecting a migration decision window for stored data based upon characteristics of hotspots in the multi-tier enterprise storage system.

In general, multi-tier enterprise storage systems include at least two types of data storage devices that have different performance characteristics. These multi-tier enterprise storage systems are designed to optimize system performance and maximize utilization of the storage devices by migrating hot data to faster storage devices, such as solid state devices (SSDs) and cold data to appropriate storage device types, such as hard disk drives (HDDs). As used herein, the term hot data refers to data that experiences a relatively large number of random accesses and the term cold data refers to data that experiences a relatively large number of sequential accesses or data that experiences a relatively low number of total accesses. In general, hotspots, which are storage areas that contain hot data, in multi-tier enterprise storage systems are dynamic, that is the locations of hotspots change over time. For example, a particular portion of storage may be considered a hotspot for a specific period of time and will not be considered a hotspot during another period of time.

Typically, access statistics for data stored in the multi-tier enterprise storage systems are collected over time and periodically reviewed. These statistics are evaluated at regular intervals, often by an automated algorithm, referred to as a decision window, in order to identify which data is hot and which data is cold. In currently available systems, the decision window is a static system parameter that is set by an administrator. In general, a static decision window may result in sub-optimal system performance and sub-optimal utilization of the storage devices in the multi-tier enterprise storage systems. For example, a decision window which is much longer than the duration of a hotspot may result in poor responsiveness in the system, since the hot data is not placed on a faster storage device in time for the large number of accesses.

SUMMARY

Methods, systems and computer program products for selecting a migration decision window for hotspots in a multi-tier enterprise storage system are provided. Aspects include collecting usage statistics for data stored in the multi-tier enterprise storage system, identifying hotspots from data stored in the multi-tier enterprise storage system based on the usage statistics, and determining one or more characteristics of the identified hotspots. Aspects further include calculating an average lifespan of the identified hotspots based on the one or more characteristics of the identified hotspots and selecting the migration decision window based on the average lifespan of the identified hotspots and the one or more characteristics of the identified hotspots.

DETAILED DESCRIPTION

Exemplary embodiments include methods, systems and computer program products for selecting a migration decision window based upon characteristics of hotspots in a multi-tier enterprise storage system. In exemplary embodiments, the migration decision window selection process is a continuously running process and, as a result, the migration decision window is dynamic and changes overtime. In exemplary embodiments, the migration decision window will change in response to changes in the characteristics of hotspots in the multi-tier enterprise storage system.

Referring now toFIG. 1, a block diagram of a multi-tier enterprise storage system100in accordance with an exemplary embodiment is shown. As illustrated the system100includes a computer system102which is in communication with a first storage device104and a second storage device106. In exemplary embodiments, the first storage device104includes one or more high performance storage devices such as SSDs and the second storage device106includes one or more low performance storage devices, such as HDDs. In exemplary embodiments, the computer system102manages data stored on both the first storage device104and the second storage device106in data blocks of a predetermined size. For example, the computer system102may manage the data in one gigabyte blocks, referred to herein as extents. In exemplary embodiments, the computer system102monitors the usage characteristics of each extent and determines which of the two storage devices to store each extent on. In general, extents that contain hot data and are identified as hotspots are stored on the first storage device104and the other extents are stored on the second storage device106.

Referring now toFIG. 2, a flowchart diagram of a method200for selecting a migration decision window based upon characteristics of hotspots in a multi-tier enterprise storage system in accordance with an exemplary embodiment is shown. As illustrated at block202, the method200includes collecting usage statistics for data stored in the multi-tier enterprise storage system. In exemplary embodiments, the usage statistics are collected for each extent and include, but are not limited to, an average input/output (I/O) size, a I/O count, a sequential read count, a random read count, a sequential write count, a random write count, and a latency. In exemplary embodiments, the usage statistics are collected for each extent periodically, such as every five or ten minutes.

Next, as shown at block204, the method200includes identifying one or more hotspots from the data stored in the multi-tier enterprise storage system based on the usage statistics. As illustrated at block206, the method200includes determining characteristics of the identified hotspots. In exemplary embodiments, the characteristics of the identified hotspots include, but are not limited to, a lifespan of the hotspot and the size of the hotspot. Next, as shown at block208, the method200includes calculating an average lifespan of the identified hotspots. As illustrated at block210, the method200includes selecting the migration decision window based on the average lifespan of the identified hotspots.

In exemplary embodiments, the migration decision window may be selected to be a time period equal to half of the average lifespan of the identified hotspots if the average lifespan of the identified hotspots is above a minimum threshold value. If the average lifespan of the identified hotspots is below the minimum threshold value, the migration decision window may be selected to be a default time period set by an administrator. In exemplary embodiments, the minimum threshold value is used to ensure that hotspots which have a very short lifespan are not moved to the high performance storage device. In many cases, by the time hotspots with very short lifespans are migrated, the hotspot is no longer hot and the migration was a waste of computing resources.

In one embodiment, the average hotspot lifespan is calculated to be 1 cycles, the size of an identified hotspot is calculated to be z extents, and the migration bandwidth is determined to be x extents per cycle. The migration bandwidth is the amount of data per cycle that the computer system can move between the first storage device and the second storage device. In exemplary embodiments, the decision window d can be calculated by the formula:

In exemplary embodiments, a hotspot will not be migrated if the average lifespan of the hotspot indicates that the hotspot will end before half of the hotspot is migrated. In other words, the hotspot will not be migrated if the following equation is true:

Referring now toFIG. 3, a flowchart diagram of a method300for selecting a migration decision window based upon characteristics of hotspots in a multi-tier enterprise storage system in accordance with an exemplary embodiment is shown. As illustrated at block302, the method300includes analyzing a workload of a multi-tier storage system. Next, as shown at block304, the method includes identifying one or more hotspots in the workload. In exemplary embodiments, any of a variety of known techniques can be used to identify one or more hotspots in the workload. As illustrated at decision block306, the method300includes determining if the average lifespan of the identified hotspots are greater than a minimum value. In exemplary embodiments, the minimum value is determined based on the average lifespan of the identified hotspots, the size of the identified hotspots and the migration bandwidth of the multi-tier storage system. If the average lifespan of the identified hotspots is greater than the minimum value, the method300proceeds to block310and includes calculating a migration decision window based on the average lifespan of the identified hotspot. Otherwise, the method300proceeds to block308and includes selecting a default time period for the migration decision window. After the migration decision window has been selected, the method300proceeds to block312and includes migrating hotspots to a high performance storage device once migration every decision window period.

In one embodiment the decision on which hotspots to migrate may also be based on the size of the first storage device. For example, in one embodiment the identified hotspots may be placed in a ranked list based on how hot the hotspots are and the hotspots may be migrated until the first storage device reaches capacity. In another embodiment, the decision on which hotspots to migrate may also be based on the predetermined threshold of hot data. For example, in one embodiment each of the identified hotspots may include a score that represents how hot the data in the hotspot is based on the access statistics of the extents in the hotspot and only hotspots which have a score that is above the predetermined threshold will be migrated.

In exemplary embodiments, hotspots that have a size that exceeds a maximum size may not be migrated. For example, in one embodiment if a hotspot is determined to have a size that is greater than half of the size of the first storage device, the hotspot may not be migrated to the first storage device.

Technical effects and benefits include a multi-tier enterprise storage system that dynamically determines a migration decision window for identified hotspots in the multi-tier enterprise storage system, which results in optimal system performance, particularly improved responsiveness, and optimal utilization of the storage devices in the multi-tier enterprise storage system.

Referring now toFIG. 4, a block diagram of an exemplary computer system400for use with the teachings herein is shown. The methods described herein can be implemented in hardware software (e.g., firmware), or a combination thereof. In an exemplary embodiment, the methods described herein are implemented in hardware, and is part of the microprocessor of a special or general-purpose digital computer, such as a personal computer, workstation, minicomputer, or mainframe computer. The system400therefore includes general-purpose computer401.

In an exemplary embodiment, in terms of hardware architecture, as shown inFIG. 4, the computer401includes a processor405, memory440coupled via a memory controller445, a storage device420, and one or more input and/or output (I/O) devices440,445(or peripherals) that are communicatively coupled via a local input/output controller435. The input/output controller435can be, for example, but not limited to, one or more buses or other wired or wireless connections, as is known in the art. The input/output controller435may have additional elements, which are omitted for simplicity, such as controllers, buffers (caches), drivers, repeaters, and receivers, to enable communications. Further, the local interface may include address, control, and/or data connections to enable appropriate communications among the aforementioned components. The storage device420may include one or more hard disk drives (HDDs), solid state drives (SSDs), or any other suitable form of storage.

The processor405is a computing device for executing hardware instructions or software, particularly that stored in memory440. The processor405can be any custom made or commercially available processor, a central processing unit (CPU), an auxiliary processor among several processors associated with the computer401, a semiconductor based microprocessor (in the form of a microchip or chip set), a macroprocessor, or generally any device for executing instructions. The processor405may include a cache470, which may be organized as a hierarchy of more cache levels (L1, L2, etc.).

The instructions in memory440may include one or more separate programs, each of which comprises an ordered listing of executable instructions for implementing logical functions. In the example ofFIG. 4, the instructions in the memory440include a suitable operating system (OS)411. The operating system411essentially controls the execution of other computer programs and provides scheduling, input-output control, file and data management, memory management, and communication control and related services.

In an exemplary embodiment, a conventional keyboard450and mouse455can be coupled to the input/output controller435. Other output devices such as the I/O devices440,445may include input devices, for example but not limited to a printer, a scanner, microphone, and the like. Finally, the I/O devices440,445may further include devices that communicate both inputs and outputs, for instance but not limited to, a network interface card (NIC) or modulator/demodulator (for accessing other files, devices, systems, or a network), a radio frequency (RF) or other transceiver, a telephonic interface, a bridge, a router, and the like. The system400can further include a display controller425coupled to a display430. In an exemplary embodiment, the system400can further include a network interface460for coupling to a network465. The network465can be an IP-based network for communication between the computer401and any external server, client and the like via a broadband connection. The network465transmits and receives data between the computer401and external systems. In an exemplary embodiment, network465can be a managed IP network administered by a service provider. The network465may be implemented in a wireless fashion, e.g., using wireless protocols and technologies, such as Wi-Fi, WiMax, etc. The network465can also be a packet-switched network such as a local area network, wide area network, metropolitan area network, Internet network, or other similar type of network environment. The network465may be a fixed wireless network, a wireless local area network (LAN), a wireless wide area network (WAN) a personal area network (PAN), a virtual private network (VPN), intranet or other suitable network system and includes equipment for receiving and transmitting signals.

If the computer401is a PC, workstation, intelligent device or the like, the instructions in the memory440may further include a basic input output system (BIOS) (omitted for simplicity). The BIOS is a set of essential routines that initialize and test hardware at startup, start the OS411, and support the transfer of data among the storage devices. The BIOS is stored in ROM so that the BIOS can be executed when the computer401is activated.

When the computer401is in operation, the processor405is configured to execute instructions stored within the memory440, to communicate data to and from the memory440, and to generally control operations of the computer401pursuant to the instructions. In exemplary embodiments, the computer system400includes one or more accelerators480that are configured to communicate with the processor405. The accelerator480may be a field programmable gate array (FPGA) or other suitable device that is configured to perform specific processing tasks. In exemplary embodiments, the computer system400may be configured to offload certain processing tasks to an accelerator480because the accelerator480can perform the processing tasks more efficiently than the processor405.

These computer program instructions may also be stored in a computer readable medium that can direct a computer, other programmable data processing apparatus, or other devices to function in a particular manner, such that the instructions stored in the computer readable medium produce an article of manufacture. Such an article of manufacture can include instructions which implement the function/act specified in the flowchart and/or block diagram block or blocks.