METHOD, DEVICE, AND COMPUTER PROGRAM PRODUCT FOR HIERARCHICAL STORAGE OF FILES

Hierarchical storage of files is described. An example method includes determining an input/output (IO) mode, a file size, and a file access frequency of a target file, and determining IO mode performances of the IO mode at different tiers of a storage system. The method further includes determining, based on the file size of the target file, respective storage costs when the target file is stored at the different tiers of the storage system. In addition, the method further includes determining a target tier at which the target file is stored based on the IO mode performances, the respective storage costs, the file size, and the file access frequency. In this way, it is possible to make full use of less expensive storage tiers and save the cost of the storage system, as well as to reduce the response time and improve the performance of the storage system.

RELATED APPLICATION

The present application claims the benefit of priority to Chinese Patent Application No. 202311246206.8, filed on Sep. 25, 2023, which application is hereby incorporated into the present application by reference herein in its entirety.

TECHNICAL FIELD

The present disclosure relates to the field of data storage and, for example, to a method, a device, and a computer program product for hierarchical storage of files.

BACKGROUND

Hierarchical storage of files is a data management method that is intended to efficiently manage and store large amounts of data and to store the data hierarchically in different storage tiers based on the access frequency and importance of the data, so as to achieve both cost effectiveness and performance optimization. These storage tiers typically include a high-performance tier, a medium-performance tier, and a low-performance tier. Storage apparatuses in the high-performance tier have the highest file access speeds and are the most expensive. Storage apparatuses in the medium-performance tier are less expensive and have relatively large capacities, but their file access speeds are lower than those in the high-performance tier. Storage apparatuses in the low-performance tier are the least expensive, but at the same time have the lowest file access speeds.

An advantage of hierarchical storage of files is that it allows businesses to find a balance between performance and cost so as to meet the needs of storage of different data. With hierarchical storage of files, businesses can reduce storage costs while still having quick access to the most commonly used data. Migration and management of data is typically performed automatically by storage management software in order to ensure that data is located at the appropriate tier. This method helps optimize the use of storage resources, improves the performance, and ensures the availability of critical data.

SUMMARY

Embodiments of the present disclosure present a method, a device, and a computer program product for hierarchical storage of files. In the solution provided by embodiments of the present disclosure, when determining a storage tier for a target file, in addition to acquiring an access frequency for the target file, an input/output (IO) mode of the target file and a file size of the target file may also be acquired. The solution may then determine, for the IO mode of the target file, a plurality of IO mode performance scores of the IO mode at different storage tiers. In addition, this solution may further determine, based on the file size of the target file, a plurality of storage costs corresponding to the target file when it is stored at different storage tiers respectively. After determining the plurality of IO mode performance scores and the plurality of storage cost scores corresponding to the plurality of storage tiers, it may be determined at which storage tier the target file is stored most appropriately based on the plurality of IO mode performance scores, the plurality of storage costs, and the file size and access frequency of the target file. In this way, the determination of the storage tier for the target file not only is based on the access frequency of the target file, but also takes into account the IO mode of the target file as well as its storage cost, which can thus optimize the file hierarchization strategy, thereby taking full advantage of the low-priced storage tiers and saving the cost of the storage system. In addition to this, files that are better suitable for storage at the high-performance tier can also be migrated to a higher storage tier, thus reducing the response time and improving the performance of the storage system.

In a first example embodiment of the present disclosure, a method for hierarchical storage of files is provided. The method includes determining an input/output (IO) mode, a file size, and a file access frequency of a target file. The method further includes determining a plurality of IO mode performances of the IO mode at a plurality of tiers of a storage system. The method further includes determining, based on the file size of the target file, a plurality of storage costs when the target file is stored at the plurality of tiers of the storage system. In addition, the method further includes determining a target tier at which the target file is stored based on the plurality of IO mode performances, the plurality of storage costs, the file size, and the file access frequency.

In a second example embodiment of the present disclosure, an electronic device is provided. The electronic device includes one or more processors; and a memory coupled to the at least one processor and having instructions stored thereon, wherein the instructions, when executed by the at least one processor, cause the electronic device to perform actions including: determining an input/output (IO) mode, a file size, and a file access frequency of a target file. The actions further include determining a plurality of IO mode performances of the IO mode at a plurality of tiers of a storage system. The actions further include determining, based on the file size of the target file, a plurality of storage costs when the target file is stored at the plurality of tiers of the storage system. In addition, the actions further include determining a target tier at which the target file is stored based on the plurality of IO mode performances, the plurality of storage costs, the file size, and the file access frequency.

In a third example embodiment of the present disclosure, a computer-readable storage medium is provided, on which a computer program is stored, which, when executed by a processor, implements a method for hierarchical storage of files. The method includes determining an input/output (IO) mode, a file size, and a file access frequency of a target file. The method further includes determining a plurality of IO mode performances of the IO mode at a plurality of tiers of a storage system. The method further includes determining, based on the file size of the target file, a plurality of storage costs when the target file is stored at the plurality of tiers of the storage system. In addition, the method further includes determining a target tier at which the target file is stored based on the plurality of IO mode performances, the plurality of storage costs, the file size, and the file access frequency.

It should be understood that the content described in this Summary part is neither intended to limit key or essential features of the embodiments of the present disclosure, nor intended to limit the scope of the present disclosure. Other features of the present disclosure will become readily understood from the following description.

DETAILED DESCRIPTION

The following will describe the embodiments of the present disclosure in more detail with reference to the accompanying drawings. Although the accompanying drawings show some embodiments of the present disclosure, it should be understood that the present disclosure may be implemented in various forms, and should not be explained as being limited to the embodiments stated herein. Rather, these embodiments are provided for understanding the present disclosure more thoroughly and completely. It should be understood that the accompanying drawings and embodiments of the present disclosure are for illustrative purposes only, and are not intended to limit the protection scope of the present disclosure.

In the description of the embodiments of the present disclosure, the term “include” and similar terms thereof should be understood as open-ended inclusion, that is, “including but not limited to.” The term “based on” should be understood as “based at least in part on.” The term “an embodiment” or “the embodiment” should be understood as “at least one embodiment.” The terms “first,” “second,” and the like may refer to different or identical objects. Other explicit and implicit definitions may also be included below.

Hierarchical storage of files enables movement of files among different storage tiers of a storage system based on specific rules. In some conventional solutions, a file hierarchization strategy can select an appropriate storage tier for a target file based on its access frequency. For example, if the target file has a high access frequency, the file hierarchization strategy can move the file to a storage tier with a high performance, and the high-performance storage tier can be, for example, a tier with solid-state disk (SSD) drives. If the target file has a low access frequency, the file hierarchization strategy can move the file to a storage tier with a low performance, and the low-performance storage tier can be, for example, a storage tier with a serially attached small computer system interface (SAS) drive or a near-line SAS (NL-SAS) drive, which can save costs and improve the efficiency of use of a storage device.

However, storage workloads resulting from client applications accessing files typically have specific IO modes, and different IO modes exhibit different performances at different storage tiers. For some IO modes, the difference between the performance exhibited on a high-performance storage tier and that on a low-performance storage tier is small; therefore, if files with these IO modes are stored at the high-performance storage tier because of high access frequencies, it will not bring significant performance improvement and at the same time take up expensive high-performance storage resources. By contrast, for some other IO modes, the difference between the performance exhibited at the high-performance storage tier and that at the low-performance storage tier is large; therefore, if files with these IO modes are stored at the low-performance storage tier because of not very high access frequencies, potential performance gains and opportunities to enhance user experience will be missed.

In addition to this, the impact of the difference in storage costs among various storage tiers on the hierarchical storage of files is not considered in conventional solutions. For example, if the difference in costs among multiple storage tiers is large, and if two files have the same file access frequency, moving the file with a larger file size to a high-performance storage tier will take up more expensive high-performance storage resources, thus producing a high storage cost, which will make the resources of the storage system unable to be used reasonably.

To this end, embodiments of the present disclosure propose a solution for hierarchical storage of files. When determining a storage tier for a target file, the method may acquire an IO mode of the target file and a file size of the target file in addition to acquiring an access frequency to the target file. The solution may then determine, for the IO mode of the target file, a plurality of IO mode performance scores of the IO mode at different storage tiers. In addition, this solution may further determine, based on the file size of the target file, a plurality of storage costs corresponding to the target file when it is stored at different storage tiers respectively. After determining the plurality of IO mode performance scores and the plurality of storage cost scores corresponding to the plurality of storage tiers, it may be determined at which storage tier the target file is stored most appropriately based on the plurality of IO mode performance scores, the plurality of storage costs, and the file size and access frequency of the target file. In this way, the determination of the storage tier for the target file not only is based on the access frequency of the target file, but also takes into account the IO mode of the target file as well as its storage cost, which can thus optimize the file hierarchization strategy, thereby taking full advantage of the low-priced storage tiers and saving the cost of the storage system. In addition to this, files that are better suitable for storage at the high-performance tier can also be migrated to a higher storage tier, thus reducing the response time and improving the performance of the storage system and the user experience.

FIG.1illustrates a schematic diagram of an example environment100in which a plurality of embodiments of the present disclosure can be implemented. As shown inFIG.1, the environment100includes a target file102for which a storage tier is to be determined, a storage system103, and a management application110, where the storage system103includes a high-performance tier104, a medium-performance tier106, and a low-performance tier108. The high-performance tier104may, for example, be a storage tier having SSD drives, which is characterized by the highest file access speeds, but the storage devices included in it are also the most expensive. The medium-performance tier106may, for example, be a storage tier having SAS drives, which is characterized by relatively low file access speeds, but the storage devices included in it are also less expensive. The low-performance tier108may, for example, be a storage tier having NL-SAS drives, which is characterized by the lowest file access speeds, but the storage devices included in it are the least expensive. It should be understood that while the storage systems in embodiments of the present disclosure all include three storage tiers, it is not intended to impose a limit on the number of storage tiers in a storage system. The solutions provided by embodiments of the present disclosure can be applied to storage systems with other numbers of storage tiers, such as storage systems with two storage tiers (e.g., a high-performance tier and a low-performance tier), storage systems with four storage tiers (e.g., a highest-performance tier, a high-performance tier, a low-performance tier, and a lowest-performance tier), and the like.

The target file102may be a file that has been stored in any of the storage tiers in the storage system103, and as needed, it can be determined by the management application110whether to migrate the file to another storage tier, and to which storage tier. As shown inFIG.1, the target file102has a plurality of attributes, including an IO mode112, an access frequency114, and a file size116. The IO mode112may be related to a data size (e.g., 4 KB or 8 KB) of one IO operation, a ratio of read operations to write operations (e.g., one hundred percent read operations, one hundred percent write operations, or seventy percent read operations and thirty percent write operations), and an access approach (e.g., sequential access or random access). In the environment100, the different storage tiers in the storage system103contain different types of storage devices, and therefore, unit costs (i.e., price per GB) of the different storage tiers are also different. As shown inFIG.1, the high-performance tier104has a unit cost of124, the medium-performance tier106has a unit cost of126, and the low-performance tier108has a unit cost of128.

As shown inFIG.1, the management application110may determine the corresponding IO mode performance and storage cost when the target file102is stored at various storage tiers of the storage system103. For example, the management application110may determine an IO mode performance134of the IO mode112at the high-performance tier104and a storage cost144consumed for storing the target file102at the high-performance tier104, an IO mode performance136of the IO mode112at the medium-performance tier106and a storage cost146consumed for storing the target file102at the medium-performance tier106, and an IO mode performance138of the IO mode112at the low-performance tier108and a storage cost148consumed for storing the target file102at the low-performance tier108. In some embodiments, the IO mode performance (e.g., the IO mode performance may be expressed in the form of a score) of the IO mode at a particular storage tier may be determined based on the bandwidth, the number of IO operations per second (IOPS), and the response time. In some embodiments, the storage cost for storing the target file102on the storage tier may be determined based on the file size116of the target file102and the unit cost of the storage tier.

As shown inFIG.1, after determining the IO mode performances134,136, and138and the storage costs144,146, and148, the management application110may determine the difference in IO mode performances of the IO mode112across different storage tiers, where a large difference in IO mode performances may indicate that a large IO mode performance gain can result from migrating the target file102to a storage tier with a high performance, and a small difference in IO mode performances may indicate that a small IO mode performance loss can result from migrating the target file102to a storage tier with a low performance. In addition to this, the management application110may also determine the difference in storage costs of storing the target file102at different storage tiers, where a larger difference in storage costs may indicate that it will cost more to migrate the target file102to a storage tier with a high performance, and a small difference in storage costs may indicate that the storage cost changes little when the target file102is migrated to other storage tiers. The management application110may then determine a target tier150based on the difference in IO mode performances, the difference in storage costs, and the access frequency114and the file size116of the target file102, and migrate the target file102from the storage tier where it is located to the target tier150.

In this manner, when the difference in IO mode performances of the IO mode112at various storage tiers is small and the difference in storage costs of storing the target file102at various storage tiers is large, although the access frequency114of the target file102may be very high, it may still be possible to choose to store the target file102at a storage tier with a low storage cost, so that the storage cost can be significantly saved while hardly affecting the file access speed. At the same time, files that have a large difference in IO mode performances but have an access frequency not enough to reach a high threshold in the previous hierarchization strategy can be migrated to a storage tier with a high performance, thereby improving the performance of the entire storage system and enhancing the user experience.

FIG.2illustrates a flow chart of a method200for hierarchical storage of files according to some embodiments of the present disclosure. As shown inFIG.2, at a block202, the method200may determine an IO mode, a file size, and a file access frequency of a target file. For example, in the environment100as shown inFIG.1, the management application110may determine the IO mode112, the file access frequency114, and the file size116of the target file102. The IO mode112may, for example, include a data size of one IO operation, a ratio of read operations to write operations, and an access approach. An example of the IO mode112may be as follows: a data size of 4 KB of one IO operation, one hundred percent read operations, and a sequential access approach.

At a block204, the method200may determine a plurality of IO mode performances of the IO mode at a plurality of tiers of a storage system. For example, in the environment100as shown inFIG.1, the management application100may determine an IO mode performance134of the IO mode112of the target file102at the high-performance tier104of the storage system103, an IO mode performance136at the medium-performance tier106, and an IO mode performance138at the low-performance tier108, and the IO mode performances134,136and138may, for example, be expressed in the form of scores. In some embodiments, a plurality of IO mode performances of each IO mode at various storage tiers may be pre-determined, and these pre-determined IO mode performances may be stored at an appropriate location. When the management application110needs to determine the target tier150for the target file102, the plurality of IO mode performances corresponding to the IO mode112of the target file102may be read from the appropriate location. In this way, it is possible to save time spent on determining the plurality of IO mode performances and to improve the response speed.

At a block206, the method200may determine, based on the file size of the target file, a plurality of storage costs when the target file is stored at the plurality of tiers. For example, in the environment100as shown inFIG.1, the management application100may determine, based on the file size116of the target file102, a storage cost144when the target file102is stored at the high-performance tier104of the storage system103, a storage cost146when the target file102is stored at the medium-performance tier106, and a storage cost148when the target file102is stored at the low-performance tier108. Typically, the storage cost144for storing the target file102at the high-performance tier104is the highest, and the storage cost for storing the target file102at the low-performance tier108is the smallest, which depends on the unit cost124of the storage device at the high-performance tier104, the unit cost126of the storage device at the medium-performance tier106, and the unit cost128of the storage device at the low-performance tier108. However, embodiments of the present disclosure do not limit the ranking of the unit costs of various storage tiers.

At a block208, the method200determines a target tier at which the target file is stored based on the plurality of IO mode performances, the plurality of storage costs, the file size, and the file access frequency. For example, in the environment100as shown inFIG.1, the management application110may determine the target tier150based on the IO mode performances134,136, and138of the IO mode112of the target file102at the high-performance tier104, the medium-performance tier106, and the low-performance tier108, respectively, the storage costs144,146, and148for storing the target file102at the high-performance tier104, the medium-performance tier106, and the low-performance tier108, respectively, the access frequency114of the target file102, and the file size116of the target file102. The management application110may then migrate the target file102to the target tier150.

In this way, the determination of the storage tier150at which the target file102is to be stored not only is based on the access frequency114of the target file102, but also takes into account the IO mode112of the target file102as well as the storage cost for the target file102, which can thus optimize the file hierarchization strategy, thereby taking full advantage of the low-priced storage tiers and saving the cost of the storage system103. In addition to this, files that are more suitable for storage at the high-performance storage tier can also be migrated to a higher storage tier, thus reducing the response time and improving the performance of the storage system103.

FIG.3illustrates a schematic diagram of an example300of IO mode performance scores for different IO modes on different types of storage devices according to some embodiments of the present disclosure. As shown inFIG.3, the example300includes four IO modes, and characteristics of the IO modes are reflected in the names of the IO modes, for example, 4K_0R_100S, 4K_0R_0S, 8K_70R_0S, and 64K_90R_100S. The 4K, 8K, and 64K in the IO mode names indicate the data size of one IO operation, and it should be understood that while only 4K, 8K, and 64K are illustrated in the example300, the data size of one IO operation may also be other values. The R in the IO mode names indicates the proportion of read operations in all IO operations (and implicitly indicates the proportion of write operations in all IO operations). For example, 0R indicates that the proportion of read operations in all IO operations is zero (the corresponding proportion of write operations is one hundred percent), 70R indicates that the proportion of read operations in all IO operations is seventy percent (the corresponding proportion of write operations is thirty percent), and 90R indicates that the proportion of read operations in all IO operations is ninety percent (the corresponding proportion of write operations is ten percent). It should be understood that while only 0R, 70R, and 90R are shown in the example300, the proportion of read operations in IO operations may also be other values. In addition, S in the IO mode names indicates the proportion of sequential operations in all IO operations (and implicitly indicates the proportion of random operations). For example, 0S indicates that the proportion of sequential operations in all IO operations is zero (the corresponding proportion of random operations is one hundred percent), and 100S indicates that the proportion of sequential operations in all IO operations is one hundred percent (the corresponding proportion of random operations is zero). It should be understood that while only 0S and 100S are shown in the example300, the proportion of sequential operations in IO operations may also be other values.

As shown inFIG.3, the example300further includes three storage device types, i.e., SSD, SAS, and NL-SAS, where SSD has the highest performance and is the most expensive, SAS has lower performance than SSD but is also less expensive than SSD, and NL-SAS has the lowest performance and is the least expensive. For example, the high-performance tier104in the environment100as shown inFIG.1may include SSD drives, the medium-performance tier106may include SAS drives, and the low-performance tier108may include NL-SAS drives. It should be understood that the storage device types in the example300are intended as examples only, and the storage device types may include other types depending on the number and configuration of storage tiers in the storage system.

As shown inFIG.3, the example300may determine the IOPS (e.g., count per second), the bandwidth (e.g., MB per second), and the response time (e.g., in seconds or milliseconds) of each IO mode on different types of storage devices (representing different storage tiers). In addition, in the example300, an IO mode performance score may also be determined based on the IOPS, the bandwidth, and the response time, where a high IO mode performance score indicates that a particular IO mode exhibits high performance on a particular type of storage devices. As shown inFIG.3, some IO modes do not achieve performance improvement after being raised from SAS or NL-SAS to SSD. For example, the performance score of the IO mode 4K_0R_100S is 0.9897 on SAS and 1.0345 on NL-SAS, but its performance score on SSD is only 0.9752, which is lower than the performance scores on SAS and NL-SAS. It can be seen that if a file with the IO mode 4K_0R_100S is migrated from SAS or NL-SAS to SSD, the performance of the storage system will not be improved while the file consumes expensive SSD resources, so it is not worthwhile to migrate such files to SSD.

As shown inFIG.3, some other IO modes can achieve significant performance improvement after being raised from SAS or NL-SAS to SSD. For example, the performance score of the IO mode 8K_70R_0S is 0.3475 on SAS and 0.2520 on NL-SAS, while its performance score on SSD is up to 2.4005. It can be seen that if a file with the IO mode 8K_70R_0S is migrated from SAS or NL-SAS to SSD, the performance of the storage system will be significantly improved, so it is worthwhile to migrate such files to SSD. In this way, files with IO modes that have a small performance difference across different storage tiers can be migrated to storage tiers that have low performance but are less expensive, thus saving costs while ensuring that the performance will not be significantly reduced.

FIG.4illustrates a flow chart of an example process400for determining a target tier for a target file according to some embodiments of the present disclosure. As shown inFIG.4, at a block402, the process400may collect various information related to a storage system, such as the bandwidth, IOPS, and response time of each IO mode at different storage tiers, a plurality of unit costs corresponding to the plurality of storage tiers, and the access frequency and file size of each file stored in the storage system. For example, in the environment100shown inFIG.1, the management application110can collect the bandwidth (e.g., MB per second), the IOPS (e.g., count per second), and the response time (e.g., seconds or milliseconds) of the IO mode112(e.g., 8 KB of data in one operation, seventy percent read operations, and one hundred percent random operations) of the target file102at the high-performance tier104(e.g., a storage tier with SSD), the medium-performance tier106(e.g., a storage tier with SAS), and the low-performance tier108(e.g., a storage tier with NL-SAS) of the storage system103, for use in determining the IO mode performances134,136, and138in subsequent steps. The management application110can also collect the file size116(e.g., MB or GB) of the target file102and the unit cost124(e.g., price per GB) of the high-performance tier104for use in determining the storage costs144,146, and148. In addition, the management application110can also collect the access frequency114of the target file102for use in determining the target tier150. In addition to the target file102, the management application110can also collect relevant information about other files for use in determining the target tiers for the other files. The process400then proceeds to a block404.

At the block404, the process400can determine whether it is currently necessary to redetermine the tile for the file. In some embodiments, the tier for the file may be redetermined periodically (e.g., every one hour, every two hours, etc.). In some embodiments, the re-hierarchization of the file may be triggered according to a predetermined scheduling plan. If it is not currently necessary to redetermine the tier for the file, the process400returns to the block402to continue collecting relevant information; otherwise, the process400proceeds to a block406.

At the block406, the process400may determine a difference in performances of the IO mode of the target file across a plurality of tiers. For example, in the environment100as shown inFIG.1, a difference in the IO mode performances134,136and138may be determined by the management application110. In some embodiments, a plurality of bandwidths, a plurality of IOPSs, and a plurality of response times corresponding to a plurality of tiers in an IO mode may be determined, and then a plurality of IO mode performances of that IO mode at the plurality of tiers of the storage system may be determined based on the plurality of bandwidths, the plurality of IOPSs, and the plurality of response times. In some embodiments, a plurality of standardized bandwidths, a plurality of standardized IOPSs, and a plurality of standardized response times corresponding to the plurality of tiers may be determined based on the plurality of bandwidths, the plurality of IOPSs, and the plurality of response times corresponding to the plurality of tiers, and then a plurality of IO mode performance scores corresponding to the plurality of tiers may be determined based on the plurality of standardized bandwidths, the plurality of standardized IOPSs, and the plurality of response times, an IO mode performance score being indicative of an IO mode performance at the corresponding tier. In some embodiments, an average IO mode performance score for the plurality of tiers may be determined based on the plurality of IO mode performance scores, and then a degree of dispersion of the plurality of IO mode performance scores may be determined, based on the plurality of IO mode performance scores and the average IO mode performance score, as the difference in performances of the IO mode across the plurality of tiers.

The process for determining the difference in performances of an IO mode of a target file across a plurality of tiers is described in detail below in conjunction withFIGS.5A to5D. The IO performance of the storage system can be determined by determining the bandwidth, the IOPS, and the response time, and since the bandwidth, the IOPS, and the response time are in different units, they need to be standardized to unify the units, and then the IO mode performance score is determined based on the standardized bandwidth (also referred to as the bandwidth score), the standardized IOPS (also referred to as the IOPS score), and the standardized response time (also referred to as the response time score).FIGS.5A to5Cillustrate schematic diagrams of example processes for determining a standardized bandwidth, a standardized IOPS, and a standardized response time according to some embodiments of the present disclosure.

FIG.5Aillustrates a schematic diagram of an example process500for determining a standardized bandwidth of an IO mode of a target file at various tiers according to some embodiments of the present disclosure. As shown inFIG.5A, the process500may determine a bandwidth504of the IO mode112of the target file102at the high-performance tier104, a bandwidth506of the IO mode112at the medium-performance tier106, and a bandwidth508of the IO mode112at the low-performance tier108. The process500may then use the bandwidths504,506, and508to standardize each of them to obtain a standardized bandwidth514for the high-performance tier104, a standardized bandwidth516for the medium-performance tier106, and a standardized bandwidth518for the low-performance tier108.

Here, the number of files on the storage system is denoted by I, and for each file fi(1≤i≤I), the size of the file fiis denoted by fsi. In addition, the type of the IO mode is denoted by J, with pjdenoting the jth IO mode. There are K storage tiers in the storage system, with tkdenoting the kth storage tier. The bandwidth of the IO mode pjon the storage tier tkis denoted by Bk, j. The larger the bandwidth of an IO mode at a storage tier, the higher the performance of that IO mode, so the standardized bandwidth NBk, jcan be calculated by the following Equation (1):

FIG.5Billustrates a schematic diagram of an example process520for determining a standardized IOPS of an IO mode of a target file at various tiers according to some embodiments of the present disclosure. As shown inFIG.5B, the process520may determine an IOPS524of the IO mode112of the target file102at the high-performance tier104, an IOPS526of the IO mode112at the medium-performance tier106, and an IOPS528of the IO mode112at the low-performance tier108. The process520may then use the IOPSs524,526, and528to standardize each of them to obtain a standardized IOPS534for the high-performance tier104, a standardized IOPS536for the medium-performance tier106, and a standardized IOPS538for the low-performance tier108.

Here, IOPSk, jis used to denote the IOPS of the IO mode pjat the storage tier tk. The larger the IOPS of an IO mode at a storage tier, the higher the performance of that IO mode, so the standardized IOPS, i.e., NIOPSk, j, can be calculated by the following Equation (2):

FIG.5Cillustrates a schematic diagram of an example process540for determining a standardized response time of an IO mode of a target file at various tiers according to some embodiments of the present disclosure. As shown inFIG.5C, the process540may determine a response time544of the IO mode112of the target file102at the high-performance tier104, a response time546of the IO mode112at the medium-performance tier106, and a response time548of the IO mode112at the low-performance tier108. The process520may then use the response times544,546, and548to standardize each of them to obtain a standardized response time534for the high-performance tier104, a standardized response time536for the medium-performance tier106, and a standardized response time538for the low-performance tier108.

Here, Lk,jis used to denote the response time of the IO mode pjat the storage tier tk. The smaller the response time of an IO mode at a storage tier, the higher the performance of that IO mode, so the standardized response time, i.e., NLk, j, can be calculated by the following Equation (3):

FIG.5Dillustrates a schematic diagram of an example process560for determining a difference in IO mode performances of an IO mode of a target file at various tiers according to some embodiments of the present disclosure. As shown inFIG.5D, the process560may determine an IO mode performance score564of the IO mode112at the high-performance tier104based on the standardized bandwidth514, the standardized IOPS534, and the standardized response time554. The process560may further determine an IO mode performance score566of the IO mode112at the medium-performance tier106based on the standardized bandwidth516, the standardized IOPS536, and the standardized response time556. In addition, the process560may further determine an IO mode performance score568of the IO mode112at the low-performance tier108based on the standardized bandwidth518, the standardized IOPS538, and the standardized response time558. The process560may then calculate an average performance score570based on the IO mode performance scores564,566, and568, and calculate a performance score standard deviation572for the IO mode performance scores564,566, and568based on the average performance score570. The performance score standard deviation572may be indicative of the degree of dispersion of the IO mode performance scores564,566, and568, and may be indicative of the difference in performances of the IO mode112of the target file102across the high-performance tier104, the medium-performance tier106, and the low-performance tier108.

Here, Perfk,jis used to denote the performance score of the IO mode pjat the storage tier tk. As mentioned above, the IO mode performance score can be determined based on the bandwidth, the IOPS, and the response time of the IO mode at the storage tier, and thus the Perfk,jcan be calculated by the following Equation (4):

Then, the standard deviation σjof the performance scores of the IO mode pjover the K storage tiers can be calculated by the following Equation (5):

where NPerfaverageis the average of the performance scores of the IO mode pjover the K storage tiers.

In this way, the IO mode performance of the IO mode at each storage tier can be determined in a quantitative manner based on the bandwidth, the IOPS, and the response time of the IO mode at each storage tier, so that the degree of dispersion of performances of the IO mode across the storage tiers can be further quantitatively determined, and a decision-making basis can be provided for determining whether it is worthwhile to migrate the target file to a storage tier with higher performance.

In some embodiments, the difference in performances of each IO mode at the plurality of tiers of the storage system may be pre-determined, and the pre-determined difference in performances may be stored at an appropriate location. When redetermining the target tier for the target file, the pre-determined difference in IO mode performances of the IO mode of the target file at different storage tiers can be directly read and found for use in subsequent operations. In this way, the time consumed in determining the difference in performances of the IO mode of the target file at different storage tiers can be saved, and computational resources can be saved to improve the performance of the management application.

Referring back toFIG.4, at a block408, the process400may calculate a difference in storage costs for the target file across the plurality of tiers. For example, in the environment100as shown inFIG.1, the management application110may determine the storage cost144when the target file102is stored at the high-performance tier104, the storage cost146when the target file102is stored at the medium-performance tier106, and the storage cost148when the target file102is stored at the low-performance tier108, as well as the differences in the storage costs144,146and148. In some embodiments, a plurality of storage costs when the target file is stored at the plurality of tiers, respectively, may be determined based on the file size of the target file and a plurality of unit storage space costs corresponding to the plurality of tiers of the storage system. In some embodiments, an average storage cost for the plurality of tiers may be determined based on the plurality of storage costs, and then a degree of dispersion of the plurality of storage costs may be determined, based on the plurality of storage costs and the average storage cost, as the difference in storage costs.

The process for determining the difference in storage costs when the target file is stored at the plurality of tiers is described in detail below in conjunction withFIG.6.FIG.6illustrates a schematic diagram of an example process600for determining a difference in storage costs when a target file is stored at various tiers according to some embodiments of the present disclosure. As shown inFIG.6, the process600may determine a storage cost604when the target file102is stored at the high-performance tier104based on the file size112of the target file102and the unit cost124of the high-performance tier104. The process600may also determine a storage cost606when the target file102is stored at the medium-performance tier106based on the file size of the target file102and the unit cost126of the medium-performance tier106. In addition, the process600may also determine a storage cost608when the target file102is stored at the low-performance tier108based on the file size of the target file102and the unit cost128of the low-performance tier108. The process600may then calculate an average storage cost610for the storage costs604,606, and608, and utilize the average storage cost610to calculate a storage cost standard deviation612for the storage costs604,606, and608. The storage cost standard deviation612may be indicative of the degree of dispersion of the storage costs604,606, and608, and may be indicative of the difference in storage costs for the target file102when it is stored at the high-performance tier104, the medium-performance tier106, and the low-performance tier108.

Here, ckis used to denote the unit cost (e.g., price per GB) of the storage tier k, and ci,kis used to denote the storage cost for the file fiwhen it is stored at the storage tier k. Then, ci,kcan be calculated by the following Equation (6):

The smaller the storage cost, the better, so the average storage cost ci,avgfor the file fiwhen it is stored at various storage tiers can be calculated by using the following Equation (7):

Then, the standard deviation σcjof the storage costs for the file fiwhen it is stored at various storage tiers can be calculated by using the following Equation (8):

When the storage cost standard deviation σcjis large, it indicates that there is a large difference in storage costs between different storage tiers, and therefore, it costs more to migrate the file fifrom a low-performance tier to a high-performance tier. When σcjis small, it indicates that there is no significant difference in storage costs between different storage tiers, and therefore, it costs less to migrate the file fifrom a low-performance tier to a high-performance tier.

In this way, when determining the storage tier for the target file, it is possible to quantify the change in storage cost brought by migration, so as to provide a cost decision-making basis when determining the tier for the target file.

Referring back toFIG.4, at a block410, a composite score for the target file may be calculated based on the difference in performances of the IO mode of the target file across the plurality of tiers and the difference in storage costs for the target file across the plurality of tiers, the composite score being indicative of a gain associated with the performance and the cost resulting from migrating the target file from a low-performance tier of the storage system to a high-performance tier. In some embodiments, a file access density of the target file may be determined based on the file size and the file access frequency of the target file, the file access density being indicative of an average access frequency per unit file size. In some embodiments, the composite score for the target file may be calculated based on the difference in IO mode performances, the difference in storage costs, and the file access density. In some embodiments, a first weight for the difference in IO mode performances and a second weight for the difference in storage costs may be determined, and the composite score for the target file may be determined based on the difference in IO mode performances, the first weight, the difference in storage costs, the second weight, and the access density.

The detailed process for determining a composite score for a target file is described in detail below in conjunction withFIG.7.FIG.7illustrates a schematic diagram of an example process700for determining a composite score of a target file according to some embodiments of the present disclosure. As shown inFIG.7, the process700may determine an access density706based on the access frequency114and the file size116of the target file102, the access density706being indicative of the access frequency per unit file size of the target file102. In addition, the process700may also assign a weight702to the performance score standard deviation572of the IO mode of the target file, and a weight704to the storage cost standard deviation612of the target file. In some embodiments, the sum of the weight702and the weight704is one, and when an administrator of the storage system values the IO performance more, the weight702may be increased and the weight704decreased accordingly; and when the administrator values the cost more, the weight704may be increased and the weight702decreased accordingly. The process700may then calculate a composite score708for the target file102based on the performance score standard deviation573, the weight702, the storage cost standard deviation612, the weight704, and the access density706. The higher the comprehensive score708, the more worthwhile the target file102is to be migrated to a high-performance storage tier.

Here, fqiis used to denote the total access frequency of the file fiduring the last monitoring cycle, so the file access density frican be calculated by the following Equation (9):

Then, the composite score csiof the target file can be calculated according to the file access density fri, the IO mode performance standard deviation σj, and the storage cost standard deviation σcjand through the following Equation (10):

where wpis the weight for the IO mode performance standard deviation σj, and wcis the weight for the storage cost standard deviation σcj, and wp+wc=1. According to Equation (10), it can be seen that the greater the difference in performances of the IO mode of the target file across different storage tiers, the more worthwhile the target file is to be migrated to a high-performance storage tier, whereas the greater the difference in storage costs when the target file is stored at various storage tiers, the more expensive a storage device at the high-performance tier is, and for cost considerations, the lower the composite score csiof the target file is.

In this way, a composite score can be determined for the target file based on the difference in performances of the IO mode across various storage tiers, the storage costs for the target file at various storage tiers, and the access density of the target file, so that the target tier for the target file can be considered more comprehensively, and therefore, the overall performance of the storage system can be improved and the cost can be reduced.

Referring back toFIG.4, at a block412, the process400may determine the target tier for the target file based on composite scores of a plurality of files. In some embodiments, a plurality of composite scores corresponding to a plurality of files to be stored may be determined, wherein the plurality of files comprise the target file. In some embodiments, the target tier at which the target file is stored may be determined based on the plurality of composite scores. In some embodiments, a ranked plurality of composite scores may be determined by ranking the plurality of composite scores. In some embodiments, tiers at which the plurality of files are stored may be determined based on the ranked plurality of composite scores, wherein a file ranked higher among the plurality of files is stored at a tier with higher performance, and a file ranked lower among the plurality of files is stored at a tier with lower performance. In some other embodiments, a plurality of thresholds corresponding to the various tiers may be pre-determined, and the target tier for the target file may be determined based on the composite score of the target file and the plurality of thresholds.

The process for determining the target tier at which the file is stored is described in detail below in conjunction withFIG.8.FIG.8illustrates a schematic diagram of an example process800for determining a target tier at which a target file is stored based on a composite score of the target file according to some embodiments of the present disclosure. As shown inFIG.8, the process800needs to determine the tiers for files802,812, and822. The process802may determine a difference in IO mode performances804, a difference in storage costs806, and an access density808of the target file802and calculate a composite score810based on these metrics. The process802may also determine a difference in IO mode performances814, a difference in storage costs816, and an access density818of the target file812and calculate a composite score820based on these metrics. In addition, the process802may also determine a difference in IO mode performances824, a difference in storage costs826, and an access density828of the target file822and calculate a composite score830based on these metrics. As shown inFIG.8, the file802has the highest access density808, and in a conventional solution, the file802will be migrated to a high-performance tier. However, in the process800, since the difference in IO mode performances804of the file802is low and the storage cost806is high, it indicates that migrating the file802to a high-performance tier will not bring a significant IO performance improvement but will increase the storage cost significantly, so the composite score810of the file802is not the highest score.

As shown inFIG.8, the file812has the lowest access density818and a high difference in storage costs816, so that although its difference in IO mode performances814is not low, the composite score820is the lowest, indicating that the access frequency of the file812is low, and the cost of storing it in a high-performance tier is high, so the composite score820of the file812is the lowest. As shown inFIG.8, the file822has the highest difference in IO mode performances824and has a medium difference in storage costs826and access density828, indicating that migrating the file822to a high-performance tier may improve performance to a great degree without paying a large storage cost, so the composite score830of the file822is the highest. The process800may then rank the files802,812, and822based on the composite scores810,820, and830, and then determine the target tier based on the ranking results. For example, in the example shown inFIG.8, the process800may migrate the file802to the medium-performance tier106, the file812to the low-performance tier108, and the file822to the high-performance tier104.

In some embodiments, a plurality of proportions corresponding to the various tiers may be pre-determined, and the files may then be hierarchized based on the plurality of proportions. For example, after ranking according to the composite scores, the top twenty percent of the files may be migrated to the high-performance tier104, the files ranked between twenty and forty percent may be migrated to the medium-performance tier106, and the other files may be migrated to the low-performance tier108. In some embodiments, a plurality of composite score thresholds corresponding to the various tiers may be pre-determined, and then the files may be hierarchized based on the plurality of composite score thresholds. For example, files with composite scores greater than a first threshold are migrated to the high-performance tier104, files with composite scores between the first and second thresholds are migrated to the medium-performance tier106, and files with composite scores less than the second threshold are migrated to the low-performance tier108.

In this way, the determination of the storage tier for the target file not only is based on the access frequency of the target file, but also takes into account the IO mode of the target file as well as its storage cost, which can thus optimize the file hierarchization strategy, thereby taking full advantage of the low-priced storage tiers and saving the cost of the storage system. In addition to this, files that are better suitable for storage at the high-performance tier can also be migrated to a higher storage tier, thus reducing the response time and improving the performance of the storage system.

FIG.9illustrates a schematic block diagram of an example device900which may be used to implement embodiments of the present disclosure. For example, the device running the management application110as shown inFIG.1may be the example device900as shown inFIG.9. As illustrated in the figure, the device900includes a computing unit901that can execute various appropriate actions and processing according to computer program instructions stored in a read-only memory (ROM)902or computer program instructions loaded from a storage unit908to a random access memory (RAM)903. Various programs and data required for the operation of the device900may also be stored in the RAM903. The computing unit901, the ROM902, and the RAM903are connected to each other through a bus904. An input/output (I/O) interface905is also connected to the bus904.

A plurality of components in the device900are connected to the I/O interface905, including: an input unit906, such as a keyboard and a mouse; an output unit907, such as various types of displays and speakers; a storage unit908, such as a magnetic disk and an optical disc; and a communication unit909, such as a network card, a modem, and a wireless communication transceiver. The communication unit909allows the device900to exchange information/data with other devices via a computer network, such as the Internet, and/or various telecommunication networks.

The computing unit901may comprise various general-purpose and/or special-purpose processing components with processing and computing capabilities. Some examples of computing units901include, but are not limited to, central processing units (CPUs), graphics processing units (GPUs), various specialized artificial intelligence (AI) computing chips, various computing units for running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The computing unit901performs various methods and processes described above, such as the method200. For example, in some embodiments, the method200may be implemented as a computer software program that is tangibly included in a machine-readable medium, such as the storage unit908. In some embodiments, part or all of the computer program may be loaded and/or installed onto the device900via the ROM902and/or the communication unit909. When the computer program is loaded to the RAM903and executed by the computing unit901, one or more steps of the method200described above may be performed. Alternatively, in other embodiments, the computing unit901may be configured to implement the method200in any other suitable manners (such as by means of firmware).

Program code for implementing the method of the present disclosure may be written by using one programming language or any combination of a plurality of programming languages. The program code may be provided to a processor or controller of a general purpose computer, a special purpose computer, or another programmable data processing apparatus, such that the program code, when executed by the processor or controller, implements the functions/operations specified in the flow charts and/or block diagrams. The program code may be executed completely on a machine, executed partially on a machine, executed partially on a machine and partially on a remote machine as a stand-alone software package, or executed completely on a remote machine or server.

In the context of the present disclosure, a machine-readable medium may be a tangible medium that may include or store a program for use by an instruction execution system, apparatus, or device or in connection with the instruction execution system, apparatus, or device. The machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. The machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the above content. More specific examples of the machine-readable storage medium may include one or more wire-based electrical connections, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combinations thereof. Additionally, although operations are depicted in a particular order, this should be understood that such operations are required to be performed in the particular order shown or in a sequential order, or that all illustrated operations should be performed to achieve desirable results. Under certain environments, multitasking and parallel processing may be advantageous. Likewise, although the above discussion contains several specific implementation details, these should not be construed as limitations to the scope of the present disclosure. Certain features that are described in the context of separate embodiments may also be implemented in combination in a single implementation. Conversely, various features that are described in the context of a single implementation may also be implemented in a plurality of implementations separately or in any suitable sub-combination.

Although the present subject matter has been described using a language specific to structural features and/or method logical actions, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the particular features or actions described above. Rather, the specific features and actions described above are merely example forms of implementing the claims.