Patent Description:
A Flash Memory (flash memory) is a non-volatile memory and is marked by that data is not lost when power is turned off, and therefore, is widely used as an external memory and an internal memory. For example, a solid-state disk (Solid State Device, SSD) that is increasingly used in a computer system in recent years is implemented based on the flash memory. A solid-state disk also be called a solid-state drive (Solid State Drive, SSD). The flash memory is marked by the limited number of times of erasure. Each read/write operation (which also be called an erase operation) from/to the SSD wears the SSD to a certain extent.

To meet a requirement for massive data storage, it is generally required to form an SSD storage array with multiple SSDs. Wear leveling (Wear Leveling) is a concept proposed for the SSD storage array, and is essentially making wear conditions of all SSDs in the SSD storage array similar and preventing certain SSDs from being erased too frequently. However, the wear leveling result in a situation that multiple SSDs fail concurrently, thereby causing data loss. <CIT> relates to a method for wear level-based allocation in a storage pool. The method includes receiving a first request to write a first data item in a storage pool, where the storage pool includes a number of physical locations associated with the storage devices, and where each of the storage devices includes metadata regarding a level of wear of the storage device.

Embodiments of the present invention provide a method, an apparatus, and a controller for managing a storage array, so as to reduce a risk that multiple storage devices fail concurrently due to wear leveling.

A first aspect of the embodiments of the present invention provides a method for managing a storage array, where the storage array is formed by N storage devices, and the method for managing a storage array includes:.

In combination with the first aspect, in a first possible implementation manner, the dividing the storage array into a first storage device subset and a second storage device subset based on the degrees of wear of the N storage devices includes:.

In combination with the first aspect, in a second possible implementation manner, the dividing the storage array into a first storage device subset and a second storage device subset based on the degrees of wear of the N storage devices includes:.

In combination with the first aspect, in a third possible implementation manner, the dividing the storage array into a first storage device subset and a second storage device subset based on the degrees of wear of the N storage devices includes:.

The grouping the N storage devices into S subsections includes:.

In combination with the first aspect, in a fourth possible implementation manner, the dividing the storage array into a first storage device subset and a second storage device subset based on the degrees of wear of the N storage devices includes:
forming the first storage device subset with storage devices whose degrees of wear are greater than or equal to a third wear threshold, and forming the second storage device subset with storage devices whose degrees of wear are smaller than the third wear threshold.

In combination with the first aspect or any one of the first to the fourth possible implementation manners of the first aspect, in a fifth possible implementation manner, the migrating data from the second storage device subset to the first storage device subset includes:.

In combination with the first aspect or the second to the third possible implementation manners of the first aspect, in a sixth possible implementation manner, an amount of data added to each subsection in the first storage device subset is equal or starts to decrease progressively from the <NUM>st subsection; and an amount of data extracted from each subsection in the second storage device subset is equal or starts to decrease progressively from the Sth subsection.

In combination with the sixth possible implementation manner of the first aspect, in a seventh possible implementation manner, when the amount of data extracted from each subsection in the second storage device subset is equal, data of (FreeSize - FreeSizeA)/(N - X) is extracted from each storage device in the second storage device subset and migrated to the first storage device subset, where FreeSize indicates a free storage space in the first storage device subset before the data migration, FreeSizeA indicates a free storage space in the first storage device subset after the data migration, and X indicates the number of storage devices in the first storage device subset.

A second aspect provides an apparatus for managing a storage array, where the storage array is formed by N storage devices, and the apparatus includes:.

In combination with the second aspect, in a first possible implementation manner, the dividing module is configured to:.

In combination with the second aspect, in a second possible implementation manner, the dividing module is configured to:.

In combination with the second aspect, in a third possible implementation manner, the dividing module is configured to:.

In combination with the second aspect, in a fourth possible implementation manner, the dividing module is configured to:
form the first storage device subset with storage devices whose degrees of wear are greater than or equal to a third wear threshold, and form the second storage device subset with storage devices whose degrees of wear are smaller than the third wear threshold.

In combination with the second aspect or any one of the first to the fourth possible implementation manners of the second aspect, in a fifth possible implementation manner, the processing module is configured to:.

In combination with the second aspect or the second or the third possible implementation manner of the second aspect, in a sixth possible implementation manner, an amount of data added to each subsection in the first storage device subset is equal or starts to decrease progressively from the <NUM>st subsection; and an amount of data extracted from each subsection in the second storage device subset is equal or starts to decrease progressively from the Sth subsection.

In combination with the sixth possible implementation manner of the second aspect, in a seventh possible implementation manner, when the amount of data extracted from each subsection in the second storage device subset is equal, data of (FreeSize - FreeSizeA)/(N - X) is extracted from each storage device in the second storage device subset and migrated to the first storage device subset, where FreeSize indicates a free storage space in the first storage device subset before the data migration, FreeSizeA indicates a free storage space in the first storage device subset after the data migration, and X indicates the number of storage devices in the first storage device subset.

In combination with the second aspect or any one of the second to the seventh possible implementation manners of the second aspect, in an eighth possible implementation manner, the apparatus further includes a comparing module, configured to compare degrees of wear of storage devices in the second storage device subset with a fourth wear threshold; and
if a degree of wear of at least one storage device in the second storage device subset is greater than or equal to the fourth wear threshold, the processing module migrates the data from the second storage device subset to the first storage device subset.

A third aspect provides a controller, including:.

A fourth aspect provides a computer program product, including a computer-readable storage medium that stores a program code, where an instruction included in the program code is used to execute the method for managing a storage array described in the first aspect.

In the embodiments of the present invention, a storage array is divided into a first storage device subset and a second storage device subset based on degrees of wear of storage devices, where a minimum degree of wear of a storage device in the first storage device subset is greater than or equal to a maximum degree of wear of a storage device in the second storage device subset, and then, data in the second storage device subset is migrated to the first storage device subset or to-be-written data is written into the first storage device subset. Therefore, service lives of storage devices in the second storage device subset be extended relatively by shortening service lives of storage devices in the first storage device subset, thereby widening an interval between time when a storage device in the first storage device subset fails and time when a storage device in the second storage device subset fails, reducing a risk that multiple storage devices fail concurrently due to wear leveling, and improving data reliability.

In the following, the word "embodiment" is to be construed as an example unless covered by the claims. To illustrate the technical solutions in the embodiments of the present invention more clearly, the following briefly introduces accompanying drawings required for describing the embodiments or the prior art. Apparently, the accompanying drawings in the following description show merely some embodiments of the present invention, and a person of ordinary skill in the art still derive other drawings based on these accompanying drawings without creative efforts.

In the following, the word "embodiment" is to be construed as an example unless covered by the claims. To make the objectives, technical solutions, and advantages of the embodiments of the present invention more comprehensible, the following clearly and completely describes the technical solutions in the embodiments of the present invention with reference to the accompanying drawings in the embodiments of the present invention. Apparently, the embodiments to be described are merely a part rather than all of the embodiments of the present invention. All other embodiments obtained by a person of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection field of the present invention.

The embodiments of the present invention provide a method, an apparatus, and a controller for managing a storage array, which reduce a risk that multiple storage devices fail concurrently due to wear leveling. A read/write operation from/to a storage device in the embodiments of the present invention is called wear to the storage device. A failure of a storage device refers to wear-out of the storage device, and the storage device needs to be replaced.

A method for managing a storage array provided in this embodiment of the present invention is implemented in a storage system. <FIG> is a schematic diagram of system architecture of a method for managing a storage array based on an embodiment of the present invention. As shown in <FIG>, the storage system includes a controller <NUM> and storage devices <NUM>. In this embodiment, that the storage devices <NUM> are solid-state disks (Solid State Device, SSD) is taken as an example for description. A solid-state disk is also called a solid-state drive (Solid State Drive, SSD), and is called a disk for short.

<FIG> is merely for description rather than limiting a specific networking manner; for example, both cascading tree networking and ring networking are applicable, provided that the controller <NUM> and the storage devices <NUM> communicate with each other.

The controller <NUM> include any computing device known in the prior art, for example, a server or a desktop computer. An operating system and other applications are installed inside the controller. The controller <NUM> manage the storage devices <NUM>, for example, control data migration between the storage devices, and acquire degrees of wear of the storage devices. Because a flash memory is marked by the limited number of times of erasure, each read/write operation (which is also called an erase operation) from/to a storage device wears the storage device to a certain extent. A degree of wear is also called a wear extent, and is used to measure a service life of a storage device. The degree of wear is represented by a percentage.

The storage devices <NUM> include storage devices known in the prior art, for example, SSDs or direct access memories (Direct Access Storage Device, DASD). In <FIG>, that the storage devices <NUM> are SSDs is taken as an example for description. N physical SSDs form a storage array (storage array). A basic idea of the storage array is to combine multiple relatively inexpensive disks to deliver a performance equal to or even stronger than that of an expensive large-capacity disk. The storage array includes N physical SSDs, and each physical SSD has a unique number, for example, a physical SSD #<NUM>, a physical SSD #<NUM>, a physical SSD #<NUM>,. , and a physical SSD #N in the figure. In addition, the number N of physical SSDs in a storage array should not be smaller than a certain lower limit, for example, <NUM>; and the number N of physical SSDs in a storage array should not be greater than a certain upper limit, for example, <NUM>. Specifically, the N physical SSDs included in the storage array is SSDs with different models and different capacities, or SSDs with different models but a same capacity, or SSDs with a same model but different capacities, or SSDs with a same model and a same capacity.

Unless otherwise specified, SSDs in this embodiment of the present invention all refer to physical SSDs.

Each physical SSD is divided into fine-grained equal-sized chunks (Chunk, CK) <NUM>. A chunk is also called a logical SSD. To measure a capacity of each physical SSD, the number of logical SSDs obtained through dividing each physical SSD is called a weight of the physical SSD. For example, if all physical SSDs in a certain storage array are divided into logical SSDs of <NUM> MB each, a weight of a physical SSD with a capacity of <NUM> GB is <NUM>, and a weight of a physical SSD with a capacity of <NUM> GB is <NUM>.

In addition, multiple CKs form a logical space based on a redundant array of independent disks (Redundant Array of Independent Disks, RAID) with a specified type, and the logical space is a chunk group (Chunk Group, CKG). All CKs included in a CKG must belong to different SSDs. The redundant array of independent disks is also called a redundant array of inexpensive disks (Redundant Array of Inexpensive Disks, RAID).

A RAID <NUM> is taken as an example. At least three disks are needed for implementing a RAID <NUM> technology, data and corresponding parity information are stored on each of disks that form the RAID <NUM>, and the parity information and corresponding data are separately stored on different disks. When data on one disk of the RAID <NUM> is corrupted, remaining data and corresponding parity information is used to restore the corrupted data.

For example, if <NUM> physical SSDs form a storage array (also called a disk group), a capacity of each physical SSD is <NUM> GB and a capacity of a logical SSD is <NUM> MB, a weight of each physical SSD is <NUM>. Assume that logical SSDs in this storage array form a RAID <NUM>. For a specific forming method, reference is made to the following:.

The <NUM> physical SSDs in the storage array execute random_select(<NUM>, <NUM>), to obtain nine physical SSDs, where in a random_select(x, y) function, y ≤ x, indicating that y objects are randomly selected from x objects.

Each SSD among the nine physical SSDs executes random_select(F, <NUM>), where F indicates logical SSDs of each physical SSD that have not been selected to form a RAID, and random_select(F, <NUM>) indicates that one logical SSD is selected from the logical SSDs that have not been selected to form the RAID. Nine logical SSDs selected from the nine physical SSDs are recorded, and the nine logical SSDs form a RAID <NUM> with parity data. Meanwhile, a value of a variable RAID_NUM is updated, where RAID_NUM is the number of established RAIDs. A formed RAID is also called a formed RAID group.

In addition, a certain space in the storage array needs to be reserved for hot backup. The hot backup space is a storage space reserved in the storage array for restoring data. For example, if at least a storage space needs to be reserved for data reconstruction after a single disk fails, it needs to determine whether the data reconstruction is possibly performed in the reserved storage space after the single disk fails. If an unused storage space is larger than a capacity of the single disk, a new RAID group is further established. If an unused storage space is smaller than the capacity of the single disk, it indicates that another RAID group cannot be established and the unused storage space needs to be used as the hot backup space. The unused storage space, that is, a storage space that has not been actually used to bear a service, is also called a free storage space or a free space.

For the RAID_NUM RAID groups established in the preceding step, striping processing is performed, and each strip or strip unit is an independent storage resource. Each strip or several adjacent strips are put into a same storage resource pool as a storage resource. Then several storage blocks are randomly selected from the storage resource pool to form a storage space with a logical unit number (Logical Unit Number, LUN) and serve the controller. For details, see <FIG>.

In the storage array established by using the preceding method, services for each LUN is evenly distributed to each physical SSD, so that a service burden on each physical SSD matches a weight (capacity) of each physical SSD. If capacities of SSDs are different, an SSD with a larger capacity needs to bear relatively more services, and an SSD with a smaller capacity needs to bear relatively fewer services. A service burden is directly proportional to a capacity. Therefore, a difference between degrees of wear of all SSDs is small, which is called wear leveling.

As shown in <FIG>, logical SSDs with a same number form a RAID, and a logical SSD without a number indicates that the logical SSD has not been used. Based on the preceding method, services for each LUN is evenly distributed to each SSD. In addition, the larger the number of logical SSDs obtained through dividing each physical SSD is, the better a leveling effect is.

The following introduces a method for managing a storage array provided in an embodiment of the present invention. As shown in <FIG>, a flowchart of a method for managing a storage array based on an embodiment of the present invention is provided. That the storage array is formed by N storage devices is taken as an example for description, and the method includes:
S102: A controller acquires degrees of wear of the N storage devices.

A degree of wear is also called a wear extent, and is used to measure a service life of a storage device. The degree of wear is generally represented by a percentage. In this embodiment of the present invention, the degree of wear is represented by T.

The controller actively send a request command at a certain time interval, to request the storage devices in the storage array to tell the controller their degrees of wear; for example, the controller delivers a SMART command to the storage devices, to request the degrees of wear of the storage devices. The storage devices also actively tell the controller their degrees of wear at a certain time interval without basing on a request of the controller. Statistics about the degrees of wear of the N storage devices are also collected based on empirical values and service durations of the storage devices. This is not limited in the embodiment of the present invention.

The controller periodically acquires the degrees of wear of the N storage devices, and specifically, the controller acquires the degrees of wear of the N storage devices at a time interval such as one week or two weeks.

A degree of wear of a storage device is measured based on an actual volume of borne services.

Generally, the number N of the storage devices in the storage array should not be smaller than a certain lower limit, for example, <NUM>, and should not be greater than a certain upper limit, for example, <NUM>.

S104: Divide the storage array into a first storage device subset and a second storage device subset based on the degrees of wear of the N storage devices, where a minimum degree of wear of a storage device in the first storage device subset is greater than or equal to a maximum degree of wear of a storage device in the second storage device subset.

Certainly, when degrees of wear of M storage devices are greater than or equal to a preset wear threshold, an action of dividing the storage array into a first storage device subset and a second storage device subset based on the degrees of wear of the N storage devices (storage array division for short) is enabled, where <NUM> ≤ M ≤ N.

Specifically, M is set to <NUM>, indicating that as long as there is one storage device whose degree of wear exceeds the preset wear threshold, the storage array division is enabled.

Certainly, M is also set to <NUM> or any value not greater than N. In addition, M is dynamically adjusted through a reserved external interface. For example, M is first set to <NUM>, indicating that if it is found that a degree of wear of a certain storage device reaches the preset wear threshold for the first time, the storage array division is enabled. Subsequently, M is set to <NUM>, indicating that if it is found that degrees of wear of two storage devices reach the preset wear threshold, the storage array division is enabled.

Specifically, M is adjusted based on N and a difference between degrees of wear of storage devices in the second storage device subset. When the difference between the degrees of wear of the storage devices in the second storage device subset is relatively small, M is properly set to a relatively small value (for example, M is set to <NUM>). In this case, the storage array division is enabled in a relatively frequent manner. When the difference between the degrees of wear of the storage devices in the second storage device subset is relatively large, M is properly set to a relatively large value (for example, M is set to <NUM>). In this case, the storage array division is not enabled in such a frequent manner.

The preset wear threshold is preset based on an empirical value, and then dynamically adjusted through a reserved external interface. The preset wear threshold is represented by a percentage. If degrees of wear of all storage devices in the storage array are smaller than the preset wear threshold, in this scenario, all storage devices properly bear a service burden of the whole array based on weight values. If there is no performance bottleneck on a single disk, the storage array division is not performed.

Specifically, for how to divide the storage array into the first storage device subset and the second storage device subset based on the degrees of wear of the N storage devices, reference is made to descriptions of <FIG>, <FIG>, and <FIG>.

Specifically, it is the first storage device subset and the second storage device subset that form the whole storage array; and it is also the first storage device subset, the second storage device subset, and a third storage device subset that form the whole storage array, where data in the third storage device subset does not need to be adjusted, that is to say, data migration, including data migration-in or migration-out, is not performed.

In this embodiment of the present invention, a storage array formed by only storage devices for which data migration is required is understood as the storage array formed by the N storage devices.

S106: Migrate data from the second storage device subset to the first storage device subset; or write to-be-written data into the first storage device subset.

Specifically, for how to perform data migration, reference is made to descriptions of <FIG>, <FIG>, and <FIG>.

If an original address is accessed for each write operation, the data in the second storage device subset is migrated to the first storage device subset; and if a new address is accessed for each write operation, the to-be-written data is directly written into a free space in the first storage device subset, instead of migrating the data from the second storage device subset to the first storage device subset.

Accordingly, the first storage device subset is called a target storage device subset or a first subset; and the second storage device subset is called a source storage device subset or a second subset.

After a period of time, no degree of wear of a newly-added storage device is greater than or equal to the preset wear threshold, degrees of wear of storage devices in the second storage device subset are smaller than or equal to those of storage devices in the first storage device subset, services in the second storage device subset are evenly distributed to the storage devices in the second storage device subset, and services in the first storage device subset are evenly distributed to the storage devices in the first storage device subset; or after a period of time, the second storage device subset is blank, and all services are evenly distributed to storage devices in the first storage device subset. Even distribution refers to distribution based on weights.

<FIG> shows a diagram of an effect achieved by using the preceding method. Dark-colored areas in <FIG> indicate degrees of wear, and the higher a ratio that a dark-colored area occupies is, the higher a degree of wear is.

In this embodiment of the present invention, a storage array is divided into a first storage device subset and a second storage device subset based on degrees of wear of storage devices, where a minimum degree of wear of a storage device in the first storage device subset is greater than or equal to a maximum degree of wear of a storage device in the second storage device subset, and then, data in the second storage device subset is migrated to the first storage device subset. Therefore, service lives of storage devices in the second storage device subset is extended relatively by shortening service lives of storage devices in the first storage device subset, thereby widening an interval between time when a storage device in the first storage device subset fails and time when a storage device in the second storage device subset fails, reducing a risk that multiple storage devices fail concurrently due to wear leveling, and improving data reliability.

If a new address is accessed for each write operation, to-be-written data is directly written into a free space in the first storage device subset, instead of migrating the data from the second storage device subset to the first storage device subset. Services borne by the storage devices in the second storage device subset are maintained by increasing services borne by the storage devices in the first storage device subset, and the service lives of the storage devices in the second storage device subset are extended relatively by shortening the service lives of the storage devices in the first storage device subset, thereby widening an interval between time when a storage device in the first storage device subset fails and time when a storage device in the second storage device subset fails, reducing a risk that multiple storage devices fail concurrently due to wear leveling, and improving data reliability.

In this embodiment, the risk that multiple storage devices fail concurrently due to wear leveling is reduced by widening the interval between the time when a storage device in the first storage device subset fails and the time when a storage device in the second storage device subset fails. Therefore, the method described in this embodiment is also called a non-wear leveling method or an anti-wear leveling method. Accordingly, in this embodiment of the present invention, the action of dividing the storage array into a first storage device subset and a second storage device subset based on the degrees of wear of the storage devices are also called an action of enabling anti-wear leveling.

Specifically, before S106 is performed, the method further include:
S105: Compare the degrees of wear of the storage devices in the second storage device subset with a fourth wear threshold; that is, determine whether the degrees of wear of all storage devices in the second storage device subset are smaller than the fourth wear threshold. If the degrees of wear of all storage devices in the second storage device subset are smaller than the fourth wear threshold, S106 is no longer performed. That is to say, a service burden on the storage devices in the second storage device subset is not heavy, and therefore, an action of anti-wear leveling is temporarily not needed; or after the data is migrated by using the management method described in this embodiment of the present invention, a service burden on the storage devices in the second storage device subset is not heavy, and therefore, an action of anti-wear leveling is temporarily not needed.

If a degree of wear of at least one storage device in the second storage device subset is greater than or equal to the fourth wear threshold, step <NUM> is performed.

The fourth wear threshold is the same as or different from the preset wear threshold described in the preceding embodiment.

In this embodiment of the present invention, a frequency for anti-wear leveling adjustment is reduced by comparing the degrees of wear of the storage devices in the second storage device subset with the fourth wear threshold.

Specifically, <FIG>, <FIG>, and <FIG> are taken as examples to describe in detail how to divide the storage array into a first storage device subset and a second storage device subset based on the degrees of wear of the N storage devices and how to migrate data from the second storage device subset to the first storage device subset. Certainly, the present invention is not limited to the methods shown in <FIG>, <FIG>, and <FIG>.

As shown in <FIG>, the following steps are included:
S301: Sequence the N storage devices based on the degrees of wear in descending order, where a storage device with a maximum degree of wear is numbered <NUM>, and a storage device with a minimum degree of wear is numbered N.

Specifically, the sequencing is performed inside a memory of a controller device.

The sequencing is performed based on the degrees of wear in descending order or in ascending order, as long as it help correctly complete division of storage device subsets. This is not limited in the embodiment of the present invention.

In this embodiment of the present invention, the sequencing performed in descending order is taken as an example for description.

S302: Calculate a difference △T between degrees of wear of an ith storage device and an (i+<NUM>)th storage device, where <NUM> < i < N;.

Specifically, the first wear threshold is determined based on service lives of the storage devices and a predicted time interval between time when two storage devices fail. There are multiple methods for determining the first wear threshold. A failure of a storage device refers to wear-out of the storage device, and the storage device needs to be replaced. If it is expected that two storage devices do not fail concurrently and an expected time interval between time when the two storage devices fail is <NUM> to <NUM> weeks, for example, <NUM> weeks, the first wear threshold is <NUM>%*<NUM> ≈ <NUM>%. It is assumed that <NUM>% is a weekly average degree of wear of a storage device and discovered through collecting statistics in an actual service. If the expected time interval between the time when the two storage devices fail is <NUM> days, for example, <NUM> weeks, the first wear threshold is <NUM>%*<NUM> ≈ <NUM>%. A specific expected time interval between the time when the two storage devices fail is determined by comprehensively considering a time interval for locally replacing a storage device.

Table <NUM> is taken as an example for description, and it is assumed that N = <NUM>, M = <NUM>, the preset wear threshold is <NUM>%, and the first wear threshold is <NUM>%.

The controller detects that degrees of wear of two storage devices in the storage array formed by <NUM> storage devices exceed <NUM>%, and starting from a storage device with a maximum degree of wear, calculation of the difference △T between the degrees of wear is performed. Ti indicates a degree of wear of the ith storage device after the sequencing. First, a difference between degrees of wear is calculated from a <NUM>st storage device and a <NUM>nd storage device after the sequencing, and △T<NUM> = T<NUM> - T<NUM> = <NUM>% - <NUM>% = <NUM>%. Because <NUM>% is greater than <NUM>%, step <NUM> is performed.

In Table <NUM>, an actual slot number in the first row is an actual physical number of a storage device in the storage array;.

S303: Take the <NUM>st to the ith storage devices as the first storage device subset, and take the (i+<NUM>)th to the Nth storage devices as the second storage device subset.

S304: Add <NUM> to i, and continue to calculate a difference between degrees of wear of an ith storage device and an (i+<NUM>)th storage device until a difference between degrees of wear is smaller than or equal to the first wear threshold or all the N storage devices are traversed.

In the example shown in Table <NUM>, because the difference between the degrees of wear of the <NUM>st storage device and the <NUM>nd storage device after the sequencing is greater than the first wear threshold, i plus <NUM> equals <NUM>, and calculation of a difference between degrees of wear of the <NUM>nd storage device and a <NUM>rd storage device is started. △T<NUM> = T<NUM> - T<NUM> = <NUM>% - <NUM>% = <NUM>%. Because <NUM>% is greater than <NUM>%, calculation is continued after i plus <NUM> equals <NUM>. In this case, calculation of a difference between degrees of wear of the <NUM>rd storage device and a <NUM>th storage device is started. △T<NUM> = T<NUM> - T<NUM> = <NUM>% - <NUM>% = <NUM>%. Because <NUM>% is smaller than <NUM>%, no calculation of a difference between degrees of wear of storage devices sequenced after the <NUM>th storage device is continued, and the traversal ends.

In this case, the <NUM>st to the ith storage devices are taken as the first storage device subset, and the (i+<NUM>)th to the Nth storage devices are taken as the second storage device subset. In the example shown in Table <NUM>, the <NUM>st to the <NUM>rd storage devices are taken as the first storage device subset subset1, and the <NUM>th to a <NUM>th storage devices are taken as the second storage device subset subset2.

The first storage device subset and the second storage device subset are obtained through division in the preceding step <NUM> or <NUM>. A characteristic of the first storage device subset is that: Differences between degrees of wear of all storage devices exceed the first wear threshold. A characteristic of the second storage device subset is that: A difference between a degree of wear of a storage device with the maximum degree of wear in the second storage device subset and that of a storage device with the minimum degree of wear in the first storage device subset does not exceed the first wear threshold.

For example, in the example shown in Table <NUM>, differences between degrees of wear of all storage devices in the first storage device subset subset1 exceed <NUM>%, and the difference between the degree of wear of the storage device with the maximum degree of wear (the <NUM>th storage device after the sequencing) in the second storage device subset subset2 and that of the storage device with the minimum degree of wear (the <NUM>rd storage device after the sequencing) in the first storage device subset subset1 does not exceed <NUM>%.

Generally, degrees of wear of all storage devices in a same storage array should be close to each other. For example, a difference between degrees of wear at a same time point does not exceed <NUM>%. Table <NUM> shows a case in which a difference between degrees of wear is relatively large, which is caused by replacing a storage device of an individual slot for a certain reason or adding a new storage device into the storage array when the storage array is in use.

If no difference of degrees of wear between the ith storage device and the (i+<NUM>)th storage device that is smaller than or equal to the first wear threshold is found even after the traversal of the N storage devices ends, distribution of service data (data for short) is re-adjusted based on a weight of each storage device, thereby achieving even distribution based on weights. That is to say, after the differences between the degrees of wear of the storage devices in the first storage device subset and those between the degrees of wear of the storage devices in the second storage device subset are widened, an even state is restored again. Then, after a period of time (for example, one week), the process is restarted from S102.

In the preceding method, it is assumed that the number of storage devices in the first storage device subset is X, and the number of storage devices in the second storage device subset is N - X. The following describes a specific method for migrating data from the second storage device subset to the first storage device subset. Certainly, a migration method is not limited to that described in the following embodiment.

S305: Collect statistics about a free storage space in the first storage device subset; that is, collect statistics about how many Chunks are still in an idle (unused) state in total in the first storage device subset. If a RAID group is established, it is understood that a Chunk that has not been used to establish the RAID group is in the idle state. It is assumed that a statistical result is represented by FreeSize.

S306: Taking that storage devices are SSDs as an example for description, averagely extract data from FreeSize/(N - X) logical SSDs of each physical SSD in the second storage device subset, and migrate the data to physical SSDs in the first storage device subset, thereby making each physical SSD in the first storage device subset to operate in a fully loaded manner. It is understood that a fully loaded operation is that every Chunk of a storage device is occupied and no space is reserved for hot backup. In this way, after a period of time, degrees of wear of all physical SSDs in the first storage device subset is higher than those of all physical SSDs in the second storage device subset due to a difference between service burdens.

Certainly, a data migration method described in another embodiment is also used.

In addition, if it is expected that a difference between a minimum degree of wear of a physical SSD in the first storage device subset and a maximum degree of wear of a physical SSD in the second storage device subset is widened rapidly, more data further be extracted from the physical SSD with the maximum degree of wear in the second storage device subset into a physical SSD in the first storage device subset.

For example, in the example shown in Table <NUM>, there are <NUM> storage devices in the first storage device subset, and there are <NUM> - <NUM> = <NUM> storage devices in the first storage device subset. Service data is averagely extracted from FreeSize/<NUM> logical SSDs and migrated to the <NUM> storage devices in the first storage device subset. To rapidly widening a difference between degrees of wear of SSDs with actual slot numbers <NUM> and <NUM>, more service data with the actual slot number <NUM> is extracted (for example, service data of <NUM> to <NUM> more Chunks is extracted) and migrated to the <NUM> storage devices in the first storage device subset.

Specifically, if a RAID is formed, when data in the second storage device subset is migrated to the first storage device subset, it is required that an SSD that is in the first storage device subset and to which the data is migrated is different from an SSD that is in the first storage device subset and included in the RAID. The objective is to ensure that each Chunk included in a RAID belongs to a different disk, thereby avoiding a case in which data of two CKs are lost due to a failure or power outage of a disk.

Because the service burden is gradually migrated, service performance of a whole system is not affected.

After step S306 is performed, the method further include:
S308: After a certain time interval, if it is found through calculation that a difference between a minimum degree of wear of a physical SSD in the first storage device subset and a maximum degree of wear of a physical SSD in the second storage device subset is already greater than or equal to the first wear threshold, restart to perform step S102.

A method shown in <FIG> is also used to describe how to divide the storage array into a first storage device subset and a second storage device subset based on the degrees of wear of the N storage devices and how to migrate data from the second storage device subset to the first storage device subset.

As shown in <FIG>, the following steps are included:
S401: Group the N storage devices into S subsections subsection based on the degrees of wear of the N storage devices, where a minimum degree of wear of a storage device in a jth subsection is greater than or equal to a maximum degree of wear of a storage device in a (j+<NUM>)th subsection, and <NUM> < j < S.

Differences between degrees of wear of storage devices in each subsection all exceed a second wear threshold. For determining of the second wear threshold, reference is made to the first wear threshold, and the second wear threshold is the same as or different from the first wear threshold. In addition, a difference between a degree of wear of a storage device with the maximum degree of wear in the (j+<NUM>)th subsection and that of a storage device with the minimum degree of wear in a storage device set in the jth subsection does not exceed the second wear threshold.

Specifically, a method for grouping into S subsections is as follows:.

Specifically, for a method for determining the second wear threshold, reference is made to a description of the embodiment of <FIG>.

S4022: Group the ith storage device into one subsection, and group the (i+<NUM>)th storage device into another subsection; then add <NUM> to i, and continue to perform step S4021.

S4023: Group the (i+<NUM>)th storage device into a subsection to which the ith storage device belongs; then add <NUM> to i, and continue to perform step S4021.

The storage array is divided into S subsections by using the preceding method.

Table <NUM> is taken as an example for description: It is assumed that N = <NUM>, M = <NUM>, a preset wear threshold is <NUM>%, and the second wear threshold is the same as the first wear threshold, that is, <NUM>%.

First, calculation of a difference between degrees of wear is performed first from a <NUM>st storage device and a <NUM>nd storage device after the sequencing, and △T<NUM> = T<NUM> - T<NUM> = <NUM>% - <NUM>% = <NUM>%. Because <NUM>% is greater than <NUM>%, step <NUM> is performed. The <NUM>nd storage device and the <NUM>st storage device are grouped into a same subsection, and then a <NUM>st subsection subsection1 is {<NUM>,<NUM>}. Then, i plus <NUM> equals <NUM>, and calculation of a difference between degrees of wear of the <NUM>nd storage device and a <NUM>rd storage device is started. △T<NUM> = T<NUM> - T<NUM> = <NUM>% - <NUM>% = <NUM>%. Because <NUM>% is greater than <NUM>%, the <NUM>rd storage device and <NUM>nd storage device are grouped into a same subsection, and then a <NUM>st subsection subsection1 is {<NUM>, <NUM>, <NUM>}. Then, calculation of a difference between degrees of wear of the <NUM>rd storage device and a <NUM>th storage device is started. △T<NUM> = T<NUM> - T<NUM> = <NUM>% - <NUM>% = <NUM>%. Because <NUM>% is smaller than <NUM>%, the <NUM>th storage device is grouped into a <NUM>nd subsection. Then, i plus <NUM> equals <NUM>, and calculation of a difference between degrees of wear of the <NUM>th storage device and a <NUM>th storage device is started. △T<NUM> = T<NUM> - T<NUM> = <NUM>% - <NUM>% = <NUM>%. Because <NUM>% is greater than <NUM>%, the <NUM>th storage device and the <NUM>th storage device are grouped into a same subsection, and then a <NUM>nd subsection subsection2 is {<NUM>, <NUM>}. Then, calculation of a difference between degrees of wear of the <NUM>th storage device and a <NUM>th storage device is started. △T<NUM> = T<NUM> - T<NUM> = <NUM>% - <NUM>% = <NUM>%. Because <NUM>% is smaller than <NUM>%, the <NUM>th storage device is grouped into a <NUM>rd subsection subsection3. Then, calculation of a difference between degrees of wear of the <NUM>th storage device and a <NUM>th storage device is started until all storage devices are traversed.

S402: Calculate the number of storage devices in the <NUM>st subsection to the jth subsection, where <NUM> < j < S; if j = <NUM>, directly calculate the number of storage devices in the <NUM>st subsection;.

S403: Take the storage devices in the <NUM>st subsection to the jth subsection as a first storage device subset, and take the storage devices in the (j+<NUM>)th subsection to an Sth subsection as a second storage device subset.

S404: Then, add <NUM> to j, and continue to calculate the number of storage devices in the <NUM>st subsection to a jth subsection until the number of the storage devices in the <NUM>st subsection to the jth subsection is greater than or equal to N/<NUM>, take the storage devices in the <NUM>st subsection to the jth subsection as the first storage device subset, and take the storage devices in a (j+<NUM>)th subsection to the Sth subsection as the second storage device subset.

When data in the second storage device subset is migrated to the first storage device subset, an amount of data added to each subsection in the first storage device subset is equal or starts to decrease progressively from the <NUM>st subsection, and an amount of data extracted from each subsection in the second storage device subset is equal or starts to decrease progressively from the Sth subsection. A specific method is as follows:
S405: Collect statistics about a free storage space in the first storage device subset; that is, collect statistics about how many Chunks are still in an idle (unused) state in total in the first storage device subset. If a RAID group is established, it is understood that a Chunk that has not been used to establish the RAID group is in the idle state. It is assumed that a statistical result is represented by FreeSize.

It is assumed that the number of storage devices in the first storage device subset is X and the number of storage devices in the second storage device subset is N - X. An average free space of all storage devices in the first storage device subset is FreeSize/X.

S406: The first subset includes j subsections. It is assumed that data of FreeSize - FreeSizeA in total in the second subset is migrated to the first subset, where FreeSizeA indicates a free storage space in the first subset after the data migration, and data of (FreeSize - FreeSizeA)/(N - X) is extracted from each storage device in the second subset and migrated to the first subset.

After the data migration, a free storage space of each subsection in the first subset is: <MAT>.

For example, the first subset includes three subsections, and after the data migration:.

After the data migration, free storage spaces of all subsections in the first subset total FreeSizeA.

S501 is the same as S401, and is not repeatedly described herein.

S502: <MAT>, indicating rounding S/<NUM> down, take storage devices in a <NUM>st subsection to the jth subsection as a first storage device subset, and take storage devices in the (j+<NUM>)th subsection to an Sth subsection as a second storage device subset.

Table <NUM> is still taken as an example for description. It is assumed that S = <NUM>, <MAT> is used, storage devices in the <NUM>st subsection and a <NUM>nd subsection are taken as the first storage device subset, and storage devices in a <NUM>rd subsection to a <NUM>th subsection are taken as the second storage device subset.

Alternatively, <MAT>, indicating rounding S/<NUM> up, take storage devices in the <NUM>st subsection to the jth subsection as the first storage device subset, and take storage devices in the (j+<NUM>)th subsection to the Sth subsection as the second storage device subset.

Accordingly, a specific method for migrating data from the second storage device subset to the first storage device subset is as follows:
S505: Collect statistics about a free storage space in the first storage device subset; that is, collect statistics about how many Chunks are still in an idle (unused) state in total in the first storage device subset. If a RAID group is established, it is understood that a Chunk that has not been used to establish the RAID group is in the idle state. It is assumed that a statistical result is represented by FreeSize.

It is assumed that the number of storage devices in the first storage device subset is X and the number of storage devices in the second storage device subset is N - X.

S506: The first subset includes j subsections. It is assumed that data of FreeSize- FreeSizeA in total in the second subset is migrated to the first subset, where FreeSizeA indicates a free storage space in the first subset after the data migration, and service data of (FreeSize- FreeSizeA)/(N-X) is extracted from each storage device in the second subset and migrated to the storage devices in the first subset.

An uneven migration solution is also used to migrate data from the second storage device subset to the first storage device subset; that is, service data added to each subsection in the first subset starts to decrease progressively from the <NUM>st subsection, and service data extracted from each subsection in the second subset starts to decrease progressively from the Sth subsection.

Table <NUM> is taken as an example for description. Under a precondition that service data migrated out of a second storage memory equals that migrated into a first storage memory: If storage devices in the <NUM>st subsection and the <NUM>nd subsection are taken as the first storage device subset, service data in the <NUM>st subsection is increased by <NUM>% (that is, all free spaces are occupied), and service data in the <NUM>nd subsection is increased by <NUM>% (that is, only <NUM>% of all free spaces are occupied); and if storage devices in the <NUM>rd subsection to the <NUM>th subsection are taken as the second storage device subset, in the second storage device subset, service data in the <NUM>th subsection is reduced by <NUM>% (<NUM>% of all data is reduced), service data in a <NUM>th subsection is reduced by <NUM>% (<NUM>% of all data is reduced), and service data in the <NUM>rd subsection is reduced by <NUM>% (<NUM>% of all data is reduced).

Certainly, a method for how to divide the storage array into a first storage device subset and a second storage device subset based on the degrees of wear of the N storage devices is not limited to that shown in <FIG>, <FIG>, or <FIG>. For example, a third wear threshold is set; and after degrees of wear are sequenced, storage devices whose degrees of wear are greater than or equal to the third wear threshold form the first storage device subset, and storage devices whose degrees of wear are smaller than the third wear threshold form the second storage device subset. The third wear threshold is preset based on an empirical value and adjusted, and is the same as or different from the preset wear threshold described in the preceding embodiment.

In this embodiment of the present invention, a storage array is divided into a first storage device subset and a second storage device subset based on degrees of wear of storage devices, where a minimum degree of wear of a storage device in the first storage device subset is greater than or equal to a maximum degree of wear of a storage device in the second storage device subset, and then, data in the second storage device subset is migrated to the first storage device subset or to-be-written data is written into the first storage device subset. Therefore, service lives of storage devices in the second storage device subset is extended relatively by shortening service lives of storage devices in the first storage device subset, thereby widening an interval between time when a storage device in the first storage device subset fails and time when a storage device in the second storage device subset fails, reducing a risk that multiple storage devices fail concurrently due to wear leveling, and improving data reliability.

All introduced in the foregoing are methods for dynamic adjustment. Certainly, if each storage device in the storage array is allocated a weight directly and capacities of all storage devices are the same, a different weight is allocated to each storage device to ensure that a weight value of each storage device is different.

Alternatively, when each storage device in the storage array is allocated the weight, it is ensured that capacities of all storage devices are different. However, a same weight is allocated to each storage device D. This makes degrees of wear of all storage devices uneven.

The method for global anti-wear leveling is implemented on various data protection models (including but not limited to: no data protection, mirror-based data protection, parity-based data protection, and the like) in the present invention, and that storage devices are SSDs is taken as an example for description.

The following takes parity-based data protection as an example for description, for example, a RAID <NUM> or a RAID <NUM>.

When physical SSDs in the SSD group all wear to a certain extent, the system selects some physical SSDs from the physical SSDs to bear more services, and accordingly, service volumes borne on other physical SSDs consequentially decrease. For example, in a RAID <NUM> system shown in <FIG> and <FIG>, data is migrated from a <NUM>th logical SSD of a physical SSD #<NUM> to a logical SSD of a physical SSD #<NUM>.

Through the preceding steps, the physical SSD #<NUM> bears a larger volume of services and wears at a higher speed, while the physical SSD #<NUM> bears a smaller volume of services and wears at a lower speed. After a period of time, degrees of wear of all physical SSDs form a staircase shape similar to that shown in <FIG>, thereby preventing multiple physical SSDs from failing concurrently, and improving system reliability.

The following further takes implementation in a mirror-based data model as an example for description, for example, a RAID <NUM> or a RAID <NUM>.

When physical SSDs in the SSD group all wear to a certain extent, the system selects some physical SSDs from the physical SSDs to bear more services, and accordingly, service volumes borne on other physical SSDs consequentially decrease. For example, in a RAID <NUM> system shown in <FIG>, data is migrated from some logical SSDs of physical SSDs #<NUM> and #<NUM> to logical SSDs of a physical SSDs #<NUM> and a physical SSD #<NUM>.

Through the preceding steps, the physical SSD #<NUM> and the physical SSD #<NUM> bear a larger volume of services and wear at a higher speed, while the physical SSDs #<NUM> and #<NUM> bear a smaller volume of services (where a volume of services borne by #<NUM> becomes much smaller than that borne by #<NUM>) and wear at a lower speed. After a period of time, degrees of wear of all physical SSDs form a staircase shape similar to that shown in <FIG>, thereby preventing multiple physical SSDs from failing concurrently, and improving system reliability.

The preceding procedure describes various scenarios where this embodiment of the present invention applies, and certainly, the present invention is not limited to these scenarios.

This embodiment of the present invention provides an apparatus for managing a storage array. The storage array is formed by N storage devices, and as shown in <FIG>, the apparatus includes:.

Specifically, the dividing module is configured to:.

Alternatively, the dividing module is configured to:.

Alternatively, the dividing module is configured to:
form the first storage device subset with storage devices whose degrees of wear are greater than or equal to a third wear threshold, and form the second storage device subset with storage devices whose degrees of wear are smaller than the third wear threshold.

Accordingly, the processing module is configured to:.

An amount of data added to each subsection in the first storage device subset is equal or starts to decrease progressively from the <NUM>st subsection; and an amount of data extracted from each subsection in the second storage device subset is equal or starts to decrease progressively from the Sth subsection.

The processing module is configured to, when the amount of data extracted from each subsection in the second storage device subset is equal, extract data of (FreeSize - FreeSizeA)/(N - X) from each storage device in the second storage device subset and migrate data to the first storage device subset, where FreeSize indicates a free storage space in the first storage device subset before the data migration, FreeSizeA indicates a free storage space in the first storage device subset after the data migration, and X indicates the number of storage devices in the first storage device subset.

Alternatively, the apparatus further includes a comparing module <NUM>, configured to compare degrees of wear of storage devices in the second storage device subset with a fourth wear threshold; and
if a degree of wear of at least one storage device in the second storage device subset is greater than or equal to the fourth wear threshold, the processing module migrates the data from the second storage device subset to the first storage device subset.

The apparatus provided in this embodiment of the present invention is disposed in the controller described in the preceding embodiment and is configured to perform the method for managing a storage array described in the preceding embodiment. For a detailed description about functions of each unit, reference is made to the description in the method embodiment, and details are not repeatedly described herein.

As shown in <FIG>, a controller provided in an embodiment of the present invention includes:
a processor <NUM>, a memory <NUM>, a system bus (a bus for short) <NUM>, and a communications interface <NUM>, where the processor <NUM>, the memory <NUM>, and the communications interface <NUM> connect to and communicate with each other through the system bus <NUM>.

The processor <NUM> is a single-core or multi-core central processing unit or a specific integrated circuit, or is configured to one or more integrated circuits implementing this embodiment of the present invention.

The memory <NUM> is a high-speed RAM memory or a non-volatile memory (non-volatile memory), for example, at least one disk memory.

The communications interface <NUM> is configured to communicate with a storage device.

The memory <NUM> is configured to store a computer execution instruction <NUM>. Specifically, the computer execution instruction <NUM> include a program code.

When a computer is running, the processor <NUM> runs the computer execution instruction <NUM> and perform any one of the method processes based on <FIG>.

An embodiment of the present invention further provides a computer program product for data processing, including a computer-readable storage medium that stores a program code, where an instruction included in the program code is used to perform any one of the method processes based on <FIG>.

A person of ordinary skill in the art understand that each aspect of the present invention or possible implementation manners of each aspect is specifically implemented as a system, a method, or a computer program product. Therefore, each aspect of the present invention or possible implementation manners of each aspect use a form of a complete hardware embodiment, a complete software embodiment (including firmware, resident software, and the like), or an embodiment combined with software and hardware, and they are collectively called "circuit", "module", or "system" herein. In addition, each aspect of the present invention or possible implementation manners of each aspect use a form of a computer program product. The computer program product refers to a computer-readable program code stored in a computer-readable medium.

The computer-readable medium is a computer-readable signal medium or a computer-readable storage medium. The computer-readable storage medium includes, but is not limited to, an electronic, a magnetic, an optical, an electromagnetic, an infrared, or a semi-conductive system, device, or apparatus, or any appropriate combination of the foregoing, such as a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, or a portable read-only memory (CD-ROM).

A processor of a computer reads the computer-readable program code stored in the computer-readable medium, so that the processor execute function actions specified in each step or a combination of steps in a flowchart; and generates an apparatus to implement function actions specified in each block or a combination of blocks in a block diagram.

The computer-readable program code is completely executed on a computer of a user, partially executed on a computer of a user, function as an independent software package, partially executed on a computer of a user and partially executed on a remote computer, or completely executed on a remote computer or a server. It should also be noted that, in some alternative implementation solutions, functions indicated in each step of the flowchart or in each block of the block diagram not occur in a sequence indicated in the flowchart or the diagram. For example, depending on involved functions, two steps or two blocks shown one after another is actually executed at nearly the same time, or sometimes, the blocks are executed in a converse sequence.

A person of ordinary skill in the art is aware that, units and algorithm steps of each example described in combination with the embodiments disclosed herein is implemented by electronic hardware, or a combination of computer software and electronic hardware. Whether the functions are performed in a hardware or software manner depends on particular applications and design constraint conditions of the technical solutions. A person skilled in the art use different methods to implement the described functions for each particular application, but it should not be considered that the implementation goes beyond the protection field of the present invention.

Claim 1:
A method, implemented by a controller, for managing a storage system, the method comprises:
determining (<NUM>) degree of wear of each of physical storage devices in a storage system is greater than or equal to a preset wear threshold;
migrating (<NUM>) data from a storage device with degree of wear lower than the preset wear threshold in the physical storage devices to a storage device with degree of wear greater than or equal to the preset wear threshold in the physical storage devices, or writing to-be-written data into the storage device with degree of wear greater than or equal to the preset wear threshold in the physical storage devices without writing the to-be-written data into the storage device with degree of wear lower than the preset wear threshold in the physical storage devices.