Patent Description:
In actual application, data migration frequently needs to be performed between different storage arrays in a storage system. For example, in the storage system, a quantity of SSD disks and a capacity of the SSD disks are large, and therefore there is a relatively high failure probability. To avoid a data loss, a to-be-invalidated SSD, that is, an SSD predicted to be invalidated, needs to be identified in advance, and then data in the SSD predicted to be invalidated is migrated to an SSD of another storage array in a cluster for backup.

In the prior art, when data is migrated between two storage arrays connected by using a network, memory space that is in a memory of a source storage array and that is dedicated to data migration first needs to be allocated, and then a controller of the source storage array reads data in a source SSD (an SSD from which the data is migrated, for example, a detected SSD predicted to be invalidated) into the memory space, and further sends a migration command to a network interface card of the source storage array. Then, the network interface card of the source storage array forwards the request to a network interface card of a target storage array. Further, the network interface card of the target storage array informs a controller of the target storage array. The controller of the target array needs to allocate a memory segment that is dedicated to data migration, and after the memory segment is allocated, the controller of the target storage array instructs the network interface card of the target storage array to migrate the data from the memory of the source storage array to a memory of the target storage array. The controller of the target storage array further writes the data in the memory of the target storage array into a flash memory of the target array.

It may be learned that, in the prior art, when data migration is performed between the storage arrays in the storage system, the migrated data needs to be migrated by using the controller of the source storage array, thereby occupying bandwidth of the controller of the source storage array and further occupying the memory of the source storage array and the memory of the target storage array. Therefore, normal access to data in another SSD in the array is affected, and performance of the entire storage system is affected. To reduce impact of data migration on the performance of the storage system, the bandwidth of the controller that is occupied during data migration is usually limited. This causes excessively long time of data migration, and increases a risk of a data loss. In addition, even if the bandwidth for data migration is limited, the performance of the storage system is still affected.

<CIT>discloses a management computer configured to: receive indication information which indicates creating a logical partition and which includes information on performance for processing an input/output instruction required by the logical partition to be created; identify a computer resource newly allocatable to a logical partition in each of a plurality of physical storage apparatuses; and issue, when there is no physical storage apparatus capable of independently creating the logical partition, to the plurality of physical storage apparatuses an indication to create a logical partition which straddles a plurality of physical storage apparatuses and to which a computer resource for satisfying the performance for pro-cessing an input/output instruction and a computer resource for transferring an input/output instruction among the physi-cal storage apparatuses are allocated.

<CIT> dislcoses an intelligent network interface card (INIC) or communication processing device (CPD) working with a host computer for data communication. The device provides a fast-path that avoids protocol processing for most messages, greatly accel-erating data transfer and offloading time-intensive process-ing tasks from the host CPU. The host retains a fallback processing capability for messages that do not fit fast-path criteria, with the device providing assistance such as vali-dation even for slow-path messages, and messages being selected for either fast-path or slow-path processing. A context for a connection is defined that allows the device to move data, free of headers, directly to or from a destination or source in the host. The context can be passed back to the host for message processing by the host. The device contains specialized hardware circuits that are much faster at their specific tasks than a general purpose CPU. When the context is held by the INIC, the INIC can advertise a receive window of memory space at the destination that corresponds to the context and is available to store data.

Embodiments of the present invention provide a method for migrating data between storage arrays. During data migration, no controller bandwidth or memory of a source storage array and a target storage array is occupied, to ensure a data migration speed without affecting performance of a storage system. The invention is defined as in the appended claims.

A first aspect of the embodiments of the present invention provides a method for migrating data between storage devices. A first storage device includes a source controller, a first solid state storage disk, SSD, and a first network interface card including a first migration cache, the second storage device includes a second solid state storage disk, SSD, and a second network interface card including a second migration cache, and the method includes: receiving, by the first network interface card, a first migration instruction sent by the source controller, wherein the first migration instruction includes an address of the data stored in the first SSD disk; in response to the receiving of the first migration instruction obtaining, by the first network interface card, the data from the first SSD disk based on the address and reading the data into the first migration cache; sending, by the first network interface card, a second migration instruction to the second network interface card, the second migration instruction comprises access information of the first migration cache and an address of the second SSD, the access information of the first migration cache comprises information about the data stored in the migration cache; in response to the sending of the second migration instruction reading, by the second network interface card, the data into the second migration cache of the second network interface card; writing, by the second network interface card, the data to the second SSD disk.

In the foregoing method for migrating data, when the data in the source storage array is migrated to the target storage array, in a data migration process, no controller bandwidth or memory of the source storage array is occupied, so that performance of the source storage array is not affected. In addition, the data migration process is not limited by the controller bandwidth of the source storage array, and therefore, a data migration speed is increased.

Further, the access information includes a memory address allocated by a controller of the source storage array to the source migration cache.

The memory address is allocated, so that the source SSD can access the source migration cache of the source intelligent network interface card.

Further, the information about the target SSD includes an IP address of the target SSD, and the IP address of the target SSD is an IP address allocated by the source intelligent network interface card to the target SSD when a connection is established between the target storage array and the source storage array. In this way, the target intelligent network interface card may be determined based on the IP address that is in the information about the target SSD and that is of the target SSD, to send the second migration instruction to the target intelligent network interface card.

In an IP address mapping manner, the SSD of the target storage array may be mapped to the source storage array, and the source storage array may find the target intelligent network interface card based on the IP address of the SSD, to transmit data.

Further, the source intelligent network interface card stores a mapping relationship between the IP address of the target SSD and a disk identifier of the target SSD; and the information about the target SSD further includes a logical address for storing the to-be-migrated data in the target SSD. In this case, when the address of the target SSD is determined based on the information about the target SSD, the disk identifier of the target SSD may be first determined based on the mapping relationship and the IP address that is in the information about the target SSD and that is of the target SSD, and then, the disk identifier of the target SSD and the logical address for storing the to-be-migrated data in the target SSD are used as the address of the target SSD.

When the source intelligent network interface card receives the information about the target SSD sent by the target storage array, a mapping relationship between an IP address allocated to each target SSD and a disk identifier of each SSD is established, so that the disk identifier of the SSD can be determined based on the mapping relationship in the data migration process and the target SSD can be further easily determined, to implement data migration.

In another implementation, the information about the target SSD includes an IP address of the target intelligent network interface card, and the IP address of the target intelligent network interface card is an IP address allocated by the source intelligent network interface card to the target intelligent network interface card when the connection is established between the target storage array and the source storage array. In this case, the source intelligent network interface card may determine the target intelligent network interface card based on the IP address that is in the information about the target SSD and that is of the target intelligent network interface card.

In this way, in a manner of allocating the IP address to the target intelligent network interface card, the source storage array easily manages a virtual hard disk obtained by mapping the target storage array to the source storage array.

Further, the information about the target SSD further includes the disk identifier of the target SSD and the logical address for storing the to-be-migrated data in the target SSD. In this case, when the address of the target SSD is determined based on the information about the target SSD, the disk identifier of the target SSD and the logical address for storing the to-be-migrated data may be used as the address of the target SSD.

A second aspect of the embodiments of the present invention provides a storage system, including a first storage device and a second storage device, wherein the first storage device includes a first solid state storage disk, SSD, and a first network interface card, wherein the first network interface card includes a first cache and a first processor, the second storage device includes a second solid state storage disk, SSD, and a second network interface card, wherein the second network interface card includes a second cache and a second processor, and wherein the first cache stores a first program instruction, and the first processor executes the first program instruction to implement corresponding steps of the method of the first aspect and the embodiments of the first aspect, executed by the first network interface card, and the second cache stores a second program instruction, and the second processor executes the second program instruction to implement corresponding steps of the method of the first aspect and the embodiments of the first aspect, executed by the second network interface card.

In various implementations of the foregoing embodiments of the present invention, migration caches provided by the source intelligent network interface card and the target intelligent network interface card may be used as data transit caches, and therefore, when the data is migrated, the migrated data does not pass through the controller or the memory of the source storage array or the target storage array, and the migrated data occupies no controller bandwidth or memory of the source storage array and the target storage array. This significantly reduces impact on the performance of the storage system and ensures the data migration speed.

To describe the technical solutions in the embodiments of the present invention or in the prior art more clearly, the following briefly describes the accompanying drawings required for describing the embodiments or the prior art.

The following clearly and completely describes the technical solutions in the embodiments of the present invention with reference to the accompanying drawings in the embodiments of the present invention. Apparently, the described embodiments are merely some but not all of the embodiments of the present invention.

As shown in <FIG> is an architectural diagram of an existing storage system <NUM>. The storage system <NUM> is a cluster including a plurality of storage arrays connected by using a network <NUM>, and is usually used as a data center. For brevity, only two storage arrays are shown in the figure, namely, a source storage array <NUM> and a target storage array <NUM>. The source storage array <NUM> is an SSD from which data is migrated, and the target storage array <NUM> is an SSD to which the data is migrated. The storage arrays communicate with each other by using a communications protocol that is based on an RDMA protocol.

Structures of the storage arrays are basically the same. The following uses the source storage array <NUM> as an example for description. The source storage array <NUM> includes a source controller <NUM>, a source memory <NUM>, a Peripheral Component Interconnect Express (Peripheral Component Interconnect Express, PCIe) bus <NUM>, a plurality of SSDs (namely, an SSD <NUM> to an SSD <NUM>), and a network interface card (Network Interface Card, NIC) <NUM>. The source memory <NUM> is connected to the source controller <NUM>. The source controller <NUM> is connected to the plurality of SSDs and the network interface card <NUM> by using the PCIe bus <NUM>. The network interface card <NUM> is connected to a network interface card of another storage array by using the network <NUM>, to implement communication between the source storage array <NUM> and the another storage array such as the target storage array <NUM>.

Link initialization is performed when a connection is established between the storage arrays in the storage system <NUM>. During link initialization, based on the RDMA protocol, each storage array can map a local SSD to another storage array in the storage system, so that the another storage array directly accesses the local SSD. The source storage array <NUM> and the target storage array <NUM> are used as an example. During link initialization, the target storage array <NUM> maps local SSDs (an SSD <NUM> to an SSD <NUM>) to the source storage array <NUM>. In this case, the SSD <NUM> to the SSD <NUM> of the target storage array <NUM> may serve as virtual disks SSD <NUM> to SSD <NUM> of the source storage array <NUM>, to be used by the source storage array <NUM>. For details, refer to <FIG>. A specific method in which the target storage array <NUM> maps the local SSD to the source storage array <NUM> to serve as the virtual SSD of the source storage array is as follows: During link initialization, the target storage array <NUM> selects an SSD that needs to be mapped to another storage array for use. In this embodiment, the target storage array <NUM> maps all local SSDs to the another storage array for use, and then sends information about each selected SSD, for example, a disk identifier, a disk size, and address space, to the another storage array such as the source storage array <NUM> by using a link to the another storage array. When SSD information sent by each storage array passes through the network interface card (network interface card, NIC) <NUM> of the source storage array <NUM>, the network interface card <NUM> generates an IP address for each SSD, adds the IP address to information about each SSD, and then establishes a correspondence between each IP address and a disk identifier of each SSD. After receiving the information about each SSD, the source storage array <NUM> records the IP address of each SSD, identifies each SSD by using the IP address of the SSD, and then creates a disk object for the information about each SSD, that is, the virtual disk (for example, the SSD <NUM> to the SSD <NUM> in <FIG>). In this way, the local SSD of the target storage array <NUM> may be mapped to the source storage array <NUM>, to serve as the virtual disk of the source storage array <NUM>. When accessing the virtual hard disk, the source storage array only needs to add an IP address of the virtual hard disk to access information.

Another method in which the target storage array <NUM> maps the local SSD to the source storage array <NUM> to serve as the virtual SSD of the source storage array is as follows: When SSD information sent by each storage array passes through the network interface card (network interface card, NIC) <NUM> of the source storage array <NUM>, the network interface card generates an IP address only for the network interface card <NUM> of the target storage array <NUM>. The source storage array <NUM> marks each SSD by using the IP address and a disk identifier, and establishes a disk object. The disk identifier is a unique identifier allocated by each storage array to each SSD in the storage array. In this case, when accessing the virtual hard disk, the source storage array adds the IP address and the disk identifier of the virtual hard disk to the access information.

In actual application, data migration frequently needs to be performed between the storage arrays. For example, when the source storage array <NUM> detects a to-be-invalidated SSD, that is, a source SSD <NUM>, data in the source SSD <NUM> needs to be migrated to another SSD, to avoid a data loss caused by invalidation of the source SSD <NUM>. During migration, the data in the source SSD <NUM> may be migrated to an SSD of the same storage array or an SSD of another storage array. In this embodiment, only a case in which the data in the source SSD <NUM> is migrated to the SSD in the another storage array such as the target storage array <NUM> is described. A hardware structure of the target storage array <NUM> is the same as a hardware structure of the source storage array <NUM>. In a description process, to differentiate elements included in the target storage array <NUM> from elements included in the source storage array <NUM>, herein, the elements included in the target storage array <NUM> are respectively referred to as a target controller <NUM>, a target memory <NUM>, and a target SSD <NUM>. The target SSD <NUM> is an SSD to which the data in the source SSD <NUM> is migrated.

With reference to <FIG>, <FIG>, and <FIG>, the following describes a prior-art method for migrating the data in the source SSD <NUM> in the source storage array <NUM> to the target SSD <NUM> of the target storage array <NUM>.

As shown in <FIG>, <FIG>, and <FIG>, in step S201, when detecting that the data in the source SSD <NUM> needs to be migrated, for example, detecting that the data of the source SSD <NUM> is to be invalidated, the source controller <NUM> selects, from the storage system <NUM>, a target SSD to which the data in the source SSD is to be migrated. The target SSD may be a local SSD or a virtual SSD. Herein, only a case in which the target SSD is the virtual SSD is described, that is, a case in which the target SSD is located in a remote storage array.

Step S202: The source controller <NUM> applies for migration memory space that is in the source memory <NUM> and that is dedicated to data migration, and obtains a data volume of the to-be-migrated data.

Step S203: The source controller <NUM> generates a read instruction based on a length of the migration memory space and the obtained data volume of the to-be-migrated data, and sends the read instruction to the source SSD <NUM>, where the read instruction instructs the source SSD <NUM> to migrate a data block whose size is equal to that of the migration memory space.

Step S204: The source SSD <NUM> reads, according to the read instruction, a data block that is in the source SSD and that is corresponding to the read instruction into the migration memory space.

Step S205: After completing execution of the read instruction, the source SSD <NUM> sends a read completion feedback instruction to the source controller <NUM>.

Step S206: The source controller <NUM> sends a write instruction to the source network interface card <NUM> according to the read completion feedback instruction, where the write instruction instructs the source network interface card <NUM> to migrate data in a migration memory in the source memory <NUM> to the target SSD.

Step S207: After receiving the write instruction, the source network interface card <NUM> determines a target network interface card based on information about the target SSD in the write instruction, and then forwards the write instruction to the target network interface card <NUM>.

Step S208: The target network interface card <NUM> further forwards the write instruction to the target controller <NUM>.

Step S209: The target controller <NUM> allocates a migration memory in the target memory <NUM> according to the write instruction, generates a migration instruction, and sends the migration instruction to the target network interface card, where a source address of the migration instruction is an address of the migration memory in the source memory <NUM>, and a target address of the migration instruction is an address of the migration memory in the target memory <NUM>.

Step S210: The target network interface card <NUM> reads, according to the migration instruction, the data in the migration memory in the source memory <NUM> into the target migration memory in the target memory <NUM>.

Step S211: After completing data migration, the target network interface card <NUM> informs the target controller that data migration is completed.

Step S212: The target controller sends a write instruction to the target SSD, so that the target SSD writes the data in the target memory into a flash memory of the target SSD.

Step S213: After receiving a migration completion command sent by the target SSD, the target network interface card <NUM> informs the source network interface card <NUM> that data migration is completed, and the source network interface card <NUM> further informs the source controller <NUM> that data migration is completed.

Step S214: After receiving the notification command, the source controller <NUM> determines whether migration of the to-be-migrated data is completed, where if migration is not completed, a procedure returns to step S201, and the procedure continues being executed until all to-be-migrated data in the source SSD <NUM> is migrated; or if migration is completed, the procedure ends.

It may be learned from the foregoing method for migrating data that, in the prior art, when the data in the source SSD is migrated to the target SSD, a data migration process needs to be controlled by the source controller and the target controller, and the migrated data needs to pass through the source memory <NUM> and the target memory <NUM> for migration. In this way, even if bandwidth of the source controller and bandwidth of the target controller that are occupied during data migration are limited, performance of the storage system is still affected. In addition, if the bandwidth of the source controller and the bandwidth of the target controller that are occupied during data migration are limited, a data migration speed is extremely low. This increases a risk of losing migrated data, and further affects reliability of the storage system.

In a method for migrating data provided in an embodiment of the present invention, when data is migrated from a source SSD to a target SSD, the data does not pass through a source memory or a target memory in a migration process, but is directly transmitted by using an interface of a PCIe switch, and a source controller only sends a command related to migration. Therefore, this occupies only little bandwidth of the source controller, and does not affect performance of a storage system. In addition, a data migration speed is not limited by the bandwidth of the source controller, and therefore, the data migration speed is increased. The following describes in detail the solution provided in this embodiment of the present invention.

As shown in <FIG> is an architectural diagram of a storage system according to Embodiment <NUM> of the present invention. An architecture of the storage system in Embodiment <NUM> is basically the same as the architecture of the storage system in the prior art in <FIG>, and a difference is that a network device connecting the source storage array <NUM> to the target storage array <NUM> is changed from the original network interface card to an intelligent network interface card having a control function. For ease of description, an intelligent network interface card in the source storage array <NUM> is referred to as a source intelligent network interface card <NUM>, and an intelligent network interface card in the target storage array <NUM> is referred to as a target intelligent network interface card <NUM>. The source intelligent network interface card <NUM> and the target intelligent network interface card <NUM> communicate with each other by using a communications protocol based on an RDMA protocol, such as an NOF protocol.

The source controller <NUM> is configured to run an application program <NUM> in the source memory <NUM>, to implement some functions provided by the source storage array <NUM>, for example, control of fetch and migration of data in the SSD <NUM> to the SSD <NUM> and control of a data transmission process of the source intelligent network interface card <NUM>. The source memory <NUM> further stores metadata <NUM> of the SSD <NUM> to the SSD <NUM> and the source intelligent network interface card <NUM>. The metadata <NUM> records information about data stored in each SSD, for example, a data volume of the stored data, logical address information of the stored data, and access information of a migration cache of the source intelligent network interface card <NUM>. The access information of the migration cache is described in detail below.

In this embodiment of the present invention, structures of the source storage array <NUM> and the target storage array <NUM> are basically the same. Herein, only the source storage array <NUM> is used as an example to describe a structural diagram of each element in the storage array. A structural diagram of the source intelligent network interface card <NUM> is first described, and a structure of the target intelligent network interface card <NUM> is the same as a structure of the source intelligent network interface card <NUM>. Herein, only the source intelligent network interface card <NUM> is used as an example for description. As shown in <FIG>, the source intelligent network interface card <NUM> includes a processor <NUM> and a cache <NUM>, a migration cache <NUM> is further disposed in the cache <NUM>, and the processor <NUM> further includes a register <NUM>. The cache <NUM> stores a program instruction (not shown in the figure), and the processor <NUM> executes the program instruction to implement a function indicated by the program instruction. The migration cache <NUM> may be accessed by the source controller <NUM> and the SSD. For a method in which the migration cache <NUM> is accessed by the source controller <NUM> and the SSD, refer to a flowchart shown in <FIG>.

Step S601: The source intelligent network interface card <NUM> sets cache information of the migration cache <NUM> in the register <NUM> of the processor <NUM>. In this embodiment, the cache information includes a size of the migration cache.

Step S602: In a BIOS startup phase of the source storage array <NUM>, the source controller <NUM> reads the cache information of the migration cache <NUM> from the register <NUM> of the processor <NUM> in the intelligent network interface card <NUM>, allocates access information to the migration cache <NUM> based on the read cache information, and records, in the metadata <NUM> of the source intelligent network interface card in the source memory <NUM>, the access information allocated to the migration cache, where the access information includes a first access address and a length of the migration cache. The first access address is address information of the source memory <NUM>, the address information is corresponding to start address information of the migration cache <NUM>, and access address space of the migration cache may be formed based on the access address and the size of the migration cache.

Step S603: The source controller <NUM> writes, into the register <NUM> of the source intelligent network interface card <NUM>, the access information allocated to the migration cache <NUM>. In specific implementation, only the first access address may be written.

Step S604: The source intelligent network interface card <NUM> selects, as the migration cache <NUM>, cache space whose size is the same as the set cache size from the cache <NUM>, and establishes a mapping relationship between the access information and the migration cache <NUM>.

When the source controller <NUM> or another SSD needs to access any location of the migration cache <NUM>, as long as an access address of the access address space is added to an access command, a physical address of the migration cache that is corresponding to the access address may be found based on a correspondence between the first access address stored in the register and a start address of the migration cache <NUM>, to implement access to data in the migration cache <NUM>. In this way, the source controller <NUM> or the SSD may access the migration cache <NUM> of the intelligent network interface card <NUM> by using the access information. After the migration cache <NUM> of the intelligent network interface card <NUM> is determined, data migration may be performed by using the migration cache <NUM>.

Structures of the SSDs in the source storage array <NUM> are also basically the same. Herein, a source SSD <NUM> is used as an example for description. The source SSD <NUM> includes a processor <NUM>, a direct memory access (Direct Memory Access, DMA) controller <NUM>, a cache <NUM>, and a flash memory <NUM>. The processor <NUM> is configured to receive a data fetch command sent by the source controller <NUM>, to control access to data <NUM> in the flash memory <NUM> according to the fetch command of the source controller <NUM>. The DMA controller <NUM> may directly access the source memory <NUM>. In addition, because the source controller <NUM> allocates the access information to the migration cache <NUM> in the intelligent network interface card <NUM>, the DMA controller <NUM> may directly access the migration cache <NUM> of the intelligent network interface card <NUM>. A specific access method is further described in detail in the following description of data migration.

After migration caches of the source intelligent network interface card <NUM> and the target intelligent network interface card <NUM> are disposed, data migration may be performed by using the migration caches. A specific migration method is shown in <FIG>, <FIG>, and <FIG>.

Step S701: The source controller <NUM> detects a preset data migration condition.

The preset migration condition may be that an SSD in the source storage array <NUM> is determined to be invalidated or when a data migration request is received, the migration request is from an application or another device.

Step S702: When the preset data migration condition is detected, the source controller <NUM> obtains to-be-migrated data information of to-be-migrated data in an SSD from which data needs to be migrated, that is, a source SSD, and determines a target SSD.

The target SSD may be a local SSD in <FIG> or a virtual SSD. However, in this embodiment, only a method for migrating data on a premise that the target SSD is a virtual SSD is described. In this embodiment, the virtual SSD <NUM> is selected as the target SSD.

The source controller <NUM> obtains the to-be-migrated data information of the to-be-migrated data in the source SSD from the metadata <NUM> stored in the source memory <NUM>. The to-be-migrated data information includes a data volume and a logical address of the to-be-migrated data.

The target SSD may be determined by a user, or may be a backup disk disposed for the source SSD in advance. There may be one or more target SSDs, and a specific quantity of target SSDs may be determined according to a data migration speed requirement.

When determining the target SSD, the user may select the target SSD based on a volume of data stored in each SSD and a busy/idle status of the SSD, that is, may select, as the target SSD, an SSD whose current service volume is relatively small and that has more free space. Generally, a control server (not shown in the figure) is disposed in a storage system. The control server includes current status information of all SSDs in the system, for example, a service volume and free space. All users may select the target SSD by accessing the cluster controller, and the cluster controller sends information about the selected target SSD to the source storage array <NUM>.

Step S703: The source controller <NUM> generates a first migration instruction, where the migration instruction includes information about a to-be-migrated data block in the source SSD, information about the target SSD, and a data length during this migration.

In this implementation, two manners of generating the first migration instruction are provided. The first manner is as follows: The to-be-migrated data is first split into a plurality of data blocks based on a preset data block length during each migration and a preset data volume of the to-be-migrated data that is in the to-be-migrated data information. It should be noted that a length of each data block may be equal to or less than a length of the migration cache. In addition, a logical address of each data block may be determined based on the logical address of the to-be-migrated data and the length of the data block obtained through splitting.

The second manner is as follows: When the first migration instruction is generated, not all to-be-migrated data is split, but one data block is obtained through splitting only for migration performed once, and a length of the data block obtained through splitting is the preset length, and may be less than or equal to the length of the migration cache; and then a logical address of the data block obtained through splitting is determined, and the first migration instruction is generated for the data block obtained through splitting.

The first migration instruction includes a source address and a target address, and the source address is a logical address that is of a data block migrated according to the first migration instruction and that is in the source SSD. As described in this embodiment, the target SSD is located in a remote storage array, and based on two mapping manners of the virtual disk described in <FIG>, the target address may be an IP address and a logical address of the SSD, or may be an IP address of the target network interface card and a disk identifier and a logical address of the SSD.

When there are a plurality of determined target SSDs, one data block is separately obtained for each target SSD in ascending order of logical addresses of the to-be-migrated data that are in the source SSD, to generate a plurality of first migration instructions.

Step S704: The source controller <NUM> sends the first migration instruction to the source intelligent network interface card <NUM>.

When generating the plurality of first migration instructions based on the plurality of target SSDs, the source controller <NUM> sequentially sends, to the source intelligent network interface card <NUM>, the plurality of first migration instructions based on a generation sequence of the plurality of first migration instructions.

Step S705: When receiving the first migration instruction, the source intelligent network interface card <NUM> generates a first read instruction, and sends the first read instruction to the source SSD. The first read instruction includes a source address, a target address, and a length of a migrated data block, the source address is a logical address of the to-be-migrated data block, and the target address is an access address of the migration cache of the intelligent network interface card.

Step S706: When receiving the read instruction, the source SSD <NUM> obtains the currently migrated data block from the to-be-migrated data based on the source address and the length of the to-be-migrated data block, and reads the data block into the migration cache of the source intelligent network interface card <NUM>.

In this implementation, after receiving the read instruction, the processor <NUM> of the source SSD <NUM> instructs the DMA controller <NUM> to find, in the flash memory of the source SSD based on the source address in the read instruction, a to-be-migrated data block corresponding to the source address, and then write, based on the target address, that is, the access address of the migration cache, the to-be-migrated data block into the migration cache of the source intelligent network interface card <NUM> that is indicated by the target address. When reading the to-be-migrated data block into the migration cache of the intelligent network interface card <NUM>, the DMA controller <NUM> first transmits, by using the PCIe bus <NUM>, the to-be-migrated data block and the target address to the migration cache of the source intelligent network interface card <NUM> that is indicated by the target address, and the source intelligent network interface card <NUM> further finds, based on the access address in the target address, a physical address of the migration cache that is corresponding to the access address, and then stores the to-be-migrated data block at a location that is in the migration cache and that is indicated by the physical address.

Step S707: After the to-be-migrated data block is fully read into the migration cache of the source intelligent network interface card <NUM>, the source SSD <NUM> sends a read completion feedback instruction to the source intelligent network interface card <NUM>.

Step S708: After receiving the read completion feedback instruction sent by the source SSD <NUM>, the source intelligent network interface card <NUM> determines the target intelligent network interface card <NUM> based on the target address in the first migration instruction, and then sends a second migration instruction to the target intelligent network interface card <NUM>, where the second migration instruction includes a source address and a target address, the source address in the second migration instruction is an access address that is of a physical interval for storing the to-be-migrated data block and that is in the migration cache of the source intelligent network interface card <NUM>, and the target address in the second migration instruction is determined based on the target address in the first migration instruction.

As described in step S703, when the target address in the first migration instruction is the IP address and the logical address of the target SSD, the source intelligent network <NUM> may determine the target intelligent network interface card based on the IP address of the target SSD, then determine, based on the IP address and the disk identifier of the SSD that are established during disk mapping, the disk identifier of the target SSD that is corresponding to the IP address of the target SSD, and then use the disk identifier and the logical address as the target address of the second migration instruction.

When the target address in the first migration instruction is the IP address of the target network interface card and the disk identifier and the logical address of the target SSD, the source intelligent network <NUM> determines the target network interface card based on the IP address of the target network interface card, and uses the disk identifier and the logical address of the target SSD as the target address of the second migration instruction.

Step S709: The target intelligent network interface card <NUM> generates a second read instruction according to the second migration instruction, where a source address of the second read instruction is the source address of the second migration instruction, a target address of the second read instruction is the access address of the migration cache of the target intelligent network interface card <NUM>, and the target intelligent network interface card <NUM> reads the to-be-migrated data block in the migration cache of the source intelligent network interface card <NUM> into the migration cache of the target intelligent network interface card <NUM> according to the second read instruction.

Herein, the target intelligent network interface card <NUM> reads the to-be-migrated data block in the migration cache of the source intelligent network interface card <NUM> by using an RDMA protocol.

Step S710: After the to-be-migrated data block is fully read, the target intelligent network interface card <NUM> sends a write instruction to the target SSD, where a source address in the write instruction is the access address of the migration cache of the target intelligent network interface card, and a target address in the write instruction is the target address in the second migration instruction.

As described above, the target address includes the disk identifier and LBA of the target SSD. Therefore, the target intelligent network interface card may find the target SSD based on the disk identifier, and find, based on the LBA, a physical address that is in the flash memory of the target SSD and that is used to store the to-be-migrated data block.

Step S711: A processor (not shown in the figure) of the target SSD <NUM> writes the to-be-migrated data block in the migration cache in the target intelligent network interface card <NUM> into the flash memory of the target SSD according to the write instruction.

Herein, it should be noted that the target SSD <NUM> has a same structure as the source SSD <NUM>, the target SSD <NUM> also includes a DMA controller (not shown in the figure), and the target SSD writes the to-be-migrated data block in the migration cache of the target intelligent network interface card <NUM> into the flash memory of the target SSD by using the DMA controller.

Step S712: After completing writing of the to-be-migrated data block, the target intelligent network interface card <NUM> sends a write completion feedback instruction to the source controller <NUM>.

Step S713: After receiving the write completion feedback instruction, the source controller <NUM> determines whether migration of the to-be-migrated data in the source SSD is completed.

Step S714: If migration is not completed, the source controller <NUM> generates a new migration instruction, and then a procedure returns to step S703; or if migration is completed, a data migration procedure ends. After receiving a plurality of feedback instructions, the source controller <NUM> generates one new migration instruction for each feedback instruction.

If the manner of generating the migration instruction in step S703 is the foregoing first manner, in step S714, one data block is determined from data blocks that are still not migrated, and a new read instruction is generated for the determined data block.

If the manner of generating the read instruction in step S703 is the foregoing second manner, in step <NUM>, one data block is obtained by splitting to-be-migrated data that is still not migrated, and a new read instruction is generated based on the data block newly obtained through splitting.

In the method provided in the foregoing embodiment, the data in the source SSD may be transmitted to the target SSD without passing through the source memory <NUM> or the target memory <NUM>. Therefore, bandwidth of the source controller <NUM> is not occupied, and a speed of migrating data between different storage arrays is increased.

In another implementation of the present invention, as shown in <FIG>, the source controller and the source memory form an independent device, that is, a host <NUM>. The host <NUM> is connected to a storage array <NUM> by using a PCIe switch <NUM>, and the storage array includes a plurality of SSDs <NUM> and an intelligent network interface card <NUM>. In this structure, the source controller still generates and sends a first migration instruction. A structure and an implementation of the intelligent network interface card <NUM> are the same as those in the first implementation. For details, refer to description in <FIG> and <FIG>, <FIG>, and <FIG>.

In addition, the source intelligent network interface card and the target intelligent network interface card may be field programmable gate arrays (Field Programmable Gate Array, FPGA). In this case, a program in the cache <NUM> is burned on a chip of the field programmable gate array instead of being stored in the cache <NUM>, and the processor <NUM> may directly perform the burned program, without a need to invoke the program instruction from the cache <NUM> for execution.

As shown in <FIG> is module diagrams of the source controller <NUM> and the source intelligent network interface card <NUM> in the source storage array <NUM> in the storage system shown in <FIG> and a function module diagram of the target intelligent network interface card <NUM> in the target storage array <NUM> in this embodiment.

The source controller <NUM> includes a generating module <NUM>, an allocation module <NUM>, a first migration instruction module <NUM>, and a determining module <NUM>. The source intelligent network interface card <NUM> includes a marking module <NUM>, a setting module <NUM>, a mapping module <NUM>, a first receiving module <NUM>, a read instruction module <NUM>, and a second migration instruction module <NUM>. The target intelligent network interface card <NUM> includes a setting module <NUM>, a mapping module <NUM>, a second receiving module <NUM>, a read module <NUM>, and a write instruction module <NUM>.

In actual implementation, structures of storage arrays in the storage system are basically the same, and function modules included in elements of the storage arrays are also basically the same, but functions implemented by the function modules are different when the storage arrays respectively serve as a source end and a destination end. In this embodiment, only a function module implemented when the storage array serves as a source storage array or a target storage array is described.

When link initialization is performed on each storage array of the storage system, the marking module <NUM> of the source intelligent network interface card generates identification information of the target intelligent network interface card based on SSD information sent by the target intelligent network interface card. In this case, the identification information may be an IP address allocated to each SSD or an IP address allocated to the target intelligent network interface card. For details, refer to related description in <FIG>. After receiving information about each SSD that is sent by the source intelligent network interface card, the generating module <NUM> of the source controller generates a virtual disk of the source storage array based on the information about each SSD. For a specific generation process, refer to related description in <FIG>.

The setting module <NUM> of the source intelligent network interface card cooperates with the allocation module <NUM> of the source controller, to implement a function of allowing the source controller and the SSD to access the migration cache in the source intelligent network interface card. This is corresponding to the method of providing the migration cache by the source intelligent network interface card shown in <FIG>. Specifically, the setting module <NUM> of the source intelligent network interface card is configured to set cache information of the migration cache in the register of the source intelligent network interface card. For a specific setting manner, refer to description of step S601.

The allocation module <NUM> of the source controller is configured to: read the migration cache information of the source intelligent network interface card in the register of the processor of the source intelligent network interface card in a BIOS startup phase of the source storage array, allocate access information to the migration cache of the source intelligent network interface card based on the read migration cache information, record, in metadata, the access information allocated to the source intelligent network interface card, and write, into the register of the source intelligent network interface card, the access information allocated to the migration cache of the source intelligent network interface card. For a specific allocation manner, refer to description of step S602.

The mapping module <NUM> of the source intelligent network interface card is configured to: select, as the migration cache, cache space whose size is the same as a set cache size from the cache, and establish a mapping relationship between the access information and the migration cache. This is corresponding to step S604.

A function executed by the setting module <NUM> and the mapping module <NUM> of the target intelligent network interface card are the same as a function executed by the setting module <NUM> and the mapping module <NUM> of the source intelligent network interface card, that is, setting the migration cache for the target intelligent network interface card. For details, refer to related description of setting the migration cache of the source intelligent network interface card.

The first migration instruction module <NUM> of the source controller is configured to: detect a preset data migration condition, and when the preset data migration condition is detected, obtain to-be-migrated data information of to-be-migrated data in a source SSD, determine a target SSD, then generate a first migration instruction based on the to-be-migrated data information of the to-be-migrated data and information about the target SSD, and send the first migration instruction to the source intelligent network interface card. A function executed by the first migration instruction module <NUM> is corresponding to steps S701 to S704 in <FIG>, <FIG>, and <FIG>. Two manners of generating the first migration instruction are the same as those in step S703.

The first receiving module <NUM> is configured to receive the first migration instruction, and the read instruction module <NUM> of the source intelligent network interface card generates a read instruction according to the first migration instruction, to instruct the source SSD to read the to-be-migrated data in the source SSD into the migration cache of the source intelligent network interface card. For details, refer to description of steps S705 to S707 in <FIG>, <FIG>, and <FIG>.

After the to-be-migrated data is migrated to the migration cache of the source intelligent network interface card, the second migration instruction module <NUM> of the source intelligent network interface card sends a second migration instruction to the target intelligent network interface card. For details, refer to description of step S708 in <FIG>, <FIG>, and <FIG>.

The second receiving module <NUM> of the target intelligent network interface card receives the second migration instruction, and the read module <NUM> of the target intelligent network interface card generates a second read instruction according to the second migration instruction, and executes the generated second read instruction to read the to-be-migrated data in the migration cache of the source intelligent network interface card into the migration cache of the target intelligent network interface card. For details, refer to description of step S709 in <FIG>, <FIG>, and <FIG>.

After the to-be-migrated data is migrated to the migration cache of the target intelligent network interface card, the write instruction module <NUM> of the target intelligent network interface card sends a write command to the target SSD, to instruct the target SSD to write the to-be-migrated data in the target intelligent network interface card into a flash memory of the target SSD. For details, refer to steps S710 to S712 in <FIG>, <FIG>, and <FIG>.

After the to-be-migrated data block is written into the flash memory of the target SSD, the determining module of the source controller further determines whether migration of the to-be-migrated data in the source SSD is completed. If migration is not completed, a new migration instruction is generated to continue performing migration until all to-be-migrated data in the source SSD is migrated; or if migration is completed, a migration procedure ends. For details, refer to description of step S713 in <FIG>, <FIG>, and <FIG>.

As shown in <FIG>, the cache of the source intelligent network interface card further stores a program instruction (not shown in the figure), and the processor of the source intelligent network interface card executes the program instruction to implement corresponding steps executed by the source intelligent network interface card in the flowchart shown in <FIG>, <FIG>, and <FIG>. Likewise, the cache of the target intelligent network interface card also stores a program instruction (not shown in the figure), and the processor of the target intelligent network interface card executes the program instruction to implement corresponding steps executed by the target intelligent network interface card in the flowchart shown in <FIG>, <FIG>, and <FIG>.

Claim 1:
A method for migrating data between storage devices, wherein a first storage device (<NUM>) comprises a source controller (<NUM>), a first solid state storage disk, SSD, (<NUM>), and a first network interface card (<NUM>) comprising a first migration cache (<NUM>), the second storage device (<NUM>) comprises a second solid state storage disk, SSD, (<NUM>), and a second network interface card (<NUM>) comprising a second migration cache, and the method comprises:
receiving (S704), by the first network interface card (<NUM>), a first migration instruction, sent by the source controller (<NUM>), wherein the first migration instruction comprises an address of the data stored in the first SSD (<NUM>);
in response to the receiving (S704) of the first migration instruction, obtaining (S706), by the first network interface card (<NUM>), the data from the first SSD (<NUM>) based on the address and reading the data into the first migration cache (<NUM>);
sending (S708), by the first network interface card (<NUM>), a second migration instruction to the second network interface card (<NUM>), the second migration instruction comprises access information of the first migration cache (<NUM>) and an address of the second SSD (<NUM>), the access information of the first migration cache (<NUM>) comprises information about the data stored in the migration cache (<NUM>);
in response to the sending of the second migration instruction (S708), reading (S709), by the second network interface card (<NUM>), the data into the second migration cache of the second network interface card (<NUM>);
sending a write instruction (<NUM>) by the second network interface card (<NUM>), to write the data to the second SSD (<NUM>).