Patent Description:
Current storage systems may use forms such as distributed storage, network-attached storage (network-attached storage, NAS), and a storage area network (storage area network, SAN). In these storage systems, clients and storage devices are connected through networks. Solid state disks (solid state disks, SSDs) are used in the foregoing storage systems because of advantages of high bandwidth, a low latency, and a high throughput. For the storage system, how the client efficiently accesses data in the solid state disk through the network is an important issue.

In the conventional technology, when the client needs to access a solid state disk of the storage device, a central processing unit (central processing unit, CPU) of the client constructs a read command or a write command, a network interface card of the client sends the read command or the write command to a network interface card of the storage device, and then the network interface card of the storage device notifies a CPU of the storage device. Then, access to the solid state disk is completed under control of the CPU of the storage device.

However, the method in the conventional technology may result in a long data access latency and excessively high CPU usage of the storage device. <CIT> describes techniques for use of vendor defined messages to execute a command to access a storage device. <CIT> describes a data storage system that includes a storage server. <CIT> describes transferring data and commands in a memory management environment.

Embodiments of this application provide a method for accessing a solid state disk and a storage device, to resolve problems of a long data access latency and excessively high CPU usage in the conventional technology.

According to a first aspect, an embodiment of this application provides a method for accessing a solid state disk as laid out in claim <NUM>.

According to a second aspect, an embodiment of this application provides a storage device as laid out in claim <NUM>.

According to the method for accessing the solid state disk and the storage device provided in the embodiments of this application, after receiving the write command, the network interface card of the storage device may directly interact with the solid state disk to complete data writing, without the participation of the CPU. Therefore, the data access latency can be greatly reduced. In addition, because the CPU does not need to participate, CPU usage of the storage device can also be greatly reduced.

<FIG> is a diagram of an example application scenario of a method for accessing a solid state disk according to an embodiment of this application. As shown in <FIG>, the method may be applied to a distributed storage scenario. In the distributed storage scenario, a storage system may include a plurality of clients and a plurality of storage devices. Any client can access any storage device. The client may be a host on a user side, or may be a control device on a storage side. When the client is a host, a virtual block service (virtual block service, VBS) and some applications may be installed on the client. When the client is a control device, an enterprise distribution system (enterprise distribution system, EDS) and management software for a storage service may be installed on the client. The storage device may be a storage server in a physical form. In <FIG>, an object storage device (object storage device, OSD) is used as an example, but the storage device is not limited to the object storage device. The client has a network interface card, and a network interface card and a hard disk (in this embodiment, a solid state disk is used as an example for description) are deployed in the storage device. When the client needs to access data stored in the OSD, the client may initiate a corresponding data access command to the OSD according to the method provided in this embodiment of this application, to instruct a network interface card in the OSD to complete access to the hard disk.

<FIG> is a diagram of another example application scenario of the method for accessing the solid state disk according to an embodiment of this application. As shown in <FIG>, the method may be applied to an enterprise storage scenario. In the enterprise storage scenario, a storage system includes a client and a storage device. In the scenario shown in <FIG>, the client is generally a control device, and the storage device is a disk enclosure. Different from a conventional disk enclosure, the disk enclosure in this embodiment not only includes a network interface card and a hard disk, but also includes a processor and a memory (which are described in detail in <FIG>).

The method in embodiments of this application may be applied to storage systems that support a remote direct access protocol. Specifically, a network protocol on which these storage systems are based is a protocol that supports remote direct access. In an example, the protocol that supports the remote direct access may be a remote direct memory access (remote direct memory access, RDMA) protocol. Correspondingly, a network protocol that supports the RDMA protocol may be an RDMA over converged Ethernet (RDMA over converged Ethernet, RoCE) protocol, an Internet wide area RDMA protocol (Internet wide area RDMA protocol, iWARP), an InfiniBand (InfiniBand) protocol, or the like.

For ease of understanding, in the following embodiments of this application, the RoCE protocol is used as an example, and a message, a command, and the like in the RoCE protocol are used to describe the technical solutions in embodiments of this application. It should be understood that this cannot be construed as a limitation on embodiments of this application. The technical solutions in embodiments of this application may further be applied to a storage system based on another RDMA network protocol.

In a possible data access manner, for example, the client requests to read data, the client notifies a CPU of a storage controller of a read command, and the CPU of the storage controller notifies a solid state disk to read the data. After the solid state disk reads the data, the CPU of the storage controller needs to be interrupted, to indicate a network interface card to send the read data and a response to the client. First, in this processing manner, the CPU of the storage controller, the network interface card, and the solid state disk need to interact to complete data access, accordingly resulting in a relatively long data access latency. In addition, the CPU of the storage controller participates in a data access process, resulting in excessively high CPU usage.

The technical solutions in embodiments of this application are intended to resolve the foregoing problems.

<FIG> is a diagram of an example system architecture of the solid state disk access method according to an embodiment of this application. As shown in <FIG>, a client and a storage device are included in this method. The client includes a CPU <NUM>, a network interface card <NUM>, and a memory <NUM>. The storage device includes a CPU <NUM>, a network interface card <NUM>, a memory <NUM>, and a solid state disk <NUM>.

The CPU <NUM> is configured to process an I/O request from outside the storage device or a request generated in the storage device. The memory <NUM> is configured to temporarily store data received from the client or data read from a hard disk. When receiving a plurality of write commands sent by the client, the network interface card <NUM> may temporarily store data in the plurality of write commands in the memory <NUM>, and notify the solid state disk <NUM> to write the temporarily stored data to a medium of the solid state disk <NUM>. The memory <NUM> includes a volatile memory, a non-volatile memory, or a combination thereof. The volatile memory is, for example, a random access-memory (random-access memory, RAM). The non-volatile memory includes, for example, various machine-readable media that can store program code, such as a flash chip, a floppy disk, a hard disk, an SSD, or an optical disc. The memory <NUM> may have a power-off protection function. The power-off protection function means that data stored in the memory <NUM> is not lost when a system is powered off and then powered on again.

The solid state disk (SSD) <NUM> includes a primary controller and a storage medium. The primary controller is configured to receive an I/O request or other information sent by the network interface card <NUM> to the solid state disk <NUM>, for example, a logical address of a data block, and the primary controller is further configured to execute the received I/O request, for example, write a data block carried in the I/O request to the storage medium, or read a data block from the storage medium. The storage medium generally includes several flash (Flash) chips. Each flash chip includes several blocks (blocks). Each block includes several pages (pages), and the primary controller writes the data block to the block by page each time.

The network interface card <NUM> includes a processor and a memory. The processor is configured to receive an I/O request or other information sent by the network interface card <NUM> of the client, and the processor is further configured to execute the received I/O request, for example, write a data block carried in the I/O request to a storage address in the memory, or send data in a storage address in the memory to the network interface card <NUM> of the client. The memory of the network interface card <NUM> may be, for example, a RAM or a read-only memory (read-only memory, ROM).

It should be noted that the foregoing uses the CPU <NUM>, the memory <NUM>, the network interface card <NUM>, and the solid state disk <NUM> on the storage device side as examples to describe these components. Functions of the CPU <NUM>, the memory <NUM>, and the network interface card <NUM> on the client side correspond to functions of the components on the storage device side.

The system architecture shown in <FIG> may be applied to the scenarios shown in <FIG>. The CPU <NUM> of the client sends a data access command to the network interface card <NUM>. In this embodiment of this application, the data access command is a write command or a read command. The write command instructs to write data to the solid state disk, and the read command instructs to read data from the solid state disk. The network interface card <NUM> then sends the data access command to the network interface card <NUM> of the storage device, and the network interface card <NUM> exchanges information with the solid state disk <NUM>, to complete access to the solid state disk.

The following describes specific functions of the foregoing components in the client and the storage device shown in <FIG> and an information exchange process between the components.

On the client side, the CPU <NUM> is configured to run a computer program. When the computer program is run, the CPU <NUM> can process a data access request, construct a data access command, and send the data access command to the storage device by using the network interface card <NUM>. The data access command may be a write command or a read command. A storage area is pre-allocated in the memory <NUM>, and is used to store a completion queue (completion queue, CQ) and a submission queue (submission queue, SQ) of the network interface card <NUM>. A command to be executed by the network interface card is placed in the SQ of the network interface card <NUM>. After completing executing one or more commands in the SQ, the network interface card <NUM> writes command execution completion information to the CQ. The command in the SQ may be written by the CPU <NUM>. After the network interface card <NUM> completes executing the one or more commands in the SQ, the network interface card <NUM> writes the command execution completion information to the CQ, and then the CPU <NUM> indicates, based on the information in the CQ, the network interface card <NUM> to execute a new command. A write buffer and a read buffer are further separately allocated in the memory <NUM>. When the client needs to write data to the storage device, after receiving a data write request, the client first temporarily stores the to-be-written data to the write buffer of the memory <NUM>, and then sends the to-be-written data in the write buffer to the storage device when sending a write command to the storage device. After the client sends a read command to the storage device, the storage device sends data requested by the read command to the read buffer of the client, and then the CPU <NUM> reads the data from the read buffer.

The network interface card <NUM> of the client is communicatively connected to the network interface card <NUM> of the storage device. The CPU <NUM> sends the data access command to the storage device by using the network interface card <NUM>, and the network interface card <NUM> receives data and a response that are returned by the storage device. After receiving the data and the response that are returned by the storage device, the network interface card <NUM> notifies the CPU <NUM> to perform processing. For a storage system based on a different network protocol, the network interface card <NUM> may communicate with the network interface card <NUM> of the storage device based on the different network protocol. For example, the network interface card <NUM> may communicate with the network interface card <NUM> based on the foregoing protocol, for example, RoCE, iWARP, or InfiniBand, or the TCP/IP protocol. Correspondingly, the network interface card <NUM> and the network interface card <NUM> are network interface cards that support the network protocol. For example, the network interface card <NUM> and the network interface card <NUM> support the RoCE protocol. In this case, both the network interface card <NUM> and the network interface card <NUM> are network interface cards having an RDMA function (RDMA enabled NICs, RNICs). In addition, the CPU <NUM>, the memory <NUM>, and the network interface card <NUM> are connected through a peripheral component interconnect express (peripheral component interconnect express, PCIe) bus. Therefore, the network interface card <NUM> has a PCIe interface, to be connected to the CPU <NUM> and the memory <NUM> through the PCIe interface and bus.

On the storage device side, the CPU <NUM> is configured to run a computer program. In this embodiment of this application, the computer program run in the CPU <NUM> is used for initialization that specifically includes initializing an SQ of the network interface card <NUM> and an SQ of the solid state disk <NUM>, and binding the SQ of the network interface card <NUM> and the SQ of the solid state disk <NUM>. A specific process is described in detail in the following initialization process.

A storage area is pre-allocated in the memory <NUM>, and is used to store the SQ of the network interface card <NUM>. In a conventional processing manner, the CPU <NUM> writes a command to be executed by the network interface card <NUM> to the SQ of the network interface card <NUM>. After the network interface card <NUM> completes executing the command in the SQ, the network interface card <NUM> writes execution completion information to a CQ, and notifies the CPU <NUM> to perform processing. However, in this embodiment of this application, a data access related command that is to be executed by the network interface card <NUM> may not be processed by the CPU <NUM>, but is written by the solid state disk <NUM> to the SQ of the network interface card <NUM>. After completing executing the command in the SQ, the network interface card <NUM> no longer needs to write the completion information to the CQ. Refer to <FIG>. Processing such as writing a command to the SQ of the network interface card <NUM> by the solid state disk <NUM> may be implemented by a network interface card queue management module disposed in the solid state disk <NUM>. The network interface card queue management module may be a hardware module or a software module. A specific form of the network interface card queue management module is not limited in this embodiment of this application.

A storage area is further pre-allocated in the memory <NUM>, and is used to store the SQ of the solid state disk <NUM>. In a conventional processing manner, the CPU <NUM> writes a command to be executed by the solid state disk <NUM> to the SQ of the solid state disk <NUM>. After the solid state disk <NUM> completes executing the command in the SQ, the solid state disk <NUM> writes execution completion information to a CQ, and notifies the CPU <NUM> to perform processing. However, in this embodiment of this application, a data access related command that is to be executed by the solid state disk <NUM> may not be processed by the CPU <NUM>, but is written by the network interface card <NUM> to the SQ of the solid state disk <NUM>. After completing executing the command in the SQ, the solid state disk <NUM> no longer needs to write the completion information to the CQ. Refer to <FIG>. Processing such as writing a command to the SQ of the solid state disk <NUM> by the network interface card <NUM> may be implemented by a solid state disk queue management module disposed in the network interface card <NUM>. The solid state disk queue management module may be a hardware module or a software module. A specific form of the solid state disk queue management module is not limited in this embodiment of this application.

It should be noted that, in addition to that both the SQ of the network interface card <NUM> and the SQ of the solid state disk <NUM> shown in <FIG> are located in the memory <NUM>, in another optional implementation, the SQ of the network interface card <NUM> may alternatively be located in the network interface card <NUM>, and the SQ of the solid state disk <NUM> may alternatively be located in the solid state disk <NUM>. Optionally, when the SQ of the network interface card <NUM> is located in the network interface card <NUM>, the SQ may be located in the RAM of the network interface card <NUM>. Optionally, when the SQ of the solid state disk <NUM> is located in the solid state disk <NUM>, the SQ may be located in the storage medium of the solid state disk <NUM>.

A write buffer and a read buffer are further separately allocated in the memory <NUM>. After receiving the to-be-written data sent by the client, the network interface card <NUM> first temporarily stores the to-be-written data in the write buffer of the memory <NUM>, and then the solid state disk <NUM> writes the to-be-written data from the write buffer to the medium of the solid state disk <NUM>. When the client requests the storage device to read data in the solid state disk <NUM>, the solid state disk <NUM> first temporarily stores the read data in the read buffer, and then the network interface card <NUM> sends the data from the read buffer to the client.

The solid state disk <NUM> can identify a data access related command that is in the SQ and that is written by the network interface card <NUM>, and after completing executing the command, return an execution result to the client by using the network interface card <NUM>. In addition, the solid state disk <NUM> is bound to the SQ of the network interface card <NUM> in advance. When performing data access processing, the solid state disk <NUM> may write a data access related command to the SQ of the network interface card <NUM>. Optionally, as described above, the network interface card queue management module in the solid state disk <NUM> may implement the command writing process. The solid state disk <NUM> includes a solid state disk SQ doorbell (doorbell), and the solid state disk SQ doorbell may be specifically a register in the solid state disk <NUM>. When writing the command to the SQ of the solid state disk <NUM>, the network interface card <NUM> may send a notification to the solid state disk SQ doorbell, so that the solid state disk <NUM> learns that the command is written to the SQ.

The network interface card <NUM> can identify and execute the data access related command that is in the SQ of the network interface card <NUM> and that is written by the solid state disk <NUM>. In addition, the network interface card <NUM> is bound to the SQ of the solid state disk <NUM> in advance. When performing data access processing, the network interface card <NUM> may write the data access related command to the SQ of the solid state disk <NUM>. Optionally, as described above, the solid state disk queue management module in the network interface card <NUM> may implement the command writing process. The network interface card <NUM> includes a network interface card SQ doorbell (doorbell), and the network interface card SQ doorbell may be specifically a register in network interface card <NUM>. When writing the command to the SQ of the network interface card <NUM>, the solid state disk <NUM> may send a notification to the network interface card SQ doorbell, so that the network interface card <NUM> learns that the command is written to the SQ.

The CPU <NUM>, the memory <NUM>, the network interface card <NUM>, and the solid state disk <NUM> are connected through a PCIe bus. Therefore, the network interface card <NUM> has a PCIe interface, to be connected to the CPU <NUM>, the memory <NUM>, and the solid state disk <NUM> through the PCIe interface and bus. In addition, the solid state disk <NUM> has a PCIe interface, to be connected to the CPU <NUM>, the memory <NUM>, and the network interface card <NUM> through the PCIe interface and bus.

The following separately describes a system initialization process, a data writing process, and a data reading process based on the system architecture shown in <FIG>.

<FIG> is a diagram of a system initialization process of the solid state disk access method according to an embodiment of this application. The initialization process may be performed before the client accesses data in the storage device. As shown in <FIG>, the system initialization process includes the following steps.

S401: The CPU <NUM> and the CPU <NUM> establish a network connection by using network interface cards.

Optionally, the CPU <NUM> and the CPU <NUM> may establish the network connection based on a specific network protocol. The specific network protocol may be, for example, the foregoing protocol, for example, RoCE, iWARP, InfiniBand, or TCP/IP.

S402: The CPU <NUM> creates a queue of the network interface card <NUM>, establishes an RDMA connection, and sends queue information of the network interface card <NUM> to the solid state disk <NUM>.

The CPU <NUM> may create the queue of the network interface card <NUM>, including creating the SQ and the network interface card SQ doorbell of the network interface card <NUM>. The created SQ of the network interface card <NUM> has a base address, and the created network interface card SQ doorbell has a doorbell address. In addition, the CPU <NUM> allocates a connection identifier to the RDMA connection of the network interface card <NUM>. In this way, the connection identifier is in a one-to-one correspondence with the SQ, the SQ base address, and the SQ doorbell of the network interface card <NUM>. One storage device may include one solid state disk <NUM>, or may include a plurality of solid state disks <NUM>. One solid state disk <NUM> may have a plurality of SQs. One RDMA connection corresponds to one SQ of one solid state disk <NUM>. One RDMA connection corresponds to one SQ and one SQ doorbell of the network interface card <NUM>, and is used by a solid state disk corresponding to the connection. An SQ of the solid state disk <NUM> corresponding to the RDMA connection is used only for command access of the corresponding RDMA connection.

For example, it is assumed that an RDMA connection <NUM> corresponds to an SQ <NUM> of a solid state disk A of the storage device, and the RDMA connection <NUM> includes a connection identifier <NUM> and an SQ <NUM> and a doorbell <NUM> of a network interface card. In this case, when the solid state disk A needs to transmit data through the RDMA connection <NUM>, the solid state disk A may write a command to the SQ <NUM> of the network interface card. After writing the command to the SQ <NUM> of the network interface card, the solid state disk A sends a notification to the doorbell <NUM> of the network interface card, so that the network interface card learns that there is a to-be-executed command in the SQ <NUM>. When the connection to which the SQ <NUM> of the current network interface card belongs receives a solid state disk access command from the client, a command needs to be written to the SQ <NUM> of the solid state disk A.

After creating the connection of the network interface card <NUM>, the CPU <NUM> sends the queue information of the network interface card <NUM>, including the connection identifier and the base address of the SQ and the address of the network interface card SQ doorbell of the network interface card <NUM>, to the solid state disk <NUM>, so that the solid state disk <NUM> uses the queue information during data access.

S403: The CPU <NUM> creates a queue of the solid state disk <NUM>, and sends queue information of the solid state disk <NUM> to the network interface card <NUM>.

The CPU <NUM> creates the queue of the solid state disk <NUM>, including creating the SQ and the solid state disk SQ doorbell of the solid state disk <NUM>. The created SQ of the solid state disk <NUM> has a base address, and the created solid state disk SQ doorbell has a doorbell address.

After creating the queue of the solid state disk <NUM>, the CPU <NUM> sends the queue information of the solid state disk <NUM>, including the base address of the SQ and the address of the solid state disk SQ doorbell of the solid state disk <NUM>, to the network interface card <NUM>, so that the network interface card <NUM> uses the queue information during data access.

After steps S402 and S403, the network interface card <NUM> sends, to the solid state disk <NUM>, the SQ base address and the doorbell address that are to be used by the solid state disk <NUM>, and the solid state disk <NUM> sends, to the network interface card <NUM>, the base address of the SQ and the address of the SQ doorbell that are to be used by the network interface card <NUM>, to complete binding the SQ of the network interface card <NUM> and the SQ of the solid state disk <NUM>.

S404: The CPU <NUM> allocates the write buffer and the read buffer.

The CPU <NUM> applies for two memory regions in the memory <NUM>, where one is used as the write buffer, and the other is used as the read buffer.

S405: The CPU <NUM> registers information about the write buffer and information about the read buffer with the network interface card <NUM> and the solid state disk <NUM>.

In a manner, the CPU <NUM> may send a start address and an end address of the write buffer and a start address and an end address of the read buffer to the network interface card <NUM> and the solid state disk <NUM>. The network interface card <NUM> and the solid state disk <NUM> may perform a read and write operation on the read buffer and the write buffer based on the addresses of the read buffer and the write buffer.

In another manner, the CPU <NUM> may send a start address and a length of the write buffer and a start address and a length of the read buffer to the network interface card <NUM> and the solid state disk <NUM>. The network interface card <NUM> and the solid state disk <NUM> may perform a read and write operation on the read buffer and the write buffer based on the start addresses and the lengths of the read buffer and the write buffer.

After receiving the information about the write buffer, the network interface card <NUM> correspondingly generates a memory region key (memory region key, MR-key) of the network interface card <NUM>. The MR-key has a unique correspondence with the write buffer, and the write buffer may be indicated by using the MR-key.

After receiving the information about the write buffer, the solid state disk <NUM> correspondingly generates an MR-key of the solid state disk. The MR-key has a unique correspondence with the write buffer, and the write buffer may be indicated by the MR-key.

S406: The CPU <NUM> sends registration information of the write buffer to the CPU <NUM> of the client by using the network interface card <NUM>.

The registration information of the write buffer includes the MR-key of the network interface card <NUM>, the MR-key of the solid state disk <NUM>, and the length of the write buffer.

The registration information of the write buffer may be sent by the network interface card <NUM> to the network interface card <NUM> of the client, and then notified by the network interface card <NUM> to the CPU <NUM> of the client. When the CPU <NUM> needs to write data to the solid state disk <NUM>, the CPU <NUM> may add the information about the write buffer in a data access command. A specific use process is described in detail in the following data writing process.

In an optional manner, the CPU <NUM> may manage the read buffer. In this case, the CPU <NUM> further sends registration information of the read buffer to the CPU <NUM> by using the network interface card <NUM>.

When the CPU <NUM> manages the read buffer, and the CPU <NUM> needs to read data from the solid state disk <NUM>, the CPU <NUM> may add the information about the read buffer in a data access command. A specific use process is described in detail in the following data reading process.

It should be noted that, in a specific implementation process, steps S402 and S403 may be performed in no particular order. S402 may be performed before S403, or S403 may be performed before S402. In addition, steps S404 to S406 and steps S402 and S403 may also be performed in no particular order. S402 and S403 may be performed before S404 to S406, or S404 to S406 may be performed before S402 and S403.

<FIG> is a schematic diagram of a data writing process of the method for accessing the solid state disk according to an embodiment of this application. As shown in <FIG>, a process in which the client writes data to the solid state disk of the storage device includes the following steps.

S501: The CPU <NUM> constructs a write command, and sends the write command and information about the to-be-written data to the network interface card <NUM>.

The to-be-written data may be stored in the write buffer of the memory <NUM> of the client. After determining that the to-be-written data is written to the write buffer, the CPU <NUM> needs to send two pieces of information to the storage device: the to-be-written data and the write command. The to-be-written data is data that needs to be written to the medium of the solid state disk <NUM> of the storage device. The write command is a command that needs to be executed by the solid state disk <NUM>. The write command includes a data source address, and the data source address may be indicated by the MR-key and an offset of the solid state disk <NUM>. As described above, during system initialization, the CPU <NUM> sends the registration information of the write buffer to the CPU <NUM> by using the network interface card <NUM>. In this step, when the CPU <NUM> needs to write the data to the solid state disk <NUM>, the CPU <NUM> adds an MR-key and an offset of a solid state disk that correspond to the write buffer to the constructed write command. The MR-key may indicate the write buffer, and the offset may indicate an offset address of the to-be-written data when the to-be-written data is stored in the write buffer. The solid state disk <NUM> may learn a storage address of the to-be-written data based on the two pieces of information, and write the to-be-written data from the data source address to the medium of the solid state disk <NUM>.

The CPU <NUM> may send the to-be-written data and the write command in any one of the following manners.

In a first manner, the CPU <NUM> constructs an access request. The access request includes information indicating the storage address of the to-be-written data in the memory <NUM> of the client, and the access request further includes the write command and the to-be-written data.

Optionally, the access request may be an RDMA_WRITE_EXT command. The RDMA_WRITE_EXT command includes the to-be-written data, the write command, and the information indicating a storage address of the to-be-written data in the memory <NUM> of the client. In the following embodiments, an example in which the access request is an RDMA_WRITE_EXT command is used for description.

In this manner, the CPU <NUM> sends the RDMA_WRITE_EXT command to the network interface card <NUM> of the client, and the network interface card <NUM> executes the command, to send the to-be-written data and the write command to the storage device.

In an example, it is assumed that the client communicates with the storage device based on the RoCE protocol. In this case, an RDMA_WRITE command may be extended to construct the RDMA_WRITE_EXT command. Specifically, immediate data (Immediate Data) in the RDMA_WRITE command is extended to a size suitable for accommodating the command, for example, extended to <NUM> bytes, and the write command is carried in the immediate data. In addition, an original part that is of the RDMA_WRITE command and that indicates the storage address of the to-be-written data may remain unchanged, and the information about the storage address of the to-be-written data may be carried in this part.

In a second manner, the CPU <NUM> may send the to-be-written data, the write command, and a solid state disk SQ doorbell notification by using three commands. The three commands are a first command, a second command, and a third command. In an example, all the three commands may be RDMA_WRITE commands, and are specifically a first RDMA_WRITE command, a second RDMA_WRITE command, and a third RDMA_WRITE command. The following uses the RDMA_WRITE command as an example for description.

In this manner, the CPU <NUM> sends the first RDMA_WRITE command, where the command includes the to-be-written data; sends the second RDMA_WRITE command, where the command includes the write command; and sends the third RDMA_WRITE command, where the command includes the solid state disk SQ doorbell notification. In this manner, in step S405, the CPU <NUM> needs to register, with the network interface card <NUM>, the base address of the SQ and the address of the SQ doorbell of the solid state disk that are obtained in step S403, and in step S406, the CPU <NUM> needs to send registration information of the base address of the SQ and the address of the SQ doorbell of the solid state disk to the CPU <NUM> by using the network interface card <NUM>. Correspondingly, when sending the second command, the CPU <NUM> calculates a destination address of the write command this time in the SQ of the solid state disk <NUM> with reference to the base address of the SQ of the solid state disk and a quantity of commands sent to the solid state disk, and adds the destination address to the second command. When sending the third command, the CPU <NUM> uses the address of the SQ doorbell as the destination address.

S502: The network interface card <NUM> sends the to-be-written data and the write command to the network interface card <NUM>.

If the CPU <NUM> constructs the command in the first optional manner, the network interface card <NUM> sends the RDMA_WRITE_EXT command to the network interface card <NUM>. Correspondingly, the network interface card <NUM> receives the RDMA_WRITE_EXT command.

The network interface card <NUM> sending the RDMA_WRITE_EXT command to the network interface card <NUM> means that the network interface card <NUM> sends the to-be-written data in the storage address in the memory <NUM> of the client and the write command to the network interface card <NUM> in a format of the RDMA_WRITE_EXT command.

If the CPU <NUM> constructs the command in the second optional manner, the network interface card <NUM> separately sends the first RDMA_WRITE command, the second RDMA_WRITE command, and the third RDMA_WRITE command to the network interface card <NUM>. Correspondingly, the network interface card <NUM> receives the first RDMA_WRITE command, the second RDMA_WRITE command, and the third RDMA_WRITE command.

S503: The network interface card <NUM> writes the to-be-written data to the memory <NUM>.

If the CPU <NUM> constructs the command in the first optional manner, the network interface card <NUM> obtains the to-be-written data from the RDMA_WRITE_EXT command and writes the to-be-written data to a segment of space that is of the write buffer and that is specified in the RDMA_WRITE_EXT command.

If the CPU <NUM> constructs the command in the second optional manner, the network interface card <NUM> obtains the to-be-written data from the first RDMA_WRITE command and writes the to-be-written data to a segment of space that is of the write buffer and that is specified in the first RDMA_WRITE command.

Optionally, the network interface card <NUM> may write the to-be-written data to a storage address in the memory <NUM>. The storage address may be the address that is in the write buffer and that is selected by the CPU <NUM> when the CPU <NUM> constructs the write command in step S501.

In step S405 in the foregoing initialization process of a storage system, when the CPU <NUM> registers the information about the write buffer and the information about the read buffer with the network interface card <NUM>, the network interface card <NUM> correspondingly generates the MR-key of the network interface card. There is the unique correspondence between the MR-key and the write buffer registered with the network interface card <NUM>. In step S406, the CPU <NUM> sends the registration information of the write buffer, including the MR-key of the network interface card and the length of the write buffer, to the CPU <NUM> by using the network interface card <NUM>. When sending the to-be-written data to the storage device, the CPU <NUM> sends the MR-key of the network interface card <NUM> and the offset address of the to-be-written data in the write buffer together with the to-be-written data. Then, in this step, when receiving the to-be-written data, the network interface card <NUM> may determine, based on the MR-key and the offset address that are sent together with the to-be-written data, an address in the write buffer to which the to-be-written data should be written. Then, starting from the address, the network interface card <NUM> writes the to-be-written data to the write buffer.

In an optional manner, the network interface card <NUM> may write the to-be-written data to the storage address in the memory in a direct memory access (direct memory access, DMA) manner. In this manner, the to-be-written data may be written to the storage address in the memory by using a DMA controller without participation of a CPU.

S504: The network interface card <NUM> writes the write command to the SQ of the solid state disk <NUM>.

If the CPU <NUM> constructs the command in the first optional manner, the network interface card <NUM> parses the RDMA_WRITE_EXT command to obtain the write command.

As described above, in the initialization process of the storage system, the CPU <NUM> binds the SQ of the network interface card <NUM> and the SQ of the solid state disk <NUM>. After the binding, the network interface card <NUM> may learn of the base address of the SQ of the solid state disk <NUM>. Therefore, the network interface card <NUM> finds the base address of the SQ of the solid state disk <NUM> based on an RDMA connection identifier in which the RDMA_WRITE_EXT command is located. Then, the network interface card <NUM> calculates the destination address of the write command this time in the SQ of the solid state disk <NUM> based on the base address of the SQ and with reference to the quantity of commands sent to the solid state disk, and writes the write command to the SQ of the solid state disk <NUM>.

As described above, the SQ of the solid state disk <NUM> may be located in the memory <NUM>, or may be located in the solid state disk <NUM>. When the SQ of the solid state disk <NUM> is located in the memory <NUM>, the solid state disk queue management module in the network interface card <NUM> writes the write command to the SQ of the solid state disk <NUM> in the memory <NUM>. When the SQ of the solid state disk <NUM> is located in the solid state disk <NUM>, the solid state disk queue management module in the network interface card <NUM> writes the write command to the solid state disk SQ in the solid state disk <NUM>. In <FIG>, an example in which the SQ of the solid state disk <NUM> is located in the memory <NUM> is used for description.

If the CPU <NUM> constructs the command in the second optional manner, the network interface card <NUM> executes the second RDMA_WRITE command, and writes data carried in the second RDMA_WRITE command, namely, the write command, to an address specified in the second RDMA_WRITE command, namely, the destination address of the SQ of the solid state disk <NUM>. The destination address of the SQ of the solid state disk <NUM> is obtained through calculation by the CPU <NUM> based on the base address of the SQ of the solid state disk <NUM> and with reference to the quantity of sent commands.

S505: The network interface card <NUM> sends the notification to the solid state disk SQ doorbell.

If the CPU <NUM> constructs the command in the first optional manner, the network interface card <NUM> may send the notification to the solid state disk SQ doorbell, so that the solid state disk <NUM> learns that the command is written to the SQ.

As described above, the solid state disk SQ doorbell may be specifically a register in the solid state disk <NUM>. During initial binding of the storage system, the network interface card <NUM> has learned of the address of the solid state disk SQ doorbell. When sending the notification to the solid state disk SQ doorbell, the network interface card <NUM> may send a PCIe packet to the address of the solid state disk SQ doorbell. After the solid state disk SQ doorbell receives the packet, the solid state disk may learn that a to-be-executed command is written to the SQ.

If the CPU <NUM> constructs the command in the second optional manner, the network interface card <NUM> executes the third RDMA_WRITE command, and writes data carried in the third RDMA_WRITE command, namely, notification information sent to the SQ doorbell of the solid state disk <NUM>, to an address specified in the third RDMA_WRITE command, namely, the address of the SQ doorbell of the solid state disk <NUM>.

S506: The solid state disk <NUM> reads the write command from the SQ of the solid state disk <NUM> and parses the write command in the SQ.

S507: The solid state disk <NUM> obtains the to-be-written data from the storage address in the memory <NUM>, and writes the to-be-written data to the medium of the solid state disk <NUM>.

As described in step S501, when constructing the write command, the client adds the MR-key and the offset of the solid state disk <NUM> that correspond to the write buffer to the write command. After parsing the write command in the SQ, the solid state disk <NUM> may parse out the MR-key and the offset of the solid state disk <NUM>. The solid state disk <NUM> may learn of the storage address of the to-be-written data in the memory <NUM> based on the two pieces of information. Then, the solid state disk <NUM> may obtain the to-be-written data from the storage address in the DMA manner, and write the to-be-written data to the medium of the solid state disk.

It should be noted that, in this embodiment of this application, the solid state disk <NUM> writing the to-be-written data to the medium of the solid state disk <NUM> means that the solid state disk <NUM> is an initiator of the data writing process. Therefore, it is referred to herein that the solid state disk <NUM> writes the data to the medium of the solid state disk <NUM>. For example, the data is written in the foregoing DMA manner. An essential process of data writing is as follows: The solid state disk <NUM> requests the DMA controller to write the to-be-written data from the storage address in the memory <NUM> to the medium of the solid state disk <NUM>. After receiving the request of the solid state disk <NUM>, the DMA controller sends the write command to a memory controller, and the memory controller writes the to-be-written data from the storage address in the memory <NUM> to the medium of the solid state disk <NUM>.

Specifically, in step S405, when the CPU <NUM> registers the write buffer with the solid state disk <NUM>, a driver (which is also a part of a program of the storage device) of the solid state disk <NUM> queries an operating system based on a virtual address of the write buffer, to obtain a corresponding physical address, generates an address translation table of the write buffer, and notifies the solid state disk <NUM> of the address translation table. The write buffer is relatively large, but a management granularity of a physical memory is generally relatively small, for example, <NUM> KB. Therefore, the address translation table includes a plurality of consecutive or inconsecutive physical addresses. The solid state disk <NUM> receives the MR-key and the offset, searches the address translation table to obtain a physical address at which the to-be-written data is stored, and may initiate a DMA operation based on the physical address.

S508: The solid state disk <NUM> writes an RDMA_SEND command to the SQ of the network interface card <NUM>.

The RDMA_SEND command carries response information about executing the write command by the solid state disk <NUM>. The response information may be a write success, a write failure, or the like.

As described above, in the initialization process of the storage system, the CPU <NUM> binds the SQ of the network interface card <NUM> and the SQ of the solid state disk <NUM>. After the binding, the solid state disk <NUM> may learn of the base address of the SQ of the network interface card <NUM>. Therefore, the solid state disk <NUM> may calculate, based on the base address of the SQ of the network interface card <NUM> and with reference to a quantity of commands written to the SQ of the network interface card, a destination address of the SQ this time, and write the RDMA_SEND command to the SQ of the network interface card <NUM>.

As described above, the SQ of the network interface card <NUM> may be located in the memory <NUM>, or may be located in the network interface card <NUM>. When the SQ of the network interface card <NUM> is located in the memory <NUM>, the network interface card queue management module in the solid state disk <NUM> writes the RDMA_SEND command to the SQ of the network interface card <NUM> in the memory <NUM>. When the SQ of the network interface card <NUM> is located in the network interface card <NUM>, the network interface card queue management module in the solid state disk <NUM> writes the RDMA_SEND command to the SQ of the network interface card <NUM> in the network interface card <NUM>. In <FIG>, an example in which the SQ of the network interface card <NUM> is located in the memory <NUM> is used for description.

S509: The solid state disk <NUM> sends a notification to the network interface card SQ doorbell.

After the command is written to the SQ of the network interface card <NUM>, the network interface card <NUM> cannot actively learn that the command is written. Therefore, the solid state disk <NUM> may send the notification to the network interface card SQ doorbell, so that the network interface card <NUM> learns that the command is written to the SQ.

As described above, the network interface card SQ doorbell may be specifically a register in the network interface card <NUM>. During the initial binding of the storage system, the solid state disk <NUM> has learned of the address of the network interface card SQ doorbell. When sending the notification to the network interface card SQ doorbell, the solid state disk <NUM> may send a PCIe packet to the network interface card SQ address. After the network interface card SQ doorbell receives the packet, the network interface card <NUM> may learn that a to-be-executed command is written to the SQ.

S510: The network interface card <NUM> reads the RDMA_SEND command from the SQ of the network interface card <NUM>.

S511: The network interface card <NUM> sends the RDMA_SEND command to the network interface card <NUM>.

S512: The network interface card <NUM> parses the RDMA_SEND command, and reports the response information carried in the command to the CPU <NUM>.

After reporting the response information to the CPU <NUM>, the network interface card <NUM> sends an ACK command of the RDMA_SEND command to the network interface card <NUM>.

S513: The network interface card <NUM> writes head pointer information of the SQ of the network interface card <NUM> to a head pointer address of the SQ of the network interface card <NUM>. A head pointer of the SQ of the network interface card <NUM> indicates a location of an executed command in the SQ of the network interface card <NUM>. By reading the head pointer, the solid state disk <NUM> may learn that the SQ of the network interface card <NUM> is empty or full, to learn whether a command can continue to be written to the SQ of the network interface card <NUM>. Optionally, the network interface card <NUM> may write the head pointer information of the SQ to an address, and the solid state disk <NUM> obtains the head pointer information of the SQ by reading the address. The address may be an address of a register in the SSD, or may be an address of the memory.

<FIG> is a schematic diagram of a data reading process of the method for accessing the solid state disk according to an embodiment of this application. As shown in <FIG>, a process in which the client reads data from the solid state disk of the storage device includes the following steps.

S601: The CPU <NUM> constructs a read command, and sends the read command to the network interface card <NUM>.

As described above, during initialization of a storage system, the CPU <NUM> allocates a read buffer in the memory <NUM> of the client. When data in the solid state disk <NUM> of the storage device needs to be read, the CPU <NUM> may use an address in the read buffer as a destination address in the read command when constructing the read command. After reading the data, the storage device sends the data to the destination address.

In a first optional manner, the CPU <NUM> may construct an RDMA_WRITE_EXT command, and the RDMA_WRITE_EXT command is the read command.

A method for constructing the RDMA_WRITE_EXT command is the same as the method in step S501. In addition, in this embodiment, the RDMA_WRITE_EXT command does not carry RDMA data.

In a second optional manner, the CPU <NUM> may send the read command and a solid state disk SQ doorbell notification by using two RDMA_WRITE commands.

In this manner, the CPU <NUM> sends a fourth RDMA_WRITE command, where the command includes the read command; and sends the foregoing third RDMA_WRITE command, where the third RDMA_WRITE command includes the solid state disk SQ doorbell notification. In this manner, in step S405, the CPU <NUM> needs to register, with the network interface card, the base address of the SQ and the address of the SQ doorbell of the solid state disk that are obtained in step S403, and in step S406, the CPU <NUM> needs to send the registration information of the base address of the SQ and the address of the SQ doorbell of the solid state disk to the CPU <NUM> by using the network interface card <NUM>. Correspondingly, when sending the fourth command, the CPU <NUM> calculates a destination address of the write command this time in the SQ of the solid state disk with reference to the base address of the SQ of the solid state disk and a quantity of commands sent to the solid state disk, and adds the destination address to the fourth command. When sending the third command, the CPU <NUM> uses the address of the SQ doorbell as the destination address.

Optionally, if the read buffer of the memory <NUM> is managed by the CPU <NUM>, when sending the read command, the CPU <NUM> may furtheradd a start address of the read buffer in the read command.

S602: The network interface card <NUM> sends the read command to the network interface card <NUM>.

S603: The network interface card <NUM> writes the read command to the SQ of the solid state disk <NUM>.

If the CPU <NUM> constructs the command in the first optional manner, the network interface card <NUM> parses the RDMA_WRITE_EXT command to obtain the read command.

As described above, in an initialization process of the storage system, the CPU <NUM> performs an RDMA connection and binds an SQ of the network interface card <NUM> and the SQ of the solid state disk <NUM>. After the binding, the network interface card <NUM> may learn of the base address of the SQ of the solid state disk <NUM>. Therefore, the network interface card <NUM> finds the base address of the SQ of the solid state disk <NUM> based on an RDMA connection identifier in which the read command is located. Then, the network interface card <NUM> calculates the destination address of the write command this time in the SQ of the solid state disk <NUM> based on the base address of the SQ and with reference to the quantity of commands sent to the solid state disk, and writes the read command to the SQ of the solid state disk <NUM>.

If the CPU <NUM> constructs the command in the second optional manner, the network interface card <NUM> executes the fourth RDMA_WRITE command, and writes data carried in the fourth RDMA_WRITE command, namely, the read command, to an address specified in the fourth RDMA_WRITE command, namely, the destination address of the SQ of the solid state disk <NUM>. The destination address of the SQ of the solid state disk <NUM> is obtained through calculation by the CPU <NUM> based on the base address of the SQ of the solid state disk <NUM> and with reference to the quantity of sent commands.

As described above, the SQ of the solid state disk <NUM> may be located in the memory <NUM>, or may be located in the solid state disk <NUM>. When the SQ of the solid state disk <NUM> is located in the memory <NUM>, the solid state disk queue management module in the network interface card <NUM> writes the read command to the SQ of the solid state disk <NUM> in the memory <NUM>. When the SQ of the solid state disk <NUM> is located in the solid state disk <NUM>, the solid state disk queue management module in the network interface card <NUM> writes the write command to the solid state disk SQ in the solid state disk <NUM>. In <FIG>, an example in which the SQ of the solid state disk <NUM> is located in the memory <NUM> is used for description.

S604: The network interface card <NUM> sends the notification to a solid state disk SQ doorbell.

As described above, the solid state disk SQ doorbell may be specifically a register in the solid state disk <NUM>. During initial binding of the storage system, the network interface card <NUM> has learned of the address of the solid state disk SQ doorbell. When sending the notification to the solid state disk SQ doorbell, the network interface card <NUM> may send a PCIe packet to the address of the solid state disk SQ doorbell. After the solid state disk SQ doorbell receives the packet, the solid state disk <NUM> may learn that a to-be-executed command is written to the SQ.

S605: The solid state disk <NUM> reads the read command from the SQ of the solid state disk <NUM> and parses the read command in the SQ.

S606: The solid state disk <NUM> executes the read command, and stores the read data at a storage address in the memory <NUM>.

Optionally, the storage address in the memory <NUM> may be an address in the read buffer of the storage device.

In a manner, if the read buffer of the memory <NUM> of the storage device is managed by the CPU <NUM>, when sending the read command, the CPU <NUM> adds the address in the read buffer. Correspondingly, the solid state disk <NUM> may store the read data at the storage address in the memory <NUM> based on the address carried in the read command, and specifically store the read data at the storage address in the read buffer.

Specifically, in step S405 in the foregoing initialization process of the storage system, when the CPU <NUM> registers the information about the write buffer and the information about the read buffer with the solid state disk <NUM>, the solid state disk <NUM> correspondingly generates the memory region key (memory region key, MR-Key) of the solid state disk <NUM>. There is the unique correspondence between the MR-Key and the read buffer registered with the solid state disk <NUM>. In step S406, when sending the information about the read buffer to the CPU <NUM> by using the network interface card <NUM>, the CPU <NUM> also sends the MR-Key of the solid state disk <NUM> to the CPU <NUM>. When the storage device sends the read command, the CPU <NUM> sends the memory region key and an offset address of the solid state disk <NUM> together with the read command. Then, in this step, when receiving the read command, the solid state disk <NUM> may determine, based on the memory region key and the offset address that are sent together with the read command, a start address that is of the read buffer and that is used by the read command. Then, starting from the start address, the solid state disk <NUM> writes the data read from a medium of the solid state disk <NUM> to the read buffer.

In another manner, the solid state disk <NUM> may alternatively manage the read buffer. In step S405 in the foregoing initialization process of the storage system, after the CPU <NUM> registers the information about the read buffer with the solid state disk <NUM>, the solid state disk <NUM> autonomously determines to use an area of the read buffer to store the data read from the solid state disk <NUM>. Optionally, the solid state disk <NUM> may write the read data to the storage address in the memory in a DMA manner. In this manner, the read data may be written to the storage address in the memory by using a DMA controller without participation of a CPU.

S607: The solid state disk <NUM> writes the RDMA_WRITE command and an RDMA_SEND command to the SQ of the network interface card <NUM>.

The RDMA_WRITE command is used to send the data in the read buffer of the memory <NUM> to the read buffer of the memory <NUM>.

The RDMA_SEND command carries response information of the read command. The response information may be a read success, a read failure, or the like.

S608: The solid state disk <NUM> sends a notification to the network interface card SQ doorbell.

The solid state disk <NUM> sends the notification to the network interface card SQ doorbell, so that the network interface card <NUM> learns that the command is written to the SQ.

S609: The network interface card <NUM> sends the RDMA_WRITE command and the RDMA_SEND command to the network interface card <NUM>.

The network interface card <NUM> executes the RDMA_WRITE command, to send the data in the read buffer of the memory <NUM> to the network interface card <NUM>. In addition, the network interface card <NUM> executes the RDMA_SEND command, to send the response information to the network interface card <NUM>.

S610: The network interface card <NUM> parses the RDMA_WRITE command, and writes the data to the read buffer of the memory <NUM> of the client in the DMA manner.

S611: The network interface card <NUM> parses the RDMA_SEND command, and reports the response information carried in the command to the CPU <NUM>.

S612: The network interface card <NUM> writes head pointer information of the SQ of the network interface card <NUM> to a head pointer address of the SQ of the network interface card <NUM>.

In embodiments of this application, in a data access process, after receiving the data access command, the network interface card of the storage device may directly interact with the solid state disk to complete data access, without participation of the CPU of the storage device. Therefore, a data access latency can be greatly reduced. In addition, CPU usage of the storage device is greatly reduced.

All or some of the foregoing embodiments may be implemented by using software, hardware, firmware, or any combination thereof. When software is used to implement the embodiments, all or some of the embodiments may be implemented in a form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or functions according to embodiments of this application are all or partially generated. The computer may be a general-purpose computer, a dedicated computer, a computer network, or another programmable apparatus. The computer instructions may be stored in a computer-readable storage medium, or transmitted from a computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions may be transmitted from a website, computer, storage node, or data center to another website, computer, storage node, or data center in a wired (for example, a coaxial cable, an optical fiber, or a digital subscriber line (DSL)) or wireless (for example, infrared, radio, or microwave) manner. The computer-readable storage medium may be any usable medium accessible by a computer, or a data storage device, for example, a storage node or a data center, integrating one or more usable media. The usable medium may be a magnetic medium (for example, a floppy disk, a hard disk, or a magnetic tape), an optical medium (for example, a DVD), a semiconductor medium (for example, a solid state disk (Solid State Disk, SSD)), or the like.

It should be understood that, in embodiments of this application, the term "first" and the like are merely intended to indicate objects, but do not indicate a sequence of corresponding objects.

A person of ordinary skill in the art may be aware that, in combination with the examples described in embodiments disclosed in this specification, units and algorithm steps can be implemented by electronic hardware or a combination of computer software and electronic hardware. Whether these functions are performed by hardware or software depends on particular applications and design constraints of the technical solutions.

It may be clearly understood by a person skilled in the art that, for convenient and brief description, for a detailed working process of the foregoing system, apparatus, and unit, refer to a corresponding process in the foregoing method embodiments.

For example, the described apparatus embodiments are merely examples. For example, division into the units is merely logical function division and may be other division during actual implementation. In addition, the displayed or discussed mutual couplings or direct couplings or communication connections may be implemented through some interfaces.

When the functions are implemented in a form of a software functional unit and sold or used as an independent product, the functions may be stored in a computer-readable storage medium. Based on such an understanding, the technical solutions of this application essentially, or the part contributing to the conventional technology or some of the technical solutions may be implemented in a form of a software product. The software product is stored in a storage medium, and includes several instructions for instructing a computer device (which may be a personal computer, a storage node, a network device, or the like) to perform all or some of the steps of the methods described in embodiments of this application. The foregoing storage medium includes any medium that can store program code, for example, a USB flash drive, a removable hard disk, a read-only memory (Read-Only Memory, ROM), a random access memory (Random Access Memory, RAM), a magnetic disk, or an optical disc.

Claim 1:
A method for accessing a solid state disk, wherein the method is applied to a storage device, the storage device comprises a network interface card (<NUM>), a memory (<NUM>), and the solid state disk (<NUM>), and the method comprises:
receiving, by the network interface card, to-be-written data and a write command, wherein the write command instructs to write the to-be-written data to the solid state disk;
writing, by the network interface card, the to-be-written data to the memory;
writing, by the network interface card, the write command to a submission queue of the solid state disk, wherein the submission queue of the solid state disk is located in the solid state disk;
notifying, by the network interface card, the solid state disk that there is a to-be-executed command in the submission queue, wherein the to-be-executed command is the write command;
obtaining, by the solid state disk, the to-be-written data from the memory and writing the to-be-written data to a medium of the solid state disk according to the write command; and
writing, by the solid state disk, an RDMA_SEND command to a submission queue of the network interface card, where the RDMA_SEND command carries response information about executing the write command by the solid state disk, wherein the submission queue of the network interface card is located in the network interface card.