Patent ID: 12216923

DETAILED DESCRIPTION

The following disclosure provides many different embodiments, or examples, for implementing different features of the provided subject matter. Specific examples of elements and arrangements are described below to simplify the present disclosure. These are, of course, merely examples and are not intended to be limiting. For example, the formation of a first feature over or on a second feature in the description that follows may include embodiments in which the first and second features are formed in direct contact, and may also include embodiments in which additional features may be formed between the first and second features, such that the first and second features may not be in direct contact. In addition, the present disclosure may repeat reference numerals and/or letters in various examples. This repetition is for the purpose of simplicity and clarity and does not in itself dictate a relationship between the various embodiments and/or configurations discussed.

Moreover, spatially relative terms, such as “beneath,” “below,” “lower,” “above,” “upper”, “on” and the like, may be used herein for ease of description to describe one element or feature's relationship to another element(s) or feature(s) as illustrated in the figures. These spatially relative terms are intended to encompass different orientations of the device in use or operation in addition to the orientation depicted in the drawings. The apparatus may be otherwise oriented (rotated 90 degrees or at other orientations) and the spatially relative descriptors used herein may likewise be interpreted accordingly.

As used herein, the terms such as “first”, “second” and “third” describe various elements, components, regions, layers and/or sections, these elements, components, regions, layers and/or sections should not be limited by these terms. These terms may be only used to distinguish one element, component, region, layer or section from another. For example, the terms such as “first”, “second” and “third” when used herein do not imply a sequence or order unless clearly indicated by the context.

As used herein, the singular forms “a,” “an” and “the” are intended to include the plural forms as well, unless the context clearly indicates otherwise. The term “connect,” and its derivatives, may be used herein to describe the structural relationship between components. The term “connected to” may be used to describe two or more components in direct physical or electrical contact with each other. The term “connected to” may also be used to indicate that two or more components are in direct or indirect (with intervening components therebetween) physical or electrical contact with each other, and/or that the two or more components collaborate or interact with each other.

Generally, the computer system includes a plurality of interconnected hosts, wherein each host can include a memory; when performing a task, the computer system uses a memory sharing mechanism to allocate the idle memory in the host to a host with a need of a memory for performing tasks; after the task is completed, the memory is released so that the computer system continues to allocate the memory to other tasks in need of a memory. However, the existing TCP/IP-based or RDMA-based memory sharing mechanism suffers from issues of high latency and low data throughput. The present application achieves the effect of reducing data transmission latency and increasing data throughput by allowing a plurality of hosts to communicate with a plurality of memory expansion devices in a one-to-one manner via a first coherence interconnection protocol and allowing the plurality of memory expansion devices to act as a bridge via a second coherence interconnection protocol. In this case, said first coherence interconnection protocol may be, for example, but not limited to, the CXL protocol; and said second coherence interconnection protocol may be, for example, but not limited to, the Ethernet protocol, the details of which are described below. For ease of discussion, in the following, the first coherence interconnection protocol is referred to as the CXL protocol and the second coherence interconnection protocol is referred to as the Ethernet protocol; however, those having ordinary knowledge in the of the related filed will recognize that this is only an illustrative example and is not intended to limit the present application.

FIG.1is an architectural diagram illustrating a computer system according to one embodiment of the present application. For ease of discussion, in the present embodiment, three sets of hosts and memory expansion devices are used as an example for discussion; however, the present application is not limited thereto. A computer system10includes a first host110A, a second host110B, a third host110C, a first memory expansion device120A, a second memory expansion device120B and a third memory expansion device120C; the first host110A includes a first central processing unit (CPU)112A and a first memory114A, wherein the first memory114A is connected to the first central processing unit112A; the second host110B includes a second central processing unit112B and a second memory114B, wherein the second memory114B is connected to the second central processing unit112B; the third host110C includes a third central processing unit112C and a third memory114C, wherein the third memory114C is connected to the third central processing unit112C. The first memory114A, the second memory114B and the third memory114C can respectively be a dynamic random access memory (DRAM).

The first memory expansion device120A is connected to the first central processing unit112A, the second memory expansion device120B is connected to the second central processing unit112B and the first memory expansion device120A, the third memory expansion device120C is connected to the third central processing unit112C, the first memory expansion device120A and the second memory expansion device120B. Specifically, the first memory expansion device120A, the second memory expansion device120B and the third memory expansion device120C all have the first interface122, the plurality of second interfaces124and the memory interface controller126; the first memory expansion device120A is connected to the first central processing unit112A via its own first interface122(e.g., the first memory expansion device120A is coupled to first processing unit112A via the first interface122inserted in the PCIe slot in the PCB), the second memory expansion device120B is connected to the second central processing unit112B via its own first interface122, the third memory expansion device120C is connected to the third central processing unit112C via its own first interface122, the plurality of second interfaces124of the first to third memory expansion devices120A-120C can be connected via cables, and can communicate, for example, at a rate of 400 Gb per second; this means that the first to third memory expansion devices120A-120C communicate with the corresponding first to third central processing units110A-110C via the CXL protocol, whereas the first memory expansion device120A, the second memory expansion device120B, and the third memory expansion device120C communicate with each other via the Ethernet protocol. With the CXL protocol, the difference in transmission performance between coarse and fine granularity in accessing remote memory can be reduced. In some embodiments, the first to third memory expansion devices120A-120C can be implemented as field programmable gate arrays (FPGAs).

With the foregoing configuration, the first central processing unit112A is capable of accessing not only the first memory114A directly, but also the second memory114B in the second host110B and the third memory114C in the third host110C indirectly via the interconnected first to third memory expansion devices120A-120C; in this way, when the first host110A is allocated to a task and the remaining space in the first memory114A is not sufficient for the currently allocated task, the first central processing unit112A can perform the allocated task by accessing the idle space in the second memory114B and/or the third memory114C, preventing the problem that the first host110A cannot operate due to insufficient resources in the first memory114A, thereby improving system performance. Furthermore, any two of the first to third memory expansion devices120A-120C communicate with each other via two different paths, for example, the first memory expansion device120A can communicate directly with the second memory expansion device120B, or it can communicate with the second memory expansion device120B via the third memory expansion device120C; therefore, even if any one of the memory expansion devices fails, it will not affect the communication between other memory expansion devices, thereby avoiding the single point of failure problem where the entire computer system10cannot work due to the failure of a single memory expansion device.

In the present embodiment, after the first host110A receives a target task, the first central processing unit112A sends a local memory access request to the first memory114A based on the storage size required by the target task; after the first memory114A receives the memory access request, the first memory114A sends the usage condition of its own storage space to the first central processing unit112A with a local reply message; the local reply message carries information about at least the size of the idle space in the first memory114A and the physical address of said idle space. Because the first memory114is the memory used by the first central processing unit112in performing regular operations, the first central processing unit112A completes the execution of the target task by accessing the first memory114A in the case where the local reply message indicates the free space in the first memory114meets the storage size required for the target task. However, in the case where the local reply message indicates that there is no sufficient idle space in the first memory114to meet the storage size required for the target task, or directly by system configuration, the first central processing unit110A may also issue a device memory access request to the first memory expansion device120A.

The memory interface controller126in the first memory expansion device120A converts the device memory access request from the first interface122into a remote device memory access request and sends it to the second memory expansion device120B and the third memory expansion device120C. The memory interface controller126in the second memory expansion device120B converts the remote device memory access request from the second interface124into a host memory access request and sends it to the second host110B, and the second host110B may access the second memory114B based on the host memory access request. Of course, the second memory expansion device120B may also forward the remote device memory access request to the third memory expansion device120C.

Specifically, the first through third memory expansion devices120A-120C are provided with one-hop switches128, shown inFIG.2, which is coupled between the plurality of second interfaces124and the memory interface controller126for determining to which memory expansion device in the computer system10the remote device memory access request is to be passed. For example, when the second memory expansion device120B receives the remote device memory access request from the first memory expansion device120A, the one-hop switch128in the second memory expansion device120B determines, based on the information carried in the remote device memory access request, whether the remote device memory access request is to be sent to the host to which it is connected; if the remote device memory access request is intended for the second host110B connected to the second memory expansion device120B, the one-hop switch128receives the remote device memory access request and transmits it to its own memory interface controller126. In contrast, if this remote device memory access request is not intended to be sent to the second host110B connected to the second memory expansion device120B, then the second memory expansion device120B sends the remote device memory access request from the second interface124to the third memory expansion device120C. The memory interface controller126in the third memory expansion device120C may convert the remote device memory access request from the second interface124into a host memory access request and correspondingly send it to the third host110C, which may access the third memory114C based on the host memory access request.

After the second memory114B or the third memory114C receives the host memory access request, it will send its own memory space usage to the corresponding second memory expansion device120B or third memory expansion device120C with a host reply message; the host reply message carries at least the size of the idle space in the second memory114A/third memory114C and the physical address of the idle space. The first memory expansion device120A receives the host reply message from the second memory expansion device120B and the third memory expansion device120C, and converts the host reply message into a device reply message readable by the first central processing unit112A before passing it to the first central processing unit112A. In this way, the first central processing unit112A can learn from the local reply information and the device reply information whether there is idle space in the computer system10to meet the storage size required for the target task. Although there is no idle space in the first memory114A that meets the target task requirement storage size, the size of the idle space in all the memories in the computer system10meets the target task requirement storage size, and the first central processing unit112A can execute the target task by accessing the first memory114A, the second memory114B, and/or the third memory114C. In other words, identifying the usage condition of the first to third memories114A-114C is a prerequisite for memory sharing.

As discussed above, the first to third memory expansion devices120A-120C communicate with the corresponding first to third central processing units110A-110C via the CXL protocol, which includes three sub-protocols, the CXL.io sub-protocol, the CXL.mem sub-protocol, and the CXL.cache sub-protocol, which together facilitate the consistent sharing of memory resources among the first to third hosts110A-110C. Specifically, the first through third central processing units112A-112C can manage the corresponding first to third memory expansion devices120A-120C via the CXL.io subprotocol. The first central processing unit112A can send device memory access requests to the first memory expansion device120A using the CXL.mem subprotocol, and the second memory expansion device120B and the third memory expansion devices120C may use the CXL.cache subprotocol to send host memory access requests to the corresponding second host112A and third host110B; the second host112A and third host110B may use the CXL.cache subprotocol to send host reply messages to the corresponding second memory expansion devices120B and third memory expansion devices120C to respond to the host memory access requests sent from the second memory expansion device120B and the third memory expansion device120C, and the first memory expansion device120A may send a device reply message using the CXL.mem subprotocol to respond to the device memory access requests.

In certain embodiments, the first to third memory expansion device120A-120C have a register130, as shown inFIG.2, configured to record the usage condition of the memory indicated in the host reply message. In one example, the memory expansion device connected to the central processing unit performing the target task records the usage condition of the memory indicated in the host reply message from the other memory expansion devices to facilitate dynamic, real-time allocation of memory space based on the target task requirements.

In general, the address used by the central processing unit is referred to as the logical address, whereas the address used by the memory is referred to as the physical address. In the present embodiment, although the second memory114B and the third memory114C are not the local memory of the first central processing unit112A that performs the target task, the physical address of their idle space equals to the physical address of the local memory (i.e., the first memory114A). In order to prevent the first central processing unit LI2A from being unable to identify which memory the data generated by executing the target task is stored in, and thus increasing the execution time of the target task or even making it impossible to execute the target task, the memory interface controller126in the expansion device120A connected to the first central processing unit112A executing the target task may be used to process logical address-to-physical address mapping information and algorithms, so as to convert the physical address carried in the host reply message into become a logical address recognizable by the first central processing unit112A upon receiving the host reply message from the second memory expansion device120A and the third memory expansion device120B. In this way, the first central processing unit112A, which performs the target task, does not have to identify the remote memory, thereby reducing the workload. The memory interface controller126further records the one-to-one mapping relationship between the physical and logical addresses of each memory in the register130.

During the execution of the target task, the memory interface controller126may convert a store/access request sent by the first central processing unit112A using a logical address into a physical address readable by the second memory114B and/or the third memory114C for accessing the second memory114B and/or the third memory114C based on the mapping relationship. In the present embodiment, the packet engines132in the first to third memory expansion devices120A-120C, are coupled between the memory interface controller126and the one-hop switch128. When performing data storage, the packet engine132can parse and unpack one or more packets sent by the first central processing unit112A and passed through the one-hop switch128. When accessing the data, the packet engine132assembles the data from memory into packets and passes them via Ethernet to the first central processing unit112A. The memory interface controllers126of the memory expansion devices120A-120C may also include a near-memory processor that can perform operations on the data from remote memory according to the target task, such as multiplication and addition of data, to reduce the workload of the first central processing unit112A. In one example, the near-memory processor may be an application-specific accelerator.

In the present embodiment, the second memory114B and the third memory114C may reserve the idle space therein for use as the storage space by other hosts in advance; of course, the system will also reserve the idle storage space in the first memory114A in advance for use by the second host110B and the third host110C. In one example, in the computer system10, such idle space pre-planned by the system for use by other hosts may have a fixed size; in such a case, it is feasible to prevent the computer system10from failing to perform certain urgent target tasks because of the limited memory resource, thereby ensuring the stable operation of computer system10. In other examples, the idle space in the second memory114B and the third memory114C may be the remaining storage space not allocated for the first to third central processing units112A-112C for executing other target tasks, and the size of these remaining storage spaces are dynamically adjusted as the performance and completion of other target tasks; this situation can significantly reduce the generation of storage fragments and improve the hardware utilization.

In some server specifications, such as a rack-mounted server, a single rack may include eight hosts. For this specification, the present application provides a corresponding embodiment as shown inFIG.3.FIG.3is an architectural diagram of a second embodiment of the computer system20of the present application. The computer system20may be a rack-mounted server and include a plurality of hosts210A-210H and a plurality of memory expansion devices220A-220H. The computer system20and the computer system10may operate according to similar principles. The difference between the computer system20and the computer system10is that the number of the second interfaces224of the memory expansion devices220A-220H in the computer system20needs to be increased to allow any two memory expansion devices224to communicate with each other via two different paths, and the number of memory expansion devices220A-220H through which at least one of the two paths passes is not more than one; whereby, all central processing units212A-212A of the computer system20achieve full connectivity within one hop of memory expansion devices220A-220H, and any one memory expansion device220A-220H breaks down does not affect the communication and the efficiency of communication between the other memory expansion devices220A-220H, thereby solving the problem of single point of failure.

Specifically, when the central processing units212A-212H of the eight hosts210A-210H in the computer system20are connected to the memory expansion devices220A-220H in a one-to-one manner, then when each memory expansion device220A-220H includes four second interfaces224, it is possible for the memory expansion device220A-220connected to these eight hosts210A-210H present in the same rack communicate with each other via two different paths, and at least one of these two paths passes through no more than one of the memory expansion devices220A-220H. For example, the memory expansion device220A may communicate with the memory expansion device220D via a plurality of paths, and one of said plurality of paths may allow no more than one of memory expansion devices220A-220H to be passed through, i.e., the memory expansion device220A communicates directly with the memory expansion device220D. In another example, the memory expansion device220A may communicate with the memory expansion device220E via a plurality of paths, and two of said plurality of paths may allow no more than one of memory expansion devices220A-220H to be passed through, one of said two paths being the memory expansion device220A communicates with the memory expansion device220E via the memory expansion device220D and the other being the memory expansion device220A communicates with the memory expansion device220E via the memory expansion device220H; thus, even if the memory expansion device220H fails, it will not affect the communication between the memory expansion device220A and the memory expansion device220E.

Communication between the memory expansion device220A and the memory expansion device220E can occur when the host210A receives a target task, but the memory214A in the host210A does not have a storage space that meets the storage space required by the target task; in such a case, the central processing unit212A in the host210A will send a device memory access request using the CXL.mem subprotocol and the memory expansion device220A converts the incoming device memory access request into a remote device memory access request and sends it to the memory expansion devices220D and220H via Ethernet; the memory expansion devices220D and220H, after determining that the aforementioned device memory access request is not an access to the hosts210D and210H to which they are connected, send the remote device memory access request to the memory expansion device220E via Ethernet; the memory expansion device220E then converts the remote device memory access request into a send host memory access request and sends the send host memory access request to the host210E using the CXL.cache subprotocol to request access to the memory214E of the host210E.

FIG.4FIG.5are respectively schematic diagrams illustrating methods for use in computer systems according to different embodiments of the present application. The computer systems in these two methods300A and300B for use in a computer system include a computer system20including a plurality of hosts and a plurality of memory expansion devices, and each host includes a central processing unit and a memory. The method300A for use in the computer system may include Steps S302, S304, S306, and S308, and the method300B for use in the computer system may include Steps S302, S304, S310, and S312S308. In Step S302ofFIG.4andFIG.5, each memory expansion device communicates with the corresponding central processing unit via the CXL protocol, whereas in Step S304, each memory expansion device communicates with a portion of the memory expansion devices in the computer system via Ethernet so that any two memory expansion devices can communicate with each other via at least two different paths, and at least one of said two paths passes through no more than one memory expansion device, thereby allowing all central processing units in the computer system to achieve full connectivity of the memory expansion devices within one hop.

After completing the connection between the host and the memory expansion device and the connection between the memory expansion devices, there are different operations depending on whether the memory expansion device is connected to the host that performs the target task. Referring toFIG.4, the memory expansion device connected to the host performing the target task receives a device memory access request sent by the central processing unit via the CXL.mem subprotocol in the CXL protocol (Step S306); this memory expansion device then converts the device memory access request into a remote device memory access request and sends it to the other memory expansion devices connected thereto via Ethernet (Step S308). Referring toFIG.5, a memory expansion device that is not connected to a host performing the target task may receive a device memory access request from another memory expansion device over the Ethernet protocol (Step S310as shown inFIG.5) and decide whether to send the remote device memory access request to the other memory expansion device or to convert the remote device memory access request into a host memory access request according to the information carried in the remote device memory access request (Step S310) and send a host memory access request (step S312) to its connected host via the CXL.mem subprotocol in the CXL protocol to access the memory of said host.

The foregoing outlines features of several embodiments of the present application so that those skilled in the art may better understand the aspects of the present disclosure. Those skilled in the art should appreciate that they may readily use the present disclosure as a basis for designing or modifying other processes and structures for carrying out the same purposes and/or achieving the same advantages of the embodiments introduced herein. Those skilled in the art should also realize that such equivalent constructions do not depart from the spirit and scope of the present disclosure, and that they may make various changes, substitutions, and alterations herein without departing from the spirit and scope of the present disclosure.