Patent ID: 12238866

DETAILED DESCRIPTION OF PREFERRED EMBODIMENTS

The following detailed description refers to the accompanying drawings. The same reference numbers in different drawings may identify the same or similar elements.

Disclosed are systems and methods that provide high-density memory arrangements with high-speed interconnects in a condensed form factor. The systems and methods may include one or more memory-dense compute units and a cooling unit.

The memory-dense compute units may interchangeably connect to one another via a high-speed interconnect fabric to form a massive memory installation within a single rack or chassis. More specifically, the connected memory-dense compute units may provide hundreds of terabytes (“TBs”) of memory from two or more rack units of a rack or chassis, and the combined memory pool may be allocated to and/or accessed by one or more applications, services, processes, and/or processors running on one or more of the memory-dense compute units.

The memory-dense compute units may also connect to the cooling unit via a detachable liquid cooling coupler or connector. The cooling unit may pump a cooling solution across the high-density memory arrangements of each connected memory-dense compute unit to keep each compute unit cool and operational despite having tens of memory modules and one or more processors positioned next to one another in a condensed form factor.

In some embodiments, each memory-dense compute unit may include a specialized motherboard or printed circuit board (“PCB”) with one or more processors and a dense arrangement of up to 48 or more Dual In-line Memory Modules (“DIMMs”), Random Access Memory (“RAM”) modules, and/or other physical memory modules in a half-width one rack unit (“1U”) form factor, wherein the memory-dense compute unit in the half-width 1U form factor may have a height of 1.75 inches, a width between 6-12 inches, and a depth of 24-48 inches. In some such embodiments, each memory-dense compute unit may support 24 TBs or more of memory, and two half-width memory-dense compute units may be inserted side-by-side in a 1U slot of a rack or chassis. Accordingly, connecting four such memory-dense compute units to the same high-speed interconnect switching fabric may create a combined memory pool of 96 TBs or more from just a two slot or two rack unit (“2U”) allocation of space from a single rack or chassis.

In some embodiments, each memory-dense compute unit may include high-speed interconnects for connecting to a high-speed interconnect switch in order to create the larger combined memory pool. The high-speed interconnect switch may be integrated in the chassis backplane of each memory-dense compute unit or a set of two or more memory-dense compute units that are inserted in the same chassis. In some embodiments, two or more chassis with two or more memory-dense compute units may be linked together based on connections between the chassis backplanes.

The high-speed interconnects may use Compute Express Link (“CXL”) and/or Peripheral Component Interconnect Express (“PCIe”) connectivity to allow any processor from a connected set of memory-dense compute units to directly access memory from any memory-dense compute unit of the connected set of memory-dense compute unit while maintaining memory coherence. Moreover, the CXL and PCIe connectivity may provide remote memory access performance that is equivalent or similar to accessing the local memory from a single motherboard or the local memory of a single memory-dense compute unit.

In some embodiments, each memory-dense compute unit may include cooling tubes and blocks interweaving and contacting each of the processors and memory modules, and a cooling connector for connecting the cooling tubes and blocks to a manifold of the cooling unit. Once the memory-dense compute unit is connected to the cooling unit, the cooling connector may receive a cooling solution from the cooling unit. The cooling solution may flow through the cooling tubes and blocks contacting each of the installed memory modules and processors. The cooling solution removes heat from each installed memory module and processor of the memory-dense compute unit, and returns to the cooling unit via the cooling connector where it is once again cooled before cycling back through each of the memory-dense compute units that are connected to cooling unit manifold. The cooling unit may be contained within a separate full length and full width 1U to 4U form factor, and may be added to the same rack or chassis as the one or more memory-dense compute units that connect to the cooling unit and that connect to one another via the high-speed interconnect switch to form the massive memory pool.

Accordingly, the systems and methods may provide hundreds of TBs, and potentially, petabytes of memory from a single rack or chassis. The massive memory pool may be accessed by the one or more processors of the memory-dense compute units that contribute to the massive memory pool, and may be directly accessed as if the entirety of the combined memory pool was local to the processors and/or without a performance penalty associated with a processor on a first memory-dense compute unit accessing a block of memory from a memory module of a second memory-dense compute unit.

FIG.1illustrates an example architecture for a memory-dense compute unit100in accordance with some embodiments presented herein. Memory-dense compute unit100may be housed within a blade, sled, tray, and/or other case with a half-width 1U form factor. In other words, memory-dense compute unit100may occupy half of one slot within a server rack or chassis. Memory-dense compute unit100may include specialized motherboard or PCB101with up to 48 depth-wise and/or horizontally aligned memory module slots103positioned on either side of one or more processors105or processor sockets. The depth-wise and/or horizontal alignment may position memory module slots103to be parallel with a front face of the rack or chassis, and/or to deviate by 90 degrees from a length-wise or vertical alignment. The length-wise or vertical alignment of memory modules is used for forced-air cooling implementations because the length-wise or vertical alignment of memory modules creates fewer obstructions for air flowing from the front to the back (or back to front) of the PCB101than a depth-wise and/or horizontal alignment or memory modules. However, the length-wise or vertical alignment of memory modules provides a less dense arrangement of memory modules than the depth-wise or horizontal alignment of memory modules in the half-width or full-width 1U form factor.

Each memory module slot103of PCB101may be separated by as little as 10 millimeters from neighboring memory module slots103. Each memory module slot103may receive a DIMM, RAM module, and/or other physical memory module. Each memory module slot103may support the maximum amount of memory that is contained by a single memory module. For instance, current capacity of a memory module may be 512 gigabytes (“GBs”). However, the memory module capacity may increase such that each memory module contains one or more TBs. Each memory module slot103may also support the maximum memory module speed or rating. For instance, each memory module slot103may support Double Data Rate 5 Synchronous Dynamic Random-Access Memory (“DDR-5 SDRAM”) modules with a clock rate of 6 gigahertz (“GHz”).

One or more processors105may include central processing units (“CPUs”), graphical processing unit (“GPUs”), and/or other integrated circuits that perform computations and/or other processing operations based on data that is stored in memory. Each processor105may manage a separate bank of memory modules. For instance, first processor105may manage access to memory installed in the first24memory module slots103, and second processor105may manage access to memory installed in the second24memory module slots103.

The rear of memory-dense compute unit100may include cooling connector107, power connector109, and high-speed interconnect interface111. These connectors107and109and interface111allow for memory-dense compute unit100to be interchangeably inserted into and out from a rack or chassis, and allow memory-dense compute unit100to be linked to other memory-dense compute units100that are connected to the same rack or chassis, to receive power from the rack or chassis, and/or to be actively cooled by a cooling unit installed on the same rack or chassis without any manual user configuration, wiring, or user-established connectivity. Memory-dense compute unit100may be inserted into a slot of the rack or chassis to connect cooling connector107to the manifold of the cooling unit, power connector109to a power supply or power interface, and high-speed interconnect interface111to a PCIe switch. In some embodiments, the switch may be located in the chassis backplane and one or more memory-dense compute units100may be connected to the chassis backplane (e.g., high-speed interconnect interface111of the one or more memory-dense compute units100may connect to a different port of the PCIe switch that is integrated in the chassis backplane).

Cooling connector107may include a dripless and/or quick disconnect connector. Cooling connector107may have a first port for receiving the cooling solution from the cooling unit, and a second port for returning the cooling solution back to the cooling unit once the cooling solution has circulated through memory-dense compute unit100and has absorbed heat from each of the memory modules, processors, and/or other heat generating components of memory-dense compute unit100.

Tubing113may be used to circulate the cooling solution from the first port of cooling connector107over the entire layout of memory-dense compute unit100and back to the second port. Specifically, tubing113may route across, through, or past each memory module slot103and processor105of memory-dense compute unit100. In some embodiments, tubing113supplies the cooling solution to or through cooling blocks115that are aligned with each memory module slot103, that abut each installed memory module, and/or that abut each processor105.

Each cooling block115may include a thermal interface material, a liquid-cooled heatsink, or other structure that makes surface contact with one side of an installed memory module or processor105. The cooling solution passes through cooling block115to cool or transfer heat away from the side that is in direct contact with one side of the memory module or processor105.

FIG.2illustrates an example of liquid cooling the high-density memory arrangement via cooling connector107, tubing113, and cooling block115in accordance with some embodiments presented herein. As shown inFIG.2, tubing113may connect to the first port of cooling connector107, and may route between or past multiple memory module slots103. Tubing113may feed the cooling solution through one end of a connected cooling block115that is positioned between two or more memory module slots103. In other words, cooling block115may be sandwiched between two memory modules or cooling block115may be positioned to directly abut one or more memory modules from one or more sides in order to draw heat away from the contacted memory modules, thereby cooling the memory modules. Specifically, the cooling solution enters an interior of cooling block115where it cools one or more abutting memory modules before exiting cooling block115and circulating through the remaining length of tubing113. In some embodiments, tubing113may receive the cooling solution on an opposite end or an output port of cooling block115after heat has been transferred away from the abutting memory modules. Tubing113continues feeding the cooling solution to other connected cooling blocks115in other parts of memory-dense compute unit100. In some embodiments, tubing113may route the cooling solution along a single path. In some other embodiments, tubing113may divert the cooling solution across multiple paths to better balance the cooling of components (e.g., memory modules and/or processors105) at the front and rear of memory-dense compute unit100.

In some embodiments, cooling block115may have a thermal interface material that is spongy, flexible, and/or compressible and through which the cooling solution may flow without being directly exposed or transferred to the contacted components or electronics. In some embodiments, cooling block115may have a hollow interior and an exterior housing formed from aluminum, copper, and/or another thermally conductive material. The thermally conductive material may aid in absorbing heat away from another surface that comes into contact with the thermally conductive exterior, and the cooling solution passing along the interior of the thermally conductive exterior may transfer the heat off the exterior surface. Cooling block115may have a rectangular shape to match the size and shape of the memory modules.

With reference back toFIG.1, power connector109may include a male or female coupler that connects to an opposite male or female couple at the rear of the rack or chassis. Power connector109may be used to supply power from the rack or chassis to memory-dense compute unit100. In some embodiments, power connector109may supply a 12 Volt (“V”) or 48V direct current (“DC”) to power operation of the installed memory modules and processors105.

High-speed interconnect interface111may provide CXL connectivity via a PCIe interface to connect memory-dense compute unit100to other memory-dense compute units100. Specifically, high-speed interconnect interface111may include a port or interface that connects to a CXL switch or a PCIe switch within the backplane of the chassis containing one or more memory-dense compute units100. High-speed interconnect interface111may provide 32 PCIe lanes with up to 64 gigabit per second (“Gbps”) or gigatransfers per second (“GT/s”) of throughput on each PCIe lane. The processors of one memory-dense compute unit100may access the installed memory on one or more other memory-dense compute units100on the same PCIe communication bus with the same or similar performance as accessing local memory due to the extremely low latency and extremely high throughput links provided by the PCIe communication bus.

Although not shown inFIG.1, memory-dense compute unit100may also include one or more input/output ports, such as Universal Serial Bus (“USB”) ports, for directly connecting to and/or managing memory-dense compute unit100, and/or a network port, such as an Ethernet port, for remotely accessing and/or managing memory-dense compute unit100. Memory-dense compute unit100may include other components or elements, or a different arrangement of the disclosed components. For instance, memory-dense compute unit100may include more or less memory module slots103, more or less processors105, different types of processors (e.g., a CPU and a GPU), temperature sensors for monitoring the temperature at different parts of memory-dense compute unit100, and/or flow regulators for adjusting the flow of the cooling solution or the amount of cooling provided to different parts of memory-dense compute unit100based on the monitored temperature. In some embodiments, the flow regulators may include electronically controlled valves for diverting the cooling solution across different sections of tubing113, or may include an interface for communicating with the cooling unit to adjust the pump pressure and/or the rate of cooling solution supplied to memory-dense compute unit100via cooling connector107.

FIG.3illustrates an example architecture for cooling unit300in accordance with some embodiment presented herein. Cooling unit300may also be housed within a blade, sled, tray, and/or other case with a 1U form factor. However, cooling unit300may be a full-length and full-width module that occupies an entire 1U slot of the rack or chassis, whereas each memory-dense compute unit100may be half-width such that two memory-dense compute units100may be placed side-by-side to occupy an entire 1U slot of the rack or chassis. In some embodiments, cooling unit300may be a larger 2U, 3U, or 4U form factor depending on the cooling requirements and/or number of memory-dense compute units100that are to be simultaneously cooled by cooling unit300. Cooling unit300may include cooling solution reservoir301, pump303, manifold305, radiator307, one or more fans309, belly pan311, and one or more sensors313.

Cooling solution reservoir301may include a tank or receptacle in which the cooling solution or liquid is stored. In some embodiment, the liquid within cooling solution reservoir301may be distilled water or a specialized liquid or gaseous coolant.

Reservoir301may be sized to contain at least enough cooling solution to route through tubing113of two or more memory-dense compute units100that may connect to cooling unit300via manifold305. For instance, if manifold305contains four ports for connecting to four different memory-dense compute units100and each memory-dense compute unit100contains 10 feet of quarter-inch tubing113, then reservoir301may be sized to contain more cooling solution than is necessary to flow through tubing113of all four memory-dense compute units100.

Pump303may generate pressure for circulating the cooling solution from cooling solution reservoir301through radiator307to manifold305and across tubing113of one or more memory-dense compute units100that are connected to manifold305. Pump303may operate at one speed or at variable speeds to generate different amounts of pressure or flow depending on the number of connected memory-dense compute units100or the temperature of the cooling solution returning to cooling unit300. In some embodiments, one or more sensors313may measure the temperature of the cooling solution flowing back to cooling unit300, and may adjust operation of pump303to increase or decrease the flow of the cooling solution based on the detected temperature.

Manifold305may include an extension that rises above and/or under cooling unit300from the rear of cooling unit300. The extension may include one or more manifold interfaces for receiving cooling connector107of memory-dense compute unit100at rack or chassis slots below or above cooling unit300. Each manifold interface may include a first port for supplying cooling solution to a connected memory-dense compute unit100(e.g., after the cooling solution is brought to an ambient or chilled temperature by radiator307), and a second port for receiving the cooling solution after it has flowed through the connected memory-dense compute unit100and absorbed heat from the components therein.

In some embodiments, manifold305may include a downward extension that reaches 1U or 1.75 inches below cooling unit300to align two pairs of horizontally-aligned manifold interfaces with cooling connectors107of two side-by-side memory-dense compute units100that are inserted in the rack or chassis slot 1U directly below cooling unit300. In some such embodiments, manifold305may also or alternatively include an upward extension that reaches 1U or 1.75 inches above cooling unit300to align two pairs of horizontally-aligned manifold interfaces with cooling connectors107of two side-by-side memory-dense compute units100inserted in the rack or chassis slot 1U directly above cooling unit300.

In some embodiments, manifold305may include a downward extension that reaches 2U or 3.5 inches below cooling unit300with a first pair of manifold interfaces that align with cooling connectors107of side-by-side memory-dense compute units100in the slot (e.g., 1U) directly below cooling unit300, and a second pair of manifold interfaces that align with cooling connectors107of side-by-side memory-dense compute units100two slots (e.g., 2U) under cooling unit300. A similar extension with two pairs of manifold interfaces may extend upward from the rear of cooling unit300for simultaneous cooling of up to eight memory-dense compute units100.

Radiator307may include a set of distributed fins that span the full length and width of cooling unit300or the 1U form factor. Radiator307may maximize the surface area across which the cooling solution returning from the connected memory-dense compute units100may be cooled via airflow. Specifically, the cooling solution received from memory-dense compute units100via manifold305may be pumped across radiator307, and wind generated by one or more fans309may flow over radiator307to transfer heat away and out of cooling unit300, thereby cooling the cooling solution within radiator307down to an ambient or lower temperature.

Belly pan311may be used to capture any of the cooling solution that leaks out from cooling solution reservoir301, pump303, radiator307, and/or the tubing of cooling unit300. One or more sensors313may include a leak detector to shut off cooling unit300and/or generate alerts in response to detecting liquid collecting in belly pan311.

In some embodiments, cooling unit300may include other components or different components than those described with reference toFIG.3. For instance, cooling unit300may include a power connector for connecting to an external power supply and/or receiving power for pump303, one or more fans309, etc. Manifold305may be modified to replace the vertical extension with the manifold interfaces with a box at the rear of cooling unit300that includes the manifold interfaces, and flexible tubing may be used to establish a water-tight connection between cooling connector107of each memory-dense compute unit100and the modified manifold305of cooling unit300.

FIG.4illustrates rack or chassis400configured with four memory-dense compute units100-1,100-2,100-3, and100-4(collectively referred to as memory-dense compute units100or individual referred to as memory-dense compute unit100) and cooling unit300in accordance with some embodiments presented herein. Each of the memory-dense compute units100may be connected to the same CXL or PCIe switch401via their respective high-speed interconnect interface111. CXL or PCIe switch401may be integrated in the backplane of rack or chassis400, or may be a standalone device that is inserted into its own slot of rack or chassis400. The CXL protocol may pool the memory from each memory-dense compute unit100together to allow any processor from the four memory-dense compute units100to directly access some or all of the memory via the PCIe link regardless of which memory-dense compute unit100hosts the memory. Accordingly, the four memory-dense compute unit100configuration illustrated inFIG.4may allocate 96 TBs of memory to a single application running on one or more processors105of memory-dense compute units100-1,100-2,100-3, and100-4.

FIG.5provides a rear exterior view of memory-dense compute unit100in accordance with some embodiments presented herein. As shown inFIG.5, high-speed interconnect interface111may include a set of blind mate connectors that extend out from behind memory-dense compute100. Cooling connector107may be located adjacent to high-speed interconnect interface111(not shown).

The set of blind mate connectors may engage with a panel mount wired harness about a chassis backplane. In particular, the set of blind mate connectors may be constructed with self-aligning features so that the blind mate connectors may guide themselves into the correct mating position with the panel mount wired harness about the chassis backplane. In some embodiments, the set of blind mate connectors may slide onto, snap on, and/or uses guided pins to connect to the panel mount wired harness.

FIG.6illustrates a partial cutaway view of chassis backplane600with a panel mount wired harness601that receives the set of blink mate connectors from four different memory-dense compute units100. Panel mount wired harness601may connect the set of blind mate connectors to the backplane PCB and the PCIe/CXL switch integrated circuit (“IC”)603. Panel mount wired harness601may provide low frequency loss for the high speed signals passing over the PCIe bus and may reduce or eliminate the need for buffering and retimers on the PCIe bus. In particular, panel mount wired harness601may minimize the PCB trace lengths so a lower cost PCB material may be used on the backplane without adversely impacting the signal integrity of the high-speed data interfaces.

FIG.7illustrates example operation of cooling unit300cooling two memory-dense compute units100-1and100-2in accordance with some embodiments presented herein. First memory-dense compute unit100-1, second memory-dense compute unit100-2, and cooling unit300may be inserted in three consecutive 1U slots of a rack or chassis, and may be connected to a backplane of the rack or chassis that has an integrated CXL or PCIe switch. Additionally, first and second memory-dense compute units100may connect to manifold305of cooling unit300.

Cooling unit300may pump (at702) the cooling solution to each memory-dense compute unit100-1and100-2. The cooling solution enters each memory-dense compute unit100-1and100-2via cooling connector107and flows (at704) through the integrated tubing113. Tubing113distributes (at706) the cooling solution across each cooling block115that is in contact with one or more memory modules or processors105of that memory-dense compute unit100-1or100-2. Heat is transferred from the memory modules and processors105to the contacting surface of each cooling block115, and is absorbed by the cooling solution flowing through each cooling block115. The cooling solution flows (at708) out of each memory-dense compute unit100and back into cooling unit300. Specifically, the cooling solution enters manifold305, is deposited into reservoir301, and is then spread across radiator307. The absorbed heat is dissipated (at710) back into the air flowing over and across radiator307. One or more fans309may expedite the cooling by increasing the volume of colder air that flows into cooling unit300and over radiator307, and by increasing the volume of air that is warmed by the heat dissipating off radiator307that flows out of cooling unit300.

FIG.8illustrates the creation and accessing of a massive memory pool using the high-speed interconnectivity between four or more memory-dense compute units100-1,100-2,100-3, and100-4in accordance with some embodiments presented herein. Memory-dense compute units100may be housed in the same rack or chassis, and may be connected to the same CXL or PCIe switch801via high-speed interconnect interface111. CXL or PCIe switch801may be integrated in the rack or chassis backplane or may be a separate device that is added to the rack or chassis and that memory-dense compute units100are connected to.

CXL is a protocol that runs atop PCIe generation5links, and that establishes cache-coherent links between the connected memory-dense compute units100. Specifically, first processor105-1on first memory-dense compute unit100-1may establish and maintain memory coherency with memory on other connected compute units100-2,100-3, and100-4with extremely low overhead and latency via the CXL protocol. First processor105-1may directly access the memory of other compute units100-2,100-3, and100-4as if the memory from other compute units100-2,100-3, and100-4was locally managed by first processor105-1of first connected compute unit100-1. In other words, first processor105-1may perform remote memory load and/or store operations on memory of any memory-dense compute unit100that is connected to the same CXL or PCIe switch801as first connected compute unit100-1.

Connected compute units100-1,100-2,100-3, and100-4may use the CXL.io protocol of CXL for initialization, link-up, device discovery and enumeration, and register access. Connected compute units100-1,100-2,100-3, and100-4may use the CXL.cache protocol to define interactions between each processor105and the memory of other connected compute units100, and/or the CXL.memory protocol to provide each processor105with direct access to the memory of other connected compute units100.

As shown inFIG.8, first processor105-1on first memory-dense compute unit100-1may be tasked (at802) with running a first application. The first application may request a first allocation of memory that first105-1obtains (at804) and/or allocates from the memory of first memory-dense compute unit100-1.

Second processor105-2on second memory-dense compute unit100-2may be tasked (at806) with running the first application in conjunction with first processor105-1. For instance, the first application may require more processing power or cores than are offered by first processor105-1alone. Accordingly, second processor105-2may be provisioned to execute the first application in conjunction with first processor105-1using the single block of memory that is allocated (at804) for the first application. Rather than transfer the data of the first application from the allocated memory of first memory-dense compute unit100-1to the local memory of second memory-dense compute unit100-2before it is processed, second processor105-2may directly access (at808) the allocated memory of first memory-dense compute unit100-1in executing the first application alongside first processor105-1. In other words, both first processor105-1of first memory-dense compute unit100-1and second processor105-2of second memory-dense compute unit100-2may perform load, store, and read operations directly to the allocated memory from first memory-dense compute unit100-1based on the memory coherency and direct access provided by the CXL protocol and the low-latency high-speed interconnection of the PCIe links created between the connected compute units100by CXL or PCIe switch801.

Fourth processor105-4on fourth memory-dense compute unit100-4may be tasked (at810) with running a second application that requires more memory than is available on fourth memory-dense compute unit100-4due to prior memory allocations. Accordingly, fourth processor105-4may allocate (at812) a first block of memory from the remaining memory of fourth memory-dense compute unit100-4, and may allocate (at812) a second block of memory from memory of second memory-dense compute unit100-2in order to run the second application. The PCIe link between connected memory-dense compute units100-1,100-2,100-3, and100-4provides sufficient bandwidth with which fourth processor105-4may access the local and remote memory with insignificant difference in the access times. Moreover, the CXL protocols allow fourth processor105-4to directly access (at814) the remote memory of second memory-dense compute unit100-2. Specifically, fourth processor105-4may bypass second processor105-2in order to directly access, load, store, and read from the allocated remote memory of second memory-dense compute unit100-2, and may perform the memory access operations without having to first transfer the data from the remote memory of second memory-dense compute unit100-2to local memory of fourth memory-dense compute unit100-4. In this manner, fourth processor105-4may directly access (at814) the remote memory of second memory-dense compute unit100in the same manner and with the same performance profile as the locally allocated memory from fourth memory-dense compute unit100-4.

FIG.9presents a process900for providing a massive memory allocation via high-speed interconnected high-density memory arrangements in accordance with some embodiments presented herein. Process900may be implemented by a system comprised of one or more PCIe linked and/or CXL interconnected memory-dense compute units100that receive cooling from cooling unit300.

Process900may include establishing (at902) high-speed interconnectivity between two or more memory-dense compute units100that are connected to a common CXL or PCIe switch. Establishing (at902) the high-speed interconnectivity may include provisioning a number of PCIe lanes with which each compute unit100connects to the CXL or PCIe switch and/or communicates with the other connected compute units100or devices. In some embodiments, the CXL or PCIe switch may support144or more PCIe lanes, and may allocate 32 PCIe lanes to each connected memory-dense compute unit100. Unused PCIe lanes may be used to link the switch to other switches that interconnect other memory-dense compute units100.

Process900may include performing (at904) discovery to detect each memory-dense compute unit100that is connected to the CXL or PCIe switch, and to detect the available memory of each compute unit100. The discovery may be implemented according to one or more of the CXL protocols.

Process900may include selecting (at906) a particular processor as a host processor for executing a particular application with a large memory requirement. In some embodiments, the system may select (at906) whichever is the least loaded processor for execution of the particular application or the associated task, or may select (at906) whichever processor is compatible with the particular application or the associated task. For instance, the task may involve rendering an image, and selecting (at906) the particular processor may include selecting an available GPU over an available CPU or other processor for the task.

Process900may include allocating (at908) memory from one or more compute units to satisfy the memory requirements of the particular application. In some embodiments, the memory may be first allocated from the local memory of the memory-dense compute unit100where the selected processor is located, and supplementing the allocation with memory from one or more interconnected memory-dense compute units100if more memory is required. The host processor may directly allocate memory from remote memory-dense compute units100using the CXL protocol, and the CXL protocol may maintain memory coherency with other processors that attempt to access the same memory.

Process900may include directly accessing (at910) the allocated memory from the one or more memory-dense compute units100using the CXL protocol and high-speed PCIe links connecting the selected processor to the memory modules that contain the allocated memory. Accordingly, the selected processor may execute the particular execution by loading and storing data directly into the allocated memory regardless of the memory being local or remote to the selected processor.

FIG.10is a diagram of example components of device1000. Device1000may be used to implement one or more of the devices or systems described above (e.g., memory-dense compute unit100, cooling unit300, etc.). Device1000may include bus1010, processor1020, memory1030, input component1040, output component1050, and communication interface1060. In another implementation, device1000may include additional, fewer, different, or differently arranged components.

Bus1010may include one or more communication paths that permit communication among the components of device1000. Processor1020may include a processor, microprocessor, or processing logic that may interpret and execute instructions. Memory1030may include any type of dynamic storage device that may store information and instructions for execution by processor1020, and/or any type of non-volatile storage device that may store information for use by processor1020.

Input component1040may include a mechanism that permits an operator to input information to device1000, such as a keyboard, a keypad, a button, a switch, etc. Output component1050may include a mechanism that outputs information to the operator, such as a display, a speaker, one or more LEDs, etc.

Communication interface1060may include any transceiver-like mechanism that enables device1000to communicate with other devices and/or systems. For example, communication interface1060may include an Ethernet interface, an optical interface, a coaxial interface, or the like. Communication interface1060may include a wireless communication device, such as an infrared (“IR”) receiver, a Bluetooth® radio, or the like. The wireless communication device may be coupled to an external device, such as a remote control, a wireless keyboard, a mobile telephone, etc. In some embodiments, device1000may include more than one communication interface1060. For instance, device1000may include an optical interface and an Ethernet interface.

Device1000may perform certain operations relating to one or more processes described above. Device1000may perform these operations in response to processor1020executing software instructions stored in a computer-readable medium, such as memory1030. A computer-readable medium may be defined as a non-transitory memory device. A memory device may include space within a single physical memory device or spread across multiple physical memory devices. The software instructions may be read into memory1030from another computer-readable medium or from another device. The software instructions stored in memory1030may cause processor1020to perform processes described herein. Alternatively, hardwired circuitry may be used in place of or in combination with software instructions to implement processes described herein. Thus, implementations described herein are not limited to any specific combination of hardware circuitry and software.

The foregoing description of implementations provides illustration and description, but is not intended to be exhaustive or to limit the possible implementations to the precise form disclosed. Modifications and variations are possible in light of the above disclosure or may be acquired from practice of the implementations.

The actual software code or specialized control hardware used to implement an embodiment is not limiting of the embodiment. Thus, the operation and behavior of the embodiment has been described without reference to the specific software code, it being understood that software and control hardware may be designed based on the description herein.

For example, while series of messages, blocks, and/or signals have been described with regard to some of the above figures, the order of the messages, blocks, and/or signals may be modified in other implementations. Further, non-dependent blocks and/or signals may be performed in parallel. Additionally, while the figures have been described in the context of particular devices performing particular acts, in practice, one or more other devices may perform some or all of these acts in lieu of, or in addition to, the above-mentioned devices.

Even though particular combinations of features are recited in the claims and/or disclosed in the specification, these combinations are not intended to limit the disclosure of the possible implementations. In fact, many of these features may be combined in ways not specifically recited in the claims and/or disclosed in the specification. Although each dependent claim listed below may directly depend on only one other claim, the disclosure of the possible implementations includes each dependent claim in combination with every other claim in the claim set.

Further, while certain connections or devices are shown, in practice, additional, fewer, or different, connections or devices may be used. Furthermore, while various devices and networks are shown separately, in practice, the functionality of multiple devices may be performed by a single device, or the functionality of one device may be performed by multiple devices. Further, while some devices are shown as communicating with a network, some such devices may be incorporated, in whole or in part, as a part of the network.

To the extent the aforementioned embodiments collect, store or employ personal information provided by individuals, it should be understood that such information shall be used in accordance with all applicable laws concerning protection of personal information. Additionally, the collection, storage and use of such information may be subject to consent of the individual to such activity, for example, through well-known “opt-in” or “opt-out” processes as may be appropriate for the situation and type of information. Storage and use of personal information may be in an appropriately secure manner reflective of the type of information, for example, through various encryption and anonymization techniques for particularly sensitive information.

Some implementations described herein may be described in conjunction with thresholds. The term “greater than” (or similar terms), as used herein to describe a relationship of a value to a threshold, may be used interchangeably with the term “greater than or equal to” (or similar terms). Similarly, the term “less than” (or similar terms), as used herein to describe a relationship of a value to a threshold, may be used interchangeably with the term “less than or equal to” (or similar terms). As used herein, “exceeding” a threshold (or similar terms) may be used interchangeably with “being greater than a threshold,” “being greater than or equal to a threshold,” “being less than a threshold,” “being less than or equal to a threshold,” or other similar terms, depending on the context in which the threshold is used.

No element, act, or instruction used in the present application should be construed as critical or essential unless explicitly described as such. An instance of the use of the term “and,” as used herein, does not necessarily preclude the interpretation that the phrase “and/or” was intended in that instance. Similarly, an instance of the use of the term “or,” as used herein, does not necessarily preclude the interpretation that the phrase “and/or” was intended in that instance. Also, as used herein, the article “a” is intended to include one or more items, and may be used interchangeably with the phrase “one or more.” Where only one item is intended, the terms “one,” “single,” “only,” or similar language is used. Further, the phrase “based on” is intended to mean “based, at least in part, on” unless explicitly stated otherwise.