Patent Description:
This object is solved by the subject matter of the independent claim.

A computing device comprises two or more compute nodes, that each include two or more processor cores. Each compute node comprises an independently coherent domain that is not coherent with other compute nodes. A central IO die is communicatively coupled to each of the two or more compute nodes. A plurality of natively-attached volatile memory units are attached to the central IO die via one or more memory controllers. The central IO die includes one or more home agents for each compute node. The home agents are configured to map memory access requests received from the compute nodes to one or more addresses within the natively-attached volatile memory units.

As discussed above, data centers typically include large numbers of discrete compute nodes, such as server computers or other suitable computing devices. Such compute nodes may be referred to as "host computing devices," or "hosts," as they may in some cases be used to host a plurality of virtual machines. It will be understood, however, that a compute node may be used for any suitable computing purpose, and need not be used for hosting virtual machines specifically. Furthermore, in some examples, a compute node may be implemented as a virtual machine.

Depending on the specific implementation, each individual compute node may have any suitable collection of computer hardware. Regardless, each individual compute node will typically include some local collection of hardware resources, including data storage, memory, processing resources, etc. However, computational workloads (e.g., associated with data center customers) are often not uniformly distributed between each of the compute nodes in the data center. Rather, in a common scenario, a subset of compute nodes in the data center may be tasked with resource-intensive workloads, while other compute nodes sit idle or handle relatively less resource-intensive tasks. Thus, the total resource utilization of the data center may be relatively low, and yet completion of some workloads may be resource-constrained due to how such workloads are localized to individual nodes. This represents an inefficient use of the available computer resources, and is sometimes known as "resource stranding," as computer resources that could potentially be applied to computing workloads are instead stranded in idle or underutilized nodes.

This problem can be mitigated when hardware resources are pulled out of individual compute nodes and are instead disaggregated as separate resource pools that can be flexibly accessed by connected compute nodes. For example, the present disclosure contemplates scenarios where resources such as physical memory, I/O interfaces, cache, and virtualization resources are pooled for use across all compute nodes in a system. For example, volatile memory hardware (e.g., random-access memory (RAM)), may be collected as part of a disaggregated memory pool, from which it may be utilized by any of a plurality of the compute nodes - e.g., in a data center. This serves to alleviate resource stranding, as compute nodes are free to request memory when needed, and release such memory when no longer needed.

This is schematically illustrated with respect to <FIG>. As shown, a plurality of compute nodes 100A-100N (where N is any suitable positive integer) are communicatively coupled with a memory pool <NUM>. In various examples, dozens, hundreds, thousands, or more individual compute nodes may share access to one or more disaggregated resource pools, including memory pool <NUM>. As used herein, disaggregated memory may refer both to memory elements that are physically disaggregated and to memory elements that are physically contiguous, but are partitioned by a memory controller.

The disaggregated memory pool is comprised of at least two memory control systems 104A and 104B, which respectively govern and maintain sets of physical memory units 106A and 106B. In this example, physical memory units 106A are considered natively-attached physical memory units while physical memory units 106B are considered to be disaggregated memory units. Memory control systems 104A and 104B may cooperate to provide a single disaggregated memory pool. In other examples, however, a disaggregated memory pool may only include one memory control system, or memory control systems 104A and 104B may operate independently from each other. The memory control systems may, as one example, be serial bus interconnect programmable pattern based memory controllers (PPMCs) (e.g., compute express link (CXL) -compliant pooled memory controllers (CPMCs)). The physical memory units may, for example, be any suitable type of volatile RAM - e.g., Double Data Rate Synchronous Dynamic RAM (DDR SDRAM). The memory control systems may facilitate use of the physical memory units by any or all of the various compute nodes 100A-100N. It will be understood that a memory pool may include any suitable number of physical memory units, corresponding to any suitable total memory capacity, and may be governed by any number of different memory control systems.

<FIG> also schematically depicts a fabric manager <NUM>. The fabric manager may be configured to monitor and govern the entire computing environment, including the plurality of compute nodes and memory pool <NUM>. The fabric manager may, for example, set and apply policies that facilitate efficient and secure use of the memory pool by each of the plurality of compute nodes. Fabric manager <NUM> may, in some examples, coordinate operations of memory control systems 104A and 104B.

Traditionally, servers and compute nodes may each be configured to be substantially self-sufficient, including processing resources, data storage, volatile/nonvolatile memory, network interface componentry, a power supply, a cooling solution, etc. However, it may be advantageous to pool resources across servers, for example by consolidating internal power supplies, cooling systems, and/or network interfaces into a central rack. This may function to reduce hardware redundancy, to provide more resources (e.g., memory) to needy nodes, and to allow for unified governance and balancing of compute node priorities with a centralized system.

Moving such a server cluster to on-silicon, system-on-a-chip(s) packages may involve balancing a number of seemingly disparate trends. On one hand, there is a trend towards on-silicon integration and building larger, monolithic systems. Emerging cooling technologies allow the operation of sizable racks of servers. Large, dense systems present opportunities for amortizing fixed costs such as platforms, racks, and cabling. However, the cost of maintaining coherence across a large core-count system increases non-linearly. On the other hand, there is a desire for right-sizing modular systems to increase utilization. Most virtual machines utilize eight or fewer virtual CPUs. At any given time, up to <NUM>% of the memory in the disaggregated memory pool is stranded, unbilled and/or unutilized. Pooling of accelerators, network interface controllers (NICs), and storage presents a significant cost savings opportunity.

In this disclosure, systems are presented for providing balanced, pooled memory (both natively-attached and disaggregated) and IO interfaces within a firm-partitioned, multi-node, chiplet-based SoC. A multi-server-node system-on-chip implementation on one package may comprise several chiplets or dice, each including several CPU Cores and an associated boot port to independently run an OS and/or hypervisor. The compute nodes may share several external DDR channels and several internal high bandwidth memory (HBM) Channels accessible via a central IO die that also includes a plurality of serial bus interconnect links. Each compute node and internal cache is independently coherent, but is not necessarily coherent with other compute nodes. This allows locally connected memory to be partitioned and disaggregated, even if not physically disaggregated from the chip, and treated differently for each compute node. Such an implementation reduces manufacturing and maintenance costs when compared to discrete implementations using traditional single-node and/or single-OS approaches.

<FIG> schematically shows an example computing device <NUM>. Computing device <NUM> may including one or more system(s)-on-a-chip (SoCs). However, other system configurations, such as multi-chiplet devices, and individual servers that are centrally hardwired may also be included in computing device <NUM>. Computing device <NUM> may be considered a large, dense, disaggregated modular system that features dynamic pooling of all available IO, memory capacity, and bandwidth resources across a plurality of compute nodes <NUM>.

Computing device <NUM> may include two or more compute nodes <NUM>. As shown, computing device <NUM> includes eight compute nodes <NUM>. Each compute node may include two or more processor cores, and each compute node may be an independently coherent domain that is not coherent with other compute nodes <NUM>. Additional features of an example compute node <NUM> are discussed herein and with regard to <FIG>.

Each compute node <NUM> may be configured to run an independent hypervisor usable to generate and operate one or more virtual machines (VMs). Using compute nodes that function as logical partition coherency domains enables modularity that allows for computing device <NUM> to run multiple hypervisors concurrently. This improves fault isolation by preventing cross-node spillover, thus reducing blast radius.

By separating compute nodes <NUM> in this way, computing device <NUM> may be provided with enough degrees of freedom to support two modes of operation at a high level. In some embodiments, the compute nodes <NUM> may indeed be operable in a first mode of operation where compute nodes <NUM> operate as a single coherent domain, similarly to current CPUs.

However, independent coherence also enables separating each compute node <NUM> into its own operating domain with its own operating system that can be booted and shut down independently, and otherwise acts like an independent system from the perspective of any related software stacks. This allows for a second mode of operation, where each compute node <NUM> operates independently.

Each of compute nodes <NUM> may thus be independently bootable, and may be configurable to independently run one of two or more operating systems (e.g. kbm, hyperV). In other words, any individual compute node <NUM> may run an operating system that is not the same operating system run by each of the other compute nodes <NUM>. This enables computing device <NUM> to run multiple operating systems simultaneously, thus allowing for assignments to be directed to the compute node <NUM> running the most relevant and/or suitable operating system for that assignment. The cores and caches within a single compute node <NUM> may be collectively coherent, even if they are not coherent with the cores and caches incudes in other compute nodes <NUM>. However, in some implementations, two or more compute nodes <NUM> may form a coherent group and/or two or more compute nodes <NUM> may concurrently run a same operating system.

Typically, each compute node <NUM> must be provisioned with its own set of resources. The topology, or method of construction described for computing device <NUM>, allows for the pooling of some or all of the platform resources such that they're available to all of the compute nodes <NUM> in computing device <NUM>.

Such poolable platform resources include memory (both bandwidth and capacity), the I/Os (e.g., PCIe devices, accelerators, links, storage), as well as legacy platform components, such as a base band management controller, a real time clock, and a trusted platform module (TPM). Pooling these resources allows computing device <NUM> to forgo duplicating each component for each compute node <NUM>, allowing for efficient utilization of the pooled resources while maintaining constraints on power and cost.

As an example, computing device <NUM> may further include a central IO die <NUM> that is communicatively coupled to each of the two or more compute nodes <NUM>, and that enables the pooling of platform resources among compute nodes <NUM>. Essentially, computing device <NUM> may be configured using multi-chip module architecture with separation of compute and IO chiplets.

Central IO die <NUM> may contain all package IO interfaces for computing device <NUM>, including DDR interfaces <NUM> (solid lines), serial bus interconnects <NUM> (dashed lines), general purpose IO (GPIO) interfaces <NUM>, etc. Each of these IO interfaces may include shared, multi-domain links for all firm partitions (e.g., compute nodes <NUM>) to operate independently while maintaining maximum available bandwidth to any CPU Core (e.g., Boot Ports, serial bus interconnects). Central IO die <NUM> may further include home agents (HA) <NUM> and memory controller agents (MC) <NUM>. In this way, each compute node <NUM> may make use of the provided functions and receive supported access to connected external devices. Additional components and structure of an example central IO die are described herein and with regard to <FIG>.

Central IO die <NUM> thus provides sufficient ports for serial bus interconnects to provision ample connectivity for a variety of external devices and to allow for increased disaggregated memory bandwidth. Central IO die <NUM> may virtually integrate a multi-node aware serial bus switch and the fabric manager may be further integrated onto central IO die <NUM> for pooling of IO devices among compute nodes <NUM>. Central IO die <NUM> may be configured to be one-way coherent with each processor core, while cache and memory serial busses may be configured to be two-way coherent with each core.

In this example, several DDR Channels are connected to central IO die <NUM> to be shared amongst all compute nodes <NUM> and their respective cores, making all memory bandwidth and capacity available to each core of each compute node <NUM> at different times.

One or more natively-attached volatile memory units <NUM> are attached to central IO die <NUM> via a memory controller agent <NUM> and a DDR interface. Central IO die <NUM> includes one or more home agents <NUM> for each compute node <NUM>, the home agents <NUM> configured to map memory access requests received from a compute node <NUM> to one or more addresses within the volatile memory units <NUM>. Each memory controller agent <NUM> may comprise one or more high bandwidth memory channels configured to be shared among the two or more compute nodes. Each memory controller may further operate one or more disaggregated caches, functioning as an optional near-memory cache.

Additionally, one or more disaggregated memory units <NUM> are attached to central IO die <NUM> via a serial bus interconnect <NUM>. Home agents <NUM> may be further configured to map memory access requests received from a compute node <NUM> to one or more addresses within the disaggregated memory units <NUM>. Disaggregated memory units <NUM> may be coupled to central IO die <NUM> via a PPMC controller <NUM> or other suitable memory controller that automates access to one or more disaggregated memory units <NUM>. Each compute node <NUM> thus has access to each of the natively-attached volatile memory units <NUM> and each of the disaggregated memory units <NUM>.

As shown, compute system <NUM> includes <NUM> channels of DDR5 RAM, including <NUM> channels of disaggregated memory units <NUM> and <NUM> channels of natively-attached volatile memory units <NUM>. As one non-limiting example, using a DIMM size of 64GB, compute system <NUM> would include <NUM> total units of natively-attached volatile memory for a total of <NUM> GB, and <NUM> total units of disaggregated memory for a total of <NUM> GB, for a total pooled capacity of <NUM> GB of RAM.

The pooling and distribution of this memory may occur at central IO die <NUM>. Accesses to natively-attached volatile memory units <NUM> may be more rapid than accesses to disaggregated memory units <NUM> that are routed through a PPMC controller <NUM>, and memory requests may be prioritized as such. Further, disaggregated memory units <NUM> may be physically disaggregated and located at different physical locations relative to central IO die <NUM>. While natively attached volatile memory units <NUM> are directly connected to central IO die <NUM>, the memory units themselves may still be considered to be disaggregated as different portions of each memory unit may be specifically assigned to any one of multiple compute nodes. In some examples, one or more natively-attached volatile memory units <NUM> may be pooled for use by several compute nodes <NUM>. Additionally or alternatively, a region of a memory unit may be dedicated for one or more compute nodes <NUM>, and not available to other compute nodes <NUM>. In some examples, additional natively-attached volatile memory units may be provided that are directly linked, and dedicated specifically to one compute node <NUM>.

Both natively-attached volatile memory units <NUM> and disaggregated memory units <NUM> may be allocated or assigned to one or more compute nodes <NUM>. Assignments to the natively-attached volatile memory units and the disaggregated memory units may be based at least on one or more of compute node-specific requirements, application-specific requirements, software-based policies, received compute node requests for additional memory and availability of one or more of natively-attached volatile memory units and disaggregated memory units. Memory reassignments may be performed periodically and/or in response to operating conditions, be they previous conditions, current conditions, or anticipated future conditions.

Further, it is contemplated that various strategies may be employed when an amount of unassigned portions/slices of natively-attached volatile memory units <NUM> and/or disaggregated memory units <NUM> runs low. In such a case, compute system <NUM> has less freedom to satisfy requests from compute nodes <NUM> to receive larger memory assignments, for example when such compute nodes <NUM> commence or prepare for more intensive computing tasks.

Mitigation strategies may include identifying a "memory pressure" situation (i.e., available pool of natively-attached volatile memory units <NUM> and/or disaggregated memory units <NUM> is low) and then activating mechanisms for freeing up memory units, which often includes revoking or unassigning memory that is currently reserved to a compute node <NUM>. Revoking memory may be conducted with reference to priority assessment - e.g., management may be conducted to override least-frequently or least-recently-used strategies, or other assessments targeted to minimizing the harm or impact of memory revocation.

In some cases, revocation can include relocating displaced data, e.g., to another disaggregated memory unit <NUM> portion managed by a different PPMC controller <NUM>, or to a larger, higher latency bulk memory location. As such, portions of one or more of the natively-attached volatile memory units may be unassigned from a first compute node and re-assigned to a second compute node based on one or more of node-specific requirements within the computing device, received requests from the second compute node for more memory, and an availability of one or more of natively-attached volatile memory units and disaggregated memory units. Relief, in some scenarios, may come in the form of a different type of memory. For example, a compute node <NUM> that has maxed out its assignment of natively-attached volatile memory units <NUM> and requests additional memory may receive an allocation of disaggregated memory units <NUM>.

Mitigation may additionally or alternatively include sending warnings to compute nodes (e.g., originating from native or pool memory controllers, or from fabric managers or other infrastructure) to prompt compute nodes to assist in relieving memory pressure. Such assistance from the compute nodes may include the nodes voluntarily relinquishing native and/or disaggregated pool memory that they are holding, or delaying or avoiding requests for more memory that they might have otherwise made in the absence of the pressure warning.

Each disaggregated memory unit <NUM> may include a plurality of slices of memory, (e.g., 1GB slices) that can be assigned, un-assigned, and reassigned to the different compute nodes <NUM>. PPMC controller <NUM> may keep track of each assignment, manages each slice, and routes read/write access requests to the appropriate slice. Operation of PPMC controller <NUM> may be regulated at least in part by a fabric manager residing on central IO die <NUM>.

In some examples, one or more slices of natively-attached volatile memory units <NUM> may be assigned to a specific compute node <NUM> based on node-specific requirements, as prefaced above. The assigned slices of natively-attached volatile memory units <NUM> may require permission from the associated memory controller <NUM> to be used. Memory controller <NUM> may then provide a portion of the allocation to the compute node <NUM>. In some examples, portions of one or more of the natively-attached volatile memory units <NUM> may be unassigned from the first compute node and re-assigned to a second compute node based on activity of one or compute nodes <NUM> within computing device <NUM>. Additionally or alternatively, a portion of the natively-attached volatile memory units assigned to the first compute node may be increased based on a change in node-specific requirements. Node-specific requirements may include, but are not limited to, node provisioning, identity, type, number, bandwidth of programs and/or applications being run, operating system(s) being run, types and number of virtual machines being executed, whether the node is part of a functional group of nodes that is designated to perform specific tasks in tandem, and priority of compute node operations in the context of the entirety of compute nodes.

As described above, natively-attached volatile memory units <NUM> may be unassigned and/or reassigned periodically, or in response to operating conditions. Memory controller <NUM> may receive requests for additional allocations of natively-attached volatile memory units <NUM> and selectively grant expanded memory assignments. For example, if two compute nodes <NUM> share an allotment of natively-attached volatile memory units <NUM>, the granting of a request from a first compute node for an additional allocation of natively-attached volatile memory units <NUM> may depend on requirements specific to a second compute node. For example, one node may be operating with a relatively higher guaranteed quality-of-service agreement, and/or may be executing higher priority applications or tasks.

Once assigned, each compute node <NUM> may determine how to manage their allotment of natively-attached volatile memory units <NUM> and disaggregated memory units <NUM>. A region of memory for a compute node <NUM> may include slices of both natively-attached volatile memory units <NUM> and disaggregated memory units <NUM>, although the relative latency may be different. A compute node <NUM> may prioritize use of natively-attached volatile memory units <NUM> in order to reduce latency for particular tasks, while assigning disaggregated memory units <NUM> for less urgent tasks. In some examples, a compute node <NUM> may interleave memory assignments to generate an average latency across all tasks.

In some examples, the amount of disaggregated memory collectively allocated to the plurality of compute nodes <NUM> may exceed the amount of memory actually provisioned in the disaggregated memory pool. This is sometimes referred to as "thin provisioning. " In general, in data center environments without thin provisioning, it can be observed that individual compute nodes <NUM> (and/or virtual machines implemented on the compute nodes <NUM>) are often provisioned with more resources (e.g., storage space, memory) than the compute nodes <NUM> end up actually using, statistically over time. For instance, the amount of memory installed for a particular compute node <NUM> may be significantly higher than the amount of memory actually utilized by that compute node <NUM> in most situations. When compounded over a plurality of compute nodes <NUM>, the amount of unused memory (or other resources) can represent a significant fraction of the total memory (or other resources) in the data center.

In one example scenario without thin provisioning, a memory pool including <NUM> GB of total memory may be distributed evenly between eight compute nodes <NUM>. As such, each compute node may be assigned 96GB of natively-attached volatile memory units <NUM> as well as 128GB from disaggregated memory units <NUM>, thus each node is allocated a total of 224GB of provisioned memory from the total pooled memory.

However, it is generally unlikely that each compute node <NUM> will fully utilize its memory allocation. Rather, in a more common scenario, each compute node <NUM> may only use a maximum of <NUM>% of its allocated memory during normal usage, and some compute nodes <NUM> may use significantly less than <NUM>%. As such, even though the <NUM> GB disaggregated memory pool will be fully assigned to the plurality of compute nodes <NUM>, only a relatively small fraction of the pooled memory may be in use at any given time, and this represents an inefficient use of the available resources.

Given this, the amount of memory actually available - i.e., "provisioned" - in the total memory pool could be reduced without significantly affecting performance of the plurality of compute nodes <NUM>. While each particular compute node <NUM> may be allocated 96GB of natively-attached volatile memory as well as 128GB of disaggregated memory, it is statistically likely that many compute nodes <NUM> will not use all, or even a significant portion, of either memory allotment at any given time. Thus, any unused natively-attached volatile memory <NUM> assigned to one compute node <NUM> may be reassigned to one or more of the other nodes <NUM> by a memory controller <NUM>, and any unused disaggregated memory <NUM> assigned to one compute node <NUM> may be reassigned to one or more of the other nodes <NUM> by a PPMC <NUM>. In this manner, any particular compute node <NUM> has the option to use up to <NUM> GB of total memory if needed, while still conserving memory in at least the disaggregated memory pool, due to the fact that each compute node <NUM> typically will not use 224GB at any given time.

Such thin provisioning may be done to any suitable extent. It is generally beneficial for the amount of available memory to exceed the amount of memory typically used by the plurality of compute nodes <NUM> under typical circumstances. In other words, if the compute nodes <NUM> typically use around 256GB, then it is generally desirable to have more than 256GB of memory actually provisioned between the natively-attached memory and the disaggregated memory, such that the compute nodes <NUM> do not exhaust the available memory during normal use. In practice, however, any suitable amount of memory may be provisioned in the disaggregated memory pool, which may have any suitable relationship with the amount of memory allocated to the plurality of compute nodes <NUM>.

When thin provisioning is implemented, there may be instances in which the plurality of compute nodes <NUM> attempts to collectively use more memory than is available in the disaggregated memory pool. As described above, this may be referred to as "pressuring" the disaggregated memory pool. Various actions may be taken to address this scenario. For example, memory assignments, be they natively-attached volatile memory units or disaggregate memory units, may be stripped away from one or more compute nodes <NUM> regarded as having a lower priority or lower need for the memory. Additionally, or alternatively, memory requests for the plurality of compute nodes <NUM> may be routed to a different disaggregated memory pool that may still have available memory, at the cost of higher latency. With natively-attached volatile memory units <NUM> assignable by a memory controller <NUM> and/or fabric manager, and disaggregated memory units <NUM> being assignable by a PPMC <NUM>, portions of either natively-attached, volatile memory units or disaggregated memory units may be routed to compute nodes <NUM> based on node-specific requirements.

Requests for additional memory may have an inherent preference for a memory type, and/or may indicate a priority and/or other parameters that indicate how preferential one memory type is over another. For example, a compute node may generally prefer natively-attached volatile memory units <NUM> to disaggregated memory units <NUM> due to latency issues and/or the impact on other systems that may be coupled to disaggregated memory units <NUM> via PPMCs <NUM>. However, this may be balanced with memory pressure on natively-attached volatile memory units <NUM>, constraints on the use of the available memory (e.g., lengthier operations that merely need to have a result retrieved at a later time point may not be prioritized for natively-attached volatile memory units <NUM>). However, when memory pressure needs to be relieved, disaggregated memory units <NUM> may be reassigned first, as the availability of the disaggregated memory pool is more important to the overall compute system <NUM>, as this pool of memory may be used to increase the overall fluidity and stability of compute system <NUM>.

A compute node <NUM> may request natively attached volatile memory units, disaggregated memory units <NUM>, and/or generic memory, depending on node-specific requirements. For example, if a compute node <NUM> experiences an increase in latency, it may request more natively attached memory. If memory controller <NUM> is unable to fulfill such a request, the request may be forwarded to one or more PPMCs <NUM>.

Notably, the memory addressing techniques described herein may be implemented with or without thin provisioning. In other words, memory address mapping as discussed herein may occur in "thin" provisioned or "thick" provisioned contexts. Furthermore, both thick and thin provisioning techniques may be used in the same implementation.

Additionally, or alternatively, each compute node <NUM> may be pre-assigned some amount of memory capacity in the disaggregated memory pool. If and when a particular compute node <NUM> completely fills its assignment and requests a larger assignment, the node <NUM> may negotiate with the memory control system (e.g., PPMC <NUM>) to determine whether and how much additional disaggregated memory the compute node <NUM> should be assigned, and this may include reducing the assignment reserved for another compute node <NUM>. In this manner, the amount of memory capacity available in the disaggregated memory pool may be carefully balanced and divided between the plurality of compute nodes <NUM> in keeping with each compute node's actual needs, rather than allow each individual compute node <NUM> to seize memory capacity they have no need for.

As such, one compute node <NUM> may be provisioned with a particular set of parameters and/or options that would be different from the way another compute node <NUM> would be provisioned. For example, certain compute nodes <NUM> may consistently require more memory capacity and bandwidth than others. Further, each compute node <NUM> may be presented with a custom partition of memory and IO devices. Thus, each compute node <NUM> may be provided with asymmetric access to resources and/or asymmetric mapping to resources, depending on the needs of the compute nodes <NUM> and the computing system <NUM> as a whole. For example, if a first compute node requires local storage but a second compute node does not, the allocated local storage for the second compute node could be assigned to the first compute node. By centralizing resources that would normally be statically provisioned at the central IO die, each compute node may be effectively treated as a different type of hardware and/or firmware partition.

Turning to <FIG>, an example compute node <NUM> is schematically shown. Compute node <NUM> may be an example of compute node <NUM>. Compute node <NUM> is shown connected to central IO die <NUM> via a pair of high-speed serial interconnect links <NUM>.

In this example, compute node <NUM> includes <NUM> cores <NUM> and <NUM> L3 caches <NUM>, for a set of <NUM> slices. However, other quantities of cores and caches are possible. For the compute system shown in <FIG>, the <NUM> nodes would thus provide a total of <NUM> cores. However, as chip technology improves, the number of cores per compute node may increase (e.g., <NUM> cores/node at <NUM>, <NUM> cores/node at <NUM>). As shown, the <NUM> L3 caches <NUM> are interconnected in groups of <NUM> caches via one of <NUM> interconnect hubs <NUM> within compute node <NUM>.

As described, cores <NUM> are firm-partitioned from the cores within other compute nodes. In this way, a reasonable die size may be maintained for each compute node <NUM>, thus maximizing yield while minimizing power consumed. However, the portioning of the cores need not be limited to a single compute node, and the total set of cores may be provisioned and re-provisioned as necessary.

As the number of cores within a coherence domain increases, the complexity increases non-linearly. Thus, by maintaining a reasonably small number of cores per compute node, the system complexity remains modest. Such a computing device is not architected to support one very large VM, rather, it is built for a number of VMs, each having a limited number of cores (e.g., <NUM>). VMs may be opportunistically distributed across nodes.

<FIG> schematically shows how an example cache organization schema for an example compute node <NUM>. Compute node <NUM> may be an example of compute nodes <NUM> and <NUM>. Compute node includes a plurality of cores <NUM> (e.g., <NUM> cores, as shown for compute node <NUM>). Each core <NUM> may include private caches for instructions (I-cache <NUM>), data (D-cache <NUM>), and an associated core-specific L2 cache <NUM>. Additionally, each core <NUM> may include one or more shared, distributed L3 caches <NUM>. As shown in <FIG>, for a <NUM> core compute node, each core may be paired with a shared L3 cache, each of which may be accessed by each core of the compute node. Cache coherency among the L3 caches of a compute node may be managed using any suitable methodology. Each compute node <NUM> further has access to DDR memory <NUM> via an address interleave decode <NUM> of a central IO die. As described with regard to <FIG>, one or more disaggregated caches may be coupled to each memory controller.

Each compute node, as an individual, hardware portioned machine has its own internal understanding of an address map for memory locations within the node. However, with multiple, potentially identically configured compute nodes within a single compute system, the central IO die may distinguish between the host physical addresses for each node using a package physical address.

<FIG> shows an example method <NUM> for memory address mapping across multiple compute nodes. Method <NUM> may be executed by a logic core of a central IO die of a multiple compute node computing system.

At <NUM>, method <NUM> includes, at a central IO die, communicatively connected to two or more independently coherent compute nodes, receiving a memory access request from a first compute node including a host physical address for the first compute node. In general, a host physical address refers to a particular compute node's internal identifier for a particular memory address within the node's larger address space. Reads and writes to a particular HPA may ultimately terminate at a physical memory unit (e.g., RAM DIMM) in the disaggregated memory pool.

At <NUM>, method <NUM> includes mapping the host physical address for the received request to a system address map including ranges of host physical addresses for each of the two or more independently coherent compute nodes. This may include, for example, receiving an indication of one or more ranges of HPAs from each compute node of a plurality of compute nodes communicatively coupled to the central IO die.

As an example, <FIG> schematically depicts an example data structure <NUM> for mapping memory addresses across multiple compute nodes. Data structure <NUM> schematically depicts the relationships between the host physical addresses as seen by each node with the package physical address as seen by the central IO die. A first compute node (e.g., compute node <NUM>) maps host physical addresses to a first range <NUM>. Each additional node maps host physical addresses to an additional range (e.g., compute node <NUM> maps to range <NUM>), such that each host physical address is mapped into the package physical address space. In this example, a system address map includes contiguously stacked address slabs of equal length for each of the two or more nodes, where each slab corresponds to the host physical address range for the respective node. In this example, each compute node includes <NUM> banks of host physical addresses. In some examples, the range of the host physical addresses is interleaved among available home agents and memory channels included in the IO die. In this way, each compute node may be provided with access to full memory bandwidth.

Returning to <FIG>, at <NUM>, method <NUM> includes outputting a package physical address based on the mapped host physical address. For example, <FIG> shows a range of package physical addresses <NUM> that includes each host physical address for each compute node (e.g., compute nodes <NUM>-<NUM>). In some examples, the package physical address includes a node ID appended to the host physical address (e.g., <NUM>-<NUM>). This may be accomplished by commandeering the upper address bits to insert the node ID when a request enters the IO die. In other words, the node ID may act as an effective area code for the host physical addresses within a compute node.

At <NUM>, method <NUM> includes mapping the package physical address to a physical element of a memory unit selectively coupled to the first compute node via the central IO die. Such mapping may include mapping the package physical address to a DIMM, bank, bank group, row, and column of a particular RAM unit. In some cases, mapping of memory addresses to physical memory units may be governed by a fabric manager, to prevent any individual compute node or memory control system from compromising the environment as a whole.

At <NUM>, method <NUM> includes providing the first compute node access to the physical element of the memory unit. For example, access may be provided using the package physical address, as this refers to the total addressable physical memory elements within the entirety of the computing system. In some examples, where the memory unit is positioned outside of the compute system, the associated memory controller may be interconnected to multiple packages. As such, a package identifier may be appended to the package physical address to allow the memory controller to distinguish between requests.

Along with mediating access to disaggregated memory and cache, the central IO die may be used to pool and distribute access to IO devices for each compute node, thus linking the compute nodes to all off-package interfaces. <FIG> shows a computing system <NUM> with a more detailed mapping of a central IO die <NUM>. Computing system <NUM> and central IO die <NUM> may be examples of computing system <NUM> and central IO die <NUM>.

Central IO die <NUM> is coupled to eight compute nodes <NUM> via dedicated compute die ports (CDP) <NUM> and at least one dedicated boot port <NUM> for each of the compute nodes <NUM>, allowing each compute node <NUM> to be booted independently using its own operating system or hypervisor. The CDPs <NUM> are each linked to a home agent (HA) <NUM>. Each home agent <NUM> maintains information about their respective connected compute nodes' internal addresses. As such, home agents <NUM> are responsible for both coherency and management of memory resources for the compute nodes <NUM>. Multiple HAs <NUM> are provided to distribute their workload for bandwidth reasons. These interconnected HAs <NUM> distribute the received accesses from the compute nodes <NUM> so that the appearance of hotspots is reduced. Additional description of home agents <NUM> is presented herein and with regard to <FIG>.

From each HA <NUM>, when access to a specific memory element is requested, the HA <NUM> decodes which of the memory controllers the memory element maps to. In some examples, the memory element may map to one of a plurality of memory controllers <NUM> locally attached to DDR memory (e.g., via DDR interfaces <NUM>, solid lines) and configured to selectively couple each compute node <NUM> to one or more natively-attached volatile memory units. Additionally or alternatively, the memory element may map to one of a plurality of pooled PPMCs coupled to a serial bus interconnect (SBI) <NUM> at a disaggregated location (e.g., via dashed lines <NUM>) to selectively couple each compute node to one or more disaggregated memory units.

A fabric manager <NUM> may be configured to mediate pooling of all IO devices, including memory, among the compute nodes <NUM>. Fabric manager <NUM> may mediate the binding of each processor, core, compute node, etc. to hardware elements and ports of central IO die <NUM>. As the different compute nodes <NUM> may have different configurations and requirements, this binding may be unbalanced. A mesh-based, on-die interconnect <NUM> may be used to couple each of these elements of central IO die <NUM>. I/O ports (IOP) <NUM> may mediate traffic to and from external elements such that IO direct memory access (DMA) traffic remains on the central IO die after translation from the input-output memory management unit IOMMU inside each IOP.

At each CDP <NUM>, traffic in and out of the associated compute node <NUM> may be metered, allowing for the provision of memory bandwidth partitioning as described with regard to <FIG>, as well as the prioritization of traffic to and from external IO devices. This allows for providing different levels of service and allocations of resources to the different compute nodes <NUM>. For memory capacity partitioning, individual compute nodes <NUM>, and even individual cores within each compute node <NUM> may be brought online so that memory is allocated differently among different hosts, as described with regard to <FIG>. In conjunction with CDP <NUM>, fabric manager <NUM> may use machine-learning principles to determine when and if certain compute nodes <NUM> need a greater memory allocation, and which compute nodes <NUM> may release at least part of their current allocation. Fabric manager <NUM> may set policies, such as ceilings and floors that automatically function to determine that is a VM is likely to be utilized, a greater allocated share of resources is provided. Additionally or alternatively, policies may set firm partitioning, such that a compute node is unable to exceed its allocation, even if other VMs are currently idle.

<FIG> shows an example coherence map for a computing system <NUM>. Computing system <NUM> may be an example of computing systems <NUM> and <NUM>, and is shown in simplified form, with a single compute node <NUM> connected to a central IO die <NUM>. Compute node <NUM> is shown coupled to a home agent <NUM> via a compute die port (CDP) <NUM>. Home agent (HA) <NUM> is coupled to IO port <NUM> and memory controller <NUM>, which is turn is coupled to DDR RAM <NUM>. HA <NUM> may be considered to be the center of coherency management in central IO die <NUM>.

When a request is received from compute node <NUM> via CDP <NUM>, CDP <NUM> decodes the address within the request and determines which of the distributed HAs within central IO die <NUM> should receive the request (e.g., by mapping to a package physical address). Once directed to HA <NUM>, the request is passed to target address decoder (TAD) <NUM>. TAD <NUM> is responsible for mapping the request to a target. For example, if a memory access request is received, TAD <NUM> determines whether the request is for natively-attached DDR. If so, the request is sent to memory controller <NUM> and the specific DDR <NUM> it maps to. Alternatively, if the memory access request is for disaggregated memory, the request is sent to the memory port <NUM> within IO port <NUM>, then to external device <NUM>, which hosts disaggregated memory <NUM>.

In addition, HA <NUM> contains a cache currency mechanism <NUM> which may be considered akin to a set of snoop filters. Such snoop filters allow HA <NUM> to ensure that the accesses are cache current and are not violating any currency rules. As such, if an updated copy of a request exists in content addressable memory (CAM) <NUM>, the snoop filters (SF) (e.g., sectored SF <NUM>, SBI $ SF <NUM>, IO $ SF <NUM>), will uncover the updated copy from compute node <NUM>, IO Wr $ <NUM> and external cache <NUM>, respectively, and present it for retrieval.

HA <NUM> is further responsible for handling requests from external devices, such as a PCIe device that is configured to perform reads and/or writes to memory. Such a request may emanate from logic within external device <NUM> and be sent to the SBI root port <NUM>. Root port <NUM> may then translate the IO virtual address that is sent by external device <NUM>, e.g., using IOMMU <NUM>, and convert it into a known physical address. That address may then be decoded and sent to HA <NUM>. IO logic <NUM> may perform this decoding using one or more hashing functions, such as the same hashing function that exists in CDP <NUM>, such that any request for a given address ends up at the same HA, thereby maintaining currency. HA <NUM> may then translate the requested address for the external device. IO port <NUM> may further include an SBI $ port <NUM>, allowing for parallel lookups to external cache <NUM> and to disaggregated memory <NUM>.

<FIG> schematically shows a traditional system <NUM> for accessing IO devices across multiple compute nodes. In system <NUM>, each compute node (e. g, nodes <NUM> and <NUM>, though more may be included) operates a plurality of virtual machines (e.g., VMs <NUM>, <NUM>, <NUM>, <NUM>, <NUM>, <NUM>). Most of the virtual machines operate in conjunction with a virtual function (e.g., VF <NUM>, <NUM>, <NUM>, <NUM>) hosted by a network interface controller (NIC) (e.g., NICs <NUM> and <NUM>), via a virtual function driver (e.g., VF drivers <NUM>, <NUM>, <NUM>, <NUM>). Each compute node includes one virtual machine (e.g., VMs <NUM>, <NUM>) that operates in conjunction with a virtual NIC (e.g., Virtual NICs <NUM> and <NUM>) using a physical function (e.g., physical functions <NUM>, <NUM>) hosted by the NIC via a PF driver (e.g., PF drivers <NUM>, <NUM>) of a virtual machine manager (e.g., VMM <NUM>, <NUM>) for the respective node. Physical functions <NUM> and <NUM> may perform the function of a network adapter that supports the single root I/O virtualization (SR-IOV) interface, which may be leveraged to achieve the pairing between the node and NIC via IOMMU <NUM>.

Each NIC represents a device that includes interfaces that are highly partitionable to software, and may include a plurality virtual functions (<NUM>, <NUM>, <NUM>, <NUM>) which can be directly assigned to software entities (e.g., virtual machines <NUM>, <NUM>, <NUM>, and <NUM>) for direct management. However, in this configuration, direct management is limited to generating n virtual machines inside of one physical machines, with each of the n virtual machines deriving virtual functions from a dedicated NIC.

<FIG> schematically shows a system <NUM> for pooling IO devices across multiple compute nodes according to the present disclosure. In system <NUM>, each compute node (e. g, nodes <NUM> and <NUM>, though more may be included) operates a plurality of virtual machines (e.g., VMs <NUM>, <NUM>, <NUM>, <NUM>, <NUM>, <NUM>). Most of the virtual machines operate in conjunction with a virtual function (e.g., VF <NUM>, <NUM>, <NUM>, <NUM>, <NUM>, <NUM>) hosted by a single NIC <NUM>, via a virtual function driver (e.g., VF drivers <NUM>, <NUM>, <NUM>, <NUM>, <NUM>). Each compute node includes one virtual machine (e.g., VMs <NUM>, <NUM>) that operates in conjunction with a virtual NIC (e.g., Virtual NICs <NUM> and <NUM>) using a physical function (e.g., physical functions <NUM>) hosted by NIC <NUM> via a PF driver (e.g., PF drivers <NUM>, <NUM>) of a virtual machine manager (e.g., VMMs <NUM>, <NUM>) for the respective node.

In contrast to system <NUM>, instead of providing NICs for each compute node, there is a single NIC <NUM> that is being pooled across nodes <NUM> and <NUM> via IOMMU <NUM> and fabric manager <NUM>. SR-IOV principles may be leveraged for pooling, while a virtual hierarchy scheme may be defined by the serial bus interface specifications.

To achieve this, fabric manager <NUM> may be configured to own the physical function of each device, while at the same time binding virtual functions to the individual nodes. All downstream configuration and IO requests may be trapped at fabric manager <NUM>, until a response can be emulated. Further, fabric manager <NUM> may program the host IO bridge for appending node IDs (as described with regard to <FIG> and <FIG>) for upstream untranslated requests, as well as for address translation services responses.

Storage machine <NUM> may include optical memory (e.g., CD, DVD, HD-DVD, Blu-Ray Disc), semiconductor memory (e.g., RAM, EPROM, EEPROM), and/or magnetic memory (e.g., hard-disk drive, floppy-disk drive, tape drive, MRAM), among others.

However, aspects of the instructions described herein alternatively may be propagated by a communication medium (e.g., an electromagnetic signal, an optical signal) that is not held by a physical device for a finite duration.

In one example, a computing device comprises two or more compute nodes, each compute node including two or more processor cores, each compute node comprising an independently coherent domain that is not coherent with other compute nodes; a central IO die communicatively coupled to each of the two or more compute nodes; and a plurality of natively-attached volatile memory units attached to the central IO die via one or more memory controllers, wherein the central IO die includes one or more home agents for each compute node, the home agents configured to map memory access requests received from the compute nodes to one or more addresses within the natively-attached volatile memory units. In such an example, or any other example, the computing device additionally or alternatively comprises one or more disaggregated memory units attached to the central IO die via a serial bus interconnect, wherein the home agents are further configured to map memory access requests received from a compute node to one or more addresses within the disaggregated memory units. In any of the preceding examples, or any other example, each compute node additionally or alternatively has access to each of the natively-attached volatile memory units and each of the disaggregated memory units. In any of the preceding examples, or any other example, portions of one or more of the natively-attached volatile memory units are additionally or alternatively assigned to a first compute node based on one or more of received requests from the first compute node for more memory and an availability of one or more of natively-attached volatile memory units and disaggregated memory units. In any of the preceding examples, or any other example, portions of one or more of the natively-attached volatile memory units are additionally or alternatively assigned to a first compute node based on node-specific requirements. In any of the preceding examples, or any other example, portions of one or more of the natively-attached volatile memory units are additionally or alternatively unassigned from the first compute node and re-assigned to a second compute node based on node-specific requirements within the computing device. In any of the preceding examples, or any other example, portions of one or more of the natively-attached volatile memory units are additionally or alternatively unassigned from the first compute node and re-assigned to a second compute node based on a received request from the second compute node for more memory. In any of the preceding examples, or any other example, portions of one or more of the natively-attached volatile memory units are additionally or alternatively unassigned from the first compute node and re-assigned to a second compute node based on an availability of one or more of natively-attached volatile memory units and disaggregated memory units. In any of the preceding examples, or any other example, one or more disaggregated caches are additionally or alternatively coupled to each memory controller. In any of the preceding examples, or any other example, each memory controller additionally or alternatively comprises one or more high bandwidth memory channels configured to be shared among the two or more compute nodes.

In another example, a computing device including a system-on-a-chip comprises two or more compute nodes, each compute node including two or more processor cores, each node comprising an independently coherent domain that is not coherent with other compute nodes; and a central IO die communicatively coupled to each of the two or more compute nodes via dedicated compute die ports, the central IO die including: one or more native memory interfaces attached to one or more memory controllers selectively coupling each compute node to one or more natively-attached volatile memory units; one or more interconnects to selectively couple each compute node to one or more disaggregated memory units; one or more connectivity links to selectively couple the two or more compute nodes to one or more external devices; and a fabric manager configured to mediate pooling of all IO devices among the two or more compute nodes. In such an example, or any other example, the central IO die additionally or alternatively includes all package IO interfaces for the two or more compute nodes. In any of the preceding examples, or any other example, the central IO die additionally or alternatively includes a multi-node aware serial bus interconnect switch configured to pool IO devices connected to the central IO die across the two or more compute nodes. In any of the preceding examples, or any other example, the pooling of IO devices is additionally or alternatively managed using a pooled single root I/O virtualization (SR-IOV) interface. In any of the preceding examples, or any other example, the pooled SR-IOV interface additionally or alternatively includes a single network interface controller (NIC) coupled to each of the compute nodes via the fabric manager and an input-output memory management unit (IOMMU). In any of the preceding examples, or any other example, the fabric manager is additionally or alternatively configured to own a physical function of the NIC and to bind virtual functions hosted by the NIC to individual compute nodes.

In yet another example, a method for memory address mapping comprises at a central IO die communicatively connected to two or more independently coherent compute nodes, receiving a memory access request from a first compute node including a host physical address for the first compute node; mapping the host physical address for the received request to a system address map including ranges of host physical addresses for each of the two or more independently coherent compute nodes; outputting a package physical address based on the mapped host physical address; mapping the package physical address to a physical element of a memory unit selectively coupled to the first compute node via the central IO die; and providing the first compute node access to the physical element of the memory unit. In such an example, or any other example, the system address map additionally or alternatively includes contiguous address slabs of equal length for each of the two or more nodes. In any of the preceding examples, or any other example, the package physical address additionally or alternatively includes a node ID appended to the host physical address. In any of the preceding examples, or any other example, a range of the host physical addresses is additionally or alternatively interleaved among available home agents and memory channels included in the central IO die.

Claim 1:
A computing device (<NUM>), comprising:
two or more compute nodes (<NUM>), each compute node including two or more processor cores, each compute node comprising an independently coherent domain that is not coherent with other compute nodes;
a central IO die (<NUM>) communicatively coupled to each of the two or more compute nodes; and
characterised in that the computing device (<NUM>) further comprises:
a plurality of natively-attached volatile memory units (<NUM>) attached to the central IO die via one or more memory controllers (<NUM>), wherein the central IO die includes one or more home agents (<NUM>) for each compute node, the home agents configured to map memory access requests received from the compute nodes to one or more addresses within the natively-attached volatile memory units.