Heatsink for a Memory and Routing Module

A heatsink is provided for a memory and routing module with a lower and upper side, both sides having multiple semiconductor chips attached. The lower side of the module has a connection component attached for connection to a motherboard. The heatsink includes a module receiving region configured to receive a lower side of the module, including a first thermally conductive portion arranged to face the semiconductor chips, an aperture through the lower heatsink component and a thermally conductive peripheral region disposed around the module receiving region. The heatsink includes an upper heatsink component which is configured to connect to the lower heatsink component at the peripheral region to retain the module. The upper heatsink component includes a lower side. The lower side includes a second thermally conductive portion arranged to face the semiconductor chips disposed on an upper side of the module and multiple second heat dissipating elements.

CROSS-REFERENCE TO RELATED APPLICATIONS

The present application claims priority to United Kingdom Patent Application No. GB2202826.0 filed Mar. 1, 2022, the disclosure of which is hereby incorporated herein by reference in its entirety.

FIELD

The present disclosure relates to a heatsink. The present disclosure also relates to an assembly comprising the heatsink and a memory and routing module.

BACKGROUND

The demand for high performance computing is ever increasing. In particular, efforts are being made to meet the demands of artificial intelligence/machine learning models which impose strenuous requirements on computing resource. It is known to address certain requirements by interconnecting a plurality of processing chips in a cluster, the processing chips being configured to operate in a co-operative manner to meet the demands of processing power required to process large AI/ML models.

Another demand which is imposed on high performance compute is the ability to have access to high-capacity memory. Attempts have been made to connect external memory to processing nodes in a cluster to increase the capacity of the memory. Such external memory may be connected by external links which provide an access path between the external memory and the processing node. For example, dynamic random access memories (DRAMs) may be mounted on dual in-line memory modules (DIMMs) on server racks. These can provide the scalable memory capacity of the order of terabytes. The provision of large amounts of external memory also presents challenges in effectively cooling the memory.

It is an aim of the disclosure to address the difficulties discussed above, and any other difficulties that would be apparent to the skilled reader from the description herein.

SUMMARY

The present inventor seeks to address the above problems by providing a heatsink for a module having high-capacity memory that is readily accessible to a cluster of processing chips and allows the processing chips to communicate with one another.

According to an aspect of the disclosure, there is provided a heatsink for a memory and routing module, the memory and routing module having a lower side and an upper side, each side having a plurality of semiconductor chips attached thereto, the lower side further having a connection component attached thereto, the memory and routing module for electrical connection to a motherboard, the heatsink comprising:a lower heatsink component comprising:a module receiving region configured to receive a lower side of the memory and routing module and comprising:a first thermally conductive portion arranged to face the plurality of semiconductor chips disposed on the lower side of the memory and routing module;an aperture through the lower heatsink component configured to permit the connection component disposed on the lower side of the memory and routing module to connect to a corresponding connection component disposed on the motherboard;a thermally conductive peripheral region disposed around the module receiving region, the peripheral region connected to the first thermally conductive portion and comprising a plurality of first heat dissipating elements configured to dissipate heat conducted from the plurality of semiconductor chips disposed on the lower side of the memory and routing module;an upper heatsink component, the upper heatsink component configured to connect to the lower heatsink component at the peripheral region so as to retain the memory and routing module between the upper heatsink component and lower heatsink component, the upper heatsink component comprising:a lower side comprising a second thermally conductive portion arranged to face the plurality of semiconductor chips disposed on an upper side of the memory and routing module;a plurality of second heat dissipating elements connected to the second thermally conductive portion and configured to dissipate heat conducted from the one or semiconductor chips disposed on the lower side of the memory and routing module.

The lower heatsink component may comprise two apertures therethrough to permit two connection components to connect to corresponding connection components on the motherboard. The first thermally conductive portion may be arranged between the two apertures to face the plurality of semiconductor chips disposed on the lower side of the memory and routing module between the two connection components.

The first thermally conductive portion may comprise thermal interface material to provide conformant thermal conduction across the plurality of semiconductor chips disposed on the lower side of the memory and routing module.

The second thermally conductive portion may comprise thermal interface material to provide conformant thermal conduction across the plurality of semiconductor chips disposed on the upper side of the memory and routing module

The module receiving region may comprise a recess corresponding in size to a substrate of the memory and routing module.

The first and/or second heat dissipating elements may comprise fins. The fins may be rectangular, circular, square or any suitable shape.

The upper heatsink component and lower heatsink component may be configured to the memory and routing module. The upper heatsink component and lower heatsink component may be configured to clamp a substrate of the memory and routing module, suitably around a peripheral edge thereof.

The heatsink may comprise a plurality of lugs extending from the upper heatsink component.

The heatsink may comprise a plurality of bores formed in the peripheral region of the lower heatsink component. The heatsink may comprise a plurality of connecting fasteners configured to extend through the lugs and into the bores to connect the upper heatsink component to the lower heatsink component. The lugs may be configured to deform towards the lower heatsink during connection.

The lower and/or upper heatsink component may be formed of or comprise aluminum. The lower and/or heatsink may be formed of or comprise a thermally conductive material such as copper or graphene.

The heatsink may be a passive heatsink. The passive heatsink may comprise a plurality of heat dissipating fins.

The heatsink may comprise a fastener receivable in a fastener receiving section of the motherboard. The fastener may be configured for a first phase of attachment in which the fastener is extended into the fastener receiving section without drawing the heatsink towards the motherboard. The fastener may be further configured for a second phase of attachment in which the fastener draws the heatsink towards the motherboard.

The fastener may comprise an unthreaded body portion retained in a channel formed in the heatsink. The unthreaded body portion may be longer than the channel. The fastener may comprise a threaded portion configured to be received in a threaded bore of the fastener receiving section. The unthreaded body portion may be configured to travel through the channel in the first phase of attachment.

The heatsink may comprise a first fastener disposed at one side of the heatsink and a second fastener disposed at an opposing side of the heatsink. The first and/or second fasteners may be disposed in a middle of their respective sides of the heatsink. Each respective side may be substantially parallel with a long edge of a respective connection component.

According to another aspect of the disclosure, there is provided an assembly comprising the heatsink defined herein and a memory and routing module retained in the heatsink.

According to another aspect of the disclosure, there is provided a system comprising the assembly defined herein and a motherboard to which the assembly is attachable via the corresponding connection component. The system may further comprise a processor disposed on the motherboard.

According to another aspect of the disclosure, there is provided a method of assembling a memory and routing module and a heatsink, comprising:placing the module in a module receiving region of a lower heatsink component, so that a thermally conductive portion of the lower heat sink component faces a plurality of semiconductor chips disposed on a lower side of the memory and routing module; andconnecting an upper heatsink component to the lower heatsink component to retain the memory and routing module between the upper heatsink component and lower heatsink component.

Further optional features of the assembly, system and method are as defined above in relation to the heatsink and may be combined in any combination.

According to another aspect of the disclosure, there is provided a mounting component comprising:a module receiving region configured to receive a memory and routing module,an aperture through the mounting component to permit a first connection component disposed on the memory and routing module to connect to a second connection component disposed on a motherboard; anda fastener receivable in a fastener receiving section of the motherboard;wherein the fastener is configured for:a first phase of attachment in which the fastener is extended into the fastener receiving section without drawing the mounting component towards the motherboard; anda second phase of attachment in which the fastener draws the mounting component towards the motherboard.

The fastener may comprise an unthreaded body portion retained in a channel formed in the mounting component. The unthreaded body portion may be longer than the channel. The fastener may comprise a threaded portion configured to be received in a threaded bore of the fastener receiving section. The fastener may be configured to descend through the channel in the first phase of attachment.

The mounting component may comprise a first fastener disposed at one side of the heatsink and a second fastener disposed at an opposing side of the heatsink.

Further optional features of the mounting component are as defined herein in relation to the heatsink and may be combined in any combination.

According to another aspect of the disclosure, there is provided an example method of mounting a memory and routing module to a motherboard, comprising: placing the module in a module receiving region of a mounting component; extending a fastener of the mounting component into a fastener receiving section of the motherboard without drawing the mounting component toward the motherboard; and extending the fastener into the fastener receiving section to draw the mounting component towards the motherboard.

In the drawings, corresponding reference characters indicate corresponding components. The skilled person will appreciate that elements in the figures are illustrated for simplicity and clarity and have not necessarily been drawn to scale. For example, the dimensions of some of the elements in the figures may be exaggerated relative to other elements to help to improve understanding of various example embodiments. Also, common but well-understood elements that are useful or necessary in a commercially feasible embodiment are often not depicted in order to facilitate a less obstructed view of these various example embodiments.

DETAILED DESCRIPTION OF EXAMPLES

In certain examples of the present disclosure, a computer comprises a plurality of processor chips and fabric chips, arranged in clusters. Within a cluster, each processor chip is connected to all of the fabric chips, and each fabric chip is connected to all of the processor chips in an all-to-all bipartite connected configuration. There are no direct connections between the fabric chips themselves in a cluster. Further, there are no direct connections between the processor chips themselves. Each fabric chip has routing logic which is configured to route incoming packets from one processor chip to another processor chip which is connected to the fabric chip. Furthermore, each fabric chip has means for attaching to external memory. The routing logic is capable of routing packets between a processor connected to the fabric chip and memory which is attached to the fabric chip. The fabric chip itself comprises a memory controller which performs memory control functions for governing memory accesses from and to memory attached to the fabric chip.

In certain embodiments, further described herein, clusters of processing chips and fabric chips may themselves be interconnected to form a larger computer system. Each processor chip within a cluster may access any of the memory attached to any of the fabric chips within the cluster. This significantly enhances the memory capacity which is rendered available to any particular processor chip.

The connection configuration described herein has the further merit that in certain embodiments, it is not necessary to use all of the edges of a processor die for surfacing external connections.

In accordance with the presently described example of the disclosure, multiple processors (also referred to herein as processor chips) are connected in a cluster using one or more “fabric chips”. Each fabric chip provides access to external memory (e.g. DRAM) and also provides routing of inter-processor traffic. Reference is made toFIG.1.FIG.1illustrates four processor chips20a,20b,20c,20d. Each processor chip comprises a processor core area22a,22b,22c,22dwhich extends to each longitudinal edge of the chip. Each processor chip has an upper beachfront area30aand a lower beachfront area30b(shown for chip20aonly). The upper beachfront area30ahas a set of external port connections C1, C2, C3, C4(labelled only on processor chip20a). It will be evident that each processor chip also has four external port connections on the upper beach front area. The lower beachfront area of each processor chip similarly has four external port connections labelled C5, C6, C7, C8. Note that the lower set of external port connections is a labelled only on the processor chip20a. It is evident that the other processor chips similarly each have a set of external port connections on their lower beachfront areas.

The cluster ofFIG.1further comprises eight “fabric chips”, which may also be referred to herein as “memory and routing chips” or “routing chips”. Each fabric chip comprises a fabric core40a,40b. . .40h. Each fabric chip has a lower beachfront area44a. . .44hwhich has a set of external ports. These external ports are provided in port connections labelled on fabric chip40aonly as FC1, FC2, FC3, FC4. It is apparent that each fabric chip has a corresponding set of external ports on each lower beachfront area. The upper beachfront area of each fabric chip is provided with one or more memory attachment interface which enables the fabric chip to connect to one or more memory, illustrated inFIG.1as respective DRAMs10a,10b,10c,10d. . . to10p. For example, the fabric core40ashown inFIG.1is connected to two DRAMS10a,10by suitable memory attachment interfaces provided on the upper beachfront46aof the fabric chip. Other high capacity memories may be connected, for example Double Data Rate DRAMs (DDRs) and later manifestations thereof such as Low Power DDRs (LPDDRs). The high bandwidth connectivity between processor chips and fabric chips within a cluster is “all-to-all bipartite”. This means that each processor chip is connected to every fabric chip, and each fabric chip is connected to every processor chip. Connections are via links such as L1between a processor port in a port connection, such as C1, and a fabric chip port in a port connection, such as FC1. Note, however, that in the example shown there are no direct high bandwidth connections between processor chips, or between fabric chips within the cluster. Moreover, in the example shown, there is no externally attached memory directly connected to each processor (although there may be High Bandwidth Memory within a chip package). Each fabric chip provides a routing function which provides pathways between every pair of processors, and between each processor and the memory attached to the fabric chip.

Furthermore, the links could be manifest in any suitable way. Each link can be connected or reconnected to different ports to set up a computer configuration. Once a computer configuration has been set up and is in operation, the links are not multiplexable and do not fan in or fan out. That is, there are no intermediate switches instead a port on a processor is directly connected to an end port on the fabric chip. Any packet transmitted over a link will be received at the port at the other end of the fixed link. It is advantageous that the links are bi-directional and preferable that they can operate in both directions at once, although this is not an essential requirement. One particular category of communication link is a SERDES link which has a power requirement which is independent of the amount of data that is carried over the link, or the time spent carrying that data. SERDES is an acronym for Serializer/DeSerializer and such links are known. For example, a twisted pair of wires may be used to implement a SERDES link. In order to transmit a signal on a wire of such links, power is required to be applied to the wire to change the voltage in order to generate the signal. A SERDES link has the characteristic that there is a fixed power for a bandwidth capacity on a SERDES link whether it is used or not. This is due to the need to provide clocking information on the link by constantly switching the current or voltage state of the wire(s) even when no data is being transmitted. As is known, data is transmitted by holding the state of the wire(s) to indicate a logic ‘0’ or logic ‘1’. A SERDES link is implemented at each end by circuitry which connects a link layer device to a physical link such as copper wires. This circuitry is sometimes referred to as PHY (physical layer). In the present example, packets are transmitted over the links using Layer 1 and Layer 2 of an Ethernet protocol. However, it will be appreciated that any data transmission protocols could be used.

There are several advantages to the computer described herein.

It is no longer necessary to dedicate a fixed proportion of processor beachfront (and therefore IO bandwidth) to fixed capacity memory or to inter-processor connectivity. All processor IO bandwidth passes via the fabric chips, where it can be used on-demand for either purpose (memory or inter-processor).

Under some popular models of multiprocessor computation, such as bulk synchronous parallel (BSP), the usage of peak DRAM bandwidth and peak inter-processor bandwidth might not be simultaneous. The total bandwidth requirement may therefore be satisfied with less processor beachfront, providing the processor chips with more core area. BSP in itself is known in the art. According to BSP, each processing node performs a compute phase and an exchange phase (sometimes called communication or message passing phase) in an alternating cycle. The compute phase and exchange phase are performed by the processing chips executing instructions. During the compute phase, each processing unit performs one or more computation tasks locally, but does not communicate any results of these computations to the other processing chips in the cluster. In the exchange phase, each processing chip is allowed to exchange one or more results of the processing from the preceding compute phase to and/from one or more others of the processing chips in the cluster. Note that different processing chips may be assigned to different groups for synchronisation purposes. According to the BSP principle, a barrier synchronisation is placed at the juncture transitioning from the compute phase into the exchange phase, or the juncture transitioning from the exchange phase into the compute phase, or both. That is, to say either all processing chips are required to complete their respective compute phase before any in the group is allowed to proceed to the next exchange phase, or all processing chips in the group are required to complete their respective exchange phase before any processing chip in the group is allowed to proceed to the next compute phase, or both of these conditions are enforced. This sequence of exchange and compute phase is repeated over multiple cycles. In BSP terminology, each repetition cycle of exchange phase and compute phase may be referred to as a “superstep”.

This has the practical effect that there are circumstances when there is no simultaneous usage of all links required for accessing memory (for the purpose of completing a compute phase) and links used to exchange data between the processing chips in an exchange phase. As a consequence, there is maximum efficient use of the fixed links, without compromising memory access times or inter-processor exchange delays. It will nevertheless be appreciated that embodiments described herein have applications other than when used with BSP or other similar synchronisation protocols.

It is possible that the links could be dynamically deactivated to consume effectively no power while not in use. However, the activation time and non-deterministic nature of machine learning applications generally render dynamic activation during program execution as problematic. As a consequence, the present inventor has determined that it may be better to make use of the fact that the link power consumption is essentially constant for any particular configuration, and that therefore the best optimisation is to maximise the use of the physical links by maintaining concurrent inter processor and processor-memory activity as far as is possible.

All of the memory in the cluster is accessible to each processor without indirection via another processor. This shared memory arrangement can benefit software efficiency.

In the example shown inFIG.1, there are two “ranks” of fabric chips, each attached to a respective upper and lower edge of the processor chip. The upper rank comprises fabric cores40a. . .40d, connected by respective links to each of the processor cores. For example, the processor core22ais connected to fabric core40aby link L1, fabric core40bby link L2, fabric core40cby link L3and fabric core40dby link L4. The lower rank comprises fabric cores40e. . .40h. The fabric core40ais also connected to each of the processor cores22a. . .22d, by corresponding links (which are shown but not labelled in the Figure for reasons of clarity). There is no use of the longitudinal processing chip edge for beachfront.

However, there are different design choices within the overall concept. For example, the long edges of the processors could be used to provide more bandwidth to the fabric chips, and all the links emerging from the beachfront of the processor chips could be passed to a single rank of fabric chips, or to three ranks etc.

The number of fabric chips in each rank may differ from the number of processor chips. What remains important to achieve the advantages of the disclosure is that the all-to-all bipartite connectivity between the processing chips and the fabric chips is maintained, with the routing functionality and external memory access provided by the fabric chips.

Note that the use of the external connectors to provide the all-to-all bipartite connectivity in the cluster according to examples of the present disclosure does not rule out the presence of other I/O ports on the processor chips or the fabric chips. For example, certain ones of the processor chips or fabric chips in the cluster may be provided with an I/O port enabling connectivity between multiple clusters or to host devices etc. In one embodiment, the fabric chips provide this additional connectivity.

Furthermore, note that additional memory may be attached directly to the processor chips, for example along the longitudinal edges. That is, additional High Bandwidth Memory (HBM) may be provided in close proximity to the processing chip which is implemented on a silicon substrate within a package which forms a processing node. In practice, the HBM is butted up against a processing chip on a silicon substrate to be as physically as close as possible to the processing chip which provides the processing function. For example, high bandwidth memory (HBM) could be attached to the processor chips, while high-capacity memory could be attached to the fabric chips—thus, combining the advantages of both memory types in the cluster

In the examples of the computers described herein, the processor chips20are not intended to be deployed on a standalone basis. Instead, their deployment is within a computer cluster in which the processor chips are supported by one or more fabric chip40. The processor chips20connect to one another through the fabric chips40, enabling use of all of the processor chip links L1, L2etc. for use simultaneously as processor-to-processor links and memory access links. In this way, the computer offers a higher capacity fast memory system when compared against existing computer systems. In current computer systems, it will become increasingly expensive to provide high capacity, high bandwidth memory. Furthermore, there remain limits on the processing power which can be obtained while delivering high bandwidth memory access and high-capacity memory. The present computer may enable those limits to be exceeded.

By providing routing logic on the fabric chip, it is not necessary for the processor chip to have routing logic for the purposes of external routing functions. This allows silicon area to be freed up to maximise the per processor chip I/O bandwidth and also to maximise area available for processing circuitry within the processor core.

By locating link ports along the north and south edges, this releases the east/west edges. This either allows the processor core to extend into the east/west edges, thereby maximizing the processing capability, or allows the east/west edges to be kept free for high bandwidth memory integration.

The computer may be operated in different topologies. In one example, a group of four processor chips and eight fabric chips (as illustrated for example inFIG.1) may constitute a cluster. Within a cluster, each group of four fabric chips connected to one of the processor chip edges is referred to herein as a rank. The cluster ofFIG.1contains two ranks.

A pod may comprise multiple clusters. Clusters may be interconnected within a pod using a processor facing link on the fabric chip. Pods may be interconnected to each other using a pod facing link on the fabric chip.

FIG.2is a schematic block diagram of components on a fabric chip40. As shown inFIG.2, routing logic46is connected to the DDR interface blocks48for transferring data packets between DDR interface block48and the other ports. The routing logic46is further attached to each processor connected link port. Each port comprises an ethernet port controller EPC. The routing logic is attached to ethernet port controllers of the pod facing ports, and to an ethernet port controller of a system facing link. The routing logic46is further attached to a PCI complex for interfacing with a host system. PCIe (Peripheral Component Interconnect Express) is an interface standard for connecting high speed computers.

FIG.2illustrates an example of a fabric chip which, in addition to enabling inter-processor communication, and processor-to-memory communication, enables a computer to be constructed by connecting together computer clusters in a hierarchical fashion. Firstly, the components of the fabric chip which are used for implementing the inter-processor communication, and the processor-to-memory communication will be described. Each fabric core port connection comprises three serial links. Each serial link comprises a port with an ethernet port controller (EPC). As mentioned, these links may be SERDES links, for example, twisted wire pairs enabling serial packet communication.

For reasons of clarity, not all of the components inFIG.2are illustrated with associated references. Each of the fabric core connections FC1, FC2, FC3and FC4have a configuration as now herein described with reference to fabric core port connection FC2which connects to the second processor (for example processor20binFIG.6). The fabric connection FC2comprises three links L2a, L2b, L2c, each comprising an ethernet port controller EPC2a, EPC2b, EPC2crespectively. Note that in other embodiments, a single physical link may be provided, or a different number of physical links may be provided in each fabric chip connection FC. Note that the link labelled L2in previous Figures may therefore comprise three individual serial links (such as L2a, L2aand L2c). Routing logic46in the fabric chip40may be implemented as a ring router, cross bar router or in any other way. The fabric chip is further connected to external memories (such as DRAMS10A,10B etc.) (not shown inFIG.2). Although two DRAMS are shown in the previous Figures, in the embodiment ofFIG.2, the fabric chip is connected to four DRAMS. In order to make this connection, the fabric chip comprises four DRAM interface blocks DIB1, DIB2, DIB3and DIB4, each associated with four DDR sub connection layers DDR sub1, DDR sub2, DDR sub3and DDR sub4. Each DDR interface block DIB48incorporates a memory controller which manages access to the memory which is attached to the block. One memory attachment interface44is shown inFIG.2, but it will be appreciated that each DDR sub layer has a respective memory attachment interface for attaching to external DRAM. The routing logic46is configured to route memory access packets received from an attached processor core to the addressed one of the data interface blocks DIB1to DIB4. The routing logic46is further configured to route packets from one attached processor chip to another attached processor chip via the respective fabric chip ports. In certain embodiments, the routing logic prevents a memory packet (such as a memory access response packet) from being routed from one memory attachment interface to another memory attachment interface. A memory response packet in such embodiments may only be routed to a processor chip via the correct port attached to the routing logic46. For example, incoming packets on link L2aof fabric core port connection FC2will be routed, based on routing information in the packet, to the addressed port connected to the routing logic46. For example, if the packet is intended to be routed to the processor20c, the routing logic46will identify the processor20cfrom the routing information in the packet and cause the packet to exit through the ethernet port controller onto the link attached to the processor20c.

Should the packet be a memory access packet, the routing logic routes the packet based on the memory address in the packet to its appropriate DDR interface block. Note that in this embodiment each DDR interface block DIB1. . . DIB4comprises four memory access channels. It will be appreciated that any number of memory access channels may be provided by each interface block DIB1. . . DIB4. The memory access channels are managed by the memory controller in each data interface block DIB1. . . DIB4.

As explained above, in the example shown inFIG.2, the fabric chip40has additional components which allow a computer to be made up of interconnected clusters. To this end, the fabric chip comprises a pod facing port connection PL. The pod facing port connection PL comprises three ports, each port comprising an ethernet port controller Pa, Pb, Pc connected to a respective link. The routing logic detects packets whose packet information indicates that the packets should not be routed to a processor within this cluster, but should instead be routed to a processor of another cluster, and routes the packet to one of the pod facing ports. Note that the pod facing port connection PL may transmit packets to a corresponding pod facing port in a fabric chip on another cluster, or may receive packets from a corresponding pod facing port on a fabric chip of another cluster.

The fabric chip ofFIG.2also permits a packet to be routed to another pod within a system. To this end, a system port SL is provided. The system port comprises a corresponding ethernet port controller EPC and is connected to a system's serial link which is connected to a corresponding port in another pod. The routing logic may determine that a packet is intended for routing to another pod in the system and transmit the packet to the system port SL. Packets may be received over the system port SL from a corresponding system port of another fabric chip in another pod in the system which is connected via a system serial link, and be applied to the routing logic.

It will be appreciated that any type of routing logic could be utilised to route traffic from one external connection of the fabric chip to another connection of the fabric chip, either to another processor chip via an external port or to attached memory via a memory attachment interface. The term data packet when used herein denotes a sequence of bits comprising a payload to be transmitted either between processor chips or between a processor chip and memory attached to a fabric chip. The packets include information, such as destination identifiers and/or memory addresses for routing purposes. In some embodiments, a destination processor identifier may be included in a packet header. One type of ring routing logic is described in Graphcore's GB patent application no. GB2115929.8.

As described herein, each processing chip is capable of implementing a processing or compute function. There are many possible different manifestations of a suitable processing chip. Graphcore have developed a intelligence processing unit (IPU) which is describe for example in U.S. patent application Ser. Nos. 15/886,009; 15/886,053; 15/886,131 [PWF Refs. 408525US, 408526US and 408527US] the contents of which are herein incorporated by reference. The processing chip may comprise a plurality of tiles that communicate with each other using a time deterministic exchange.

The description above sets out the logical arrangement of the computer systems described herein, including the processor cores22or chips20, fabric chips40and DRAMs10. Hereinbelow, the physical layout and construction of some elements of the computer systems will be described in further detail.

Turning now toFIGS.3ato6b, there is shown a memory and routing module100according to an example of the disclosure.

The module100comprises a plurality of fabric chips140, a plurality of DRAMs110, and two connection components160. The fabric chips140and DRAMs110correspond to the fabric chips40and DRAMs10discussed hereinabove. That is to say, the fabric chips140and DRAMs110discussed below incorporate the features discussed above in respect of fabric chips40and DRAMs10. The fabric chips140on the module100include memory controllers for accessing the DRAMs110.

The fabric chips140, DRAMs110and connection components160are attached to a substrate170, which takes the form of a planar board. The board may be approximately 80 mm×70 mm, for example 77 mm×69 mm to give a surface area of approximately 5300 to 5400 mm2.

An upper side171of the substrate170supports eight DRAMs110a, which may for example be arranged in a two×four grid, extending from one edge170aof the substrate to an opposing edge170b. The two×four grid of DRAMs110ais arranged approximately equidistant between two other edges170c,170d, effectively forming a strip along the middle of the module100.

The lower side172of the substrate also supports eight DRAMS110b. The DRAMs110bon the lower side172of the substrate are positioned at locations corresponding to the DRAMs110aon the upper side171. In other words, each DRAM110aon the upper side171is positioned directly above a DRAM110bon the lower side172.

It will be appreciated that “upper side” and “lower side” used herein are merely labels referring to the two sides of the substrate170, and that in use the module100may be mounted such that the lower side172is not below the upper side171.

Each DRAM110may be a DDR (double data rate) DRAM. In one example, each DRAM is an LPDDR (low-power DDR) DRAM, such as an LPDDRS DRAM. Each DRAM may have a capacity of 16 GB, though in other examples the capacity may be 24 GB or 32 GB. LPDDR DRAMs are designed for mobile computing contexts (e.g. on mobile telephones or laptop computers). However, the inventors have found that such memory can advantageously provide high-capacity, low-latency memory suitable for meeting the demands of artificial intelligence/machine learning models in high-performance computing contexts.

The module100further comprises four fabric chips140, located on the upper side171. The fabric chips140may also be referred to herein as “routing chips” or “memory attachment and routing chips”, in view of their above-described function of routing signals between different processor cores22and between the processor cores22and the DRAMS110. Each fabric chip140is positioned in a region170eor170fbetween an edge170cor170dof the substrate and the strip of DRAMs110.

Each fabric chip140is proximate to a different pair of DRAMs110aon the upper side171, and so consequently, a further pair of DRAMs110bon the lower side172. The fabric chip140is connected to those four proximate DRAMs110. In one example, the fabric chip140is connected only to those four proximate DRAMs110.

The module100can therefore be divided into four notional quadrants by a first notional line170yextending between the middle of edge170aand170b, and a second notional line170xextending between, each quadrant comprising a fabric chip140and four DRAMs110a,bconnected to the fabric chip140. The module is reflectionally symmetrical in both lines170xand170y. One quadrant102qof the module100is indicated onFIG.4. Each quadrant102qcan be considered a sub-module of the module100.

The module100comprises two connection components160disposed on the lower side172of the substrate170. One connection component160is positioned on the underside of region170eand the other connection component160is positioned on the underside of region170f, and so each connection component160is underneath two of the fabric chips140. Each connection component160is configured to mate with a corresponding connection component formed on a motherboard400, which will be discussed in more detail below. Accordingly, the module100is connectable to, and disconnectable from, the motherboard by virtue of the connection components160. The connection components160therefore form the electrical coupling or link between the module100and the rest of the system beyond the module100.

The module100, and more particularly each fabric chip140, connects to the processor cores22through the connection components160. Each connection component provides a plurality of connectors (e.g. pins). Each fabric chip140connects via one or more connectors of the connection component160which it is disposed above. Accordingly, the connectors of the connection component160can be considered part of the physical embodiment of the links L1-L4discussed above in relation toFIG.1, in that they are part of the signal path that extends between the processor cores22and fabric chips140.

The connectors of the connection component160also provide part of the physical embodiment of the link between each module100and other modules100in other pods, and the links to the rest of the system. The module100therefore does not include a processor core22, but instead provides routing for data between processor cores22as well as memory access. In other words, the only processing power on the module100may be that provided in the fabric chips140. The processor cores22are disposed remote from the module100and do not form part of the module100.

Furthermore, as discussed above, there are no high-bandwidth direct connections between fabric chips140. Accordingly, each fabric chip140on the module100is not connected to the other fabric chips140on the same module100.

Each connection component160may take the form of a mezzanine connector. The connection components160may be hermaphroditic mezzanine connectors, which may for example have eleven rows, each row having fifteen pairs of pins, which may also be referred to as connectors. An example pair of pins161is labelled onFIG.4—for clarity the remaining pins have not been labelled. The mezzanine connector may be a Mirror Mezz connector supplied by Molex®. In other examples, other connection components160may be employed. For example, connectors supplied Samtec®, TE Connectivity® or Amephenol® may be used. The connection components160are also part of the physical linkage between the module100and the motherboard, providing physical support to the module100.

The structure of the substrate170and the connections between the elements of the module100and the substrate170will now be further discussed.

The substrate170is a package substrate. Accordingly, the substrate170is not a traditional printed circuit board, but instead is a substrate of the type that is typically used inside a chip package to support a die of the chip. The substrate170may also be referred to as a high-density interconnect (HDI) substrate or interposer substrate. It will be understood that the use of the term “interposer” in this context does not imply that the substrate acts as an intermediate or interposed layer, but instead is merely a reference to the type of substrate employed. As will be apparent from the description herein, the package substrate170is the main substrate of the module100and does not act as an interposer.

In one example, the package substrate170is a High Tg glass epoxy multilayer material, such as MCL-E-705G provided by Hitachi®.

In one example, the substrate170is monolithic. In other words, the substrate170is a single, unbroken substrate. In other examples, the substrate170may comprise two or more substrates coupled together, either physically, electrically or both.

As shown inFIG.6a, the interposer substrate170comprises a core173, and a plurality of build-up layers174formed on the core173. The core173has two layers173a,173b, the first core layer137abeing insulative and acting to provide strength to the substrate. The second core layer173bmay be a copper layer. The core173may be approximately 1.2 mm in thickness.

The build-up layers174, which are shown in more detail inFIG.6b, each carry a plurality of conductive lines or wires177that electrically connect the elements of the module100. The build-up layers174may each comprise a copper foil sublayer174afrom which the conductive lines are formed and an insulative sublayer174b, to isolate each build-up layer174from other build-up layers174. Each copper foil sublayer174amay be approximately 12 microns thick. Each insulative sublayer174bmay be approximately 30 microns thick. Accordingly, it will be appreciated thatFIG.6ais not to scale and exaggerates the size of the build-up layers174relative to the core173.

In one example, six build-up layers174are formed on each side of the core173, to give a6:2:6interposer substrate. The build-up layers174on one side of the substrate170may each have a different function. For example, one or more of the layers174may be a ground layer comprising conductive lines177connected to ground. One or more of the layers may be a VDD layer, comprising conductive lines177connected to VDD. One or more of the layers174may be a signal layer, carrying signals between the connection components160and the fabric chips140, and between the fabric chips140and the DRAMs110. In one example, two of the build-up layers174are signal layers. The outermost layer174may comprise pads (not shown) for connection to the other elements of the module170.

Furthermore, as illustrated inFIG.6a, vias174cmay be formed between build-up layers174, so that the conductive lines177may pass between layers174. In addition, core vias175may also be formed through the core173, so that the conductive lines177may pass from the upper side of the substrate170to the lower side of the substrate170.

The fabric chips140are flip chips, secured directly to the substrate170. In other words, the fabric chips140are semiconductor chips fabricated to include solder bumps on a surface of the die thereof. These bumps are then directly attached to the substrate170. In one example, the bump pitch in a core region of the chip is approximately as follows:x=261 micronsy=154 micronsdiagonal pitch=151 microns
where x is the width direction between long edges of the chip, and y is the length direction between short edges of the chip. The pitch may be wider in areas where the connection is provided to the DRAMs110, for example x=286 microns, y=164 microns, diagonal pitch=167 microns.

The conductive lines177of the interposer substrate170are sufficiently fine in the regions of the substrate170under each fabric chip140to allow the lines to breakout from the footprint of the chips140.

The DRAMs110are attached to the substrate170using a ball grid array (BGA). That is to say that the DRAMs110each take the form of a packaged semiconductor chip comprising a die and a package substrate. The die is secured to the upper side of the package substrate, and electrically connected thereto. The package substrate has a grid of solder balls formed on the underside thereof, which are in turn secured to corresponding conductive pads on the substrate170. The BGA may for example have a pitch of 650 microns. The pitch of the balls is therefore substantially coarser than the bumps of the fabric chip140.

The connection components160may also be connected to the substrate170via a BGA. Each connection component160therefore may comprise a grid of solder balls disposed on a face of the connection component160opposing the face including the pins161.

Returning toFIGS.3aand4, the die of the fabric chip140is rectangular and has two opposing long edges140a,140band two opposing short edges140c,140d. The short edges140c,140dare approximately 6 mm long. The long edges140a,140bare approximately 15 mm long. In one example, each fabric chip is 5.5 mm×15.3 mm. Accordingly, the aspect ratio of the chip140is approximately 3:1. In one example, the fabric chip140is a single or monolithic die.

The fabric chip140comprises a plurality of memory controllers. Each memory controller142is a circuit formed in the die of the chip, which acts as an interface to the DRAMs100. In examples where the DRAMs are LPDDR DRAMs, the memory controllers are LPDDR interfaces. The LPDDR interfaces implement the relevant LPDDR standard (e.g. a JEDEC standard, for example JESD209-5B). In examples where the DRAMs110are of a different type, the memory controllers142may accordingly implement the standard necessary to access the DRAMs110.

The memory controllers142are arranged on one long side140aof the chip140. Each chip140is arranged on the module100so that this side140afaces the DRAMs100. This arrangement may ease breaking out the connections to the DRAMs100from under the footprint of the chip140.

Each memory controller forms part of the DDR interface blocks48discussed above. Accordingly, the routing logic of the fabric chip140is configured to route signals to and from the DRAMs110using the trunk nodes46.

The fabric chip140furthermore comprises a plurality of link controllers. Each link controller may comprise a circuit formed in the die. Each communication lane provided by the link controllers may be a serial link, such as a SERDES link as discussed above. Accordingly, the link controller may comprise an analogue circuit. In one example, the link controllers provide 100 Gbps links.

The link controllers provide communication lanes for communication with respective processor cores22. As discussed above in relation toFIG.1, three links (e.g. L2a, L2b, L2c) are provided to each processor core. Accordingly, each communication lane corresponds to an EPC (e.g. EPC2a, EPC2b, EPC2c) shown onFIG.1.

The link controllers may also provide three lanes of pod-facing communication. The communication lanes may implement EPCs corresponding to Pa, Pb, Pc ofFIG.1to implement pod-facing links PLa, PLb, PLc. The pod-facing links PL may also be termed cluster connecting links, in that they connect the fabric chip to another cluster.

The link controllers may also comprise a lane of system facing communication. Accordingly, one of the communication lanes implements an EPC (i.e. corresponding to PCS ofFIG.1) to provide the system link SL. The system link SL is for example connected to a switching fabric. The link controllers may also provide a PCIe link to a host computer.

The link controllers may be arranged along the opposite long edge140bto the memory controllers142. Again, this may assist in easing the breakout of lines from under the fabric chip140.

FIG.17illustrates an example power supply arrangement for the module100. As shown inFIG.17, the motherboard400to which the module100may be attached comprises power supply components400P. The power supply components400P may for example comprise a point of load power supply, disposed on an opposing side of the motherboard400to the module100. Power is supplied from the power supply components400P through the connection components420and160to the fabric chip140.

Accordingly, the module100may not comprise a power supply, such as a point of load power supply. This allows the substrate170to be made smaller, reducing the use of the relatively high-cost substrate material. Furthermore, the positioning of the fabric chip140on the other side of the substrate170directly above the connection component160minimises the distance between the power supply components400P and the fabric chip140, reducing the IR drop.

As set out above, the memory and routing module100comprises a plurality of fabric chips140and a plurality of DRAMs110. In use, these components generate heat, which may lead to damage of the fabric chips140and/or DRAMs110, as well as to other proximate components either on the motherboard to which the module is attached, or otherwise disposed within an enclosure housing the motherboard. Accordingly, there is a need to effectively dissipate the heat generated by the module100.

Issues may arise with the double-sided construction of the module100, and in particular the DRAMs110bdisposed on the lower side172of the module100. These DRAMs110bare disposed between the substrate170and the motherboard in use, and the DRAMs110bare positioned in a relatively narrow channel defined by the connection components160on either side of the DRAMs110b. This may make it difficult to effectively dissipate heat from the underside of the module100.

FIGS.7to13eillustrate a heatsink200provided to address these issues.

The heatsink200comprises a lower heatsink component210and an upper heatsink component220.

The lower heatsink component210is configured to receive the lower side172of the module100. The lower heatsink component210comprises a plate211having a recess212formed therein. The plate211may have dimensions of approximately 90 mm×100 mm, with the recess212formed approximately in the centre of the plate211. The main body of the plate211has four peripheral edge regions: two opposing edge regions211a,211bwhich are respectively arranged at the edges170a,170bof the module100in use, and two opposing edge regions211c,211d, which are respectively arranged at the edges170c,170dof the module100in use. The edge regions211a-dmay be collectively referred to as the peripheral region of the lower heatsink component210.

The upper side of recess212has a length and width that correspond to the dimension of the substrate170of the module100. The lower side of the recess212includes a flange212a, which extends laterally into the recess212to engage with a peripheral edge region of the lower side172of the substrate170. Accordingly, once inserted into the recess212, the module100cannot move laterally and is effectively captured by the recess212of the lower heatsink component210. Therefore, the lower heatsink component210effectively comprises a tray in which the substrate170sits. The recess212is an example of a module receiving region of the lower heatsink component210.

The recess212comprises two apertures214a,b, which extend through the plate211. The apertures214a,bare sized to allow a respective one of the connections components160a,bto extend therethrough, so that they protrude from the underside of the lower heatsink component210when the substrate170rests on the flange212a.

The lower heatsink component210further comprises a cross member213extending across the recess212from peripheral edge region170ato peripheral edge region170b. The cross member213forms the inner boundary of each of the two apertures214a,b. The cross member213is coupled to the underside of the plate211. The cross member213is positioned under the DRAMs110bof the module100, with its major surface facing the DRAMs110b. The cross member213is therefore arranged to conduct heat generated by the DRAMs110bto the plate211.

The cross member213comprises thermal interface material231disposed on its upper surface. The thermal interface material231has high thermal conductivity. This assists in heat transfer and ensures conformant thermal conduction over the cross member213. This assists in situations where a particular DRAM of the DRAMs110bis generating excess heat, but other DRAMs110bare not. The cross member213, or more particularly the thermal interface material231disposed thereon, may be in direct contact with the DRAMs110b.

The thermal interface material231may be a thermally conductive pad, for example a silicone pad. Particularly, the thermal interface material231may be a pad of Tpli™200, supplied by Laird™. However, other suitable thermal interface materials may be employed, including thermal pastes, thermal adhesives, phase-change materials and so on.

The plate211of the lower heatsink component210comprises a plurality of fins215. The fins215are upstanding on an upper surface of the plate211. In the example shown, the fins215are substantially rectangular, but in other examples the shape may be varied or other heat dissipating structures or elements may be employed. For example, square or circular fins may be employed, or fins of any other suitable shape. The fins215are generally arranged in two rows, one disposed along peripheral edge211aand one along peripheral edge211b.

Gaps215aare formed in the rows of fins215to accommodate means for attaching the upper heatsink component220to the lower heatsink component210, which will be discussed in more detail below. Two breaks215amay be formed in each row, to provide four locations for the attachment of the upper heatsink component220. In each gap215a, a threaded bore215bis formed in the plate211.

The peripheral edges211c,dof the lower heatsink component210comprise part of the mounting system250for attaching the module100and heatsink200to the motherboard400, which will be discussed in further detail below with reference toFIG.13.

The upper heatsink component220comprises a base plate221. The lower side222of the base plate221faces the upper side171of the module100, so as to conduct heat generated by the fabric chips140and DRAMs110adisposed on the upper side171. The lower side222of the base plate221comprises three regions222a,b,c. Regions222aand222brespectively correspond to the regions170e,170fin which the fabric chips140are located. Region222cis disposed in between the regions222a,222band corresponds to the location of the DRAMs110aon the upper side171of the module100. The region222cmay be recessed to accommodate the DRAMs110a, which may be taller than the fabric chips140.

Thermal interface material may be disposed on the lower side222. As discussed above, the thermal interface material231has high thermal conductivity and thus assists in conformant thermal conduction over the plurality of fabric chips140and DRAMs110adisposed on the upper side171. The thermal interface material may be provided in three portions232,233,234, each portion disposed in a corresponding one of the regions222a,b,cThe lower side222, or more particularly the thermal interface material232-234disposed thereon, may be in direct contact with the DRAMs110aor fabric chips140.

Like the thermal interface material231, the thermal interface materials232,233,234may be a pad of silicone, for example Tpli™200, supplied by Laird™. However, other suitable thermal interface materials may be employed, including thermal pastes, thermal adhesives, phase-change materials and so on. Different thermal interface materials may be employed for each of material sections231-234—it need not be the case that all of the thermal interface materials are the same.

The base plate221further comprises lugs or tangs223extending laterally outwards from the sides thereof. The lugs223are disposed at locations corresponding to the threaded bores215b, so that threaded fasteners240can be passed through the lugs223and driven into the bores215bto secure the upper heatsink component220to the top of the lower heatsink component210. As shown inFIG.10, the lugs223are configured to deform, for example by bending downwards during attachment. Furthermore, the upper heatsink component220may contact the substrate170in a region227directly above the flange212a, so as to securely clamp the module100in place.

The force required to mate the connection component160with the corresponding connection component420is high. Furthermore, the material of the substrate170may be relatively weak, as well as being high cost. By clamping the substrate170in this manner, damage to the substrate170and components or solder joints thereof may be avoided, because the whole assembly300is attached and detached from the motherboard400. In other words, the substrate170need not be directly handled during attachment or detachment of the assembly300. Accordingly the components of the heatsink200provide stiffness to the substrate170, preventing the bending thereof and generally protecting the substrate170.

The upper side224of the base plate221comprises a plurality of fins225. The fins215are upstanding on an upper surface of the plate211. In the example shown, the fins215are substantially rectangular, but in other examples the shape may be varied or other heat dissipating structures or elements may be employed. The fins215are generally arranged in a single row, covering almost all of the base plate221apart from regions226which accommodate the mounting system250.

In some examples, the heatsink200is substantially formed of aluminum. For example, the component210and220may be formed of or comprise aluminum. In other examples, the heatsink200may be formed of or comprise other thermally conductive materials, including copper and graphene. In some examples, the fasteners240are formed of another material, which may be stronger than aluminum. For example, the fasteners240may comprise steel, such as plated steel.

In use, an assembly300is formed from the heatsink200and module100as follows. Firstly, thermal interface material as231is applied to the cross member213. The module100is then seated in the recess212of the lower heatsink component210, with the connection components160extending through the apertures214.

Subsequently, the thermal interface material232,233,234is applied to the lower side222of the upper heatsink component220. The upper heatsink component220is then placed on top of the module100, with the DRAMS110ahoused in the recessed region222cof the lower side222. The fasteners240are driven through the lugs223and into the bores215b, until the lugs223deform downward to securely clamp the upper heatsink component220to the lower heatsink component210.

The heatsink200assembled with the module100provides a passive cooling mechanism for the semiconductor chips (i.e. the fabric chips140and DRAMs110) disposed on the module100. By passive, it is meant that the heatsink200comprises no moving parts such as fans or other powered cooling mechanisms such as refrigerated liquid or gas. It will be appreciated that the enclosure in which the heatsink200is disposed may comprise fans and/or means of cooling the air (or other gas) within the enclosure, so as to provide cooling by forced convection.

During operation, the heatsink200conducts heat generate by the DRAMs110in a lateral direction through the cross member213and out to the peripheral edge region211of the lower heatsink component210, whereupon the large surface area of the fins215assists in dissipating the heat. In addition, the tight coupling of the heatsink components210,220allows heat to be conducted from the lower heatsink component210to the upper heatsink component220, whereupon it can be dissipated by fins225. Furthermore, heat generated by the fabric chips140and DRAMS110aon the upper side171of the module is conducted through the base plate221and dissipated by the fins225.

FIG.11schematically illustrates an example motherboard400. The motherboard400may be a printed circuit board. The motherboard400supports a plurality of assemblies300, each assembly300comprising a heatsink200and module100. The motherboard400also supports a plurality of processor chips20. In the example shown, the motherboard400supports four processor chips20arranged in a row extending across the motherboard400. The motherboard400also comprises eight assemblies300, arranged in two rows401,402. The rows401,402are disposed in parallel on opposing sides of the row of processor chips20.

As can be seen inFIG.12, the motherboard400comprises a plurality of connection components420, which are configured to mate with connection components160of the respective modules. The motherboard400comprises conductive lines or other wiring (not shown) that connects the connection components420, and thus the installed modules100, to the processors20. The motherboard400may also comprise conductive lines or wiring (not shown) that can connect to networking hardware, power sources and the like.FIG.12also demonstrates that each processor20may be provided with a cooling apparatus480, which may for example comprise a vapour chamber and/or an active cooling means such as a means for supplying cooled air or a refrigerated liquid to the processor20.

FIG.13a-eillustrate a process for attaching the assembly300to the motherboard400.

As discussed hereinabove, the connection components160may take the form of mezzanine connectors comprising a relatively large number of pins161. For example, one mezzanine connector160may comprise 688 pins. The pins161are relatively small and are susceptible to bending and other deformation during connection to and disconnection from the corresponding connection component420on the motherboard400. This may be caused by the connection component160being misaligned with the corresponding connection component420during connection or disconnection. This may occur when one end of the connection component160is lifted from the corresponding connection component420whilst the other end remains in place, a problem which can be exacerbated by the relatively long and thin shape of the connection components160,420. It may also occur when one of the two connection components160aof the module100is lifted from its corresponding connection component420, whilst the other connection component160bremains engaged. This causes the module to pivot at an angle about the connection component160bthat remains engaged, potentially deforming its pins161. Whilst the above issues are explained in the context of a mezzanine connector160, similar issues may arise with other connection components with relatively fine pins161or that are otherwise susceptible to damage during engagement or disengagement.

To address these issues, the assembly300comprises a mounting system250that ensures the connection components160are aligned with their corresponding connection components420and that limits the angle of the substrate170with respect to the motherboard400during installation and removal of the assembly300.

The mounting250system is shown inFIGS.7-9and13a-e, and comprises mounting fasteners260aand260b, which are configured to be received in respective fastener receiving sections460of the motherboard400. As can be best seen inFIGS.12and13e, the fastener receiving sections460take the form of threaded bores, which for example extend from the surface of the motherboard400on which the assembly300is mounted into bosses arranged on the opposing (e.g. lower) surface of the motherboard400.

A first one of the mounting fasteners260ais located at the end of the heatsink200corresponding in position to edge170cof the module100. A second mounting fastener250bis located at the opposing end of the heatsink200, proximate to edge170dof the module.

Accordingly, the mounting fasteners260are positioned along a long edge of each of the connection components160aand160b, for example substantially in the middle of the long edge.

The mounting fasteners260a,beach extend through a respective aperture251a,bin the upper heatsink component220. As discussed above, breaks226in the fins225of the upper heatsink component220accommodate and allow access to the fasteners260. The mounting fasteners260aand200bfurther extend through respective apertures252aand252bin the lower heatsink component210. The apertures252aand252bare formed in the peripheral edge regions211cand211dof the lower heatsink component, for example substantially in the middle of the respective edges. The apertures251aand252aform a channel through which the fastener260aextends. The apertures251band252blikewise form a channel through which the fastener260bextends.

A mounting fastener260is shown in more detail inFIG.14. The fastener260may take the form of a screw.

The fastener260comprises a head section261at a first or top end thereof. The head section261is configured to receive a driver (not shown), for rotating the fastener260. In the examples shown, the head section261has a hex socket for receiving an allen key or hex key, though in other examples different drivers may be employed.

A threadless body section262is disposed below the head section261, which may for example be a cylindrical body section. The threadless body section262is narrower than the head section261. A collar263is formed around a lower region of the threadless body section262, wherein the collar263extends radially outward from the body section262, and thus forms a wider section of the threadless body section262.

Below the threadless body262, the fastener260comprises a threaded section264configured to engage with the threaded bore of the receiving section260. The second or bottom end of the fastener260comprises an unthreaded locating projection265having a substantially flat tip266, which may be referred to as a “dog point”.

As shown inFIG.13a, the fastener260is installed in the heatsink200so that the threadless body section262extends through the channel formed by the apertures251and252. The head section261and the collar263are wider than the channel, so that once installed the fastener260is held captive in the channel. The body section262is longer than the channel, so that it can move vertically within the channel. However, the range of vertical motion of the fastener260is limited by the head261and collar263. In one example, the portion of the threadless body section262between the head261and collar is263is 2 mm longer than the channel. The length difference between the channel and the threadless body section262defines the length of the first phase of attachment discussed below.

The fastener260may be installed in the channel formed by the apertures251and252after the upper heatsink component220and lower heatsink component210are assembled, for example by inserting a collarless fastener260into the channel and subsequently attaching the collar263.

Engagement of the mounting system250with the motherboard400will now be discussed with particular reference toFIGS.13a-e.

Firstly, as shown inFIGS.13aand13b, a user installing the assembly300uses the unthreaded locating projection265to locate the fastener260in the receiving section460. The locating projection265acts as guide, allowing the fastener260to be partially received in the receiving section460before making a threaded connection. Locating projections265of both fasteners260a,260bmay be located in their respective receiving sections, so that the assembly300is correctly aligned with the motherboard400and connection components420thereon before a threaded connection is made.

Next, as shown inFIG.13c, a first phase of attachment commences in which the user advances the fastener260by rotating it to make a threaded connection with the receiving section460. As the fastener260advances into the receiving section460, the connection is made. Consequently, the unthreaded body section262descends through the channel. However, the descent of the fastener260in this phase of attachment does not cause the assembly300to be drawn towards the motherboard400to make the connection between the connection components160a,band connection components420. This is because the head261of the fastener260is not engaged with the heatsink200in the area around the aperture251a. As noted above, the length of this phase of attachment is defined by the length difference between the channel and the unthreaded body section262. This may correspond to a particular number of turns of the fastener260. The first phase of attachment ends when the head261of the fastener260engages the heatsink200.

In one example, the user ceases the descent of one of the fasteners260at the end of the first phase of attachment (i.e. when the head261engages the heatsink200), and carries out the corresponding process with the other fasteners260. Accordingly, at the end of the first phase of attachment, the assembly300is physically attached to the motherboard400, with the connection components160aligned with the corresponding connection components420. However, the connection components160are not engaged with the corresponding connection components420.

Finally, in a second phase of attachment, the fasteners260are further descended to engage the connection components160with the corresponding connection components420. The second phase of attachment ends when the threaded section264is fully advanced into the receiving section, so that the bottom of the unthreaded body section262, which may be referred to as the shoulder of the fastener260, contacts the receiving section460.

By providing the two phases of attachment, the assembly300is physically attached to the motherboard400and correctly aligned at the point at which the connection components160,420are connected. Furthermore, at the end of the first phase of attachment the possible range of motion of the assembly300with respect to the motherboard400is limited by the fasteners260. Accordingly, the possibility of the major plane of the substrate170of the module100being placed at a sufficiently large angle with respect to the major plane of the motherboard400to cause damage to connectors160,420is reduced. Furthermore, employing the first phase of attachment ensures that sufficient thread is engaged during the attachment process to secure the assembly300to the motherboard400. That is to say, providing the two phases of engagement ensures the user drives the fasteners260sufficiently far into the receiving sections460to provide suitable engagement. In one example, the fastener260is sized so that the engaged thread is equal to or greater than the diameter of the fastener260.

To remove the assembly300from the motherboard400, one of the fasteners (e.g. fastener260a) is rotated in the opposite direction, until its collar263contacts the underside of the lower heatsink component210.

Subsequently, the fastener260can be rotated further until its threaded section264is fully removed from the bore of the receiving section.FIG.13eillustrates the assembly300in this state, with the corresponding connection components420of the motherboard400omitted for clarity. During retraction of the fastener260, the collar263acts on the underside of the peripheral edge211cto raise the edge211cand disengage the connection component160a.

Accordingly, at the time that the collar263begins to act on the underside of peripheral edge211c, only a portion of the thread264of the fastener remains engaged with the receiving section460. This then limits the distance that the edge211ccan be raised to a distance corresponding to the length of the threaded portion264remaining engaged. Accordingly, the retraction of the thread cannot cause the side211cto pivot with respect to side211dbeyond a safe angle α. In one example, the safe angle α is 1.3°.

FIG.15shows an example method of assembling a memory and routing module and a heatsink. The method comprises a step S1501of placing the module in a module receiving region of a lower heatsink component, so that a thermally conductive portion of the lower heat sink component faces a plurality of semiconductor chips disposed on a lower side of the memory and routing module. The method comprises a step S1502of connecting an upper heatsink component to the lower heatsink component to retain the memory and routing module between the upper heatsink component and lower heatsink component. The method may comprise further steps, as described herein.

FIG.16shows an example method of mounting a memory and routing module to a motherboard. The method comprises a step S1601of placing the module in a module receiving region of a mounting component. The mounting component may for example be a heatsink that also acts to dissipate heat generated by the module. The method comprises a step S1602of extending a fastener of the mounting component into a fastener receiving section of the motherboard without drawing the mounting component toward the motherboard. The method comprises a step S1603of extending the fastener into the fastener receiving section to draw the mounting component towards the motherboard.

Various modifications may be made to the module100described above. In some examples, the number of fabric chips140and DRAMS110present in the module100may be varied from the examples discussed above. For example, each fabric chip140may be connected to fewer DRAMs110(i.e. 1, 2 or 3 DRAMs) or more DRAMs110(5 or more DRAMs, for example 8 DRAMs). In other examples, the module100may comprise fewer fabric chips140or more fabric chips140. The module100may comprise fewer of the notional quadrants discussed above (e.g. 2 of the quadrants, 6 of the quadrants, 8 of the quadrants or any other suitable number). In some examples, the number of connection components160provided may also be varied. For example, only one connection component160, or more than two connection components160may be provided. Furthermore, the elements of the fabric chip140may be varied.

Various modifications may also be made to the heatsink200described above. For example, the upper and lower heatsink components may be secured together by alternative means, including hinges, catches, latches and the like. Although the above examples include a single integral upper heatsink component and a single integral lower heatsink component, in other examples the upper and lower heatsink components may comprise multiple subcomponents coupled together. The size and shape of the heatsink may be varied to accommodate larger or smaller modules, or modules that have more or fewer connection components.

The mounting system may comprise more or fewer fasteners. Furthermore, the fasteners may have different configurations or designs that permit the above-described two-phase attachment. Furthermore, in some examples the mounting system may be disposed in a component that does not act as a heatsink. For example, a mounting component comprising a module receiving section but no fins or crossmember may be provided. In other examples, the mounting system may be disposed in the upper heatsink component or the lower heatsink component only. That is to say, the channel in which the fastener is disposed may be formed in the upper heatsink component only, the lower heatsink component only or both as discussed above.

Advantageously, the module100provides routing functionality and a high-capacity, high-bandwidth and low-latency memory for a processor20, rendering it suitable for the processing of large machine learning models. When arranged as shown inFIG.8with a ratio of two modules100to each processor20, each processor has access to 512 GB at an example bandwidth of 9.6 Tbit/s. Furthermore, the use of module100limits the links required on the processor20, saving valuable beach front space that would ordinarily be used for memory access and routing.

Advantageously, the module100comprises directly flip-chip attached fabric chips140and DRAMs110and connection components160attached to the module100via BGAs. By permitting direct flip-chip attachment of the fabric chips140to the substrate, the fabric chips140do not require additional packaging, and thus the overall size of the module100can be reduced.

Furthermore, the above-described heatsink provides a means of conducting heat generated by chips disposed on the underside of the module to dissipate it, thereby preventing damaged caused by overheating. The mounting system advantageously ensures alignment of the connection components during connection and disconnection of the assembly from the motherboard and may prevent too large an angle developing between the plane of the module and the plane of the motherboard. In addition, the heatsink may support the module during connection and disconnection. Accordingly, damage to the connection components or the module more generally may be avoided.