Patent ID: 12248679

DETAILED DESCRIPTION

Semiconductor devices, packaging architectures and associated methods are disclosed. In one embodiment, a multi-chip module (MCM) is disclosed. The MCM includes a common substrate and a first integrated circuit (IC) chip disposed on the common substrate. The first IC chip includes a first memory interface. A second IC chip is disposed on the common substrate and includes a second memory interface. A first memory device is disposed on the common substrate and includes memory and a first port coupled to the memory. The first port is configured for communicating with the first memory interface of the first IC chip. A second port is coupled to the memory and communicates with the second memory interface of the second IC chip. In-memory processing circuitry is coupled to the memory and controls transactions between the first memory device and the first and second IC chips. By including the in-memory processing circuitry on the memory device, controlled accesses to the memory for operations associated with the first IC chip and the second IC chip may be carried out with lower latency and lower cost. For some embodiments, the in-memory processing circuitry takes the form of a co-processor or accelerator that is capable of carrying out a processing function that is off-loaded from the first IC chip or second IC chip on data retrieved from the memory. In other embodiments, the in-memory processing circuitry may include network-on-chip (NoC) circuitry to control the transactions between the memory and the first IC chip and the second IC chip.

Throughout the disclosure provided herein, the term multi-chip module (MCM) is used to represent a semiconductor device that incorporates multiple semiconductor die or sub-packages in a single unitary package. An MCM may also be referred to as a system in a chip (SiP). With reference toFIG.1, a multi-chip module (MCM) is shown, generally designated100. For one embodiment, the MCM includes a substrate102that serves as a common substrate for a first integrated circuit (IC) chip104, a second IC chip106and a memory device108. For some embodiments, the various chips are interconnected in a manner that allows for use of a relatively inexpensive non-silicon or organic substrate as the common substrate. The use of a non-silicon common substrate102avoids size and signaling constraints typically associated with silicon-based substrates. This allows the substrate102to be larger, incorporate a more relaxed bump pitch for external interface contacts, and provide low-loss traces.

With continued reference toFIG.1, the first IC chip104is mounted to the common substrate102and may take the form of a computer processing unit (CPU), graphics processing unit (GPU), artificial intelligence (AI) processing circuitry or the like. For one embodiment, the first IC chip104includes first interface circuitry105for communicating with the memory device108. For one embodiment, the first interface circuitry105supports transactions with the first memory device108via a high-speed link118. Various embodiments for compatible interface schemes are disclosed in U.S. patent application Ser. No. 17/973,905, titled “Method and Apparatus to Reduce Complexity and Cost For Multi-Chip Modules (MCMs)”, filed Oct. 26, 2022, incorporated by reference in its entirety, and assigned to the assignee of the instant application. The second IC chip106may be formed similar to the first IC chip104, including second interface circuitry107for communicating with the memory device108. Like the first IC chip104, the second IC chip106may take the form of a computer processing unit (CPU), graphics processing unit (GPU), artificial intelligence (AI) processing circuitry or the like.

With continued reference toFIG.1, one embodiment of the memory device108includes a first port112for interfacing with the first IC chip104via the first high-speed link118, and a second port114for interfacing with the second IC chip106via a second link120. Memory110is coupled to the first port112and the second port114and is configured with a unified memory space that, for one embodiment, is fully accessible to each of the first and second ports112and114. While only two ports are shown for clarity, for some embodiments, three or more ports may be employed, corresponding to the edges of a standard IC chip and the available edge space for the interface circuitry.

Further referring toFIG.1, in-memory processing circuitry116provides processing resources in the memory device108to provide a variety of functions. For some embodiments, described more fully below, the in-memory processing circuitry116may take the form of a co-processor or accelerator that carries out functions offloaded from the first IC chip104or the second IC chip106. In other embodiments, the in-memory processing circuitry116may instead (or additionally) include a router functionality in the form of network-on-chip (NoC) circuitry for controlling access between the memory device108and the first and second IC chips104and106, and, in some embodiments, controlling forwarding and receiving operations involving other IC chips (not shown) that may be disposed on the MCM100. Further detail regarding embodiments of the NoC circuitry are provided below.

FIG.2illustrates a cross-sectional view of one embodiment of the MCM100ofFIG.1that employs one specific embodiment of the memory device108. As shown, for one embodiment, the memory device108may be configured as a 3-dimensional (3D) packaging architecture with one or more memory die202stacked and assembled as a sub-package203that is vertically stacked with a logic base die204. For some embodiments, the logic base die204is configured as an interface die for the stack of memory die203and may be compatible with various dynamic random access memory (DRAM) standards, such as high-bandwidth memory (HBM), or non-volatile memory standards such as Flash memory. The stack of memory die203and the logic base die204may be packaged together as a sub-package to define the memory device108, with the logic base die204further formed with an external interface in the form of an array of contact bumps, at206. Various alternative 3D embodiments for the memory device are disclosed in the above-referenced U.S. patent application Ser. No. 17/973,905. Additionally, while shown as a 3D stacked architecture, the memory device108may alternatively take the form of a 2.5D architecture, where the various die are positioned in a horizontal relationship. Such architectures are also described in U.S. patent application Ser. No. 17/973,905.

Referring now toFIG.3, for one embodiment, the logic base die204incorporated in the memory device108is manufactured in accordance with a logic process that incorporates node feature sizes similar to those of the first IC chip and the second IC chip, but with a much smaller overall size and footprint. As a result, operations carried out by the logic base die204may be more power efficient than those carried out by the larger IC chips104and106. In some embodiments, the logic base die204includes memory interface circuitry302that defines the first and second ports112and114(FIG.1), allowing the first and second IC chips104and106to access the entirety of the memory space of the memory110. For one embodiment, the first and second ports112and114take the form of spatial signaling path resources that access the memory via multiplexer or switch circuitry, such that either IC chip has access to any portion of the memory during a given time interval. In this manner, where both of the first and second IC chips share the entirety of the memory110, the memory device108becomes unified, thereby avoiding many of the latency problems associated with separately disposed memory spaces dedicated to separate IC chips.

Further referring toFIG.3, for one embodiment, the logic base die204realizes at least a portion of the in-memory processing circuitry116as co-processing circuitry304. The co-processing circuitry304provides co-processor or accelerator resources in the memory device108to allow for off-loading of one or more CPU/GPU/AI processing tasks involving data retrieved from the memory110without the need to transfer the data to either of the first or second IC chips104or106. For example, in some embodiments, the co-processing circuitry304may be optimized to perform straightforward multiply-accumulate operations on data retrieved from the memory110, thus avoiding the need for the larger and more power-hungry IC chips104or106to perform the same operations. The co-processing circuitry304may be accessed by providing application programming interfaces (APIs) in software frameworks (such as, for example, Pytorch, Spark, Tensorflow) in a manner that avoids re-writing application software. By carrying out offloaded processing tasks in this manner, data transfer latencies may be reduced, while power efficiency associated with the processing tasks may be increased.

For some embodiments, and with continued reference toFIG.3, the logic base die204also provides network-on-chip (NoC) circuitry306for the memory device108. The NoC circuitry306generally serves as a form of network router or switch for cooperating with other NoC circuits that may be disposed in various other IC chips or memory devices disposed on the MCM100. Thus, the NoC circuitry306is generally capable of transferring and/or receiving data and/or control signals via a packet-switched protocol to any other nodes within the MCM100that also have NoC circuitry.

FIG.4illustrates one specific embodiment of the NoC circuitry306ofFIG.3. The NoC circuitry306includes input buffer circuitry410that receives data and/or control signals from a separate NoC circuit associated with another IC chip or node on the MCM100. Depending on how many separate edge interfaces, or ports, are employed by the memory device108, the input buffer circuitry410may include two (corresponding to, for example, “east” and “west” ports such as those shown inFIG.1), three, or four queues (“N INPUT”, “S INPUT”, “E INPUT” OR “W INPUT”) to temporarily store signals received from the multiple ports. The memory interface302of the memory device108may also provide input data/control signals for transfer by the NoC circuitry306to another NoC node in the MCM100.

Further referring toFIG.4, the input buffer circuitry410feeds a crossbar switch406that is controlled by a control unit408in cooperation with a scheduler or arbiter404. Output buffer circuitry412couples to the crossbar switch406to receive data/control signals from the memory device108or the data/control signals from the input buffer circuitry410for transfer to a selected output port/interface (“N OUTPUT”, “S OUTPUT”, “E OUTPUT” OR “W OUTPUT”). The crossbar switch406may also feed any of the signals from the input buffer circuitry410to the memory interface302of the memory device108.

FIG.5illustrates a chip topology on an MCM, generally designated500, that is similar to the architecture ofFIG.1, including a CPU as the first IC chip104, a GPU as the second IC chip106, and an HBM/NoC memory device as the first memory device108. The MCM500also includes additional memory devices504and506that are configured as single-port memory devices and are disposed on the common substrate102in a distributed manner.

FIG.6illustrates an additional architecture that incorporates the topology ofFIG.5, and also includes further memory devices602and604coupled to the memory device504. For one embodiment, the additional memory devices602and604provide additional memory capacity for the first IC chip104without the need for additional corresponding I/O interface circuitry at the edge of the first IC chip104. The first IC chip104thus may access memory device602via the first and second ports of memory device504. Accessing memory device604by the first IC chip104is performed similarly via the first and second ports of memory device504and602. The connection of additional memory devices602and604through memory device504to the first IC chip104can be purely for extending the total memory to the first IC chip104, and such memory extension does not necessarily need a NOC to connect them to other chips in the package. In some embodiments, the interconnected memory devices504,602and604may, for example, provide different memory hierarchies for the first IC chip104. As a result, for the first IC chip104, the memory device504may serve as low-latency memory (such as cache memory) for data accessed more often with minimal latency, while the second and third memory devices602and604may serve as backing store media and/or other forms of storage where additional latency may be tolerated. Further, the addition of the memory devices602and604has little to no electrical impact on the MCM due to the buffering nature of the memory device504(where the aggregate load of the memory devices504,602and604is seen as a single load from the perspective of the first IC chip104). As a result, system software memory management tasks may be simplified as memory capacity is added to the MCM. Use of the unified memory architecture described above for each memory device contributes to a lower cost of use since the unified architecture is able to provide a variety of storage functions for a myriad of applications.

FIG.7illustrates yet another topology that is similar to the MCM ofFIG.5, but further scales the architecture to include a further disaggregated second level of processing and memory resources that are straightforwardly interconnected. Such a topology enables complex application specific integrated circuit (ASIC) chips to be partitioned into smaller interconnected chiplets, such as at702and704, that together form a virtual ASIC706. Having the smaller processing chiplets702and704virtualized in this manner allows for beneficial pairing and sized matching of memory device chiplet packages708to the smaller processing chiplets. Moreover, for embodiments where each memory device and processor chip includes NoC circuitry, any of the IC chips and memory devices of the MCM ofFIG.7may communicate with any other of the IC chips and memory devices.

FIG.8illustrates one embodiment of an MCM800that is similar to the architecture ofFIG.6, with a CPU resource104coupled to a pair of inline memory devices108and504via a single link802. This allows for memory capacity upgrades without requiring additional physical I/O space (multiple interfaces for coupling to multiple links) along the edge of the CPU104. By adding an additional single-port memory device504and coupling it to the multi-port memory device108, accesses to the added memory device504may be made by the CPU104via the in-memory processing circuitry, such as the NoC circuitry, that is disposed in the multi-port memory device108. A similar configuration is shown at the far right of the MCM800with memory devices110and506that are in communication with a GPU106via a second link804.FIG.8also shows a pair of multi-port memory devices112and114that are interconnected by a simultaneous bidirectional link, at806. The simultaneous bidirectional link806allows for concurrent accesses to a given distal memory device by the CPU104(where it accesses memory device114via memory device112) and the GPU106(where it accesses memory device112via memory device114). Having the ability to perform concurrent accesses significantly increases the bandwidth of the system. As an example of scaling the architecture ofFIG.8even larger,FIG.9illustrates an MCM900that adds a second row of devices, at902, that interconnect to a first row of devices, at904, essentially doubling the resources provided in the architecture ofFIG.8. Additional rows of devices may also be employed to scale the capacity even further, if desired.

When received within a computer system via one or more computer-readable media, such data and/or instruction-based expressions of the above described circuits may be processed by a processing entity (e.g., one or more processors) within the computer system in conjunction with execution of one or more other computer programs including, without limitation, net-list generation programs, place and route programs and the like, to generate a representation or image of a physical manifestation of such circuits. Such representation or image may thereafter be used in device fabrication, for example, by enabling generation of one or more masks that are used to form various components of the circuits in a device fabrication process.

In the foregoing description and in the accompanying drawings, specific terminology and drawing symbols have been set forth to provide a thorough understanding of the present disclosure. In some instances, the terminology and symbols may imply specific details that are not required to practice embodiments of the disclosure. For example, any of the specific numbers of bits, signal path widths, signaling or operating frequencies, component circuits or devices and the like may be different from those described above in alternative embodiments. Also, the interconnection between circuit elements or circuit blocks shown or described as multi-conductor signal links may alternatively be single-conductor signal links, and single conductor signal links may alternatively be multi-conductor signal links. Signals and signaling paths shown or described as being single-ended may also be differential, and vice-versa. Similarly, signals described or depicted as having active-high or active-low logic levels may have opposite logic levels in alternative embodiments. Component circuitry within integrated circuit devices may be implemented using metal oxide semiconductor (MOS) technology, bipolar technology or any other technology in which logical and analog circuits may be implemented. With respect to terminology, a signal is said to be “asserted” when the signal is driven to a low or high logic state (or charged to a high logic state or discharged to a low logic state) to indicate a particular condition. Conversely, a signal is said to be “deasserted” to indicate that the signal is driven (or charged or discharged) to a state other than the asserted state (including a high or low logic state, or the floating state that may occur when the signal driving circuit is transitioned to a high impedance condition, such as an open drain or open collector condition). A signal driving circuit is said to “output” a signal to a signal receiving circuit when the signal driving circuit asserts (or deasserts, if explicitly stated or indicated by context) the signal on a signal line coupled between the signal driving and signal receiving circuits. A signal line is said to be “activated” when a signal is asserted on the signal line, and “deactivated” when the signal is deasserted. Additionally, the prefix symbol “/” attached to signal names indicates that the signal is an active low signal (i.e., the asserted state is a logic low state). A line over a signal name (e.g., ‘<signal name>’) is also used to indicate an active low signal. The term “coupled” is used herein to express a direct connection as well as a connection through one or more intervening circuits or structures. Integrated circuit device “programming” may include, for example and without limitation, loading a control value into a register or other storage circuit within the device in response to a host instruction and thus controlling an operational aspect of the device, establishing a device configuration or controlling an operational aspect of the device through a one-time programming operation (e.g., blowing fuses within a configuration circuit during device production), and/or connecting one or more selected pins or other contact structures of the device to reference voltage lines (also referred to as strapping) to establish a particular device configuration or operation aspect of the device. The term “exemplary” is used to express an example, not a preference or requirement.

While aspects of the disclosure have been described with reference to specific embodiments thereof, it will be evident that various modifications and changes may be made thereto without departing from the broader spirit and scope of the disclosure. For example, features or aspects of any of the embodiments may be applied, at least where practicable, in combination with any other of the embodiments or in place of counterpart features or aspects thereof. Accordingly, the specification and drawings are to be regarded in an illustrative rather than a restrictive sense.