TILED COMPUTE AND PROGRAMMABLE LOGIC ARRAY

Examples herein describe a three-dimensional (3D) die stack. The 3D die stack includes a programmable logic (PL) die and a compute die stacked on top of the PL die. The PL die includes a plurality of configurable blocks and a plurality of first electrical connections on a top side of the PL die. The compute die includes a plurality of data processing engines and a plurality of second electrical connections on a bottom side of the compute die. The three-dimensional die stack includes a plurality of tiles, each tile comprising M configurable blocks included in the plurality of configurable blocks and N data processing engines included in the plurality of data processing engines.

TECHNICAL FIELD

Examples of the present disclosure generally relate to integrated circuit (IC) devices, and more specifically, to a tiled compute and programmable logic array.

BACKGROUND

Increasingly, high-performance computing systems implement large numbers of data processing engines and programmable logic (PL) (e.g., a field-programmable gate array or “FPGA”) within the same die and/or integrated circuit (IC) package. Such systems generally provide a flexible and highly parallel computing interface that can be adapted to a wide variety of applications. However, the architectures implemented in current systems suffer from a number of drawbacks.

For example, such systems commonly implement network-based communications in which data processing engines communicate with programmable logic and other IC components via an edge interface. One drawback of this configuration is that, as more and more processing elements need to communicate through an edge interface, the routing channels associated with the edge interface become saturated. As the routing channels approach saturation, routing congestion increases, limiting bandwidth and/or increasing latency between data processing engines and programmable logic. Additionally, due to routing congestion, data processing engines and programmable logic positioned far away from an edge of the interface may have difficulty meeting timing closure requirements, effectively limiting the total number of resources that can be utilized for a given process.

SUMMARY

Techniques for implementing a three-dimensional (3D) die stack. The 3D die stack includes a programmable logic (PL) die and a compute die stacked on top of the PL die. The PL die includes a plurality of configurable blocks and a plurality of first electrical connections on a top side of the PL die. The compute die includes a plurality of data processing engines and a plurality of second electrical connections on a bottom side of the compute die. The three-dimensional die stack includes a plurality of tiles, each tile comprising M configurable blocks included in the plurality of configurable blocks and N data processing engines included in the plurality of data processing engines.

One example described herein is a computing system. The computing system includes a memory and a three-dimensional (3D) die stack coupled to the memory. The 3D die stack includes a programmable logic (PL) die and a compute die stacked on top of the PL die. The PL die includes a plurality of configurable blocks and a plurality of first electrical connections on a top side of the PL die. The compute die includes a plurality of data processing engines and a plurality of second electrical connections on a bottom side of the compute die. The three-dimensional die stack includes a plurality of tiles, each tile comprising M configurable blocks included in the plurality of configurable blocks and N data processing engines included in the plurality of data processing engines.

DETAILED DESCRIPTION

Examples herein describe techniques that implement a tiled compute and programmable logic (PL) (e.g., a field-programmable gate array (FPGA), programmable logic device(s) (PLD), and/or any other type of logic device that is reprogrammable). In various embodiments, the techniques may include vertically aligning, in a three-dimensional integrated die stack, data processing engines (e.g., DPEs110) included in a compute die with programmable elements (e.g., CLBs310) included in a programmable logic (PL) die300. Electrical connections (e.g., through-silicon vias) included on the bottom of the compute die may be pitch-matched and bonded to electrical connections included on the top of the PL die300, enabling orders of magnitude more connections and, as a result, higher bandwidth between the data processing engines and the programmable elements. In some embodiments, this high-bandwidth coupling to programmable logic fabric included in the PL die300enables compute memory (e.g., SRAM or UltraRAM, also referred to as “URAM”) included in each data processing engine to be distributed (e.g., cascaded) between multiple data processing engines, extending the amount of memory available for a given use case. Additionally, in some embodiments, each data processing engine (or tile of data processing engines) may be associated with substantially the same number and type(s) of programmable elements, enabling modular, “soft” intellectual property (IP) blocks to be “stamped” across the compute die and PL die300in a repeatable manner that generates predictable timing, bandwidth, and/or latency. Further, data processing engines (or tiles of data processing engines) may be connected to one another via the programmable logic fabric included in the PL die300in a specific topology, enabling, for example, advanced in-line processing, broadcasting, and other advanced functionality that cannot be efficiently performed via conventional systems that implement an edge interface.

FIG.1is a block diagram of a SoC100that includes a data processing engine (DPE) array105and programmable logic (PL)125, according to an example. The DPE array105includes a plurality of DPEs110which may be arranged in a grid, cluster, or checkerboard pattern in the SoC100. AlthoughFIG.1illustrates arranging the DPEs110in a 2D array with rows and columns, the embodiments are not limited to this arrangement. Further, the array105can be any size and have any number of rows and columns formed by the DPEs110.

In one embodiment, the DPEs110are identical. That is, each of the DPEs110(also referred to as tiles or blocks) may have the same hardware components or circuitry. Further, the embodiments herein are not limited to DPEs110. Instead, the SoC100can include an array of any kind of processing elements. for example, the DPEs110could be digital signal processing circuits, cryptographic circuits, Forward Error Correction (FEC) circuits, or other specialized hardware for performing one or more specialized tasks.

InFIG.1, the array105includes DPEs110that are all the same type (e.g., a homogeneous array). However, in another embodiment, the array105may include different types of circuits. For example, the array105may include digital signal processing circuits, cryptographic circuits, graphic processing circuits, and the like. Regardless of whether the array105is homogenous or heterogeneous, the DPEs110can include direct connections between DPEs110which permit the DPEs110to transfer data directly as described in more detail below.

In one embodiment, the DPEs110are formed from software-configurable hardened logic (i.e., are hardened). One advantage of doing so is that the DPEs110may take up less space in the SoC100relative to using programmable logic to form the hardware elements in the DPEs110. That is, using hardened logic circuitry to form the hardware elements in the DPE110such as program memories, an instruction fetch/decode unit, fixed-point vector units, floating-point vector units, arithmetic logic units (ALUs), multiply accumulators (MAC), and the like can significantly reduce the footprint of the array105in the SoC100. Although the DPEs110may be hardened, this does not mean the DPEs110are not programmable. That is, the DPEs110can be configured when the SoC100is powered on or rebooted to perform different functions or tasks.

The DPE array105also includes a SoC interface block115(also referred to as a shim) that serves as a communication interface between the DPEs110and other hardware components in the SoC100. In this example, the SoC100includes a network on chip (NoC)120that is communicatively coupled to the SoC interface block115. Although not shown, the NoC120may extend throughout the SoC100to permit the various components in the SoC100to communicate with each other. For example, in one physical implementation, the DPE array105may be disposed in an upper right portion of the integrated circuit forming the SoC100. However, using the NoC120, the array105can nonetheless communicate with, for example, PL125, a processor subsystem (PS)130, input/output (I/O)135, or memory controller circuit (MC)140which may be disposed at different locations throughout the SoC100.

In addition to providing an interface between the DPEs110and the NoC120, the SoC interface block115may also provide a connection directly to a communication fabric in the PL125. In this example, the PL125and the DPEs110form a heterogeneous processing system since some of the kernels in a dataflow graph may be assigned to the DPEs110for execution while others are assigned to the PL125. WhileFIG.1illustrates a heterogeneous processing system in a SoC, in other examples, the heterogeneous processing system can include multiple devices or chips. For example, the heterogeneous processing system could include two FPGAs or other specialized accelerator chips that are either the same type or different types. Further, the heterogeneous processing system could include two communicatively coupled SoCs.

In one embodiment, the SoC interface block115includes separate hardware components for communicatively coupling the DPEs110to the NoC120and to the PL125that is disposed near the array105in the SoC100. In one embodiment, the SoC interface block115can stream data directly to a fabric for the PL125. For example, the PL125may include an FPGA fabric which the SoC interface block115can stream data into, and receive data from, without using the NoC120. That is, the circuit switching and packet switching described herein can be used to communicatively couple the DPEs110to the SoC interface block115and also to the other hardware blocks in the SoC100. In another example, SoC interface block115may be implemented in a different die than the DPEs110. In yet another example, DPE array105and at least one subsystem may be implemented in a same die while other subsystems and/or other DPE arrays are implemented in other dies. Moreover, the streaming interconnect and routing described herein with respect to the DPEs110in the DPE array105can also apply to data routed through the SoC interface block115.

AlthoughFIG.1illustrates PL125as one contiguous block, the SoC100may include multiple blocks of PL125(also referred to as logic sub-regions) that can be disposed adjacent to one another and/or at different locations in the SoC100. Each logic sub-region (also referred to as a fabric sub-region) may include a set of configuration logic blocks (CLBs) that can include look-up tables (LUTs). In some embodiments, each logic sub-region is driven by a separate clock signal. In such embodiments, the logic sub-regions may be referred to as “clock regions.” PL125may include hardware elements that form a field programmable gate array (FPGA), programmable logic devices (PLD), and/or any other type of logic device that is reprogrammable. However, in other embodiments, the SoC100may not include any PL125—e.g., the SoC100may be an application-specific integrated circuit (ASIC).

FIG.2is a block diagram of a DPE110in the DPE array105illustrated inFIG.1, according to an example. The DPE110includes an interconnect205, a core210, and a memory230. The interconnect205permits data to be transferred from the core210and the memory230to different cores in the array105. That is, the interconnect205in each of the DPEs110may be connected to each other so that data can be transferred north and south (e.g., up and down) as well as east and west (e.g., right and left) in the array of DPEs110.

Referring back toFIG.1, in one embodiment, the DPEs110in the upper row of the array105rely on the interconnects205in the DPEs110in the lower row to communicate with the SoC interface block115. For example, to transmit data to the SoC interface block115, a core210in a DPE110in the upper row transmits data to its interconnect205which is in turn communicatively coupled to the interconnect205in the DPE110in the lower row. The interconnect205in the lower row is connected to the SoC interface block115. The process may be reversed where data intended for a DPE110in the upper row is first transmitted from the SoC interface block115to the interconnect205in the lower row and then to the interconnect205in the upper row that is the target DPE110. In this manner, DPEs110in the upper rows may rely on the interconnects205in the DPEs110in the lower rows to transmit data to and receive data from the SoC interface block115.

In one embodiment, the interconnect205includes a configurable switching network that permits the user to determine how data is routed through the interconnect205. In one embodiment, unlike in a packet routing network, the interconnect205may form streaming point-to-point connections. That is, the streaming connections and streaming interconnects (not shown inFIG.2) in the interconnect205may form routes from the core210and the memory230to the neighboring DPEs110or the SoC interface block115. Once configured, the core210and the memory230can transmit and receive streaming data along those routes. In one embodiment, the interconnect205is configured using the Advanced Extensible Interface (AXI) 4 Streaming protocol.

In addition to forming a streaming network, the interconnect205may include a separate network for programming or configuring the hardware elements in the DPE110. Although not shown, the interconnect205may include a memory mapped interconnect which includes different connections and switch elements used to set values of configuration registers in the DPE110that alter or set functions of the streaming network, the core210, and the memory230.

In one embodiment, streaming interconnects (or network) in the interconnect205support two different modes of operation referred to herein as circuit switching and packet switching. In one embodiment, both of these modes are part of, or compatible with, the same streaming protocol—e.g., an AXI Streaming protocol. Circuit switching relies on reserved point-to-point communication paths between a source DPE110to one or more destination DPEs110. In one embodiment, the point-to-point communication path used when performing circuit switching in the interconnect205is not shared with other streams (regardless whether those streams are circuit switched or packet switched). However, when transmitting streaming data between two or more DPEs110using packet-switching, the same physical wires can be shared with other logical streams.

The core210may include hardware elements for processing digital signals. For example, the core210may be used to process signals related to wireless communication, radar, vector operations, machine learning applications, and the like. As such, the core210may include program memories, an instruction fetch/decode unit, fixed-point vector units, floating-point vector units, arithmetic logic units (ALUs), multiply accumulators (MAC), and the like. However, as mentioned above, this disclosure is not limited to DPEs110. The hardware elements in the core210may change depending on the circuit type. That is, the cores in a digital signal processing circuit, cryptographic circuit, or FEC may be different.

The memory230includes a DMA circuit215, memory banks220, and hardware synchronization circuitry (HSC)225or other type of hardware synchronization block. In one embodiment, the DMA circuit215enables data to be received by, and transmitted to, the interconnect205. That is, the DMA circuit215may be used to perform DMA reads and write to the memory banks220using data received via the interconnect205from the SoC interface block or other DPEs110in the array.

The memory banks220can include any number of physical memory elements (e.g., SRAM). For example, the memory230may be include 4, 8, 16, 32, etc. different memory banks220. In this embodiment, the core210has a direct connection235to the memory banks220. Stated differently, the core210can write data to, or read data from, the memory banks220without using the interconnect205. That is, the direct connection235may be separate from the interconnect205. In one embodiment, one or more wires in the direct connection235communicatively couple the core210to a memory interface in the memory230which is in turn coupled to the memory banks220.

In one embodiment, the memory230also has direct connections240to cores in neighboring DPEs110. Put differently, a neighboring DPE in the array can read data from, or write data into, the memory banks220using the direct neighbor connections240without relying on their interconnects or the interconnect205shown inFIG.2. The HSC225can be used to govern or protect access to the memory banks220. In one embodiment, before the core210or a core in a neighboring DPE can read data from, or write data into, the memory banks220, the core (or the DMA circuit215) requests a lock acquire to the HSC225when it wants to read or write to the memory banks220(e.g., when the core/DMA circuit want to “own” a buffer, which is an assigned portion of the memory banks220. If the core or DMA circuit does not acquire the lock, the HSC225will stall (e.g., stop) the core or DMA circuit from accessing the memory banks220. When the core or DMA circuit is done with the buffer, they release the lock to the HSC225. In one embodiment, the HSC225synchronizes the DMA circuit215and core210in the same DPE110(e.g., memory banks220in one DPE110are shared between the DMA circuit215and the core210). Once the write is complete, the core (or the DMA circuit215) can release the lock which permits cores in neighboring DPEs to read the data.

FIG.3illustrates a field programmable gate array (FPGA) implementation of a programmable logic (PL) die300, according to an example. The PL die300includes configurable logic blocks (CLBs)310, random access memory blocks (BRAMs)320, digital signal processing blocks (DSPs)330, and interconnect340. In some embodiments, each CLB310includes one or more programmable interconnect elements (INTs)312and one or more configurable logic elements (CLEs)314that can be programmed to implement user logic. The PL die300may further include other components, such as input/output blocks (IOBs), analog-to-digital converters (ADCs), system monitoring logic, and so forth. AlthoughFIG.3illustrates the CLBs310, BRAMs320, and DSPs330arranged in columns and rows, any other configuration including any number of CLBs310, BRAMs320, and DSPs330may be implemented.

In some embodiments, each programmable interconnect element312includes connections to input and output terminals of a CLE314within the same CLB310. Each programmable interconnect element312can also include connections to adjacent programmable interconnect element(s)312and connections to general routing resources between logical blocks included in the PL die300. A BRAM320can include a BRAM logic element (BRL) and one or more programmable interconnect elements (not shown). A DSP320can include a DSP logic element (DSPL) in addition to an appropriate number of programmable interconnect elements.

In some embodiments, interconnect340may be configured as a horizontal area near the center of the PL die300and may be used for configuration, clock, and other control logic. The PL die300may further include additional logic blocks that disrupt the regular columnar structure making up a large part of the programmable logic. The additional logic blocks can be programmable blocks and/or dedicated logic.

Note thatFIG.3is intended to illustrate only an exemplary programmable logic architecture. For example. the numbers of logic blocks (e.g., CLBs310) in a column or row, the relative width of the columns and rows, the number and order of columns and rows, the types of logic blocks included in the columns or rows, the relative sizes of the logic blocks, and the interconnect/logic implementations included at the top ofFIG.3are exemplary.

Tiled Compute and Programmable Logic Array

Increasingly, high-performance computing systems implement large numbers of data processing engines and programmable logic (PL) (e.g., a field-programmable gate array or “FPGA”) within the same die and/or integrated circuit (IC) package. Such systems generally provide a flexible and highly parallel computing interface that can be adapted to a wide variety of applications. However, the architectures implemented in current systems suffer from a number of drawbacks.

For example, such systems commonly implement network-based communications, such as a network-on-chip (NoC) interface, in which data processing engines communicate with programmable logic and other IC components via an edge interface. For example, an array of data processing engines may be positioned along one edge of an interface (e.g., a NoC interface), and programmable logic may be positioned along another edge of the interface. One drawback of this configuration is that, as more and more processing elements need to communicate through an edge interface, the routing channels associated with the edge interface become saturated. As the routing channels approach saturation, routing congestion increases, limiting bandwidth and/or increasing latency between data processing engines and programmable logic. Additionally, due to routing congestion, data processing engines and programmable logic positioned far away from an edge of the interface may have difficulty meeting timing closure requirements, effectively limiting the total number of resources that can be utilized for a given process. Additionally, data processing engines and programmable logic positioned far away from an edge of the interface may have difficulty meeting timing closure requirements, effectively limiting the total number of resources that can be utilized for a given process.

In various embodiments, the tiled compute and programmable logic array techniques disclosed herein vertically align, in a three-dimensional die stack, data processing engines (e.g., DPEs110) included in a compute die with programmable elements (e.g., CLBs310) included in a programmable logic (PL) die300. Electrical connections (e.g., through-silicon vias) included on the bottom of the compute die may be pitch-matched and bonded to electrical connections included on the top of the PL die300, enabling orders of magnitude more connections and, as a result, higher bandwidth between the data processing engines and the programmable elements. In some embodiments, this high-bandwidth coupling to programmable logic fabric included in the PL die300enables compute memory (e.g., SRAM or UltraRAM, also referred to as “URAM”) included in each data processing engine to be distributed (e.g., cascaded) between multiple data processing engines, extending the amount of memory available for a given use case. Additionally, in some embodiments, each data processing engine (or tile of data processing engines) may be associated with substantially the same number and type(s) of programmable elements, enabling modular, “soft” intellectual property (IP) blocks to be “stamped” across the compute die and PL die300in a repeatable manner that generates predictable timing, bandwidth, and/or latency. Further, data processing engines (or tiles of data processing engines) may be connected to one another via the programmable logic fabric included in the PL die300in a specific topology, enabling, for example, advanced in-line processing, broadcasting, and other advanced functionality that cannot be efficiently performed via conventional systems that implement an edge interface. Such techniques are described below in further detail in conjunction withFIGS.3,4A-4B,5,6A-6B,7, and8A-8C.

FIG.4Aillustrates a schematic elevation view of a compute die400and PL die300, according to an example.FIG.4Billustrates a schematic elevation view of a three-dimensional (3D) die stack450that includes compute die400and PL die300, according to an example.

As shown inFIG.4A, compute die400includes a plurality of data processing engines410(e.g., DPEs110) and interconnect412. Interconnect412permits communication between the compute die400and the PL die300. In some embodiments, interconnect412may be positioned between two or more data processing engines410and may include TSVs, FIFOs, and/or level shifters for domain crossing. The compute die400may include regions of white space405where no circuitry is fabricated. Alternatively, in some embodiments, most or all of the compute die400may include circuitry, such as data processing engines410and interconnect412.

As shown inFIG.48, when the compute die400is stacked on top of the PL die300and the resulting 3D die stack450is viewed from above, each data processing engine410is electrically connected (e.g., at a die-to-die interface between the PL die300and compute die400) to substantially the same number of programmable elements (e.g., CLBs310). In various embodiments, the 3D die stack450includes a plurality of tiles, where each tile includes N total DPEs410(in the compute die400) that are electrically connected to M total CLBs210(in the PL die300). In some embodiments, each tile may include an integer number N total DPEs410that are electrically connected to an integer number M total CLBs210, where electrical connections of each DPE410are electrically connected (e.g., at a die-to-die interface between the PL die300and the compute die400) to electrical connections of the same number of CLBs210. For example, the 3D die stack450shown inFIG.4Bmay include 8 tiles, where each tile includes 6 DPEs410(e.g., 2×3 DPEs, 3×2 DPEs, 1×6 DPEs, 6×1 DPEs, etc.) that are electrically connected-at the die-to-die interface between the PL die300and compute die400—to 72 CLBs210(e.g., 2×36 CLBs, 36×2 CLBs, 1×72 CLBs, 72×1 CLBs, etc.). In another example, each tile may include 24 DPEs410(e.g., 6×4 DPEs, 4×6 DPEs, 3×8 DPEs, 8×3 DPEs, etc.) and 288 CLBs (e.g., 4×72 CLBs, 72×4 CLBs, etc.). In general, each tile may include any integer number M of CLBs210and any integer number N of DPEs410having any dimensions, such that M and N are the same for each tile in the 3D die stack450.

For clarity of illustration, white space405has been omitted fromFIG.4Bto enable components sitting below the compute die400to be more easily viewed. In some embodiments, each data processing engine410(or tile of data processing engines410) and the programmable elements with which it is vertically aligned and/or electrically connected form a module that operates in a similar manner to generate predictable timing, bandwidth, and/or latency. Although the PL die300shown inFIG.4Ahas a uniform, uninterrupted structure of CLBs310and, for example, does not include any DSPs330, in some embodiments, DSPs330and/or other elements may be included in the PL die300(e.g., as shown inFIG.3) and/or compute die400while still maintaining a substantially uniform allocation of programmable elements to each data processing engine410.

In various embodiments, interconnect412included in the compute die400may optionally be vertically aligned with an interconnect (e.g., programmable interconnect elements312) included in the PL die300such that electrical connections can be more easily formed in a z-direction between the compute die400and the PL die300. In some embodiments, the compute die400and the PL die300are electrically connected by hybrid oxide bonding through-silicon vias (TSVs) included on a bottom side of the compute die400to electrical connections (e.g., one or more metallization layers) included on a top side of the PL die300. For example, as shown inFIG.5, multiple input and output connections (e.g., 32 input and 32 output connections) may be formed in the z-direction between each programmable interconnect element312included in the PL die300and a data processing engine410and/or interconnect412included in compute die400. In some embodiments, each data processing engine410is directly coupled to substantially the same number of programmable interconnect elements312, where directly coupled means that a TSV (or similar connection) of the data processing engine410on the bottom of the compute die400couples to a programmable interconnect element312without passing through any intermediate logic (e.g., another programmable interconnect element312, a different CLB310, etc.) in PL die300. AlthoughFIG.5illustrates a CLE314and a DSP330positioned adjacent to the programmable interconnect elements312, in various embodiments, any type(s) of component (e.g., two CLEs314, two DSPs330, etc.) may be implemented in conjunction with the programmable interconnect elements312.

FIG.6Aillustrates a schematic elevation view of a compute die400and PL die300, according to an example.FIG.6Billustrates a schematic elevation view of a three-dimensional (3D) die stack650in which an interconnect412included in compute die400does not vertically align with programmable interconnect elements312included in PL die300, according to an example. In some embodiments, TSVs disposed on the bottom side of the compute die400are aligned to the interconnects412disposed on the upper side of the PL die300, enabling communication between the compute die400and the PL die300. Techniques for electrically connecting the compute die400and the PL die300when the interconnect(s)412included in compute die400do not vertically align with the programmable interconnect element(s)312included in PL die300are described below in further detail in conjunction withFIG.7.

As shown inFIG.6B, when the compute die400is stacked on top of the PL die300to form 3D die stack650, each data processing engine410is vertically aligned with substantially the same number of programmable elements (e.g., CLBs310). Similar toFIG.4B, for clarity of illustration, white space405has been omitted fromFIG.6Bto enable components sitting below the compute die400to be more easily viewed. Although the PL die300shown inFIG.6Ahas a uniform, uninterrupted structure of CLBs310and, for example, does not include any DSPs330, in some embodiments, DSPs330and/or other elements may be included in the PL die300(e.g., as shown inFIG.3) and/or compute die400while still maintaining a substantially uniform allocation of programmable elements to each data processing engine410.

In various embodiments, interconnect412included in the compute die400is not vertically aligned with (or is only partially vertically aligned with) an interconnect (e.g., programmable interconnect elements312) included in the PL die300. In such embodiments, electrical connections can be routed in the x-direction and/or y-direction via one or more metal layers on and/or within the compute die400to enable interconnect(s)412to be vertically connected (e.g., via TSVs disposed at the die-to-die interface on the bottom side of the compute die400) to interconnect(s) included in the PL die300. For example, as shown inFIG.7, a bottom side of the compute die400may include z-interface cells710, and metal tracks712may be fabricated between the z-interface cells710and an interconnect (e.g., programmable interconnect elements312) included in the PL die300. Accordingly, such embodiments provide additional flexibility with respect to the vertical alignment between components included in the PL die300and compute die400.

FIGS.8A-8Cillustrate techniques for programming a 3D die stack450,650, according to an example. For example,FIG.8Aillustrates a technique for high-bandwidth distributed compute memory810, according to an example. Conventionally, each data processing engine410may include a fixed amount of high-speed memory810(e.g., SRAM, URAM, etc.) built into its array. Because the memory810is not extendable, use cases that require more than the fixed amount of memory810generally can access memory810only in adjacent arrays that are in close proximity to the data processing engine410(or tile of data processing engines410) due to timing closure requirements. Alternatively, the fixed amount of memory810included in each data processing engine410can be increased to support the use case(s), which may significantly increase die area requirements. Accordingly, in various embodiments, the high-bandwidth coupling between the compute die400and the programmable logic fabric included in the PL die300enables the fixed memory810included in each data processing engine to be distributed (e.g., cascaded) between multiple data processing engines, significantly extending the amount of memory810available for a given use case. Additionally, bit depth and/or bit width may be user-programmable, providing flexibility for a wide range of applications. In some embodiments, larger RAM modules included in the compute die400may be interleaved in order to improve memory speed.

As shown inFIG.8B, functions that do not target well to a particular compute architecture (e.g., integer operations, trigonometry functions, etc.) and/or operations for which there is no fixed-function hardware may be synthesized directly underneath a data processing engine410(or a tile of data processing engines410) in the programmable logic fabric of the PL die300. For example, as shown inFIG.8B, a tile of nine data processing engines410and resources included in PL die300(e.g., CLBs310, BRAM320, DSPs330, etc.) underlying the nine data processing engines410may be implemented to execute each instance of function820, and a tile of ten data processing engines410and resources included in PL die300underlying the ten data processing engines410may be implemented to execute an instance of function822. As another example, a tile of X data processing engines410and resources included in PL die300underlying the X data processing engines410may be implemented to execute any function that is not ubiquitous enough and/or not used frequently enough to warrant using the silicon area of every tile. In such implementations, the programmable logic fabric included in the PL die300may serve as a coprocessor to the data processing engines410included in the compute die400, enabling a modular, fully-customizable function (e.g., ReLU/sigmoid, encryption/decryption, search, compression, etc.) to be tiled multiple times across a compute die with predictable and repeatable results. Such configurations may also enable variable precision copies of fixed-precision data processing engine410functions and user-programmed instruction set extensions to be created.

As shown inFIG.8C, customized compute interconnects can be generated to connect data processing engines410to each other in a specific topology via resources included in PL die300(e.g., CLBs310, BRAM320, DSPs330, etc.). The compute interconnects may be lower latency than a conventional 2D, edge interface. The interconnects may enable, for example, dedicated point-to-point connections between data processing engines410and complex compute interconnect topologies (e.g., torus, hypercube, fat tree, etc.). Additionally, the customized compute interconnects may enable in-line processing to be performed along a path830of data processing engines410, broadcasting in a fully-connected layer, and other advanced functionality that cannot be efficiently performed via conventional systems that implement an edge interface.