Fast memory for programmable devices

An integrated circuit device may include a programmable fabric die having programmable logic fabric and configuration memory that may configure the programmable logic fabric. The integrated circuit device may also include a base die that may provide fabric support circuitry, including memory and/or communication interfaces as well as compute elements that may also be application-specific. The memory in the base die may be directly accessed by the programmable fabric die using a low-latency, high capacity, and high bandwidth interface.

BACKGROUND

This disclosure relates to interfaces for transfer of data in a multi-dimensional programmable logic device.

Programmable logic devices are a class of integrated circuits that can be programmed to perform a wide variety of operations. To that end, programmable logic devices may include circuitry for sending and receiving data. For example, a programmable logic device may include programmable logic elements programmed by a form of memory known as configuration random access memory (CRAM). To program a circuit design into a programmable logic device, the circuit design, which may be compiled into a bitstream, is transmitted and loaded into CRAM cells. Once programmed (e.g., with the bitstream), the programmable logic device may perform operations associated with the circuit design. Operations may, among other things, include data transmission and/or data reception. As such, programmable logic devices may perform operations (e.g., configuration operations, logic operations) that may include data exchange.

The highly flexible nature of programmable logic devices makes them an excellent fit for accelerating many computing tasks. Thus, programmable logic devices are increasingly used as accelerators for networking, storage, data center systems, communications, mobile applications, machine learning, video processing, voice recognition, image recognition, and many other highly specialized tasks, particularly those that would be too slow or inefficient in software running on a processor. As the computing tasks performed by the programmable logic devices become more complex, more flexible and faster interfaces for data exchange processes may be of benefit.

DETAILED DESCRIPTION OF SPECIFIC EMBODIMENTS

The highly flexible nature of programmable logic devices makes them an excellent fit for accelerating many computing tasks. Programmable logic devices are devices that may include customizable and reprogrammable circuitry that can perform digital operations and/or logic functions. To that end, programmable logic devices may be programmed by loading configuration data into configuration memory (e.g., configuration random access memory (CRAM)) that may be embedded in the programmable fabric. The configuration memory may store a logic design (e.g., state machines, truth tables, functions, etc.) that may control configurable logic circuitry to facilitate performance of the programmed tasks. The flexibility in the operations of programmable logic devices also allows reprogramming of the configuration memory (e.g., programming a portion of a circuit design). For example, a system using programmable logic devices may change context (e.g., change the type of operation performed) by loading new configuration data to the configuration memory. Due to the flexibility afforded by the customizable and reconfigurable design, programmable logic devices are increasingly used as accelerators for machine learning, video processing, voice recognition, image recognition, and many other highly specialized tasks, particularly those that would be too slow or inefficient in software running on a processor. As the complexity of the tasks increases, the dimensions of the configuration data may also increase and may benefit from a high-speed interface for loading configuration memory.

As discussed above, programmable circuitry in the programmable logic device (e.g., configuration memory, programmable logic elements, and embedded memory) may send and receive data (e.g., configuration data, user data, incoming data for processing by the programmable logic elements, data processed by the programmable logic data elements). In order to exchange data with external circuitry or other programmable circuitry in the programmable logic device, the device may include high-speed interfaces. A high-speed interface may be used to increase the speed of the programming and/or reprogramming operations, which may reduce the amount of idle time during which the reprogramming tasks is taking place. The high-speed interface may also be used to increase the speed of data transfer from the programmable logic elements, to facilitate data processing operations. Moreover, programmable logic devices may also have user memory that may be directly accessed by the interface. Direct access to the user memory may facilitate diagnostic operations, such as debugging, testing, or emulation of a system design. The direct access further may provide faster access to the user memory to increase the overall speed of diagnostic tasks.

In some embodiments, the programmable logic device may utilize one or more die, such as a programmable logic (or fabric) die having a fabric of programmable logic elements and a base die having fabric support circuitry, in a three-dimensional arrangement. In some systems, the programmable logic die may be sectorized, as detailed below. In such systems, the fabric support circuitry in the base die may include network on chip (NOC) circuitry to send or receive data (e.g., configuration data, user data) with systems external to the programmable logic device and/or between sectors in the programmable logic devices. The fabric support circuitry may also include sector-aligned memory. In some embodiments, the sector-aligned memory may operate as a temporary storage (e.g., cache) for the configuration data or user memory. By incorporating the NOC into the fabric support circuitry, the NOC may resolve periphery shoreline bandwidth issues of the fabric, while increasing the bandwidth of the fabric. In addition, the communication routes available via the NOC embedded in the fabric support circuitry may enable the fabric to implement design relocations or reconfigurations, provide alternate pathways around powered-down sectors of the fabric, and provide security isolation features. The NOC may be a source of configuration data or fabric data and may be integrated to access the multi-purpose high-speed interface. With the foregoing in mind, the embodiments described herein are related to the use of sector-aligned memory to increase the speed and the capacity of low-latency memory for programmable logic applications. Aggregation of bandwidth between the programmable die and the base die is also discussed.

In addition to the above-described features, the fabric support circuitry may include, among other things, a device controller (sometimes referred to as a secure device manager (SDM)), a sector controller (sometimes referred to as a local sector manager (LSM), region controller), a configuration network on chip (CNOC), data routing circuitry, local (e.g., sectorized, sector-aligned, region-aligned) memory used to store and/or cache configuration programs (bitstreams) or data, memory controllers used to program the programmable logic fabric, input/output (I/O) interfaces or modules for the programmable logic fabric, external memory interfaces (e.g., for a high bandwidth memory (HBM) device), an embedded processor (e.g., an embedded INTEL® XEON® processor by Intel Corporation of Santa Clara, Calif.) or an interface to connect to a processor (e.g., an interface to an INTEL® XEON® processor by Intel Corporation of Santa Clara, Calif.), voltage control circuitry, thermal monitoring circuitry, decoupling capacitors, power clamps, or electrostatic discharge circuitry, to name just a few circuit elements that may be present on the second die.

By way of introduction,FIG. 1illustrates a block diagram of a system10that may employ a programmable logic device12that can communicate via a NOC disposed on a separate die (e.g., base die), in accordance with embodiments presented herein. Using the system10, a designer may implement a circuit design functionality on an integrated circuit, such as a reconfigurable programmable logic device12, such as a field programmable gate array (FPGA).

The designer may implement a circuit design to be programmed onto the programmable logic device12using a design description14. The design description14may include a register-transfer level (RTL) design abstraction with constraints. Additionally or alternatively, the design description14may include high-level programming languages that may be converted to the a lower-level design description. For instance, the design description12may be specified using, OpenCL, C, data parallel C++ (DPC++), and the like. The design descriptions14may be submitted to a compiler16, such as a version of INTEL® QUARTUS® by Intel Corporation of Santa Clara, Calif., to generate a low-level circuit-design defined by a bitstream18, sometimes known as a program object file and/or configuration program, which programs the programmable logic device12. Thus, the compiler16may provide machine-readable instructions representative of the circuit design to the programmable logic device12. For example, the programmable logic device12may receive one or more configuration programs (bitstreams)18that describe the hardware implementations that should be stored in the programmable logic device12. A configuration program (e.g., bitstream)18may be programmed into the programmable logic device12as a configuration program20. The configuration program20may, in some cases, represent an accelerator function to perform machine learning functions, video processing functions, voice recognition functions, image recognition functions, networking functions, or other highly specialized task.

To carry out the systems and methods of this disclosure, the programmable logic device12may take any suitable form that includes the multi-purpose high-speed parallel interface, which increases the speed of exchange of fabric data and/or configuration data across different portions (e.g., sectors, multiple die) of the programmable logic device12. The multi-purpose parallel interface may also allow reconfiguration of portions of the programmable logic device12while concurrently operating a circuit design by allowing concurrent exchange of fabric data and configuration data through distinct microbump channels. As such, in one embodiment, the programmable logic device12may have two separate integrated circuit die coupled via the multi-purpose parallel interface. The integrated circuit die may include controllers for the multi-purpose parallel interface, which may be hard coded circuitry, a soft IP block, and/or custom logic.

One example of the programmable logic device12is shown inFIG. 2, but any suitable programmable logic device may be used. In the example ofFIG. 2, the programmable logic device12includes fabric die22and respective base die24that are connected to one another via microbumps26. Although microbumps are discussed throughout, any bonding techniques that are suitable for coupling the fabric die22and the base die24together may be used. Furthermore, although the microbumps26are located in theFIG. 2in a particular location (e.g., at the edge), the microbumps26may be located at any suitable location. The microbumps26may couple an interface in the fabric die22(e.g., a fabric or FPGA microbump interface (FMIB)) to an interface in the base die24(e.g., a base microbump interface (BMIB)), as detailed below. In the illustrated diagram ofFIG. 2, the fabric die22A and base die24A are illustrated in a one-to-one relationship and in an arrangement in which a single base die24B may attach to several fabric die22B and22C. Other arrangements, such as an arrangement in which several base die24may attach to a single fabric die22, or several base die24may attach to several fabric die22(e.g., in an interleaved pattern along the x- and/or y-direction) may also be used. Peripheral circuitry28may be attached to, embedded within, and/or disposed on top of the base die24. The base die24may attach to a package substrate32via bumps34. The bumps34may include controlled collapse chip connection (C4) bumps. The base die24may include one or more through-silicon vias (TSVs)35that enable the fabric die22to couple to the bumps34via the microbumps26. In the example ofFIG. 2, two pairs of fabric die22and base die24are shown communicatively connected to one another via a silicon bridge36(e.g., an embedded multi-die interconnect bridge (EMIB) and microbumps38at a silicon bridge interface39.

Although the microbumps26and the microbumps38are described as being employed between the fabric die22and the base die24or between the edge devices, such as the silicon bridge36and the silicon bridge interface39, it should be noted that microbumps may be employed at any suitable position between the components of the programmable logic device12. For example, the microbumps may be incorporated in any suitable position (e.g., middle, edge, diagonal) between the fabric die22and the base die24. In the same manner, the microbumps may be incorporated in any suitable pattern or amorphous shape to facilitate interconnectivity between various components (e.g., NOC) described herein.

In combination, the fabric die22and base die24may operate as a programmable logic device such as a field programmable gate array (FPGA) device. For example, the fabric die22and the base die24may operate in combination as an FPGA40, shown inFIG. 3. It should be understood that the FPGA40shown inFIG. 3is meant to represent the type of circuitry and/or a logical arrangement of a programmable logic device when both the fabric die22and the base die24operate in combination. That is, some of the circuitry of the FPGA40shown inFIG. 3may be found in the fabric die22and some of the circuitry of the FPGA40shown inFIG. 3may be found in the base die24. Moreover, for the purposes of this example, the FPGA40is referred to as an FPGA, though it should be understood that the device may be any suitable type of programmable logic device (e.g., an application-specific integrated circuit and/or application-specific standard product).

In the example ofFIG. 3, the FPGA40may include transceiver circuitry44for driving signals off of the FPGA40and for receiving signals from other devices. The transceiver circuitry44may be part of the fabric die22, the base die24, or a separate die altogether. Interconnection resources46may be used to route signals, such as clock or data signals, through the FPGA40. The FPGA40ofFIG. 3is shown to be sectorized, meaning that programmable logic resources may be distributed through a number of discrete programmable logic sectors48(e.g., region, portion).

Each programmable logic sector48may include a number of programmable logic elements50(also referred herein as FPGA fabric) having operations defined by configuration memory52(e.g., configuration random access memory (CRAM)). The programmable logic elements50may include combinatorial or sequential logic circuitry. For example, the programmable logic elements50may include look-up tables, registers, multiplexers, routing wires, and so forth. A designer may program the programmable logic elements50to perform a variety of desired functions. The programmable logic sector48may also include user memory53. User memory may be in the form of embedded random access memory (ERAM), and/or memory blocks, such as M20K. A power supply54may provide a source of voltage and current to a power distribution network (PDN)56that distributes electrical power to the various components of the FPGA40. Operating the circuitry of the FPGA40causes power to be drawn from the power distribution network56.

There may be any suitable number of programmable logic sectors48on the FPGA40. Indeed, while the illustrated system includes 29 programmable logic sectors48shown, it should be appreciated that more or fewer may appear in an actual implementation (e.g., in some cases, on the order of 50, 100, or 1000 sectors or more). Each programmable logic sector48may include a sector controller (SC)58that controls the operation of the programmable logic sector48. Each sector controller58may be in communication with a device controller (DC)60. Each sector controller58may accept commands and data from the device controller60and may read data from and write data into its configuration memory52or user memory53based on control signals from the device controller60. To that end and as detailed below, the device controller60may employ a data register (DR) and/or an address register (AR) to access data from the configuration memory52or user memory53of the various programmable logic sectors48.

In addition to these operations, the sector controller58and/or device controller60may be augmented with additional capabilities. As described herein, a high-speed parallel interface may be used to coordinate memory transactions between local in-fabric memory (e.g., local fabric memory or CRAM being used for data storage) and sector-aligned memory associated with that particular programmable logic sector48. Moreover, the NOC may be used to facilitate memory transactions between multiple sectors, multiple die, and/or between the programmable logic device and external systems, as discussed herein. The NOC may further be employed for decrypting configuration data (bitstreams)18, for locally sequencing reads and writes to implement error detection and correction on the configuration memory52or user memory53, and sequencing test control signals to effect various test modes.

The sector controllers58and the device controller60may be implemented as state machines and/or processors. For example, each operation of the sector controllers58or the device controller60may be implemented as a separate routine in a memory containing a control program. This control program memory may be fixed in a read-only memory (ROM) or stored in a writable memory, such as random-access memory (RAM). The ROM implementation may use more memory than would be used to store only one copy of each routine. This additional memory may allow each routine to have multiple variants depending on “modes,” and the local controller may be placed into any of those modes. When the control program memory is implemented as random access memory (RAM), the RAM may be written with new routines to implement new operations and functionality into the programmable logic sectors48. This may provide usable extensibility in an efficient and easily understood way. This may be useful because new commands could bring about large amounts of local activity within the sector at the expense of only a small amount of communication between the device controller60and the sector controllers58.

The sector controller58may include circuitry to manage the high-speed parallel interface (e.g., DR Intercept interface (DRIP)). The high-speed parallel interface may allow fast loading of DR circuitry, which may be used for configuration operations (e.g., CRAM operations), user memory operations (e.g., ERAM operations), and/or test (e.g., scan chains operations). For example, a high-speed interface controller (e.g., DRIP Controller), located in a fabric die, and a base die DRIP Controller, located in a base die, may be used to coordinate operations in the microbump interface, and may be part of the sector controller58. The DRIP controllers and the NOC circuitry may also coordinate operations to perform multi-sector high-speed data exchange between the base die24and a respective fabric die22. Each sector controller58thus may communicate with the device controller60, which may coordinate the operations of the sector controllers58and convey commands initiated from outside the FPGA40. To support this communication, the interconnection resources46may act as a network between the device controller60and each sector controller58. The interconnection resources may support a wide variety of signals between the device controller60and each sector controller58. In one example, these signals may be transmitted as communication packets.

The FPGA40may be electrically programmed. With electrical programming arrangements, the programmable logic elements50may include one or more logic elements (wires, gates, registers, etc.). For example, during programming, configuration data is loaded into the configuration memory52using transceiver circuitry44and input/output circuitry42. In one example, the configuration memory52may be implemented as configuration random-access-memory (CRAM) cells. The use of configuration memory52based on RAM technology is described herein is intended to be only one example. Moreover, configuration memory52may be distributed (e.g., as RAM cells) throughout the various programmable logic sectors48the FPGA40. The configuration memory52may provide a corresponding static control output signal that controls the state of an associated programmable logic element50or programmable component of the interconnection resources46. The output signals of the configuration memory52may be applied to configure the gates of metal-oxide-semiconductor (MOS) transistors that control the states of the programmable logic elements50or programmable components of the interconnection resources46.

As stated above, the logical arrangement of the FPGA40shown inFIG. 3may result from a combination of the fabric die22and base die24. The circuitry of the fabric die22and base die24may be divided in any suitable manner. In one example, shown in block diagram form inFIG. 4, the fabric die22contains primarily programmable logic fabric resources, such as the programmable logic elements50, configuration memory52, and user memory53, and may be arranged in sectors such as sectors65A and65B. In some cases, this may also entail certain fabric control circuitry such as the sector controller (SC)58or device controller (DC)60. The base die24may include supporting circuitry to operate the programmable logic elements50, configuration memory52, and user memory53. As an example, the programmable logic elements50may exchange fabric data with the supporting circuitry in the base die24and configuration memory may exchange configuration data with the supporting circuitry in the base die24. As shown here, the base die24includes support circuitry70A, which may support fabric sector65A, and support circuitry70B, which may support fabric sector65B. The base die24may also include support circuitry for other sectors of the fabric die22.

As discussed above, the multi-purpose interface may benefit from the presence of NOC circuitry in the base die (e.g., base die24). The block diagrams inFIGS. 5, 6, 7, and8illustrate examples of physical arrangements of the fabric die22and the base die24that may implement a NOC system. For example, a physical arrangement of the fabric die22and the base die24is shown byFIGS. 5 and 6. InFIG. 5, the fabric die22is shown to contain an array of fabric sectors80that include fabric resources82(e.g., programmable logic elements programmed by CRAM and/or certain fabric control circuitry such as the sector controller (SC)58or device controller (DC)60) and interface circuitry84. The interface circuitry84may include data routing and/or clocking resources or may include an interface to data routing and/or clocking resources on the base die24. Thus, the interface circuitry84may connect with a microbump interface to connect to the base die24.

FIG. 6provides an example complementary arrangement of the base die24. The base die24may represent an active interposer with several sectors90surrounded by peripheral circuitry28and the silicon bridge interface39. Although not shown inFIG. 6, each sector90may include a variety of fabric support circuitry, which may described in greater detail below. In any case, the base die24, in some embodiments, may include data and/or configuration routers98, and/or data or configuration pathways99. In some embodiments, portions of the data or configuration pathways99may communicate data in one direction, while other portions may communicate data in the opposite direction. In other embodiments, the data or configuration pathways99may communicate data bi-directionally.

With the foregoing in mind, the data and/or configuration pathways99may make up a network on chip (NOC)100. In the embodiment depicted inFIG. 6, the NOC100may be integrated between each sector90of the base die24. As such, the NOC100may enable each of the sectors90disposed on the base die24to be accessible to each other. Indeed, the NOC100may provide communication paths between each sector90via routers98or the like. In certain embodiments, the routers98may route user data between sectors90of the base die24, to sectors48of the fabric die22, and the like. Since the base die24is separate from the fabric die22, the NOC100may be continuously powered on, even when various sectors48of the fabric die22are powered down. In this way, the NOC100of the base die24may provide an available route to different sectors48of the fabric die22regardless of the positions of powered down sectors48.

In some embodiments, the NOC100may include features such as Quality of Service management, Security Management, Debug and Performance measurement and Address virtualization services, and the like. In addition, the NOC100may support caching features and interconnect protocols allowing the memory components of the programmable logic device12to be part of a coherent memory system supported by a caching agent.

By vertically aligning the fabric die22and the base die24, the NOC100disposed on the base die24may physically span across the same surface area of the fabric die22. In certain embodiments, microbumps may be positioned at various locations between the base die24and the fabric die22to enable the NOC100to communicate data between sectors90of the base die and sectors48of the fabric die22. In the example embodiment of the NOC100depicted inFIG. 6, the NOC100may be positioned around each sector90, which may be aligned with a corresponding sector48of the fabric die22. As such, the NOC100may provide additional horizontal and vertical routing wires or pathways to facilitate communication between sectors48of the fabric die22, between sectors90of the base die24, or between sectors48of the fabric die22and sectors90of the base die24. The additional horizontal and vertical lines provided by the NOC100may reduce the amount of quantization performed by the programmable logic device12.

Although data or configuration pathways99of the NOC100are illustrated inFIG. 6as being routed around the sectors90of the base die24, it should be noted that data or configuration pathways99of the NOC100may be routed across the base die24in any suitable manner. By way of example,FIG. 7illustrates the NOC100implemented with data or configuration pathways99disposed across the center of the sector90. As such, the router98may also be positioned in the center of the sector90.

In addition, in some embodiments, the data or configuration pathways99of the NOC100may be wider in certain sectors90as compared to other sectors90. In any case, it should be understood that the data or configuration pathways99may be incorporated into the base die24, such that they serve desired functions or operations that may be specific to the operations performed by the programmable logic device12. That is, if the programmable logic device12includes functions that involve regularly transferring data across the fabric die22, it may be beneficial to use more of the base die24space to include data or configuration pathways99that take up more space on the base die24to provide increased bandwidth.

With the foregoing in mind, the NOC100may include the data or configuration pathways99that allow for efficient multi-dimensional (e.g., three-dimensional, two-dimensional) integration of the NOC100on the programmable logic device12. Indeed, the NOC100may enable the fabric die22to connect to the peripheral circuitry28and other parts of the fabric without communicating via the fabric die22or the programmable logic sectors48of the fabric die22. That is, by employing the NOC100in the base die24, the programmable logic device12may increase the available bandwidth for communication across the programmable logic sectors48because the NOC100provides additional pathways to different parts of the programmable logic device12.

In addition, the NOC100resolves shoreline issues, supports fast reconfiguration, and enables relocate-ability of functions in the programmable logic sectors48based on the increased bandwidth and accessibility to different portions of the fabric die22via the base die24. In some embodiments, the portions may include or be equal to the sectors48. However, in certain embodiments, the portions may include embedded processors, such as an INTEL® XEON® processor or a reduced-instruction processor. Furthermore, the relocate-ability of the functions may be at least partially attributable to the regularity of the sectors48. In other words, at least some portion of the sectors may be similarly arranged that enables movement of functions between sectors48readily since the two sectors for the function are similar due to a limited number of types of sectors48.

As shown inFIG. 9, the NOC100may be integrated with the sector-aligned memory92of the base die24, such that it spans across the entire base die24. As such, the NOC100may access different fabric sectors80through various routes in the base die24. In addition, the additional routes enable the NOC100to serve as functional bypass around powered down sectors80without affecting the performance of the programmable logic device12by avoiding blockages across the programmable logic sectors48of the programmable logic device12. That is, in some situations, certain sectors80of the fabric die22may be powered down, thereby preventing communication across the powered down sectors80. In this case, the NOC100may provide alternate communication pathways around the powered down sectors80to maintain communicative connectedness across the sectors80of the fabric die22regardless of whether certain sectors80are powered down.

The design relocate-ability of the programmable logic sectors48is also enhanced by the NOC100because the NOC100may access different sectors80of the fabric die22. That is, the increased communication flexibility provided by the NOC100being disposed in the base die24enables the programmable logic sectors48to be repositioned in various sectors80of the fabric die22, while maintaining communication capabilities between the relocated programmable logic sectors48.

AlthoughFIGS. 6 and 7illustrate two embodiments with different configurations for the NOC100, it should be noted that the base die24may be configured to include a data or configuration pathways99in a variety of shapes, forms, positions, and the like. For example, the data or configuration pathways99of different sectors90may overlap each other, the entire sector90may incorporate the data or configuration pathway99, or the like. In addition, microbumps may be used to facilitate communication between the NOC100and various sectors80of the fabric die22and the like.

In addition to facilitating communication of data between sectors90, sectors80, and the like, the presence of the NOC100in the base die24may also enable the programmable logic device12to incorporate additional circuit features by leveraging the NOC100of the base die24to improve the communication across the fabric die22. By way of example,FIG. 8provides another embodiment of an arrangement of the base die24. Similar to the base die24described above, the base die24ofFIG. 8may represent an active interposer with several sectors90surrounded by peripheral circuitry28and the silicon bridge interface39. As shown inFIG. 8, each sector90may include a variety of fabric support circuitry, such as sector-aligned memory92, memory control circuitry93, non-user input control circuitry94, non-user output control circuitry95, a voltage regulator such as a fully integrated voltage regulator (FIVR)96, one or more sensors97(e.g., thermal, voltage, and the like), data and configuration routers98, and/or data or configuration pathways99.

Although the following description of the additional circuit features enabled by the NOC100embedded in the base die24focuses on the ability to transfer data to and from the sector-aligned memory92, it should be noted that the additional circuit features are not limited to technologies involving the sector-aligned memory92. Indeed, the NOC100embedded in the base die24may enable a variety of circuit operations to be performed more efficiently and effectively via the programmable logic device12. However, for the purposes of discussion, certain operations that involve the sector-aligned memory92and the NOC100will be described to illustrate some of the functions enabled by incorporating the NOC100into the base die24.

Referring back toFIG. 8, the memory control circuitry93may be used to program the sector-aligned memory92, the CRAM of the fabric die22, or both. The non-user input control circuitry94and non-user output control circuitry95may allow the circuitry of the sectors90to exchange data and/or control signals (e.g., via configurable data routing network-on-chip (NOC) or a configuration network on chip (CNOC)). In one example, the non-user input control circuitry94and non-user output control circuitry95may operate as the sector controller (SC)58for a corresponding fabric sector80(as shown inFIG. 5).

The FIVR96and the one or more thermal sensors97may be used to provide a desired voltage to the corresponding fabric sector80(as shown inFIG. 5), enabling the voltage to be selectively scaled up or down, or removed, depending on power and thermal specifications (e.g., based at least in part on temperature as measured by a thermal sensor97and/or in accordance with a dynamic voltage and frequency scaling (DVFS) scheme). Even though the thermal sensors97are in a separate die from that of the programmable logic fabric elements, when the base die24is directly adjacent to the fabric die22as in this example, the temperature measured by the thermal sensor97in the base die24may correspond well enough to the fabric die22to allow for temperature-based operations (e.g., turn off power to the corresponding fabric sector80to prevent a permanent-denial-of-service (PDOS) condition).

In certain embodiments, the data or configuration pathways99that make up the NOC100may provide communication paths between each sector90via routers98mentioned above. As shown inFIG. 9, by vertically aligning the fabric die22and the base die24and incorporating the NOC100in the base die24, memory located in the base die24may be accessible in parallel to fabric sectors80of the fabric die22.FIG. 9shows an example in which sector-aligned memory92may be contained in the base die24. The sector-aligned memory92may be directly accessible to respective fabric sectors80of the fabric die22and may contain user data (generated by or accessible by a circuit design programmed into the programmable logic fabric of the base die24) or configuration data that may be used to program the programmable logic fabric of the respective fabric die22. In this disclosure, “directly accessible” refers to a connection between a particular fabric sector80and a region of the sector-aligned memory92that is associated with the particular fabric sector80. In some embodiments, each respective region of the sector-aligned memory92associated with a particular fabric sector80may be directly accessible to that particular fabric sector80, thereby providing each fabric sector80with direct access to respective regions of the sector-aligned memory92. For example, there may be N regions of sector-aligned memory92that can be accessible by N corresponding fabric sectors80at the same time (e.g., in parallel). In some cases, the sector-aligned memory92may be accessible to more than one fabric sector80or multiple sectors of sector-aligned memory92may be accessible to a single fabric sector80. Thus, in some cases, the same region of sector-aligned memory92may be directly accessible to multiple fabric sectors80, while in other cases, a region of sector-aligned memory92may be directly accessible only to a single fabric sector80. In the example ofFIG. 9, the fabric sectors80may access specific regions of sector-aligned memory92. The sector-aligned memory92is shown inFIG. 9as vertically stacked memory. This may allow a large amount of memory to be located within the base die24. However, the sector-aligned memory92may occupy a single plane of the base die24in some embodiments.

It should be noted that the fabric sectors80may initiate a transfer of data directly between memory locations of the sector-aligned memory92of the base die24using the NOC100, between different fabric sectors80of the fabric die22, between fabric sectors80and memory locations of the sector-aligned memory92, and the like. In certain embodiments, the sector controller (SC)58may initiate the transfer of data between sectors80of the fabric die22, between memory locations of the sector-aligned memory92, between sectors80of the fabric die22and memory locations of the sector-aligned memory92, and the like. That is, the sector controller (SC)58may coordinate the operation of the NOC100to facilitate the transfer of the data between the source and destination targets, as specified by the section controller (SC)58. In some embodiments, the section controller (SC)58may act as a master to initiate the respective transfer and the transfer would then be performed using the NOC100after the section controller (SC)58initiates the data transfer process.

By way of example,FIG. 10includes a block diagram illustrating the transfer of data between sectors80of the programmable logic fabric via the NOC100of the base die24. Referring toFIG. 10, in one embodiment, the sector controller (SC)58may initiate a transfer of data from sector “A” to sector “B” of the fabric die22using the NOC100of the base die24. That is, the sector controller (SC)58may transfer data to a region of the sector-aligned memory92aligned with sector “A” of the fabric die22, use the NOC100to transfer the data to a second region of the sector-aligned memory92aligned with sector “B” of the fabric die22, and transfer the data from the second region of the sector-aligned memory92to sector “B” of the fabric die22. Although the route of the data transfer illustrated inFIG. 10corresponds to straight paths, it should be noted that the data transferred to different sectors80of the fabric die22or regions of the sector-aligned memory92may use a variety of directions and routes.

In another example,FIG. 11includes a block diagram illustrating the transfer of data from a sector80of the programmable logic fabric to a region of the sector-aligned memory92via the NOC100of the base die24. Referring toFIG. 11, in one embodiment, the sector controller (SC)58may initiate a transfer of data from sector “A” of the fabric die22to region “C” of the sector-aligned memory92using the NOC100of the base die24. That is, the sector controller (SC)58may transfer data to a first region of the sector-aligned memory92aligned with sector “A” of the fabric die22and use the NOC100to transfer the data to region “C” of the sector-aligned memory92via different regions of the sector-aligned memory92or the like. LikeFIG. 10, although the route of the data transfer illustrated inFIG. 11corresponds to straight paths, it should be noted that the data transferred to different regions of the sector-aligned memory92may use a variety of directions and routes. It should be noted that other suitable circuits may also be employed to transfer the data in accordance with the embodiments presented herein.

As shown inFIGS. 10 and 11, the sector controller (SC)58may initiate a transfer of data directly between memory locations within the base die24using the NOC100. In this case, the sector controller (SC)58may act as the master to initiate the transfer, but then the transfers would be performed directly in the sector-aligned memory92and the NOC100of the base die24after the sector controller (SC)58initiates the transfer. It should also be mentioned that, in some embodiments, that the sector controller (SC)58and similar components of the fabric die22may also initiate the components (e.g., memory control circuitry93, non-user input control circuitry94, non-user output control circuitry95) of the base die24to perform transfers between the sector-aligned memory92, the peripheral circuitry28, and other components attached to the base die. As a result, data transfers may occur in the base die24without involvement of components in the fabric die22.

In certain embodiments, the NOC100may also enable the programmable logic device12to provide security isolation for one or more of the programmable logic sectors48. That is, the NOC100may be employed to communicate certain sensitive or secure data to a portion of programmable logic sectors48that may be designated as a security portion of the programmable logic device12. Third party programmers may be unable to access the security portion of the programmable logic device12without access to the NOC100. Instead, the NOC100may be limited to communication by certain programmers with a level of security credentials.

Since resources of the programmable logic device12may be pooled or assigned in different patterns for different applications/designs executed on the programmable logic device12, the arrangement of such resources (e.g., programmable elements in the fabric die22and memory in the base die24) may be configurable. Indeed, the bandwidth, mapping, and transportation of data within programmable logic device12may be programmable for the fabric die22or the base die24.

FIG. 12illustrates a programmable logic device12that includes a programmable array120as part of its configurable resources. The programmable logic device12may include a reconfigurable on-chip memory pool122as another part of its configurable resources. The programmable logic device12also includes a compute accelerator124(e.g., dot-product engines) that may perform computations in-memory and/or near-memory to obtain improved performance and power efficiency. Using local computes and the reconfigurable on-chip memory pool122, the programmable logic device12may include memory with a density greater than memory density attained otherwise. For example, the memory density of the programmable logic device12with the reconfigurable on-chip memory pool may be five times greater than the density using dedicated memory blocks (e.g., M20K memory blocks) inside the programmable array120. Furthermore, the reconfigurable on-chip memory pool122may be accessed with a bandwidth higher than high-bandwidth memory (HBM) with a lower energy per bit (e.g., less than 5 times the energy per bit). Due to the in-memory and/or near-memory compute using the compute accelerators124and the memory pools122, the programmable logic device12may perform more tera operations per second (TOPS) with an increased efficiency.

The programmable logic device12may include an I/O interface126and an application processor128. The I/O interface126may provide channels to inject data into and/or extract data from the programmable array120. For example, the I/O interface126may include I/O interfaces provided as part of the support circuitry in the base die24. The application processor128may include a controller to control operation of the programmable array120. For example, the application processor128may include the SCs58and/or the DC60.

FIG. 13provides a model for a 3D stacked programmable fabric device130that includes available memory in the fabric die22, the available memory in the base die24, and the bandwidth132between the fabric die22and the base die24. For instance, a portion (e.g., sector48or entire) of the fabric die22may include a number (e.g., 1680) of logic array blocks (LABs), a number (e.g., 205) of memory blocks (e.g., M20Ks), and a number (e.g., 168) of digital signal processing (DSP) blocks while the base die may provide an amount (e.g., 20 Mb) of memory. Some components (e.g., fabric) of the fabric die22or some components (e.g., memory) of the base die24may be configurable to various different aggregations, such as having a number (e.g., 3) different levels (e.g., fine, medium, and large) of aggregation of memory in the base die22. Each level may have its own associated properties, such as a capacity of memory in the base die22and bandwidth or latency between the fabric die22and the base die24. As illustrated, the programmable fabric device130includes fine-grained interfaces133and medium/shared-grained interfaces134that provide access between the base die24and the fabric die22via corresponding portions of the bandwidth132. For instance, the fine-grained interfaces133may access only a part136(e.g., ⅛th) of a portion of the memory of the base die22. Specifically, the fine-grained interfaces133are used in a fine grained aggregation138since the fine-grained interfaces133may not use a memory management unit (MMU) or an address translation unit (ATU). As illustrated inFIG. 14, in a medium-grained aggregation140, the fabric die22may utilize the medium/shared-grained interfaces134in a deterministic manner (e.g., without using the MMU/ATU), but the whole memory of the portion of the memory of base die24may be accessed using the medium-grained aggregation140from the fabric die22. As illustrated inFIG. 15, in a shared aggregation150, the medium/shared-grained interfaces134are used with the MMU/ATU to enable the portion of fabric die22to access any memory in the base die24including any portions of memory not vertically below the fabric die22.

The fine-grained and medium-grained aggregations of memory blocks may be used to obtain adjustable latency in a programmable logic device system140. In fine-grained and medium-grained aggregation, the latency may be deterministic and the memory of the base die24may behave as a M20K. Fine-grained and medium-grained aggregations of memory blocks may be deterministic by providing direct access bandwidths132between respective portions (e.g., sectors48or portions of sectors) of the fabric die22and corresponding portions of the base die24. As may be appreciated, the latency of the fine-grained aggregation may be a low level (e.g., 1-15 ns) while the latency of the medium-grained aggregation of memory blocks may be higher than the low level (e.g., 16-32 ns). The shared aggregation150may have variable latency that virtualized access that allows shared access to a large portion (e.g., the entire) of the memory of the base die24.

FIG. 16illustrates an aggregation160using fine-grained deterministic access capacity aggregation. As illustrated, a programmable fabric may include memory modules162(e.g., M20K) that may be used in the fabric die22. The aggregation160shows the relationship between the memory modules162that may be used in the fabric die22and the consolidated aggregated memory that may be placed in the base die24in fine-grained memory aggregation. The aggregation160illustrates an aggregation from a single portion164(e.g., a sector, a part of a sector, parts of multiple sectors, etc.) of fabric of the fabric die22into a corresponding portion166. The portion166may be a sub-portion of a portion168(e.g., support circuitry70). Aggregating the memory modules162from the fabric die22to the base die24may provide consolidation of memory and may improve the deterministic latency even though bandwidth may be reduced.

FIG. 17illustrates a fine-grained aggregation170with deterministic access capacity aggregation of the memory modules162of multiple portions164of the fabric die22into corresponding a portion168having the portions166contained therein. Some portions164may include entire sectors48and/or portions of sectors. The aggregation170may employ fabric resources172in the fabric die22to coordinate memory access. For instance, the aggregation170may employ a memory soft wrapper in the fabric of the fabric die22.

FIG. 18illustrates a relationship between a fine-grained aggregation170and a medium-grained aggregation180aggregating memory of the memory modules162that may be used in fabric die22with the consolidated aggregated memory that may be placed in the base die. The aggregation180aggregates memory from the portions164to the portion168. The medium-grained memory aggregation may avoid the use of fabric resources172to coordinate memory access since the medium-grained aggregation180supports multiple access points to the same physical address space (e.g., the portion168). In some embodiments, the latency of accesses may decrease over implementing the memory in the memory modules162even when bandwidth may be reduced. As may be appreciated, in both the fine-grained aggregation170and the medium-grained aggregation180, the latency is deterministic and may employ a soft wrapper in fabric.

FIG. 19illustrates a non-uniform memory access (NUMA) aggregations of the memory of the memory modules162of thirty portions164to use a portion198of the base die24. The portion198may include a part of a support circuitry70, an entire support circuitry70, parts of multiple support circuitries70, or any combination thereof. A comparison of a fine-grained memory aggregation200and a medium-grained memory aggregation202is provided for a coarse NUMA aggregation204. The fine-grained memory aggregation200may employ extensive use of fabric resources172and may provide a deterministic latency that is low relative to on-die memory with a lower bandwidth. The medium-grained memory aggregation202may employ reduced use of fabric resources172and may provide a deterministic latency that is low relative to on-die memory with a lower bandwidth than the fine-grained memory aggregation200. In the coarse NUMA aggregation204, no fabric aggregation is employed and, thus, the use of fabric resources172is substantially decreased. The bandwidth may be reduced below that of the medium-grained memory aggregation202. In some embodiments of the coarse NUMA aggregation205, the base die24may perform address translation to access memory when responding to memory access requests.

FIG. 20illustrates how bandwidth aggregation may be used in fine-grained memory aggregation use models and medium-grained memory aggregation use models. In fine-grained memory aggregation210, instead of accessing a single memory bank212in the base die24via a single access port214, the memory bank212may be replicated to other memory banks212to enable the use of additional access ports214to perform bandwidth aggregation. The fine-grained memory aggregation210may increase the bandwidth but may lead to a reduction in the total memory capacity. In read-and-write memory, the fabric may coordinate reads and writes across the memory banks212using the fabric resources172to prevent inconsistent data across the replicated memory banks212. In a medium-grained memory aggregation216, the fabric die22and the base die24may aggregate access ports214to generate a high-bandwidth communication link between the fabric die22and the base die24. The multi-port access may be managed by the fabric die22. To aggregate the memory, the fabric may employs a memory soft wrapper in the fabric.

In the deterministic aggregations ofFIG. 20, the deterministic access use models may have limited memory capacities. Furthermore, memory access may be performed directly from the fabric without translation (e.g., to distributed address space), whereas in coarse-grained aggregations (e.g., NUMA), one or more memory banks212may be shared and accessed via address translation of an memory management unit (MMU) in the base die24.

FIG. 21illustrates a bandwidth aggregation220in coarse-grained memory aggregation use cases. Bandwidth aggregation may be obtained by increasing the number of access ports between the base die24and the fabric die22. In such situations, multiple accessors (e.g., logic blocks in the programmable logic) that share the memory may use the fabric resources172to manage access to the base die24via multiple access ports214. The base die24may have an MMU222having an address translation unit (ATU)223to assist address translation and access management.

FIG. 22illustrates NUMA shared memory accesses to shared-memory in a coarse-grained aggregation use model. In the illustrated example, accessors230,232, and234may share a common memory236using the illustrated mapping of various logical address ranges238. The base die24(e.g., via the MMU222) may provide translation to physical addresses240from logical addresses242. For example, the base die24may translate respective logical addresses242A,242B, and242C to respective physical addresses240A,240B, and240C. Furthermore, the base die24(e.g. via the MMU222) may provide access protection (e.g., authorization, locking) for the common memory236. The base die24may share responsibilities related to memory consistency and/or coherency with the fabric die22(e.g., via the fabric).

As may be appreciated, the memory of the base die24may have higher capacity than the in-die memory (e.g., M20K, memory logic array block (MLAB), adaptive logic modules (ALM)). Furthermore, the memory of the base die24may have a higher bandwidth than shore-line/peripheral memory (e.g., HBM, dual-data rate (DDR)). However, the memory of the base die24may have a lower bandwidth than in-die memory. Therefore, in situations in which high bandwidth access may be useful or is to be used at a rate higher than available via the base die24, the programmable logic device12may perform memory paging from the base die24to the in-die memory on the fabric die22. More generally, the programmable logic device12may employ paging of the memory between in-die and base-die memory regions to manage bandwidth of data exchanges.

FIG. 23illustrates paging of memory between the in-die memory and the base-die memory (e.g., buffering resources such as M20K, MLAB, and ALM-FF modules). In some situations, a design to be implemented in the fabric may benefit from a bandwidth at a level available an in-die memory (e.g., memory modules162) but not the base die24. However, the available in-die memory resources may not be sufficient. In such situations, the memory of the base die24may be used to provide additional capacity, and programmable logic device12may utilize paging to place work sets in the in-die memory. Paging may be performed with or without fabric involvement.

The paging model employs the fabric die22and the base die24as alternative readers/writers of the in-die memory. Such alternative writing and reading may utilize significant fabric resources172. To reduce consumption of fabric resources172, a paging model may employ hardened pathways to write to in-die memory without fabric involvement. To enable the base die24to read/write from the in-die memory, in-die memory module (e.g., memory module162) being paged may be paused for some period (e.g., <1 ms) to perform the paging.

As previously noted, memory in the base die24may be copied, broadcast, gathered, scattered, and/or transformed in the base die24. For example, this movement of data may be made using direct memory accesses (DMA) in the base die24without moving the data through fabric die22.FIG. 24illustrates example DMA movements250,252,254, and256by the base die24to move and/or transform data. The movement250illustrates copy and/or broadcasting data from a first portion258A (e.g., memory bank212, support circuitry70, etc.) of memory of the base die24to other portions258B,258C, and258D. Although the fabric of the fabric die22may master and/or initiate the DMA, the data moved may not be passed through the access port214and leaving fabric resources172unencumbered by the DMA.

The movement252includes scattering of data from one portion258A with parts of the data from the portion258A being scattered to respective portions258B,258C, and258D. Similarly, the movement254includes gathering the data to the portion258A from multiple portions258A,258B,258C, and258D. Similar to the copy/broadcast of data, the movements252and254may be performed without fabric involvement or usage of the access port214in passing the data between portions258. The movement256includes transforming data in the base die24by performing one or more operations on the data in the portion258A without moving the data to the fabric die22for processing.

Fast partial reconfiguration may be used on the fabric of the programmable logic device12to reconfigure a portion of the fabric dynamically while the remaining fabric design continues to function.FIG. 25illustrates a partial reconfiguration that may be facilitate transformation of the fabric using configurations stored in the base die24. The PR may be performed employing background loading and/or swapping of configuration data in the fabric from the base die24. Accordingly, during background load259configuration data for portions260(e.g., sectors48) of the fabric die22may be loaded into the corresponding locations in the base die24(e.g., support circuitry70). For example, configuration data for the portion260A may be stored in the portion262A, configuration data for the portion260B may be stored in the portion262B, configuration data for the portion260C may be stored in the portion262C, and configuration data for the portion260D may be stored in the portion262D even if configurations are already loaded in into and/or being used in the portions260A,260B,260C, and260D. When one of the portions (e.g., portion260A) is to be reconfigured, a partial reconfiguration264may be employed. In the partial reconfiguration264, the configuration of the portion260A is loaded into the fabric of the portion260A from the portion262A in a relatively short period (e.g., <1 ms) over the access port214compared to loading from shoreline memory. Since the portions260are aligned with the portions262, multiple reconfigurations may be loaded in parallel using multiple access ports214. For example, a parallel loading266loads respective configurations into the portions260A,260B,260C, and260D of the fabric die22from respective portions262A,262B,262C, and262D of the base die24in a relatively short period (e.g., <1 ms total) relative to sequential loading from the base die24and/or loading from shoreline memory. In other words, the foregoing background loading of configurations into base die24and swapping configurations increases the usefulness and speed of performing partial configurations. Additionally or alternatively, the portions260A,260B,260C, and/or260D may pull configurations from any of the portions262A,262B,262C, and262D.

As previously noted inFIG. 12, the base die may include a compute-near-memory (CnM) architecture with compute accelerators124(e.g., dot-product engines) located near memory (e.g., memory pool122).FIG. 26illustrates a CnM architecture that may be used to increase speed of computation. In the illustrated embodiment, the compute accelerators124include dot-product engines (DPEs). However, the compute accelerators124may include any suitable computation circuitry that may be used to implement, for example, finite impulse response (FIR) filters, fast Fourier transform (FFT) algorithms, and the like. The computations may use multiple precisions (e.g., integer, floating point, Gbit, 16 bit), different Endianness, and/or may allow data reutilization. The CnM architecture may increase the number of available programmable logic device12resources by adding extra compute power (e.g., DPEs) and memory (e.g., RAMs). In some embodiments, the compute accelerators124may be organized as fixed regions. A user and/or administrator may allocate one or more portions274(e.g., partition) of the fabric die22to user design(s). Each of the user designs, includes at least a portion274(e.g., sector) of the fabric die22. Each user design is also allocated at support circuitry276(e.g., support circuitry70) in the base die24. For instance, each support circuitry276may be allocated based on allocation of a corresponding portion274of the fabric. Resources accessed by the fabric may be distributed via microbumps as previously discussed. The fabric may broadcast control to the support circuitry276. As illustrated, the support circuitry276may include DPEs278arranged in rows between rows of memory blocks280. Additionally or alternatively, the memory blocks280and the DPEs278may be interleaved in any other suitable configuration. Furthermore, at least some of the DPEs278may be replaced and/or supplemented with other compute accelerators124configured to perform computations near the memory pool122(e.g., the memory blocks280).

Using the base die24to perform CnM may enable the base die24to perform a portion of operations for the programmable logic device12. The base die24may perform tensor operations (e.g., matrix-vector and matrix-matrix multiplications). For example, if the compute accelerators124include the DPEs278, the base die24may provide INT8 precision for each DPE278that includes a 40-bit accumulator.

The programmable logic device12may be a data processing system or may be a component of a data processing system. For example, the programmable logic device12may be a component of a data processing system500, shown inFIG. 27. The data processing system500includes a host processor502, memory and/or storage circuitry504, and a network interface506. The data processing system500may include more or fewer components (e.g., electronic display, user interface structures, application specific integrated circuits (ASICs)). The host processor502may include any suitable processor, such as an INTEL® XEON® processor or a reduced-instruction processor (e.g., a reduced instruction set computer (RISC), an Advanced RISC Machine (ARM) processor) that may manage a data processing request for the data processing system500(e.g., to perform machine learning, video processing, voice recognition, image recognition, data compression, database search ranking, bioinformatics, network security pattern identification, spatial navigation, or the like). The memory and/or storage circuitry504may include random access memory (RAM), read-only memory (ROM), one or more hard drives, flash memory, or the like. The memory and/or storage circuitry504may be considered external memory to the programmable logic device12and may hold data to be processed by the data processing system500. In some cases, the memory and/or storage circuitry504may also store configuration programs (e.g., bitstream) for programming the programmable logic device12. The network interface506may enable the data processing system500to communicate with other electronic devices. The data processing system500may include several different packages or may be contained within a single package on a single package substrate.

In one example, the data processing system500may be part of a data center that processes a variety of different requests. For instance, the data processing system500may receive a data processing request via the network interface506to perform machine learning, video processing, voice recognition, image recognition, data compression, database search ranking, bioinformatics, network security pattern identification, spatial navigation, or some other specialized task. The host processor502may cause the programmable logic fabric of the programmable logic device12to be programmed with a particular accelerator related to requested task. For instance, the host processor502may instruct that configuration data (bitstream) stored on the memory/storage circuitry504or cached in sector-aligned memory of the programmable logic device12to be programmed into the programmable logic fabric of the programmable logic device12. The configuration data (bitstream) may represent a circuit design for a particular accelerator function relevant to the requested task. Due to the high density of the programmable logic fabric, the proximity of the substantial amount of sector-aligned memory to the programmable logic fabric, or other features of the programmable logic device12that are described here, the programmable logic device12may rapidly assist the data processing system500in performing the requested task. Indeed, in one example, an accelerator may assist with a voice recognition task less than a few milliseconds (e.g., on the order of microseconds) by rapidly accessing and processing large amounts of data in the accelerator using sector-aligned memory.

Placement of computation and memory in spatial architectures where compute and memory have three-dimensional spatial locality may be performed statically. Additionally or alternatively, the programmable logic device12may dynamically allocate, relocate, and de-allocate compute and memory on such spatial architectures. These techniques enable the static mapping and dynamic management of systems using such architectures. Moreover, using flexible allocation schemes enables the programmable logic device12to find and support optimal co-placement of compute with memory in a static setting in a known sequence of static settings and in a dynamic setting when the allocation of compute and memory is not known a priori. Such usage of static and/or dynamic placement in a three-dimensional spatial locality provides the ability to extend compilation to support the simultaneous synthesis, placement, and routing of spatial computation with a spatially distributed memory to enable users to leverage an architecture with a much richer memory sub-system. The support for dynamic management of the computation and memory allows users/administrators to build dynamic runtime systems for spatial architectures for the programmable logic device12.

Static Compilation to a Multi-Dimensional Device

An embodiment of the programmable logic device12is provided inFIG. 28. As illustrated, a portion (e.g., a sector) of the fabric die22and the base die24are connected via the access ports214in a stacked-die architecture. The fabric die22contains a spatial compute fabric which contains one or more memories (e.g., M20Ks). The fabric die22has a local sector manager (LSM)518that controls the configuration of the fabric. In some embodiments, the LSM518coordinates the communication between the fabric die22and the base die24. The fabric die22has a fabric multi-die interface bus (FMIB)520that is used to communicate with a base multi-die interface bus (BMIB)522in the base die24via the access ports214. The base die24has a controller524that coordinates the communication between the fabric die22and the base die24. As previously discussed, the base die24contains memory (e.g., memory block526). In some embodiments, the base die24also contains compute units528(e.g., DPE or the like). Physically, the fabric and various memory sectors in the 3D columnar construction may have different sizes and/or use different technologies. Furthermore, even though the discussion related to two die in the programmable logic device12, the programmable logic device12may contain more than two die.

FIG. 29illustrates the programmable logic device12having four portions (e.g., sectors). In some embodiments, the spatial fabric portions are stitched using configurable wires532(also referred to as fabric resources) to allow spatial fabric portions to communicate with each other. In some embodiments, the base portions are stitched together using an interconnect534, such as the NOC100. The interconnect534allows each FMIB520and BMIB530to access any of the base memory blocks526. In some embodiments, the portions of the fabric die22and/or the base die24may have additional components or fewer components.

Although the fabric may be divided into portions (e.g., sectors), the fabric may be viewed as a continuous 3D architecture as illustrated inFIG. 30. The compilation of the programmable logic device12may include optimizing spatial alignment of the compute/memory to minimize latency and maximize bandwidth. In some embodiments, access to the compute/memory in another die uses deterministic latency. In some embodiments, to realize this deterministic latency requirement, the memory may be located within a defined proximity of the accessor. In some embodiments, access to the memory in another die does not use deterministic latency (e.g., the NUMA model). In such embodiments, the accessor and accessed memory have no proximity limitations to be functional, but the compiler16may choose to optimize the placement of the accessors and accessed memories to minimize latency and/or maximize bandwidth.

In some embodiments, a user (and/or administrator) interacts with the spatial fabric as illustrated inFIG. 31. User logic560,562, and564specifies one or more designs for the fabric die22and/or the base die24. For example, these designs may be specified in the same manner as would be done in a 2D or 2.5D programmable logic devices. In some embodiments, the user logic560,562, and/or564may be described in a hardware description language (HDL). In some embodiments, the user logic560,562, and/or564may be described in a high-level language. To communicate with the spatial memory/compute in another plane, the user logic560,562, and/or564is coupled to an appropriate interface and/or bus. For example, the fabric in the fabric die22may be coupled to the memory blocks526or the compute units528via the FMIBs520. In some embodiments, the interface provides a direct memory interface (e.g. SRAM/M20K interface). In some embodiments, the interfaces use a well-defined protocol such as an AVALON® Memory-Mapped (AVMM) interface or advanced extensible interface (AXI).

As illustrated inFIG. 32, the user (and/or administrator) may specify memory arrays570to be mapped in the base die24. In some embodiments, these memory arrays570are explicitly tagged to be mapped to the base die24. Additionally or alternatively, the compiler16may infer the memory arrays570to be in the base die24. Furthermore, the compiler16may also infer the FMIBs520.

In some embodiments, the user (and/or administrator) creates read/write associations572between the FMIBs520and the memory arrays570as shown inFIG. 34. In some embodiments, the read/write associations572are defined as read only, write only, and read/write. In some embodiments, the read/write associations and FMIBs520are inferred by the compiler16. For access that are to be deterministic, the user (and/or administrator) specifies an attribute (e.g. direct access574) to inform the compiler16that the respective FMIB520and respective memory array570are to be located within a defined proximity. In the absence of a specific definition, other connections without this attribute may be assumed to have non-deterministic access times (i.e. NUMA). In some embodiments, deterministic accessors are mapped to specific FMIBs on the spatial fabric of the fabric die22. The FMIB520may be shared between deterministic and non-deterministic accessors with the deterministic accessors mapped to this same FMIBs as non-deterministic accessors. In such embodiments, the accessors themselves are configured to enable deterministic access.

After the FMIBs520and the respective memory arrays570are associated and/or inferred, the FMIBs520interfaces are elaborated to support the defined access. In the illustrated embodiments ofFIG. 34, each FMIB520contains one or more offsets576,578,580, and580for the memory arrays570that the respective FMIB520is defined to access. For example, a first FMIB520may include an offset576for a first memory array570that the first FMIB520is defined to access. Similarly, a second FMIB520may include the offset576along with offsets578and580for respective second and third memory arrays570that the second FMIB520is defined to access. Furthermore, a third FMIB520may contain the offsets578and580along with an offset582for a fourth memory array570that the third FMIB520is defined to access. These offsets576,578,580, and582allow the user to define addressing to the memory of the base die24using logical addresses. In some embodiments, the FMIBs520provide hardwired circuitry to support the offsets using hardware. Additionally or alternatively, programmable circuitry is used to support the offsets using software. These offsets are used to support multiple array accesses from a single FMIB520. In certain embodiments, the FMIBs520do not perform the logical-to-physical address translation of the base memory address space. In such embodiments, that address translation is done using the MMU222, as discussed below.

As previously discussed, the programmable logic device12may perform memory paging between the fabric die22and the base die24. To this point and as illustrated inFIG. 35, memory paging relationships584may be defined between base memory arrays570and groups of spatial fabric memories586(e.g. M20Ks). In some embodiments, the relationships584are defined as read-only, write-only, or read/write. In some embodiments, the spatial fabric memories586are defined as a dynamic partial reconfiguration region. As previously discussed, this partial reconfiguration allows the paging to happen while the rest of the spatial fabric continues to execute design-implemented operations.

As illustrated inFIG. 36, an address space588is defined for memories590in the base die24and is defined for memories594(e.g. M20Ks) in the spatial fabric. As illustrated, the definitions in the physical address space488may be flat and any FMIB520can access any memory (e.g., memories590and/or594) in the physical address space488. In some embodiments, the physical address space488may not be flat, and an address translation unit as part of a memory management unit (MMU) of the base die24translates logical addresses from the FMIB520into physical addresses in the base die24.

In some embodiments, the address translation unit (ATU)223has restrictions on the logical-to-physical address translation. InFIG. 37, an address translation is shown in the restricted space. In this example, the upper number (e.g., four) bits600may define a portion (e.g., sector48). For example, the bits “0100” identify a portion602(e.g., sector48or support circuitry70) of memory. A next number (e.g., six) bits603define a sub-portion of the identified portion as the high address space of the memory in that portion. For example, the bits “011001” identifies a block604and a sub-block606of memory in the portion602. Specifically, the block604is identified by the bits “01” and the bits “1001” identify the sub-block606within the block604. Remaining bits608may be used to define specific memory locations within the sub-block606. In some embodiments, the bits (e.g., ten most significant bits) corresponding to the portions, the block, and the sub-block may be translated while the remaining (e.g., lower ten bits) are not translated. Other translation schemes that use different number and/or allocation of bits for address translation may be employed to find a balance between the size of the ATU223and the richness of translation. The limits of the ATU223provide restrictions to the compiler16when finding the placement of the defined compute and memory.

In some embodiments, the compiler16takes user input descriptions of the design and the definition of the architecture, physical address space488, and ATU223restrictions to determine the placement of the user logic560,562, and564and the respective FMIBs520as shown inFIG. 38.FIG. 38illustrates a depiction of the user logic560,562, and564mapped into the physical address space488. Memory accesses that are labeled direct access in the designs are to be aligned within a defined proximity of the memory they access, such as the FMIB520and the array570A. In some embodiments, memory access that are not defined as direct access are placed to minimize access latency and maximize bandwidth.

In some embodiments, the compiler16takes the user input description of the design and the definition of the architecture, physical address space488, and the ATU223restrictions to determine the placement of the user logic560,562, and564, the respective FMIBs520, base memory arrays570, and spatial fabric memory586as shown inFIG. 39. Memory accesses that are labeled direct access in the designs are to be aligned within a defined proximity of the memory they access, such as the FMIB520and the array570A. In some embodiments, memory accesses that are not defined as direct access are placed to minimize access latency and maximize bandwidth. In some embodiments, the MMU222may use the ATU223to translate the logical address space defined in the user's design to the physical address space488of the memory in the fabric die22or base die24. In some embodiments, the MMU222may provide security to disable unauthorized access to memory. In some embodiments, the MMU222disables unauthorized access at the accessor. In some embodiments, the MMU222provides locking capabilities to isolate the read and/or write access of multiple accessors to a subset of addresses to enable memory consistency.

In some embodiments, after the FMIBs520, the spatial fabric memory586, and the base memory arrays570are placed within the restrictions of the ATU223, the compiler16configures the MMU222as shown inFIG. 40. Specifically, the MMU222(and the ATU223) may store a table620used to translate virtual addresses to physical addresses for the memory arrays570in the base die24. Similarly, the MMU222(and the ATU223) may store a table622used to translate virtual addresses to physical addresses for the memories594in the fabric die22. In some embodiments, the ATU223is configured in coordination with the offsets of the FMIBs520for multiple array accessing. The MMU222(and the ATU223) may store mappings624for bulk movements of data to and from the base die24to the memories594of the fabric die22.

FIG. 41is a flow diagram of a process625that may be deployed by the compiler16when organizing the programmable logic device12. The compiler16maps implementations of designs (e.g., user logic560,562, and/or564) to one or more FMIBs520(block626). As previously noted, this mapping may be associated in the designs and/or inferred by placement of the designs. The compiler16then maps the FMIB(s)520to one or more memory arrays570of the base die24(block627). The mapping may include a mapping from the FMIB(s)520to a corresponding BMIB522. The mapping may include a forced direct access between an FMIB520and a corresponding memory array570when the latency and/or bandwidth between the designs and the array570is deterministic. The mapping may also include offsets in the FMIB520that provides a virtual starting address for the memory arrays570. The compiler16may also map the one or more memory arrays570to memory in the fabric die22for bulk transfer and/or memory paging (block628). As previously noted, the mapping between the memory array(s)570and the in-die memory may be based at least in part on associations set by a user and/or administrator. Store mapping in the ATU223and/or the MMU222.

Sector-Aligned Dynamic Partial Reconfiguration

The programmable logic device12may be used to perform a partial reconfiguration (PR) of the fabric where a portion of the fabric is reconfigured while one or more other portions of the fabric remain in use during the PR. The alignment of sectors or portions as of the fabric die22or base die24combined with sector alignment of PR enables an increase in the PR performance. The static compilation previously discussed focused on logic and memory placement of a single design. However, the concepts discussed related to the memory in the base die24may be extended beyond user data. For example, compilation may be used to store multiple partial reconfiguration personas. The separation of the memory for data and personas is part of the static compilation. A partial reconfiguration630is illustrated inFIG. 42. The partial reconfiguration630has PR regions632and are defined as part of a static compilation. The PR regions632and634have few restrictions on their size or shape and allow static routes636to go through the PR regions632and634. The PR regions632and634may include inter-sector routes638that enable communication through the sectors48within a respective PR region632or634. Each PR region632and634is capable of supporting any number of PR personas that have been compiled and/or relocated to the respective PR region632or634and use a subset of an input and output interface of the PR region632and634.

The partial reconfiguration may be restricted to be aligned to the sectors48of the fabric. Sector-aligned dynamic PR, as shown inFIG. 43, is a sector-restricted form of PR that forces the PR regions632and634to be defined on boundaries of the sectors48. This sector-restricted method allows entire sectors48to be reconfigured by using a configuration write instead of the read-modify-write process of traditional PR. Inter-sector routes638between sectors48contained in the same PR region632or634may cross boundaries of sectors48. Static routes636, on the other hand, may not cross through PR regions632or634as permitted in traditional PR. These routing restrictions of sector-aligned PR restrict the PR regions632and634to accommodate the static routes636outside of the PR regions632and634.

As illustrated inFIG. 44, to reduce the restrictions imposed by sector-aligned PR, a network-on-chip (NOC)640may be employed in the programmable logic device12. In some embodiments, the NOC640is created with soft logic. In such embodiments, the soft logic NOC is part of the fabric resources in the PR regions632and634with the programmable logic device12tolerating portions of the NOC640disappearing during a PR operation. In some embodiments, the NOC640is created with hard logic (e.g. the NOC100). In some embodiments, sector-aligned PR uses the NOC640to support the static routes636. Furthermore, regardless of implementation type, the use of the NOC640enables adjacent sectors48to be used for different regions without leaving sector-sized gaps between sector-aligned PR regions632and634to support the static routes636.

In some embodiments, the NOC640is implemented as the NOC100in the base die24as shown inFIG. 45. In some embodiments, the NOC100of the base die24provides fabric-to-fabric communication as the spatial fabric NOC640did inFIG. 44. In some embodiments, the NOC100provides communication to the memory of the base die24and communication is done through memory reads and writes of the base die24via the access ports214.

In some embodiments, sequencing of PR personas using sector-aligned PR is coordinated as a series of static compilations that adhere to the original base compilation of the PR regions. These static compilations of personas may be later swapped in a sequence. InFIG. 46, two sector-aligned PR regions with PR personas F1and F3are shown. F1and F3are shown to communicate through the base die24via the access ports214. If the next persona is statically known, it may be background loaded to the base die24, as previously discussed, to take advantage of the speed of sector-aligned PR in a three-dimensional setting.FIG. 47illustrates a persona F2with its own static routes636and inter-sector routes638loaded into memory of the base die24in the background while F1and F3in the fabric continue to execute until the loading is performed using the access ports214and corresponding FMIBs520.FIG. 48illustrates the persona F2loaded into the fabric die22.

FIG. 49is a static sequence650of PR personas. In some embodiments, the next persona(s) are background loaded. The background loading allows the configuration data to be loaded into the memory of the base die24in preparation of sector-aligned PR operation between the fabric die22and the base die24to reconfigure the spatial fabric. In a first part651of the static sequence650, a first persona is executing in the fabric die22while a second persona is loaded into the base die24via the access ports214and respective FMIBs520. During a second part652of the static sequence650, the second personal is loaded from the base die24to the fabric die22. In a third part653, a third persona is loaded into the base die24while the second persona is executing in the fabric die22. In some embodiments, the third persona and the first persona may be the same configuration. In a fourth part654, the third persona is loaded into the fabric die22from the base die24via the access ports214and respective FMIBs520.

FIG. 50is a flow diagram of a process655. One or more personas are loaded into the fabric die (block656). The personas may be sector-aligned and may include one or more sectors48inside each region corresponding to a persona. One or more background loaded personas are loaded into the base die (block657). At a later time, the one or more background loaded personas are loaded from the base die24into the fabric die22(block658).

Execution of Dynamic Actions

The programmable logic device12may be used to perform dynamic actions. The execution of the dynamic actions may not leave the compute or memory allocations unchanged, re-allocations, or de-allocation. The ability to execute these dynamic actions may be part of the static compilation process.

Sector fabric memory paging has been previously discussed as part of the static compilation process. In some embodiments, a spatial fabric memory paging may use partial reconfiguration and through the FMIB520(and access port214) as shown inFIG. 51. In some embodiments, the spatial fabric invokes a PR operation isolated to the memory in the fabric die22and interfacing logic which directs the system to move data to/from the memory in the fabric die22from/to the memory in the base die24. In some embodiments, the MMU222protects the system from unauthorized spatial fabric memory paging. During spatial fabric memory paging, the page may be aborted if the memory being paged is corrupted and/or precautions may be taken (e.g., restrictions on writes) during the spatial fabric memory paging. In some embodiments, the associations between the memories involved in paging is communicated, and memory corruption is avoided using soft logic. In some embodiments, the memory corruption is avoided using hard logic in the MMU222.

The static placement of the memory in the base die24may be decided by the compiler16. In some embodiments, direct memory access (DMA) operations are performed on the memory in the base die24. The DMA operations may include DMA scatter operations, DMA gather operations, parallel DMA transfer operations, and the like. Furthermore, the DMA operations may be involved in near-to-memory compute operations, and/or other operations that involve memory accesses. In some embodiments, the spatial fabric communicates a DMA descriptor to a DMA engine660located in the base die24. For instance, a DMA engine660may be included in one or more of the support circuitries70of the base die24. The base die24then executes the DMA operation to completion.

An embodiment of a DMA scatter operation661is shown inFIG. 52. In some embodiments, given the description of the FMIB520and memory associations for the base die24, some read and write operations may be restricted during the DMA operation to prevent write corruptions. A specific restriction given the example DMA scatter operation is also shown inFIG. 52. To avoid corruption of the memory of the base die24during the DMA operation, restricted memory operations664may be blocked while allowed memory operations666are allowed. In some embodiments, these restrictions are communicated via the soft logic in the spatial fabric. In some embodiments, the DMA engines660use the MMUs222in the base die24to disallow the potentially corrupting reads and writes from the restricted memory operations664.

FIG. 53shows a gather operation670where data is gathered from various locations in the base die24. Similar to the scatter operation661, some memory access operations may be susceptible to causing memory corruption during the gather operation670. Accordingly during the gather operation670, the restricted memory operations664that may be susceptible to having corrupted reads or writes may be blocked while the allowed memory operations666without such susceptibilities may be allowed. In some embodiments, the memory has no restrictions on how it is scattered or gathered. In some embodiments, the DMA operations are restricted based on user inputs.

In some embodiments, parallel DMA operations672,674, and676are issued as shown inFIG. 54. The restricted memory operations664and a single allowed memory operation666are displayed for operations during the parallel DMA operations672,674, and676.

As previously noted, compute accelerators124may also be included in the base die24. In some embodiments, the compute accelerators124are tightly coupled with the memory. In some embodiments, the compute accelerators124are loosely coupled with the memory. In some embodiments, the spatial compute fabric is coupled with different base die24instances to enable selection of application-specific acceleration. In some embodiments, as shown inFIG. 55, a tightly coupled compute-near-memory instance of the base die24is illustrated. In the illustrated embodiment, portions680(e.g., support circuitry70) include multiple banks of memories682coupled with arithmetic circuitries684(e.g., adder, multiplier, etc.) of the compute accelerators124. In some embodiments, the spatial compute fabric provides hooks to dynamically control the compute accelerators in the base die24via the controller524. For example, the controller524may be used to load weights into memories686or688to be used in arithmetic operations performed by the respective arithmetic circuitries684on the data of respective banks of memories682. The memories686and/or688may include registers used to store weights for use in the arithmetic operations. In some embodiments, the user creates the instances to control the compute accelerators124, and the compiler16maps the controls to appropriate FMIBs520. In some embodiments, the compute-near-memory performs an application-specific computation (e.g. dot-product). In some embodiments, the compute-near-memory enables tight interactions with the memory to enable efficient memory operations such as transposing, shuffling, and the like to the memories. In some embodiments, the compute-near-memory provides simple arithmetic logic unit (ALU) operations (e.g. increment/decrement) that operate on the memories. In some embodiments, these other forms of computations are controlled by the spatial fabric through the FMIBs520. As previously discussed, the DMA660may access/store data in the banks of memories682. In some embodiments, a DMA compute690that may be used to perform computations on the data being passed using the DMA660.

FIG. 56is a block diagram of a process700using the compute accelerators124. The controller524loads one or more weight(s) into one or more memories686or688(block702). The DMA660receives and scatters data to the banks of memories682(block704). The compute accelerators124then perform corresponding arithmetic calculations on data in the respective banks of memories682(block706). The DMA660then gathers the data from the respective compute accelerators124(block708). The gathered data is then transmitted to an appropriate location (block710). For example, the gathered data may be transmitted to the fabric via the BMIB522, transmitted to the NOC100, scattered by the DMA660through the portion680, used alongside with a new input vector input to the one or more compute accelerators124, and/or the like.

FIG. 57illustrates steps of the process700inside the portion680of the base die24. As illustrated in the portion680A, the controller524loads a first weight712into the memory686of a first compute accelerator124. Loading the first weight712into the memory686may include loading the first weight712to the controller524via the BMIB522. In the portion680B, the controller524loads a second weight714into the memory688of the first compute accelerator124where the controller524receives the second weight714via the BMIB522. In the portion680C, the controller524loads a third weight716into the memory686of another compute accelerator124after the controller524has received the third weight716from the BMIB522.

In the portion680D, the DMA660scatters/broadcasts data718to respective banks of memories682. The DMA660may receive the data718from the controller524that receives the data718from the BMIB522and/or the NOC100. Once the data has been distributed to the compute accelerators124, each compute accelerator124loaded with appropriate data, the compute accelerators124perform computations as illustrated in the portion680E. As illustrated in the portion680F, the DMA660gathers the results720from the compute accelerators124. The DMA660may then transmit the results720to an FMIB520via a respective BMIB522(illustrated in portion680G) and/or to the NOC100(illustrated in portion680H).

Dynamic Allocation, Re-Allocation, and De-Allocation of Compute and Memory

In some embodiments, a compute and related memory may be initially statically allocated, but then the memory may be dynamically allocated, re-allocated, and/or de-allocated. During a movement (re-allocation)749of memory750, as illustrated inFIG. 58, the re-allocation of the memory in the memory pool122is handled by the DMA660and the MMU222engines where the DMA660moves the memory750and the MMU222updates the logical to physical address mappings. In some embodiments, the spatial fabric initiates the DMA660operations and updates the MMUs222. By performing the re-allocation of the memory750, the memory750may be moved closer to a corresponding compute752.

FIG. 59shows an allocation, re-allocation, and de-allocation of memory in programmable logic device12. In a diagram754A, an FMIB520A dynamically requests memory beyond statically allocated resources760. In a diagram754B, the programmable logic device12then dynamically allocates memory762. In some embodiments, the dynamic allocation of the memory762is handled by a runtime system that monitors the available memory and manages the allocation, re-allocation, and de-allocation of the memory762. In some embodiments, the dynamic runtime system is implemented as soft logic in the spatial fabric and/or as hard logic in the spatial fabric (e.g. hard processor system (HPS)). In some embodiments, the dynamic runtime system is implemented on a third die that communicates with the 3D spatial device to provide these runtime services. In some embodiments, the runtime system re-allocates the memory and/or de-allocates the memory. As illustrated in diagram754C, in a re-allocation, the system may re-allocate the memory762to memory764as long as used and free memories in the memory pool122are tracked. Once the allocated memory764has been used and is no longer to be used by the FMIB520A, the FMIB520A may de-allocate the memory764to free memory766as illustrated in diagram754D.

In some embodiments, the compute752and the memory750are initially statically allocated, but the compute752(rather than the memory750) is later dynamically allocated, re-allocated, and de-allocated. In some embodiments, moves (re-allocates)770of the compute752are performed as illustrated inFIG. 60. In some embodiments, the move (re-allocation)770of compute is performed using the NOC100of the base die24(or other memory interconnects). This move770of the compute752results in the memory750being close to the compute752to within a threshold distance to improve efficiency of the programmable logic device12.

FIG. 61illustrates diagrams771of movements of the compute752. In diagram771A, an FMIB520B requests a memory772. The system dynamically re-allocates the compute752to the memory772in the diagram751B.

In some embodiments, the dynamic allocation is handled by the runtime system that monitors the available compute resources and manages the allocation, re-allocation, and de-allocation of the computer resources. In some embodiments, the dynamic runtime system is implemented as soft logic and/or hard logic in the spatial fabric (e.g. HPS). In some embodiments, the dynamic runtime system is implemented on a third die that communicates with the 3D spatial device to provide these runtime services. In some embodiments, the runtime system re-allocates the compute752and/or de-allocates the compute752.

During the diagram771B, the FMIB520B is to use a memory774. The system dynamically re-allocates the compute to the memory774. Once the operations for the FMIB520are completed, the compute752may be de-allocated, as illustrated in diagram771D.

FIG. 62illustrates a diagram780and data flow782for a spatial fabric using sector-aligned PR personas F5, F6, F7, and F8and the memory of the base die24for communication between the personas. As also shown, in some embodiments, the communication between the personas is done using FMIBs520communicating through the base die24using its memory. In some embodiments, the communication is done directly between multiple FMIBs520in the fabric die22and/or via the NOC100of the base die24.

FIG. 63illustrates a diagram784and data flow786for a spatial fabric using the sector-aligned PR personas F5, F6, F7, and F8with PR personas F1, F3, F4, F5, F6, F7, F8, and F9stored in the memory of the base die24.FIG. 64illustrates a diagram788and data flow790for a spatial fabric using the sector-aligned PR personas F5, F6, F7, and F8with PR personas F1, F3, F4, F5, F6, F7, F8, and F9stored in the memory of the base die24. Furthermore, in the illustrated diagram788, the PR persona storage is mixed with user memory storage and interconnects in the base die24.

As previously noted, the PR personas may be loaded into the fabric to change personas over time. Indeed, multiple copies of the same personas may be loaded into the fabric.FIG. 65illustrates a diagram792and a data flow794for a fabric die22loaded with a different composition. Specifically, in the illustrated embodiment, four copies of the F4persona and two copies of the F5persona are loaded into the fabric die22. In other words, the spatial fabric allows relocation of the personas to allow one copy of the persona to be copied to multiple locations in the spatial fabric, and computation is dynamically unrolled to match the design for the fabric.

As previously noted, in some embodiments, the spatial fabric allows the relocation of sector-aligned computes. In some embodiments, each sector48provides the same interfaces to the base die24, and the sector-aligned compute personas may align to a region containing multiple sectors. In some embodiments with relocation, the sector-aligned personas may be compiled a single time. Further, the programmable logic fabric12may have a large number of possible compositions using the 9 personas. For example, a 3×3 sector spatial fabric with 9 sector-aligned personas may have a number of combinations with repetition equal to 24,310 different compositions. In some embodiments, after the one static compilation, dynamic composition as described previously composes the 9 personas as the system demands. As previously discussed, these personas may be stored in the base die24. The personas may be precompiled before runtime of the programmable logic device. Additionally or alternatively, a runtime system, as previously discussed, may dynamically compose the personas.

In some situations and as previously noted, the programmable logic device12may copy compute and/or memory. As illustrated inFIG. 66, a copy of base memory800in a first period802is made to create additional copies of the base memory800in a second period804is performed using the DMA660in the base die24. In some embodiments, a copy of a compute806is performed by copying a corresponding persona from a resident copy in the base die24to be deployed in the second period804. In some embodiments, the copy is done without the corresponding persona being stored in the base die24by copying the compute806directly. Furthermore, relative positions of the copied compute806and it respective base memory800are maintained through copying. For example, each copy of the compute806may be located in a sector48adjacent to a sector directly above the respective base memory800.

Specifically, a compute may be copied to unroll the computation while copying the memory to the base die24.FIG. 67illustrates such a copy of a compute810from a first period812to have multiple copies in a second period814. Memories816are copied to the base die24without changing a compute818and corresponding memories820. The copy and unroll of the compute810along with copying the memories816enable parallelization of the computation810.

As previously discussed, both computes and memory may be relocated using moving. In addition to performing the moving separately, the computes and memory may be moved in parallel. For instance,FIG. 68shows a move of a compute822and related memory824at a first time826. The move of the822and the related memory824are relocated at a later time828. Furthermore, as illustrated, the compute822and the related memory824may be kept in same relative positions to each other before and after the move. Additionally or alternatively, the relative positions between the compute822and the related memory824may change during the moving and relocating.

Over time compute and/or memory may become scattered due to the dynamic allocation, re-allocation, and de-allocation in the system resulting in a fragmentation of the spatial computes and/or base memories. As illustrated inFIG. 69, a fragmented fabric832may undergo a de-fragmentation to form a de-fragmented fabric834. As illustrated, the de-fragmentation aligns computes836with the respective memories838to improve performance, power, and availability of computes836and memories838for more tasks. In some embodiments, the compute836is defragmented independently of the memory838, the memory838is de-fragmented independently of the compute836, and/or the compute836and the memory838are de-fragmented simultaneously.

Memory and related computes may be allocated, re-allocated, and/or de-allocated one at-a-time in either order. Additionally or alternatively, the memory and related computes may be allocated, re-allocated, and/or de-allocated simultaneously. For example,FIG. 70illustrates a simultaneous allocation of a compute840and a memory842after a compute844and a respective memory846has previously been allocated.FIG. 71illustrates a simultaneous de-allocation of the compute844and the respective memory846.

Although much of the foregoing discusses and/or illustrates the fabric of the fabric die22vertically above the memory of the base die24that is used by the fabric die22, memory in the memory die24that is not vertically below the fabric die22may also be used by the fabric. For instance, as illustrated inFIG. 72, vertical memory900in the base die24below the fabric die22may be used by the fabric die22using deterministic (e.g., fine or medium aggregations) or via shared usage. Additional memory902not vertically below the fabric die22. This additional memory902may be accessed via the NOC of the base die24.

The methods and devices of this disclosure may be incorporated into any suitable circuit. For example, the methods and devices may be incorporated into numerous types of devices such as microprocessors or other integrated circuits. Exemplary integrated circuits include programmable array logic (PAL), programmable logic arrays (PLAs), field programmable logic arrays (FPLAs), electrically programmable logic devices (EPLDs), electrically erasable programmable logic devices (EEPLDs), logic cell arrays (LCAs), field programmable gate arrays (FPGAs), application specific standard products (ASSPs), application specific integrated circuits (ASICs), and microprocessors, just to name a few.

The embodiments set forth in the present disclosure may be susceptible to various modifications and alternative forms, specific embodiments have been shown by way of example in the drawings and have been described in detail herein. However, it may be understood that the disclosure is not intended to be limited to the particular forms disclosed. The disclosure is to cover all modifications, equivalents, and alternatives falling within the spirit and scope of the disclosure as defined by the following appended claims. In addition, the techniques presented and claimed herein are referenced and applied to material objects and concrete examples of a practical nature that demonstrably improve the present technical field and, as such, are not abstract, intangible or purely theoretical. Further, if any claims appended to the end of this specification contain one or more elements designated as “means for [perform]ing [a function] . . . ” or “step for [perform]ing [a function] . . . ” it is intended that such elements are to be interpreted under 35 U.S.C. 112(f). For any claims containing elements designated in any other manner, however, it is intended that such elements are not to be interpreted under 35 U.S.C. 112(f).

APPENDIX

a fabric die including a programmable fabric; and

a base die that vertically overlaps the fabric die, wherein the base die includes base memory, the programmable fabric and the base memory each include respective portions that have columnar alignment, and a capacity of the base memory and a bandwidth between the fabric die and the base die is selectively allocated to different granularity memory organizations.

2. The programmable logic device of statement 1, wherein the portion of the programmable fabric includes a sector of the programmable fabric, and the portion of the base die includes support circuitry for the sector.

3. The programmable logic device of statement 1, wherein the allocated bandwidth between the base die and the fabric die may provide a latency deterministic direct access between portions of the fabric die and the base die, wherein the latency deterministic direct access includes a logical address space of the programmable fabric being the same as a physical address space of the base memory.
4. The programmable logic device of statement 3, wherein the latency deterministic direct access includes access to different subsets of the portion of the base memory of the base die where at least some of the base memory is inaccessible.
5. The programmable logic device of statement 3, wherein the latency deterministic direct access includes access to different subsets of the portion of the base memory of the base die where the entire base memory of the base die is accessible.
6. The programmable logic device of statement 5, wherein accessing different subsets of the portion of the base memory or accessing the same subsets from different portions of the programmable fabric have different latencies.
7. The programmable logic device of statement 1, wherein allocated bandwidth provides latency non-deterministic access to multiple portions of the base memory.
8. The programmable logic device of statement 7, wherein the non-deterministic access supports virtualized access where logical addresses from the programmable fabric are translated into physical addresses in the base memory.
9. The programmable logic device of statement 1, including a plurality of access ports between the fabric die and the base die, wherein the plurality of access ports include:

a plurality of widths; or

a plurality of communication protocols.

10. The programmable logic device of statement 1, including a plurality of access ports between the fabric die and the base die, wherein the plurality of access ports provide exclusive access to the base memory.

11. The programmable logic device of statement 1, wherein transfer of data between the base memory and in-die memory in the fabric die are transferred through inter-die connections without using fabric resources between portions of the programmable fabric.

12. The programmable logic device of statement 11, wherein the inter-die connections include a network on chip of the base die.

allocating a plurality of computes to respective portions of a programmable fabric of a fabric die of a programmable logic device;

allocating corresponding memory to respective portions of base memory of a base die of the programmable logic device; and

moving memory in the base memory without involvement of the programmable fabric.

14. The method of statement 13, wherein the portions of the programmable fabric include sectors of the programmable fabric, and the portions of the base memory include support circuitries for the sectors having one or more memory blocks.

15. The method of statement 13, including using the programmable fabric to aggregate capacity of the base memory.

16. The method of statement 15, wherein aggregating capacity includes accessing the portion of the base memory through a single port or accessing multiple portions of the base memory using multiple ports within a portion of the programmable fabric.

17. The method of statement 13, wherein moving the memory in base memory includes using direct memory access operations (DMA) without involvement of the fabric die.

18. The method of statement 17, wherein the DMA operations include copying data to different portions of the base memory, broadcasting copy data to different portions of the base memory, a gather of data from the base memory, a scatter of data to the base memory, or a movement of data with transformation computations using the DMA.
19. A method, including:

allocating, in a static allocation before run time of a programmable logic device, a compute to a respective portion of a programmable fabric of a fabric die of the programmable logic device;

assigning, in the static allocation before run time, one or more portions of base memory of a base die of the programmable logic device to the compute via a multi-ported connection between the fabric die and the base die;

moving memory in the base memory without involvement of the programmable fabric; and

using a memory management unit of the base die to provide consistency or coherency of the multi-ported connection between the fabric die and the base die.

20. The method of statement 19, including aggregating capacity within the one or more portions of the base memory using the programmable fabric via one or more ports from a respective portion of the programmable fabric.

mapping an implementation of a design in a programmable fabric on a fabric die to a fabric microbump interface of the fabric die;

mapping the fabric microbump interface to one or more memory arrays in a base die; and

associating the one or more memory arrays to in-die memory on the fabric die.

22. The method of statement 21, including using the in-die memory to perform memory paging between the one or more memory arrays and the in-die memory.

23. The method of statement 21, including receiving the association received from a user or administrator, received from hardware, or received from software.

24. The method of statement 21, including using the in-die memory to perform a bulk transfer between the one or more memory arrays and the in-die memory.

25. The method of statement 21, wherein mapping the fabric microbump interface to the one or more memory arrays in the base die includes mapping the fabric microbump interface to a respective base microbump interface of the base die.

26. The method of statement 21, wherein the fabric microbump interface includes an offset that provides a virtual starting address for the one or more memory arrays.

27. The method of statement 21, wherein mapping the implementation includes assigning location of the implementation using a compiler.

28. The method of statement 27, wherein the location of the implementation is assigned based on an associated location in the design.

29. The method of statement 27, wherein the location of the implementation is assigned based on an inference from the designs and one or more other designs in the programmable fabric.

30. The method of statement 21, wherein mapping the fabric microbump interface includes mapping a direct access between the fabric microbump interface and the one or more memory arrays based at least in part on a forced direct access between the fabric microbump interface and the one or more memory arrays based on a specified latency in the design.
31. The method of statement 21, including storing the mapping of the fabric microbump interface to the one or more memory arrays in a memory management unit of the base die.
32. The method of statement 31, including translating, in the memory management unit, an address from the fabric microbump interface to the one or more memory arrays.
33. The method of statement 31, wherein storing the mapping in the memory management unit includes storing the mapping in an address translation unit of the memory management unit.
34. The method of statement 33, including translating, in the address translation unit, an address from the fabric microbump interface to the one or more memory arrays.
35. Tangible, non-transitory, and computer-readable media having instructions stored thereon, that when executed by a processor, are configured to cause the processor to:

compile a configuration of a programmable logic device having a fabric die and a base die by:mapping a plurality of implementations of designs in a programmable fabric on the fabric die to a plurality of fabric microbump interfaces of the fabric die;mapping the plurality of fabric microbump interfaces to one or more memory arrays in the base die;mapping the one or more memory arrays to in-die memory on the fabric die; and

storing the configuration in the programmable logic device.

36. The tangible, non-transitory, and computer-readable media of statement 35, wherein the instructions are configured to cause the processor to store offsets in the plurality of fabric microbump interfaces to provide virtual starting addresses for the one or more memory arrays.
37. The tangible, non-transitory, and computer-readable media of statement 36, wherein the offsets support mapping a single fabric microbump interface to multiple of the one or more memory arrays.
38. The tangible, non-transitory, and computer-readable media of statement 35, wherein the instructions are configured to cause the processor to store the mappings of the plurality of fabric microbump interfaces to the one or more memory arrays in a memory management interface of the base die.
39. An electronic device, including:

a fabric die having a programmable fabric;

a base die, including:plurality of compute accelerators that perform arithmetic operations;memory;a direct memory access that:scatters data to a subset of the memory; andgathers computed data from the subset of the memory after the arithmetic operations are performed on the data; anda controller that:loads weights into the memory to control how the arithmetic operations are computed; andtransmit the gathered computed data.
40. The electronic device of statement 39, wherein the memory is interspersed with the plurality of compute accelerators.
41. The electronic device of statement 39, wherein the base die includes a direct memory access compute, wherein gathering the computed data includes performing computations on the gathered data using the direct memory access compute.
42. The electronic device of statement 39, wherein transmitting the gathered data includes transmitting the data from a base microbump interface of the base die via the controller.
43. A programmable fabric device, including:

a fabric die having a programmable fabric including:a plurality of partial reconfiguration regions each corresponding to a design for the programmable fabric, wherein the partial reconfiguration regions of the plurality of partial reconfiguration regions are aligned to sectors of the programmable fabric;a plurality of external sectors outside of the plurality of partial reconfiguration regions; andfabric resources that couple the external sectors to adjacent sectors of the plurality of the partial reconfiguration regions; and

a base die coupled to the external sectors and that provides interconnection between the external sectors.

44. The programmable fabric device of statement 43, wherein the external sectors enable communications between regions using external paths outside of the plurality of partial reconfiguration regions.

45. The programmable fabric device of statement 44, wherein communications between partial reconfiguration regions of the plurality of partial reconfiguration regions uses at least one of the plurality of external sectors.

46. The programmable fabric device of statement 43, wherein communications between partial reconfiguration regions of the plurality of partial reconfiguration regions uses a network on chip of the base die.

47. The programmable fabric device of statement 43, wherein background partial reconfiguration personas for the plurality of partial reconfiguration region are stored in the base die.

48. The programmable fabric device of statement 43, wherein the plurality of partial reconfiguration regions are reconfigured using a configuration write.

49. The programmable fabric device of statement 43, wherein the programmable fabric includes static routes within a partial reconfiguration region of the plurality of partial reconfiguration regions.

50. The programmable fabric device of statement 43, wherein communications between the plurality of partial reconfiguration regions uses a soft logic network on chip in the fabric die or a hardened network-on-chip in the fabric die.

51. The programmable fabric device of statement 50, wherein the soft logic network on chip is tolerant of disappearing sections during a partial reconfiguration of the programmable fabric.

52. The programmable fabric of statement 43, wherein communications between the plurality of partial reconfiguration regions utilizes connections between a fabric microbump interface of the fabric die and a base microbump interface of the base die.

loading a plurality of partial reconfiguration personas into a programmable fabric of a fabric die of programmable logic device, wherein the plurality of partial reconfiguration personas are aligned to sectors of the programmable fabric;

loading a background partial reconfiguration persona into a base die of the programmable logic device while performing operations using the plurality of partial reconfiguration personas; and

loading the background partial reconfiguration persona into the programmable fabric from the base die.

54. The method of statement 53, wherein loading the background partial reconfiguration persona includes loading the background partial reconfiguration persona into the programmable fabric using a base microbump interface of the base die.

55. The method of statement 54, wherein loading the background partial reconfiguration persona includes loading the background from base microbump interface of the base die via a fabric microbump interface of the fabric die.

56. The method of statement 53, wherein communications between a plurality of partial reconfiguration regions loaded with the plurality of partial reconfiguration personas includes pathways around the loaded plurality of partial reconfiguration regions.

57. The method of statement 56, wherein the pathways include connections through the base die.

58. The method of statement 57, wherein the connections through the base die include a network on chip of the base die.

a programmable fabric die having a programmable fabric of programmable elements that is sequentially configured using a sequence of partial reconfiguration personas that control how the programmable elements are programmed for one or more portions of the programmable fabric; and

a base die having one or more memory blocks that store a first subset of the partial reconfiguration personas for loading into the fabric die at a future time while a second subset of the partial reconfiguration personas are used to perform an operation in the programmable fabric.

60. The programmable fabric device of statement 59, wherein the sequence includes a static sequence of the partial reconfiguration personas.

61. The programmable fabric device of statement 59, wherein the partial reconfiguration personas are alighted to sectors of the programmable fabric.

62. The programmable fabric device of statement 59, wherein the sequence of the partial reconfiguration personas includes at least one of the partial reconfiguration personas occurring more than once in the sequence.

allocating a compute to a portion of a programmable fabric of a fabric die of a programmable logic device;

allocating a first portion of memory in a base die to the compute; and

moving the allocation of the first portion of the memory to a second portion of the memory of the base die, wherein the second portion is closer to the compute in the fabric die than the first portion.

64. The method of statement 63, wherein the portion of the programmable fabric of the fabric die includes a sector of the programmable fabric.

65. The method of statement 63, wherein the second portion of the memory includes support circuitry in the base die.

66. The method of statement 65, wherein the second portion of the support circuitry in the base die located directly beneath the relevant compute portion of the programmable fabric.

67. The method of statement 63, including receiving a request from software, firmware, a fabric microbump interface, or soft logic for memory beyond statically allocated resources.

68. The method of statement 67, including, in response to receiving the request, allocating a third portion of the memory in the base die to the fabric microbump interface.

69. The method of statement 68, wherein a runtime system of the programmable logic device receives the request and allocates the third portion of the memory.

70. The method of statement 69, wherein the runtime system is located on a third die of the programmable logic device.

71. The method of statement 63, including tracking used and free memories in a memory pool of the base die.

72. The method of statement 71, including determining that a third portion of the memory is no longer being used by a corresponding fabric microbump interface.

73. The method of statement 72, including, response to the determination that the third portion of the memory is no longer being used, de-allocate the memory to free memory for other computes.

a fabric die having a programmable fabric, wherein a compute is allocated to a first portion of the programmable fabric that performs operations in the programmable fabric using programmable elements of the programmable fabric;

a base die located below the fabric die and having base memory with a portion of the base memory allocated to the compute; and

a third die executing a run-time system management that moves the allocation of the compute from the first portion to a second portion of the programmable fabric based at least in part on the second portion of the programmable fabric being closer to the portion of the base memory than the first portion of the programmable fabric.

75. The programmable logic device of statement 74, wherein the first portion of the programmable fabric includes a first sector, and the second portion of the programmable fabric includes a second sector.

76. The programmable logic device of statement 74, wherein the third die receiving a request from a fabric microbump interface for memory beyond statically allocated resources.

77. The programmable logic device of statement 76, wherein the third die, in response to receiving the request, allocates a second portion of the memory in the base die to the fabric microbump interface.

78. The programmable logic device of statement 77, wherein the third die tracks used and free portions of the memory in the base die.

79. The programmable logic device of statement 78, wherein the third die:

determines that a third portion of the memory is no longer being used by a corresponding fabric microbump interface; and

de-allocates a compute associated with the fabric microbump interface.

allocating a compute to a first portion of a programmable fabric of a fabric die of a programmable logic device;

allocating a base memory of a base die of the programmable logic device to the compute;

copying the compute to a second portion of the programmable fabric; and

copying the base memory to a copy base memory of the base die that is closer to the second portion of the programmable fabric.

81. The method of statement 80, wherein copying the compute to maintain a relative position between the base memory and the compute for the copy base memory and the second portion after the copy.

82. The method of statement 80, wherein copying the compute to the second portion includes copying a corresponding persona from a resident copy in the base die.