Patent Description:
A heterogeneous computing platform (HCP) refers to a data processing system that includes a host processor coupled to one or more other devices through interface circuitry. The devices typically differ from the host processor architecturally. The host processor is capable of offloading tasks to the devices. The devices are capable of performing the tasks and making the results available to the host processor. As an illustrative example, the host processor is typically implemented as a central processing unit while the devices are implemented as graphics processing units (GPUs) and/or digital signal processors (DSPs).

In other HCPs, one or more of the devices that perform tasks offloaded from the host processor include devices adapted for hardware acceleration (referred to as "hardware accelerators"). The hardware accelerators include circuitry that is capable of performing a task offloaded from the host as opposed to executing software or program code to perform the task. The circuitry of the hardware accelerator is functionally equivalent to executing software, but is typically able to complete the task in less time.

Examples of hardware accelerators include programmable integrated circuits (ICs) such as field programmable gate arrays (FPGAs), partially programmable ICs, application specific ICs (ASICs), and so forth. Appreciably, an HCP may include a combination of different devices where one or more are adapted to execute program code and one or more others are adapted for hardware acceleration. Document "Groute: An Asynchronous Multi-GPU Programming Model for Irregular Computations" refers to asynchronous Multi-GPU Programming Model for Irregular Computations and proposes constructs for asynchronous multi-GPU programming, and describes their implementation in a thin runtime environment called Groute. Document "NVIDIA TESLA P <NUM>, The most advanced datacenter accelerator ever built" is a whitepaper about NVIDIA TESLA P100, a general-purpose graphics processing unit for high-performance computing market.

In an embodiment according to the invention as defined in claim <NUM> a system includes a host processor coupled to a communication bus, a first hardware accelerator communicatively linked to the host processor through the communication bus, and a second hardware accelerator communicatively linked to the host processor through the communication bus. The first hardware accelerator and the second hardware accelerator are directly coupled through an accelerator link independent of the communication bus. The host processor is configured to initiate a data transfer between the first hardware accelerator and the second hardware accelerator directly through the accelerator link.

In an embodiment according to the invention as defined in claim <NUM> a method includes receiving, within a first hardware accelerator, an instruction and a target address for a data transfer sent from a host processor over a communication bus, the first hardware accelerator comparing the target address with an upper bound of an address range corresponding to the first hardware accelerator, and, in response to determining that the target address exceeds the address range based on the comparing, the first hardware accelerator initiating a transaction with a second hardware accelerator to perform a data transfer using an accelerator link that directly couples the first hardware accelerator and the second hardware accelerator.

This Summary section is provided merely to introduce certain concepts and not to identify any key or essential features of the claimed subject matter. Other features of the inventive arrangements will be apparent from the accompanying drawings and from the following detailed description.

The inventive arrangements are illustrated by way of example in the accompanying drawings. The drawings, however, should not be construed to be limiting of the inventive arrangements to only the particular implementations shown. Various aspects and advantages will become apparent upon review of the following detailed description and upon reference to the drawings.

While the disclosure concludes with claims defining novel features, it is believed that the various features described within this disclosure will be better understood from a consideration of the description in conjunction with the drawings.

This disclosure relates to hardware acceleration and, more particularly, to facilitating use of multiple hardware accelerators through a unified address space and low latency communication links. Using hardware accelerators with data processing systems has become an effective technique for offloading tasks from the host processor thereby reducing the workload on the host processor. The hardware accelerators are typically attached to the host processor through a bus. For example, a hardware accelerator may be attached to a circuit board that is inserted into an available bus slot of the host system. Typically, each hardware accelerator is attached to a corresponding circuit board. Adding an additional hardware accelerator to a system usually entails inserting an additional circuit board with the hardware accelerator into an available bus slot.

Within conventional systems, applications executed by the host processor must be updated and/or rewritten to specifically access any newly added hardware accelerators (e.g., by hardware address). Further, to transfer data from one hardware accelerator to another, the data is moved from the source hardware accelerator to the host processor, and then from the host processor down to the target hardware accelerator. The data moves to and from each hardware accelerator through the host processor via the bus. As such, each additional hardware accelerator added to a system increases the number of devices on the bus thereby creating contention for bandwidth on the bus. As the complexity, number, and/or size of tasks performed by hardware accelerators (or other devices) increases, available bandwidth on the bus is further constrained.

In accordance with the inventive arrangements described within this disclosure, a unified address space for devices is provided. Further, direct communication links between hardware accelerators, referred to herein as "accelerator links", are provided that are capable of operating independently of the bus. A runtime library and driver executed by the host are capable of leveraging the unified address space so that applications executed by the host processor may operate without directly referencing (e.g., addressing) particular hardware accelerators in the system. The runtime library is capable of determining the proper addresses to use to effectuate data transfers among hardware accelerators. As such, the applications need not be modified to access additional hardware accelerators that may be added to the system. Further, data transfers may be performed over the accelerator links allowing data to be transferred directly from one hardware accelerator to another without passing through the host processor effectively bypassing the bus. As such, the bandwidth used by hardware accelerators on the bus may be significantly reduced, thereby increasing overall system performance.

As noted, additional hardware accelerators can be added to a system using the existing address space without requiring a corresponding change or modification to the program code (e.g., applications) executed by the host processor. This is supported, at least in part, through implementation of an automated discovery process for hardware accelerator boards and of adding such boards to the system, use of remote versus local buffer flags, automated switching to accelerator links for data transfers in at least some cases, and automated address translation for remote buffers.

Further aspects of the inventive arrangements are described below in greater detail with reference to the figures. For purposes of simplicity and clarity of illustration, elements shown in the figures have not necessarily been drawn to scale. Further, where considered appropriate, reference numbers are repeated among the figures to indicate corresponding, analogous, or like features.

<FIG> illustrates an example of a system <NUM> with multiple hardware accelerators. System <NUM> is an example of computer hardware that may be used to implement a computer, a server, or other data processing system. System <NUM> is also an example of a heterogeneous computing system. As pictured, system <NUM> includes at least one host processor <NUM> coupled to host memory <NUM> through interface circuitry <NUM>.

System <NUM> also includes a plurality of hardware accelerators <NUM>. In the example of <FIG>, system <NUM> includes three hardware accelerators <NUM>-<NUM>, <NUM>-<NUM>, and <NUM>-<NUM>. While the example of <FIG> illustrates three hardware accelerators, it should be appreciated that system <NUM> may include fewer than three hardware accelerators or more than three hardware accelerators. Further, system <NUM> may include one or more other devices such as graphics processing units (GPUs) or digital signal processors (DSPs).

System <NUM> is capable of storing computer readable instructions (also referred to as "program code") within host memory <NUM>. Host memory <NUM> is an example of computer readable storage media. Host processor <NUM> is capable of executing the program code accessed from host memory <NUM> via interface circuitry <NUM>. In one or more embodiments, host processor <NUM> communicates with host memory <NUM> through a memory controller (not shown).

Host memory <NUM> may include one or more physical memory devices such as, for example, a local memory and a bulk storage device. Local memory refers to non-persistent memory device(s) generally used during actual execution of program code. Examples of local memory include random access memory (RAM) and/or any of the various types of RAM that are suitable for use by a processor during execution of program code such as DRAM, SRAM, DDR SDRAM, and the like. A bulk storage device refers to a persistent data storage device. Examples of bulk storage devices include, but are not limited to, a hard disk drive (HDD), a solid-state drive (SSD), flash memory, a read-only memory (ROM), an erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or other suitable memory. System <NUM> may also include one or more cache memories (not shown) that provide temporary storage of at least some program code to reduce the number of times program code must be retrieved from a bulk storage device during execution.

Host memory <NUM> is capable of storing program code and/or data. For example, host memory <NUM> may store an operating system <NUM>, instructions <NUM>, and data <NUM>. In the example of <FIG>, instructions <NUM> may include one or more applications <NUM>, a runtime library (referred to herein as the "runtime") <NUM>, and a driver <NUM> that is capable of communicating with hardware accelerators <NUM>. Runtime <NUM> is capable of handling completion events, managing command queues, and providing notifications to application(s) <NUM>. Data <NUM>, among other types of data items, may include buffer objects such as buffer objects <NUM> and <NUM>, which facilitate direct data transfers between hardware accelerators <NUM>. Buffer object <NUM> includes a remote flag <NUM>, while buffer object <NUM> includes a remote flag <NUM>. For purposes of illustration remote flag <NUM> is not set, while remote flag <NUM> is set. System <NUM>, e.g., host processor <NUM>, is capable of executing operating system <NUM> and instructions <NUM> to perform the operations described within this disclosure.

Examples of interface circuitry <NUM> include, but are not limited to, a system bus and an input/output (I/O) bus. Interface circuitry <NUM> may be implemented using any of a variety of bus architectures. Examples of bus architectures may include, but are not limited to, Enhanced Industry Standard Architecture (EISA) bus, Accelerated Graphics Port (AGP), Video Electronics Standards Association (VESA) local bus, Universal Serial Bus (USB), and Peripheral Component Interconnect Express (PCle) bus. Host processor <NUM> may be coupled to host memory <NUM> through different interface circuitry than is used to couple to hardware accelerators <NUM>. For purposes of illustration, an endpoint for interface circuitry <NUM> through which host processor <NUM> communicates with other devices is not shown.

System <NUM> further may include one or more other I/O devices (not shown) coupled to interface circuitry <NUM>. The I/O devices may be coupled to system <NUM>, e.g., interface circuitry <NUM>, either directly or through intervening I/O controllers. Examples of I/O devices include, but are not limited to, a keyboard, a display device, a pointing device, one or more communication ports, and a network adapter. A network adapter refers to circuitry that enables system <NUM> to become coupled to other systems, computer systems, remote printers, and/or remote storage devices through intervening private or public networks. Modems, cable modems, Ethernet cards, and wireless transceivers are examples of different types of network adapters that may be used with system <NUM>.

In the example of <FIG>, each of hardware accelerators <NUM>-<NUM>, <NUM>-<NUM>, and <NUM>-<NUM> is coupled to a memory <NUM>-<NUM>, <NUM>-<NUM>, and <NUM>-<NUM>, respectively. Memories <NUM>-<NUM>, <NUM>-<NUM>, and <NUM>-<NUM> are implemented as RAMs as generally described in connection with host memory <NUM>. In one or more embodiments, each hardware accelerator <NUM> is implemented as an IC. The IC may be a programmable IC. An example of a programmable IC is a Field Programmable Gate Array (FPGA).

In the example of <FIG>, each of hardware accelerators <NUM> includes an endpoint <NUM>, a link circuit <NUM>, a memory controller (abbreviated "MC" in <FIG>) <NUM>, and interconnect circuitry <NUM>. Each hardware accelerator <NUM> also includes one or more compute units (abbreviated "CU" in <FIG>). A compute unit is a circuit that is capable of performing the tasks offloaded from host processor <NUM>. For purposes of illustration, each of hardware accelerators <NUM> is shown to include a compute unit <NUM> and a compute unit <NUM>. It should be appreciated that hardware accelerators <NUM> may include fewer or more compute units than shown.

In one example, each of endpoints <NUM> is implemented as a PCle endpoint. It should be appreciated that endpoints <NUM> may be implemented as any type of endpoint suitable for communicating over the particular type or implementation of interface circuitry <NUM> that is used by system <NUM>. Each of memory controllers <NUM> is coupled to a respective memory <NUM> to facilitate access (e.g., reading and writing) of that memory <NUM> by hardware accelerator <NUM>.

In one or more embodiments, hardware accelerator <NUM>-<NUM> and memory <NUM>-<NUM> are attached to a first circuit board (not shown), hardware accelerator <NUM>-<NUM> and memory <NUM>-<NUM> are attached to a second circuit board (not shown), and hardware accelerator <NUM>-<NUM> and memory <NUM>-<NUM> are attached to a third circuit board (not shown). Each of these circuit boards may include suitable connectors for coupling to a bus port or slot. For example, each of the circuit boards may have a connector configured for insertion into an available PCle slot (or other bus/interface connector) of system <NUM>.

Each of link circuits <NUM> is capable of establishing an accelerator link with at least one other, e.g., neighboring, link circuit <NUM>. As used herein, an "accelerator link" refers to a communication link that directly connects two hardware accelerators. For example, each of the circuit boards having a hardware accelerator <NUM> may be coupled through wires that connect to link circuits <NUM>. Link circuits <NUM> may establish the accelerator links over the wires.

In particular embodiments, link circuits <NUM> are communicatively linked using a ring topology. Data that is sent via the accelerator link(s) established by link circuits <NUM> master from left to right as indicated by the directional arrows. For example, referring to the example of <FIG>, the link circuit on the left (e.g., link circuit <NUM>-<NUM>) may operate as a master, while the neighboring link circuit to the right (e.g., link circuit <NUM>-<NUM>) may operate as a slave. Similarly, link circuit <NUM>-<NUM> may operate as a master with respect to link circuit <NUM>-<NUM>. Link circuit <NUM>-<NUM> may operate as a master with respect to link circuit <NUM>-<NUM>.

In one or more embodiments, each link circuit <NUM> includes a table or register specifying the amount (or size) of memory <NUM> for each hardware accelerator (e.g., on each board). Using the table, each link circuit <NUM> is capable of modifying addresses specified in transactions for purposes of exchanging information using the accelerator links. In particular embodiments, the table or register is static. In one or more other embodiments, the driver is able to read and/or update the information stored in the table or register dynamically, e.g., at runtime.

For purposes of illustration, operation of hardware accelerator <NUM>-<NUM> is described. It should be appreciated that like numbered components in each respective hardware accelerator are capable of operating in the same or similar manner. Accordingly, referring to hardware accelerator <NUM>-<NUM>, link circuit <NUM>-<NUM> is capable of receiving a transaction from any of a variety of different sources or initiators and routing the transaction to any of a variety of targets. For example, link circuit <NUM>-<NUM> is capable of receiving a transaction from endpoint <NUM>-<NUM> (e.g., originating from host processor <NUM>), compute unit <NUM>-<NUM>, compute unit <NUM>-<NUM>, hardware accelerator <NUM>-<NUM> via link circuit <NUM>-<NUM>, or hardware accelerator <NUM>-<NUM> via link circuit <NUM>-<NUM> flowing to link circuit <NUM>-<NUM> and then on to link circuit <NUM>-<NUM>. Link circuit <NUM>-<NUM> is capable of routing the transaction to any target such as endpoint <NUM>-<NUM> (e.g., to host processor <NUM>), compute unit <NUM>-<NUM>, compute unit <NUM>-<NUM>, memory controller <NUM>-<NUM>, hardware accelerator <NUM>-<NUM> via link circuit <NUM>-<NUM> and on to link circuit <NUM>-<NUM>, or hardware accelerator <NUM>-<NUM> via link circuit <NUM>-<NUM>, where the target is different from the source or initiator.

For example, host processor <NUM> is capable of accessing any location in memory <NUM>-<NUM>, memory <NUM>-<NUM>, and/or memory <NUM>-<NUM> as part of the unified address space. In accessing such memories, however, host processor <NUM> may do so by accessing a selected hardware accelerator, e.g., hardware accelerator <NUM>-<NUM>, and then reaching any target such as memory <NUM>-<NUM>, memory <NUM>-<NUM>, or memory <NUM>-<NUM> through the selected hardware accelerator using the accelerator links.

As an illustrative and nonlimiting example, host processor <NUM> may initiate a data transfer involving hardware accelerators <NUM>-<NUM> and <NUM>-<NUM>. Hardware accelerator <NUM>-<NUM> may be the initiator. In this example, host processor <NUM>, e.g., runtime <NUM> and/or driver <NUM>, creates buffer object <NUM> corresponding to hardware accelerator <NUM>-<NUM> and buffer object <NUM> corresponding to hardware accelerator <NUM>-<NUM>. Host processor <NUM> sets remote flag <NUM> indicating that the target address for the data transfer (located in hardware accelerator <NUM>-<NUM>) is remote relative to the initiating hardware accelerator (hardware accelerator <NUM>-<NUM>).

Endpoint <NUM>-<NUM> is capable of receiving the task offloaded from host processor <NUM> via interface circuitry <NUM>. In one or more embodiments, host processor <NUM>, by way of executing runtime <NUM> and driver <NUM>, is capable of viewing hardware accelerators <NUM> as a unified address space. Endpoint <NUM>-<NUM> may provide the task (e.g., data) to compute unit <NUM>-<NUM>. The task may specify a target address within memory <NUM>-<NUM> from which compute unit <NUM>-<NUM> is to retrieve data for performing the offloaded task. Hardware accelerator <NUM>-<NUM>, using link circuit <NUM>-<NUM>, is able to initiate and perform the data transfer directly with hardware accelerator <NUM>-<NUM> by way of the accelerator link established between link circuit <NUM>-<NUM> and link circuit <NUM>-<NUM>.

While the data transfer may be initiated by host processor <NUM>, the data transfer is performed using link circuits <NUM> and occurs without involving host processor <NUM>, host memory <NUM>, or interface circuitry <NUM>. The data transfer occurs directly between the hardware accelerators. In conventional systems, the data transfer would occur by host processor <NUM> retrieving the data from hardware accelerator <NUM>-<NUM> via interface circuitry <NUM> and then providing the data to hardware accelerator <NUM>-<NUM> via interface circuitry <NUM>.

The ability of hardware accelerators <NUM> to read and write data among themselves without having that data travel through host processor <NUM> significantly reduces the amount of data passed over interface circuitry <NUM> (e.g., the PCle bus). This saves considerable bandwidth of interface circuitry <NUM> for use in conveying data between host processor <NUM> and other hardware accelerators <NUM>. Further, the speed of operation of system <NUM> may be increased due to the reduction in the time required for hardware accelerators <NUM> to share data.

System <NUM> may include fewer components than shown or additional components not illustrated in <FIG> depending upon the particular type of device and/or system that is implemented. In addition, the particular operating system, application(s), and/or I/O devices included may vary based upon system type. Further, one or more of the illustrative components may be incorporated into, or otherwise form a portion of, another component. For example, a processor may include at least some memory. System <NUM> may be used to implement a single computer or a plurality of networked or interconnected computers each implemented using the architecture of <FIG> or an architecture similar thereto.

<FIG> illustrates an example implementation of hardware accelerator <NUM>-<NUM> of <FIG>. Within <FIG>, an example implementation of link circuit <NUM>-<NUM> is provided. It should be appreciated that the architecture illustrated for link circuit <NUM>-<NUM> in <FIG> may be used to implement any of the link circuits <NUM> illustrated in <FIG>.

In one or more embodiments, link circuit <NUM>-<NUM> is capable of converting transactions that are to be sent to other hardware accelerators into data stream based packets and route the packets over the accelerator links established among link circuits <NUM>. In particular embodiments, link circuit <NUM>-<NUM> is capable of converting AMBA eXtensible Interface (AXI) compliant memory mapped transactions into AXI data streams for transmission. Within this disclosure, AXI is used as an example communication protocol. It should be appreciated that other communication protocols may be used. In this regard, use of AXI is intended for purposes of illustration and not limitation. Link circuit <NUM>-<NUM> is also capable of handling incoming packets from other hardware accelerators (e.g., hardware accelerators <NUM>-<NUM> and <NUM>-<NUM>), convert the packets into memory mapped transactions, and route the data locally within hardware accelerator <NUM>-<NUM>. Further, link circuit <NUM>-<NUM> is capable of converting received packets into memory mapped transactions, modifying the transaction, converting the memory mapped transaction into packets, and passing the packets to a next hardware accelerator. Data received via the accelerator links may be routed internally within hardware accelerator <NUM>-<NUM> as memory mapped transactions.

In the example of <FIG>, link circuit <NUM>-<NUM> includes transceivers <NUM> and <NUM>, retransmit engines (RTEs) <NUM> and <NUM>, and memory map to stream (MM-stream) mappers <NUM> and <NUM>. MM-stream mappers <NUM> and <NUM> are coupled to interconnect circuitry <NUM>.

As pictured, transceiver <NUM> may be coupled to a corresponding transceiver in hardware accelerator <NUM>-<NUM>, while transceiver <NUM> is coupled to a corresponding transceiver in hardware accelerator <NUM>-<NUM>. Transceivers <NUM> and <NUM> implement a physical layer of the accelerator links established with other hardware accelerators. Each of transceivers <NUM> and <NUM> is capable of implementing a lightweight, serial communications protocol for multi-gigabit communication links. In one or more embodiments, each of transceivers <NUM> and <NUM> is capable of implementing a bi-directional interface to a transceiver in a neighboring IC. Transceivers <NUM> and <NUM> are capable of automatically initializing the accelerator links with the other hardware accelerators. In general, transceivers <NUM> and <NUM> are capable of bi-directional communication to implement low level signaling and low PHY level protocols relating to flow control. Data flows, however, are implemented using a ring topology and flow from master to slave (e.g., in a single direction around the ring) as previously described.

For example, transceiver <NUM> is capable of communicating bi-directionally with a corresponding transceiver within link circuit <NUM>-<NUM> of hardware accelerator <NUM>-<NUM>. Transceiver <NUM> is capable of communicating bi-directionally with a corresponding transceiver within link circuit <NUM>-<NUM> of hardware accelerator <NUM>-<NUM>. Each of transceivers <NUM> and <NUM> is capable of communicating with a neighboring transceiver using data streams, e.g., AXI data streams.

In particular embodiments, transceivers <NUM> and <NUM> are capable of sending and receiving data to a neighboring hardware accelerator using 8B/10B coding rules. Each of transceivers <NUM> and <NUM> is capable of detecting single-bit and most multi-bit errors using the 8B/10B coding rules.

In one or more embodiments, each of transceivers <NUM> and <NUM> is implemented as an Aurora 8B/10B IP Core, which is available from Xilinx, Inc. of San Jose, California. It should be appreciated, however, that the particular core noted is provided for purposes of illustration and is not intended as a limitation. Other transceivers that are capable of operating as described herein may be used.

Transceiver <NUM> is coupled to RTE <NUM>. Transceiver <NUM> and RTE <NUM> are capable of communicating through a plurality of data streams running in each direction supporting bidirectional communication. Transceiver <NUM> is coupled to RTE <NUM>. Transceiver <NUM> and RTE <NUM> are capable of communicating through a plurality of data streams running in each direction supporting bidirectional communication.

RTEs <NUM> and <NUM> are capable of managing transactions. In one or more embodiments, RTE <NUM> and RTE <NUM> each implements additional layers of communication protocol upon those implemented by transceivers <NUM> and <NUM>, respectively. For example, RTE <NUM> and RTE <NUM> each implement a transaction layer (TL) / Link Layer (LL) and a user layer. These additional layers provide extra assurance regarding data integrity. After initialization, applications are able to pass data across the accelerator links as streams of data. The additional data integrity measures are particularly beneficial since control signals are merged with data when converting memory mapped transactions to stream data. A data integrity issue may result in corrupt control signals. On-chip interconnects and/or buses are intolerant of data loss with respect to the control signals.

The TL/LL implements a token based flow control to guarantee lossless data communication. In one or more embodiments, the communication channels between neighboring transceivers and between transceivers and RTEs are <NUM> bits in width. When sending data, each RTE is capable of checking that the receiving link circuit in the target hardware accelerator has sufficient buffering resources (e.g., a token) for receiving the entire transaction to be sent before actually sending the transaction to the physical layer implemented by the transceiver. For example, RTE <NUM> may check that receiving link circuit <NUM>-<NUM> in hardware accelerator <NUM>-<NUM> has sufficient buffer resources for receiving data prior to providing the data to transceiver <NUM> (within link circuit <NUM>-<NUM>) for sending.

RTEs <NUM> and <NUM> are capable of detecting data corruption. For example, each of RTEs <NUM> and <NUM> is capable of verifying packet length information, packet sequence information, and/or the Cyclic Redundancy Check (CRC) checksum for each packet that is received. When an RTE slave (e.g., receiving RTE) detects a packet error, the RTE may enter an error abort mode. In the error abort mode, the RTE drops the packet with the error as a failed packet. The RTE further drops all subsequent packets of the transaction. In particular embodiments, initiation of the error abort mode causes the RTE to launch a link retry sequence. Once the link retry sequence is successful, the link master (e.g., sending RTE) is able to re-cover the transmission by starting from the failing point.

RTE <NUM> is coupled to MM-stream mapper <NUM>. RTE <NUM> is capable of communicating with MM-stream mapper <NUM> via a plurality data streams running in each direction supporting bidirectional communication. RTE <NUM> is coupled to MM-stream mapper <NUM>. RTE <NUM> is capable of communicating with MM-stream mapper <NUM> via a plurality data streams running in each direction supporting bidirectional communication.

Each of MM-stream mapper <NUM> and MM-stream mapper <NUM> is coupled to interconnect circuitry <NUM>. Interconnect circuitry <NUM> is capable of routing data among the MM-stream mappers <NUM> and <NUM> as well as other master and/or slave circuits of hardware accelerator <NUM>-<NUM> coupled thereto. Interconnect circuitry <NUM> may be implemented as one or more on-chip interconnects. An example of an on-chip interconnect is an AXI bus. An AXI bus is an embedded microcontroller bus interface for use in establishing on-chip connections between circuit blocks and/or systems. Other example implementations of interconnect circuitry may include, but are not limited to, other buses, cross-bars, network on chips (NoCs), and so forth.

MM-stream mappers <NUM> and <NUM> are capable of converting received data streams from RTEs <NUM> and <NUM>, respectively, to memory mapped transactions that may be provided to interconnect circuit block <NUM>. In this regard, the data stream may be demultiplexed into multiple channels supporting memory mapped transactions. MM-stream mappers <NUM> and <NUM> are also capable of converting received memory mapped transactions from interconnect circuit block <NUM> to stream data that may be provided to RTE <NUM> and <NUM>, respectively. MM-stream mappers <NUM> and <NUM> are capable of multiplexing multiple channels supporting memory mapped transactions (e.g., including control signals as discussed) into a single data stream for sending to RTEs <NUM> and <NUM>, respectively.

In one or more embodiments, each of MM-stream mappers <NUM> and <NUM> is capable of adjusting a target address received in a transaction. MM-stream mapper <NUM>, for example, in receiving a transaction from hardware accelerator <NUM>-<NUM> via an accelerator link may subtract an upper bound of the address range for hardware accelerator <NUM>-<NUM> (e.g., the address range of memory <NUM>-<NUM>) from the target address of the transaction. By adjusting the target address as the transaction passes through link circuits <NUM>, a transaction may be directed from one hardware accelerator to another via the accelerator links. Further details relating to the operation of addresses in using the accelerator links are described in greater detail in connection with <FIG>.

For purposes of illustration, other portions of hardware accelerator <NUM>-<NUM> are described in relation to link circuit <NUM>-<NUM>. In the example of <FIG>, interconnect circuitry <NUM> is coupled to a direct memory access (DMA) master circuit <NUM>. DMA master circuit <NUM>, for example, includes a memory mapped interface for communicating with interconnect circuit block <NUM>. DMA master circuit <NUM> is coupled to PCle endpoint <NUM>. PCle endpoint <NUM>, which is an example implementation of endpoint <NUM>-<NUM> of <FIG>, is communicatively linked to host processor <NUM>.

In the example of <FIG>, interconnect circuitry <NUM> is also coupled to one or more compute unit masters <NUM>-<NUM> through <NUM>-N. Each compute unit master <NUM> provides a bidirectional interface between a compute unit implemented within hardware accelerator <NUM>-<NUM> and interconnect circuit block <NUM>. Each compute unit master <NUM> further includes a memory mapped interface for communicating with interconnect circuit block <NUM>. Each of compute unit <NUM>-<NUM> and compute unit <NUM>-<NUM> may be connected to interconnect circuitry <NUM> via a slave interface (not shown).

In the example of <FIG>, interconnect circuitry <NUM> is also coupled to one or more memory controller slave circuits <NUM>-<NUM> through <NUM>-N. Each memory controller slave circuit <NUM> facilitates read and write operations for memory <NUM>-<NUM>. Memory <NUM>-<NUM> may be implemented as one or more off-chip memories accessible by hardware accelerator <NUM>-<NUM>. Each of memory controllers <NUM>-<NUM> through <NUM>-N further includes a memory mapped interface for communicating with interconnect circuit block <NUM>.

<FIG> illustrates an example implementation of RTE <NUM>. The example architecture described in connection with <FIG> implements a credit-based flow control/retransmission control scheme using Flow Control Units (FLITs). RTE <NUM> is capable of translating between FLIT-based protocol and/or interface used internally to a protocol and/or interface that may be used by applications.

RTE <NUM> includes a transmit channel <NUM>. Transmit channel <NUM> is capable of decapsulating data (e.g., AXI) streams into FLIT-based transactions. In the example of <FIG>, transmit channel <NUM> includes a transmit (TX) packet Cyclic Redundancy Check (CRC) generator <NUM>, a Retry Pointer Return Command (PRET) Packet / Init Retry Command (IRTRY) Packet Generator and Return Retry Pointer (RRP) embedder <NUM>, a Token Return (TRET) packet generator and Sequence (SEQ) number / Forward Retry Pointer (FRP) / Return Token Count (RTC) embedder <NUM>, a flow control circuit <NUM>, and an output buffer <NUM>. TRET generator and SEQ/FRP/RTC embedder <NUM> is also coupled to retry buffer <NUM>.

RTE <NUM> includes a receive channel <NUM>. Receive channel <NUM> is capable of encapsulating a FLIT-based interface and converting the interface into data (e.g., AXI) streams. In the example of <FIG>, receive channel <NUM> includes a packet boundary detector <NUM>, a receive (RX) packet CRC circuit <NUM>, an RX packet processor <NUM>, and an input buffer <NUM>. Rx packet processor <NUM> is coupled to error handler <NUM> and to retry sequence circuit <NUM>.

RTE <NUM> is provided for purposes of illustration and not limitation. It should be appreciated that other architectures suitable for implementing a credit-based flow control/retransmission control scheme may be used. The architecture described in connection with <FIG> may also be used to implement RTE <NUM> of <FIG> with a flipped or reversed orientation in terms of data flow.

<FIG> illustrates an example method <NUM> of operation for a system with a plurality of hardware accelerators. Method <NUM> illustrates an example of data transfer directly among the hardware accelerators. Method <NUM> may be performed by a system the same as, or similar to, system <NUM> described in connection with <FIG>. Method <NUM> illustrates how insufficient bandwidth on the bus coupling the host processor and the hardware accelerators may be alleviated. Data transfers that otherwise occur on the bus may be diverted to the accelerator links thereby freeing bandwidth on the bus for other operations.

In block <NUM>, the system is capable of automatically discovering the hardware accelerator sequence. In one or more embodiments, the hardware accelerators, e.g., boards of the hardware accelerators, are arranged in a ring topology within the system. The host processor is aware of the existing PCle topology and, as such, the number of hardware accelerators that exist within the system coupled to the PCle bus. Further, the host processor, e.g., by way of the runtime, is aware of the particular circuitry (e.g., image or configuration bitstream) loaded into each hardware accelerator. As such, the host processor is aware that the hardware accelerators support accelerator links as described herein. The host processor still must determine the sequence of hardware accelerators. The driver, for example, is capable of performing the automatic discovery of the hardware accelerator sequence described. This automatic discovery capability supports the addition of new and/or additional hardware accelerators to the system without having to modify the applications executed by the host processor.

Each hardware accelerator may have a known and same address range. For example, each hardware accelerator may be assumed to have an address range of <NUM> GB corresponding to <NUM> GB of memory <NUM>. In one or more embodiments, the host processor is capable of writing a unique value to memory addresses at <NUM> GB intervals. The host processor may then read back the values to determine the sequence of hardware accelerators within the ring topology based upon the written and read values.

In block <NUM>, the host processor is capable of creating a buffer on each hardware accelerator at start up. For example, the runtime executed by the host processor is capable of communicating with each hardware accelerator to create a buffer within the memory of each respective hardware accelerator. Referring to <FIG>, hardware accelerator <NUM>-<NUM> creates a buffer within memory <NUM>-<NUM>. Hardware accelerator <NUM>-<NUM> creates a buffer within memory <NUM>-<NUM>. Hardware accelerator <NUM>-<NUM> creates a buffer within memory <NUM>-<NUM>.

In block <NUM>, the host processor initiates a data transfer between hardware accelerators. The data transfer, for example, may be part of a task that is to be offloaded from the host processor to a hardware accelerator. As an illustrative and nonlimiting example, host processor <NUM> may offload a task for an application to compute unit <NUM>-<NUM> of hardware accelerator <NUM>-<NUM>. The task may include instructions and a target address from which compute unit <NUM>-<NUM> is to obtain data for the task. The target address in this example is located in hardware accelerator <NUM>-<NUM> (e.g., in memory <NUM>-<NUM>). Accordingly, to perform the task offloaded from the host processor, compute unit <NUM>-<NUM> must retrieve the data from the target address in memory <NUM>-<NUM>.

In block <NUM>, the runtime may request a data transfer between hardware accelerators <NUM>-<NUM> and <NUM>-<NUM>. For example, the runtime may request a read of hardware accelerator <NUM>-<NUM> by, or from, hardware accelerator <NUM>-<NUM>.

In block <NUM>, the driver is capable of creating a buffer object in the host memory corresponding to hardware accelerator <NUM>-<NUM> and a buffer object in the host memory corresponding to hardware accelerator <NUM>-<NUM>. A buffer object is a shadow data structure implemented in host memory. Each buffer object may correspond to, or represent, a device in the system. A buffer object may include data that supports administrative functions performed by the runtime executed by the host processor.

In one or more embodiments, buffer objects created in the host memory may include a remote flag. The remote flag may be set to indicate that the buffer object is remote from the perspective of the hardware accelerator that is initiating a transaction. In this example, hardware accelerator <NUM>-<NUM> is reading data from hardware accelerator <NUM>-<NUM>. As such, hardware accelerator <NUM>-<NUM> is initiating the transaction. The driver sets the remote flag in the buffer object corresponding to hardware accelerator <NUM>-<NUM> upon creation.

In block <NUM>, the runtime library initiates access to the buffer object (e.g., remote buffer object) by the initiating hardware accelerator. The runtime library initiates access of the buffer object corresponding to hardware accelerator <NUM>-<NUM> from hardware accelerator <NUM>-<NUM>. For example, the runtime determines that the remote flag is set within the buffer object for hardware accelerator <NUM>-<NUM>. In response to determining that the remote flag is set, the runtime library schedules the transfer using the accelerator links established by the link circuits. In scheduling the transfer using the accelerator links between the hardware accelerators, the runtime determines the address to be used by hardware accelerator <NUM>-<NUM> to access the data from hardware accelerator <NUM>-<NUM>.

For purposes of illustration, consider an example where each of hardware accelerators <NUM> has an address range of <NUM>-<NUM>. In such an example, the runtime may determine that the data to be retrieved from hardware accelerator <NUM>-<NUM> by hardware accelerator <NUM>-<NUM> is located in a buffer at address <NUM> corresponding to hardware accelerator <NUM>-<NUM> (e.g., at address <NUM> corresponding to memory <NUM>-<NUM>). In this example, the runtime adds <NUM> to the target address resulting in an address of <NUM>, which is provided to hardware accelerator <NUM>-<NUM> as the target address for reading data upon which to operate for the offloaded task.

As another example, if the data were stored at address <NUM> within memory <NUM>-<NUM>, the runtime would add <NUM>, assuming each of hardware accelerators <NUM> has an address range of <NUM>-<NUM>, in order for the transaction to reach hardware accelerator <NUM>-<NUM>. In general, as known, return path data may be tracked through the on-chip bus interconnects (e.g., AXI interconnects) used. When a read request from a master is issued, for example, the read request is routed to the slave through the interconnects with a series of address decoding and/or address shifting (performed by the mm-stream mappers) as the read request traverses across each hardware accelerator. Each individual interconnect is capable of keeping track of which masters have outstanding transactions to each slave. Upon the read data being returned, the read data may be sent back over the correct interface(s). In some cases, identifier (ID) bits may be used to associate particular read data back with a particular master in order to return the read data.

In block <NUM>, the initiating hardware accelerator (e.g., the first hardware accelerator) receives the task from the host processor. End point <NUM>-<NUM>, for example, may receive the task and provide the task to compute unit <NUM>-<NUM>. The task specifies that the data to be operated on by compute unit <NUM>-<NUM> is located at the target address, which is <NUM> in this example. Compute unit <NUM>-<NUM>, for example, may have a control port to which the target address may be stored. In attempting to access the data located at address <NUM>, compute unit <NUM>-<NUM> recognizes that the address is not within the range of hardware accelerator <NUM>-<NUM>. For example, compute unit <NUM>-<NUM> is capable of comparing the address with the upper bound of the address range of <NUM> and determining that the address exceeds the upper bound. In this example, compute unit <NUM>-<NUM> is capable of initiating a read transaction from address <NUM>. For example, compute unit <NUM>-<NUM> may initiate the read transaction as a memory mapped transaction sent over interconnect <NUM>.

In block <NUM>, the initiating hardware accelerator accesses the target hardware accelerator (e.g., the second hardware accelerator) over the accelerator link. For example, link circuit <NUM>-<NUM> is capable of converting the memory mapped transaction initiated by compute unit <NUM>-<NUM> into stream based packets (e.g., using the MM-stream mapper). Link circuit <NUM>-<NUM> is further capable of encoding the packets with additional data supporting data integrity checking, retransmitting, initialization, and error reporting (e.g., using the RPE). The ring topology may master from left to right. As such, the packets may be output by the transceiver of link circuit <NUM>-<NUM> to link circuit <NUM>-<NUM>.

Link circuit <NUM>-<NUM> receives the data stream in transceiver <NUM> and processes the transaction in RTE <NUM>. MM-stream mapper <NUM>, in response to receiving the stream data based packets, is capable of performing a variety of operations. MM-stream mapper <NUM>, for example, is capable of converting the stream based packets into a memory mapped transaction. Further, MM-stream mapper <NUM> is capable of decrementing the target address of <NUM> by the upper bound of the address range of hardware accelerator <NUM>-<NUM>. As noted, the upper bound may be stored in a table or register within link circuit <NUM>-<NUM>, e.g., in MM-stream mapper <NUM>. In this example, MM-stream mapper <NUM> decrements the target address of <NUM> by <NUM> resulting in a target address of <NUM>. Since the target address is local to hardware accelerator <NUM>-<NUM>, hardware accelerator <NUM>-<NUM> is capable of acting on the received transaction. In this example, MM-stream mapper <NUM> provides the memory mapped transaction to interconnect <NUM>. The memory mapped transaction may be provided to memory controller <NUM>-<NUM> (e.g., through a memory controller slave) to perform the read transaction. In this manner, hardware accelerator <NUM>-<NUM> is capable of reading data from (or writing data to) hardware accelerator <NUM>-<NUM>. The requested data may be provided from memory <NUM>-<NUM> back to the requestor using the same path used to send the read request. For example, the data read from memory <NUM>-<NUM> is sent from hardware accelerator <NUM>-<NUM> to hardware accelerator <NUM>-<NUM> without having to traverse forward through the ring topology to hardware accelerator <NUM>-<NUM> and then to hardware accelerator <NUM>-<NUM>.

If, for example, the target address was <NUM>, the result of decrementing would be <NUM>. In that case, MM-stream mapper <NUM> determines that the target address is not located in hardware accelerator <NUM>-<NUM> since the target address is larger than the upper bound of the address range (e.g., <NUM>) for hardware accelerator <NUM>-<NUM>. In that case, MM-stream mapper <NUM> may send the transaction through the interconnect circuitry to MM-stream mapper <NUM> to forward on to the next hardware accelerator.

In block <NUM>, compute unit <NUM>-<NUM> in hardware accelerator <NUM>-<NUM> is capable of generating an interrupt to the host processor informing the host processor that the data transfer between the hardware accelerators is complete. In block <NUM>, the runtime is capable of providing any notifications necessary to applications that the data transfer is complete. The runtime, for example, is capable of handling completion events, command queues, and notifications to applications.

In one or more embodiments, the PCle endpoint and DMA master are capable of writing to a target address that is located in a different hardware accelerator. As an illustrative and non-limiting example, the host processor may send data to hardware accelerator <NUM>-<NUM> with a target address that is located in hardware accelerator <NUM>-<NUM>. In that case, the DMA master is capable of recognizing that the target address is located in a different hardware accelerator and schedule the data transfer over the accelerator link. For example, the DMA master may compare the target address with the upper bound of the address range for hardware accelerator <NUM>-<NUM>. In response to determining that the target address exceeds the upper bound, the DMA master is capable of initiating a memory mapped transaction over the interconnect circuitry to MM-stream mapper <NUM> in link circuit <NUM>-<NUM> for sending to hardware accelerator <NUM>-<NUM> via the accelerator link.

In one or more embodiments, the host processor is capable of using accelerator links for purposes of load balancing. For example, the host processor is capable of using the runtime to determine the status of the DMA channels (e.g., DMA master) in a selected hardware accelerator to which data is to be provided or a task is to be offloaded. In response to determining that the DMA master is busy or operating above a threshold amount of activity, the host processor may send the data to a different hardware accelerator via the bus. The data may specify a target address within the selected hardware accelerator. The DMA master within the receiving hardware accelerator, upon receiving the data from the host processor, is capable of forwarding the data to the selected hardware accelerator over the accelerator link(s). In particular embodiments, the host processor is capable of choosing the receiving hardware accelerator based upon a determination that the DMA master therein is not busy or is operating below the threshold amount of activity.

For purposes of illustration, an example of a write transaction from hardware accelerator <NUM>-<NUM> to hardware accelerator <NUM>-<NUM> is generally described as initiated by the host processor. The host processor, by way of the runtime and driver, sets the remote flag for the target hardware accelerator and determines an address of <NUM> (using the prior example where the desired address is located at address <NUM> in hardware accelerator <NUM>-<NUM>). The host processor provides instructions to hardware accelerator <NUM>-<NUM> to write to address <NUM>. Within hardware accelerator <NUM>-<NUM>, the transaction with an address of <NUM> is presented to interconnect <NUM>. Since the address exceeds the upper limit of hardware accelerator <NUM>-<NUM>, interconnect <NUM> sends the transaction to link circuit <NUM>-<NUM>. Link circuit <NUM>-<NUM> sends the transaction to link circuit <NUM>-<NUM>. The MM-stream mapper in hardware accelerator <NUM>-<NUM> decrements the address by <NUM> resulting in a new address of <NUM>. The new address is still remote as <NUM> exceeds the upper address bound of hardware accelerator <NUM>-<NUM>. As such, the transaction is forwarded to hardware accelerator <NUM>-<NUM>.

The MM-stream mapper in hardware accelerator <NUM>-<NUM> decrements the address resulting in a new address of <NUM>. The transaction is then provided, via interconnect <NUM> in hardware accelerator <NUM>-<NUM> to a memory controller and the data written to memory <NUM>-<NUM>. In the examples described, the address is used by each hardware accelerator to determine whether the transaction can be serviced by the hardware accelerator and, if so, where to route the transaction internally (e.g., to a memory controller or other circuit block), or should be forwarded to the next hardware accelerator. In particular embodiments, the address is different from the actual address to which the data is written in memory. The write acknowledgement is sent, as described, from hardware accelerator <NUM>-<NUM> through hardware accelerator <NUM>-<NUM> to hardware accelerator <NUM>-<NUM>.

For purposes of illustration, another example of a read transaction initiated by hardware accelerator <NUM>-<NUM> to hardware accelerator <NUM>-<NUM> is generally described as initiated by the host processor. The host processor, by way of the runtime and driver, sets the remote flag for the target hardware accelerator and determines an address of <NUM> (using the prior example where the desired address is located at address <NUM> in hardware accelerator <NUM>-<NUM>). The host processor provides instructions to hardware accelerator <NUM>-<NUM> to read from address <NUM>. Within hardware accelerator <NUM>-<NUM>, the transaction with an address of <NUM> is presented to interconnect <NUM>. Since the address exceeds the upper limit of hardware accelerator <NUM>-<NUM>, interconnect <NUM> sends the transaction to link circuit <NUM>-<NUM>. Link circuit <NUM>-<NUM> sends the transaction to link circuit <NUM>-<NUM>. The MM-stream mapper in hardware accelerator <NUM>-<NUM> decrements the address by <NUM> resulting in a new address of <NUM>. The new address is still remote as <NUM> exceeds the upper address bound of hardware accelerator <NUM>-<NUM>. As such, the transaction is forwarded to hardware accelerator <NUM>-<NUM>.

The MM-stream mapper in hardware accelerator <NUM>-<NUM> decrements the address resulting in a new address of <NUM>. The transaction is then provided, via interconnect <NUM> in hardware accelerator <NUM>-<NUM> to a memory controller and the data read from memory <NUM>-<NUM>. In the examples described, the address is used by each hardware accelerator to determine whether the transaction can be serviced by the hardware accelerator and, if so, where to route the transaction internally, or should be forwarded to the next hardware accelerator. In particular embodiments, the address is different from the actual address from which the data is read from memory. The data that is read is sent, as described, from hardware accelerator <NUM>-<NUM> through hardware accelerator <NUM>-<NUM> to hardware accelerator <NUM>-<NUM>.

<FIG> illustrates an example of a system including hardware accelerators and one or more additional devices. In the example of <FIG>, hardware accelerators <NUM>-<NUM> and <NUM>-<NUM> are shown and are coupled by an accelerator link using the link circuit in each respective hardware accelerator. For purposes of illustration, hardware accelerator <NUM>-<NUM> is not shown. The system also includes a GPU <NUM>, which is coupled to memory <NUM>, and an I/O device <NUM>.

In the example of <FIG>, GPU <NUM> may write data to hardware accelerator <NUM>-<NUM> or read data from hardware accelerator <NUM>-<NUM>. In this example, the host processor (not shown) provides handle <NUM>-N to GPU <NUM>. In particular embodiments, handles may be implemented as file descriptors. Handle <NUM>-N may point to a buffer object <NUM>-N, which corresponds to hardware accelerator <NUM>-<NUM>. By GPU <NUM> using handle <NUM>-N for the read or write operation, the host processor initiates action on a buffer object corresponding to handle <NUM>-N, e.g., buffer object <NUM>-N. The host processor determines whether buffer object <NUM>-N is local or remote. The host processor may retrieve the data from memory <NUM>-<NUM> over PCle and provide the data to GPU <NUM> over PCle since the remote flag in buffer object <NUM>-N is not set.

In one or more other embodiments, the host processor may initiate retrieval of data from memory <NUM>-<NUM> by accessing a different hardware accelerator. For example, the host processor may initiate communication via PCle with hardware accelerator <NUM>-<NUM> to retrieve data from memory <NUM>-<NUM>. In that case, hardware accelerator <NUM>-<NUM> may communicate directly with hardware accelerator <NUM>-<NUM> using the link circuits to retrieve data from memory <NUM>-<NUM>. Hardware accelerator <NUM>-<NUM> may then provide the data back to the host processor, which in turn provides the data to GPU <NUM> over PCle.

In another example, I/O device <NUM>, e.g., a camera, may write data to hardware accelerator <NUM>-<NUM>. In that case, the host processor is capable of providing handle <NUM>-<NUM> to I/O device <NUM>. Handle <NUM>-<NUM> may point to a buffer object <NUM>-<NUM>, which corresponds to hardware accelerator <NUM>-<NUM>. By I/O device <NUM> using handle <NUM>-<NUM> for the write operation, the host processor initiates action on a buffer object corresponding to handle <NUM>-<NUM>, e.g., buffer object <NUM>-<NUM>. The host processor determines whether buffer object <NUM>-<NUM> is local or remote. The host processor may receive data from I/O device <NUM> and provide such data over PCle to hardware accelerator <NUM>-<NUM> for writing in memory <NUM>-<NUM> and/or further processing since the remote flag in buffer object <NUM>-<NUM> is not set.

In one or more embodiments, the driver is capable of setting the remote flag within a buffer object only in cases of data transfers between hardware accelerators that are capable of using accelerator links as described. <FIG> illustrates that while other types of devices may be used with hardware accelerators, data transfers between such other devices and the hardware accelerators occur over the bus and involve the host processor.

<FIG> illustrates an example architecture <NUM> for an IC. In one aspect, architecture <NUM> may be implemented within a programmable IC. For example, architecture <NUM> may be used to implement a field programmable gate array (FPGA). Architecture <NUM> may also be representative of a system-on-chip (SOC) type of IC. An SOC is an IC that includes a processor that executes program code and one or more other circuits. The other circuits may be implemented as hardwired circuitry, programmable circuitry, and/or a combination thereof. The circuits may operate cooperatively with one another and/or with the processor.

As shown, architecture <NUM> includes several different types of programmable circuit, e.g., logic, blocks. For example, architecture <NUM> may include a large number of different programmable tiles including multi-gigabit transceivers (MGTs) <NUM>, configurable logic blocks (CLBs) <NUM>, random access memory blocks (BRAMs) <NUM>, input/output blocks (IOBs) <NUM>, configuration and clocking logic (CONFIG/CLOCKS) <NUM>, digital signal processing blocks (DSPs) <NUM>, specialized I/O blocks <NUM> (e.g., configuration ports and clock ports), and other programmable logic <NUM> such as digital clock managers, analog-to-digital converters, system monitoring logic, and so forth.

In some ICs, each programmable tile includes a programmable interconnect element (INT) <NUM> having standardized connections to and from a corresponding INT <NUM> in each adjacent tile. Therefore, INTs <NUM>, taken together, implement the programmable interconnect structure for the illustrated IC. Each INT <NUM> also includes the connections to and from the programmable logic element within the same tile, as shown by the examples included at the top of <FIG>.

For example, a CLB <NUM> may include a configurable logic element (CLE) <NUM> that may be programmed to implement user logic plus a single INT <NUM>. A BRAM <NUM> may include a BRAM logic element (BRL) <NUM> in addition to one or more INTs <NUM>. Typically, the number of INTs <NUM> included in a tile depends on the height of the tile. As pictured, a BRAM tile has the same height as five CLBs, but other numbers (e.g., four) also may be used. A DSP tile <NUM> may include a DSP logic element (DSPL) <NUM> in addition to an appropriate number of INTs <NUM>. An IOB <NUM> may include, for example, two instances of an I/O logic element (IOL) <NUM> in addition to one instance of an INT <NUM>. The actual I/O pads connected to IOL <NUM> may not be confined to the area of IOL <NUM>.

In the example pictured in <FIG>, a columnar area near the center of the die, e.g., formed of regions <NUM>, <NUM>, and <NUM>, may be used for configuration, clock, and other control logic. Horizontal areas <NUM> extending from this column may be used to distribute the clocks and configuration signals across the breadth of the programmable IC.

Some ICs utilizing the architecture illustrated in <FIG> include additional logic blocks that disrupt the regular columnar structure making up a large part of the IC. The additional logic blocks may be programmable blocks and/or dedicated circuitry. For example, a processor block depicted as PROC <NUM> spans several columns of CLBs and BRAMs.

In one aspect, PROC <NUM> may be implemented as dedicated circuitry, e.g., as a hardwired processor, that is fabricated as part of the die that implements the programmable circuitry of the IC. PROC <NUM> may represent any of a variety of different processor types and/or systems ranging in complexity from an individual processor, e.g., a single core capable of executing program code, to an entire processor system having one or more cores, modules, coprocessors, interfaces, or the like.

In another aspect, PROC <NUM> may be omitted from architecture <NUM> and replaced with one or more of the other varieties of the programmable blocks described. Further, such blocks may be utilized to form a "soft processor" in that the various blocks of programmable circuitry may be used to form a processor that can execute program code as is the case with PROC <NUM>.

The phrase "programmable circuitry" refers to programmable circuit elements within an IC, e.g., the various programmable or configurable circuit blocks or tiles described herein, as well as the interconnect circuitry that selectively couples the various circuit blocks, tiles, and/or elements according to configuration data that is loaded into the IC. For example, circuit blocks shown in <FIG> that are external to PROC <NUM> such as CLBs <NUM> and BRAMs <NUM> are considered programmable circuitry of the IC.

In general, the functionality of programmable circuitry is not established until configuration data is loaded into the IC. A set of configuration bits may be used to program programmable circuitry of an IC such as an FPGA. The configuration bit(s) typically are referred to as a "configuration bitstream. " In general, programmable circuitry is not operational or functional without first loading a configuration bitstream into the IC. The configuration bitstream effectively implements a particular circuit design within the programmable circuitry. The circuit design specifies, for example, functional aspects of the programmable circuit blocks and physical connectivity among the various programmable circuit blocks.

Circuitry that is "hardwired" or "hardened," i.e., not programmable, is manufactured as part of the IC. Unlike programmable circuitry, hardwired circuitry or circuit blocks are not implemented after the manufacture of the IC through the loading of a configuration bitstream. Hardwired circuitry is generally considered to have dedicated circuit blocks and interconnects, for example, that are functional without first loading a configuration bitstream into the IC, e.g., PROC <NUM>.

In some instances, hardwired circuitry may have one or more operational modes that can be set or selected according to register settings or values stored in one or more memory elements within the IC. The operational modes may be set, for example, through the loading of a configuration bitstream into the IC. Despite this ability, hardwired circuitry is not considered programmable circuitry as the hardwired circuitry is operable and has a particular function when manufactured as part of the IC.

In the case of an SOC, the configuration bitstream may specify the circuitry that is to be implemented within the programmable circuitry and the program code that is to be executed by PROC <NUM> or a soft processor. In some cases, architecture <NUM> includes a dedicated configuration processor that loads the configuration bitstream to the appropriate configuration memory and/or processor memory. The dedicated configuration processor does not execute user-specified program code. In other cases, architecture <NUM> may utilize PROC <NUM> to receive the configuration bitstream, load the configuration bitstream into appropriate configuration memory, and/or extract program code for execution.

<FIG> is intended to illustrate an example architecture that may be used to implement an IC that includes programmable circuitry, e.g., a programmable fabric. For example, the number of logic blocks in a column, the relative width of the columns, the number and order of columns, the types of logic blocks included in the columns, the relative sizes of the logic blocks, and the interconnect/logic implementations included at the top of <FIG> are purely illustrative. In an actual IC, for example, more than one adjacent column of CLBs is typically included wherever the CLBs appear, to facilitate the efficient implementation of a user circuit design. The number of adjacent CLB columns, however, may vary with the overall size of the IC. Further, the size and/or positioning of blocks such as PROC <NUM> within the IC are for purposes of illustration only and are not intended as limitations.

Architecture <NUM> may be used to implement a hardware accelerator as described herein. In particular embodiments, one or more or each of the endpoint, link circuit, and memory controller may be implemented as hardwired circuit blocks. In particular embodiments, one or more or each of the endpoint, link circuit, and memory controller may be implemented using programmable circuitry. In still other embodiments, one or more of the noted circuit blocks may be implemented as hardwired circuit blocks while the others are implemented using programmable circuitry.

The embodiments described within this disclosure may be used in any of a variety of applications such as, for example, database acceleration, processing multiple video stream, real time network traffic monitoring, machine learning, or any other application that may involve multiple hardware accelerators.

For purposes of explanation, specific nomenclature is set forth to provide a thorough understanding of the various inventive concepts disclosed herein. The terminology used herein, however, is for the purpose of describing particular aspects of the inventive arrangements only and is not intended to be limiting.

As defined herein, the singular forms "a," "an," and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise.

As defined herein, the term "approximately" means nearly correct or exact, close in value or amount but not precise. For example, the term "approximately" may mean that the recited characteristic, parameter, or value is within a predetermined amount of the exact characteristic, parameter, or value.

As defined herein, the terms "at least one," "one or more," and "and/or," are open-ended expressions that are both conjunctive and disjunctive in operation unless explicitly stated otherwise. For example, each of the expressions "at least one of A, B, and C," "at least one of A, B, or C," "one or more of A, B, and C," "one or more of A, B, or C," and "A, B, and/or C" means A alone, B alone, C alone, A and B together, A and C together, B and C together, or A, B and C together.

As defined herein, the term "automatically" means without user intervention. As defined herein, the term "user" means a human being.

As defined herein, the term "computer readable storage medium" means a storage medium that contains or stores program code for use by or in connection with an instruction execution system, apparatus, or device. As defined herein, a "computer readable storage medium" is not a transitory, propagating signal per se. A computer readable storage medium may be, but is not limited to, an electronic storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing. The various forms of memory, as described herein, are examples of computer readable storage media. A non-exhaustive list of more specific examples of a computer readable storage medium may include: a portable computer diskette, a hard disk, a RAM, a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an electronically erasable programmable read-only memory (EEPROM), a static random access memory (SRAM), a portable compact disc read-only memory (CD-ROM), a digital versatile disk (DVD), a memory stick, a floppy disk, or the like.

As defined herein, the term "if" means "when" or "upon" or "in response to" or "responsive to," depending upon the context. Thus, the phrase "if it is determined" or "if [a stated condition or event] is detected" may be construed to mean "upon determining" or "in response to determining" or "upon detecting [the stated condition or event]" or "in response to detecting [the stated condition or event]" or "responsive to detecting [the stated condition or event]" depending on the context.

As defined herein, the term "responsive to" and similar language as described above, e.g., "if," "when," or "upon," means responding or reacting readily to an action or event. The response or reaction is performed automatically. Thus, if a second action is performed "responsive to" a first action, there is a causal relationship between an occurrence of the first action and an occurrence of the second action. The term "responsive to" indicates the causal relationship.

As defined herein, the terms "one embodiment," "an embodiment," "one or more embodiments," "particular embodiments," or similar language mean that a particular feature, structure, or characteristic described in connection with the embodiment is included in at least one embodiment described within this disclosure. Thus, appearances of the phrases "in one embodiment," "in an embodiment," "in one or more embodiments," "in particular embodiments," and similar language throughout this disclosure may, but do not necessarily, all refer to the same embodiment. The terms "embodiment" and "arrangement" are used interchangeably within this disclosure.

As defined herein, the term "processor" means at least one hardware circuit. The hardware circuit may be configured to carry out instructions contained in program code. The hardware circuit may be an integrated circuit. Examples of a processor include, but are not limited to, a central processing unit (CPU), an array processor, a vector processor, a digital signal processor (DSP), an FPGA, a programmable logic array (PLA), an ASIC, programmable logic circuitry, and a controller.

As defined herein, the term "output" means storing in physical memory elements, e.g., devices, writing to display or other peripheral output device, sending or transmitting to another system, exporting, or the like.

As defined herein, the term "real time" means a level of processing responsiveness that a user or system senses as sufficiently immediate for a particular process or determination to be made, or that enables the processor to keep up with some external process.

As defined herein, the term "substantially" means that the recited characteristic, parameter, or value need not be achieved exactly, but that deviations or variations, including for example, tolerances, measurement error, measurement accuracy limitations, and other factors known to those of skill in the art, may occur in amounts that do not preclude the effect the characteristic was intended to provide.

The terms first, second, etc. may be used herein to describe various elements. These elements should not be limited by these terms, as these terms are only used to distinguish one element from another unless stated otherwise or the context clearly indicates otherwise.

A computer program product may include a computer readable storage medium (or media) having computer readable program instructions thereon for causing a processor to carry out aspects of the inventive arrangements described herein. Within this disclosure, the term "program code" is used interchangeably with the term "computer readable program instructions. " Computer readable program instructions described herein may be downloaded to respective computing/processing devices from a computer readable storage medium or to an external computer or external storage device via a network, for example, the Internet, a LAN, a WAN and/or a wireless network. The network may include copper transmission cables, optical transmission fibers, wireless transmission, routers, firewalls, switches, gateway computers and/or edge devices including edge servers.

Computer readable program instructions for carrying out operations for the inventive arrangements described herein may be assembler instructions, instruction-set-architecture (ISA) instructions, machine instructions, machine dependent instructions, microcode, firmware instructions, or either source code or object code written in any combination of one or more programming languages, including an object oriented programming language and/or procedural programming languages. Computer readable program instructions may include state-setting data. In the latter scenario, the remote computer may be connected to the user's computer through any type of network, including a LAN or a WAN, or the connection may be made to an external computer (for example, through the Internet using an Internet Service Provider). In some cases, electronic circuitry including, for example, programmable logic circuitry, an FPGA, or a PLA may execute the computer readable program instructions by utilizing state information of the computer readable program instructions to personalize the electronic circuitry, in order to perform aspects of the inventive arrangements described herein.

Certain aspects of the inventive arrangements are described herein with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products. It will be understood that each block of the flowchart illustrations and/or block diagrams, and combinations of blocks in the flowchart illustrations and/or block diagrams, may be implemented by computer readable program instructions, e.g., program code.

These computer readable program instructions may also be stored in a computer readable storage medium that can direct a computer, a programmable data processing apparatus, and/or other devices to function in a particular manner, such that the computer readable storage medium having instructions stored therein comprises an article of manufacture including instructions which implement aspects of the operations specified in the flowchart and/or block diagram block or blocks.

The computer readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operations to be performed on the computer, other programmable apparatus or other device to produce a computer implemented process, such that the instructions which execute on the computer, other programmable apparatus, or other device implement the functions/acts specified in the flowchart and/or block diagram block or blocks.

The flowchart and block diagrams in the Figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various aspects of the inventive arrangements. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of instructions, which comprises one or more executable instructions for implementing the specified operations.

Claim 1:
A system, comprising:
a host processor (<NUM>) coupled to a communication bus;
a first hardware accelerator (<NUM>-<NUM>) communicatively linked to the host processor (<NUM>) through the communication bus; and
a second hardware accelerator (<NUM>-<NUM>) communicatively linked to the host processor (<NUM>) through the communication bus;
wherein the first hardware accelerator (<NUM>-<NUM>) and the second hardware accelerator (<NUM>-<NUM>) are directly coupled through an accelerator link independent of the communication bus; and
wherein the host processor (<NUM>) is configured to initiate a data transfer between the first hardware accelerator (<NUM>-<NUM>) and the second hardware accelerator (<NUM>-<NUM>) directly through the accelerator link;
characterised in that
the second hardware accelerator (<NUM>-<NUM>) is configured to decrement a target address for the data transfer by an upper bound of an address range for the second hardware accelerator (<NUM>-<NUM>) in response to receiving a transaction via the accelerator link and determine whether the decremented target address is local.