Patent Description:
A system on a chip (SoC) can include a mix of programmable logic (e.g., programmable fabric) and software-configurable hardened logic such as processing cores or engines. Typically, a user must understand in detail the programmable and software configurable hardened logic (and how they communicate) in order to write programs which can be compiled into a bitstream for configuring the programmable and binary code for configuring the software-configurable hardened logic to perform a user function. But using hardware description language (HDL) or Open Computing Language (OpenCL) to write programs for a SoC with a mix of programmable and hardened logic is cumbersome and difficult to parallelize. Data-parallelism and thread-parallelism are also used to express computations over an array of processors but these techniques do not extend naturally to programmable logic where heterogeneous computations with different interfaces need to be expressed.

Dubach et al. describe compiling a high-level language in GPUs in C.

<CIT> discloses mapping a computer program to an asymmetric multiprocessing apparatus.

Techniques for implementing a dataflow graph on a heterogeneous processing system are described. One example is a method that includes receiving graph source code, the graph source code defining a plurality of kernels and a plurality of communication links, where each of the plurality of communication links couple a respective pair of the plurality of kernels to form a dataflow graph. The method also includes compiling the graph source code to implement the dataflow graph on a system in a heterogeneous processing system. Compiling the graph source code includes assigning the plurality of kernels to programmable logic and an array of data processing engines (DPEs) in the heterogeneous processing system, assigning a communication type to the plurality of communication links, and selecting synchronization techniques for transferring data between the plurality of kernels using the plurality of communication links.

In some embodiments, assigning the plurality of kernels to the heterogeneous processing system includes identifying that a first kernel and a second kernel are communicatively coupled by a first one of the plurality of communication links as defined by the graph source code, assigning the first kernel to a first data processing engine (DPE) in the heterogeneous processing system, and assigning the second kernel to a second DPE in the heterogeneous processing system that directly neighbors the first DPE.

In some embodiments, the first DPE and a second DPE both have a direct connection to a shared memory module and the method includes assigning a double buffer in the shared memory module for transferring data between the first kernel and the second kernel.

In some embodiments, assigning the plurality of kernels to the heterogeneous processing system includes identifying that a first kernel and a second kernel are communicatively coupled by a first one of the plurality of communication links as defined by the graph source code, assigning the first kernel to a first DPE in the heterogeneous processing system, assigning the second kernel to programmable logic in the heterogeneous processing system, and configuring the second kernel to perform a direct memory access (DMA) using an interconnect to transfer data to the first kernel, wherein the interconnect interconnects an array of DPEs that includes the first DPE to each other and to programmable logic.

In some embodiments, assigning the plurality of kernels to the heterogeneous processing system includes identifying that a first kernel and a second kernel are communicatively coupled by a first one of the plurality of communication links as defined by the graph source code, clustering the first and second kernels to a first core in an array of DPEs in the heterogeneous processing system in response to determining that the first and second kernels have a combined cycle count that is less than or equal to a cycle budget for the first core, and assigning a buffer in a memory module for transmitting data between the first and second kernels where the memory module has a direct connection to the first core.

In some embodiments, assigning the communication type to the plurality of communication links includes selecting whether to use one of streaming and windowing to transmit data for each of the plurality of communication links based on the definition of the plurality of communication links in the graph source code.

In some embodiments, windowing comprises dividing received data into individual windows with a predefined or parameterized block size, and each of the plurality of kernels configured to receive the individual windows waits until receiving a window on every invocation before processing the received windows. Further, for at least one of the communication links that performs windowing, the individual windows have data at the beginning that overlaps with ends of previously transmitted windows so that a receiving kernel of the plurality of kernels that receives the individual windows maintains its state.

In some embodiments, selecting the synchronization techniques includes identifying a double buffer assigned to a first one of the plurality of communication links and configuring a locking protocol so that a first kernel and a second kernel corresponding to the first one of the plurality of communication links can access the double buffer in parallel.

In some embodiments, the method includes transmitting a bitstream and binary code based on compiling the graph source code that configures the heterogeneous processing system to execute the dataflow graph and controlling execution of the dataflow graph in the heterogeneous processing system using a control program.

In some embodiments, the heterogeneous processing system includes a first chip and a second chip, where the plurality of kernels are assigned to the first chip, the graph source code defines a second plurality of kernels, and compiling the graph source code includes assigning the second plurality of kernels to the second chip, wherein the second plurality of kernels assigned to the second chip are configured to communicate with the plurality of kernels assigned to the first chip.

In some embodiments, the graph source code is independent of a hardware design of a SoC forming the heterogeneous processing system and can be implemented by the compiler onto multiple different types of SoCs each having different hardware designs.

In some embodiments, the heterogeneous processing system comprises programmable logic and an array of DPEs, where at a least one of the plurality of kernels is assigned to the programmable logic and at least one of the plurality of kernels is assigned to one of the DPEs.

In some embodiments, the method includes encapsulating a sub-graph into the dataflow graph where the sub-graph is defined by a graph class separate from the graph source code and generating a constrained graph that adds constraints to the dataflow graph and the sub-graph where the constrained graph serves as a wrapper for the dataflow graph.

In some embodiments, each of the plurality of kernels includes a least one port to enable each of the plurality of kernels to communicate with another kernel in the dataflow graph, and, in the dataflow graph, each one of the plurality of communication links couples a first port on a first kernel to a second port on a second kernel.

One example described herein is a host that includes a processor, graph source code defining a plurality of kernels and a plurality of communication links where each of the plurality of communication links couple a respective pair of the plurality of kernels to form a dataflow graph, and a compiler configured to compile the graph source code to implement the dataflow graph in a heterogeneous processing system. Compiling the graph source code includes assigning the plurality of kernels to programmable logic and an array of DPEs in the heterogeneous processing system, assigning a communication type to the plurality of communication links, and selecting synchronization techniques for transferring data between the plurality of kernels using the plurality of communication links.

So that the manner in which the above recited features can be understood in detail, a more particular description, briefly summarized above, may be had by reference to example implementations, some of which are illustrated in the appended drawings. It is to be noted, however, that the appended drawings illustrate only typical example implementations and are therefore not to be considered limiting of its scope.

It is contemplated that elements of one example may be beneficially incorporated in other examples.

Various features are described hereinafter with reference to the figures. It should be noted that the figures may or may not be drawn to scale and that the elements of similar structures or functions are represented by like reference numerals throughout the figures. It should be noted that the figures are only intended to facilitate the description of the features. They are not intended as an exhaustive description or as a limitation on the scope of the claims. In addition, an illustrated example need not have all the aspects or advantages shown. An aspect or an advantage described in conjunction with a particular example is not necessarily limited to that example and can be practiced in any other examples even if not so illustrated, or if not so explicitly described.

Examples herein describe techniques for generating dataflow graphs using source code for defining kernels and communication links between those kernels. In one embodiment, the graph is formed using nodes (e.g., kernels) which are communicatively coupled by edges (e.g., the communication links between the kernels). A compiler converts the source code into a bitstream and binary code which configures programmable logic and software-configurable hardened logic in a heterogeneous processing system of a SoC to execute the graph. Rather than requiring the programmer to understand in detail the programmable and software-configurable hardened hardware in the heterogeneous processing system, the compiler can use the graph expressed in source code to determine which kernels to assign to programmable logic blocks and which to assign to hardened logic blocks. Further, the compiler can, using the parameters provided in the graph source code, select the specific communication techniques to establish the communication links between the kernels (e.g., shared memory, windowing, direct memory access (DMA), etc.). Furthermore, the compiler can automatically determine whether synchronization should be used in a communication link and set up that synchronization without input from the programmer - i.e., without the programmer providing the details of the synchronization within the graph source code. Thus, the programmer can express the dataflow graph at a high-level (using source code) without understanding how the dataflow graph is implemented using the programmable and hardened hardware in the SoC. As a result, the graph source code is independent of a hardware design of a particular SoC and can be implemented (using the compiler) onto multiple different types of SoCs each having different hardware designs.

<FIG> is a block diagram of a SoC <NUM> that includes a data processing engine (DPE) array <NUM>, according to an example. The DPE array <NUM> includes a plurality of DPEs <NUM> which may be arranged in a grid, cluster, or checkerboard pattern in the SoC <NUM>. Although <FIG> illustrates arranging the DPEs <NUM> in a 2D array with rows and columns, the embodiments are not limited to this arrangement. Further, the array <NUM> can be any size and have any number of rows and columns formed by the DPEs <NUM>.

In one embodiment, the DPEs <NUM> are identical. That is, each of the DPEs <NUM> (also referred to as tiles or blocks) may have the same hardware components or circuitry. Further, the embodiments herein are not limited to DPEs <NUM>. Instead, the SoC <NUM> can include an array of any kind of processing elements, for example, the DPEs <NUM> could be digital signal processing engines, cryptographic engines, Forward Error Correction (FEC) engines, or other specialized hardware for performing one or more specialized tasks.

In <FIG>, the array <NUM> includes DPEs <NUM> that are all the same type (e.g., a homogeneous array). However, in another embodiment, the array <NUM> may include different types of engines. For example, the array <NUM> may include digital signal processing engines, cryptographic engines, graphic processing engines, and the like. Regardless if the array <NUM> is homogenous or heterogeneous, the DPEs <NUM> can include direct connections between DPEs <NUM> which permit the DPEs <NUM> to transfer data directly as described in more detail below.

In one embodiment, the DPEs <NUM> are formed from software-configurable hardened logic - i.e., are hardened. One advantage of doing so is that the DPEs <NUM> may take up less space in the SoC <NUM> relative to using programmable logic to form the hardware elements in the DPEs <NUM>. That is, using hardened logic circuitry to form the hardware elements in the DPE <NUM> such as program memories, an instruction fetch/decode unit, fixed-point vector units, floating-point vector units, arithmetic logic units (ALUs), multiply accumulators (MAC), and the like can significantly reduce the footprint of the array <NUM> in the SoC <NUM>. Although the DPEs <NUM> may be hardened, this does not mean the DPEs <NUM> are not programmable. That is, the DPEs <NUM> can be configured when the SoC <NUM> is powered on or rebooted to perform different functions or tasks.

The DPE array <NUM> also includes a SoC interface block <NUM> (also referred to as a shim) that serves as a communication interface between the DPEs <NUM> and other hardware components in the SoC <NUM>. In this example, the SoC <NUM> includes a network on chip (NoC) <NUM> that is communicatively coupled to the SoC interface block <NUM>. Although not shown, the NoC <NUM> may extend throughout the SoC <NUM> to permit the various components in the SoC <NUM> to communicate with each other. For example, in one physical implementation, the DPE array <NUM> may be disposed in an upper right portion of the integrated circuit forming the SoC <NUM>. However, using the NoC <NUM>, the array <NUM> can nonetheless communicate with, for example, programmable logic (PL) <NUM>, a processor subsystem (PS) <NUM> or input/output (I/O) <NUM> which may disposed at different locations throughout the SoC <NUM>.

In addition to providing an interface between the DPEs <NUM> and the NoC <NUM>, the SoC interface block <NUM> may also provide a connection directly to a communication fabric in the PL <NUM>. In this example, the PL <NUM> and the DPEs <NUM> form a heterogeneous processing system since some of the kernels in a dataflow graph may be assigned to the DPEs <NUM> for execution while others are assigned to the PL <NUM>. While <FIG> illustrates a heterogeneous processing system in a SoC, in other examples, the heterogeneous processing system can include multiple devices or chips. For example, the heterogeneous processing system could include two FPGAs or other specialized accelerator chips that are either the same type or different types. Further, the heterogeneous processing system could include two communicatively coupled SoCs.

This can be difficult for a programmer to manage since communicating between kernels disposed in heterogeneous or different processing cores can include using the various communication interfaces shown in <FIG> such as the NoC <NUM>, the SoC interface block <NUM>, as well as the communication links between the DPEs <NUM> in the array <NUM> (which as shown in <FIG>).

In one embodiment, the SoC interface block <NUM> includes separate hardware components for communicatively coupling the DPEs <NUM> to the NoC <NUM> and to the PL <NUM> that is disposed near the array <NUM> in the SoC <NUM>. In one embodiment, the SoC interface block <NUM> can stream data directly to a fabric for the PL <NUM>. For example, the PL <NUM> may include an FPGA fabric which the SoC interface block <NUM> can stream data into, and receive data from, without using the NoC <NUM>. That is, the circuit switching and packet switching described herein can be used to communicatively couple the DPEs <NUM> to the SoC interface block <NUM> and also to the other hardware blocks in the SoC <NUM>. In another example, SoC interface block <NUM> may be implemented in a different die than the DPEs <NUM>. In yet another example, DPE array <NUM> and at least one subsystem may be implemented in a same die while other subsystems and/or other DPE arrays are implemented in other dies. Moreover, the streaming interconnect and routing described herein with respect to the DPEs <NUM> in the DPE array <NUM> can also apply to data routed through the SoC interface block <NUM>.

Although <FIG> illustrates one block of PL <NUM>, the SoC <NUM> may include multiple blocks of PL <NUM> (also referred to as configuration logic blocks) that can be disposed at different locations in the SoC <NUM>. For example, the SoC <NUM> may include hardware elements that form a field programmable gate array (FPGA). However, in other embodiments, the SoC <NUM> may not include any PL <NUM> - e.g., the SoC <NUM> is an ASIC.

<FIG> is a block diagram of a DPE <NUM> in the DPE array <NUM> illustrated in <FIG>, according to an example. The DPE <NUM> includes an interconnect <NUM>, a core <NUM>, and a memory module <NUM>. The interconnect <NUM> permits data to be transferred from the core <NUM> and the memory module <NUM> to different cores in the array <NUM>. That is, the interconnect <NUM> in each of the DPEs <NUM> may be connected to each other so that data can be transferred north and south (e.g., up and down) as well as east and west (e.g., right and left) in the array of DPEs <NUM>.

Referring back to <FIG>, in one embodiment, the DPEs <NUM> in the upper row of the array <NUM> relies on the interconnects <NUM> in the DPEs <NUM> in the lower row to communicate with the SoC interface block <NUM>. For example, to transmit data to the SoC interface block <NUM>, a core <NUM> in a DPE <NUM> in the upper row transmits data to its interconnect <NUM> which is in turn communicatively coupled to the interconnect <NUM> in the DPE <NUM> in the lower row. The interconnect <NUM> in the lower row is connected to the SoC interface block <NUM>. The process may be reversed where data intended for a DPE <NUM> in the upper row is first transmitted from the SoC interface block <NUM> to the interconnect <NUM> in the lower row and then to the interconnect <NUM> in the upper row that is the target DPE <NUM>. In this manner, DPEs <NUM> in the upper rows may rely on the interconnects <NUM> in the DPEs <NUM> in the lower rows to transmit data to and receive data from the SoC interface block <NUM>.

In one embodiment, the interconnect <NUM> includes a configurable switching network that permits the user to determine how data is routed through the interconnect <NUM>. In one embodiment, unlike in a packet routing network, the interconnect <NUM> may form streaming point-to-point connections. That is, the streaming connections and streaming interconnects (not shown in <FIG>) in the interconnect <NUM> may form routes from the core <NUM> and the memory module <NUM> to the neighboring DPEs <NUM> or the SoC interface block <NUM>. Once configured, the core <NUM> and the memory module <NUM> can transmit and receive streaming data along those routes. In one embodiment, the interconnect <NUM> is configured using the Advanced Extensible Interface (AXI) <NUM> Streaming protocol.

In addition to forming a streaming network, the interconnect <NUM> may include a separate network for programming or configuring the hardware elements in the DPE <NUM>. Although not shown, the interconnect <NUM> may include a memory mapped interconnect which includes different connections and switch elements used to set values of configuration registers in the DPE <NUM> that alter or set functions of the streaming network, the core <NUM>, and the memory module <NUM>.

In one embodiment, streaming interconnects (or network) in the interconnect <NUM> support two different modes of operation referred to herein as circuit switching and packet switching. In one embodiment, both of these modes are part of, or compatible with, the same streaming protocol - e.g., an AXI Streaming protocol. Circuit switching relies on reserved point-to-point communication paths between a source DPE <NUM> to one or more destination DPEs <NUM>. In one embodiment, the point-to-point communication path used when performing circuit switching in the interconnect <NUM> is not shared with other streams (regardless whether those streams are circuit switched or packet switched). However, when transmitting streaming data between two or more DPEs <NUM> using packet-switching, the same physical wires can be shared with other logical streams.

The core <NUM> may include hardware elements for processing digital signals. For example, the core <NUM> may be used to process signals related to wireless communication, radar, vector operations, machine learning applications, and the like. As such, the core <NUM> may include program memories, an instruction fetch/decode unit, fixed-point vector units, floating-point vector units, arithmetic logic units (ALUs), multiply accumulators (MAC), and the like. However, as mentioned above, this disclosure is not limited to DPEs <NUM>. The hardware elements in the core <NUM> may change depending on the engine type. That is, the cores in a digital signal processing engine, cryptographic engine, or FEC may be different.

The memory module <NUM> includes a direct memory access (DMA) engine <NUM>, memory banks <NUM>, and hardware synchronization circuitry (HSC) <NUM> or other type of hardware synchronization block. In one embodiment, the DMA engine <NUM> enables data to be received by, and transmitted to, the interconnect <NUM>. That is, the DMA engine <NUM> may be used to perform DMA reads and write to the memory banks <NUM> using data received via the interconnect <NUM> from the SoC interface block or other DPEs <NUM> in the array.

The memory banks <NUM> can include any number of physical memory elements (e.g., SRAM). For example, the memory module <NUM> may be include <NUM>, <NUM>, <NUM>, <NUM>, etc. different memory banks <NUM>. In this embodiment, the core <NUM> has a direct connection <NUM> to the memory banks <NUM>. Stated differently, the core <NUM> can write data to, or read data from, the memory banks <NUM> without using the interconnect <NUM>. That is, the direct connection <NUM> may be separate from the interconnect <NUM>. In one embodiment, one or more wires in the direct connection <NUM> communicatively couple the core <NUM> to a memory interface in the memory module <NUM> which is in turn coupled to the memory banks <NUM>.

In one embodiment, the memory module <NUM> also has direct connections <NUM> to cores in neighboring DPEs <NUM>. Put differently, a neighboring DPE in the array can read data from, or write data into, the memory banks <NUM> using the direct neighbor connections <NUM> without relying on their interconnects or the interconnect <NUM> shown in <FIG>. The HSC <NUM> can be used to govern or protect access to the memory banks <NUM>. In one embodiment, before the core <NUM> or a core in a neighboring DPE can read data from, or write data into, the memory banks <NUM>, the HSC <NUM> provides a lock to an assigned portion of the memory banks <NUM> (referred to as a "buffer"). That is, when the core <NUM> wants to write data, the HSC <NUM> provides a lock to the core <NUM> which assigns a portion of a memory bank <NUM> (or multiple memory banks <NUM>) to the core <NUM>. Once the write is complete, the HSC <NUM> can release the lock which permits cores in neighboring DPEs to read the data.

Because the core <NUM> and the cores in neighboring DPEs <NUM> can directly access the memory module <NUM>, the memory banks <NUM> can be considered as shared memory between the DPEs <NUM>. That is, the neighboring DPEs can directly access the memory banks <NUM> in a similar way as the core <NUM> that is in the same DPE <NUM> as the memory banks <NUM>. Thus, if the core <NUM> wants to transmit data to a core in a neighboring DPE, the core <NUM> can write the data into the memory bank <NUM>. The neighboring DPE can then retrieve the data from the memory bank <NUM> and begin processing the data. In this manner, the cores in neighboring DPEs <NUM> can transfer data using the HSC <NUM> while avoiding the extra latency introduced when using the interconnects <NUM>. In contrast, if the core <NUM> wants to transfer data to a non-neighboring DPE in the array (i.e., a DPE without a direct connection <NUM> to the memory module <NUM>), the core <NUM> uses the interconnects <NUM> to route the data to the memory module of the target DPE which may take longer to complete because of the added latency of using the interconnect <NUM> and because the data is copied into the memory module of the target DPE rather than being read from a shared memory module.

In addition to sharing the memory modules <NUM>, the core <NUM> can have a direct connection to cores <NUM> in neighboring DPEs <NUM> using a core-to-core communication link (not shown). That is, instead of using either a shared memory module <NUM> or the interconnect <NUM>, the core <NUM> can transmit data to another core in the array directly without storing the data in a memory module <NUM> or using the interconnect <NUM> (which can have buffers or other queues). For example, communicating using the core-to-core communication links may use less latency (or have high bandwidth) than transmitting data using the interconnect <NUM> or shared memory (which requires a core to write the data and then another core to read the data) which can offer more cost effective communication. In one embodiment, the core-to-core communication links can transmit data between two cores <NUM> in one clock cycle. In one embodiment, the data is transmitted between the cores on the link without being stored in any memory elements external to the cores <NUM>. In one embodiment, the core <NUM> can transmit a data word or vector to a neighboring core using the links every clock cycle, but this is not a requirement.

In one embodiment, the communication links are streaming data links which permit the core <NUM> to stream data to a neighboring core. Further, the core <NUM> can include any number of communication links which can extend to different cores in the array. In this example, the DPE <NUM> has respective core-to-core communication links to cores located in DPEs in the array that are to the right and left (east and west) and up and down (north or south) of the core <NUM>. However, in other embodiments, the core <NUM> in the DPE <NUM> illustrated in <FIG> may also have core-to-core communication links to cores disposed at a diagonal from the core <NUM>. Further, if the core <NUM> is disposed at a bottom periphery or edge of the array, the core may have core-to-core communication links to only the cores to the left, right, and bottom of the core <NUM>.

However, using shared memory in the memory module <NUM> or the core-to-core communication links may be available if the destination of the data generated by the core <NUM> is a neighboring core or DPE. For example, if the data is destined for a non-neighboring DPE (i.e., any DPE that DPE <NUM> does not have a direct neighboring connection <NUM> or a core-to-core communication link), the core <NUM> uses the interconnects <NUM> in the DPEs to route the data to the appropriate destination. As mentioned above, the interconnects <NUM> in the DPEs <NUM> may be configured when the SoC is being booted up to establish point-to-point streaming connections to non-neighboring DPEs to which the core <NUM> will transmit data during operation.

<FIG> illustrate a memory module 230A shared by multiple DPEs <NUM> in a DPE array, according to an example. As shown, the memory module 230A has direct connections to four cores - i.e., cores 210A-D. The memory module 230A is in the same DPE (i.e., DPE 110A) as the core 210A. As such, the direct connection <NUM> is an intra-engine connection. However, the memory module 230A is in a different DPE than the cores 210B-D. As such, the direct neighboring connections 240A-C are inter-engine connections since these connections <NUM> span across an interface between DPEs <NUM> in the array. For clarity, the interconnects in each of the DPEs <NUM> have been omitted.

In <FIG>, the memory module 230A in the DPE 110A is disposed to the right of the core <NUM>0A. The same is true for the DPE 110D located to the right of the DPE 110A (i.e., is east of the DPE 110A). As such, the core 210D in the DPE 110D directly neighbors the memory module 230A which makes establishing the direct neighboring connection 240B between the memory module 230A and the core 210D easier than if the memory module 230D were disposed to the left of the core 210D - i.e., if the memory module 230D were disposed between the memory module 230A and the core 210D.

Unlike the DPEs 110A and 110D, in the DPEs 110B and 110C, the cores 210B and 210C are disposed to the right of the memory modules 230B and 230C. As a result, the cores 210B and 210C are disposed directly above and directly below the memory module 230A (i.e., the cores 210B and 210C are north and south of the memory module 230A). Doing so makes establishing the direct neighboring connections 240A and 240C between the shared memory module 230A and the cores <NUM>0B and 210C easier than if the cores 210B and 210C were disposed to the left of the memory modules 230B and 230C. Using the arrangement shown in <FIG>, the memory module 230A has direct connections <NUM> and <NUM> to the cores 210A-D that are located in the same DPE and neighboring DPEs which means the memory module 230A is a shared memory for the DPEs 110A-D. Although <FIG> illustrates sharing the memory module 230A between four cores <NUM>, in other embodiments the memory module 230A may be shared by more or fewer cores. For example, the memory module 230A may also have direct connections to neighboring DPEs that are arranged at a diagonal relative to the DPE 110A.

The arrangement of the DPEs <NUM> illustrated in <FIG> is just one example of a suitable arrangement of the DPEs <NUM> to provide direct connections to the memory module 230A from the neighboring cores <NUM>. In <FIG>, the DPEs <NUM> in the different rows are staggered. That is, instead of the DPEs <NUM> in the same column being aligned, the DPEs <NUM> are offset. In this arrangement, the cores 210B and 210C are disposed to the left of the memory modules 230B and 230C (unlike what is shown in <FIG>) and still are directly above and beneath the shared memory module 230A by shifting the DPEs 110B and 110C to the right relative to the DPE 110A. As such, the direct connection 240A-C can be formed in the SoC to enable the memory module 230A to be shared by the cores 210A-D.

Moreover, although not shown in <FIG>, the memory modules 230B-D may also be shared memory modules. For example, the memory module 230D may have direct connection to cores in DPEs that are disposed above, below, and to the right (i.e., to the north, south, and east) of the DPE 110D. In this manner, the memory module 230D can be shared with cores in neighboring DPEs. However, the memory modules <NUM> in DPEs disposed at the edges or periphery of the array may be shared by fewer numbers of cores (or may not be shared at all).

<FIG> is a block diagram of a computing system <NUM> for implementing a dataflow graph <NUM> on the SoC <NUM> illustrated in <FIG>, according to an example. The system <NUM> includes a host <NUM> (e.g., a host computing system) which includes a processor <NUM> and memory <NUM>. The processor <NUM> represents any number of processing elements which each can contain any number of processing cores. The memory <NUM> can include volatile and non-volatile memory elements. Moreover, the memory <NUM> can be disposed within the same apparatus (e.g., a server) or can be distributed across the computing system <NUM> (e.g., a cloud computing environment).

The memory <NUM> includes a heterogeneous programming environment <NUM> for generating graph source code <NUM>, kernel source code <NUM>, control source code <NUM>. The memory <NUM> also includes a compiler <NUM>. The graph source code <NUM> can be written in various types of object orientated programming languages (e.g., C++, Python, Javascript, Swift, Go, LabView, or Simulink). Generally, the graph source code <NUM> defines kernels (e.g., nodes) which are connected via communication links (e.g., edges). The combination of the kernels and the communication links form the graph <NUM>.

One advantage of providing a heterogeneous programming environment <NUM> for defining a dataflow graph <NUM> using the source code <NUM> is that different aspects of compiling dataflow graphs on the heterogeneous processing system can be directly expressed and controlled in the heterogeneous programming environment <NUM>. A programmer can start with a parallel definition (e.g., the graph) which the compiler <NUM> then implements in the hardware of the SoC <NUM>. The graph <NUM> enables the data to flow between the nodes (e.g., the kernels) in a continuous pipelined manner. A node starts processing as soon as the data at its inputs is available, otherwise it stalls. Moreover, the graph <NUM> provides the programmer with significant freedom to map the computation and the data flows to DPEs <NUM> and programmable logic <NUM> in the SoC <NUM>.

While various types of dataflow graphs can be used, in one embodiment, the semantics of the graph <NUM> established by the graph source code <NUM> is based upon the general theory of Kahn Process Networks which provides a computation model for deterministic parallel computation that is applied to the heterogeneous architecture in the SoC <NUM> (which includes both programmable and hardened blocks). Moreover, the graph source code <NUM> is tolerant for communication latencies between the nodes in the graph <NUM>, and as a result, extends naturally to graphs that map to multiple super logic regions and multiple SoC devices (e.g., multiple FPGAs). For example, the graph source code <NUM> can include a first plurality of kernels which the compiler assigns to a first chip (e.g., a SoC, FPGA, etc.) and a second plurality of kernels the compiler assigns to a second chip. The first and second plurality of kernels can be part of the same dataflow graph, and as such, may communicate with each other when executed on the first and second chips.

Another advantage of using the source code <NUM> to define a dataflow graph is that a sequential program, in contrast, fixes the control flow and the order of computation. When using a dataflow graph, predictable and reproducible responses to input are obtained without a race condition. While there is a risk of deadlock, this can be solved or mitigated by managing the storage assigned to each node or kernel.

The kernel source code <NUM> can be written in various types of object orientated programming languages. The kernel source code <NUM> defines the attributes of a particular kernel or node in the dataflow graph <NUM>. In one embodiment, the kernel source code <NUM> defines the operation of each kernel within the graph source code <NUM>.

The control source code <NUM> can be written in various types of object orientated programming languages. In one embodiment, the control source code <NUM> defines a control program, that when executed, controls the execution of the graph <NUM> when implemented on the SoC <NUM>. For example, the control source code <NUM> may control when the graph <NUM> executes, the number of iterations the graph <NUM> executes, and when the graph <NUM> stops executing. The control program generated from the control source code <NUM> can execute on the host <NUM> (e.g., in a datacenter solution) or within the SoC <NUM> (e.g., the PS <NUM>).

The compiler <NUM> is a software application that can compile the source code <NUM>, <NUM>, and <NUM>. For example, using the graph source code <NUM> (and other libraries not shown in <FIG>), the compiler <NUM> can generate the graph <NUM> which can be implemented on the SoC <NUM> which will be described in more detail below. In one embodiment, the graph <NUM> includes a bitstream <NUM> that configures the programmable logic in the SoC <NUM> (e.g., the PL <NUM>, NoC <NUM>, SoC Interface block <NUM>, and I/O <NUM>) and binary code <NUM> (which can include many targeted commands) which configures the software-configurable hardened logic in the SoC <NUM> (e.g., the DPEs <NUM> and PS <NUM>). The bitstream <NUM> and the binary code <NUM> may be transmitted over a memory bus to the SoC <NUM> to configure the SoC <NUM> to execute the graph <NUM>.

<FIG> is a flowchart of a method <NUM> for compiling source code to implement a dataflow graph on a SoC with programmable logic and software-configurable hardened logic, according to an example. At block <NUM>, the host provides a heterogeneous programming environment for defining a dataflow graph as object oriented source code (e.g., C++, Python, Javascript, Swift, Go, LabView, or Simulink). That is, the programmer uses the heterogeneous programming environment (which is described in more detail in <FIG>) to generate source code that defines the dataflow graph. At block <NUM>, the compiler receives the source code establishing the dataflow graph that defines kernel and communication links between the kernels. In one embodiment, the source code received by the compiler includes graph source code.

For clarity, <FIG> are discussed in tandem with the blocks described in method <NUM>.

<FIG> is graph source code <NUM> for defining a dataflow graph, according to an example. That is, <FIG> is one example of graph source code <NUM> generated in a heterogeneous programming environment that permits a programmer to define a plurality of kernels and communication links for establishing a dataflow graph. The source code <NUM> using a namespace "Namespace A" which may reference one or more libraries which can be used to define a dataflow graph in the source code <NUM>. In one embodiment, the graph source code <NUM> can be thought of establishing a data structure in the heterogeneous programming environment which the programmer builds using the kernels <NUM> and communication links <NUM>.

In this example, the graph source code <NUM> includes six kernels <NUM>: a, b, c, d, e, and f. The kernels <NUM> are defined within the class "radio". While <FIG> illustrates source code <NUM> for performing a radio function, as mentioned above, the techniques described herein can be used for a plurality of different functions such as radar, vector operations, machine learning applications, and the like.

The source code <NUM> includes wrappers 610A-F that define the function or operation performed by each of the kernels <NUM>. The wrappers <NUM> create mechanisms to invoke the corresponding C++ function (e.g., polarclip, feedback, equalizer, fir_tap11, fir_tap7, and scale). That is, the wrappers <NUM> permit the programmer to define the kernel using the example functions which may be part of another C++ library. In this example, the kernels <NUM> are functions calls rather than single instructions. In one embodiment, a kernel <NUM> executes only when the kernel <NUM> receives the data from all its triggering inputs and executes in a non-blocking manner to produce output which can be transmitted to a downstream kernel <NUM>. A kernel may also block during execution on a stream input if the stream data is not present when accessed.

One advantage of abstracting the kernels as function calls using the wrappers <NUM> is that doing so means the programmer can express kernels that are to be executed on the DPEs or the programmable logic in the same uniform framework. The programmer writes the kernels <NUM> differently but the kernels <NUM> are packaged in the same way and can be expressed in the same framework. The programmer does not need to worry about integrating kernels assigned to the DPE with kernels assigned to the PL fabric. Here, the programmer selects or indicates the types of communication links <NUM> in the graph source code <NUM> and all the synchronization between the kernels <NUM> using those types of communication links <NUM> is handled by the compiler.

The source code <NUM> also includes constraints <NUM> which include instructions to limit how the compiler maps the objects defined in the source code <NUM> (e.g., the kernels <NUM> and the communication links <NUM>) to the hardware in the SoC. In this example, the constraints <NUM> instruct the compiler to assign the kernels a and f to the fabric (e.g., the programmable logic) in the SoC rather than assigning these kernels to a DPE. For reasons described below, assigning the kernels a and f to the fabric rather than the DPEs can offer performance improvement. Thus, while the graph source code <NUM> does not require the programmer to assign the kernels <NUM> to the hardware in the SoC (and thus the programmer does not need to understand the underlying hardware architecture of the SoC), the namespace provided to the programmer permits her to use the constraints <NUM> to instruct the compiler how to assign one or all of the kernels <NUM> if the programmer knows doing so improves performance.

The communication links <NUM> define how data is communicated between the kernels <NUM>. For example, the communication link 620A indicates that streaming data is converted into window data which has a length of <NUM> bytes. Further, each window is transmitted with an <NUM> byte overlap. However, for communication link 620B, windowing data of length <NUM> bytes is transmitted between kernel b and kernel c without any overlapping data. The details of windowing data (and overlapping the windows) are described in more detail below.

Further, each communication link <NUM> defines which port on the upstream kernel is connected to which port on the downstream kernel. For example, in link 620A, the output port a. out[<NUM>] of kernel a is coupled to the input port b. in[<NUM>] of kernel b. Each kernel can have multiple input ports and multiple output ports. For example, in communication link 620D, a first output port d. out[<NUM>] of kernel d is coupled to the input port e. Also, in communication link 620F, a second output port d. out[<NUM>] of kernel d is coupled to the input port f.

Like how the graph source code <NUM> abstracts the kernels <NUM> so they can be expressed in the same uniform framework, the source code <NUM> can abstract (or hide) synchronization on the communication links <NUM> from the programmer. As described in more detail below, the compiler can select the optimal communication technique to transmit data between the kernels <NUM> based on whether the kernels <NUM> are in the fabric or in the DPE array, or whether the kernels <NUM> are neighbors in the DPE array.

In one embodiment, the ability to define the kernels <NUM>, wrappers <NUM>, constraints <NUM>, and communication links <NUM> in the graph source code <NUM> are tools provided by the heterogeneous programming environment (and supported by the libraries in the namespace) that permit a programmer to generate object orientated source code that implements a dataflow graph.

<FIG> illustrates a dataflow graph <NUM> defined by the source code <NUM> in <FIG>, according to an example. That is, the graph <NUM> is a graphical representation of the graph defined by the graph source code <NUM>. As shown, the graph <NUM> includes the six kernels a-f that are communicatively coupled using the communication links 620A-E. Further, the graph <NUM> includes an input <NUM> which transfers data into the kernel a and an output <NUM> that receives data from the output of the kernel f. The data received at the input <NUM> can be provided by, e.g., an application executing on the host, a radio transceiver, a camera, or from a file or database. The output <NUM> can transmit data processed by the graph <NUM> to the host or into a file or database.

<FIG> is an abstract view of the graph <NUM> where the kernels (e.g., nodes) are coupled by the links <NUM> at respective input and output ports. That is, <FIG> illustrates the data flow between the kernels a-f using the links 620A-F but does not illustrate the hardware implementation on which the kernels are executed or the particular type of communication link <NUM> being used - e.g., shared memory, NoC, DMA, etc. Nonetheless, the programmer can design the graph <NUM> at the abstract view illustrated in <FIG> and then the compiler can implement the kernels a-f and the communication links <NUM> in the hardware of the SoC.

<FIG> is kernel source code <NUM> for defining a kernel in a dataflow graph, according to an example. In one embodiment, the wrapper <NUM> in the source code in <FIG> permits the arguments of the function defined by the kernel to be accessed as ports. In <FIG>, the kernel source code <NUM> includes arguments <NUM> that specify a pointer (i.e., *inputw) to the input data and a pointer (*outputw) to the output data. When two kernels are communicatively coupled by a link as described above, the compiler can allocate data memory which is supplied to the kernel (or the function called by the kernel) when the kernel is called. In one embodiment, the kernel operates on the input data provided by the arguments <NUM> using an application programming interface (API).

In <FIG>, the kernel source code <NUM> includes window APIs for processing the input data before it is outputted. For example, the window_readincr is an API which reads the next window using the pointer inputw. Once the operation is performed, which is illustrated here generally as performing math using sbuff, another API can be used to output the processed data - e.g., window_writeincr.

In one embodiment, the programmer generates kernel source code for each kernel defined in the graph source code. However, if the graph source code has multiple instances of the same kernel, these multiple instances can be defined using the same kernel source code.

Returning to the method <NUM>, at block <NUM> the compiler compiles the source code (e.g., the graph, kernel, and control source code). For ease of explanation, this compilation is divided into at least three sub-blocks. At block <NUM>, the compiler assigns the kernels to the DPEs and programmable logic in the SoC. The compiler can use constraints provided by the programmer in the source code (e.g., the constraints <NUM> in <FIG>), but absent constraints, can assign the kernels in the graph source code to the DPEs and the programmable logic in the SoC.

In one embodiment, the compiler evaluates the graph to determine how to assign the kernels to the hardware in the SoC. For example, if two kernels are communicatively coupled to each other in the graph, the compiler may assign the kernels to neighboring DPEs in the DPE array to take advantage of faster communication protocol such as shared memory between the DPEs. Further, the compiler may determine the cycle count and the fraction of time used by each of the kernels to determine whether multiple kernels can be assigned to the same DPE.

<FIG> is an abstract view of implementing the dataflow graph <NUM> in <FIG>, according to an example. <FIG> illustrates the kernels a-f as well as the communication links <NUM>. Further, <FIG> illustrates the hardware on which the kernels are assigned in the SoC. As shown, the kernels a and f are disposed in the PL <NUM>, the kernels b and c are implemented in the DPE 110A, and the kernels d and e are implemented in the DPE 110B.

In one embodiment, the compiler chose to place the kernels a and f in the PL <NUM> based on the constraint provided in the graph source code. However, in another embodiment, the compiler may have recognized these kernels as input/output kernels which may be better suited for being implemented in programmable logic rather than the DPEs.

The compiler may have assigned the kernels b and c to the same DPE 110A using the estimated fraction of the cycle count of each kernel or in response to a constraint from the programmer. This is referred to generally as clustering. For example, if the kernel b uses only <NUM>% of the cycle count of the DPE 110A and the kernel c uses only <NUM>% of the cycle count, then the compiler can place them on the same DPE 110A. In another example, the programmer may use a constraint to instruct the compiler to place the kernels b and c on the same DPE 110A. That way, although the programmer describes the graph as a parallelized data structure, the programmer can use the estimate cycle counts of the kernels to force some of the kernels to be sequential - i.e., assigned to the same DPE. That is, because each DPE can execute only one task at a time (i.e., are not parallelized), placing two different kernels on the same DPE means only one of the kernels can execute at a time rather than the scenario where the kernels are assigned to their own DPEs. However, this clustering would still meet the overall cycle count.

Returning to the method <NUM>, at block <NUM> the compiler assigns the connections between the kernels to streaming or windowing. In one embodiment, these connections are controlled by the communication links defined in the graph source code. That is, the programmer can indicate how data should be passed between each pair of kernels. In another example, the compiler assigns a DMA engine <NUM> in the memory module <NUM> of one DPE <NUM> to transfer window data from memory bank <NUM> to another DPE <NUM> through the interconnect <NUM>. In yet another example, the compiler assigns a stream channel on the interconnect <NUM> and a stream channel on the receiving core <NUM> or the receiving DMA engine <NUM>.

At block <NUM>, the compiler selects synchronization techniques for transferring data between the kernels. This is illustrated in <FIG> where the communication links 620A-F (which, in this example, use windowing) include either a double buffer <NUM> or a single buffer <NUM> to transmit data between the kernels. If the kernels are on different (or heterogeneous) processing cores (e.g., PL <NUM> versus the DPEs <NUM>) as in the case with the link 620A between kernels a and b and the link 620F between the kernels d and f, the compiler assigns a double buffer <NUM>. Moreover, if the kernels are on different DPEs as in the case with the link 620C between kernels c and d and the link 620E between kernels e and b, the compiler again uses a double buffer <NUM>. However, for transferring data between kernels on the same DPE as in the case of the link 620B between kernels b and c and the link 620D between kernels d and e, the compiler can assign a single buffer <NUM>. As described below, single buffering may provide lower latency than double buffering.

The compiler also handles synchronization between the kernels when performing double or single buffering. For example, when performing double buffering, the compiler can establish a locking protocol for accessing the double buffers <NUM> which may not be needed when performing single buffering (e.g., when the kernels are on the same DPE <NUM>). In another example, the compiler may select a ping/pong synchronization technique for the double buffers <NUM>. In any case, the synchronization can be established by the compiler using the parameters provided by the programmer in the source code.

Returning to the method <NUM>, at block <NUM>, the compiler transmits a bitstream and/or binary code (e.g., a series of memory-mapped store transactions) for configuring the SoC to execute the dataflow graph using the compiled source code. That is, the SoC can receive the bitstream/binary code and then execute the graph using the hardware elements stipulated by the compiler. The compiler can determine where each kernel should be placed in the SoC, the type of communication links between those kernels, and the synchronization used by the communication links.

<FIG> is a hardware view <NUM> of implementing the dataflow graph in <FIG> in a SoC, according to an example. That is, the hardware view <NUM> illustrates a portion of the SoC used to implement the dataflow graph illustrated in <FIG>. In this example, <FIG> illustrates a part of the SoC that includes the PL <NUM> and at least a portion of the DPEs in the DPE array which includes five cores <NUM> and five memory modules <NUM>.

The kernels a and f are formed using configurable logic blocks (CLBs) in the PL <NUM>. The kernel a is communicatively coupled to the memory module 230A via the interconnect <NUM>. Although not shown, this communication link between kernel a and the memory module 230A may also include the NoC and the SoC interface block which permits a core <NUM> in the DPE array to communicate with other hardware modules in the SoC (e.g., the PL <NUM>). In this embodiment, the kernel a transmits data to a DMA engine 215A in the memory module 230A which stores the received data into the double buffer 905A in the memory banks 220A. Thus, the compiler has decided to implement the communication link 620A illustrated in <FIG> by assigning the double buffer 905A to the memory banks 220A. Using DMA writes, the kernel a can store data in the double buffer 905A which can then be accessed by the kernel b hosted on the core 210B.

In this example, the double buffer 905A is assigned four of the banks in the memory banks 220A. In one embodiment, each memory bank holds <NUM> bytes which means that the total size of the double buffer 905A is <NUM> bytes. However, the compiler can assign more memory banks or fewer memory banks to the double buffer 905A depending on the expected needs of the kernels a and b. The kernel a can write data into two of the memory banks 220A in the double buffer 905A while the kernel b is reading data out of the other two memory banks 220A in the buffer 905A. In one embodiment, the compiler establish a ping/pong synchronization protocol between the kernels a and b so that the kernels do not try to access the same pair of memory banks. As mentioned above, the compiler can handle the synchronization protocol so that the kernel a on the PL <NUM> can communicate with the kernel b on the core 210B with only the programmer indicating the type of communication (e.g., windowing or streaming) that should occur between these kernels in the graph source code.

In one embodiment, because the core 210B which host the kernel b directly neighbors the memory module 230A, kernel b can directly access the double buffer 905A without having to the use the interconnect <NUM> (unlike kernel a). Thus, when assigning the double buffer 905A and the kernel b to hardware elements, the compiler selected a memory module 230A and core 210B which directly neighbor each other so that the kernel b can use the direct connection between the core 210B and the memory module 230A which has higher throughput than using the interconnect <NUM>.

Because the kernels b and c are hosted or assigned to the same core 210B as shown in <FIG>, the compiler attempts to assign the single buffer 910A to a neighboring memory module <NUM>. In this case, the compiler assigned the single buffer 910A to the memory module 230C but could have used any of the neighboring memory modules - e.g., modules 230A or 230B. The compiler may have selected the memory module 230C rather than the modules 230A or 230B so that these memory modules have more available space to be used by cores further north in the array (not shown). Regardless of the reason, the kernels b and c can use the direct connection between the core 210B and the memory module 230C to transfer data into and out of the single buffer 910A. Because the kernels b and c are assigned to the same core 210B and as a result are executed sequentially rather in parallel, a single buffer 910A rather than a double buffer is sufficient since only one of the kernels is being executed by the core 210B at any given time. In this example, the single buffer 910A includes two banks of the memory banks 220C but the compiler can assign more banks or fewer banks depending on the expected needs of the kernels b and c.

For the inter-core communication link between kernel c and kernel d (which is illustrated as communication link 620C in <FIG>), the compiler assigns the double buffer 905B to the memory banks 220B in the memory module 230B. As above, the compiler may establish a ping/pong synchronization protocol for the kernels c and d to simultaneously write and read two respective pairs of memory banks 220B in the double buffer 905B. Moreover, by using a memory module 230B which neighbors both the core 210B which hosts kernel c and the core 210C which hosts the kernel d, the compiler takes advantage of the direct connections these cores 210B-C have to the memory module 230B for reading and storing data in the double buffer 905B.

For the intra-core communication link between kernels d and e (which is illustrated as communication link 620D in <FIG>), the compiler assigns the single buffer 910B to the memory module 230C. Like with the communication link between the kernels b and c, the single buffer 910B is sufficient since the kernels d and e are executed sequentially on the core 210C.

For the inter-core communication link between kernels e and b (which is illustrated as communication link 620E in <FIG>), the compiler assigns the double buffer 905D to the remaining four memory banks 220C in the memory module 230C which are not being used by the single buffers 910A and 910B. The compiler can again establish a synchronization protocol between the kernels b and e for accessing the double buffer 905D.

For the heterogeneous communication link between kernels d and f (which is illustrated as communication link 620F in <FIG>) where the kernels are hosted on different types of processing cores (e.g., the PL <NUM> and the DPE containing the core 210C), the compiler assigns the double buffer 905C to the memory banks 220D in the memory module 230D. The kernel d can access the double buffer 905C using the direct connection between the core 210C and the memory module 230D. However, because the kernel f is hosted on the PL <NUM> rather than one of the cores <NUM>, the kernel f can access the double buffer 905C using the DMA engine 215D and the interconnect (as well as the NoC and the SoC interface buffer which are not shown). The compiler can again establish a synchronization protocol between the kernels d and f to permit them to access the double buffer 905C in parallel.

While <FIG> illustrates placing kernels in the DPE array that communicate with each other either in the same core <NUM> or in cores <NUM> that have direct connections to the same memory module, in other embodiments the compiler may place two kernels on cores that do not have direct connections to the same memory module <NUM>. That is, the compiler may assign two kernels that directly communicate in the graph to two non-neighboring cores <NUM>. In that case, the compiler may configure the kernels to perform DMA read/writes or a streaming connection using the interconnect <NUM> (similar to the kernels located in the PL <NUM>) in order to communicate between the kernels rather than using shared memory.

In this manner, the compiler can determine where to place the kernels in the heterogeneous system, determine the type of communication links between the kernels (whether double buffer, single buffer, windowing, or streaming), and establish a synchronization protocol between the kernels using the parameters (e.g., the parameters defining the communication links) defined by the programmer in the source code. However, as mentioned above, the programmer can provide optimization instructions to the compiler using constraints if the programmer knows beforehand an optimal solution for implementing the graph defined in the source code on the SoC.

<FIG> illustrates overlapping windows <NUM> used when transmitting data between kernels, according to an example. In one embodiment, the overlapping windows <NUM> may be formed from streaming data that is received at one kernel (e.g., kernel a in <FIG>) which then chunks up the data to generate the overlapping windows <NUM> illustrated in <FIG>. In another example, the kernel may have received overlapping windows from an upstream kernel and then transmits overlapping windows to a downstream kernel. In one embodiment, the window 1100A is stored in one of the double buffers 905A-D and the window 1100B is in the other buffer due to ping-pong synchronization. The compiler is then responsible for ensuring that the overlap <NUM> is copied from one buffer to the other before the next invocation of the kernel.

Overlapping windows <NUM> may be useful in some embodiments but not in others. For example, overlapping windows <NUM> can be useful in wireless domain so the SoC can maintain the state of a kernel between executing different windows. In one embodiment, after a core finishes executed the kernel, the registers associated with the kernel are cleared and thus the state of the kernel is lost. However, by providing an overlap <NUM> between the windows 1100A and 1100B where the data in the overlap <NUM> is the same, the kernel can regain the state it finished processing the window 1100A when the kernel then begins to process the new data in the window 1100B. Put differently, by processing the overlap <NUM> in the window 1100B (which contains the last samples in the window 1100A), the kernel regains the state it had at the end of processing the window 1100A. The kernel can then begin to process the new data in the window 1100B which was not in the window 1100A. Thus, the block size <NUM> of the window 1100B indicates the new data being processed by the kernel that was not in the previous window 1100A. In this manner, the graph can use windows <NUM> (which can reduce stalls at the kernels relative to streaming data) to process the received data but still maintain an infinite stream illusion by using the overlap <NUM>.

If a communication link between kernels uses windows (rather than streaming), in one embodiment, the receiving kernel does not process the data until a window <NUM> of data is received from all its inputs, which makes processing data non-blocking. Once all the windows <NUM> of data are received, the kernel processes the data without being stalled for further data and outputs a window to the downstream kernel or kernels. For example, the kernel d in <FIG> outputs a window <NUM> of data to both the kernels f and e in parallel using the communication links 620F and 620D, respectively. The window <NUM> of data outputted by the kernel d to the kernels f and e can be the same data or different data.

In another embodiment, the user can program a kernel to determine when it receives input data or outputs data, rather than waiting until all the windows are received or all the data is ready to be outputted. For example, referring back to <FIG>, the communication link 620E is asynchronous where the source code defining kernel b determines when it receives data from the kernel e.

Returning to the method <NUM>, a control program controls the execution of the dataflow graph on the SoC. That is, once the kernels and communication links have been assigned to the various hardware components and configured as illustrated in <FIG>, the control program can provide instructions to the SoC for controlling the execution of the graph. As mentioned above, the control program can execute on a host computing system (as may be preferably in a datacenter) or within the PS of the SoC. In one embodiment, the control program is compiled using control source code.

<FIG> is control source code <NUM> defining a control program for a dataflow graph, according to an example. The source code <NUM> provides connections <NUM> indicating to the compiler how data should be read into the graph and read out from the graph. The main class includes control APIs for initializing the graph (e.g., init()), running the graph (e.g., run()), and ending the graph (e.g., end()). For example, the programmer can use the control source code <NUM> to indicate the number of iterations the graph should run before stopping. This may be useful for debug purposes. However, in other examples, the control program may permit the graph to operate indefinitely depending on the application. These control APIs are discussed in more detail later.

In one embodiment, the programmer may want large look-up tables (LUT) that exceed the size of the memory modules. Once the compiler identifies a large LUT that is too big for any of the memory modules in the DPE array, the compiler can spread the LUT across multiple memory modules. The compiler can allocate the LUT directly onto the array. The programmer can declare the LUT as static data and as an array parameter and connect the static data and the array parameter to a kernel. The compiler treats the LUT as internal data to the kernel (similar to a coefficient table). This declaration of the LUT is in the graph and gets allocated as a graph component. In one embodiment, the large LUTs are not double buffered and are only accessible by one kernel at a time.

In one embodiment, kernels can read/write directly to streams from cores in the DPEs. In the kernel source code, the streams can be declared as function parameters. If data is not available on a streaming port in the core, the kernel can stall (and thus, does not need a locking mechanism). It is an element by element synchronization implemented by the hardware of the stream itself, although the core can stall because no input data is available, there is a memory conflict on a bank, or an output buffer is full.

In one embodiment, if a kernel requires more cycle count than any on core can provide, it is split between cores and cascade streams are used to connect the sub-divided kernel. In the source code, the programmer expresses multiple kernels that are chained together to form a cascade. The overall computing is an accumulated sum of the entire chain. The compiler spreads the computation of the cascaded kernels across multiple cores. The cores perform a cycle by cycle accumulation in a register in the cores, that is, using internal registers in the cores and not using the memory modules. As such, the cores can use register-to-register communication to execute the chain without using the memory modules as buffers (e.g., the single and double buffers described above). In one embodiment, rather than the programmer chaining multiple kernels to form a cascade, the compiler (or some other software application) could perform this transformation where the kernel is split between cores to form the cascade.

<FIG> is a flowchart of a method <NUM> for compiling source code to implement a dataflow graph using constraints, according to an example. At block <NUM>, the compiler identifies a user-defined constraint in source code establishing the dataflow graph. For example, referring to <FIG>, the programmer can add the constraints <NUM> to the graph source code <NUM>. However, in other embodiments, the programmer places constraints in the kernel source code. In still other embodiments, the programmer may define constraints in a separate file. The graph source code can reference or link to the file so that the compiler can identify the constraints when implementing the dataflow graph.

User-defined constraints are external constraints since they are generated by the programmer rather than the compiler when compiling the source code for implementation on the SoC. In one embodiment, the number of external constraints provided by the programmer may vary depending on the intelligence of the compiler. If the compiler has internal constraints that result in well-optimized implementations of the dataflow graph, the programmer may choose to provide few constraints. Thus, the capabilities of the compiler can affect the number of external constraints the programmer decides to use. As newer more intelligent versions of the compiler become available, the programmer may provide fewer constraints.

The types of constraints can vary. Moreover, the number of constraints that a programmer provides may be correlated to how much the programmer understands the underlying hardware in the SoC. If the programmer knows little about the hardware of the SoC, the constraints may dictate an overall performance of the dataflow graph (e.g., a desired performance of the dataflow graph such as cycle time or latency of the graph). If the programmer understands some basics hardware constructs in the SoC (e.g., DPEs, PL, types of communication links, and the like), the programmer may also provide constraints for these specific graph objects. Thus, some constraints can be hardware agnostic (such as performance constraints which affect the graph as a whole) while other constraints are hardware aware and affect particular graph objects (or groups of graph objects) in the dataflow graph.

As an example of a hardware aware constraint, the programmer may stipulate where in the DPE array a particular kernel should be located (e.g., a kernel location constraint). Or the programmer can stipulate a location relationship between two kernels (e.g., the two kernels should be hosted on the same core or hosted on neighboring cores). In another example, a constraint can stipulate where a particular buffer for a communication link (or a port for a kernel) should be placed in the DPE array. The location requirement of the buffer could be absolute address or a memory bank, or a relative location with respect to another buffer or kernel or the stack associated with the processor where the kernel executes. Another type of constraint can indicate whether a particular buffer should be disposed in a memory module that neighbors a core hosting a particular kernel. Another type of constraint could apply to the dataflow graph as a whole. Using these types of constraints, the programmer can control how the compiler places the graph objects (e.g., kernels, ports, communication links, etc.) in the SoC.

The programmer can also provide performance constraints which can be hardware agnostic. For example, the programmer may want the latency of the graph to be less than a certain number of processing cycles. The compiler can test its implementation of the graph to determine whether it satisfies the performance constraint, and if not, reconfigure the graph until the constraint is satisfied. For example, the compiler may split two kernels into two different cores if they were previously co-located on the same core, or move a buffer to a shared memory module so the kernel can access the data directly without having to use the interconnect in the DPE array.

In another embodiment, the constraint may define a utilization of a core/port/FIFO/memory module or a preferred FIFO depth. The compiler can test its implementation of the graph to determine whether it satisfies the performance constraint, and if not, reconfigure the graph. Because with performance constraints the compiler often tests the graph to determine whether the constraint is satisfied, these constraints can also be referred to as derived constraints.

At block <NUM> the compiler identifies a graph object corresponding to the constraint using a unique name in the constraint. In this example, each of the graph objects can be assigned a unique name - e.g., each kernel, communication link, port, etc. When formatting the constraints, the programmer can use the unique names to inform the compiler to which graph object the constraint applies.

In one embodiment, the programmer can provide unique names to each graph object in an index. The index can then be accessible to the compiler. In another embodiment, the compiler assigns the unique names to the graph objects. For example, the compiler can form a hierarchical tree of all the graph objects in the graph and assign unique names to the objects by traversing the tree from the root to the leaves. The hierarchical tree is also accessible to the programmer so she can assign constraints to particular object using the unique names.

At block <NUM>, the compiler configures the graph object to satisfy the constraint when compiling the source code. Various examples of placing graph objects according to the constraints are illustrated in <FIG>.

<FIG> is a DPE array <NUM> with graph objects implemented using user-defined constraints, according to an example. In this example, the graph objects include kernels a-d and a buffer <NUM>. In one embodiment, the compiler places the kernel a on the core <NUM> in response to a location constraint provided by the programmer. For example, the programmer can use unique addresses <NUM> assigned to the cores <NUM> to instruct the compiler to place the kernel a on the core <NUM>. That is, the constraint may include the address <NUM> of the core <NUM> (i.e., <NUM>,<NUM>) which instructs the compiler to place the kernel a on the core <NUM>.

<FIG> also illustrates a colocation constraint <NUM> which indicates that the kernels b and d should be collocated on the same core 210E. While the programmer could format the constraint in source code to require the compiler to place both kernels b and d on the core 210E (e.g., using its address <NUM>,<NUM>), in another embodiment the constraint may not stipulate a particular core which gives the compiler freedom to identify on its own the best core <NUM> to host the kernels b and d.

<FIG> also illustrates a relative location constraint <NUM> which instructs the compiler to place the kernel c and kernel b in neighboring cores - i.e., core 210D and 210E. Again, while the programmer could format the constraint to indicate which two of the cores <NUM> in the DPE array <NUM> should host the kernels c and b, in another embodiment the compiler has the freedom to choose the cores <NUM> to use based on other metrics such as availability.

Moreover, <FIG> illustrates placing the buffer <NUM> according to a constraint provided by the programmer. In one embodiment, the programmer stipulates in a constraint that the buffer <NUM> should be placed in the memory module 230B using, for example, the address of the tile (<NUM>,<NUM>). Alternatively, the constraint may not provide an absolute location of the memory module in the array <NUM> but instead stipulate that the buffer <NUM> be disposed in a memory module <NUM> that can be directly accessed by the core corresponding to kernel d. Doing so gives the compiler the freedom to choose one of the four memory modules <NUM> surrounding the core 210E to implement the buffer using a metric such as availability. In another embodiment, a plurality of buffers may be mapped to the same memory group by a constraint (e.g., stack/reserved memory of set of kernels is mapped to same memory group).

<FIG> only illustrates a few location constraints that can be used to place graph objects in the DPE array <NUM>. As mentioned above, the programmer can provide other external constraints (or the compiler can identify other derived constraints) not illustrated in <FIG> that can be used to customize the graph according to a programmers' preferences. Further constraint types can include routing resources a path should take to transport data from one point to another point, whether a data path should be circuit switched or packet switched, and how much delay should be inserted on the data path. Some constraints may aid the compiler to make better decisions when generating the compiled code. Other constraints can improve performance of the SoC such as buffer-to-buffer placement constraints to avoid memory conflicts.

Returning to the method <NUM>, at block <NUM> the compiler implements the dataflow graph in the heterogeneous processing system of the SoC according to the constraint. As mentioned above, the compiler can generate a bitstream and binary code which configures the heterogeneous processing system in the SoC to execute the dataflow graph.

In one embodiment, the dataflow graph can extend across multiple SoCs (e.g., multiple FPGAs). In that case, the graph source code may include a first constraint used to configure a first graph object in a heterogeneous processing system of a first SoC and a second constraint used to configure the a second graph object in a heterogeneous processing system of a second SoC.

<FIG> is an inheritable abstract interface <NUM>, according to an example. The abstract interface 1505defines an interface for, in this example, a filter chain <NUM> that includes ports <NUM>. The interface <NUM> may be defined by a software class that can be implemented by the programmer in different ways. For example, the filter chain <NUM> inherits the abstract interface <NUM> and includes kernels a and b. The filter chain <NUM>, in contrast, also inherits the abstract interface <NUM> but includes kernels a, b, and c. For example, the filter chain <NUM> may require more granular processing than the filter chain <NUM>. Because the abstract interface <NUM> can be defined using an object orientated programming language, the interface <NUM> can be inherited and used for different implementations.

<FIG> is a dataflow graph <NUM> with multiple sub-graphs <NUM>, according to an example. <FIG> differs from <FIG> in that the source code for the dataflow graph <NUM> includes two instances of the sub-graph - i.e., sub-graph 1505A and 1505B. That is, the sub-graph <NUM> can be defined once and the multiple instantiations of that sub-graph <NUM> can be inserted into the graph <NUM>. For example, the receiver chain defined by the graph <NUM> may use two of the filters defined by the sub-graphs <NUM> because it corresponds to a two channel system rather than the one channel system of <FIG>. In this manner, a sub-graph <NUM> can be separately defined from the graph source code (e.g., in its own file) and then instantiated any number of times.

In <FIG>, the kernel b is modified to include a first port 1510B to transmit data windows to the sub-graph 1505A and a second port 1510A to transmit data windows to the sub-graph 1505B. This can be defined by the programmer in the source code.

<FIG> is a constrained dataflow graph <NUM>, according to an example. <FIG> includes the graph <NUM> illustrated in <FIG> which include multiple instantiations of the sub-graphs <NUM>. However, the graph <NUM> is contained within the constrained dataflow graph <NUM>. In one embodiment, the constrained graph <NUM> is a wrapper graph that adds constraints to logic design. That is, by encapsulating the graph <NUM> in the constrained graph <NUM> (which is accessible using the ports <NUM>), the programmer can add overall constraints to the execution of the graph <NUM>. Moreover, the compiler automatically propagates the constraints from constrained graph <NUM> that can transform the graph <NUM> into a different implementation which can then be instantiated into another dataflow graph.

<FIG> is a constraint processing flow <NUM> for merging constraints from multiple sources, according to an example. The flow <NUM> includes graph source code <NUM> which includes constraints <NUM> which can include any of the constraint types discussed above. Moreover, the flow <NUM> includes constraints from other sources <NUM> which again can include any of the constraint types discussed above. These latter constraints can be defined in a javascript object notation (JSON) file format, a TCL file format, or by using a graphical user interface (GUI). As such, the constraints from the other source <NUM> are not embedded within the source code <NUM> but are separate files.

During constraint processing <NUM>, the compiler merges the constraints <NUM> in the source code <NUM> with the constraints from the other sources <NUM>. In one embodiment, the constraints (regardless where they are defined) have a format so they can be merged with the internal data structure of the compiler. In one embodiment, a programmer can specify the constraints for each sub-graph separately and the compiler can handle reading and merging these constraints with the parent graph program defined by the source code <NUM>.

Constraint clients <NUM> such as a partitioner, mapper, and router receive the merged constraints and ensure the solution <NUM> satisfies the constraints. That is, the constraint clients <NUM> ensure that the implementation of the dataflow graph in the SoC satisfies the constraints <NUM> embedded in the source code <NUM> as well as the constraints from the other sources <NUM>.

<FIG> is a block diagram of a computing system <NUM> for implementing a dataflow graph on the SoC, according to an example. The computing system <NUM> includes many of the same components discussed above in <FIG> which are not discussed in detail here. However, <FIG> differs from <FIG> in that the computing system <NUM> includes control APIs <NUM> which may or may not be in the computing system illustrated in <FIG>. As shown, the control APIs <NUM> are disposed in the control source code <NUM>.

In general, the programmer can use the control APIs <NUM> to change parameters that control the execution of the dataflow graph <NUM> on the SoC <NUM>. That is, embodiments herein use the APIs <NUM> and corresponding methods to control, interact, and at least partially reconfigure a user application (e.g., the dataflow graph <NUM>) executing on the heterogeneous processing system of the SoC <NUM> through a local control program compiled from the control source code <NUM>, or by executing the control source code on the PS itself). Using the control APIs <NUM>, users can manipulate such remotely executing graphs directly as local objects and perform control operations on them, (e.g., for loading and initializing the graphs; dynamically adjusting parameters for adaptive control; monitoring application parameters, system states and events; scheduling operations to read and write data across the distributed memory boundary of the platform; controlling the execution life-cycle of a subsystem; and partially reconfiguring the computing resources for a new subsystem).

For example, the kernels or other graph objects in the SoC <NUM> may have parameters, such as a gain or filter coefficients that control the operation of these objects. These parameters can be dynamically controlled using the control program that executes on the host or the SoC itself. The compiler <NUM> can configure the control program to change the parameters, which means the programmer can express the APIs <NUM> at a high-level (using source code) while the compiler <NUM> handles the hardware details for adjusting the parameters such as configuring registers, identifying routes, identifying the location of the graph objects, and the like.

Advantageously, the compiler <NUM> can configure drivers <NUM>, registers, and other hardware in the SoC <NUM> so that the APIs <NUM> can perform the desired function. For example, the drivers <NUM> may be used to perform a DMA to read data in DDR memory in the SoC <NUM> into one of the DPEs <NUM> executing a kernel in the dataflow graph <NUM>. While the drivers <NUM> are illustrated as part of the PS <NUM>, in other in other embodiments, the drivers <NUM> could be implemented using controllers in the PL <NUM> or through control signals transmitted to the SoC <NUM> from a remote controller using a network.

Without the control APIs <NUM>, the programmer would have to configure the driver <NUM> directly which may require the programmer to know the location of the kernel (e.g., the host DPE) as well as the route to reach the kernel. Instead, the compiler <NUM> can configure the drivers <NUM> in response to detecting the corresponding API <NUM> in the control source code <NUM>. That is, when defining the API <NUM>, the programmer simply identifies the graph object (e.g., a particular kernel or kernel port) and the compiler <NUM> can do the rest - e.g., configure the drivers <NUM> and program registers to perform the DMA.

<FIG> and <FIG> illustrate control APIs for controlling the execution of a dataflow graph on the SoC, according to examples. <FIG> illustrates a list of control APIs <NUM> that can be used to control the operation of a dataflow graph. <FIG> includes comments next to each API <NUM> explaining its purpose. For example, the graph() API defines an empty dataflow graph class constructor. All user defined graphs are extensions of this class.

The init() API initializes a dataflow graph, the run() APIs execute the graph, the wait() APIs wait for the graph to complete the previous run or to wait for a number of cycles and the pause the graph, the resume() API resumes the graph after a pause, and the end() APIs wait for the last run to complete and then disables the DPE. Thus, using these APIs <NUM>, the programmer can control when the graph begins operating, how long it operates, and end the graph.

The update() APIs permit the programmer to update runtime parameters in the dataflow graph by specifying a graph object (e.g., by using the input_port& p pointer). Using the provided information, the compiler can configure the hardware in the SoC to perform the update using a trigger which is discussed below.

Using the read() APIs, the programmer can read runtime parameters from the executing dataflow graph. This is especially useful for controlling graph execution based on dynamic data-dependent decisions.

<FIG> illustrates other control APIs <NUM> that may be part of the programming model. <FIG> includes a global memory input/output (GMIO) class with special APIs for moving data between the DPE array and DDR memory in the SoC. For example, the init() API initializes a GMIO object by providing a set of memory addresses that exist in the DDR memory. The gm2me_nb() APIs can use the DMA registers in the shim to transfer data from the global memory to the DPE array. In one embodiment, the compiler configures the registers in the shim to perform the APIs <NUM> within the GMIO class. Further, these APIs <NUM> are non-blocking commands which means the PS (which may host the control program) can perform other functions concurrently with the GMIO reads and writes. In one embodiment, the GMIO APIs permit the SoC to use the same set of DDR memory to transfer data into the DPE array and read data out from the array. That is, the programmer can use the GMIO APIs to read data from the DDR memory into the DPE array which then processes the data and stores the processed data in the same DDR memory.

<FIG> also includes a programmable logic input/output (PLIO) class with an API for moving data between the PL and the DPE array. The PLIO API is more straightforward than the GMIO APIs since it may only be used for simulation environments where data is transferred between the DPE array and input/output files.

<FIG> also has an event class with APIs for monitoring performance or executing an event trace for a particular graph object (e.g., a particular kernel of GMIO port). The event APIs permit the programmer to track specific hardware events, count occurrences of hardware events and measure aggregate performance metrics. In one example, the programmer can measure latency of a graph by the tracking the input and output of the dataflow graph. For example, in response to the APIs, the compiler can establish a performance counter that counts the number of processing cycles between when the first data is inputted into the dataflow graph and when the first data is outputted by the dataflow graph. In another example, the programmer can measure the throughput of a graph executing within the DPE. The compiler can establish performance counters to count the number of cycles and the number of data items produced during some number of iterations of graph execution.

<FIG> illustrates logically dividing a DPE array <NUM> into different regions, according to an example. In this embodiment, a TopRegion <NUM> includes the entire DPE array <NUM> and its DPEs <NUM>. The RCregion <NUM> includes a subset of the columns in the DPE array <NUM>. The regions 2115A and 2115B define sub-regions within the RCregion <NUM>. In this manner, the DPE array <NUM> can be divided into a hierarchy of regions. In this example, the RCregion <NUM> is a subregion of the TopRegion <NUM> while the regions 2115A and 2115B are sub-regions contained within the RCregion <NUM>.

Using the APIs and constraints discussed above, the programmer can assign different dataflow graphs to different regions in the array <NUM>. For example, a plurality of dataflow graphs may process digital data obtained from a radio transceiver which can, depending on the time of day, receive data using different numbers of antennas. To disable or enable dataflow graphs corresponding to the antennas, the programmer can use the placement constraints to place each dataflow graph in a separate RCregion <NUM> so that the process control corresponding to a particular antenna can be selectively enabled and disabled. Thus, placing different dataflow graphs in different regions gives the programmer control so that one dataflow graph can be enabled or disabled without affecting the dataflow graphs operating in different regions. In one embodiment, the programmer provides a plurality of logically independent container graphs derived from the class RCGraph and assigns a plurality of dataflow graphs to them. The compiler then determines the specific hardware regions for each container graph so each dataflow graph can be controlled independently.

In another embodiment, the programmer can use the control APIs discussed above to establish a plurality of alternative graphs within a single container graph. Alternative graphs are dataflow graphs that share the same logical container graph, and thus, share the same hardware region. If the number of alternative graphs for a container graphs is greater than one, this means different dataflow graphs share the same hardware region but execute at different times. In one embodiment, the container graph and the assignment of the alternative dataflow graphs to a particular region is defined in a package binary that is provided to the SoC by the compiler.

<FIG> illustrates dynamically changing the execution of the dataflow graph, according to an example. That is, <FIG> illustrates using one or more control APIs to dynamically reconfigure a dataflow graph <NUM> (e.g., change a run-time parameter) to alter how the graph <NUM> processes data. This reconfiguration can occur without changing the underlying hardware. That is, after the SoC is initialized, the dataflow graph <NUM> can switch between different states on the fly without requiring the hardware to be reconfigured.

The dataflow graph <NUM> illustrates a processing scheme that includes a dedicated LTE20 channel <NUM>, a dedicated LTE10 channel <NUM> and a reconfigurable channel <NUM> which can be selectively changed between a LTE20 and a LTE10 channel using run-time parameters <NUM>. For example, to configure the channel <NUM> as a LTE20 channel, the parameter <NUM> controls a mux <NUM> such that it outputs the data received from a half-band filter. The control APIs can alter the parameters <NUM> such that the mux <NUM> ignores the data outputted by the half-band filter and the delay alignment block so that the channel <NUM> processes data similar as the LTE10 channel <NUM>.

In one embodiment, a plurality of reconfigurable alternatives within the dataflow graph <NUM> can be assigned to the same region in the SoC. This is illustrated in <FIG>. For example, the graph <NUM> may be assigned to the TopRegion <NUM> in the SoC. Alternatively, the different channels in the graph <NUM> may be assigned to different regions. In this example, rather than having a reconfigurable channel <NUM> that includes the mux <NUM>, the graph <NUM> is built with two alternatives for a reconfigurable container RCRegion <NUM>. One alternative AltO <NUM> is the LTE20 channel and the other alternative Alt1 <NUM> carries two LTE10 channels along with a mixer <NUM>. The fixed LTE20 channel in AltO <NUM> can be assigned to its own region in the SoC, separate from the region or regions to which the two LTE10 channels are assigned, or it could be made part of the TopRegion <NUM>. Thus, when the RCRegion <NUM> should function as an LTE20 channel, the control APIs can reconfigure the region to load the graph AltO <NUM> (without affecting the dedicated LTE20 channel disposed in other regions). However, when the RCRegion <NUM> should function as two LTE10 channels, the control APIs can reconfigure the region to load the alternative graph Alt1 <NUM>. While doing so avoids the circuitry illustrated in <FIG> used to dynamically reconfigure the channel <NUM> such as the mux <NUM> and reuses the same DPE resources for the two alternatives (which can reduce the amount of space the graph <NUM> uses in the SoC), it typically takes more time to reconfigure the hardware in the region between the LTE20 and LTE10 embodiments than to control the parameters <NUM> for the muxes <NUM>.

<FIG> illustrate triggered and asynchronous parameters, according to examples. For example, unlike windows and streams which correspond to streaming data, parameters can be used to control the execution of the dataflow graph using non-streaming data. In one embodiment, the programmer uses a synchronization trigger at the start of a kernel execution to change the parameters in the dataflow graph. In another embodiment, the change in the parameter can take place asynchronously with the execution of a kernel. In one embodiment, the control program (whether executing on the PS or the host) initiates the triggered or asynchronous change in parameters. In another embodiment, the programmable logic initiates the triggered or asynchronous change in parameters. Examples of parameters that can be altered using triggers include parameters in a function or method call or changing the size of the windows.

<FIG> illustrates triggered parameters where a kernel waits on a new parameter every time the corresponding function is invoked. As a result, the kernel does not execute until the control program <NUM> provides the triggered parameter. For example, the control program <NUM> generates a write transaction 2315A to the ping buffer of a parameter which is received by the DPE executing a kernel <NUM>. In response, the kernel <NUM> processes data during an execution block 2325A. Concurrently, the control program <NUM> is free to perform other activities during the time block <NUM>. That is, the control program <NUM> can transmit the triggered parameter value to the ping buffer (which is non-blocking) and then can perform other tasks during time block <NUM>.

Notably, when the kernel <NUM> finishes the execution block 2325A, it does not begin to immediately process more data even if that data is available at its inputs. Instead, the kernel <NUM> waits until receiving the second write transaction 2315B at the pong buffer which includes the triggered parameters (which can have the same values as in the write transaction 2315A or different values) to perform the execution block 2325B. Once finished with execution block 2325B, the kernel <NUM> again waits until receiving the triggered parameters in the write transaction 2315C to begin the execution block 2325C. In this manner, triggered parameters permit the control program <NUM> to transmit updated parameters to the kernel <NUM> before each execution block.

<FIG> illustrates asynchronous parameters where the kernel <NUM> executes using the previously received parameters. As shown, the control program <NUM> transmits the write transaction 2315D to the ping buffer which includes updated parameters for the kernel <NUM> to use when processing data during execution block 2325D. Concurrently, the control program <NUM> can perform other activities during the time block <NUM> like in <FIG>. However, unlike in <FIG>, once the execution block 2325D is complete, the kernel <NUM> can immediately begin processing data during execution block 2325E and 2325F. Because the kernel <NUM> has not received new parameters from the control program <NUM>, the kernel <NUM> processes input data during the execution blocks 2325E and 2325F using the same parameters during execution block 2325D.

During execution block 2325E, the control program <NUM> transmits a new write transaction 2315E to the pong buffer which includes updated parameters for the kernel <NUM>. The updated parameter value is available for use by the kernel <NUM> after the completion of the write transaction 2315E. Thus, when the kernel <NUM> begins execution block <NUM>, the kernel <NUM> uses the updated parameters (which may be different from the values of the parameters used during blocks 2325D-F). In this manner, the kernel <NUM> can continuously execute using the same parameters until the control program <NUM> transmits updated parameters to the kernel <NUM>.

In one embodiment, when the kernel is invoked, the compiler creates locking criteria which ensures all the data is available before the kernel starts processing the received data and all the data is ready to be transmitted before outputting a data window. For asynchronous communication, however, the graphs does not have to make either of those checks but the user can create an API that defines the criteria used when acquiring an input window to read, or outputting a window to write. Put differently, the criteria provided by the user defines the point when the kernel synchronizes. In <FIG> for example, the connection from kernel e to kernel b is asynchronous. So kernel e can prepare the window and then kernel b determines (using the criteria provided by the user in the API) whether it should skip over the first few frames before synchronizing with the kernel b. That is, it is up to kernel b and e respectively to determine when it will receive or output the window using the criteria provided by the user.

In the preceding, reference is made to embodiments presented in this disclosure. However, the scope of the present disclosure is not limited to specific described embodiments. Instead, any combination of the described features and elements, whether related to different embodiments or not, is contemplated to implement and practice contemplated embodiments. Furthermore, although embodiments disclosed herein may achieve advantages over other possible solutions or over the prior art, whether or not a particular advantage is achieved by a given embodiment is not limiting of the scope of the present disclosure. Thus, the preceding aspects, features, embodiments and advantages are merely illustrative and are not considered elements or limitations of the appended claims except where explicitly recited in a claim(s).

Accordingly, aspects may take the form of an entirely hardware embodiment, an entirely software embodiment (including firmware, resident software, micro-code, etc.) or an embodiment combining software and hardware aspects that may all generally be referred to herein as a "circuit," "module" or "system. " Furthermore, aspects may take the form of a computer program product embodied in one or more computer readable medium(s) having computer readable program code embodied thereon.

In the context of this document, a computer readable storage medium is any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus or device.

Aspects of the present disclosure are described below with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems) and computer program products according to embodiments presented in this disclosure.

The flowchart and block diagrams in the Figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various examples of the present invention.

Claim 1:
A computer-implemented method (<NUM>) of implementing a dataflow graph (<NUM>, <NUM>) on a heterogeneous processing system (<NUM>) formed by a System on Chip, SoC, comprising programmable logic (<NUM>) and software-configurable hardened logic (<NUM>, <NUM>), the method comprising:
using (<NUM>), by a user, a heterogeneous programming environment, to generate object oriented source code that defines the dataflow graph (<NUM>, <NUM>);
receiving (<NUM>), by a compiler (<NUM>), graph source code (<NUM>) generated in the heterogeneous programming environment, the graph source code (<NUM>) defining a plurality of kernels (<NUM>) and a plurality of communication links (<NUM>), wherein each of the plurality of communication links (<NUM>) couple a respective pair of the plurality of kernels (<NUM>) to form the dataflow graph (<NUM>, <NUM>); and
compiling (<NUM>), by the compiler (<NUM>), the graph source code (<NUM>) to implement the dataflow graph (<NUM>, <NUM>) in the heterogeneous processing system (<NUM>), and transmitting (<NUM>), to the SoC, a configuration based on compiling the graph source code for configuring the software-configurable hardened logic (<NUM>, <NUM>) and the programmable logic of the heterogeneous processing system (<NUM>) to execute the dataflow graph (<NUM>, <NUM>), wherein compiling (<NUM>) the graph source code (<NUM>) comprises:
assigning (<NUM>) the plurality of kernels (<NUM>) to different processing elements of the heterogeneous processing system (<NUM>) based on the definition of the plurality of kernels (<NUM>) in the graph source code (<NUM>),
assigning (<NUM>) a communication type to the plurality of communication links (<NUM>) defined in the graph source code, and
selecting (<NUM>) synchronization techniques for transferring data between the plurality of kernels using the plurality of communication links (<NUM>),
wherein assigning (<NUM>) the plurality of kernels (<NUM>) to the heterogeneous processing system (<NUM>) comprises identifying that a first kernel and a second kernel are communicatively coupled by a first one of the plurality of communication links (<NUM>) as defined by the graph source code (<NUM>);
assigning the first kernel to a first data processing engine, DPE, (<NUM>) in the heterogeneous processing system (<NUM>); and
assigning the second kernel to a second DPE (<NUM>) in the heterogeneous processing system (<NUM>) that directly neighbors the first DPE (<NUM>).