Implied fence on stream open

Techniques related to executing a plurality of instructions by a processor comprising receiving a first instruction configured to cause the processor to output a first data value to a first address in a first data cache, outputting, by the processor, the first data value to a second address in a second data cache, receiving a second instruction configured to cause a streaming engine associated with the processor to prefetch data from the first data cache, determining that the first data value has not been outputted from the second data cache to the first data cache, stalling execution of the second instruction, receiving an indication, from the second data cache, that the first data value has been output from the second data cache to the first data cache, and resuming execution of the second instruction based on the received indication.

BACKGROUND

Modern digital signal processors (DSP) faces multiple challenges. Workloads continue to increase, requiring increasing bandwidth. Systems on a chip (SOC) continue to grow in size and complexity. Memory system latency severely impacts certain classes of algorithms. As transistors get smaller, memories and registers become less reliable. As software stacks get larger, the number of potential interactions and errors becomes larger. Even wires become an increasing challenge. Wide busses are difficult to route. Wire speeds continue to lag transistor speeds. Routing congestion is a continual challenge.

Generally, bus bandwidth is proportional to the width of the bus in bits times the bus clock rate. To increase bandwidth to the processor required a wider bus running at a faster clock rate. However, that can lead to more wires and greater latency, because faster clock rates typically require greater pipelining. More wires produce more routing issues. Thus, processor needs tend to lead either to lower clock rates, overly large chips or both.

Memory systems continue to provide scalability challenges to the central processing unit (CPU). For example, in certain CPUs, a level one data (L1D) cache line can be consumed in fewer CPU cycles than it takes to read a new line into the cache. Streaming workloads therefore pay a very large cache penalty even if all their data resides in level two (L2) RAM due to sometimes limited buffering, limits to how many cache lines may be retrieved at a time, and by roundtrip latency ranging from tens to hundreds of processor cycles for various forms of cache and memory. The in-order nature of the CPU limits the ability to hide this latency penalty. A prefetcher can help, but even with a prefetcher, streaming workloads can saturate memory systems over a period of time.

Memory system overhead limits performance, and traditional approaches to solving memory system issues do not necessarily scale well. However, applications continue to demand increasing performance. Future application workloads will not only continue to place more tasks on DSPs, but also have those tasks communicate directly with tasks running under a traditional virtual memory operating system such as running on a traditional general-purpose processor. Larger systems might even include virtualization so that multiple virtual machines need to interact with the DSPs.

SUMMARY

This disclosure relates generally to the field of DSPs. More particularly, but not by way of limitation, aspects of the present disclosure relate to an implied fence on stream open, including a method for executing a plurality of instructions by a processor. This method includes receiving a first instruction configured to cause the processor to output a first data value to a first address in a first data cache. The method also includes outputting, by the processor, the first data value to a second address in a second data cache. The method further includes receiving a second instruction configured to cause a streaming engine associated with the processor to prefetch data from the first data cache. The method also includes determining that the first data value has not been outputted from the second data cache to the first data cache. The method further includes stalling execution of the second instruction. The method also includes stalling execution of the second instruction. The method further includes stalling execution of the second instruction.

Another aspect of the present disclosure relates to a processor comprising a streaming engine capable of autonomously accessing a first data cache and an instruction execution pipeline controller. The instruction execution pipeline controller includes circuitry configured to output the first data value to a second address in a second data cache. The instruction execution pipeline controller also includes circuitry configured to receive a second instruction configured to cause the streaming engine to prefetch data from the first data cache. The instruction execution pipeline controller also include circuitry configured to determine that the first data value has not been outputted from the second data cache to the first data cache. The instruction execution pipeline controller also include circuitry configured to stall execution of the second instruction. The instruction execution pipeline controller also include circuitry configured to receive an indication, from the second data cache, that the first data value has been output from the second data cache to the first data cache. The instruction execution pipeline controller also include circuitry configured to resume execution of the second instruction based on the received indication.

Another aspect of the present disclosure relates to a processing system comprising a memory space and a processor. The processor includes a streaming engine capable of autonomously accessing a first data cache of the memory space and an instruction execution pipeline controller. The instruction execution pipeline controller includes circuitry configured to receive a first instruction configured to cause the processor to output a first data value to a first address in the first data cache, the first instruction associated with a first color value. The circuitry is further configured to output the first data value to a second address in a second data cache of the memory space. The circuitry is further configured to receive a second instruction configured to cause the streaming engine to prefetch data from the first data cache, the second instruction associated with the first color value, The circuitry is further configured to determine that the first data value has not been outputted from the second data cache to the first data cache. The circuitry is further configured to stall execution of the second instruction. The circuitry is further configured to receive an indication, from the second data cache, that the first data value has been output from the second data cache to the first data cache. The circuitry is further configured to resume execution of the second instruction based on the received indication. The circuitry is further configured to receive a third instruction configured to cause the processor to output a third data value to a third address in the first data cache, the third instruction associated with the first color value. The circuitry is further configured to receive a fourth instruction associated with a second color value different from the first color value, the fourth instruction configured to cause the streaming engine to prefetch data from the third address. The circuitry is further configured to execute the fourth instruction without stalling execution of the fourth instruction

DETAILED DESCRIPTION

Digital signal processors (DSP) are optimized for processing streams of data that may be derived from various input signals, such as sensor data, a video stream, a voice channel, radar signals, biomedical signals, etc. Memory bandwidth and scheduling are concerns for digital signal processors operating on real-time data. An example DSP processing core described in this disclosure includes a streaming engine to improve processing efficiency and data scheduling.

Digital signal processors (DSP) are optimized for processing streams of data that may be derived from various input signals, such as sensor data, a video stream, a voice channel, radar signals, biomedical signals, etc. Memory bandwidth and scheduling are concerns for digital signal processors operating on real-time data. The example DSP processing core described hereinbelow includes a streaming engine to improve processing efficiency and data scheduling.

One or more DSP processing cores may be combined with various peripheral circuits, blocks of memory, etc. on a single integrated circuit (IC) die to form a system on chip (SoC). See, for example, “66AK2Hx Multicore Keystone™ DSP+ARM® System-on-Chip,” 2013, which is incorporated by reference herein.

In at least one example DSP core of this disclosure, an autonomous streaming engine is coupled to the DSP. In this example, the streaming engine can manage two data streams simultaneously; in another example the streaming engine may be capable of managing only a single stream, while in other examples the streaming engine may be capable of handling more than two streams. In each case, for each stream the streaming engine includes an address generation stage, a data formatting stage, and some storage for formatted data waiting for consumption by the processor. In the examples described hereinbelow, addresses are derived from algorithms that can involve multi-dimensional loops, each dimension maintaining an iteration count. In one example, the streaming engine supports six levels of nested iteration. In other examples, more or fewer levels of iteration may be supported.

In one or more examples of this disclosure, a processing core within a given family may have a different number of instruction pipeline stages, depending on a particular technology and cost/performance tradeoffs. The example described here is representative and includes multiple pipelines stages.

In the drawings, like elements are denoted by like reference numerals for consistency.

An example DSP core is described in detail hereinbelow with reference toFIGS. 1-4. A streaming engine capable of managing two data streams using six-dimensional nested loops is described in detail hereinbelow with reference toFIGS. 5-10.

FIG. 1illustrates an example processor100that includes dual scalar/vector data paths115,116. Processor100includes separate level one instruction cache (L1I)121and level one data cache (L1D)123. Processor100includes a L2 combined instruction/data cache (L2)130that holds both instructions and data.FIG. 1illustrates connection between L1I cache and L2 combined instruction/data cache130by way of 512-bit bus142.FIG. 1illustrates connection between L1D cache123and L2 combined instruction/data cache130by way of 512-bit bus145. In this example of processor100, L2 combined instruction/data cache130stores instructions to back up L1I cache121and data to back up L1D cache123. In this example L2 combined instruction/data cache130is further connected to higher level cache and/or main memory using known or later developed memory system techniques but not illustrated inFIG. 1. The size of L1I cache121, L1D cache123, and L2 cache130may be implemented in different sizes in various examples; in this example, L1I cache121and L1D cache123are each 32 kilobytes, and L2 cache130is 1024 kilobytes. In this example central processing unit core110, L1I cache121, L1D cache123and L2 combined instruction/data cache130are formed on a single integrated circuit. In some examples, the integrated circuit described above includes other circuits.

Central processing unit core110fetches instructions from L1I cache121as controlled by instruction fetch unit111. Instruction fetch unit111determines the next instructions to be executed and recalls a fetch packet sized set of such instructions. The nature and size of fetch packets are further detailed below. Instructions are directly fetched from L1I cache121upon a cache hit (if these instructions are stored in L1I cache121). Upon a cache miss (the specified instruction fetch packet is not stored in L1I cache121), these instructions are sought in L2 combined cache130. In this example the size of a cache line in L1I cache121equals the size of a fetch packet, which is 512 bits. The memory locations requested by these instructions are either a hit in L2 combined cache130or a miss. A hit is serviced from L2 combined cache130. A miss is serviced from a higher level of cache (not illustrated) or from main memory (not illustrated). In this example, the requested instruction is simultaneously supplied to both L1I cache121and central processing unit core110to speed use.

In the example illustrated inFIG. 1, central processing unit core110includes plural functional units to perform instruction specified data processing tasks. Instruction dispatch unit112determines the target functional unit of each fetched instruction. In this example central processing unit110operates as a very long instruction word (VLIW) processor capable of operating on plural instructions in corresponding functional units simultaneously. In some examples, a complier organizes instructions in execute packets that are executed together. Instruction dispatch unit112directs each instruction to its target functional unit. The functional unit assigned to an instruction is completely specified by the instruction produced by a compiler. The hardware of central processing unit core110does not assign functional units. In this example instruction dispatch unit112may operate on several instructions in parallel. The number of such parallel instructions is set by the size of the execute packet. This will be further detailed hereinbelow.

Instruction decode unit113decodes each instruction in a current execute packet. Decoding includes identification of the functional unit performing the instruction, identification of registers used to supply data for the corresponding data processing operation from among possible register files and identification of the register destination of the results of the corresponding data processing operation. As further explained below, instructions may include a constant field in place of one register number operand field. The result of this decoding are control signals which cause the target functional unit to perform the data processing operation specified by the corresponding instruction on the specified data.

Central processing unit core110includes control registers114. Control registers114store information for control of the functional units in scalar data path side A115and vector data path side B116. This information may include mode information or the like.

The decoded instructions from instruction decode unit113, and information stored in control registers114, are supplied to scalar data path side A115and vector data path side B116. As a result, functional units within scalar data path side A115and vector data path side B116perform instruction specified data processing operations upon instruction specified data and store the results in an instruction specified data register or registers. Each of scalar data path side A115and vector data path side B116include plural functional units that operate in parallel. Scalar data path side A115, vector data path side B116, and their plural functional units will be further detailed below in conjunction withFIG. 2. There is a data path117between scalar data path side A115and vector data path side B116permitting data exchange.

Central processing unit core110includes further non-instruction-based modules. Emulation unit118permits determination of the machine state of central processing unit core110in response to instructions. This capability will typically be employed for algorithmic development. Interrupts/exceptions unit119enables central processing unit core110to be responsive to external, asynchronous events (interrupts) and to respond to attempts to perform improper operations (exceptions).

Processor100includes streaming engine125. Streaming engine125supplies two data streams from predetermined addresses typically cached in L2 combined cache130to register files of vector data path side B of central processing unit core110. This provides controlled data movement from memory (as cached in L2 combined cache130) directly to functional unit operand inputs.

L1D cache123exchanges data with register files in scalar data path side A115via bus143. Bus143is a 64-bit bus in this example. L1D cache123exchanges data with register files in vector data path side B116via bus144. Bus144is a 512-bit bus in this example. Busses143and144are illustrated as bidirectional supporting both central processing unit110data reads and data writes. L1D cache123exchanges data with L2 combined cache130via bus145. Bus145is a 512-bit bus in this example. Bus145is illustrated as bidirectional supporting cache service for both central processing unit110data reads and data writes.

Processor data requests are fetched directly from L1D cache123upon a cache hit (if the requested data is stored in L1D cache123). Upon a cache miss (the specified data is not stored in L1D cache123), this data is sought in L2 combined cache130. As noted previously, the memory locations of this requested data are either a hit in L2 combined cache130or a miss. A hit is serviced from L2 combined cache130. A miss is serviced from another level of cache (not illustrated) or from main memory (not illustrated). The requested data may be simultaneously supplied to both L1D cache123and central processing unit core110to speed the use of the requested data.

L2 combined cache130supplies data of a first data stream to streaming engine125via bus146. Bus146is a 512-bit bus in this example. Streaming engine125supplies data of this first data stream to functional units of vector data path side B116via bus147. Bus147is a 512-bit bus in this example. L2 combined cache130supplies data of a second data stream to streaming engine125via bus148. Bus148is a 512-bit bus in this example. Streaming engine125supplies data of this second data stream to functional units of vector data path side B116via bus149, which is a 512-bit bus in this example. Busses146,147,148and149are illustrated as unidirectional from L2 combined cache130to streaming engine125and to vector data path side B116in accordance with this example.

Streaming engine data requests are directly fetched from L2 combined cache130upon a cache hit (if the requested data is stored in L2 combined cache130). Upon a cache miss (the specified data is not stored in L2 combined cache130), this data is sought from another level of cache (not illustrated) or from main memory (not illustrated). It is technically feasible in some examples for L1D cache123to cache data not stored in L2 combined cache130. If such operation is supported, then upon a streaming engine data request that is a miss in L2 combined cache130, L2 combined cache130may snoop L1D cache123for the streaming engine requested data. If L1D cache123stores this data its snoop response would include the data, which is then supplied to service the streaming engine request. If L1D cache123does not store this data, its snoop response would indicate this and L2 combined cache130would then service this streaming engine request from another level of cache (not illustrated) or from main memory (not illustrated).

FIG. 2illustrates further details of functional units and register files within scalar data path side A115and vector data path side B116. Scalar data path side A115includes L1 unit221, S1 unit222, M1 unit223, N1 unit224, D1 unit225and D2 unit226. Scalar data path side A115includes global scalar register file211, L1/S1 local register file212, M1/N1 local register file213and D1/D2 local register file214. Vector data path side B116includes L2 unit241, S2 unit242, M2 unit243, N2 unit244, C unit245and P unit246. Vector data path side B116includes global vector register file231, L2/S2 local register file232, M2/N2/C local register file233and predicate register file234. There are limitations upon which functional units may read from or write to which register files. These will be described in more detail hereinbelow.

Scalar data path side A115includes L1 unit221. L1 unit221generally accepts two 64-bit operands and produces one 64-bit result. The two operands are each recalled from an instruction specified register in either global scalar register file211or L1/S1 local register file212. L1 unit221may perform the following instruction selected operations: 64-bit add/subtract operations; 32-bit min/max operations; 8-bit Single Instruction Multiple Data (SIMD) instructions such as sum of absolute value, minimum and maximum determinations, circular min/max operations, and various move operations between register files. The result may be written into an instruction specified register of global scalar register file211, L1/S1 local register file212, M1/N1 local register file213or D1/D2 local register file214.

Scalar data path side A115includes S1 unit222. S1 unit222generally accepts two 64-bit operands and produces one 64-bit result. The two operands are each recalled from an instruction specified register in either global scalar register file211or L1/S1 local register file212. In this example, S1 unit222performs the same type of operations as L1 unit221. In another example, there might be slight variations between the data processing operations supported by L1 unit221and S1 unit222. The result may be written into an instruction specified register of global scalar register file211, L1/S1 local register file212, M1/N1 local register file213or D1/D2 local register file214.

Scalar data path side A115includes M1 unit223. M1 unit223generally accepts two 64-bit operands and produces one 64-bit result. The two operands are each recalled from an instruction specified register in either global scalar register file211or M1/N1 local register file213. In this example, M1 unit223performs the following instruction selected operations: 8-bit multiply operations; complex dot product operations; 32-bit bit count operations; complex conjugate multiply operations; and bitwise logical operations, moves, adds and subtracts. The result may be written into an instruction specified register of global scalar register file211, L1/S1 local register file212, M1/N1 local register file213or D1/D2 local register file214.

Scalar data path side A115includes N1 unit224. N1 unit224generally accepts two 64-bit operands and produces one 64-bit result. The two operands are each recalled from an instruction specified register in either global scalar register file211or M1/N1 local register file213. In this example, N1 unit224performs the same type operations as M1 unit223. There may be certain double operations (called dual issued instructions) that employ both the M1 unit223and the N1 unit224together. The result may be written into an instruction specified register of global scalar register file211, L1/S1 local register file212, M1/N1 local register file213or D1/D2 local register file214.

Scalar data path side A115includes D1 unit225and D2 unit226. D1 unit225and D2 unit226generally each accept two 64-bit operands and each produce one 64-bit result. D1 unit225and D2 unit226generally perform address calculations and corresponding load and store operations. D1 unit225is used for scalar loads and stores of 64-bits. D2 unit226is used for vector loads and stores of 512-bits. In this example, D1 unit225and D2 unit226also perform: swapping, pack and unpack on the load and store data; 64-bit single instruction, multiple data arithmetic operations; and 64-bit bitwise logical operations. D1/D2 local register file214will generally store base and offset addresses used in address calculations for the corresponding loads and stores. The two operands are each recalled from an instruction specified register in either global scalar register file211or D1/D2 local register file214. The calculated result may be written into an instruction specified register of global scalar register file211, L1/S1 local register file212, M1/N1 local register file213or D1/D2 local register file214.

Vector data path side B116includes L2 unit241. L2 unit241generally accepts two 512-bit operands and produces one 512-bit result. The two operands are each recalled from an instruction specified register in either global vector register file231, L2/S2 local register file232or predicate register file234. In this example, L2 unit241performs instruction similar to L1 unit221except on wider 512-bit data. The result may be written into an instruction specified register of global vector register file231, L2/S2 local register file232, M2/N2/C local register file233or predicate register file234.

Vector data path side B116includes S2 unit242. S2 unit242generally accepts two 512-bit operands and produces one 512-bit result. The two operands are each recalled from an instruction specified register in either global vector register file231, L2/S2 local register file232or predicate register file234. In this example, S2 unit242performs instructions similar to S1 unit222. The result may be written into an instruction specified register of global vector register file231, L2/S2 local register file232, M2/N2/C local register file233or predicate register file234.

Vector data path side B116includes M2 unit243. M2 unit243generally accepts two 512-bit operands and produces one 512-bit result. The two operands are each recalled from an instruction specified register in either global vector register file231or M2/N2/C local register file233. In this example, M2 unit243performs instructions similarly to M1 unit223except on wider 512-bit data. The result may be written into an instruction specified register of global vector register file231, L2/S2 local register file232or M2/N2/C local register file233.

Vector data path side B116includes N2 unit244. N2 unit244generally accepts two 512-bit operands and produces one 512-bit result. The two operands are each recalled from an instruction specified register in either global vector register file231or M2/N2/C local register file233. In this example, N2 unit244performs the same type operations as M2 unit243. There may be certain double operations (called dual issued instructions) that employ both M2 unit243and the N2 unit244together. The result may be written into an instruction specified register of global vector register file231, L2/S2 local register file232or M2/N2/C local register file233.

Streaming engine125(FIG. 1) transfers data in certain restricted circumstances. Streaming engine125controls two data streams. A stream includes of a sequence of elements of a particular type. Programs that operate on streams read the data sequentially, operating on each element in turn. Every stream has the following basic properties: the stream data has a well-defined beginning and ending in time; the stream data has a fixed element size and type throughout the stream; the stream data has a fixed sequence of elements. Thus, programs cannot seek randomly within the stream. The stream data is read only while active. Programs cannot write to a stream while simultaneously reading from it. Once a stream is opened, streaming engine125performs the following operations: calculates the address; fetches the defined data type from L2 unified cache130(which may require cache service from a higher-level memory, e.g., in the event of a cache miss in L2); performs data type manipulation such as zero extension, sign extension, data element sorting/swapping such as matrix transposition; and delivers the data directly to the programmed data register file within processor core110. Streaming engine125is thus useful for real time digital filtering operations on well behaved data. Streaming engine125frees these memory fetch tasks from the corresponding processor enabling other processing functions.

Streaming engine125provides several benefits. For example, streaming engine125permits multi-dimensional memory accesses. Streaming engine125increases the available bandwidth to the functional units. Streaming engine125reduces the number of cache miss stalls since the stream buffer bypasses L1D cache123. Streaming engine125reduces the number of scalar operations required to maintain a loop. Streaming engine125manages address pointers. Streaming engine125handles address generation automatically freeing up the address generation instruction slots and D1 unit225and D2 unit226for other computations.

Processor core110(FIG. 1) operates on an instruction pipeline. Instructions are fetched in instruction packets of fixed length further described below. All instructions require the same number of pipeline phases for fetch and decode but require a varying number of execute phases.

FIG. 3illustrates the following pipeline phases: program fetch phase310, dispatch and decode phases320and execution phases330. Program fetch phase310includes three stages for all instructions. Dispatch and decode phases320include three stages for all instructions. Execution phase330includes one to four stages dependent on the instruction.

Fetch phase310includes program address generation (PG) stage311, program access (PA) stage312and program receive (PR) stage313. During program address generation stage311, the program address is generated in the processor and the read request is sent to the memory controller for the L1I cache. During the program access stage312the L1I cache processes the request, accesses the data in its memory and sends a fetch packet to the processor boundary. During the program receive stage313the processor registers the fetch packet.

Instructions are fetched in a fetch packet that includes sixteen 32-bit wide words.FIG. 4illustrates sixteen instructions401to416of a single fetch packet. Fetch packets are aligned on 512-bit (16 word) boundaries. This example employs a fixed 32-bit instruction length. Fixed length instructions are advantageous for several reasons. Fixed length instructions enable easy decoder alignment. A properly aligned instruction fetch can load multiple instructions into parallel instruction decoders. Such a properly aligned instruction fetch can be achieved by predetermined instruction alignment when stored in memory by having fetch packets aligned on 512-bit boundaries coupled with a fixed instruction packet fetch. Conversely, variable length instructions require an initial step of locating each instruction boundary before they can be decoded. A fixed length instruction set generally permits more regular layout of instruction fields. This simplifies the construction of each decoder which is an advantage for a wide issue VLIW central processor.

Processor core110(FIG. 1) and L1I cache121pipelines (FIG. 1) are decoupled from each other. Fetch packet returns from L1I cache can take a different number of clock cycles, depending on external circumstances such as whether there is a hit in L1I cache121or a hit in L2 combined cache130. Therefore, program access stage312can take several clock cycles instead of one clock cycle as in the other stages.

FIG. 5illustrates a conceptual view of the streaming engine125of the example processor100ofFIG. 1.FIG. 5illustrates the process of a single stream that is representative of each of the two streams that are controlled by streaming engine125. Streaming engine500includes stream address generator501. Stream address generator501sequentially generates addresses of the elements of the stream and supplies these element addresses to system memory510. Memory510recalls data stored at the element addresses (data elements) and supplies these data elements to data first in first out (FIFO) buffer502. Data FIFO buffer502provides buffering between memory510and processor520. Data formatter503receives the data elements from data FIFO memory502and provides data formatting according to the stream definition. This process will be described in more detail hereinbelow. Streaming engine500supplies the formatted data elements from data formatter503to the processor520. A program executing on processor520consumes the data and generates an output.

Stream elements typically reside in system memory. The memory does not impose a specific structure upon the stream. Programs define streams and thereby impose structure by specifying the stream attributes such as: address of the first element of the stream; size and type of the elements in the stream; formatting for data in the stream; and the address sequence associated with the stream.

The streaming engine defines an address sequence for elements of the stream in terms of a pointer walking through memory. A multiple level nested loop controls the path the pointer takes. An iteration count for a loop level indicates the number of times that level repeats. A dimension gives the distance between pointer positions of that loop level.

In a basic forward stream, the innermost loop consumes physically contiguous elements from memory. The implicit dimension of this innermost loop is 1 element. The pointer itself moves from element to element in consecutive, increasing order. In each level outside the inner loop, that loop moves the pointer to a new location based on the size of that loop levers dimension. This form of addressing allows programs to specify regular paths through memory in a small number of parameters. Table 1 lists the addressing parameters of a basic stream.

TABLE 1ParameterDefinitionELEM_BYTESSize of each element in bytesICNT0Number of iterations for the innermost loop level 0.At loop level0 all elements are physically contiguous.Implied DIM0 = ELEM_BYTESICNT1Number of iterations for loop level 1DIM1Number of elements between the starting points forconsecutive iterations of loop level 1ICNT2Number of iterations for loop level 2DIM2Number of elements between the starting points forconsecutive iterations of loop level 2ICNT3Number of iterations for loop level 3DIM3Number of elements between the starting points forconsecutive iterations of loop level 3ICNT4Number of iterations for loop level 4DIM4Number of elements between the starting points forconsecutive iterations of loop level 4ICNT5Number of iterations for loop level 5DIM5Number of elements between the starting points forconsecutive iterations of loop level 5

In this example, ELEM_BYTES ranges from 1 to 64 bytes as shown in Table 2.

The streaming engine fills each vector CPU core110fetch with as many elements as the streaming engine can from the innermost stream dimension. If the innermost dimension is not a multiple of the vector length, the streaming engine pads that dimension out to a multiple of the vector length with zeros. As noted below the streaming engine will also mark these lanes invalid. Thus, for higher-dimension streams, the first element from each iteration of an outer dimension arrives in lane 0 of a vector. The streaming engine maps the innermost dimension to consecutive lanes in a vector. For transposed streams, the innermost dimension includes groups of sub-elements along dimension 1, not dimension 0, as transposition exchanges these two dimensions.

Two-dimensional streams exhibit greater variety than one-dimensional streams. A basic two-dimensional stream extracts a smaller rectangle from a larger rectangle. A transposed 2-D stream reads a rectangle column-wise instead of row-wise. A looping stream, where the second dimension overlaps the first dimension, executes a finite impulse response (FIR) filter tap, which loops repeatedly or FIR filter samples which provide a sliding window of input samples.

FIG. 6illustrates a region of memory600that may be accessed using a basic two-dimensional stream. The inner two dimensions, represented by ELEM_BYTES, ICNT0, DIM1 and ICNT1 (refer to Table 1) give sufficient flexibility to describe extracting a smaller rectangle620having dimensions621and622from a larger rectangle610having dimensions611and612. In this example, rectangle620is a 9-by-13 rectangle of 64-bit values and rectangle610is a larger 11-by-19 rectangle. The following stream parameters define this stream: ICNT0=9; ELEM_BYTES=8; ICNT1=13; DIM1=88 (11 times 8).

Thus, the iteration count in the 0 dimension621is 9. The iteration count in the 1 dimension622is 13. Note that the ELEM_BYTES only scales the innermost dimension. The first dimension has ICNT0 elements of size ELEM_BYTES. The stream address generator does not scale the outer dimensions. Therefore, DIM1=88, which is 11 elements scaled by 8 bytes per element.

FIG. 7illustrates the order of elements within the example stream ofFIG. 6. The streaming engine fetches elements for the stream in the order illustrated in order700. The first nine elements come from the first row of rectangle620, left-to-right in hops 1 to 8. The 10th through 18th elements comes from the second row, and so on. When the stream moves from the 9th element to the 10th element (hop 9 inFIG. 7), the streaming engine computes the new location based on the pointer's position at the start of the inner loop, not where the pointer ended up at the end of the first dimension. This makes DIM1 independent of ELEM_BYTES and ICNT0. DIM1 represents the distance between the first bytes of each consecutive row.

FIG. 8illustrates the details of streaming engine800, which is representative of streaming engine125ofFIG. 1. Streaming engine800contains three major sections: Stream 0810; Stream 1820; and Shared L2 Interfaces830. Stream 0810and Stream 1820both contain identical hardware that operates in parallel. Stream 0810and Stream 1820both share L2 interfaces830. Each stream810and820provides central processing unit core110(FIG. 1) data at a rate of up to 512 bits/cycle, every cycle. The streaming engine architecture enables this through its dedicated stream paths and shared dual L2 interfaces.

Each streaming engine800includes a respective dedicated 6-dimensional stream address generator811/821that can each generate one new non-aligned request per cycle. Address generators811/821output 512-bit aligned addresses that overlap the elements in the sequence defined by the stream parameters. This will be further described hereinbelow.

Each address generator811/821connects to a respective dedicated micro table look-aside buffer (μTLB)812/822. The μTLB812/822converts a single 48-bit virtual address to a 44-bit physical address each cycle. Each μTLB812/822has 8 entries, covering a minimum of 32 kB with 4 kB pages or a maximum of 16 MB with 2 MB pages. Each address generator811/821generates 2 addresses per cycle. The μTLB812/822only translates one address per cycle. To maintain throughput, streaming engine800takes advantage of the fact that most stream references will be within the same 4 kB page. Thus, the address translation does not modify bits 0 through 11 of the address. If aout0 and aout1 line in the same 4 kB page (aout0[47:12] are the same aout1[47:12]), then the μTLB812/822only translates aout0 and reuses the translation for the upper bits of both addresses.

Translated addresses are queued in respective command queue813/823. These addresses are aligned with information from the respective corresponding Storage Allocation and Tracking block814/824. Streaming engine800does not explicitly manage μTLB812/822. A system memory management unit (MMU) invalidates μTLBs as necessary during context switches.

Storage Allocation and Tracking814/824manages the stream's internal storage, discovering data reuse and tracking the lifetime of each piece of data (described in greater detail below).

Respective reference queue815/825stores the sequence of references generated by the respective corresponding address generator811/821. This information drives the data formatting network so that it can present data to central processing unit core110in the correct order. Each entry in respective reference queue815/825contains the information necessary to read data out of data storage816/826and align it for central processing unit core110. Respective reference queue815/825maintains the information listed in Table 3 in each slot.

TABLE 3Data Slot LowSlot number for the lower half of data associated withaout0Data Slot HighSlot number for the upper half of data associated withaout1RotationNumber of bytes to rotate data to align next elementwith lane 0LengthNumber of valid bytes in this reference

Storage allocation and tracking814/824inserts references in reference queue815/825as address generator811/821generates new addresses. Storage allocation and tracking814/824removes references from reference queue815/825when the data becomes available and there is room in the stream head registers818/828. As storage allocation and tracking814/824removes slot references from reference queue815/825and formats data, it checks whether the references represent the last reference to the corresponding slots. Storage allocation and tracking814/824compares reference queue815/825removal pointer against the slot's recorded Last Reference. If they match, then storage allocation and tracking814/824marks the slot inactive once it is done with the data.

Streaming engine800has respective data storage816/826for a selected number of elements. Deep buffering allows the streaming engine to fetch far ahead in the stream, hiding memory system latency. The amount of buffering in a given product may vary. In the current example, streaming engine800dedicates 32 slots to each stream. Each slot holds 64 bytes of data.

Data storage816/826, respective storage allocation/tracking logic814/824, and reference queues815/825implement the data FIFO502discussed with reference toFIG. 5.

Respective butterfly network817/827includes a seven-stage butterfly network. Butterfly network817/827receives 128 bytes of input and generates 64 bytes of output. The first stage of the butterfly is actually a half-stage. It collects bytes from both slots that match a non-aligned fetch and merges them into a single, rotated 64-byte array. The remaining six stages form a standard butterfly network. Respective butterfly network817/827performs the following operations: rotates the next element down to byte lane 0; promotes data types by a power of two, if requested; swaps real and imaginary components of complex numbers, if requested; converts big endian to little endian depending on the current endian mode of central processing unit core110. The user specifies element size, type promotion and real/imaginary swap as part of the stream's parameters.

Streaming engine800attempts to fetch and format data ahead of central processing unit core110's demand for it so that it can maintain full throughput. Respective stream head registers818/828provide a small amount of buffering so that the process remains fully pipelined. Respective stream head registers818/828are not directly architecturally visible, except for the fact that streaming engine800provides full throughput. Each stream also has a respective stream valid register819/829. Valid registers819/829indicate which elements in the corresponding stream head registers818/828are valid.

The two streams810/820share a pair of independent L2 interfaces830: L2 Interface A (IFA)833and L2 Interface B (IFB)834. Each L2 interface provides 512 bits/cycle throughput direct to the L2 controller for an aggregate bandwidth of 1024 bits/cycle. The L2 interfaces use a credit based multicore bus architecture (MBA) protocol. An example of such a MBA protocol is described in more detail in U.S. Pat. No. 9,904,645, “Multicore Bus Architecture with Non-Blocking High Performance Transaction Credit System,” which is incorporated by reference herein. The L2 controller assigns each interface its own pool of command credits. The pool should have sufficient credits so that each interface can send sufficient requests to achieve full read return bandwidth when reading L2 RAM, L2 cache and multicore shared memory controller (MSMC) memory, as described in more detail hereinbelow.

To maximize performance, in this example both streams can use both L2 interfaces, allowing a single stream to send a peak command rate of two requests/cycle. Each interface prefers one stream over the other, but this preference changes dynamically from request to request. IFA833and IFB834always prefer opposite streams, i.e., when IFA833prefers Stream 0, IFB834prefers Stream 1 and vice versa.

Respective arbiter831/832ahead of each respective interface833/834applies the following basic protocol on every cycle it has credits available. Arbiter831/832checks if the preferred stream has a command ready to send. If so, arbiter831/832chooses that command. Arbiter831/832next checks if an alternate stream has at least two requests ready to send, or one command and no credits. If so, arbiter831/832pulls a command from the alternate stream. If either interface issues a command, the notion of preferred and alternate streams swap for the next request. Using this algorithm, the two interfaces dispatch requests as quickly as possible while retaining fairness between the two streams. The first rule ensures that each stream can send a request on every cycle that has available credits. The second rule provides a mechanism for one stream to borrow the other's interface when the second interface is idle. The third rule spreads the bandwidth demand for each stream across both interfaces, ensuring neither interface becomes a bottleneck by itself.

Respective coarse grain rotator835/836enables streaming engine800to support a transposed matrix addressing mode. In this mode, streaming engine800interchanges the two innermost dimensions of its multidimensional loop. This accesses an array column wise rather than row wise. Respective rotators835/836are not architecturally visible, except as enabling this transposed access mode.

FIG. 9illustrates an example stream template register900. The stream definition template provides the full structure of a stream that contains data. The iteration counts and dimensions provide most of the structure, while the various flags provide the rest of the details. In this example, a single stream template900is defined for all data-containing streams. All stream types supported by the streaming engine are covered by this template. The streaming engine supports a six-level loop nest for addressing elements within the stream. Most of the fields in the stream template map directly to the parameters in that algorithm. The numbers above the fields are bit numbers within a 256-bit vector. Table 4 shows the stream field definitions of a stream template.

Loop 0 is the innermost loop and loop 5 is the outermost loop. In the current example DIM0 is always equal to is ELEM_BYTES defining physically contiguous data. Thus, the stream template register900does not define DIM0. Streaming engine2800interprets all iteration counts as unsigned integers and all dimensions as unscaled signed integers. An iteration count at any level (ICNT0, ICNT1, ICNT2, ICNT3, ICNT4 or ICNT5) indicates an empty stream. Each iteration count must be at least 1 to define a valid stream. The template above fully specifies the type of elements, length and dimensions of the stream. The stream instructions separately specify a start address. This would typically be by specification of a scalar register in scalar register file211which stores this start address. This allows a program to open multiple streams using the same template but different registers storing the start address.

The central processing unit110(FIG. 1) exposes the streaming engine800(FIG. 8) to programs through a number of assembly instructions and specialized registers. For example, a SEOPEN instruction opens a stream. The SEOPEN instruction may specify a stream number indicating opening stream 0 or stream 1. The SEOPEN instruction may specify a data register storing the start address of the stream. The SEOPEN instruction may also specify a template defining a stream of data for the streaming engine to retrieve from memory. In executing the SEOPEN instruction, the corresponding streaming engine begins to prepare operations and may start fetching data from L2 memory. Once open, an instruction can reference the streaming engine data using a SE0 or SE1 instruction. The SE0 and SE1 instruction causes streaming engine data to be supplied from the last referenced address. For example, a SE0 immediately following an SEOPEN will supply the streaming engine data at the base address, but does not advance the address. An SE0++ or SE1++ instruction causes streaming engine data to be supplied from the last referenced address and then advances the address sequentially based on the template.

As discussed with respect toFIG. 1, a processor may include multiple levels of cache, such as a L1D cache123and L2 cache130. In certain cases, the streaming engine125may obtain data from the L2 cache130. After processing by the processor, data is written out, but does not output data to the L2 cache130directly, but rather via the L1D cache123. As the streaming engine125may be decoupled from the processor core110there may be ambiguity if the streaming engine attempts to access a data address after a data write from the processor to that same data address. For example, the processor may execute a STORE instruction followed by an SEOPEN instruction. In certain cases, the STORE instruction may attempt to store data into a memory address that would be accessed as a part of the SEOPEN instruction. As discussed above, the processor first writes outputs to the L1D cache, which then writes to the L2 cache. As the streaming engine can prefetch data directly from the L2 cache, the SEOPEN instruction may cause the streaming engine125to begin accessing data from the L2 cache. This potentially sets up a race condition between the time the data from the STORE instruction is written to the memory address in the L2 cache and when the SEOPEN instruction attempts to access the data in the same memory address in the L2 cache.

Rather than attempting to analyze the STORE command and then accessing both the L1D cache and the L2 cache to send data to the streaming engine, or otherwise imposing a load/store architecture on the streaming engine, an implicit memory fence may be defined based on the streaming engine opening a stream. This implicit memory fence provides a guarantee that a store operation to an address will complete before a subsequent access to the address by the streaming engine. However, the reverse, e.g., a SEOPEN followed by a STORE to the same address, is not guaranteed and it is ambiguous as to what value will be in the address when accessed by the SEOPEN.

The memory fence operation may be performed when a SEOPEN instruction is executed by checking the L1 D to see if there are any outstanding writes for data stored in the L1D that have not been written to the L2 cache. When the SEOPEN command is received, a pipeline controller of the processor sends the command to the streaming engine and then stalls on the command until the streaming engine starts to return data. The pipeline controller checks with the L1D memory controller to see if there are any outstanding or pending L2 writes. If there are outstanding or pending L2 writes, the streaming engine stalls waiting for the L2 writes to complete before the streaming engine starts to pre-fetch data from L2 and return data to the CPU. After the L1D sends data to the L2 memory controller, the L2 memory controller sends back an acknowledgement to the L1D indicating that the L2 cache has consumed the data. The L1D memory controller tracks which data has not yet been sent to the L2 along with outstanding write requests and completed write requests.

Stalling the processor and streaming engine every time to wait for writes to be committed from the L1D cache can be expensive and unnecessary when a program tries to issue a set of writes, but it is not necessary for the writes to be visible to the streaming engine. According to certain aspects, a memory tag may be used to help fence one or more commands together. For example, a set of stream-open and stream-write operations may be tagged with a color value based on a processor register field. As used herein, a color value refers to an identifier that may be associated with a set of memory operations to help group or otherwise organize the set of memory operations. Each memory transaction may include a source field comprising a set of one or more bits that encode the color value associated with the memory transaction. Where multiple color values are used, the streaming engine can receive multiple signals from the L1D memory controller, indicating whether there are any pending writes associated with a particular color value. If there are pending writes associated with a specified color value, then the streaming engine can be stalled waiting for those pending writes to complete. If there are pending writes associated with a color value different from the specified color value, then execution proceeds without stalling.

In certain cases, memory tags may be used in conjunction with implicit memory fences. For example, a color value associated with a stream-open command may be passed to the streaming engine. When the stream-open command is executed by the streaming engine, the associated color value may be passed to the L1D memory controller and checked against color values associated with pending writes. If there are pending writes with a matching color value, then the streaming engine can be stalled until the pending writes are completed. If there are no pending writes with a matching color value, then execution proceeds without stalling.

The color value is based on a k value provided by a processor register field, such as a task state register (TSR) field. This processor register field is read by every load and store operation to determine the color value associated with the memory operation. According to certain aspects, the TSR may be included as a part of a set of control registers, such as control registers114ofFIG. 1. Two or more color values may be available. According to certain aspects an instruction, such as a MTAG instruction may be provided to programmatically adjust the processor register field, and thus color value, based on an argument provided with the MTAG instruction. The color value may be adjusted, for example, by either directly assigning a color value, jumping to the next color value, or flipping the color value. Once the color value is adjusted, memory operations in the next execute packet may be tagged with the adjusted color value.

Generally, the instruction fetch pipeline may be programmatically stalled while certain memory operations with the same color value are completed. In some cases, multiple memory fencing behaviors may be supported. For example, a MFENCE k command may stall the instruction fetch pipeline until all pending load and store operations with the same color value are completed. The k argument may define the specific color value, such that when k=00000b, the instruction fetch pipeline is stalled until all memory transactions with color value=0 complete. Likewise, when k=00001b, the instruction fetch pipeline is stalled until all memory transactions with color value=1 complete. Additionally, when k=1111 b, the instruction fetch pipeline is stalled until all memory transactions, regardless of memory tag color, are completed. A second command, MFENCEST k may stall the instruction fetch pipeline until all pending store operations, but not load operations, with the color value specified by k are completed. The k argument for MFENCEST may be operationally similar to MFENCE and when k=00000b or k=00001b, the instruction fetch pipeline will stall until all store transactions with color value=0 or 1 are completed, respectively. Similarly, when k=1111 b, the instruction fetch pipeline is stalled until all store transactions, regardless of memory tag color, are completed.

FIG. 10is a flow diagram1000illustrating a technique for executing a plurality of instructions by a processor, in accordance with aspects of the present disclosure. At block1002, a first instruction is received. The first instruction is configured to cause the processor to output a first data value to a first address in a first data cache. For example, the processor may receive a STORE instruction to store a value in L2. At block1004, the processor outputs the first data value to an address in a second data cache. For example, the processor may be coupled to the L2 cache via the L1 D cache and the processor outputs the data value to the L1 D cache. At block1006, the processor receives a second instruction configured to cause a streaming engine associated with the processor to prefetch data from the first data cache. For example, the processor may receive a SEOPEN instruction telling the streaming engine to begin fetching data from L2. At block1008, the processor determines that the first data value has not been outputted from the second data cache to the first data cache. For example, a check may be performed with the L1D memory controller to verify whether there are any outstanding or pending L2 writes. At1010, execution of the second instruction may be stalled, for example, by inserting no-op instructions into the pipeline. At block1012, the pipeline controller receives an indication, from the second data cache, that the first data value has been outputted from the second data cache to the first data cache. For example, the L1D memory controller may receive an acknowledgment that the L2 cache has written the data. At block1014, the pipeline controller resumes execution of the second instruction based on the received indication.

In this description, the term “couple” or “couples” means either an indirect or direct wired or wireless connection. Thus, if a first device couples to a second device, that connection may be through a direct connection or through an indirect connection via other devices and connections. The recitation “based on” means “based at least in part on.” Therefore, if X is based on Y, X may be a function of Y and any number of other factors.