Computing architecture and method for processing data

A processing device includes an execute processor configured to execute data processing instructions; and an access processor configured to be coupled with a memory system to execute memory access instructions; wherein the execute processor and the access processor are logically separated units, the execute processor having an execute processor input register file with input registers, and a data processing instruction is executed as soon as all operands for the respective data processing instruction are available in the input registers.

FOREIGN PRIORITY

This application claims priority to Great Britain Patent Application No. 1319217.4, filed Oct. 31, 2013, and all the benefits accruing therefrom under 35 U.S.C. § 119, the contents of which in its entirety are herein incorporated by reference.

BACKGROUND

The present invention relates to computing architectures, in particular to computing architectures having logically separated access and execute processors. The present invention further relates to methods for providing data from the access processor to the execute processor.

For some time, there has been a growing gap between processor performance and memory performance, since the processing frequency is increasing faster than the access frequency for retrieving memory data due to memory latency. In other words, the retrieval speed of data from the memory is decreasingly able to catch up with the processing speed of the processors.

Decoupled access/execute computer architectures have long been well-known and have been introduced e.g., by J. E. Smith, “Decoupled Access/Execute Computer Architectures”, ACM Transactions on Computer Systems, Vol. 2, No. 4, November 1984, pages 289-308, wherein the concept of a decoupled access/execute computer architecture as a means to tolerate larger memory latencies for general purpose computing has been introduced.

The above document describes the execute processor with an out-of-order execution that separates the fetch and decode stages from the general execute stage in a pipelined processor by using a buffer. The purpose of the buffer is to partition the memory access functions and execute functions provided by a computer program and to achieve high performance by exploiting a fine-grained parallelism between the two. Thereby, the memory latency can be effectively hidden from the processor execution.

However, the proposed architecture could so far not be established for general purpose computing. One drawback becomes obvious in case of a branch misprediction in the processor, i.e., if the entire buffer will need to be deleted/flushed wasting clock cycles, thereby reducing the efficiency of the decoupled access/execute computer architecture. Consequently, control-intensive code, in particular data-dependent branches in the code thread, cannot be handled in a satisfactory manner. Memory latency problems in general purpose computers have instead been addressed by alternative concepts, such as hardware-based, out-of-order execution and prefetching.

Hence, the above document is focused on special-purpose processing, in particular on DSP accelerators, which are generally facing the same fast-growing gap between processor and memory performance. For specific DSP workloads, memory bandwidth has become the bottleneck factor that limits the performance since these processes do not include frequent code branching.

In document A. Milidonis et al., “Decoupled Processor Architecture for Accelerating Data-Intensive Applications Using Scratch-Pad Memory Hierarchy”, Journal of Signal Processing Systems, June 2010, Vol. 59, Issue 3, pp. 281-296, a decoupled architecture is described having a memory hierarchy built with scratch-pad memories between which data is transferred using DMA operations that are under the control of an access processor. An access processor of the decoupled architecture is further configured to control the exchange of data between an L1 scratch-pad memory and a register file of the execute processor. Data transfers to and from the register file of the execute processor are initiated by the execute processor which for that purpose submits a read or write request to the access processor by writing a bit flag into a buffer.

SUMMARY

In one embodiment, a processing device includes an execute processor configured to execute data processing instructions; and an access processor configured to be coupled with a memory system to execute memory access instructions; wherein the execute processor and the access processor are logically separated units, the execute processor having an execute processor input register file with input registers, and a data processing instruction is executed as soon as all operands for the respective data processing instruction are available in the input registers.

In another embodiment, a computing system includes a memory system; an execute processor configured to execute data processing instructions; and an access processor coupled with the memory system to execute memory access instructions; wherein the execute processor and the access processor are logically separated units, the execute processor having an execute processor input register file with input registers, and a data processing instruction is executed as soon as all operands for the respective data processing instruction are available in the input registers.

In another embodiment, a method for operating a processing device, the processing device including an execute processor configured to execute data processing instructions, and an access processor configured to be coupled with a memory system to execute memory access instructions includes reading data out of the memory system; providing the data to the execute processor as operands; and determining whether all operands are available in the execute processor for one of one or more waiting data processing instructions; wherein the data processing instruction which is triggered by the availability of all operands associated with one of the waiting data processing instructions is executed; wherein the execute processor and the access processor are logically separated units, the execute processor having an execute processor input register file with input registers, and a data processing instruction is executed as soon as all operands for the respective data processing instruction are available in the input registers.

DETAILED DESCRIPTION

According to an embodiment, a processing device is provided, including an execute processor for performing data processing instructions; and an access processor to be coupled with a memory system and for performing memory access instructions, wherein the execute processor and the access processor are logically separated units, the execute processor has an execute processor input register file with input registers, and data processing instructions are executed in the execute processor as soon as all operands for the respective data processing instruction are available in the input registers.

Conventionally, a decoupled architecture provides a computing system having an access processor and an execute processor. The access and execute processors are logically separated hardware units which are provided with split threads of memory access instructions and data processing instructions, respectively. The synchronization of data flow between the access processor and the execute processor is usually performed through a set of architectural queues. The access processor performs the calculation of addresses for read and write operations to and from the memory system, wherein data retrieved from the memory stored in a buffer can be consumed by the execute processor as required. Mostly, this data is retrieved from the memory before it is actually processed by the execute processor, so that larger access latencies regarding the data retrieval from the memory can be tolerated. Further queues may be provided which provide additional mechanisms for synchronization between access processor and execute processor in order to exchange information on certain branches of the code thread that have to be processed. In case the access processor has to wait for data from the execute processor for calculating an address, the memory access stream will be blocked temporarily. This effect is known as loss of decoupling, which will negatively affect the overall performance of the computing system having such a decoupled architecture.

One idea of the above processing device is to omit the data buffer in the read data path from the access processor to the execute processor and to directly transfer the required data from the memory system to an execute processor input register file. Processing in the execute processor core is then carried out upon the availability/provision of operands/data stored in the execute processor input register file which are needed as input for the waiting data processing instruction. In other words, data processing instructions are waiting in the execute processor wherein the processing of a respective data processing instruction is initiated/triggered by the availability of all operands required to perform the respective instruction.

Such a decoupled architecture may allow for better efficiency, particularly if the instruction code thread has frequently nested branches, which lowers the efficiency of a conventional processing device of decoupled architecture using a read buffer in the read data path between the access processor and the execute processor. According to embodiments of the invention, the overall memory system may be substantially simplified and the power consumption may also be significantly reduced.

Moreover, according to embodiments, data processing instructions may be triggered by the availability of the operands associated with the respective data processing instruction, i.e., by the arrival of the last required operand in the execute processor input register file.

According to an embodiment, the operands may be transferred as read data from the memory system by means of a read data line, wherein the read data line directly couples the memory system with the execute processor input register file.

Furthermore, the input register in which read data are to be written may be addressed by means of read tag data provided by the access processor, wherein the read tag data indicates an address of the input register file to which the read data is to be written.

It may be provided that the availability of operands in the input registers is indicated by a respective data valid flag associated to each of the input registers in the execute processor, so that a respective one of the waiting data processing instructions is executed as soon as all data valid flags associated to the input registers keeping the operands needed, are set. In particular, the access processor may provide read data to those input registers of the execute processor whose data valid flags are not set and/or to those input registers which shall keep/obtain operands for data processing instructions waiting in the execute processor.

According to the data processing instruction which has been executed, the data valid flag of the associated operands may be either reset or kept. In particular, the resulting data may be written either back to one of an execute processor input register for processing a further instruction waiting in the execute processor and an execute processor output register for writing the output data to a memory system. Moreover, the result/output data, which are an outcome of the data processing instructions executed in the execute processor, may be written to the memory system via a write buffer, wherein particularly the process of writing data from the write buffer to the memory system is prioritized over any further memory accesses.

According to an embodiment, the execute processor may include at least one of a microcontroller, an ALU, a butterfly unit for an FFT calculation and the like.

The access processor may be directly coupled to the execute processor input register file to transmit internal instructions directly to the execute processor. One or more execute processors may be provided to handle a plurality of execution threads each associated with a separate portion of the memory system.

According to an embodiment of a further aspect, a computing system is provided comprising the above processing device and a memory system. Furthermore, the access processor and the memory system may be logically integrated.

According to an embodiment of a further aspect, a method for operating the above processing device is provided, wherein data to be processed in the execute processor is read out of a memory system and provided to the execute processor as operands, wherein it is determined whether all operands are available in the execute processor for one of the one or more waiting data processing instructions, and wherein the respective data processing instruction which is triggered by the availability of all operands associated with the respective waiting data processing instructions is executed.

FIG. 1schematically shows a block diagram of a computing system with a processing device1of a decoupled architecture. The processing device1comprises an execute processor2and an access processor3, which are logically separated hardware units integrated in one single or multiple hardware blocks, such as integrated circuits.

The access processor3is provided with an access processor core32for performing memory access instructions and the execute processor2is provided with an execute processor core22for performing data processing instructions. In general, the access processor3and the execute processor2might be integrated as small cores with dedicated instruction sets and their own or shared instruction memory (not shown).

The access processor3is configured to perform the calculation of addresses for read and write operations from and to a memory system4coupled with the processing device1or included therein. According to another embodiment, the access processor3may be integrated in the memory system4. The organization of the memory system4, e.g., number of memory banks, bandwidth, access and cycle times, refresh cycles and the like, is freely exposed to the access processor3. The access processor3may include a programmable address mapping scheme to control access of the memory system4.

Moreover, conventional memory controller functions may be integrated into the access processor2in a programmable way; in particular, the ability to program the access processor2to control the page open and close and refresh operations of DRAMs in the memory system4allows to even better adapt the memory operations to the application characteristics.

In operation, the access processor core32and the execute processor core22are supplied with a memory access instruction thread (memory access code thread) via an access instruction bus34and a data processing instruction thread (data processing code thread) via a processing instruction bus24, respectively. The memory access instruction thread and the data processing instruction thread are obtained as a result of a suitable compilation of an original program code to perform the program code in the processing device1.

Furthermore, the execute processor2is provided with an execute processor input register file21having a number of input registers25and with an execute processor output register file23having a number of output registers26. To bring/transfer data to the execute processor2, a read data bus5is provided which connects the memory system4and optionally the access processor3with the execute processor2, so that read data can be directly transferred from the memory system4to an execute processor input register file21of the execute processor2under control of the access processor3. In particular, no buffer is provided within the read data bus5.

The access processor3is further configured to schedule instructed read accesses from the memory4and via the read data bus5such that the data is directly transferred from the memory system4to input registers25of the execute processor input register file21in the execute processor2when data is to be retrieved from the memory system4. Furthermore, the execute processor output register file23is connected via a write data bus6to a write buffer7for writing resulting/processed data available in the execute processor output register file23to the memory system4.

Each of the input registers25of the execute processor input register file21can be addressed via read tag data provided via a read tag address line8connecting the access processor3with the execute processor2. The read tag data is generated by the access processor core32which provides an input register address for the targeted input register25in the execute processor input register file21. Thereby, read accesses are performed by providing the read data and the input register address to channel the read data to the addressed input register25in the execute processor input register file21. Hence, read accesses are substantially initiated and controlled by the access processor3. In this way, it is under the control of the access processor core32to start a read operation for the execute processor2and a corresponding data transfer at any time.

The execute processor output registers26are used to directly write any output data (result data), which are obtained by processing in the execute processor core22, to the memory system4. When data resulting from processing in the execute processor2is to be written to the memory system4, the write data is directly written to the write buffer7via the output registers26of the execute processor output register file23and via the write data bus6.

The memory address location to which the write data is to be written is selected by write tag data that is generated or determined by the execute processor core22and communicated via a write tag address line9connecting the execute processor2with the write buffer7and the access processor3. The write tag address line9is substantially used for communicating the write address as write tag data for the respective output data to be written back to the memory system4. Usually, the write tag data might be equal to or derived from the read tag data associated with the input data that was retrieved from the memory system4and contains the operand values for an instruction.

For addressing the memory system4to write output data, a write address table10can be used. The access processor3is configured to pre-calculate write addresses for directing the output data to be written from the write buffer7to a desired memory location in the memory system4. The write addresses are stored in the write address table10. The write address table10also receives write tag data via the write tag data line9and selects the write address to be used for writing the output data in response of the write tag data, i.e., the write addresses are associated with the write tag data in the write address table10. The write buffer7is then configured to buffer the output data to be written. The output data is only buffered for more than one clock cycle when the memory system4, e.g., the memory banks involved in the write operation, are busy with any other ongoing access.

It can be provided that the write accesses from the write buffer7to the memory system4may have priority over any read accesses required by the access processor3. In general, the execute processor2initiates the process of providing output data that is to be written from the output register file23of the execute processor2over the write data bus6to the memory system4. In a typical implementation as described above, the access processor3can be the master and the execute processor2can be the slave, with the access processor3performing the scheduling of memory accesses, the busses and the execution in the execute processor2.

In contrast to the microcontroller-based approach as described above, the execute processor2might further be a general processing unit that can also be a simple arithmetic logical unit (ALU) or a set of dedicated hardware function units that can only perform a single type of operation, for example a butterfly unit for an FFT calculation or a unit for an FIR filter. In this case the execute processor core22is reduced to a simple data path.

Processing by the execute processor2is triggered by the availability of the operands/data in the execute processor input register file21or by the arrival of the last operand/data among all operands needed to perform a respective data processing instruction waiting to be processed in the execute processor2. In other words, as soon as the operands, i.e., the read data required for a specific instruction waiting to be processed in the execute processor2, are fully available, the respective instruction is executed being triggered by the arrival/availability of the last missing operand for the specific data processing instruction.

The availability of each of the required operands might be indicated by a data valid flag associated with each input register25of the execute processor input register file21. The data valid flag is set upon the arrival of any new read data via the read data bus5and the respective read data tag indicating the input register address of the input register25in the execute processor input register file21. The data valid flag associated with each of the input registers25can be reset by the execute processor core22upon completion of the processing of the specific instruction triggered before.

It might also be possible, for example, for FFT calculations using twiddle factors as input operands, to re-use the same input data multiple times as long as the data valid flag is kept set during that time. The access processor3may include multiple hardware threads that are executed in parallel or in an interleaved fashion. Each thread can perform a specific type of address generation and the corresponding execution. For example, for an FFT application, one thread can be used to generate the addresses to access the samples, and a second thread can be used to generate the addresses for the twiddle factors.

Communication between the execute processor2and the access processor3enables both the memory accesses and the data processing of the access processor3and the execute processor2to be performed out of order, which can result in substantial performance improvements because it may resolve dependencies of locking, e.g., due to certain memory banks being occupied by other accesses.

Furthermore, the communication scheme can be used to prevent the access processor3from overloading the execute processor2. By limiting the number of different read tag values that can be used or by applying the restriction that no new read tag can be generated for which there is still an entry in the write table, an efficient kind of back pressure can be applied to guarantee correct operation without requiring additional queues in the read data bus5, share buffer memory or by any other form of communication between the execute processor2and access processor3. The conventionally used read data buffers are replaced by a management of input register addresses supplied via the read tag address line8thereby assigning a free (not used) input register25with the next read data to be supplied to the execute processor2. By means of a dedicated compiler, the waiting instruction watches respective input registers25for operands to perform its instruction as soon as all operands are available in the execute processor input register file21.

To improve communication between the access processor3and execute processor2, particularly for controlling each other's operation (mutual operational control), it may be provided that the access processor3can write data directly to the execute processor input register file21. Similarly, the execute processor2can write directly to an input register file31of the access processor3. For this purpose, specially reserved registers can be used or these operations can be explicitly scheduled by the access processor3.

In particular, dedicated instruction sets for both the execute processor2and the access processor3might be applied. Both processors2,3can be based on a programmable state machine (B-FSM controller), which allows efficient data validation testing for the execute processor2as well as efficient loop controls for the access processor3. For instance, a programmable state machine in the execute processor2can be used to determine if all the operands for a given instruction are available. Moreover, a programmable state machine of the access processor3can be used to allow parallel testing of multiple counters and other conditions, in order to enable an efficient programmable instruction scheduling.

One further advantage of the above-described architecture is that it is scalable in terms of memory configuration. For example, memories with different cycle times, any numbers of memory banks, while the memory lines and the like requires only the access processor3to be adapted while the main processing in the execute processor2substantially remains unchanged. This allows for the combination of different memory systems to one system, wherein each memory system has its specific program and all memory systems are served by the same execute processor2.

InFIG. 2, a flow diagram for illustrating a typical data flow scheme in the computing system ofFIG. 1is shown. In operation S1, one or more data processing instructions are loaded into a data processing instruction memory in the execute processor2. In the instruction memory, instructions are waiting to be processed while watching the associated input registers25for operands becoming valid. As described above, the loaded data processing instructions require the availability of operands in the associated input registers25of the execute processor input register file21.

In operation S2, memory access instructions are loaded into an access instructions memory of the access processor3. The access instructions are performed successively, as long as free input registers25are available or as long as the execute processor2has executional instructions that are waiting for operands to appear in specific input registers25. As described above, an operand “appears” in a respective input register25when the respective data valid flag associated to the respective input register25has been set.

In operation S3, data to be processed is read out of the memory system4according to a mapping scheme implemented in the access processor3. The scheduling and mapping scheme manages the data access, e.g., for optimizing the latency and access times, and provides the data retrieved from the memory system4directly to an addressed input register25of the execute processor input register file21. The input register25is addressed using the read tag data associated with the read data to be transmitted to the execute processor2.

If in operation S4 it is determined in the execute processor2that for one of the one or more waiting instructions all operands are available (Alternative: Yes) the process is continued with operation S5. Otherwise (Alternative: No), operation S3 is repeated.

In operation S5, the instruction for which all operands have been determined as available is executed and the valid flags are reset as required by the instruction. The output data obtained by performing the respective instruction is stored in a dedicated output register26of the execute processor output register file23. In other words, as soon as the last operand required by one of the waiting instructions is available, i.e., the respective data valid flag is set, the instruction is executed and the respective output is written via an output register26to the write buffer7. In some embodiments it might be required that the input data is valid for more than one instruction to be executed, so that the reset operation is not performed on the respective input register25.

If write data arrives in the write buffer7in operation S6, it will be written directly to the memory system4with priority over any read access performed in operation S3. The memory location to which the output data stored/buffered in the write buffer7is to be written is indicated by the write addresses pre-calculated by the access processor3and selected by the write tag data supplied by the execute processor2as a memory address associated with the respective instruction that has been executed. The process continues with operation S1, wherein the instruction buffers within the access processor3and execute processor2are filled up as necessary and the process continues.

By using multiple threads running on a single execute processor2or, alternatively, having multiple execute processors2, attached to the memory system4comprised of multiple memory banks and controlled by the access processor3, it becomes possible to program pipelined modes of operation. In this case, some of the memory banks of the memory system4can be programmed to “serve” as the buffers (“latches”) separating the pipeline stages, each of which is executed on different execute processors2(or in special cases, in an interleaved fashion on a single execute processor).

For example, a first pipeline stage implemented as code executed by a first execution thread (or a first execute processor), involves processing input data stored in one or more first memory banks of the memory system4and storing the resulting output data in one or more second memory banks of the memory system4(different from the first memory banks). This first pipeline may access the one or more first memory banks multiple times, for example, to store and retrieve intermediate processing results. A second pipeline stage implemented as code executed by a second execution thread (or second execute processor) involves processing the output data from the first pipeline stage that it retrieves from the one or more second memory banks, and storing the results of that processing e.g., in one or more third memory banks of the memory system4.

The number of execution threads (or execute processors) and the number of memory banks can be varied based upon the actual processing and memory access requirements by the various pipeline stages, enabling a very flexible adaptation and tuning of the available resources to obtain an optimal performance.

REFERENCE LIST