Enforcing strongly-ordered requests in a weakly-ordered processing

The disclosure is directed to a weakly-ordered processing system and method for enforcing strongly-ordered memory access requests in a weakly-ordered processing system. The processing system includes a plurality of memory devices and a plurality of processors. Each of the processors are configured to generate memory access requests to one or more of the memory devices, with each of the memory access requests having an attribute that can be asserted to indicate a strongly-ordered request. The processing system further includes a bus interconnect configured to interface the processors to the memory devices, the bus interconnect being further configured to enforce ordering constraints on the memory access requests based on the attributes.

BACKGROUND

The present disclosure relates generally to processing systems, and more particularly, to a method and apparatus for enforcing strongly-ordered requests in a weakly-ordered processing system.

Computers and other modern processing systems have revolutionized the electronics industry by enabling sophisticated tasks to be performed with just a few strokes of a keypad. These sophisticated tasks often involve a number of devices that communicate with one another in a fast and efficient manner using a bus. The bus provides a shared communication link between devices in a processing system.

The types of devices connected to a bus in a processing system may vary depending on the particular application. Typically, the sending devices on the bus may be processors, and the receiving devices on the bus may be memory devices or memory mapped devices. In these systems, the processors often achieve performance benefits by allowing memory operations to be performed out-of-order. For example, a sequence of memory operations could be reordered to allow all operations to the same page in memory to be executed before a new page is opened. Processing systems that are allowed to reorder memory operations are generally referred to as “weakly-ordered” processing systems.

In certain instances, the reordering of memory operations may unpredictably affect program behavior. For instance, an application may require a processor to write data to memory before the processor reads from that memory location. In a weakly-ordered processing system, there is no guarantee that this will occur. This result may be unacceptable.

Various techniques have been employed for executing ordered memory operations in a weakly-ordered processing system. One technique is simply to delay certain memory operations until all memory operations before it are executed. In the previous example, the processor may delay issuing a read request until after it writes to the memory location. Another technique is to use a bus command referred to as a memory barrier when an ordered memory operation is required. A “memory barrier” may be used to ensure that all memory access requests issued by a processor before the memory barrier are executed before all memory access requests issued by the processor after the memory barrier. Again, in the previous example, a memory barrier could be sent to the memory by the processor before issuing a read request. This would ensure that the processor writes to memory before it reads from the same memory location.

Both techniques are effective, but inefficient from a system performance perspective. The memory barrier may be particularly inefficient in processing systems with multiple memory devices. In these processing systems, a memory barrier would need to be issued by the processor to every memory device it can access to enforce an ordering constraint on memory operations. Thus, there is a continuing need for more efficient methods to perform ordered memory operations in a weakly-ordered processing system.

SUMMARY

One aspect of a weakly-ordered processing system is disclosed. The processing system includes a plurality of memory devices and a plurality of processors. Each of the processors are configured to generate memory access requests to one or more of the memory devices, with each of the memory access requests having an attribute that can be asserted to indicate a strongly-ordered request. The processing system further includes a bus interconnect configured to interface the processors to the memory devices, the bus interconnect being further configured to enforce ordering constraints on the memory access requests based on the attributes.

Another aspect of a weakly-ordered processing system is disclosed. The processing system includes a plurality of memory devices, and a plurality of processors. Each of the processors are configured to generate memory access requests to one or more of the memory devices, with each of the memory access requests having an attribute that can be asserted to indicate a strongly-ordered request. The processing system further includes a bus interconnect having means for interfacing the processors to the memory devices, and means for enforcing ordering constraints on the memory access requests based on the attributes.

One aspect of a bus interconnect is disclosed. The bus interconnect includes a bus switch configured to interface a plurality of processors to a plurality of memory devices in a weakly-ordered processing system. Each of the processors are configured to generate memory access requests to one or more of the memory devices, with each of the memory access requests having an attribute that can be asserted to indicate a strongly-ordered request. The bus interconnect further includes a controller configured to enforce ordering constraints on the memory access requests based on the attributes.

One aspect of a method for enforcing strongly-ordered memory access requests in a weakly-ordered processing system is disclosed. The method includes receiving, from a plurality of processors, memory access requests for a plurality of memory devices, one of the memory access requests having a memory attribute indicating a strongly-ordered memory access request. The method further includes enforcing an ordering constraint for the strongly-ordered memory access request in relation to the other memory access requests in response to the attribute.

DETAILED DESCRIPTION

The detailed description set forth below in connection with the appended drawings is intended as a description of various embodiments of the invention and is not intended to represent the only embodiments in which the invention may be practiced. The detailed description includes specific details for the purpose of providing a thorough understanding of the invention. However, it will be apparent to those skilled in the art that the invention may be practiced without these specific details. In some instances, well known structures and components are shown in block diagram form in order to avoid obscuring the concepts of the invention.

FIG. 1is a conceptual block diagram illustrating an example of a weakly-ordered processing system. The processing system100may be a computer, resident in a computer, or any other system capable of processing, retrieving and storing information. The processing system100may be a stand-alone system, or alternatively, embedded in a device, such as a wireless or wired telephone, a personal digital assistant (PDA), a desktop computer, a laptop computer, a game console, a pager, a modem, a camera, automotive equipment, industrial equipment, video equipment, audio equipment, or any other suitable device requiring processing capability. The processing system100may be implemented as integrated circuit, part of an integrated circuit, or distributed across multiple integrated circuits. Alternatively, the processing system100may be implemented with discrete components, or any combinations of discrete circuits and integrated circuit. Those skilled in the art will recognize how best to implement the processing system100for each particular application.

The processing system100is shown with multiple processors102a-102cin communication with multiple memory devices104a-104cover a bus106. The actual number of processors and memory devices required for any particular application may vary depending on the computational power required and the overall design constraints. A bus interconnect108may be used to manage bus transactions between the processors102a-102cand memory devices104a-104cusing point-to-point switching connections. In at least one embodiment of the bus interconnect108, multiple direct links may be provide to allow several bus transactions to occur simultaneously. Alternatively, the bus interconnect108may be configured to support a shared bus arrangement.

Each processor102a-102cmay be implemented as any type of bus mastering device including, by way of example, a general purpose processor, a digital signal processor (DSP), application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic, discrete gate or transistor logic, discrete hardware components, or any other processing entity or arrangement. One or more of the processors102a-102cmay be configured to execute instructions under control of an operating system or other software. The instructions may reside in one or more of the memory devices104a-104c. Data may also be stored in the memory devices104a-104c, and retrieved by the processors102a-102cto execute certain instructions. The new data resulting from the execution of these instructions may be written back into the memory devices104a-104c. Each memory device104a-104cmay include a memory controller (not shown) and a storage medium (not shown). The storage medium may include RAM memory, DRAM memory, SDRAM memory, flash memory, ROM memory, PROM memory, EPROM memory, EEPROM memory, CD-ROM, DVD, registers, hard disk drive, a removable disk, or any other suitable storage medium.

Each processor102a-102cmay be provided with a dedicated channel106a-106con the bus106for communicating with the bus interconnect108. Similarly, the bus interconnect108may use a dedicated channel106d-106fon the bus to communicate with each memory device104a-104c. By way of example, a first processor102acan access a target memory device104bby sending a memory access request over its dedicated channel106aon the bus106. The bus interconnect108determines the target memory device104bfrom the address of the memory access request and forwards the request to the target memory device104bover the appropriate channel106eon the bus106. A “memory access request” may be a write request, a read request, or any other bus related request. An originating processor102a-102cmay issue a write request to a target memory device104a-104cby placing the appropriate address with the payload on the bus106and asserting a write enable signal. An originating processor102a-102cmay issue a read request to a target memory device104a-104cby placing the appropriate address on the bus106and asserting a read enable signal. In response to the read request, the target memory device104a-104cwill send the payload back to the originating processor102a-102c.

In at least one embodiment of the processing system100, the processors102a-102cmay transmit a “memory attribute” with each memory access request. The “memory attribute” may be any parameter that describes the nature of the memory access request. The memory attribute may be transmitted with the address over the address channel. Alternatively, the memory attribute may be transmitted using sideband signaling or some other methodology. The memory attribute may be used to indicate whether or not the memory access request is strongly-ordered. A “strongly-ordered” request refers to a memory access request that cannot be executed out of order.

The bus interconnect108may be used to monitor the memory attribute for each memory access request from the processors102a-102c. If a memory attribute indicates a strongly-ordered memory access request, the bus interconnect108may enforce an ordering constraint on that request. By way of example, a memory access request from a first processor102ato a target memory device104amay include a memory attribute. The bus interconnect108may be used to determine from the memory attribute whether the request is strongly-ordered. If the bus interconnect108determines that the request is strongly-ordered, it sends a memory barrier to every memory device104band104cthat the first processor102ais capable of accessing, other than the target memory device104a. The bus interconnect108also sends the memory access request to the target memory104awithout a memory barrier because the target memory device104awill implicitly handle it as a strongly-ordered request due to the memory attribute associated with the memory access request.

FIG. 2is a functional block diagram illustrating an example of a bus interconnect in a weakly-ordered processing system. The manner in which the bus interconnect is actually implemented will depend on the particular application and the design constraints imposed on the overall system. Those skilled in the art will recognize the interchangeability of various designs, and how best to implement the functionality described herein for each particular application.

Referring toFIG. 2, a bus register202may be used to receive and store information from the bus106. The bus register202may be any type of storage device such as a first-in-first-out (FIFO) memory, or other suitable storage device. The information received and stored by the bus register202may be any bus related information, but typically includes the address and memory attribute for each memory access request, and in the case of a write request, the payload. The address for each memory access request is also provided to a decoder204. The decoder204may be used to determine the target memory device for each memory access request in the bus register202. This determination is used to generate a signal205which controls a bus switch206. The bus switch206is used to demultiplex each memory access request in the bus register202to the appropriate channel of the bus106for its target memory device. A controller208may be used to control the timing of memory access requests released from the bus register202.

FIG. 3is a functional block diagram illustrating an example of a controller in a bus interconnect for a weakly-ordered processing system. The controller208is primarily responsible for enforcing ordering constraints on memory operations based on a signal209it receives from the decoder204. The signal209includes information relating to the memory attribute for each memory access request, which may be stored in a first register302. The signal209also includes information that identifies each memory device, other than the target memory device, that the originating processor is capable of accessing. The particular memory devices accessible by each processor are preconfigured during the design stage, and therefore, can be programmed or hard wired into the decoder. In any event, a second register304may be used to store this data. The first and second registers302,304may be separate registers as shown inFIG. 3, or alternatively a single register. In some embodiments of the controller208, the information from the decoder204may be stored in registers shared with other bus interconnect functions. Each register may be a FIFO or any other suitable storage medium.

The controller208enforces ordering constraints on memory operations by controlling the timing of memory access requests released from the bus register202. The process will first be described in connection with a memory attribute which indicates a that strongly-ordered memory access request is ready to be released from the bus register202. In this case, the memory attribute is provided by the first register302to a memory barrier generator306as an enabling signal. At the same time, the data stored in the second register304is provided to the input of the memory barrier generator306. As indicated above, the data stored in the second register304includes data that identifies each memory device, other than the target memory device, that the originating processor is capable of accessing. When the memory barrier generator306is enabled by the memory attribute, this information is used to generate a memory barrier for each memory device identified by the data. Each memory barrier may be provided to the appropriate memory device by issuing a bus command with an attribute identifying the originating processor which initiated the strongly-ordered request. Alternatively, the memory barriers may be provided to the appropriate memory devices using sideband signaling, or by other suitable means.

Logic308in the controller208may be used to monitor feedback from the memory devices for memory barrier acknowledgements. A “memory barrier acknowledgement” is a signal from a memory device indicating that every memory access request received by that memory device from the originating processor issuing the strongly-ordered request that precedes the memory barrier will be computed before the completion of any subsequent memory access request from the originating processor. The data from the second register304is used by the logic308to determine which memory devices need to be monitored for memory barrier acknowledgements. When the logic308determines that all memory barrier acknowledgements have been received, it generates a trigger that is used to release the corresponding memory access request from the bus register202via a first connection211. More specifically, the memory attribute from the first register302is provided to the select input of a multiplexer310. The multiplexer310is used to couple the trigger generated by the logic308to the bus register202when the memory attribute indicates that the memory access request is strongly-ordered. The trigger output from the multiplexer310is also coupled to the decoder to synchronize the timing of the bus switch206via a second connection210(seeFIG. 2).

Once the memory access request is released from the bus register, it is routed to the target memory device through the bus switch206(seeFIG. 2). A second multiplexer312in the controller208may be used to delay the release of data from the first and second registers302,304until a memory access acknowledgement is received from the target memory device when a memory attribute indicating a strongly-ordered request is applied to the select input. As discussed earlier, the memory attribute included in the memory access request enforces an ordering constraint on the target memory device. Namely, the target memory device executes all outstanding memory access requests issued by the originating processor before executing the strongly-ordered memory access request. A memory access acknowledgement is generated by the target memory device following the execution of the strongly-ordered request. The memory access acknowledgement is fed back to the multiplexer312in the controller208, where it is used generate a trigger to release new data from the first and second register302,304corresponding to the next memory access request in the bus register202. If the new data includes a memory attribute indicating that the corresponding memory access request in the bus register202is strongly-ordered, then the same process is repeated. Otherwise, the memory access request can be released immediately from the bus register202.

The controller208is configured to immediately release a memory access request from the bus register202when the corresponding memory attribute in the first register302indicates that the request is weakly-ordered. In that case, the memory attribute is used to disable the memory barrier generator306. In addition, the memory attribute forces the multiplexer310into a state which couples an internally generated trigger to the bus register202to release the memory access request via the first connection211. The memory access request is released from the bus register202and coupled to the target memory device through the bus switch206(seeFIG. 2). The data corresponding to the next memory access request is then released from the first and second registers302,304by an internally generated trigger output from the second multiplexer312in the controller208.

FIG. 4is a functional block diagram illustrating another example of a controller in a bus interconnect for a weakly-ordered processing system. In this example, a strongly-ordered memory access request is released from the bus register202by the controller208at the same time the memory barriers are provided to the appropriate memory devices. More specifically, a memory attribute for a memory access request in the bus register202is provided by the first register302to the memory barrier generator306. If the memory attribute indicates that the corresponding memory access request is strongly-ordered, then the memory barrier generator306is enabled. When the memory barrier generator306is enabled, the data in the second register304is used to generate a memory barrier for each memory device accessible by the originating processor, other than the target memory device.

With the memory barrier generator306enabled, logic314in the controller208may be used to prevent subsequent memory access requests from being released from the bus register202until the strongly-ordered request is executed by the target memory device. A delay316may be used to allow an internally generated trigger to release the strongly-ordered memory request from the bus register202before the trigger is gated off by the memory attribute. In this way, the memory access request can be provided to the target memory device concurrently with the memory barriers for the other memory devices accessible by the originating processor.

Logic318may be used to monitor feedback from the target memory device for the memory access acknowledgement, and the other memory devices accessible by the originating processor for the memory barrier acknowledgements. The data from the second register304is used by the logic318to determine which memory devices need to be monitored for memory barrier acknowledgements. When the logic318determines that the various acknowledgements have been received, it generates a trigger to release new data from the first and second registers302,304corresponding to the next memory access request in the bus register202. The trigger is coupled through a multiplexer320which is forced into the appropriate state by the memory attribute from the first register202. If the new data includes a memory attribute indicating that the corresponding memory access in the bus register202is strongly-ordered, then the same process is repeated. Otherwise, the memory access request can be released immediately from the bus register202with an internally generated trigger via the logic314. An internally generated trigger may also be coupled through the multiplexer320to release the data from the first and second registers302,304for the next memory access request in the bus register202.

The previous description is provided to enable any person skilled in the art to practice the various embodiments described herein. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other embodiments. Thus, the claims are not intended to be limited to the embodiments shown herein, but is to be accorded the full scope consistent with the language claims, wherein reference to an element in the singular is not intended to mean “one and only one” unless specifically so stated, but rather “one or more.” All structural and functional equivalents to the elements of the various embodiments described throughout this disclosure that are known or later come to be known to those of ordinary skill in the art are expressly incorporated herein by reference and are intended to be encompassed by the claims. Moreover, nothing disclosed herein is intended to be dedicated to the public regardless of whether such disclosure is explicitly recited in the claims. No claim element is to be construed under the provisions of 35 U.S.C. §112, sixth paragraph, unless the element is expressly recited using the phrase “means for” or, in the case of a method claim, the element is recited using the phrase “step for.”