Patent Publication Number: US-11030127-B2

Title: Multi-threaded architecture for memory controller data paths

Description:
BACKGROUND 
     The present invention generally relates to integrated circuits, and, more particularly, to a memory controller. 
     Integrated circuits (ICs) often include multiple processing cores for processing data packets and a shared memory for storing the data packets. ICs also include a memory controller that manages communication between the cores and the shared memory. To access the data packets stored in the memory, a core issues an access request that includes a memory address. The memory controller grants access to the core only after an on-going transaction with the memory is completed. Thus, due to contention, the core must wait before it can access the memory. This waiting period is referred to as the dynamic latency of the core. 
     A known technique to reduce dynamic latency involves interleaving of memory addresses, which requires the shared memory to be divided into multiple memory banks. Each memory bank is accessible, independent of other memory banks. Interleaving of addresses involves mapping contiguous addresses to memory locations in separate memory banks. The interleaving scheme may depend on the size of a contiguous address block mapped to each memory bank, for instance, interleaving based on a page size, a cache-line, and an address boundary. The cores generate access requests that include addresses mapped to memory locations present in separate memory banks due to interleaving of the addresses. Thus, address interleaving permits a core to sequentially access separate memory banks. Address interleaving also permits different cores to simultaneously access separate memory banks, leading to a reduction in dynamic latency. However, since only one core can access a memory bank in one access cycle, a memory access conflict arises when multiple cores try to simultaneously access the same memory bank. 
     A known technique to resolve memory access conflicts involves including an arbiter in the memory controller. The memory controller assigns a priority level to each core based on factors such as the core type and the access request type and then provides the cores access to the memory based on the assigned priority levels, thereby establishing a sequence of access for the cores. 
     To ensure fair access to the cores (i.e., to prevent starvation of low-priority access requests), the arbiter can modify the access sequence using arbitration techniques such as rotating priority, round robin, and least recently accessed core. However, these arbitration techniques do not allow a user to dynamically determine the access sequence, and hence, a select logic circuit is included in the arbiter to allow a user to configure the access sequence. However, including such select logic requires redesigning the existing arbiter, which increases the complexity of the circuit and the circuit area. 
     It would be advantageous to have a memory controller that provides multiple cores access to memory with reduced dynamic latency and contention and dynamically determines the access sequence without significantly increasing the complexity of the memory controller and the circuit area. 
    
    
     
       BRIEF DESCRIPTION OF THE DRAWINGS 
       Embodiments of the present invention are illustrated by way of example and are not limited by the accompanying figures, in which like references indicate similar elements. Elements in the figures are illustrated for simplicity and clarity and have not necessarily been drawn to scale. For example, the thicknesses of layers and regions may be exaggerated for clarity. 
         FIG. 1  is a schematic block diagram of an integrated circuit (IC) including a memory controller in accordance with an embodiment of the present invention; 
         FIG. 2  is a schematic block diagram of an arbiter of the memory controller of  FIG. 1  in accordance with an embodiment of the present invention; 
         FIG. 3  is a schematic flow diagram of memory-access processing according to a conventional implementation of the IC of  FIGS. 1 and 2 ; and 
         FIG. 4  is a schematic flow diagram of memory-access processing according to an inventive implementation of the IC of  FIGS. 1 and 2 . 
     
    
    
     DETAILED DESCRIPTION 
     The detailed description of the appended drawings is intended as a description of the currently preferred embodiments of the present invention, and is not intended to represent the only form in which the present invention may be practiced. It is to be understood that the same or equivalent functions may be accomplished by different embodiments that are intended to be encompassed within the spirit and scope of the present invention. 
     As used herein, the singular forms “a”, “an”, and “the”, are intended to include the plural forms as well, unless the context clearly indicates otherwise. It further will be understood that the terms “comprises”, “comprising”, “has”, “having”, “includes”, or “including” specify the presence of stated features, steps, or components, but do not preclude the presence or addition of one or more other features, steps, or components. It also should be noted that, in some alternative implementations, the functions/acts noted might occur out of the order noted in the figures. For example, two figures shown in succession may in fact be executed substantially concurrently or may sometimes be executed in the reverse order, depending upon the functionality/acts involved. The term “or” is to be interpreted as inclusive unless indicated otherwise. 
     In an embodiment of the present invention, an article of manufacture comprises N processing cores, wherein N is an integer greater than 1; M memory banks, wherein M is an integer greater than or equal to 1; and a memory controller configured to enable the N processing cores to access the M memory banks. The memory controller is configured to (i) receive a sequence of one or more access requests from each of one or more of the processing cores, wherein each access request is a request for access by the corresponding processing core to a specified memory bank, and (ii) process access requests from one or more of the processing cores to generate a current sequence of one or more granted access requests to each of one or more of the memory banks. For each processing core, the article comprises first and second buffers configured to alternately store access requests after address decoding. Initially, the request from the first buffer is presented for arbitration, and the result of the arbitration is granted in the same cycle. When this grant is received back after registering in the next cycle, the first buffer is configured to receive a new access request, and the request from the second buffer is presented for arbitration. Similarly, when the grant is received for the second buffer&#39;s request, the second buffer is configured to receive a new access request, and processing is performed to determine whether to grant an access request stored in the first buffer. 
     FIGS. 1 and 2 are from U.S. Patent Application Publication No. US 2017/0168934 A1 (“the &#39;934 application”), the teachings of which are incorporated herein by reference in their entirety. 
       FIG. 1  is a schematic block diagram of an integrated circuit (IC)  100  in accordance with an embodiment of the present invention. The IC  100  includes a plurality of cores  102  including first through fourth cores  102   a - 102   d , a memory controller  104 , and a memory  106 . The memory  106  includes a plurality of memory segments  108  including first and second memory segments  108   a  and  108   b . Each memory segment  108  includes a plurality of memory banks  110 . In an example, the first memory segment  108   a  includes first and second memory banks  110   a  and  110   b , and the second memory segment  108   b  includes third and fourth memory banks  110   c  and  110   d . In one implementation, the memory banks  110  are static random-access memories (SRAMs). In other implementations, the memory banks  110  may be other types of memories such as (without limitation) dynamic random-access memories (DRAMs) and flash memories. 
     Although the IC  100  of  FIG. 1  has four cores  102   a - 102   d  and four memory banks  110   a - 110   d , in other implementations, ICs of the present invention may have other numbers of cores  102  and other numbers of memory banks  110 . In an example, real-world implementation, an IC of the present invention has 64 cores  102  vying for access to 24 memory banks  110 . 
     For each memory segment  108 , the memory controller  104  includes an address decoder  112 , a system bus  114 , and an arbiter  116  corresponding to each memory bank  110  in the memory segment  108 . In an example, the memory controller  104  includes first and second address decoders  112   a  and  112   b , first and second system buses  114   a  and  114   b , and first through fourth arbiters  116   a - 116   d . The first address decoder  112   a  and the first system bus  114   a  correspond to the first memory segment  108   a , and the second address decoder  112   b  and the second system bus  114   b  correspond to the second memory segment  108   b . The first through fourth arbiters  116   a - 116   d  are connected to the corresponding first through fourth memory banks  110   a - 110   d , respectively, by way of an interface  118 . 
     The first through fourth cores  102   a - 102   d  generate first through fourth access requests AR 1 -AR 4 , respectively. The first through fourth access requests AR 1 -AR 4  include first through fourth sets of address least significant bits (LSBs), respectively. 
     The first and second address decoders  112   a  and  112   b  are connected to the first through fourth cores  102   a - 102   d  to receive the first through fourth access requests AR 1 -AR 4 . The first address decoder  112   a  identifies each of the first and second memory banks  110   a  and  110   b  in the first memory segment  108   a  based on the first through fourth sets of address LSBs. The second address decoder  112   b  identifies each of the third and fourth memory banks  110   c  and  110   d  in the second memory segment  108   b  based on the first through fourth sets of address LSBs. 
     The first and second system buses  114   a  and  114   b  are connected to the first and second address decoders  112   a  and  112   b , respectively. Each system bus  114  may receive and route one or more of the first through fourth access requests AR 1 -AR 4  to the corresponding memory banks  110  when the corresponding memory banks  110  are available for a memory access. Each system bus  114  stalls its access requests to the corresponding memory banks  110  when the corresponding memory banks  110  are unavailable for a memory access. Each system bus  114  routes multiple access requests to a single memory bank  110  in a sequence. This sequence indicates an order of predetermined priority levels of the cores  102  corresponding to the access requests. Each system bus  114  maintains an (N×M) matrix that stores and routes access requests by N cores  102  to M corresponding memory banks  110 . In the example embodiment of  FIG. 1 , each system bus  114  maintains a (4×2) matrix that stores and routes access requests by the four cores  102   a - 102   d  to two corresponding memory banks, i.e., memory banks  110   a  and  110   b  for system bus  114   a  and memory banks  110   c  and  110   d  for system bus  114   b.    
     The first arbiter  116   a  is connected to the first system bus  114   a  and is configured to modify the access sequence to avoid starvation of the cores  102 . When one or more of the first through fourth cores  102   a - 102   d  request access to the first memory bank  110   a , the first arbiter  116   a  provides one of those cores  102  access to the first memory bank  110   a  in an access cycle. The second arbiter  116   b  is similarly connected to the first system bus  114   a , and the third and fourth arbiters  116   c  and  116   d  are similarly connected to the second system bus  114   b.    
       FIG. 2  is a schematic block diagram of each arbiter  116  of  FIG. 1 . The arbiter  116  includes first and second sets  202   a  and  202   b  of muxes  206 , a priority encoder  204 , and a grant signal generator  208 . The first set  202   a  includes first through fourth 2:1 muxes  206   a - 206   d , and the second set  202   b  includes fifth through eighth 2:1 muxes  206   e - 206   h . As understood by those skilled in the art, the number of sets  202  of muxes  206  and the number of muxes  206  in each set  202  depend on the number of cores  102  handled by the arbiter  116 . The sets  202  of muxes  206  correspond to and perform the combinatorial logic of the arbiter  116  based on the selection signals SEL 1  and SEL 2  applied to the muxes  206 . 
     The arbiter  116  receives one or more of the first through fourth access requests AR 1 -AR 4  from the corresponding system bus  114  of  FIG. 1 . The operations of the combinatorial logic of the arbiter  116  are described in detail in the &#39;934 application and are not repeated here. 
     The priority encoder  204  compares the priority levels between different requests generated by different cores for the same memory bank and selects the request with the highest priority for grant generation. Priority is assigned for different cores beforehand. 
     The grant generator  208  is connected to the output terminal of the priority encoder  204  and receives one of the first through fourth access requests AR 1 -AR 4 . The grant generator  208  generates a grant signal (GS) at a first logic state (e.g., logic 1) when the corresponding memory bank  110  is available for memory access and at a second logic state (e.g., logic 0) when the corresponding memory bank  110  is unavailable for memory access. The corresponding memory bank  110  is unavailable for memory access in an access cycle when the arbiter  116  provides one of the first through fourth cores  102   a - 102   d  access to the corresponding memory bank  110  during that access cycle. The corresponding address decoder  112  receives the grant signal (GS) and determines whether the corresponding memory bank  110  is available for memory access. 
     Based on the operations of each arbiter  116 , the memory controller  104  of  FIG. 1  implements an interleaving and arbitration scheme and resolves any contention among the first through fourth cores  102   a - 102   d  for access to each memory bank  110  by determining a corresponding sequence of access for the first through fourth cores  102   a - 102   d . The memory controller  104  can modify the sequence of the predetermined priority levels of the first through fourth cores  102   a - 102   d  to avoid starvation of cores  102  having lower priority levels. In this way, the memory controller  104  achieves a high bandwidth of data transfer in each access cycle. Further, the memory controller  104  can handle wide access requests by providing a core  102  access to adjacent memory banks  110  in a single access cycle. 
       FIG. 3  is a schematic flow diagram of memory-access processing according to a conventional implementation of an IC analogous to the IC  100  of  FIGS. 1 and 2  having N cores  102   1 - 102   N  analogous to the cores  102   a - 102   d  of  FIG. 1  and M memory banks  110   1 - 110   M  (not shown in  FIG. 3 ) analogous to the memory banks  110   a - 110   d  of  FIG. 1 , where N is a positive integer greater than one and M is a positive integer. As shown in  FIG. 3 , each core  102   i  has a corresponding address FIFO (first in, first out) register  103   i  that stores a set of addresses in the memory banks  110  to be sequentially accessed by that core  102   i . For each core  102   i , the current address output from the corresponding address FIFO  103   i  is decoded  112   i  and a corresponding current access request is generated  113   i . 
     If any cores  102  have current access requests for the first memory bank  110   1 , then those access requests are provided to the first priority mirroring stage  116 A 1  of the first arbiter  116   1 . If any cores  102  have current access requests for the second memory bank  110   2 , then those access requests are provided to the second priority mirroring stage  116 A 2  of the second arbiter  116   2  (not explicitly shown in  FIG. 3 ). And so on such that, if any cores  102  have current access requests for the Mth memory bank  110   M , those access requests are provided to the Mth priority mirroring stage  116 A M  of the Mth arbiter  116   M . 
     Each priority mirroring stage  116 A j  implements combinatorial logic analogous to the combinatorial logic of  FIG. 2  to prioritize the different access requests received from the different cores  102  for the corresponding memory bank  110   j . Note that the design of the combinatorial logic will depend on the number M of different memory banks  110 . Each set of prioritized access requests is then provided to a corresponding bank arbitration stage  116 B j  of the jth arbiter  116   j . The bank arbitration stage  116 B j  returns the grant for the highest priority access request. Collectively, the M bank arbitration stages  116 B output up to M granted access requests in each access cycle, one for each memory bank  110   j  receiving an access request. 
     In step  120 , each core  102   i  that had an access request granted is informed of that fact so that that core  102   i  will then process the next access request in its address FIFO  103   i . In step  122 , each granted access request is performed using a suitable procedure as understood by those skilled in the art. 
     One goal is to ensure that all of the memory-access processing of  FIG. 3  for the current set of up to N access requests from the N cores  102   1 - 102   N  are performed in a single clock cycle to make sure that there are no gaps in between consecutive accesses to any single memory bank  110   j  so that, in the next cycle, another set of up to N access requests can be processed in order to maintain optimal bandwidth utilization. 
     As described previously,  FIG. 2  shows the combinatorial logic of each arbiter  116  in the IC  100  of  FIG. 1  having only four cores  102   a - 102   d , where the IC  100  has four respective arbiters  116   a - 116   d  for the four different memory banks  110   a - 110   d . As described previously, in some real-world implementations, an IC can have many more than four cores and many more than four memory banks. For example, a typical IC may have N=64 different cores  102   1 - 102   64  and M=24 different memory banks  110   1 - 110   24 . Such an IC will have 24 separate arbiters  116   1 - 116   24 , each having combinatorial logic that is larger and more complex than the combinatorial logic of  FIG. 2 . In an IC having 24 separate, large, and complex arbiters  116   1 - 116   24 , the arbiter&#39;s combinatorial logic causes congestion and routing delays at the physical design level, thus limiting the frequency of operation. 
     One way to address the timing problems associated with ICs having many cores  102  accessing many memory banks  110  is to add pipelining step between steps  120  and  122 , where the granted access requests processed in step  120  are stored in (e.g., flip-flop) registers during one clock cycle and then processed in step  122  during the next clock cycle. 
     Assume, for example, a scenario in which the first core  102   1  generates three respective access requests a1_1, a2_1, a3_1 for the first memory bank  110   1  during the first three clock cycles 1, 2, and 3 and that the second core  102   2  generates three respective access requests b1_1, b2_1, b3_1 for that same first memory bank  110   1  during those same first three clock cycles 1, 2, and 3. In that case, the first and second cores  102   1  and  102   2  are contending for the same first memory bank  110   1 , but each memory bank  110   j  can be accessed by only one core  102   i  at a time. 
     Note that, depending on the particular situation, each of the other cores  102   3 - 102   N  may also be generating an access request for any one of the memory banks  110  during each of these same clock cycles. For this particular example, it is assumed that only the first and second cores  102   1  and  102   2  generate access requests for the first memory bank  110   1  during the clock cycles 1, 2, and 3, and that no access requests are generated for the first memory bank  110   1  by any core  102  during the next three clock cycles 4, 5, and 6. 
     Assume further that the first core  102   1  has higher priority than the second core  102   2 , such that all of the pending access requests for the first core  102   1  should be performed prior to any of the pending access requests for the second core  102   2 . In that case, the IC should be configured to perform the following sequence of access events for the first memory bank  110   1 :
         Clock cycle 1: Perform access request a1_1 for the first core  102   1 ;   Clock cycle 2: Perform access request a2_1 for the first core  102   1 ;   Clock cycle 3: Perform access request a3_1 for the first core  102   1 ;   Clock cycle 4: Perform access request b1_1 for the second core  102   2 ;   Clock cycle 5: Perform access request b2_1 for the second core  102   2 ; and   Clock cycle 6: Perform access request b3_1 for the second core  102   2 .       

     However, the pipelining added to the memory-access processing of  FIG. 3  results in a gap between two consecutive accesses by a particular core  102   i  to a particular memory bank  110   j . As a result, for the above example scenario, the IC will perform the following, improper sequence of access events for the first memory bank  110   1 :
         Clock cycle 1: Perform access request a1_1 for the first core  102   1 ;   Clock cycle 2: Perform access request b1_1 for the second core  102   2 ;   Clock cycle 3: Perform access request a2_1 for the first core  102   1 ;   Clock cycle 4: Perform access request b2_1 for the second core  102   2 ; and   Clock cycle 5: Perform access request a3_1 for the first core  102   1 ;   Clock cycle 6: Perform access request b3_1 for the second core  102   2 .       

       FIG. 4  is a schematic flow diagram of memory-access processing according to an inventive implementation of the IC of  FIGS. 1 and 2  having N cores  102   1 - 102   N  similar to the N cores  102   1 - 102   N  of  FIG. 3  and M memory banks  110   1 - 110   M  (not shown in  FIG. 4 ) similar to the M memory banks  110   1 - 110   M  of  FIG. 3 . The processing of  FIG. 4  is identical to the processing of  FIG. 3  except for the inclusion of a parallel address processing stage  400   i  for each core  102   i  and the addition of a register-based pipelining step  121  between steps  120  and  122 . 
     As shown in  FIG. 4 , each parallel address processing stage  400   i  has two buffers  402   1  and  402   2  and two corresponding masks  404   1  and  404   2  in addition to a request generator  113   i  that is similar to the request generator  113   i  of  FIG. 3 . When a mask  404   k  is enabled, an address stored in the corresponding buffer  402   k  is prevented from being propagated towards the request generator  113   i . When a mask  404   k  is disabled, an address stored in the corresponding buffer  402   k  is propagated towards the request generator  113   i , overwriting whatever request was being generated previously. 
     When the IC is initially powered up, both masks  404   1  and  404   2  in all of the parallel address processing stages  400   1 - 400   N  are enabled. When the first core  102   1  (eventually) decodes ( 112   1 ) its first access request from its address FIFO  103   1 , the first core&#39;s parallel address processing stage  400   1  stores the resulting first decoded address in its first buffer  402   1  and disables its first mask  404   1 , thereby allowing the first decoded address to be propagated from the first buffer  402   1  into the first core&#39;s request generator  113   1 . 
     When the first core  102   1  (eventually) decodes ( 112   1 ) its second access request from its address FIFO  103   1 , assuming that the access request associated with that first decoded address has not yet been granted, the first core&#39;s parallel address processing stage  400   1  stores the resulting second decoded address in its second buffer  402   2  but keeps its second mask  404   2  enabled, thereby preventing the second decoded address from being propagated from the second buffer  402   2  into the first core&#39;s request generator  113   1 . 
     When the access request associated with the first core&#39;s first decoded address and generated by the request generator  113   1  is eventually granted by the first arbiter  116   1  in step  120 , the pipelining step  121  will inform the first core  102   1  of that event in the next clock cycle. In response, the first core&#39;s parallel address processing stage  400   1  enables its first mask  404   1  and disables its second mask  404   2 , thereby allowing the first core&#39;s second decoded address to be propagated from its second buffer  402   2  into its request generator  113   1 . In addition, if and when available, the first core&#39;s parallel address processing stage  400   1  stores the first core&#39;s third decoded address into its first buffer  402   1 . 
     When the access request associated with the first core&#39;s second decoded address is eventually granted, the pipelining step  121  will inform the first core  102   1  of that event. In response, the first core&#39;s parallel address processing stage  400   1  disables its first mask  404   1  and enables its second mask  404   2 , thereby allowing the first core&#39;s third decoded address (if available) to be propagated from its first buffer  402   1  into its request generator  113   1 . In addition, if and when available, the first core&#39;s parallel address processing stage  400   1  stores the first core&#39;s fourth decoded address into its second buffer  402   2 . 
     The first core&#39;s parallel address processing stage  400   1  continues this alternating processing between its two buffers  402   1  and  402   2  for all of the subsequently generated access requests by the first core  102   1  as the first core&#39;s access requests are sequentially granted. 
     The (N−1) other parallel address processing stages  400   2 - 400   N  of  FIG. 4  perform analogous, respective alternating processing for the access requests by the (N−1) other cores  102   2 - 102   N . 
     The following description corresponds to the previous example scenario in which the first core  102   1  generates three respective access requests a1_1, a2_1, a3_1 for the first memory bank  110   1  during the first three clock cycles 1, 2, and 3 and the second core  102   2  generates three respective access requests b1_1, b2_1, b3_1 for that same first memory bank  110   1  during those same first three clock cycles 1, 2, and 3, where the first core  102   1  has higher priority than the second core  102   2 . 
     For this scenario, assume that, before clock cycle 1, the decoded address for the first access request a1_1 from the first core  102   1  was stored in the first core&#39;s first buffer  402   1 , the decoded address for the second access request a2_1 from the first core  102   1  was stored in the first core&#39;s second buffer  402   2 , and the address for the third access request a3_1 from the first core  102   1  was stored in the first core&#39;s address FIFO  103   1 . Similarly, before clock cycle 1, the decoded address for the first access request b1_1 from the second core  102   2  was stored in the second core&#39;s first buffer  402   1 , the decoded address for the second access request b2_1 from the second core  102   2  was stored in the second core&#39;s second buffer  402   2 , and the address for the third access request b3_1 from the second core  102   1  was stored in the second core&#39;s address FIFO  103   2 . 
     In that case, in clock cycle 1, the first core&#39;s first mask  404   1  is disabled, and the first core&#39;s second mask  404   2  is enabled, such that the decoded address for the first core&#39;s first access request a1_1 is propagated towards the first core&#39;s request generator  113   1 , which generates the first core&#39;s first access request a1_1. Similarly, in clock cycle 1, the second core&#39;s first mask  404   1  is disabled, and the second core&#39;s second mask  404   2  is enabled, such that the decoded address for the second core&#39;s first access request b 1 _1 is propagated towards the second core&#39;s request generator  113   2 , which generates the second core&#39;s first access request b1_1. As such, in clock cycle 1, the first arbiter  116   1  (i) receives the first core&#39;s first access request a1_1 from the first core&#39;s request generator  113   1  and the second core&#39;s first access request b1_1 from the second core&#39;s request generator  113   2  and (ii) selects the first core&#39;s first access request a1_1 for the grant generation step  120  because the first core  102   1  has higher priority than the second core  102   2 . 
     The pipelining step  121  will inform the first core  102   1  of that event in the next clock cycle 2. In response, the first core&#39;s parallel address processing stage  400   1  enables its first mask  404   1  and disables its second mask  404   2 , thereby allowing the decoded address for the first core&#39;s second access request a2_1 to be propagated from its second buffer  402   2  into the first core&#39;s request generator  113   1 , which generates the first core&#39;s second access request a2_1. In addition, the first core&#39;s parallel address processing stage  400   1  stores the decoded address for the first core&#39;s third access request a3_1 into its first buffer  402   1 . Note that, during clock cycle 2, the state of the second core&#39;s parallel address processing stage  400   2  does not change. 
     In clock cycle 2, the first arbiter  116   1  (i) receives the first core&#39;s second access request a2_1 from the first core&#39;s request generator  113   1  and the second core&#39;s first access request b1_1 from the second core&#39;s request generator  113   2  and (ii) selects the first core&#39;s second access request a2_1 for the grant generation step  120  because the first core  102   1  has higher priority than the second core  102   2 . 
     The pipelining step  121  will inform the first core  102   1  of that event in the next clock cycle 3. In response, the first core&#39;s parallel address processing stage  400   1  disables its first mask  404   1  and enables its second mask  404   2 , thereby allowing the decoded address for the first core&#39;s third access request a3_1 to be propagated from its first buffer  402   1  into the first core&#39;s request generator  113   1 , which generates the first core&#39;s third access request a3_1. In this example, there are no other addresses in the first core&#39;s address FIFO  103   1 . Note that, during clock cycle 3, the state of the second core&#39;s parallel address processing stage  400   2  does not change. 
     In clock cycle 3, the first arbiter  116   1  (i) receives the first core&#39;s third access request a3_1 from the first core&#39;s request generator  113   1  and the second core&#39;s first access request b1_1 from the second core&#39;s request generator  113   2  and (ii) selects the first core&#39;s third access request a3_1 for the grant generation step  120  because the first core  102   1  has higher priority than the second core  102   2 . 
     The pipelining step  121  will inform the first core  102   1  of that event in the next clock cycle 4. Note that the first core&#39;s parallel address processing stage  400   1  has logic (not shown in  FIG. 4 ) that determines whether the decoded addresses stored in the first core&#39;s first and second buffers  402   1  and  402   2  have not yet been serviced. In this case, since the decoded addresses stored in those buffers  402   1  and  402   2  have both been serviced, the logic will enable both masks  404   1  and  404   2 , thereby preventing any decoded address from being propagated towards the first core&#39;s request generator  113   1 . 
     As such, in clock cycle 4, the first arbiter  116   1  (i) receives only the second core&#39;s first access request b1_1 from the second core&#39;s request generator  113   2  and (ii) selects the second core&#39;s first access request b1_1 for the grant generation step  120  because it is the only pending access request for the first memory bank  110   1 . 
     The pipelining step  121  will inform the second core  102   2  of that event in the next clock cycle 5. In response, the second core&#39;s parallel address processing stage  400   2  enables its first mask  404   1  and disables its second mask  404   2 , thereby allowing the decoded address for the second core&#39;s second access request b2_1 to be propagated from its second buffer  402   2  into the second core&#39;s request generator  113   2 , which generates the second core&#39;s second access request b2_1. In addition, the second core&#39;s parallel processing stage  400   2  stores the decoded address for the second core&#39;s third access request b3_1 into its first buffer  402   1 . Note that, during clock cycle 5, the state of the first core&#39;s parallel address processing stage  400   1  does not change. 
     In clock cycle 5, the first arbiter  116   1  (i) receives only the second core&#39;s second access request b2_1 from the second core&#39;s request generator  113   2  and (ii) selects the second core&#39;s second access request b2_1 for the grant generation step  120  because it is the only pending access request for the first memory bank  110   1 . 
     The pipelining step  121  will inform the second core  102   2  of that event in the next clock cycle 6. In response, the second core&#39;s parallel address processing stage  400   2  disables its first mask  404   1  and enables its second mask  404   2 , thereby allowing the decoded address for the second core&#39;s third access request b3_1 to be propagated from its second buffer  402   2  into the second core&#39;s request generator  113   2 , which generates the second core&#39;s third access request b3_1. In this example, there are no other addresses in the second core&#39;s address FIFO  103   2 . Note that, during clock cycle 6, the state of the first core&#39;s parallel address processing stage  400   1  does not change. 
     In clock cycle 6, the first arbiter  116   1  (i) receives only the second core&#39;s third access request b3_1 from the second core&#39;s request generator  113   2  and (ii) selects the second core&#39;s third access request b3_1 for the grant generation step  120  because it is the only pending access request for the first memory bank  110   1 . 
     The pipelining step  121  will inform the second core  102   2  of that event in the next clock cycle. Note that the second core&#39;s parallel address processing stage  400   2  also has logic (not shown in  FIG. 4 ) that determines whether the decoded addresses stored in the second core&#39;s first and second buffers  402   1  and  402   2  have not yet been serviced. In this case, since the decoded addresses stored in those buffers  402   1  and  402   2  have both been serviced, the logic will enable both masks  404   1  and  404   2 , thereby preventing any decoded address from being propagated towards the second core&#39;s request generator  113   2 . 
     In this way, the memory-access processing of  FIG. 4  provides the following desired sequence of memory-access events for the first memory bank  110   1 :
         Clock cycle 1: Perform access request a1_1 for the first core  102   1 ;   Clock cycle 2: Perform access request a2_1 for the first core  102   1 ;   Clock cycle 3: Perform access request a3_1 for the first core  102   1 ;   Clock cycle 4: Perform access request b1_1 for the second core  102   2 ;   Clock cycle 5: Perform access request b2_1 for the second core  102   2 ; and   Clock cycle 6: Perform access request b3_1 for the second core  102   2 .       

     As such, the memory-access processing of  FIG. 4  provides the desired sequence of memory-access events without inducing any bubble/gap between two consecutive accesses to the same memory bank, thereby achieving optimal bandwidth utilization while properly applying the relative priority levels between the different cores  102  and the added pipelining step  121  will reduce the timing congestion at the physical layer. 
     The invention has been described in the context of a simple example in which only two cores  102  are vying for the same memory bank  110 . Those skilled in the art will understand how the invention would work when more than two cores  102  vie for the same memory bank  110 . 
     Although the invention has been described in the context of alternating processing implemented using buffers  402  and masks  404 , in other implementations, other suitable mechanisms can be employed to achieve analogous alternating processing. 
     Although the invention has been described in the context of arbitration techniques based on programmable priority mirroring, the invention can also be implemented in the context of other arbitration techniques, such as fixed priority schemes or adaptive priority schemes such as (without limitation) rotating priority, round robin, and least recently accessed core. 
     While various embodiments of the present invention have been illustrated and described, it will be clear that the present invention is not limited to these embodiments only. Numerous modifications, changes, variations, substitutions, and equivalents will be apparent to those skilled in the art, without departing from the spirit and scope of the present invention, as described in the claims. 
     Although the invention is described herein with reference to specific embodiments, various modifications and changes can be made without departing from the scope of the present invention as set forth in the claims below. Accordingly, the specification and figures are to be regarded in an illustrative rather than a restrictive sense, and all such modifications are intended to be included within the scope of the present invention. Any benefits, advantages, or solutions to problems that are described herein with regard to specific embodiments are not intended to be construed as a critical, required, or essential feature or element of any or all the claims. 
     It should be understood that the steps of the exemplary methods set forth herein are not necessarily required to be performed in the order described, and the order of the steps of such methods should be understood to be merely exemplary. Likewise, additional steps may be included in such methods, and certain steps may be omitted or combined, in methods consistent with various embodiments of the invention. 
     Although the elements in the following method claims, if any, are recited in a particular sequence with corresponding labeling, unless the claim recitations otherwise imply a particular sequence for implementing some or all of those elements, those elements are not necessarily intended to be limited to being implemented in that particular sequence. 
     Reference herein to “one embodiment” or “an embodiment” means that a particular feature, structure, or characteristic described in connection with the embodiment can be included in at least one embodiment of the invention. The appearances of the phrase “in one embodiment” in various places in the specification are not necessarily all referring to the same embodiment, nor are separate or alternative embodiments necessarily mutually exclusive of other embodiments. The same applies to the term “implementation.” 
     While various embodiments of the present invention have been illustrated and described, it will be clear that the present invention is not limited to these embodiments only. Numerous modifications, changes, variations, substitutions, and equivalents will be apparent to those skilled in the art, without departing from the spirit and scope of the present invention, as described in the claims.