Patent Description:
Computer systems typically use inexpensive and high density dynamic random access memory (DRAM) chips for main memory. Most DRAM chips sold today are compatible with various double data rate (DDR) DRAM standards promulgated by the Joint Electron Devices Engineering Council (JEDEC). DDR DRAMs use conventional DRAM memory cell arrays with high-speed access circuits to achieve high transfer rates and to improve the utilization of the memory bus.

A typical DDR memory controller maintains a queue to store pending read and write requests to allow the memory controller to pick the pending requests out of order and thereby to increase efficiency. For example, the memory controller can retrieve multiple memory access requests to the same row in a given rank of memory (referred to as "page hits") from the queue out of order and issue them consecutively to the memory system to avoid the overhead of precharging the current row and activating another row repeatedly. However, scanning and picking accesses from a deep queue while taking advantage of the bus bandwidth available with modern memory technologies such as DDR5 has become difficult to achieve with known memory controllers. Memory controllers may employ techniques such as creating streaks of read commands or write commands to improve bus efficiency. However, such techniques come with performance trade-offs, such as latency concerns that arise from delaying commands not part of the current streak, and additional performance overhead associated with "turning around" the command bus from a read streak to a write streak, and vice versa.

<CIT> discloses a memory controller that maintains separate read and write command queues, and calculates costs of switching between read and write modes based on the latency costs compared with bus turnaround costs. <CIT>discloses a memory controller and method that determines an end of a read mode during which read requests are sent to the memory device, switches from the read mode to the write mode, determines how many writes are currently pending, and continues the write burst until that number of writes have been sent before turning the bus back around to the read mode. <CIT>discloses a memory controller and method that group memory requests into multiple memory rank queues, schedule at least a minimum burst number of memory requests within one of the memory rank queues, and when the burst number has been reached in the one of the plurality of memory rank queues, and if a memory request exceeds an aging threshold, then that memory request will be serviced. <CIT> discloses a method for optimizing DRAM bus switching between reads and writes using a last level cache (LLC) by keeping track of a credit value remaining in a particular direction (read or write) before the DRAM bus switches to the opposite direction when the counter reaches zero, in which the credit value is decremented when a transaction is sent or upon the occurrence of other conditions.

In the following description, the use of the same reference numerals in different drawings indicates similar or identical items. Unless otherwise noted, the word "coupled" and its associated verb forms include both direct connection and indirect electrical connection by means known in the art, and unless otherwise noted any description of direct connection implies alternate embodiments using suitable forms of indirect electrical connection as well.

A memory controller includes a command queue having a first input for receiving memory access requests, and a memory interface queue having an output for coupling to a memory channel adapted for connecting to at least one dynamic random access memory (DRAM). An arbiter is connected to the command queue for selecting entries from the command queue, and placing them in the memory interface queue causing them to be transmitted over the memory channel. The arbiter is operable to transact streaks of consecutive read commands and streaks of consecutive write commands over the memory channel. The arbiter is operable to transact a streak for at least a minimum burst length based on a number of commands of a designated type available to be selected by the arbiter. Following the minimum burst length, the arbiter is operable to decide to start a new streak of commands of a different type based on a first set of one or more conditions indicating intra-burst efficiency.

A method includes causing streaks of consecutive read commands and streaks of consecutive write commands to be transacted over a memory channel. The method includes transacting a streak for at least a minimum burst length based on a number of commands of a designated type available to be selected by the arbiter. Following the minimum burst length, the method includes deciding to start a new streak of commands of a different type based on a first set of one or more conditions indicating intra-burst efficiency.

A data processing system includes a central processing unit, a data fabric connected to the central processing unit, and a memory controller connected to the data fabric for fulfilling memory requests from the central processing unit. The memory controller includes a command queue having a first input for receiving memory access requests, and a memory interface queue having an output for coupling to a memory channel adapted for connecting to at least one DRAM. An arbiter is connected to the command queue for selecting entries from the command queue, and placing them in the memory interface queue causing them to be transmitted over the memory channel. The arbiter is operable to transact streaks of consecutive read commands and streaks of consecutive write commands over the memory channel. The arbiter is operable to transact a streak for at least a minimum burst length based on a number of commands of a designated type available to be selected by the arbiter. Following the minimum burst length, the arbiter is operable to decide to start a new streak of commands of a different type based on a first set of one or more conditions indicating intra-burst efficiency.

<FIG> illustrates in block diagram form an accelerated processing unit (APU) <NUM> and memory system <NUM> known in the prior art. APU <NUM> is an integrated circuit suitable for use as a processor in a host data processing system, and includes generally a central processing unit (CPU) core complex <NUM>, a graphics core <NUM>, a set of display engines <NUM>, a memory management hub <NUM>, a data fabric <NUM>, a set of peripheral controllers <NUM>, a set of peripheral bus controllers <NUM>, and a system management unit (SMU) <NUM>.

CPU core complex <NUM> includes a CPU core <NUM> and a CPU core <NUM>. In this example, CPU core complex <NUM> includes two CPU cores, but in other embodiments CPU core complex <NUM> can include an arbitrary number of CPU cores. Each of CPU cores <NUM> and <NUM> is bidirectionally connected to a system management network (SMN), which forms a control fabric, and to data fabric <NUM>, and is capable of providing memory access requests to data fabric <NUM>. Each of CPU cores <NUM> and <NUM> may be unitary cores, or may further be a core complex with two or more unitary cores sharing certain resources such as caches.

Graphics core <NUM> is a high performance graphics processing unit (GPU) capable of performing graphics operations such as vertex processing, fragment processing, shading, texture blending, and the like in a highly integrated and parallel fashion. Graphics core <NUM> is bidirectionally connected to the SMN and to data fabric <NUM>, and is capable of providing memory access requests to data fabric <NUM>. In this regard, APU <NUM> may either support a unified memory architecture in which CPU core complex <NUM> and graphics core <NUM> share the same memory space, or a memory architecture in which CPU core complex <NUM> and graphics core <NUM> share a portion of the memory space, while graphics core <NUM> also uses a private graphics memory not accessible by CPU core complex <NUM>.

Display engines <NUM> render and rasterize objects generated by graphics core <NUM> for display on a monitor. Graphics core <NUM> and display engines <NUM> are bidirectionally connected to a common memory management hub <NUM> for uniform translation into appropriate addresses in memory system <NUM>, and memory management hub <NUM> is bidirectionally connected to data fabric <NUM> for generating such memory accesses and receiving read data returned from the memory system.

Data fabric <NUM> includes a crossbar switch for routing memory access requests and memory responses between any memory accessing agent and memory management hub <NUM>. It also includes a system memory map, defined by basic input/output system (BIOS), for determining destinations of memory accesses based on the system configuration, as well as buffers for each virtual connection.

Peripheral controllers <NUM> include a universal serial bus (USB) controller <NUM> and a Serial Advanced Technology Attachment (SATA) interface controller <NUM>, each of which is bidirectionally connected to a system hub <NUM> and to the SMN bus. These two controllers are merely exemplary of peripheral controllers that may be used in APU <NUM>.

Peripheral bus controllers <NUM> include a system controller or "Southbridge" (SB) <NUM> and a Peripheral Component Interconnect Express (PCIe) controller <NUM>, each of which is bidirectionally connected to an input/output (I/O) hub <NUM> and to the SMN bus. I/O hub <NUM> is also bidirectionally connected to system hub <NUM> and to data fabric <NUM>. Thus for example a CPU core can program registers in USB controller <NUM>, SATA interface controller <NUM>, SB <NUM>, or PCIe controller <NUM> through accesses that data fabric <NUM> routes through I/O hub <NUM>. Software and firmware for APU <NUM> are stored in a system data drive or system BIOS memory (not shown) which can be any of a variety of non-volatile memory types, such as read-only memory (ROM), flash electrically erasable programmable ROM (EEPROM), and the like. Typically, the BIOS memory is accessed through the PCIe bus, and the system data drive through the SATA interface.

SMU <NUM> is a local controller that controls the operation of the resources on APU <NUM> and synchronizes communication among them. SMU <NUM> manages power-up sequencing of the various processors on APU <NUM> and controls multiple off-chip devices via reset, enable and other signals. SMU <NUM> includes one or more clock sources (not shown), such as a phase locked loop (PLL), to provide clock signals for each of the components of APU <NUM>. SMU <NUM> also manages power for the various processors and other functional blocks, and may receive measured power consumption values from CPU cores <NUM> and <NUM> and graphics core <NUM> to determine appropriate power states.

Memory management hub <NUM> and its associated physical interfaces (PHYs) <NUM> and <NUM> are integrated with APU <NUM> in this embodiment. Memory management hub <NUM> includes memory channels <NUM> and <NUM> and a power engine <NUM>. Memory channel <NUM> includes a host interface <NUM>, a memory channel controller <NUM>, and a physical interface <NUM>. Host interface <NUM> bidirectionally connects memory channel controller <NUM> to data fabric <NUM> over a serial presence detect link (SDP). Physical interface <NUM> bidirectionally connects memory channel controller <NUM> to PHY <NUM>, and in the exemplary embodiment conforms to the DDR PHY Interface (DFI) Specification. Memory channel <NUM> includes a host interface <NUM>, a memory channel controller <NUM>, and a physical interface <NUM>. Host interface <NUM> bidirectionally connects memory channel controller <NUM> to data fabric <NUM> over another SDP. Physical interface <NUM> bidirectionally connects memory channel controller <NUM> to PHY <NUM>, and conforms to the DFI Specification. Power engine <NUM> is bidirectionally connected to SMU <NUM> over the SMN bus, to PHYs <NUM> and <NUM> over the APB, and is also bidirectionally connected to memory channel controllers <NUM> and <NUM>. PHY <NUM> has a bidirectional connection to memory channel <NUM>. PHY <NUM> has a bidirectional connection memory channel <NUM>.

Memory management hub <NUM> is an instantiation of a memory controller having two memory channel controllers and uses a shared power engine <NUM> to control operation of both memory channel controller <NUM> and memory channel controller <NUM> in a manner that will be described further below. Each of memory channels <NUM> and <NUM> can connect to state-of-the-art DDR memories such as DDR version five (DDR5), DDR version four (DDR4), low power DDR4 (LPDDR4), graphics DDR version five (GDDR5), and high bandwidth memory (HBM), and can be adapted for future memory technologies. These memories provide high bus bandwidth and high speed operation. At the same time, they also provide low power modes to save power for battery-powered applications such as laptop computers, and also provide built-in thermal monitoring.

Memory system <NUM> includes a memory channel <NUM> and a memory channel <NUM>. Memory channel <NUM> includes a set of dual inline memory modules (DIMMs) connected to a DDRx bus <NUM>, including representative DIMMs <NUM>, <NUM>, and <NUM> that in this example correspond to separate ranks. Likewise, memory channel <NUM> includes a set of DIMMs connected to a DDRx bus <NUM>, including representative DIMMs <NUM>, <NUM>, and <NUM>.

APU <NUM> operates as the central processing unit (CPU) of a host data processing system and provides various buses and interfaces useful in modern computer systems. These interfaces include two double data rate (DDRx) memory channels, a PCIe root complex for connection to a PCIe link, a USB controller for connection to a USB network, and an interface to a SATA mass storage device.

APU <NUM> also implements various system monitoring and power saving functions. In particular one system monitoring function is thermal monitoring. For example, if APU <NUM> becomes hot, then SMU <NUM> can reduce the frequency and voltage of CPU cores <NUM> and <NUM> and/or graphics core <NUM>. If APU <NUM> becomes too hot, then it can be shut down entirely. Thermal events can also be received from external sensors by SMU <NUM> via the SMN bus, and SMU <NUM> can reduce the clock frequency and/or power supply voltage in response.

<FIG> illustrates in block diagram form a memory controller <NUM> that is suitable for use in an APU like that of <FIG>. Memory controller <NUM> includes generally a memory channel controller <NUM> and a power controller <NUM>. Memory channel controller <NUM> includes generally an interface <NUM>, a memory interface queue <NUM>, a command queue <NUM>, an address generator <NUM>, a content addressable memory (CAM) <NUM>, replay control logic <NUM> including a replay queue <NUM>, a refresh control logic block <NUM>, a timing block <NUM>, a page table <NUM>, an arbiter <NUM>, an error correction code (ECC) check circuit <NUM>, an ECC generation block <NUM>, a data buffer <NUM>, and refresh logic <NUM>.

Interface <NUM> has a first bidirectional connection to data fabric <NUM> over an external bus, and has an output. In memory controller <NUM>, this external bus is compatible with the advanced extensible interface version four specified by ARM Holdings, PLC of Cambridge, England, known as "AXI4", but can be other types of interfaces in other embodiments. Interface <NUM> translates memory access requests from a first clock domain known as the FCLK (or MEMCLK) domain to a second clock domain internal to memory controller <NUM> known as the UCLK domain. Similarly, memory interface queue <NUM> provides memory accesses from the UCLK domain to a DFICLK domain associated with the DFI interface.

Address generator <NUM> decodes addresses of memory access requests received from data fabric <NUM> over the AXI4 bus. The memory access requests include access addresses in the physical address space represented in a normalized format. Address generator <NUM> converts the normalized addresses into a format that can be used to address the actual memory devices in memory system <NUM>, as well as to efficiently schedule related accesses. This format includes a region identifier that associates the memory access request with a particular rank, a row address, a column address, a bank address, and a bank group. On startup, the system BIOS queries the memory devices in memory system <NUM> to determine their size and configuration, and programs a set of configuration registers associated with address generator <NUM>. Address generator <NUM> uses the configuration stored in the configuration registers to translate the normalized addresses into the appropriate format. Command queue <NUM> is a queue of memory access requests received from the memory accessing agents in APU <NUM>, such as CPU cores <NUM> and <NUM> and graphics core <NUM>. Command queue <NUM> stores the address fields decoded by address generator <NUM> as well other address information that allows arbiter <NUM> to select memory accesses efficiently, including access type and quality of service (QoS) identifiers. CAM <NUM> includes information to enforce ordering rules, such as write after write (WAW) and read after write (RAW) ordering rules.

Error correction code (ECC) generation block <NUM> determines the ECC of write data to be sent to the memory. ECC check circuit <NUM> checks the received ECC against the incoming ECC.

Replay queue <NUM> is a temporary queue for storing selected memory accesses picked by arbiter <NUM> that are awaiting responses, such as address and command parity responses. Replay control logic <NUM> accesses ECC check circuit <NUM> to determine whether the returned ECC is correct or indicates an error. Replay control logic <NUM> initiates and controls a replay sequence in which accesses are replayed in the case of a parity or ECC error of one of these cycles. Replayed commands are placed in the memory interface queue <NUM>.

Refresh control logic <NUM> includes state machines for various powerdown, refresh, and termination resistance (ZQ) calibration cycles that are generated separately from normal read and write memory access requests received from memory accessing agents. For example, if a memory rank is in precharge powerdown, it must be periodically awakened to run refresh cycles. Refresh control logic <NUM> generates refresh commands periodically and in response to designated conditions to prevent data errors caused by leaking of charge off storage capacitors of memory cells in DRAM chips. Refresh control logic <NUM> includes an activate counter <NUM>, which in this embodiment has a counter for each memory region which counts a rolling number of activate commands sent over the memory channel to a memory region. The memory regions are memory banks in some embodiments, and memory sub-banks in other embodiments as further discussed below. In addition, refresh control logic <NUM> periodically calibrates ZQ to prevent mismatch in on-die termination resistance due to thermal changes in the system.

Arbiter <NUM> is bidirectionally connected to command queue <NUM> and is the heart of memory channel controller <NUM>, and improves efficiency by intelligent scheduling of accesses to improve the usage of the memory bus. Arbiter <NUM> uses timing block <NUM> to enforce proper timing relationships by determining whether certain accesses in command queue <NUM> are eligible for issuance based on DRAM timing parameters. For example, each DRAM has a minimum specified time between activate commands, known as "tRC". Timing block <NUM> maintains a set of counters that determine eligibility based on this and other timing parameters specified in the JEDEC specification, and is bidirectionally connected to replay queue <NUM>. Page table <NUM> maintains state information about active pages in each bank and rank of the memory channel for arbiter <NUM>, and is bidirectionally connected to replay queue <NUM>.

In response to write memory access requests received from interface <NUM>, ECC generation block <NUM> computes an ECC according to the write data. Data buffer <NUM> stores the write data and ECC for received memory access requests. It outputs the combined write data/ECC to memory interface queue <NUM> when arbiter <NUM> picks the corresponding write access for dispatch to the memory channel.

Power controller <NUM> generally includes an interface <NUM> to an advanced extensible interface, version one (AXI), an advanced peripheral bus (APB) interface <NUM>, and a power engine <NUM>. Interface <NUM> has a first bidirectional connection to the SMN, which includes an input for receiving an event signal labeled "EVENT_n" shown separately in <FIG>, and an output. APB interface <NUM> has an input connected to the output of interface <NUM>, and an output for connection to a PHY over an APB. Power engine <NUM> has an input connected to the output of interface <NUM>, and an output connected to an input of memory interface queue <NUM>. Power engine <NUM> includes a set of configuration registers <NUM>, a microcontroller (µC) <NUM>, a self refresh controller (SLFREF/PE) <NUM>, and a reliable read/write timing engine (RRW/TE) <NUM>. Configuration registers <NUM> are programmed over the AXI bus, and store configuration information to control the operation of various blocks in memory controller <NUM>. Accordingly, configuration registers <NUM> have outputs connected to these blocks that are not shown in detail in <FIG>. SLFREF/PE <NUM> is an engine that allows the manual generation of refreshes in addition to the automatic generation of refreshes by refresh control logic <NUM>. Reliable read/write timing engine <NUM> provides a continuous memory access stream to memory or I/O devices for such purposes as DDR interface maximum read latency (MRL) training and loopback testing.

Memory channel controller <NUM> includes circuitry that allows it to pick memory accesses for dispatch to the associated memory channel. In order to make the desired arbitration decisions, address generator <NUM> decodes the address information into predecoded information including rank, row address, column address, bank address, and bank group in the memory system, and command queue <NUM> stores the predecoded information. Configuration registers <NUM> store configuration information to determine how address generator <NUM> decodes the received address information. Arbiter <NUM> uses the decoded address information, timing eligibility information indicated by timing block <NUM>, and active page information indicated by page table <NUM> to efficiently schedule memory accesses while observing other criteria such as quality of service (QoS) requirements. For example, arbiter <NUM> implements a preference for accesses to open pages to avoid the overhead of precharge and activation commands required to change memory pages, and hides overhead accesses to one bank by interleaving them with read and write accesses to another bank. In particular during normal operation, arbiter <NUM> normally keeps pages open in different banks until they are required to be precharged prior to selecting a different page. Arbiter <NUM>, in some embodiments, determines eligibility for command selection based on at least on respective values of activate counter <NUM> for target memory regions of the respective commands.

<FIG> illustrates a block diagram of a portion <NUM> of memory controller <NUM> of <FIG> according to some embodiments. Portion <NUM> includes arbiter <NUM> and a set of control circuits <NUM> associated with the operation of arbiter <NUM>. Arbiter <NUM> includes a set of sub-arbiters <NUM> and a final arbiter <NUM>. Sub-arbiters <NUM> include a sub-arbiter <NUM>, a sub-arbiter <NUM>, and a sub-arbiter <NUM>. Sub-arbiter <NUM> includes a page hit arbiter <NUM> labeled "PH ARB", and an output register <NUM>. Page hit arbiter <NUM> has a first input connected to command queue <NUM>, a second input, and an output. Register <NUM> has a data input connected to the output of page hit arbiter <NUM>, a clock input for receiving the UCLK signal, and an output. Sub-arbiter <NUM> includes a page conflict arbiter <NUM> labeled "PC ARB", and an output register <NUM>. Page conflict arbiter <NUM> has a first input connected to command queue <NUM>, a second input, and an output. Register <NUM> has a data input connected to the output of page conflict arbiter <NUM>, a clock input for receiving the UCLK signal, and an output. Sub-arbiter <NUM> includes a page miss arbiter <NUM> labeled "PM ARB", and an output register <NUM>. Page miss arbiter <NUM> has a first input connected to command queue <NUM>, a second input, and an output. Register <NUM> has a data input connected to the output of page miss arbiter <NUM>, a clock input for receiving the UCLK signal, and an output. Final arbiter <NUM> has a first input connected to the output of refresh control logic <NUM>, a second input from a page close predictor <NUM>, a third input connected to the output of output register <NUM>, a fourth input connected to the output of output register <NUM>, a fifth input connected to the output of output register <NUM>, a first output for providing a first arbitration winner to queue <NUM> labeled "CMD1", and a second output for providing a second arbitration winner to queue <NUM> labeled "CMD2".

Control circuits <NUM> include timing block <NUM> and page table <NUM> as previously described with respect to <FIG>, and a page close predictor <NUM>, a current mode register <NUM>, and cross-mode enable logic <NUM>. Timing block <NUM> has an output connected to cross-mode enable logic <NUM>, an input and an output connected to page hit arbiter <NUM>, page conflict arbiter <NUM>, and page miss arbiter <NUM>. Page table <NUM> has an input connected to an output of replay queue <NUM>, an output connected to an input of replay queue <NUM>, an output connected to the input of command queue <NUM>, an output connected to the input of timing block <NUM>, and an output connected to the input of page close predictor <NUM>. Page close predictor <NUM> has an input connected to one output of page table <NUM>, an input connected to the output of output register <NUM>, and an output connected to the second input of final arbiter <NUM>. Cross-mode enable logic <NUM> has an input connected to current mode register <NUM>, and input connected to command queue <NUM>, an input and output connected to final arbiter <NUM>, and an input and output connected to page hit arbiter <NUM>, page conflict arbiter <NUM>, and page miss arbiter <NUM>.

In operation, arbiter <NUM> selects memory access commands from command queue <NUM> and refresh control logic <NUM> by taking into account the current mode (indicating whether a read streak or write streak is in progress), the page status of each entry, the priority of each memory access request, and the dependencies between requests. The priority is related to the quality of service or QoS of requests received from the AXI4 bus and stored in command queue <NUM>, but can be altered based on the type of memory access and the dynamic operation of arbiter <NUM>. Arbiter <NUM> includes three sub-arbiters that operate in parallel to address the mismatch between the processing and transmission limits of existing integrated circuit technology. The winners of the respective sub-arbitrations are presented to final arbiter <NUM>. Final arbiter <NUM> selects between these three sub-arbitration winners as well as a refresh operation from refresh control logic <NUM>, and may further modify a read or write command into a read or write with auto-precharge command as determined by page close predictor <NUM>.

Cross-mode enable logic <NUM> operates to cause streaks of read commands and streaks of write commands over the memory channel. During a current streak of either type of commands, cross-mode enable logic <NUM> monitors an indicator of data bus efficiency of the memory channel as further described below with respect to <FIG> and <FIG>. In response to the indicator of data bus efficiency indicating that data bus efficiency is less than a designated threshold, cross-mode enable logic <NUM> stops the current streak, starts a streak of the other type, and changes the current mode in current mode register <NUM>.

Each of page hit arbiter <NUM>, page conflict arbiter <NUM>, and page miss arbiter <NUM> has an input connected to the output of timing block <NUM> to determine timing eligibility of commands in command queue <NUM> that fall into these respective categories. Timing block <NUM> includes an array of binary counters that count durations related to the particular operations for each bank in each rank. The number of timers needed to determine the status depends on the timing parameter, the number of banks for the given memory type, and the number of ranks supported by the system on a given memory channel. The number of timing parameters that are implemented in turn depends on the type of memory implemented in the system. For example, GDDR5 memories require more timers to comply with more timing parameters than other DDRx memory types. By including an array of generic timers implemented as binary counters, timing block <NUM> can be scaled and reused for different memory types. The inputs from cross-mode enable logic <NUM> signal the sub-arbiters which type of commands, read or write, to provide as candidates for final arbiter <NUM>.

A page hit is a read or write cycle to an open page. Page hit arbiter <NUM> arbitrates between accesses in command queue <NUM> to open pages. The timing eligibility parameters tracked by timers in timing block <NUM> and checked by page hit arbiter <NUM> include, for example, row address strobe (RAS) to column address strobe (CAS) delay time (tRCD) and CAS latency (tCL). For example, tRCD specifies the minimum amount of time that must elapse before a read or write access to a page after it has been opened in a RAS cycle. Page hit arbiter <NUM> selects a sub-arbitration winner based on the assigned priority of the accesses. In one embodiment, the priority is a <NUM>-bit, one-hot value that therefore indicates a priority among four values, however it should be apparent that this four-level priority scheme is just one example. If page hit arbiter <NUM> detects two or more requests at the same priority level, then the oldest entry wins.

A page conflict is an access to one row in a bank when another row in the bank is currently activated. Page conflict arbiter <NUM> arbitrates between accesses in command queue <NUM> to pages that conflict with the page that is currently open in the corresponding bank and rank. Page conflict arbiter <NUM> selects a sub-arbitration winner that causes the issuance of a precharge command. The timing eligibility parameters tracked by timers in timing block <NUM> and checked by page conflict arbiter <NUM> include, for example, active to precharge command period (tRAS). Page conflict arbiter <NUM> selects a sub-arbitration winner based on the assigned priority of the access. If page conflict arbiter <NUM> detects two or more requests at the same priority level, then the oldest entry wins.

A page miss is an access to a bank that is in the precharged state. Page miss arbiter <NUM> arbitrates between accesses in command queue <NUM> to precharged memory banks. The timing eligibility parameters tracked by timers in timing block <NUM> and checked by page miss arbiter <NUM> include, for example, precharge command period (tRP). If there are two or more requests that are page misses at the same priority level, then the oldest entry wins.

Each sub-arbiter outputs a priority value for their respective sub-arbitration winner. Final arbiter <NUM> compares the priority values of the sub-arbitration winners from each of page hit arbiter <NUM>, page conflict arbiter <NUM>, and page miss arbiter <NUM>. Final arbiter <NUM> determines the relative priority among the sub-arbitration winners by performing a set of relative priority comparisons taking into account two sub-arbitration winners at a time. The sub-arbiters may include a set of logic for arbitrating commands for each mode, read and write, so that when the current mode changes, a set of available candidate commands are quickly available as sub-arbitration winners.

After determining the relative priority among the three sub-arbitration winners, final arbiter <NUM> then determines whether the sub-arbitration winners conflict (i.e. whether they are directed to the same bank and rank). When there are no such conflicts, then final arbiter <NUM> selects up to two sub-arbitration winners with the highest priorities. When there are conflicts, then final arbiter <NUM> complies with the following rules. When the priority value of the sub-arbitration winner of page hit arbiter <NUM> is higher than that of page conflict arbiter <NUM>, and they are both to the same bank and rank, then final arbiter <NUM> selects the access indicated by page hit arbiter <NUM>. When the priority value of the sub-arbitration winner of page conflict arbiter <NUM> is higher than that of page hit arbiter <NUM>, and they are both to the same bank and rank, final arbiter <NUM> selects the winner based on several additional factors. In some cases, page close predictor <NUM> causes the page to close at the end of the access indicated by page hit arbiter <NUM> by setting the auto precharge attribute.

Within page hit arbiter <NUM>, priority is initially set by the request priority from the memory accessing agent but is adjusted dynamically based on the type of accesses (read or write) and the sequence of accesses. In general, page hit arbiter <NUM> assigns a higher implicit priority to reads, but implements a priority elevation mechanism to ensure that writes make progress toward completion.

Whenever page hit arbiter <NUM> selects a read or write command, page close predictor <NUM> determines whether to send the command with the auto-precharge (AP) attribute or not. During a read or write cycle, the auto-precharge attribute is set with a predefined address bit and the auto-precharge attribute causes the DDR device to close the page after the read or write cycle is complete, which avoids the need for the memory controller to later send a separate precharge command for that bank. Page close predictor <NUM> takes into account other requests already present in command queue <NUM> that access the same bank as the selected command. If page close predictor <NUM> converts a memory access into an AP command, the next access to that page will be a page miss.

By using different sub-arbiters for different memory access types, each arbiter can be implemented with simpler logic than if it were required to arbitrate between all access types (page hits, page misses, and page conflicts; although embodiments including a single arbiter are envisioned). Thus the arbitration logic can be simplified and the size of arbiter <NUM> can be kept relatively small.

In other embodiments, arbiter <NUM> could include a different number of sub-arbiters. In yet other embodiments, arbiter <NUM> could include two or more sub-arbiters of a particular type. For example, arbiter <NUM> could include two or more page hit arbiters, two or more page conflict arbiters, and/or two or more page miss arbiters.

<FIG> is a flow diagram <NUM> of a process for managing streak efficiency according to some embodiments. In some versions, the process is embodied in monitoring logic circuity inside the memory controller's arbiter (such as arbiter <NUM>, <FIG>). In other versions, the process may be performed by digital logic or a controller having similar functionality while using different methods of arbitration than the sub-arbiters <NUM> and final arbiter <NUM> described above. The process generally works to decide when to perform a turnaround of the streak of commands, changing the current mode to read from write, or to write from read, to improve data bus utilization efficiency. The process may be used in combination with other techniques of determining the length of a streak of read or write commands.

The process starts at block <NUM>, at the beginning of each streak of commands, and determines snapshot or count of the commands currently in the command queue for the mode of commands (read or write) which will be burst in the streak. In some embodiments, this snapshot count is tracked by the streak turnaround process and is available at the beginning of a new streak. In some embodiments, the process updates the snapshot count to account for any new commands that have entered the command queue after the decision was made to end the prior streak.

At block <NUM>, the process determines the minimum burst length, the minimum number of commands to be sent in the streak, based on the snapshot of the number of commands from block <NUM>. Thus, the minimum burst length is adaptive to the current conditions at the memory controller. In this embodiment, the minimum burst length is calculated by scaling or multiplying the snapshot by a predetermined coefficient provided to the arbiter. A first coefficient "READ eCoefficiency" is used for read streaks, and a second coefficient "WRITE eCoefficiency" is used for write streaks, as shown at block <NUM>. In some embodiments, the snapshot is adjusted to account for commands of the new current mode available to be selected by the arbiter that are not "blocked", that is, do not become page conflicts due to a cross-mode activation. This adjustment is made by setting the minimum burst length equal to the lower of the scaled snapshot number and the total number of new current-mode requests not blocked by a cross-mode activation. This adjustment is made due to the scenario that at a streak turnaround, cross-mode requests become page hits and same bank current-mode requests become page conflicts. The process does not include these current-mode page conflicts conflicting with cross-mode hits in the count of commands available to be scheduled for the new streak, otherwise the minimum streak length would counteract the benefit of cross-mode ACTs due to the cross-mode activation.

At block <NUM>, the process starts sending commands for the streak, and monitors the size of the streak (the number of commands sent in the streak) until the minimum burst length set at block <NUM> is achieved.

Blocks <NUM> and <NUM> are performed for each command sent after the minimum burst length is achieved. At block <NUM>, following the minimum burst length, the process monitors a first set of one or more conditions indicating intra-burst efficiency, as further described below. In this embodiment, a second set of conditions indicating inter-burst efficiency is also monitored at block <NUM>. In some other embodiments, efficiency conditions are monitored only for intra-burst efficiency following the minimum burst length. Monitoring the set(s) of conditions in some embodiments include calculating one or more indicators such as bus usage efficiency or CAS latency. Monitoring the efficiency conditions may also include monitoring conditions at the memory controller such as the available current-mode or cross-mode commands. An exemplary embodiment using multiple CAS latency conditions is described below with respect to <FIG>.

At block <NUM>, the process decides whether to end the streak based on whether the monitored conditions indicate that ending the streak and starting a new streak of the other mode will be more efficient. In some embodiments, at least the first set of conditions (intra-burst efficiency) is employed to make the decision at block <NUM>. In other embodiments, a combination of the first and second sets of conditions are employed. The first and second sets of conditions may each include one or more conditions in various embodiments. If the process does not decide to end the streak at block <NUM>, it returns to block <NUM> to continue monitoring the streak as new commands are sent. If the process decides to end the streak at block <NUM>, it goes to block <NUM> where it changes the current mode and starts a new streak of commands of the type of commands that were cross-mode commands in the prior streak. To determine if the set(s) of conditions indicate a streak end, one or more conditions are compared to thresholds, or may be compared to each other. In some embodiments, the second set of conditions related to inter-burst efficiency is compared to a threshold that is based on at least partially on the time it takes to turnaround the process to start a new streak.

Generally, the depicted process has several advantages over other known streak management processes that make it suitable for managing a wide variety of memory access workloads. It also solves several problems that tend to occur with various streak management techniques. For example, the use of intra-burst efficiency management alone tends to cause excessive turnover of streaks because the intra-burst management will frequently decide to end a streak when there is not an efficient set of commands to burst for the cross-mode streak. Especially for workloads with poor bank level parallelism, such excessive turnover hurts overall efficiency. While the use of a minimum threshold may seem to mitigate such a problem, it has associated problems. If the minimum threshold is too large, a burst can become inefficient even before reaching minimum threshold. If the minimum burst length is too low, there will be more turnarounds. Minimum burst threshold is also workload dependent, meaning a threshold selected for one type of workload might not be suitable for another type of workload.

While the use of an adaptive minimum burst length alone tends to improve inter-burst efficiency, it does not provide sufficient management of the many different situations and workloads with which intra-burst efficiency can suffer. For example, the bursts may not continue long enough to achieve a high level of efficiency for a particular workload. The depicted process addresses this issue by combining intra-burst efficiency management with an adaptive minimum burst length.

<FIG> is a flow diagram <NUM> of a process for managing streak efficiency according to some additional embodiments. The process is typically performed by monitoring logic inside the memory controller's arbiter (such as arbiter <NUM>, <FIG>) to transact streaks of consecutive read commands and streaks of consecutive write commands sent to the system memory. The depicted process is an exemplary implementation of the process of <FIG>, and generally employs intervals between two or more adjacent CAS commands as intra-burst efficiency indicators, and also employs other CAS intervals as inter-burst efficiency indicators.

When a new streak begins, block <NUM> the turnaround monitor process is started. An adaptive minimum burst length is determined as described above with respect to <FIG>. At block <NUM>, the initial commands of the streak are sent over the command bus until the minimum burst length is met.

After the minimum burst length is met, the process performs several checks for each command sent in the streak to determine if the streak should end or continue. At block <NUM>, the current command is sent. At block <NUM>, the process determines whether only current mode commands are available to be sent at the arbiter. For example, if the current streak is a write streak, block <NUM> determines whether only write commands are pending. If so, the process continues the streak, returning to block <NUM> to send the next command. If not, the process goes to block <NUM>, where it determines if only cross-mode commands are available at the arbiter. If so, the process ends the current streak and begins a new streak.

At block <NUM>, if there are still current mode commands available, the process continues to block <NUM>, where it monitors intra-burst efficiency for each command based on the interval between CAS commands, referred to as "CAS-to-CAS" interval. In some embodiments, the interval is determined for multiple candidate commands at the arbiter. The interval is a measurement or projection of one or more time intervals between column-address-strobe (CAS) commands. The interval includes a time interval between a most recently transmitted CAS command and a time at which a selected subsequent CAS command can be transmitted. This calculation yields a projection of the intra-burst data bus efficiency of the respective candidate command. In this embodiment, the interval in clock cycles is compared to a predetermined threshold in to determine if it is to be considered a high efficiency command.

At block <NUM>, the process determines whether any cross-mode commands available as candidates have a high efficiency. If there are no high-efficiency cross-mode commands available, the process continues the current streak. Block <NUM> employs one or more efficiency conditions indicating inter-burst efficiency, which in this example is an efficiency indicator based on the potential CAS-to-CAS threshold of cross-mode commands. Generally, at block <NUM> an efficiency indicator is calculated for cross-mode commands which are available to be selected at the arbiter, and compared to one of thresholds <NUM> or <NUM> to determine if the cross-mode command is high efficiency. If the indicator is the CAS-to-CAS interval calculated directly in clock cycles, this indicator may be treated as a cost function in which low cost signals high efficiency. This check has the advantage of preventing a scenario, in which a turnaround is conducted, and then no high efficiency commands are available in the other mode and the process turns around again repeatedly. The thresholds used for cross-mode commands are a "Cross-Mode Write CAS Gap Threshold" <NUM>, used for the comparison when read commands are the current mode, and a "Cross-Mode Read CAS Gap Threshold" <NUM>, used when write commands are the current mode. In this embodiment, thresholds <NUM> and <NUM> are set based on an adjustable configuration register value, which is selected to account for the minimum CAS-to-CAS timing for the respective command type of command, the time it takes to perform a streak turnaround, the burst length employed (the number of data beats sent or pulled for each CAS command), and other considerations such as the possibility of rank switches and bank group switches.

If there are high-efficiency cross-mode commands available at block <NUM>, the process goes to block <NUM>, where it uses another type of intra-burst efficiency indicator to prevent a continuing sequence of commands that are just slightly within the high efficiency threshold but overall provide an inefficient streak. Block <NUM> calculates a "last <NUM>" current mode CAS-to-CAS interval, which provides the interval between the most recently transmitted CAS command, and a prior CAS command occurring three CAS commands ago. If this "last <NUM>" interval is greater than a designated threshold, the process ends the current streak and performs a turnaround.

If the "last <NUM>" interval is within the designated threshold, the process at block <NUM> goes to block <NUM>. At block <NUM>, it checks whether there are high-efficiency current-mode commands available for the current mode by comparing the CAS-to-CAS intervals of the candidate commands to one of thresholds <NUM> or <NUM>. If there are high-efficiency current-mode commands available, the process continues the current streak. If not, the process goes to block <NUM> where it ends the current streak. Ending the streak in each depicted case causes a turnaround process in which a streak of the other mode is begun.

As depicted by the threshold values feeding into block <NUM>, two different thresholds <NUM> and <NUM> are used at block <NUM> to monitor efficiency of commands depending on whether the current mode is read or write. The thresholds used for current mode commands are a "Current-Mode Write CAS Gap Threshold" <NUM>, used for the comparison when write commands are the current mode, and a "Current-Mode Read CAS Gap Threshold" <NUM>, used when read commands are the current mode. In this embodiment, thresholds <NUM> and <NUM> are set based on an adjustable configuration register value, which is selected to account for the minimum CAS-to-CAS timing for the respective command type of command, the burst length employed (the number of data beats sent or pulled for each CAS command), and other considerations such as the possibility of rank switches and bank group switches, for example. Because a minimum desired burst length is already achieved, the depicted process allows greater flexibility in managing intra-burst efficiency as compared to a scheme that also needs to account for excessive streak turnaround related to the intra-burst efficiency management. In this embodiment, the intra-burst efficiency conditions monitored at block <NUM> are employed at blocks <NUM> and <NUM> to determine whether to end the streak. In other embodiments, block <NUM> is not used. In various embodiments, other process decision flows are employed to achieve a similar result.

While the process in this embodiment employs several different determinations in deciding whether to end the current streak, other embodiments may include fewer than all of the depicted conditional determinations. Further, while the various determinations in flowchart <NUM> are depicted in a particular order, this is not limiting and various embodiments can achieve similar functionality using circuitry which conducts selected blocks in a different order or simultaneously.

Memory controller <NUM> of <FIG> or any portions thereof, such as arbiter <NUM> and refresh control logic <NUM>, may be described or represented by a computer accessible data structure in the form of a database or other data structure which can be read by a program and used, directly or indirectly, to fabricate integrated circuits. For example, this data structure may be a behavioral-level description or register-transfer level (RTL) description of the hardware functionality in a high level design language (HDL) such as Verilog or VHDL. The description may be read by a synthesis tool which may synthesize the description to produce a netlist including a list of gates from a synthesis library. The netlist includes a set of gates that also represent the functionality of the hardware including integrated circuits. The netlist may then be placed and routed to produce a data set describing geometric shapes to be applied to masks. The masks may then be used in various semiconductor fabrication steps to produce the integrated circuits. Alternatively, the database on the computer accessible storage medium may be the netlist (with or without the synthesis library) or the data set, as desired, or Graphic Data System (GDS) II data.

Claim 1:
A memory controller (<NUM>), comprising:
a command queue (<NUM>) having a first input for receiving memory access requests; and
an arbiter (<NUM>) coupled to the command queue for selecting entries from the command queue, and causing them to be transmitted over a memory channel (<NUM>), the arbiter operable to:
transmit streaks of consecutive read commands and streaks of consecutive write commands over the memory channel; and
transmit a current streak of one of consecutive read commands or consecutive write commands for at least a minimum burst length based on a number of commands of a designated type available to be selected by the arbiter;
characterised in that the arbiter (<NUM>) is further operable to:
following the minimum burst length, based on whether no commands for a new streak are available for which a column-address-strobe, CAS, command can be sent within a first defined CAS-to-CAS interval, continue the current streak.