DDR memory error recovery

In one form, a memory controller includes a command queue, an arbiter, and a replay queue. The command queue receives and stores memory access requests. The arbiter is coupled to the command queue for providing a sequence of memory commands to a memory channel. The replay queue stores the sequence of memory commands to the memory channel, and continues to store memory access commands that have not yet received responses from the memory channel. When a response indicates a completion of a corresponding memory command without any error, the replay queue removes the corresponding memory command without taking further action. When a response indicates a completion of the corresponding memory command with an error, the replay queue replays at least the corresponding memory command. In another form, a data processing system includes the memory controller, a memory accessing agent, and a memory system to which the memory controller is coupled.

BACKGROUND

Computer systems typically use inexpensive and high-density dynamic random access memory (DRAM) chips for main memory. Most DRAM chips sold today are compatible with various double data rate (DDR) DRAM standards promulgated by the Joint Electron Devices Engineering Council (JEDEC). DDR memory controllers are used to manage the interface between various memory accessing agents and DDR DRAMs according to published DDR standards.

Modern DDR memory controllers maintain queues to store pending memory access requests to allow them to pick the pending memory access requests out of order in relation to the order in which they were generated or stored to increase efficiency. For example, the memory controllers can retrieve multiple memory access requests to the same row in a given rank of memory from the queue and issue them consecutively to the memory system to avoid the overhead of precharging the current row and activating another row.

DDR memory systems include a variety of mechanisms for error detection and recovery, such as parity bits, cyclic redundancy codes (CRCs), error detection code (EDC), or other error correcting codes (ECCs) that are stored along with the data in the DDR DRAMs. When performing memory accesses, DDR memory controllers compare the stored CRC or ECC bits to CRC or ECC bits calculated with the memory access. In response to detecting an error, the DDR memory controller corrects the error if possible, and reports the error to the operating system, and the operating system determines further corrective action to be taken. However DRAM buses operate at relatively high clock rates, such as 2400 MegaHertz (MHz), and transfer data on both transitions of the clock cycle. Because of the high data rates, DDR memory buses are susceptible to occasional random errors or “glitches” on the memory bus. While known DDR memory controllers have mechanisms to detect and correct these errors, the mechanisms reduce system performance significantly due to the overhead caused by the operating system calls.

In the following description, the use of the same reference numerals in different drawings indicates similar or identical items. Unless otherwise noted, the word “coupled” and its associated verb forms include both direct connection and indirect electrical connection by means known in the art, and unless otherwise noted any description of direct connection implies alternate embodiments using suitable forms of indirect electrical connection as well.

DETAILED DESCRIPTION OF ILLUSTRATIVE EMBODIMENTS

In one form, a memory controller includes a command queue, an arbiter, and a replay queue. The command queue receives and stores memory access requests. The arbiter is coupled to the command queue for providing a sequence of memory commands to a memory channel. The replay queue stores the sequence of memory commands to the memory channel, and continues to store memory access commands that have not yet received responses from the memory channel. When a response indicates a completion of a corresponding memory command without any error, the replay queue removes the corresponding memory command without taking further action. When a response indicates a completion of the corresponding memory command with an error, the replay queue replays at least the corresponding memory command.

In another form, a memory controller includes a command queue, an arbiter, a memory interface queue, and a replay queue. The command queue receives and stores memory access requests. The arbiter is coupled to the command queue for providing a sequence of memory commands to a memory channel. The memory interface queue is coupled to the command queue for receiving and storing memory access requests. The replay queue is coupled to the memory interface queue and stores the sequence of memory commands to the memory channel, and continues to store memory access commands that have not yet received responses from the memory channel. The memory controller remains in a normal state as long as it does not detect any errors, wherein in the normal state the memory interface queue continues to receive commands from the command queue that are picked by the arbiter. In response to detecting an error, the memory controller enters a recovery state in which the replay queue replays at least one corresponding memory command by sending the at least one corresponding memory command to the memory interface queue.

In yet another form, a data processing system includes a memory accessing agent, a memory system, and a memory controller. The memory accessing agent provides memory access requests. The memory controller is coupled to the memory accessing agent and the memory system. The memory controller includes a command queue, an arbiter, and a replay queue. The command queue receives and stores memory access requests. The arbiter is coupled to the command queue for providing a sequence of memory commands to the memory system. The replay queue stores the sequence of memory commands to the memory channel, and continues to store memory access commands that have not yet received responses from the memory system. When a response indicates a completion of a corresponding memory command without any error, the replay queue removes the corresponding memory command without taking further action. When a response indicates a completion of the corresponding memory command with an error, the replay queue replays at least the corresponding memory command.

In still another form, a method includes receiving and storing memory access requests. A sequence of memory commands are provided to a memory channel from stored memory access requests, and memory commands that have not yet received error-free responses from the memory channel continue to be stored. Whether or not a memory error occurred is detected in responses received from the memory channel. In response to detecting no memory errors, a memory controller remains in a normal state, and while in the normal state commands continue to be provided from among stored memory access requests. In response to detecting an error, a recovery state is entered, and while in the recovery state stored memory commands are replayed starting from a corresponding memory command at which the error occurred.

FIG.1illustrates in block diagram form a data processing system100according to some embodiments. Data processing system100includes generally a data processor110in the form of an accelerated processing unit (APU), a memory system120, a peripheral component interconnect express (PCIe) system150, a universal serial bus (USB) system160, and a disk drive170. Data processor110operates as the central processing unit (CPU) of data processing system100and provides various buses and interfaces useful in modern computer systems. These interfaces include two double data rate (DDRx) memory channels, a PCIe root complex for connection to a PCIe link, a USB controller for connection to a USB network, and an interface to a Serial Advanced Technology Attachment (SATA) mass storage device.

Memory system120includes a memory channel130and a memory channel140. Memory channel130includes a set of dual inline memory modules (DIMMs) connected to a DDRx bus132, including representative DIMMs134,136, and138that in this example correspond to separate ranks. Likewise memory channel140includes a set of DIMMs connected to a DDRx bus142, including representative DIMMs144,146, and148.

PCIe system150includes a PCIe switch152connected to the PCIe root complex in data processor110, a PCIe device154, a PCIe device156, and a PCIe device158. PCIe device156in turn is connected to a system basic input/output system (BIOS) memory157. System BIOS memory157can be any of a variety of non-volatile memory types, such as read-only memory (ROM), flash electrically erasable programmable ROM (EEPROM), and the like.

USB system160includes a USB hub162connected to a USB master in data processor110, and representative USB devices164,166, and168each connected to USB hub162. USB devices164,166, and168could be devices such as a keyboard, a mouse, a flash EEPROM port, and the like.

Disk drive170is connected to data processor110over a SATA bus and provides mass storage for the operating system, application programs, application files, and the like.

Data processing system100is suitable for use in modern computing applications by providing a memory channel130and a memory channel140. Each of memory channels130and140can connect to state-of-the-art DDR memories such as DDR version four (DDR4), low power DDR4 (LPDDR4), graphics DDR version five (gDDR5), and high bandwidth memory (HBM), and can be adapted for future memory technologies. These memories provide high bus bandwidth and high speed operation. At the same time, they also provide low power modes to save power for battery-powered applications such as laptop computers, and also provide built-in thermal monitoring.

FIG.2illustrates in block diagram form an APU200suitable for use in data processing system100ofFIG.1. APU200includes generally a central processing unit (CPU) core complex210, a graphics core220, a set of display engines230, a memory management hub240, a data fabric250, a set of peripheral controllers260, a set of peripheral bus controllers270, a system management unit (SMU)280, and a set of memory controllers290.

CPU core complex210includes a CPU core212and a CPU core214. In this example, CPU core complex210includes two CPU cores, but in other embodiments CPU core complex can include an arbitrary number of CPU cores. Each of CPU cores212and214is bidirectionally connected to a system management network (SMN), which forms a control fabric, and to data fabric250, and is capable of providing memory access requests to data fabric250. Each of CPU cores212and214may be unitary cores, or may further be a core complex with two or more unitary cores sharing certain resources such as caches.

Graphics core220is a high performance graphics processing unit (GPU) capable of performing graphics operations such as vertex processing, fragment processing, shading, texture blending, and the like in a highly integrated and parallel fashion. Graphics core220is bidirectionally connected to the SMN and to data fabric250, and is capable of providing memory access requests to data fabric250. In this regard, APU200may either support a unified memory architecture in which CPU core complex210and graphics core220share the same memory space, or a memory architecture in which CPU core complex210and graphics core220share a portion of the memory space, while graphics core220also uses a private graphics memory not accessible by CPU core complex210.

Display engines230render and rasterize objects generated by graphics core220for display on a monitor. Graphics core220and display engines230are bidirectionally connected to a common memory management hub240for uniform translation into appropriate addresses in memory system120, and memory management hub240is bidirectionally connected to data fabric250for generating such memory accesses and receiving read data returned from the memory system.

Data fabric250includes a crossbar switch for routing memory access requests and memory responses between any memory accessing agent and memory controllers290. It also includes a system memory map, defined by BIOS, for determining destinations of memory accesses based on the system configuration, as well as buffers for each virtual connection.

Peripheral controllers260include a USB controller262and a SATA interface controller264, each of which is bidirectionally connected to a system hub266and to the SMN bus. These two controllers are merely exemplary of peripheral controllers that may be used in APU200.

Peripheral bus controllers270include a system controller or “Southbridge” (SB)272and a PCIe controller274, each of which is bidirectionally connected to an input/output (I/O) hub276and to the SMN bus. I/O hub276is also bidirectionally connected to system hub266and to data fabric250. Thus for example a CPU core can program registers in USB controller262, SATA interface controller264, SB272, or PCIe controller274through accesses that data fabric250routes through I/O hub276.

SMU280is a local controller that controls the operation of the resources on APU200and synchronizes communication among them. SMU280manages power-up sequencing of the various processors on APU200and controls multiple off-chip devices via reset, enable and other signals. SMU280includes one or more clock sources not shown inFIG.2, such as a phase locked loop (PLL), to provide clock signals for each of the components of APU200. SMU280also manages power for the various processors and other functional blocks, and may receive measured power consumption values from CPU cores212and214and graphics core220to determine appropriate power states.

APU200also implements various system monitoring and power saving functions. In particular one system monitoring function is thermal monitoring. For example, if APU200becomes hot, then SMU280can reduce the frequency and voltage of CPU cores212and214and/or graphics core220. If APU200becomes too hot, then it can be shut down entirely. Thermal events can also be received from external sensors by SMU280via the SMN bus, and SMU280can reduce the clock frequency and/or power supply voltage in response.

FIG.3illustrates in block diagram form a memory controller300and an associated physical interface (PHY)330suitable for use in APU200ofFIG.2according to some embodiments. Memory controller300includes a memory channel310and a power engine320. Memory channel310includes a host interface312, a memory channel controller314, and a physical interface316. Host interface312bidirectionally connects memory channel controller314to data fabric250over a scalable data port (SDP). Physical interface316bidirectionally connects memory channel controller314to PHY330over a bus that conforms to the DDR-PHY Interface Specification (DFI). Power engine320is bidirectionally connected to SMU280over the SMN bus, to PHY330over the Advanced Peripheral Bus (APB), and is also bidirectionally connected to memory channel controller314. PHY330has a bidirectional connection to a memory channel such as memory channel130or memory channel140ofFIG.1. Memory controller300is an instantiation of a memory controller for a single memory channel using a single memory channel controller314, and has a power engine320to control operation of memory channel controller314in a manner that will be described further below.

FIG.4illustrates in block diagram form another memory controller400and associated PHYs440and450suitable for use in APU200ofFIG.2according to some embodiments. Memory controller400includes memory channels410and420and a power engine430. Memory channel410includes a host interface412, a memory channel controller414, and a physical interface416. Host interface412bidirectionally connects memory channel controller414to data fabric250over an SDP. Physical interface416bidirectionally connects memory channel controller414to PHY440, and conforms to the DFI Specification. Memory channel420includes a host interface422, a memory channel controller424, and a physical interface426. Host interface422bidirectionally connects memory channel controller424to data fabric250over another SDP. Physical interface426bidirectionally connects memory channel controller424to PHY450, and conforms to the DFI Specification. Power engine430is bidirectionally connected to SMU280over the SMN bus, to PHYs440and450over the APB, and is also bidirectionally connected to memory channel controllers414and424. PHY440has a bidirectional connection to a memory channel such as memory channel130ofFIG.1. PHY450has a bidirectional connection to a memory channel such as memory channel140ofFIG.1. Memory controller400is an instantiation of a memory controller having two memory channel controllers and uses a shared power engine430to control operation of both memory channel controller414and memory channel controller424in a manner that will be described further below.

FIG.5illustrates in block diagram form a memory controller500according to some embodiments. Memory controller500includes generally a memory channel controller510and a power controller550. Memory channel controller510includes generally an interface512, a queue514, a command queue520, an address generator522, a content addressable memory (CAM)524, a replay queue530, a refresh logic block532, a timing block534, a page table536, an arbiter538, an error correction code (ECC) check block542, an ECC generation block544, and a data buffer (DB)546.

Interface512has a first bidirectional connection to data fabric250over an external bus, and has an output. In memory controller500, this external bus is compatible with the advanced extensible interface version four specified by ARM Holdings, PLC of Cambridge, England, known as “AXI4”, but can be other types of interfaces in other embodiments. Interface512translates memory access requests from a first clock domain known as the FCLK (or MEMCLK) domain to a second clock domain internal to memory controller500known as the UCLK domain. Similarly, queue514provides memory accesses from the UCLK domain to the DFICLK domain associated with the DFI interface.

Address generator522decodes addresses of memory access requests received from data fabric250over the AXI4 bus. The memory access requests include access addresses in the physical address space represented in a normalized format. Address generator522converts the normalized addresses into a format that can be used to address the actual memory devices in memory system120, as well as to efficiently schedule related accesses. This format includes a region identifier that associates the memory access request with a particular rank, a row address, a column address, a bank address, and a bank group. On startup, the system BIOS queries the memory devices in memory system120to determine their size and configuration, and programs a set of configuration registers associated with address generator522. Address generator522uses the configuration stored in the configuration registers to translate the normalized addresses into the appropriate format. Command queue520is a queue of memory access requests received from the memory accessing agents in data processing system100, such as CPU cores212and214and graphics core220. Command queue520stores the address fields decoded by address generator522as well other address information that allows arbiter538to select memory accesses efficiently, including access type and quality of service (QoS) identifiers. CAM524includes information to enforce ordering rules, such as write after write (WAW) and read after write (RAW) ordering rules.

Replay queue530is a temporary queue for storing memory accesses picked by arbiter538that are awaiting responses, such as address and command parity responses, write cyclic redundancy check (CRC) responses for DDR4 DRAM or write and read CRC responses for GDDR5 DRAM. Replay queue530accesses ECC check block542to determine whether the returned ECC is correct or indicates an error. Replay queue530allows the accesses to be replayed in the case of a parity or CRC error of one of these cycles.

Refresh logic532includes state machines for various powerdown, refresh, and termination resistance (ZQ) calibration cycles that are generated separately from normal read and write memory access requests received from memory accessing agents. For example, if a memory rank is in precharge powerdown, it must be periodically awakened to run refresh cycles. Refresh logic532generates refresh commands periodically to prevent data errors caused by leaking of charge off storage capacitors of memory cells in DRAM chips. In addition, refresh logic532periodically calibrates ZQ to prevent mismatch in on-die termination resistance due to thermal changes in the system. Refresh logic532also decides when to put DRAM devices in different power down modes.

Arbiter538is bidirectionally connected to command queue520and is the heart of memory channel controller510. It improves efficiency by intelligent scheduling of accesses to improve the usage of the memory bus. Arbiter538uses timing block534to enforce proper timing relationships by determining whether certain accesses in command queue520are eligible for issuance based on DRAM timing parameters. For example, each DRAM has a minimum specified time between activate commands to the same bank, known as “tRC”. Timing block534maintains a set of counters that determine eligibility based on this and other timing parameters specified in the JEDEC specification, and is bidirectionally connected to replay queue530. Page table536maintains state information about active pages in each bank and rank of the memory channel for arbiter538, and is bidirectionally connected to replay queue530.

In response to write memory access requests received from interface512, ECC generation block544computes an ECC according to the write data. DB546stores the write data and ECC for received memory access requests. It outputs the combined write data/ECC to queue514when arbiter538picks the corresponding write access for dispatch to the memory channel.

Power controller550generally includes an interface552to an advanced extensible interface, version one (AXI), an APB interface554, and a power engine560. Interface552has a first bidirectional connection to the SMN, which includes an input for receiving an event signal labeled “EVENT_n” shown separately inFIG.5, and an output. APB interface554has an input connected to the output of interface552, and an output for connection to a PHY over an APB. Power engine560has an input connected to the output of interface552, and an output connected to an input of queue514. Power engine560includes a set of configuration registers562, a microcontroller (μC)564, a self refresh controller (SLFREF/PE)566, and a reliable read/write training engine (RRW/TE)568. Configuration registers562are programmed over the AXI bus, and store configuration information to control the operation of various blocks in memory controller500. Accordingly, configuration registers562have outputs connected to these blocks that are not shown in detail inFIG.5. Self refresh controller566is an engine that allows the manual generation of refreshes in addition to the automatic generation of refreshes by refresh logic532. Reliable read/write training engine568provides a continuous memory access stream to memory or I/O devices for such purposes as DDR interface read latency training and loopback testing.

Memory channel controller510includes circuitry that allows it to pick memory accesses for dispatch to the associated memory channel. In order to make the desired arbitration decisions, address generator522decodes the address information into predecoded information including rank, row address, column address, bank address, and bank group in the memory system, and command queue520stores the predecoded information. Configuration registers562store configuration information to determine how address generator522decodes the received address information. Arbiter538uses the decoded address information, timing eligibility information indicated by timing block534, and active page information indicated by page table536to efficiently schedule memory accesses while observing other criteria such as QoS requirements. For example, arbiter538implements a preference for accesses to open pages to avoid the overhead of precharge and activation commands required to change memory pages, and hides overhead accesses to one bank by interleaving them with read and write accesses to another bank. In particular during normal operation, arbiter538may decide to keep pages open in different banks until they are required to be precharged prior to selecting a different page.

Arbiter538uses timing block534to determine timing eligibility for pending accesses, and then picks eligible accesses from command queue520based on a set of criteria that ensure both efficiency and fairness. Arbiter538supports two mechanisms to ensure both efficiency and fairness. First, arbiter538performs read/write transaction management to ensure both efficiency and fairness by examining attributes of memory access requests stored in command queue520as well as programmable threshold values to control the conditions in which reads are allowed to proceed while writes are allowed to make progress. Second, arbiter538includes streak counters that ensure that streaks of accesses of certain types are not allowed to hold the memory bus indefinitely. These two mechanisms will now be described.

DDR Memory Error Recovery

As mentioned above, replay queue530is a temporary queue for storing memory accesses picked by arbiter538that are awaiting responses, such as address and command parity responses, write cyclic redundancy check (CRC) responses for DDR4 DRAM, or write and read CRC responses for GDDR5 DRAM. Replay queue530accesses ECC check block542to determine whether the returned ECC is correct or indicates an error. Replay queue530allows the accesses to be replayed in the case of a parity or CRC error of one of these cycles. In addition, replay queue530takes advantage of error reporting mechanisms available in current DDR DRAMs to make a decision about replay. By assuming that memory errors are normally temporary and that the memory channel will shortly recover, replay queue530provides a graceful backup and replay mechanism to avoid lengthy and disruptive recovery sequences.

Some devices support data protection on transfers (e.g., GDDR5 read and write data transfers with error detection and correction (EDC); DDR4 write data transfers protected by write CRC). GDDR5 devices provide a uni-directional EDC bus to transfer CRC data whereby the EDC values always travel from the devices to the controller independent of whether the request was a read or write. During GDDR5 read response data transfers, the EDC bundle is returned with or soon after the response data based on the parameter tcrcrl. During a write data transfer the EDC bundle is returned after the GDDR5 device receives the write data (as it calculates the EDC value from the received write data). On reads, memory controller500calculates the EDC value from the received read data response and compares it to the EDC data received from that read. On writes, memory controller500calculates the expected EDC value and temporarily stores it in memory controller500for later comparison with the EDC packet returned from the GDDR5 device after a write data cycle. The expected write data EDC value is stored in EDC Queue logic (EDCQ) in memory controller500.

Memory controller500supports “early response” to reduce latency, and replay queue530returns the early response back to the memory channel “early” relative to the time the EDC response is returned. This “Early Response” support then requires that a response be “cancelled” should the EDC come back “bad”. Memory controller500responds to the memory channel with an “early response” packet and “response cancel” should the EDC come back bad. If the EDC is returned “good” then no further action is required. Upon a failed read or write request, replay queue530performs a retry of the cycle request. A write request is acknowledged back to the memory channel when issued. Should the write fail, replay queue530retries the command and maintains write data ordering, independent of the memory channel.

DDR4 devices support CRC checking on write commands only. The CRC information is sent out along with the write data during the last two bit-times. Therefore, unlike GDDR5 EDC, CRC information is checked in the DDR4 device and the device asserts the ALERT# signal upon an error detection. Because the ALERT# signal is open-drain and considered asynchronous to the DRAM MEMCLK or any internal controller clock, thus requiring synchronization, the ability of memory controller500to identify the particular write transaction that caused the error in a sequence of consecutive bursts is limited, and memory controller500replays a range of previously issued write commands to ensure the replay of the failed write. For both GDDR5 and DDR4 memories, replay queue530replays the write and read transactions, and halts any new transactions being issued from command queue520until the failed cycles have completed successfully.

A specific implementation of replay queue530and its operation will now be described.

FIG.6illustrates a state diagram associated with the operation of a finite state machine600of memory controller500ofFIG.5. Finite state machine600is defined by a NORMAL state610, a wait acknowledge state620labeled “WAIT_ACK”, a command replay state630labeled “CMD_REP”, and an error recovery state640labeled “ERR_REC”. Memory controller500remains in NORMAL state610as long as there is no DRAM error detected. During this state, arbiter538retains control of the sequencing of memory access commands to the memory system, and sends normal traffic by picking commands from command queue520according to its normal priority rules. Memory controller500leaves NORMAL state610when it detects a DRAM error returned from replay queue530, and temporarily enters WAIT_ACK state620. Queue514then stops accepting commands selected by arbiter538, and starts receiving commands from replay queue530in a selected recovery phase. Replay queue530takes control of issuing memory access commands and performs a recovery sequence based on the DRAM type and the error type. For example in the case of a command/address error in a DDR4 system, memory controller500enters ERR_REC state640, and remains in this state until a wait time after replay queue530sends out the last recovery command to guarantee the command is provided correctly. At that time, memory controller500transitions to CMD_REP state630. In CMD_REP state630, replay queue530replays the command or commands on which the error may have occurred. CMD_REP state630ends when replay queue530sends out the last command on which an error may have occurred.

For example, a typical sequence proceeds as follows:

1) On bootup, finite state machine600starts in NORMAL state610.

2) For DDR4 systems, replay queue530samples an error (ALERT_n=0), or in GDDR5 systems, once replay queue530receives a CRC error returned from queue514, it requests control from queue514, and finite state machine600moves to WAIT_ACK state620.

3) Arbiter538and queue514need to wake any DRAMs from power down mode before acknowledging the recovery request, and disable dynamic power down while replay queue530is taking control.

4) Once queue514acknowledges the request, state machine600either moves to ERR_REC state640to perform a command/address error recovery sequence, or moves to CMD_REP state630directly.

5) Once command/address error recovery is done, then state machine600moves to CMD_REP state630.

6) In CMD_REP state630, the error transactions get replayed and resent to the memory system. In case of further errors, state machine600will stay in CMD_REP state630or move to ERR_REC state640in the case of a command/address error.

7) Once replay is done and a cool down time has passed, replay queue530releases control of transactions to arbiter538, and finite state machine600returns to NORMAL state610and memory controller500is again ready to provide normal commands. If memory controller500replays a memory command and receives an error for a predefined number of times, it indicates a system error. The incidence of repeated errors indicates a real system failure instead of a temporary condition on the memory bus, and it requires other remedial action to be taken by the operating system.

The memory controller ofFIG.5may be implemented with various combinations of hardware and software. For example, the hardware circuitry may include priority encoders, finite state machines, programmable logic arrays (PLAs), and the like. For example, arbiter538could be implemented with a microcontroller executing stored program instructions to evaluate the relative timing eligibility of the pending commands. In this case, some of the instructions may be stored in a non-transitory computer memory or computer readable storage medium for execution by the microcontroller. In various embodiments, the non-transitory computer readable storage medium includes a magnetic or optical disk storage device, solid-state storage devices such as Flash memory, or other non-volatile memory device or devices. The computer readable instructions stored on the non-transitory computer readable storage medium may be in source code, assembly language code, object code, or other instruction format that is interpreted and/or executable by one or more processors.

The memory controller ofFIG.5or any portion thereof, such as replay queue530, may be described or represented by a computer accessible data structure in the form of a database or other data structure which can be read by a program and used, directly or indirectly, to fabricate integrated circuits. For example, this data structure may be a behavioral-level description or register-transfer level (RTL) description of the hardware functionality in a high level design language (HDL) such as Verilog or VHDL. The description may be read by a synthesis tool which may synthesize the description to produce a netlist comprising a list of gates from a synthesis library. The netlist includes a set of gates that also represent the functionality of the hardware comprising integrated circuits. The netlist is then placed and routed to produce a data set describing geometric shapes to be applied to masks. The masks are then used in various semiconductor fabrication steps to produce the integrated circuits. Alternatively, the database on the computer accessible storage medium may be the netlist (with or without the synthesis library) or the data set, as desired, or Graphic Data System (GDS) II data.

While particular embodiments have been described, various modifications to these embodiments will be apparent to those skilled in the art. For example, the internal architecture of memory channel controller510and/or power engine550may vary in different embodiments. Memory controller500may interface to other types of memory besides DDRx memory, such as high bandwidth memory (HBM), RAMbus DRAM (RDRAM), and the like. While the illustrated embodiment showed each rank of memory corresponding to separate DIMMs, in other embodiments each DIMM can support multiple ranks.

Accordingly, it is intended by the appended claims to cover all modifications of the disclosed embodiments that fall within the scope of the disclosed embodiments.