Patent Description:
Computer systems typically use inexpensive and high density dynamic random access memory (DRAM) chips for main memory. Most DRAM chips sold today are compatible with various double data rate (DDR) DRAM standards promulgated by the Joint Electron Devices Engineering Council (JEDEC). As JEDEC has promulgated new DDR standards, there have been significant periods in which multiple generations of DDR DRAMs, such as DDR3 and DDR4, are popular. In addition, JEDEC specifies another class of DRAM that is designed specifically for the needs of modern graphics processors, known as graphics DDR (gDDR) memory, and one generation, gDDR5, remains popular today. Thus, it is important for memory controllers to be able to flexibly interface to any one of these plus potentially other emerging memory types.

Memory controller flexibility is also important for the memory system to meet the needs of the different types of products that use it. For example, memories are typically designed with a power-of-two density to simplify layout and decoding. Memory chip densities have historically increased exponentially as modem integrated circuit lithography techniques have evolved. Thus historically DRAM sizes have evolved from <NUM> kilobit (64Kb) available in the mid <NUM>, to 128Kb, to <NUM> Kb, and so on until the present in which DDR DRAMs are commonly available in <NUM>-, <NUM>-, and <NUM>-gigabit (Gb) densities. There are two reasons why this trend may not continue. First, semiconductor lithography technology may be approaching physical limits. Thus memory manufacturers may offer intermediate sizes that are not power-of-two. Second, designers may need memory having densities that are not close to the nearest power of two size, and may not want the extra product cost that comes with the next higher density. Thus memory manufacturers have started designing non power-of-two memory sizes to better meet these realities. Interfacing to non power-of-two memories places additional burdens on memory controller manufacturers to design the circuitry that meets all possible configurations without excessive cost.

Memory systems operate more efficiently if the memory controller is able to access different banks in an interleaved fashion without causing page conflicts. By interleaving accesses to different banks, the memory controller is able to partially hide the overhead that would be required for a series of accesses to different rows in the same bank. Known memory controllers use a circuit that scrambles or "swizzles" the input address so that sequential accesses to the same rank and bank will be spread across multiple banks. For example, the memory controller uses certain address bits to scramble the bank address so that memory accesses in a relatively small region of the address space are mapped to different banks. The bank scramble algorithm implemented by this memory controller provides a pattern of accesses with a desirable level of interleaving for some systems but not for others, depending on the type of system, the characteristics of the accesses generated by the application program and the operating system, etc.
<CIT>) relates to an arbiter and arbitration method of multiple data accesses. <CIT>) relates to a system and method for dynamic memory interleaving and de-interleaving. <CIT>) discloses a memory controller and method that detects that a number of activate commands to a particular row or a set of rows of a memory exceeds a threshold, and in response changes a normal activate command into an activate adjacent (ACTADJ) command to cause the memory to refresh rows adjacent to the particular row.

In the following description, the use of the same reference numerals in different drawings indicates similar or identical items. Unless otherwise noted, the word "coupled" and its associated verb forms include both direct connection and indirect electrical connection by means known in the art, and unless otherwise noted any description of direct connection implies alternate embodiments using suitable forms of indirect electrical connection as well.

<FIG> illustrates in block diagram form a data processing system <NUM> according to some embodiments. Data processing system <NUM> includes a data processor <NUM> in the form of an accelerated processing unit (APU), a memory system <NUM>, a peripheral component interconnect express (PCIe) system <NUM>, a universal serial bus (USB) system <NUM>, and a disk drive <NUM>. Data processor <NUM> operates as the central processing unit (CPU) of data processing system <NUM> and provides various buses and interfaces useful in modern computer systems. These interfaces include two double data rate (DDRx) memory channels, a PCIe root complex for connection to a PCIe link, a USB controller for connection to a USB network, and an interface to a Serial Advanced Technology Attachment (SATA) mass storage device.

Memory system <NUM> includes a memory channel <NUM> and a memory channel <NUM>. Memory channel <NUM> includes a set of dual inline memory modules (DIMMs) connected to a DDRx bus <NUM>, including representative DIMMs <NUM>, <NUM>, and <NUM> that in this example correspond to separate ranks. Likewise memory channel <NUM> includes a set of DIMMs connected to a DDRx bus <NUM>, including representative DIMMs <NUM>, <NUM>, and <NUM>.

PCIe system <NUM> includes a PCIe switch <NUM> connected to the PCIe root complex in data processor <NUM>, a PCIe device <NUM>, a PCIe device <NUM>, and a PCIe device <NUM>. PCIe device <NUM> in turn is connected to a system basic input/output system (BIOS) memory <NUM>. System BIOS memory <NUM> can be any of a variety of non-volatile memory types, such as read-only memory (ROM), flash electrically erasable programmable ROM (EEPROM), and the like.

USB system <NUM> includes a USB hub <NUM> connected to a USB master in data processor <NUM>, and representative USB devices <NUM>, <NUM>, and <NUM> each connected to USB hub <NUM>. USB devices <NUM>, <NUM>, and <NUM> could be devices such as a keyboard, a mouse, a flash EEPROM port, and the like.

Disk drive <NUM> is connected to data processor <NUM> over a SATA bus and provides mass storage for the operating system, application programs, application files, and the like.

Data processing system <NUM> is suitable for use in modem computing applications by providing a memory channel <NUM> and a memory channel <NUM>. Each of memory channels <NUM> and <NUM> can connect to state-of-the-art DDR memories such as DDR version four (DDR4), low power DDR4 (LPDDR4), graphics DDR version five (gDDR5), and high bandwidth memory (IIBM), and can be adapted for future memory technologies. These memories provide high bus bandwidth and high speed operation. At the same time, they also provide low power modes to save power for battery-powered applications such as laptop computers, and also provide built-in thermal monitoring.

<FIG> illustrates in block diagram form an APU <NUM> suitable for use in data processing system <NUM> of <FIG>. APU <NUM> includes a central processing unit (CPU) core complex <NUM>, a graphics core <NUM>, a set of display engines <NUM>, a memory management hub <NUM>, a data fabric <NUM>, a set of peripheral controllers <NUM>, a set of peripheral bus controllers <NUM>, a system management unit (SMU) <NUM>, and a set of memory controllers <NUM>.

CPU core complex <NUM> includes a CPU core <NUM> and a CPU core <NUM>. In this example, CPU core complex <NUM> includes two CPU cores, but in other embodiments CPU core complex can include an arbitrary number of CPU cores. Each of CPU cores <NUM> and <NUM> is bidirectionally connected to a system management network (SMN), which forms a control fabric, and to data fabric <NUM>, and is capable of providing memory access requests to data fabric <NUM>. Each of CPU cores <NUM> and <NUM> may be unitary cores, or may further be a core complex with two or more unitary cores sharing certain resources such as caches.

Graphics core <NUM> is a high performance graphics processing unit (GPU) capable of performing graphics operations such as vertex processing, fragment processing, shading, texture blending, and the like in a highly integrated and parallel fashion. Graphics core <NUM> is bidirectionally connected to the SMN and to data fabric <NUM>, and is capable of providing memory access requests to data fabric <NUM>. In this regard, APU <NUM> may either support a unified memory architecture in which CPU core complex <NUM> and graphics core <NUM> share the same memory space, or a memory architecture in which CPU core complex <NUM> and graphics core <NUM> share a portion of the memory space, while graphics core <NUM> also uses a private graphics memory not accessible by CPU core complex <NUM>.

Display engines <NUM> render and rasterize objects generated by graphics core <NUM> for display on a monitor. Graphics core <NUM> and display engines <NUM> are bidirectionally connected to a common memory management hub <NUM> for uniform translation into appropriate addresses in memory system <NUM>, and memory management hub <NUM> is bidirectionally connected to data fabric <NUM> for generating such memory accesses and receiving read data returned from the memory system.

Data fabric <NUM> includes a crossbar switch for routing memory access requests and memory responses between any memory accessing agent and memory controllers <NUM>. It also includes a system memory map, defined by BIOS, for determining destinations of memory accesses based on the system configuration, as well as buffers for each virtual connection.

Peripheral controllers <NUM> include a USB controller <NUM> and a SATA interface controller <NUM>, each of which is bidirectionally connected to a system hub <NUM> and to the SMN bus. These two controllers are merely exemplary of peripheral controllers that may be used in APU <NUM>.

Peripheral bus controllers <NUM> include a system controller or "Southbridge" (SB) <NUM> and a PCIe controller <NUM>, each of which is bidirectionally connected to an input/output (I/O) hub <NUM> and to the SMN bus. I/O hub <NUM> is also bidirectionally connected to system hub <NUM> and to data fabric <NUM>. Thus for example a CPU core can program registers in USB controller <NUM>, SATA interface controller <NUM>, SB <NUM>, or PCIe controller <NUM> through accesses that data fabric <NUM> routes through I/O hub <NUM>.

SMU <NUM> is a local controller that controls the operation of the resources on APU <NUM> and synchronizes communication among them. SMU <NUM> manages power-up sequencing of the various processors on APU <NUM> and controls multiple off-chip devices via reset, enable and other signals. SMU <NUM> includes one or more clock sources not shown in <FIG>, such as a phase locked loop (PLL), to provide clock signals for each of the components of APU <NUM>. SMU <NUM> also manages power for the various processors and other functional blocks, and may receive measured power consumption values from CPU cores <NUM> and <NUM> and graphics core <NUM> to determine appropriate power states.

APU <NUM> also implements various system monitoring and power saving functions. In particular one system monitoring function is thermal monitoring. For example, if APU <NUM> becomes hot, then SMU <NUM> can reduce the frequency and voltage of CPU cores <NUM> and <NUM> and/or graphics core <NUM>. If APU <NUM> becomes too hot, then it can be shut down entirely. Thermal events can also be received from external sensors by SMU <NUM> via the SMN bus, and SMU <NUM> can reduce the clock frequency and/or power supply voltage in response.

<FIG> illustrates in block diagram form a memory controller <NUM> and an associated physical interface (PHY) <NUM> suitable for use in APU <NUM> of <FIG> according to some embodiments. Memory controller <NUM> includes a memory channel <NUM> and a power engine <NUM>. Memory channel <NUM> includes a host interface <NUM>, a memory channel controller <NUM>, and a physical interface <NUM>. Host interface <NUM> bidirectionally connects memory channel controller <NUM> to data fabric <NUM> over a scalable data port (SDP). Physical interface <NUM> bidirectionally connects memory channel controller <NUM> to PHY <NUM> over a bus that conforms to the DDR-PHY Interface Specification (DFI). Power engine <NUM> is bidirectionally connected to SMU <NUM> over the SMN bus, to PHY <NUM> over the Advanced Peripheral Bus (APB), and is also bidirectionally connected to memory channel controller <NUM>. PHY <NUM> has a bidirectional connection to a memory channel such as memory channel <NUM> or memory channel <NUM> of <FIG>. Memory controller <NUM> is an instantiation of a memory controller for a single memory channel using a single memory channel controller <NUM>, and has a power engine <NUM> to control operation of memory channel controller <NUM> in a manner that will be described further below.

<FIG> illustrates in block diagram form another memory controller <NUM> and associated PHYs <NUM> and <NUM> suitable for use in APU <NUM> of <FIG> according to some embodiments. Memory controller <NUM> includes memory channels <NUM> and <NUM> and a power engine <NUM>. Memory channel <NUM> includes a host interface <NUM>, a memory channel controller <NUM>, and a physical interface <NUM>. Host interface <NUM> bidirectionally connects memory channel controller <NUM> to data fabric <NUM> over an SDP. Physical interface <NUM> bidirectionally connects memory channel controller <NUM> to PHY <NUM>, and conforms to the DFI Specification. Memory channel <NUM> includes a host interface <NUM>, a memory channel controller <NUM>, and a physical interface <NUM>. Host interface <NUM> bidirectionally connects memory channel controller <NUM> to data fabric <NUM> over another SDP. Physical interface <NUM> bidirectionally connects memory channel controller <NUM> to PHY <NUM>, and conforms to the DFI Specification. Power engine <NUM> is bidirectionally connected to SMU <NUM> over the SMN bus, to PHYs <NUM> and <NUM> over the APB, and is also bidirectionally connected to memory channel controllers <NUM> and <NUM>. PHY <NUM> has a bidirectional connection to a memory channel such as memory channel <NUM> of <FIG>. PHY <NUM> has a bidirectional connection to a memory channel such as memory channel <NUM> of <FIG>. Memory controller <NUM> is an instantiation of a memory controller having two memory channel controllers and uses a shared power engine <NUM> to control operation of both memory channel controller <NUM> and memory channel controller <NUM> in a manner that will be described further below.

<FIG> illustrates in block diagram form a memory controller <NUM> according to some embodiments. Memory controller <NUM> includes a memory channel controller <NUM> and a power controller <NUM>. Memory channel controller <NUM> includes an interface <NUM>, a queue <NUM>, a command queue <NUM>, an address generator <NUM>, a content addressable memory (CAM) <NUM>, a replay queue <NUM>, a refresh logic block <NUM>, a timing block <NUM>, a page table <NUM>, an arbiter <NUM>, an error correction code (ECC) check block <NUM>, an ECC generation block <NUM>, and a data buffer (DB) <NUM>.

Interface <NUM> has a first bidirectional connection to data fabric <NUM> over an external bus, and has an output. In memory controller <NUM>, this external bus is compatible with the advanced extensible interface version four specified by ARM Holdings, PLC of Cambridge, England, known as "AXI4", but can be other types of interfaces in other embodiments. Interface <NUM> translates memory access requests from a first clock domain known as the FCLK (or MEMCLK) domain to a second clock domain internal to memory controller <NUM> known as the UCLK domain. Similarly, queue <NUM> provides memory accesses from the UCLK domain to the DFICLK domain associated with the DFI interface.

Address generator <NUM> decodes addresses of memory access requests received from data fabric <NUM> over the AXI4 bus. The memory access requests include access addresses in the physical address space represented in a normalized format. Address generator <NUM> converts the normalized addresses into a format that can be used to address the actual memory devices in memory system <NUM>, as well as to efficiently schedule related accesses. This format includes a region identifier that associates the memory access request with a particular rank, a row address, a column address, a bank address, and a bank group. On startup, the system BIOS queries the memory devices in memory system <NUM> to determine their size and configuration, and programs a set of configuration registers associated with address generator <NUM>. Address generator <NUM> uses the configuration stored in the configuration registers to translate the normalized addresses into the appropriate format. Command queue <NUM> is a queue of memory access requests received from the memory accessing agents in data processing system <NUM>, such as CPU cores <NUM> and <NUM> and graphics core <NUM>. Command queue <NUM> stores the address fields decoded by address generator <NUM> as well other address information that allows arbiter <NUM> to select memory accesses efficiently, including access type and quality of service (QoS) identifiers. CAM <NUM> includes information to enforce ordering rules, such as write after write (WAW) and read after write (RAW) ordering rules.

Replay queue <NUM> is a temporary queue for storing memory accesses picked by arbiter <NUM> that are awaiting responses, such as address and command parity responses, write cyclic redundancy check (CRC) responses for DDR4 DRAM or write and read CRC responses for gDDR5 DRAM. Replay queue <NUM> accesses ECC check block <NUM> to determine whether the returned ECC is correct or indicates an error. Replay queue <NUM> allows the accesses to be replayed in the case of a parity or CRC error of one of these cycles.

Refresh logic <NUM> includes state machines for various powerdown, refresh, and termination resistance (ZQ) calibration cycles that are generated separately from normal read and write memory access requests received from memory accessing agents. For example, if a memory rank is in precharge powerdown, it must be periodically awakened to run refresh cycles. Refresh logic <NUM> generates refresh commands periodically to prevent data errors caused by leaking of charge off storage capacitors of memory cells in DRAM chips. In addition, refresh logic <NUM> periodically calibrates ZQ to prevent mismatch in on-die termination resistance due to thermal changes in the system.

Arbiter <NUM> is bidirectionally connected to command queue <NUM> and is the heart of memory channel controller <NUM>. It improves efficiency by intelligent scheduling of accesses to improve the usage of the memory bus. Arbiter <NUM> uses timing block <NUM> to enforce proper timing relationships by determining whether certain accesses in command queue <NUM> are eligible for issuance based on DRAM timing parameters. For example, each DRAM has a minimum specified time between activate commands, known as "tRC". Timing block <NUM> maintains a set of counters that determine eligibility based on this and other timing parameters specified in the JEDEC specification, and is bidirectionally connected to replay queue <NUM>. Page table <NUM> maintains state information about active pages in each bank and rank of the memory channel for arbiter <NUM>, and is bidirectionally connected to replay queue <NUM>.

In response to write memory access requests received from interface <NUM>, ECC generation block <NUM> computes an ECC according to the write data. DB <NUM> stores the write data and ECC for received memory access requests. It outputs the combined write data/ECC to queue <NUM> when arbiter <NUM> picks the corresponding write access for dispatch to the memory channel.

Power controller <NUM> includes an interface <NUM> to an advanced extensible interface, version one (AXI), an APB interface <NUM>, and a power engine <NUM>. Interface <NUM> has a first bidirectional connection to the SMN, which includes an input for receiving an event signal labeled "EVENT_n" shown separately in <FIG>, and an output. APB interface <NUM> has an input connected to the output of interface <NUM>, and an output for connection to a PHY over an APB. Power engine <NUM> has an input connected to the output of interface <NUM>, and an output connected to an input of queue <NUM>. Power engine <NUM> includes a set of configuration registers <NUM>, a microcontroller (µC) <NUM>, a self refresh controller (SLFREF/PE) <NUM>, and a reliable read/write timing engine (RRW/TE) <NUM>. Configuration registers <NUM> are programmed over the AXI bus, and store configuration information to control the operation of various blocks in memory controller <NUM>. Accordingly, configuration registers <NUM> have outputs connected to these blocks that are not shown in detail in <FIG>. Self refresh controller <NUM> is an engine that allows the manual generation of refreshes in addition to the automatic generation of refreshes by refresh logic <NUM>. Reliable read/write timing engine <NUM> provides a continuous memory access stream to memory or I/O devices for such purposes as DDR interface maximum read latency (MRL) training and loopback testing.

Memory channel controller <NUM> includes circuitry that allows it to pick memory accesses for dispatch to the associated memory channel. In order to make the desired arbitration decisions, address generator <NUM> decodes the address information into predecoded information including rank, row address, column address, bank address, and bank group in the memory system, and command queue <NUM> stores the predecoded information. Configuration registers <NUM> store configuration information to determine how address generator <NUM> decodes the received address information. Arbiter <NUM> uses the decoded address information, timing eligibility information indicated by timing block <NUM>, and active page information indicated by page table <NUM> to efficiently schedule memory accesses while observing other criteria such as QoS requirements. For example, arbiter <NUM> implements a preference for accesses to open pages to avoid the overhead of precharge and activation commands required to change memory pages, and hides overhead accesses to one bank by interleaving them with read and write accesses to another bank. In particular during normal operation, arbiter <NUM> normally keeps pages open in different banks until they are required to be precharged prior to selecting a different page.

<FIG> illustrates in block diagram form a memory controller <NUM> that can be used to implement a portion of memory controller <NUM> of <FIG> according to some embodiments. Memory controller <NUM> implements address decoder <NUM> of <FIG> as a non power-of-two address decoder <NUM> to accommodate non-traditional memory sizes. For example, memory controller <NUM> is able to receive the request from data fabric <NUM> and programmably map it onto non power-of-two memory sizes, such as 6GB DIMMs. This operation will be explained in greater detail below.

Memory controller <NUM> has an input port for receiving memory access requests from data fabric <NUM> using the SDP. Each memory access request includes a set of control signals labeled "CONTROL", a <NUM>-bit address labeled "NORMALIZED ADDRESS", and a set of <NUM> data signals labeled "DATA". The CONTROL signals include a tag for the access request, the size of the request, the quality of service requested, the type of access such as read or write, and so on. The NORMALIZED ADDRESS includes all of the supported address bits and is a <NUM>-bit address with implied leading zeros appended. The DATA signals include a sufficient number of signals that are associated with a single memory access request. For example, a CPU core may include a last level cache that has a <NUM>-bit cache line size; thus a writeback of a cache line to memory will require a <NUM>-bit transfer. The physical interface such as PHY <NUM> of <FIG> may perform a corresponding memory access as a burst of eight to a <NUM>-bit or <NUM>-bit (<NUM> bits plus <NUM> bits of error correcting code) DDR DIMM, but the memory controller receives all <NUM> DATA bits as part of the memory access request.

Memory controller <NUM> includes non power-of-two address decoder <NUM> and write data buffer <NUM> as previously illustrated in <FIG>. Address decoder <NUM> includes an input for receiving the NORMALIZED ADDRESS, and outputs for providing a set of n chip select signals labeled "CS[n]", a set of three chip identification signals labeled "CHIP_ID", a decoded row address labeled "ROW_ADDRESS", a decoded column address labeled "COLUMN_ADDRESS", a bank group signal labeled "BG", and a bank address signal labeled "BA". Memory controller <NUM> provides these outputs of address decoder <NUM> along with the CONTROL signals to command queue <NUM> to allow command queue <NUM> to store them so that arbiter <NUM> can make decisions about the efficient ordering of memory access requests. Memory controller <NUM> also provides the COLUMN_ADDRESS, BG, and BA signals to page table <NUM> to allow page table <NUM> to associate access requests with open pages in each DRAM chip.

Write data buffer <NUM> is a holding buffer having an input for receiving the <NUM>-bit DATA signal, and an output connected to an input of BEQ <NUM>. Since data fabric <NUM> provides interspersed read and write memory access requests to memory controller <NUM>, write data buffer <NUM> will not be used for all received memory access requests but only for writes.

In operation, the system BIOS queries the serial presence detect (SPD) ROM on each memory module of memory channels <NUM> and <NUM> at boot-up to determine their respective densities and organizations. The system BIOS uses this information to program configurable address decoder registers of address decoder <NUM> to define the address map for a given workload and memory chip configuration. The system BIOS also makes this information available to the operating system to allow it to program page tables used for virtual address translation from logical addresses to physical addresses, which is the format of the NORMALIZED ADDRESS. After the registers are configured by the system BIOS, address decoder <NUM> uses them to decode the NORMALIZED ADDRESS to map each access request to a specific region having a corresponding chip select.

For example, if the memory is DDR4 memory having a power-of two size, address decoder <NUM> decodes the NORMALIZED ADDRESS into various output signals as shown in TABLE I below:.

To operate with some DIMMs, memory controller <NUM> also supports a feature known as rank multiplication. In systems with rank multiplication, each packaged integrated circuit on a given DIMM includes a three-dimensional (3D) stack of memory chips interconnected using through-silicon-via (TSV) technology. For example the DDR4 standard specifies a <NUM>-bit chip identification input signal C[<NUM>:<NUM>] to support stacks of <NUM>, <NUM>, and <NUM> memory chips. In this way each memory chip in the stack is selected by both a common chip select signal and an encoded C[<NUM>:<NUM>] signal to identify the selected logical rank within the region. To implement rank multiplication, address decoder <NUM> programmably decodes the NORMALIZED ADDRESS into logical ranks and activates a one-hot chip select signal for the selected region and also provides the encoded C[<NUM>:<NUM>] signal corresponding to the selected logical rank.

Address decoder <NUM> supports non power-of-two address decoding. The construction of address decoder <NUM> will now be described.

<FIG> illustrates in block diagram form a simplified block diagram of non-power-of-two decoder <NUM> of <FIG> according to some embodiments. Non power-of-two address decoder <NUM> includes a set of region decoders <NUM> associated with different ranks defined by a corresponding chip select signal. In the example illustrated in <FIG>, address decoder <NUM> includes four region decoders <NUM> associated with four chip select signals respectively labeled "CS0", "CS1", "CS2", and "CS3". Each region decoder <NUM> includes a primary decoder <NUM>, a secondary decoder <NUM>, a logic circuit labeled "OR" <NUM>, a first set of configuration registers <NUM> associated with primary decoder <NUM> labeled "CFG", and a second set of similarly labeled configuration registers <NUM> associated with secondary decoder <NUM>. Note that configuration registers <NUM> and <NUM> are logically associated with primary decoder <NUM> and secondary decoder <NUM>, respectively, and may either be physically distinct or may be combined with other configuration registers in a central register set such as configuration registers <NUM> of <FIG>.

Each of configuration registers <NUM> and <NUM> has an input connected to the SMN bus, and an output for providing register values for use by a respective one of primary decoder <NUM> and secondary decoder <NUM>. Primary decoder <NUM> has a first input for receiving the NORMALIZED ADDRESS, a second input connected to the output of configuration registers <NUM>, and an output for providing a primary chip select signal. The primary chip select signals are labeled "CSPRI0", "CSPRI1", "CSPRI2", and "CSPRI3", respectively. Secondary decoder <NUM> has a first input for receiving the NORMALIZED ADDRESS, a second input connected to the output of configuration registers <NUM>, and an output for providing a primary chip select signal. The primary chip select signals are labeled "CSSEC0", "CSSEC1", "CSSEC2", and "CSSEC3", respectively. Logic circuit <NUM> has a first input connected to the output of primary decoder <NUM>, a second input connected to the output of secondary decoder <NUM>, and an output for providing a respective one of signals "CS0", "CS1", "CS2", and "CS3".

Each set of configuration registers <NUM> and <NUM> includes several registers sufficient to define the attributes of the region such that the NORMALIZED ADDRESS can be decoded and mapped to the region. In one example, a base address register defines the starting address of the region and corresponds to the lowest address in the region, whereas an address mask register defines the size of the region and thus identifies significant bits to be used in the decoding. Each decoder compares the significant bits of the NORMALIZED ADDRESS, masked according to the address mask register, to the corresponding bits of the base address register. If there is a match, then the decoder outputs its respective chip select signal, and logic circuit <NUM> outputs a final chip select signal. For active high chip select signals, logic circuit <NUM> is implemented using a logical OR function.

In one embodiment, each primary decoder supports regions of size of <NUM>N, and each secondary decoder supports regions of size <NUM>(N-<NUM>), where N is an integer. For example, if N is equal to <NUM>, then primary decoder <NUM> supports a region size of 4GB and secondary decoder supports a region size of 2GB, for a total region size of 6GB.

By providing both a primary and secondary decoder, assigned to the same region and combining their results, region decoder <NUM> supports non power-of-two memory sizes without complicated bit-by-bit decoding, thereby reducing the size of the decoders. Since each of the primary and secondary decoders have a power-of-two size, they can perform region decoding on a subset of the NORMALIZED ADDRESS bits quickly and efficiently using a compact circuit. By reducing the number of bits required in the decoding operation, address decoder <NUM> is able to decode addresses faster. For example a full bit-by-bit comparison of <NUM> bits of the <NUM>-bit NORMALIZED ADDRESS to base and limit registers of an arbitrary region size would require more than a single clock cycle to resolve for higher clock rates using contemporary CMOS logic processes.

While address decoder <NUM> can be used to support non power-of-two region sizes with both a primary decoder and a secondary decoder, additional configurations are possible according to other embodiments. For example, each address decoder could include a primary decoder, a secondary decoder, and a tertiary decoder that have respective sizes of <NUM>N, <NUM>(N-<NUM>), and <NUM>(N-<NUM>). For example if N is equal to <NUM>, this configuration allows the decoding of normalized addresses into region sizes of and of <NUM>-7GB in 1GB increments. This concept could be further extended to four or more decoders as well.

<FIG> illustrates a diagram <NUM> showing the address mapping performed by the address decoder of <FIG> when programmed for a non-power-of-two address space using two region decoders. Diagram <NUM> includes a normalized address space <NUM>, a first region <NUM>, and a second region <NUM>. Normalized address space <NUM> has a 4GB sub-region <NUM> associated with decoded signal CSPRI0, a 4GB sub-region <NUM> associated with decoder signal CSPRI1, a 2GB sub-region <NUM> associated with decoded signal CSSEC0, and a 2GB sub-region associated with decoded signal CSSEC1. Sub-region <NUM> starts at address 0x0 and extends to address 0x0_FFFF_FFFF, which is <NUM> - <NUM> (<NUM><NUM> - <NUM>), in which 0x indicates a <NUM>-bit hexadecimal address with implied leading zeros. Sub-region <NUM> starts at address 0x1_0000_0000 (<NUM><NUM>) and extends to address 0x1_FFFF_FFFF, which is <NUM> - <NUM> (<NUM><NUM> + <NUM><NUM> - <NUM>). Sub-region <NUM> starts at address 0x2_0000_0000 (<NUM>) and extends to address 0x2_7FFF_FFFF (<NUM> - <NUM>). Sub-region <NUM> starts at address 0x2_8000_0000 (<NUM>) and extends to address 0x2_FFFF_FFFF (<NUM> - <NUM>). First region <NUM> is a 6GB region associated with CS0 and has a 4GB primary portion <NUM> and a 2GB secondary portion <NUM>. First region <NUM> is implemented with <NUM> (<NUM>,<NUM>) rows of <NUM> (<NUM>,<NUM>) bytes each, in which a primary region <NUM> is implemented with <NUM> (<NUM>,<NUM>) rows extending from row address 0x0 to row address 0x7FFF, and a secondary region <NUM> is implemented with <NUM> rows extending from row address 0x8000 to 0xbFFF. Likewise second region <NUM> is implemented with <NUM> rows of <NUM> bytes each, in which a primary region <NUM> is implemented with <NUM> rows extending from row address 0x0 to row address 0x7FFF, and a secondary region <NUM> is implemented with <NUM> rows extending from row address 0x8000 to 0xbFFF.

To perform this decoding operation, the system BIOS programs configuration registers as shown in TABLE II:.

Known memory controllers use an additional interleaving mode known as bank swizzle mode in which certain bits of the input address are decoded to form the bank address. These particular bits include certain bits of the access address such as low-order row address bits to generate new bank (or in the case of DDR4 both bank and bank group) bits. In this way different portions of a set of contiguous addresses that would have otherwise caused page conflicts are divided between banks, resulting in greater efficiency.

According to some embodiments, a memory controller as described herein includes programmable mechanisms to interleave the physical address space across a set of distinct regions. In this way, a memory controller as described above, for example memory controller <NUM> or <NUM> of <FIG> or memory controller <NUM> of <FIG>, can operate more efficiently by spreading a series of accesses that may be encountered during execution of a program across multiple ranks of DRAM. Thus overhead cycles such as page precharges and page activates can be hidden within useful cycles. The first mechanism is chip select interleaving that can be accomplished using the primary and secondary region decoders.

<FIG> illustrates a diagram <NUM> showing the address mapping performed by the address decoder of <FIG> when programmed to implement chip select interleave using two address decoders according to some embodiments. Diagram <NUM> includes a normalized address space <NUM>, a first region <NUM>, and a second region <NUM>. Normalized address space <NUM> has a 2GB sub-region <NUM> associated with decoded signal CSPRI0, a 2GB sub-region <NUM> associated with decoder signal CSPRI1, a 2GB sub-region <NUM> associated with decoded signal CSSEC0, and a 2GB sub-region <NUM> associated with decoded signal CSSEC1. Sub-region <NUM> starts at address 0x0 and extends to address 0x0_7FFF_FFFF, which is <NUM> - <NUM> (<NUM><NUM>- <NUM>). Sub-region <NUM> starts at address 0x8_0000_0000 (<NUM><NUM>) and extends to address 0x0_FFFF_FFFF, which is <NUM> - <NUM> (<NUM><NUM> + <NUM><NUM>- <NUM>). Sub-region <NUM> starts at address 0x1_0000_0000 (4GB) and extends to address 0x1_7FFF_FFFF (<NUM> - <NUM>). Sub-region <NUM> starts at address 0x1_8000_0000 (<NUM>) and extends to address 0x1_FFFF_FFFF (<NUM> - <NUM>). First region <NUM> is a 4GB region associated with CS0 and has a 2GB portion <NUM> and a 2GB portion <NUM>. First region <NUM> is implemented with <NUM> rows of <NUM> bytes each, in which a primary region <NUM> is implemented with <NUM> rows extending from row address 0x0 to row address 0x3FFF, and a secondary region <NUM> is implemented with <NUM> rows extending from row address 0x4000 to 0x7FFF. Likewise second region <NUM> is implemented with <NUM> rows of <NUM> bytes each, in which a primary region <NUM> is implemented with <NUM> rows extending from row address 0x0 to row address 0x3FFF, and a secondary region <NUM> is implemented with <NUM> rows extending from row address 0x4000 to 0x7FFF.

In the chip select interleave mode, memory controller <NUM> interleaves the physical address space over multiple DIMM ranks on a channel, as opposed to a single DIMM rank occupying a contiguous set of addresses in the normalized address space. Chip select (CS) interleave reduces page conflicts as potentially more DRAM banks can be used over a smaller address region, effectively making more DRAM banks available. To use chip select interleave, there are two requirements. First, the number of interleaved chip select signals is a power of two. Second, the regions are the same size. In the example in <FIG>, there are two (<NUM><NUM>) chip selects for two regions having the same size (4GB).

Memory controller <NUM> programmably implements chip select interleave by swapping upper order normalized address bits used to select a region and chip select signal with lower order bits of the normalized address corresponding to the desired interleave size. CS interleave mode can be configured by setting the BaseAddrCS and AddrMaskCS registers to indicate the size of interleave. For example, if interleaving only the primary decoders and then only the secondary decoders on a 1024KB normalized address range in a two-CS system, the register settings will be as shown in TABLE III below:.

Thus regions <NUM> and <NUM> are now two-way interleaved across addresses 0x0 to 0x0_FFFF_FFFF, and address bit <NUM> determines whether CS0 or CS1 is used. Likewise, regions <NUM> and <NUM> are also two-way interleaved across addresses 0x1_0000_0000 to 0x1_FFFF_FFFF, and address bit <NUM> again determines whether CS0 or CS1 is used.

In the example shown in TABLE I, address bit A[<NUM>] (corresponding to a size of <NUM> KB) was mapped to column address bit <NUM> (COL[<NUM>]). This mapping allows the firmware additional flexibility when implementing interleaving. In the current example A[<NUM>] is used for CS interleaving, and COL[<NUM>] is mapped instead to A[<NUM>], and higher order address bits are used for bank and bank group addresses, namely A[<NUM>] for BA0, A[<NUM>] for BA1, etc. according to the pattern of TABLE I.

The second mechanism is known as hashing. Hashing provides a finer granularity by allowing more address bits to be used. The number and location of the address bits used in hashing can be programmed by the user, providing flexibility so that the hashing operation can be tailored for the specific application and the characteristics of the software code.

<FIG> illustrates in block diagram form a portion of a memory controller <NUM> that can be used to implement address decoder <NUM> of <FIG> according to some embodiments. Memory controller <NUM> includes an address decoder <NUM> and a set of configuration registers <NUM>. Address decoder <NUM> includes a chip select (CS) hashing circuit <NUM> and a bank address (BA) hashing circuit <NUM>. CS hashing circuit <NUM> has an input for receiving the <NUM>-bit NORMALIZED ADDRESS from interface <NUM>, and an output for providing a set of hashed chip select signals labeled "CS_HASH". BA hashing circuit <NUM> has a first input connected to the output of CS hashing circuit <NUM>, a second input for receiving the NORMALIZED ADDRESS, and outputs for providing the CHIP_ID, ROW_ADDRESS, COLUMN_ADDRESS, BG, and BA_HASH signals. Memory controller <NUM> provides decoded memory accesses using these outputs along with the CONTROL signals to command queue <NUM> to allow command queue <NUM> to store them so that arbiter <NUM> can make decisions about the efficient ordering of memory access requests.

Address decoder <NUM> receives fields from various configuration registers for the base address and size of each memory chip in the memory channel as described above. In addition, address decoder <NUM> uses additional configuration registers to support a programmable hashing function, and <FIG> shows only these additional configuration registers. In the illustrated embodiment, address decoder <NUM> supports up to four chip select signals and up to thirty-two banks. Thus configuration registers <NUM> include a set of two CS hash registers <NUM> each corresponding to a bit of the encoded CS signal, and a set of five BA hash registers <NUM>, each corresponding to a bit of the encoded BA signal.

CS hash registers <NUM> include a first CS hash register <NUM> associated with CS_HASH[<NUM>] and a second CS hash register <NUM> associated with CS_HASH[<NUM>]. Each CS hash register is a <NUM>-bit register that includes a <NUM>-bit field labeled "NORMALIZED ADDRESS XOR[<NUM>:<NUM>]" in register bits <NUM>:<NUM> and an enable field labeled "EN" in register bit <NUM>. BA hash registers <NUM> include BA hash registers <NUM>, <NUM>, <NUM>, <NUM>, and <NUM> associated with hashed bank address bits BA_HASH[<NUM>], BA_HASH[<NUM>], BA_HASH[<NUM>], BA_HASH[<NUM>], and BA_HASH[<NUM>], respectively. Each BA hash register is a thirty-two bit register with three fields: an <NUM>-bit row exclusive-OR (XOR) field labeled "ROWXOR[<NUM>:<NUM>]" in register bits <NUM>:<NUM>, a <NUM>-bit column XOR field labeled "COLXOR[<NUM>:<NUM>]" in register bits <NUM>:<NUM>, and an enable field labeled "EN" in register bit <NUM>.

Address decoder <NUM> performs bitwise XOR operations using selected bits of the NORMALIZED ADDRESS. CS hashing circuit <NUM> first hashes the chip select bits using selected ones of the most significant thirty-one bits of the NORMALIZED ADDRESS. Each bit of the NORMALIZED ADDRESS XOR field of the CS hash register is used to selectively perform a bitwise exclusive OR (XOR) operation on the indicated bits of the NORMALIZED ADDRESS. The two chip select signals are hashed according to equations [<NUM>] and [<NUM>] below: <MAT> <MAT> in which ^ represents the XOR operator, and ^() represents the bitwise XOR operator on respective pairs of bits.

Memory decoder <NUM> first locates the CS[<NUM>:<NUM>] bits based on the size of the memory. It then performs CS hashing to calculate the CS_HASH values using equations [<NUM>] and [<NUM>]. After CS hashing circuit <NUM> determines the hashed CS_HASH values, BA hashing circuit <NUM> performs BA hashing to calculate the BA_HASH values using equations [<NUM>]-[<NUM>]: <MAT> <MAT><MAT><MAT><MAT> Note that the NORMALIZED ADDRESS bits corresponding to the CS bits cannot themselves be used to hash the CS bits, because otherwise it would force all CS_HASH values to be <NUM>. An additional restriction on setting these register values will be described further below.

In some embodiments, the CS and BA hashing functions can be extended to additional levels of memory organization. For example, HBM memories implement a concept known as a "pseudo channel". The pseudo channel can be also hashed using a corresponding hashing equation and a corresponding pseudo channel register, as described in Equation [<NUM>] below: <MAT> In this case, the memory controller uses an additional hashing circuit and an additional configuration register.

Known memory controllers only hash bank addresses and use a fixed hashing function. Memory controller <NUM> provides two additional mechanisms to increase its flexibility. First, memory controller <NUM> selectively hashes chip selects to allow a greater flexibility in dividing accesses. For example, a memory with four ranks and four chip selects can be used to spread a set of proximal memory accesses more widely over four times more memory banks. This wider spreading allows memory controller <NUM> to hide overhead better. Second, memory controller <NUM> allows the hashing function itself to be programmable and therefore changeable to better fit the processing environment. For example, mobile systems tend to run a smaller number of tasks and to use a higher code and data concentration in the physical address space than desktop systems or servers, and therefore would benefit from a more complex hashing algorithm to ensure that more memory accesses to relatively small areas of physical memory are spread more widely across multiple chips and banks. On the other hand, desktop and server systems tend to be more multi-tasked and multi-threaded, so a simpler hashing algorithm may be sufficient. In either case, the hashing algorithm is programmable through a set of hashing registers that can be selectively programmed by the system BIOS.

<FIG> illustrates in block diagram form another portion of a memory controller <NUM> that can be used to implement address decoder <NUM> of <FIG> according to some embodiments. As shown here memory controller <NUM> includes an address decoder <NUM> having a hashing circuit <NUM> followed by a non-power-of-two decoder <NUM>. Hashing circuit <NUM> has an input for receiving an access address and an output and can be implemented with hashing circuit <NUM> of <FIG>. Non-power-of-two decoder <NUM> has an input connected to the output of hashing circuit <NUM>, and an output for providing a decoded address, and can be implemented with non-power-of-two decoder circuit <NUM> of <FIG> and <FIG>. Memory controller <NUM> illustrates that not only can the hashing mechanism and the non-power-of-two memory size decoder be implemented separately, they also can be used together in a single memory decoder <NUM>. In this case, the hashing mechanism seamlessly precedes the non-power-of-two decoding to provide a memory controller with further enhanced flexibility by supporting both functions.

The hashing operation can also be used seamlessly with non power-of-two memory sizes. The way in which hashing circuit <NUM> performs the hashing operation with a non-power-of-two memory address size can be described with respect to a particular example. In this example, a dual-rank <NUM> Gb memory is implemented using two region decoders, in which the first region decoder for CS0 maps to <NUM> Gb of the memory space using a primary decoder (CS0p) and a secondary decoder (CS0s), and a second region decoder for CS1 maps to <NUM> Gb of the memory space using a primary decoder (CS1p) and a secondary decoder (CS1s). In this example, the interleaving occurs in a straightforward fashion:.

When the system BIOS configures non-power-of-two decoder <NUM> to set up a non-power-of-two size, there is an additional restriction on the hashing function due to the DRAM architecture. For example according to the configuration illustrated in <FIG> and <FIG>, there are only <NUM> rows, and the ROWXOR bits corresponding to Row[MSB:MSB-<NUM>] should not be enabled for the hashing operation because it would not correctly translate the three states into the correct number of bank states. Instead these bits can only be used in memories with a power-of-two size.

Therefore the memory controller described above is able to perform flexible address mapping through a variety of configurable options, providing the user a range of choices. These address mapping choices include support for non power-of-two memory sizes, interleaving, and hashing, all of which are implemented in one combined decoder (address decoder <NUM>).

The memory controller of <FIG> and <FIG> may be implemented with various combinations of hardware and software. For example decoder <NUM> may be implemented with hardware circuitry for speed and efficiency purposes. This hardware circuitry may include priority encoders, finite state machines, programmable logic arrays (PLAs), and the like. In some embodiments, other functional blocks of memory controller <NUM> can be performed by a data processor under the control of software. Some of the software components may be stored in a computer readable storage medium for execution by at least one processor, and may correspond to instructions stored in a non-transitory computer memory or computer readable storage medium. In various embodiments, the non-transitory computer readable storage medium includes a magnetic or optical disk storage device, solid-state storage devices such as Flash memory, or other non-volatile memory device or devices. The computer readable instructions stored on the non-transitory computer readable storage medium may be in source code, assembly language code, object code, or other instruction format that is interpreted and/or executable by one or more processors.

Memory controller <NUM> of <FIG> or address decoder <NUM> of <FIG> or any portions thereof may be described or represented by a computer accessible data structure in the form of a database or other data structure which can be read by a program and used, directly or indirectly, to fabricate integrated circuits. For example, this data structure may be a behavioral-level description or register-transfer level (RTL) description of the hardware functionality in a high level design language (HDL) such as Verilog or VHDL. The description may be read by a synthesis tool which may synthesize the description to produce a netlist comprising a list of gates from a synthesis library. The netlist includes a set of gates that also represent the functionality of the hardware comprising integrated circuits. The netlist may then be placed and routed to produce a data set describing geometric shapes to be applied to masks. The masks may then be used in various semiconductor fabrication steps to produce the integrated circuits. Alternatively, the database on the computer accessible storage medium may be the netlist (with or without the synthesis library) or the data set, as desired, or Graphic Data System (GDS) IIdata.

Claim 1:
An apparatus (<NUM>/<NUM>) having a memory controller (<NUM>), the memory controller (<NUM>) comprising:
a host interface (<NUM>) for receiving memory access requests, said memory access requests including access addresses;
a memory interface (<NUM>) for providing memory accesses to a memory system (<NUM>);
an address decoder (<NUM>/<NUM>) coupled to said host interface (<NUM>) for programmably mapping said access addresses to selected ones of a plurality of regions, wherein said address decoder (<NUM>/<NUM>) is programmable to map said access addresses to a first region having a non-power-of-two size using a primary decoder (<NUM>) receiving said access addresses and having a first power-of-two size and a secondary decoder (<NUM>) receiving said access addresses and having a second power-of-two size different from said first power-of-two size, and provides a first region mapping signal in response;
a command queue (<NUM>) coupled to said address decoder (<NUM>/<NUM>) for storing said memory access requests and region mapping signals; and
an arbiter (<NUM>) for picking said memory access requests from said command queue (<NUM>) based on a plurality of criteria, said plurality of criteria evaluated based in part on said region mapping signals, and providing corresponding memory accesses to said memory interface (<NUM>) in response, characterised in that:
said address decoder (<NUM>/<NUM>) comprises a plurality of region decoders (<NUM>), wherein for each region decoder (<NUM>):
said primary decoder (<NUM>) is for receiving said access addresses and providing a primary region select signal, wherein said primary decoder (<NUM>) has a first base address and a first power-of-two size;
said secondary decoder (<NUM>) is for receiving said access addresses and providing a secondary region select signal, wherein said secondary decoder (<NUM>) has a second base address and a second power-of-two size; and
a region decoder (<NUM>) further comprises a logic circuit (<NUM>) for activating a corresponding region mapping signal in response to an activation of at least one of said primary region select signal and said secondary region select signal.