BUFFER COMMUNICATION FOR DATA BUFFERS SUPPORTING MULTIPLE PSEUDO CHANNELS

A memory module has data buffers coupled to a registered clock driver (RCD) via buffer communication (BCOM) bus. The memory module includes memory devices managed as a first pseudo channel and a second pseudo channel. The data buffers manage data transmission between the memory devices and a host based on commands received over the BCOM bus. The RCD can send a first BCOM command on the BCOM bus to the data buffer, the first BCOM command to specify a rank and a burst length for the first pseudo channel. The RCD can send a second BCOM command on the BCOM bus to the data buffer, the second BCOM command to specify a rank and a burst length for the second pseudo channel, and a timing offset relative to the first BCOM command.

FIELD

Descriptions are generally related to memory subsystems, and more particular descriptions are related to communication to memory module data buffers.

BACKGROUND

System memory in computer systems is often provided with a DIMM (dual inline memory module) that includes multiple DRAM (dynamic random access memory) devices. To reduce the loading on the system memory bus by the DRAM devices, LRDIMMs (load reduced DIMM) can be used, which includes an RCD (registered clock driver) and multiple data buffers. The RCD receives the commands and passes a subset of the commands to the data buffers to manage the data transmission between the DRAM devices and the host.

The RCD communications with the data buffers via a BCOM (buffer communication) bus, which can indicate the specific command (e.g., Read and Write commands). The BCOM commands are traditionally sent with very specific timing to control exactly when the data buffers transfer the data.

There are LRDIMM implementations that divide the devices on the DIMM into two pseudo channels that can transfer data simultaneously and improve data throughput. The data buffers for the pseudo channels time multiplex the data from both pseudo channels onto the host data bus. With two pseudo channels, the BCOM bus needs to provide twice as much information to the data buffers to manage two separate channels.

Traditional BCOM commands do not have enough bits to indicate all possible operating modes for two separate channels. Thus, prior implementations of pseudo channels have had to sacrifice either burst on the fly operation, the use of two ranks per pseudo channel, or non-target ODT. Changing the BCOM structure to include more bits would limit cross- compatibility of system designs, increasing complexity and cost.

Descriptions of certain details and implementations follow, including non-limiting descriptions of the figures, which may depict some or all examples, and well as other potential implementations.

DETAILED DESCRIPTION

As described herein, a memory module has data buffers coupled to a registered clock driver (RCD) via buffer communication (BCOM) bus. The memory module includes memory devices managed as a first pseudo channel and a second pseudo channel. The data buffers manage data transmission between the memory devices and a host based on commands received over the BCOM bus.

The BCOM commands can be separated for Read/Write transactions to serve both pseudo channels with one command. Thus, even though the commands provide information to multiple channels, the system can maintain the same number of signal lines on the BCOM bus and the same number of bits in the BCOM commands. The system can effectively provide more information with the same number of bits by applying a different protocol or interpretation of the bits in the BCOM commands. The different command layout or command protocol can leverage the fact that the timing of the commands provides information that can be inferred, and thus certain information does not need explicit bits to indicate as it can be inferred.

The use of a different protocol can increase the configurations that can be supported. The protocol can allow the BCOM commands to signal a read/write command for the same start time for separate pseudo channels. The RCD can send a first BCOM command on the BCOM bus to the data buffer, the first BCOM command to specify a rank and a burst length for the first pseudo channel. The RCD can send a second BCOM command on the BCOM bus to the data buffer, the second BCOM command to specify a rank and a burst length for the second pseudo channel, and a timing offset relative to the first BCOM command.

For purposes of description herein, reference is made to DRAM (dynamic random access memory) devices and DIMMs (dual inline memory modules). More specific examples are directed to load reduced (LRDIMMs). Reference to an LRDIMM or a memory module will be understood as referring to a module or unit that includes multiple DRAM devices accessed through one or more data buffers. The DRAM devices on the module can be managed as multiple pseudo channels, where the BCOM commands to the data buffer enable the data buffer to manage the access to the DRAM devices with desired timing and configurations. In addition to DIMMs, other types of memory module that allow for the parallel connection of memory devices can be used, such as a multichip package (MCP) with multiple memory devices in a stack.

In one specific example, the use of DRAM devices in an LRDIMM as multiple pseudo channels can be governed by a standard. An application of an LRDIMM with DDR5(double data rate version 5, JESD79-5, originally published by JEDEC (Joint Electron Device Engineering Council) in July 2020)) DRAMs can be defined for an MCR (muxed combined ranks) configuration. In the MCR configuration, the DRAMs can be configured in ranks (e.g., devices on the front and devices on the back of the DIMM board), with multi-channel LRDIMMs (e.g., Channel0and Channel1or Channel A and Channel B), as well as being divided in pseudo channels (e.g., PS[0] (Pseudo channel0) and PS[1] (Pseudo channel1).

The host memory controller is aware of the configuration of memory as channels and ranks. The host memory controller is aware of the configuration of the memory as pseudo channels, and sends separate commands to the RCD for each pseudo channel. The commands are time multiplexed on the CA (command and address) bus from the host to the RCD. In one example, the host sends commands for PS[0] on even clocks and commands for PS[1] on odd clocks. The command rate on the CA bus from the host to the RCD can be double the rate of the RCD to the DRAMs to enable the host to send a command to each pseudo channel on every DRAM clock. The pseudo channels are described in more detail below.

FIG. 1is a block diagram of an example of a system with a memory module having a buffer communication bus. System100illustrates memory coupled to a host. Host110represents a host computing system. Host110includes host hardware such as processor112and memory controller120. The host hardware also includes hardware interconnects and driver/receiver hardware to provide the interconnection between host110and memory module140. Memory module140represents a DIMM or LRDIMM or other multidevice package with memory devices coupled to host110. Memory module140includes data buffers144to buffer data for data access to DRAMs142. Memory controller120controls access from the host side to DRAMs142of memory module140. RCD150can control access to DRAMs142on memory module140.

The host hardware supports the execution of host software on host110. The host software can include host OS (operating system). The host OS represents a software platform under which other software will execute. During execution, software programs, including the host OS, generate requests to access memory. The requests can be directly from host OS software, from other software programs, from requests through APIs (application programming interfaces), or other mechanisms. In response to a host memory access request, memory controller120can generate a memory access request for memory module140.

In one example, memory controller120includes command logic122, which represents logic in memory controller120to generate commands to send to the memory devices of memory module140. The commands can include Read commands for Read transactions and Write commands for Write transactions. Memory controller120includes scheduler124to schedule how commands will be sent to the memory devices of memory module140, including controlling the timing of the commands.

Memory controller120includes I/O (input/output)132, which represents interface hardware to interconnect host110with memory. I/O134represents interface hardware on memory module140to interconnect with host110. I/O132and I/O134can have one or more system buses to interconnect them. System100represents data136and command (CMD)138between I/O132and I/O134. Data136represents a data bus, which is typically a bidirectional point to point bus, where the collection of the signal lines to the individual data buffer144is collectively referred to as the data bus. Command138represents a command bus or command and address (CA) bus, which is typically a unidirectional multidrop bus from the host to the memory.

Memory module140includes multiple DRAMs142, which represent memory devices. Memory module140includes data buffers144, which buffer data136between DRAMs142and host110. Data162represents the data bus signal lines on memory module140from I/O134to data buffers144and from the data buffers to DRAMs142. Command (CMD)164represents the signal lines on memory module140from I/O134to RCD150. Command (CMD)168represents signal lines on memory module140from RCD150to DRAMs142to provide command and device selection (e.g., chip select (CS)) signals. BCOM (buffer communication)166represents signal lines from RCD150to data buffers144to control the operation of the data buffers for memory access commands involving the exchange of data (i.e., read and write commands).

RCD150receives commands from host110and generates commands on memory module140to memory devices to which the host commands are directed. Logic152can represent control logic within RCD150to control the retiming of command signals. Logic152can represent control logic within RCD150to control the operation of data buffers144. More specifically, logic152can generate BCOM commands to control the operation and the timing of data buffers144.

In one example, logic152includes firmware or software logic. In one example, logic152includes hardware logic. In one example, logic152includes a combination of hardware and software/firmware logic.

In contrast to a traditional BCOM implementation that has limitations on the features and configurations that can be supported by the application of pseudo channels on memory module140, RCD150, through logic152, can generate BCOM commands that can remove at least some of the traditional limitations. In one example, memory module140can support the use of pseudo channels with burst on the fly and multiple ranks per pseudo-channel. In one example, memory module140can support the use of pseudo-channels with non-target ODT (on die termination) control.

BCOM166can be referred to as a BCOM bus. The BCOM commands can direct data buffers144to control the transfer of data from a first pseudo channel and a second pseudo channel implemented on memory module140. In one example, the format and the interpretation of the BCOM command can be different depending on the timing between BCOM commands on the BCOM bus. In one example, RCD150sends a first BCOM command to control the data transfer for the first pseudo channel. The first BCOM command can specify a rank and a burst length for the first pseudo channel. RCD150can send a second BCOM command to control the data transfer for the second pseudo channel. The second BCOM command can have different formats depending on its timing relative to the first BCOM command. In one example, the format of the second BCOM command is the same as the format of the first BCOM command. In one example, the second BCOM command has a different format, which specifies a rank and a burst length for the second pseudo channel, and a timing offset relative to the first BCOM command.

In some implementations, the data buffers (e.g., data buffers144) can make inferences about received BCOM commands that are received within certain timing. In one example, consecutive BCOM commands made to the same pseudo channel must be separated by a delay period (e.g., 8 clocks). Thus, for any second BCOM command sent by the RCD within the delay period of a first BCOM command, the data buffer(s) can infer that the second BCOM command is directed to the same command type and that it is directed to the other pseudo channel.

The data buffer cannot receive data on the data bus for a write at the same time as sending data on the data bus for a read, thus, a second BCOM command received before the prior data access transaction is completed must be for the same direction of data transfer, and thus, the command must be the same command type (if both are directed to commands that transfer data). Thus, when the first BCOM command specifies a read command, the data buffer can infer the second BCOM command is directed to a read command. Similarly, when the first BCOM command specifies a write command, and the data buffer can infer the second BCOM command is directed to a write command.

Additionally, if there is a requirement for a delay period between intra-pseudo-channel commands, a BCOM command received within the delay period must be for the other pseudo channel. Thus, the second format can leverage the inferences that the data buffer can make, and does not need to include certain information that can be understood by inference. In one example, the system always uses the first format unless the second command is exactly consecutive to the first command. In one example, the system always uses the second format when the second command is directly consecutive to the first command, and only uses the second format when the second command is directly consecutive to the first command, as described below in reference toFIG. 3B. In one example, the system always uses the first format unless the second command is exactly consecutive to the first command, but then the system can select to use either the first format or the second format for the second command, depending on the delay to be indicated, as described below in reference toFIG. 4.

FIG. 2Ais an example of BCOM timing for a system with pseudo channels. Diagram202represents a timing diagram of the timings for BCOM commands, such as BCOM commands sent by RCD150on BCOM bus166to data buffers144. More specifically, diagram202represents the timing for signals when the access command for the two pseudo channels is for the same clock.

Signal210represents a host clock (HOST CLK) signal on one or more signal lines, which represents the clock signal that controls the timing of command signals from the host to the RCD. System100does not specifically represent a clock signal, but the clock signal can accompany the command bus to indicate the command timing. Commands are represented as two clocks. The zeros and ones represent the fixed slots for each pseudo channel on the host bus.

Signal212represents a host command (HOST CA) signal on one or more signal lines. The two commands represented on signal212are labeled as CMD PS[0] for an access command to the first pseudo channel, which can be either a read command or a write command, and CMD PS[1] for an access command to the second pseudo channel, which will be the same type of access command as CMD PS[0]. Signal212is sent by the host or the host memory controller to the RCD (e.g., from memory controller120of host110to RCD150, over CMD138).

Signal220represents a memory clock (MEM CLK) signal on one or more signal lines, which represents the clock signal that controls the timing of BCOM command signals from the RCD to the memory devices or DRAMs (e.g., from RCD150to DRAMs142).

Signal222represents a command (CA) signal on one or more signal lines for a first pseudo (PS) channel, identified as PS[0]. Thus, signal222can represent the PS[0] CA signal, or the command signal sent by the RCD to the memory devices of the first pseudo channel. For example, RCD150could indicate CMD PS[0] of signal line212over CMD168to the DRAMs of the first pseudo channel.

Signal224represents a command (CA) signal on one or more signal lines for a second pseudo (PS) channel, identified as PS[1]. Thus, signal224can represent the PS[1] CA signal, or the command signal sent by the RCD to the memory devices of the second pseudo channel. As with the first pseudo channel command, RCD150could indicate CMD PS[1] of signal line212over CMD168to the DRAMs of the second pseudo channel.

Signal226represents a BCOM command on one or more signal lines from the RCD to the data buffer or data buffers. For example, signal226can represent BCOM commands from RCD150to data buffers144over BCOM166. Consider the following examples in diagram202based on the specific times indicated in the diagram.

At time t0, the host sends CMD PS[0] (e.g., a Read or a Write) on signal212, which triggers access to DRAMs in PS[0]. At time t1, the host sends CMD PS[1] on signal212, which triggers access to DRAMs in PS[1]. At time t2, the RCD generates the command on signal222to PS[0] with CMD PS[0] and on signal224to PS[1] with CMD PS[1]. It will be observed that diagram202represents the commands as taking two clocks or two clock cycles on the memory module from the RCD based on signal220, whereas the command on the host bus to the RCD is only one clock relative to signal220, two clock cycles for signal210. In one example, the use of two pseudo channels can allow the memory module to use a slower communication speed (e.g., half) as compared to the host bus. Thus, the commands on signal222and signal224can represent the same command as the command on signal212, but transmitted at half the speed. For identification, the command to PS[0] is illustrated with cross-hatch and the command to PS[1] is illustrated with shading.

In one example, at time t2, or approximately at the same time as t2, the RCD generates a BCOM command on signal226to the data buffer(s) that buffer the memory devices selected by the access command. In one example, there is a time delay, such as a one clock cycle delay or a two clock cycle delay, between the commands on the pseudo channel CA buses and the BCOM command. Thus, the signals on signal226can start at one or more clock cycles after time t2. In one example, the RCD determines which of the pseudo channels to signal first. The determination of which pseudo channel to signal first can be a matter of configuration, such as always signaling PS[0] first, and then signaling PS[1] (or the reverse). For purposes of diagram202, the RCD generates the BCOM command for PS[0], and then generates the BCOM command for PS[1].

In one example, at time t2, the RCD sends PS[0] BCOM0, which is the first BCOM transfer for pseudo channel PS[0]. The BCOM commands (as illustrated below) are two clock commands, with a first transfer on one clock and the second transfer on the next clock, as indicated on signal226. In one example, PS[0] BCOM0is of Format-1. The RCD sends the second part of the first BCOM transfer at time t3, represented as PS[0] BCOM1, which is the second transfer of the Format-1 signal.

At time t4, the RCD sends the second BCOM command to PS[1] on signal226. The second BCOM command is also a two cycle command, with the RCD sending the PS[1] BCOM0as the first part of the command, and sending PS[1] BCOM1as the second part of the PS[1] command at time t5. Time t4, when the RCD sends the second BCOM command, is exactly two clock cycles after the BCOM command to PS[0]. It will be understood that the command delay refers to the beginning of the first BCOM command to the beginning of the second BCOM command. Measurement by other references would result in different timing.

While the timing is illustrated and described as being “exactly two clock cycles”, it will be understood that the meaning of the timing is that the second BCOM command is sent directly after the first BCOM command, with no intervening clock cycles or timing between the BCOM commands. Thus, for a system that uses a different BCOM timing, such as one cycle commands or three cycle commands, the timing would be, respectively, one clock cycle or three clock cycles, or whatever timing would cause the second BCOM command to be sent directly after the first BCOM command without delay between the commands.

The timing of the consecutive commands can be referred to as having a separation of the number of clock cycles between the start of the first command and the start of the second command. The commands can be said to be separated by the number of clock cycles between the start of the sending the two commands. When the second command starts on the next clock cycles after the first command ends, the two command can be said to be directly consecutive.

In one example, when the RCD sends two BCOM commands consecutively without intervening delay, the format of the first BCOM command is the first format (Format-1) and the format of the second BCOM command is a second format (Format-2), as indicated with PS[0] BCOM0and PS[0] BCOM1being Format-1 and PS[1] BCOM0and PS[1] BCOM1being Format-2. Examples of differences in format are provided below. In general, the difference in the second format relative to the first format is that the second format indicates a timing offset relative to the first BCOM command. Thus, the second BCOM command (Format-2) can indicate if the BCOM command has the correct timing, whether it is supposed to align in timing with the first BCOM command (i.e., both PS[0] and PS[1] will transfer data on the same clock cycle), or whether it is supposed to be one clock cycle offset from the first BCOM command (i.e., PS[1] is supposed to start data transfer one clock cycle after PS[0] starts).

In diagram202, the Format-2 BCOM command to PS[1] will indicate that the PS[1] command has the same timing as the command for PS[0], and thus, the data for the two will be on the same clock. In one example, the Format-2 BCOM command to PS[1] will indicate a −2 clock delay, since the DRAM command (at time t2) was sent 2 clocks before the BCOM command (at time t4).

FIG. 2Bis an example of BCOM timing for a system with pseudo channels. Diagram204represents a timing diagram of the timings for BCOM commands when the access command the second pseudo channel is one clock after the command for the first pseudo channel.

Similar to diagram202, diagram204illustrates host clock (HOST CLK) signal210, host command (HOST CA) signal212, memory clock (MEM CLK) signal220, PS[0] command (PS[0] CA) signal222, PS[1] command (PS[1] CA) signal224, and BCOM command signal226. These signals can be the same signal lines. The timings illustrated also begin with time t0, which is understood as an initial time for the signaling scenario where the second pseudo channel command comes one clock after the first pseudo channel command. The timing indicators in diagram204are not to be understood the same as the timing indicators for diagram202.

At time t0, the host sends CMD PS[0] (e.g., a Read or a Write) on signal212, which triggers access to DRAMs in PS[0]. The host does not send the CMD PS[1] on the next time slot for the PS[1] pseudo channel (i.e., the ‘1’ above signal210directly after CMD PS[0]). Thus, the next time to send the command for PS[1] is at the next time slot for PS[1], or one clock cycle later, at time t2.

At time t1, before the host sends the CMD PS[1] command on signal212, the RCD generates the command on signal222to PS[0] with CMD PS[0]. In one example, at, or approximately at, the same time as t1, the RCD generates the first transfer of the BCOM command for PS[0] on signal226, indicated as PS[0] BCOM0, which is a Format-1 command.

At time t3, in response to CMD PS[1] on signal212at time t2, the RCD sends CMD PS[1] for the second pseudo channel on signal224. Time t3is a clock cycle after the RCD sent CMD PS[0] on signal222. Also at time t3, the RCD sends the second transfer of the BCOM command to PS[0] on signal226, indicated as PS[0] BCOM1, which is the second transfer of the Format-1 command.

At time t4, the RCD sends the first transfer of the BCOM command to PS[1] on signal226, indicated as PS[1] BCOM0, which is the first transfer of the command. In one example, because the RCD sends PS[1] BCOM0two clocks after PS[0] BCOM0, the PS[1] BCOM command is a Format-2 command. At time t5, the RCD the second transfer of the BCOM command to PS[1] on signal226, indicated as PS[1] BCOM1, which is the second transfer of the Format-2 command. In one example, the PS[1] BCOM command (at time t4and time t5) will indicate a delay of −1 to indicate that the DRAM command was send 1 clock before the BCOM command.

FIG. 2Cis an example of BCOM timing for a system with pseudo channels. Diagram206represents a timing diagram of the timings for BCOM commands when the access command the second pseudo channel is two clocks after the command for the first pseudo channel.

Similar to diagram202, diagram206illustrates host clock (HOST CLK) signal210, host command (HOST CA) signal212, memory clock (MEM CLK) signal220, PS[0] command (PS[0] CA) signal222, PS[1] command (PS[1] CA) signal224, and BCOM command signal226. These signals can be the same signal lines. The timings illustrated also begin with time t0, which is understood as an initial time for the signaling scenario where the second pseudo channel command comes one clock after the first pseudo channel command. The timing indicators in diagram206are not to be understood the same as the timing indicators for diagram202.

At time t0, the host sends CMD PS[0] (e.g., a Read or a Write) on signal212, which triggers access to DRAMs in PS[0]. The host sends the CMD PS[1] two clocks later, thus, not on the next time slot for the PS[1] pseudo channel (i.e., the ‘1’ above signal210directly after CMD PS[0]), but two time slots later. Thus, the host sends CMD PS[1] on signal212at time t3.

At time t1, before the host sends the CMD PS[1] command on signal212, the RCD generates the command on signal222to PS[0] with CMD PS[0]. In one example, at, or approximately at, the same time as t1, the RCD generates the first transfer of the BCOM command for PS[0] on signal226, indicated as PS[0] BCOM0, which is a Format-1 command. At time t2, the RCD sends the second transfer of the BCOM command to PS[0] on signal226, indicated as PS[0] BCOM1, which is the second transfer of the Format-1 command.

At time t4, in response to CMD PS[1] on signal212at time t3, the RCD sends CMD PS[1] for the second pseudo channel on signal224. Time t4is two clock cycles after the RCD sent CMD PS[0] on signal222.

At time t4, the RCD sends the first transfer of the BCOM command to PS[1] on signal226, indicated as PS[1] BCOM0, which is the first transfer of the command. In one example, because the RCD sends PS[1] BCOM0two clocks after PS[0] BCOM0, the PS[1] BCOM command is a Format-2 command. At time t5, the RCD the second transfer of the BCOM command to PS[1] on signal226, indicated as PS[1] BCOM1, which is the second transfer of the Format-2 command.

In one example, the PS[1] BCOM command (at time t4and time t5) will indicate a delay of 0 to indicate that the DRAM command was send 0 clocks before the BCOM command. In one example, where the BCOM command format includes a field to indicate the format of the BCOM command, the RCD would send the PS[1] BCOM command as a Format-1 command instead of a Format-2 command.

FIG. 2Dis an example of BCOM timing for a system with pseudo channels. Diagram208represents a timing diagram of the timings for BCOM commands when the access command the second pseudo channel is three clocks after the command for the first pseudo channel. The representation of three clocks later would be the same for more than three clocks later.

Similar to diagram202, diagram208illustrates host clock (HOST CLK) signal210, host command (HOST CA) signal212, memory clock (MEM CLK) signal220, PS[0] command (PS[0] CA) signal222, PS[1] command (PS[1] CA) signal224, and BCOM command signal226. These signals can be the same signal lines. The timings illustrated also begin with time t0, which is understood as an initial time for the signaling scenario where the second pseudo channel command comes one clock after the first pseudo channel command. The timing indicators in diagram208are not to be understood the same as the timing indicators for diagram202.

At time t0, the host sends CMD PS[0] (e.g., a Read or a Write) on signal212, which triggers access to DRAMs in PS[0]. The host sends the CMD PS[1] three clocks later, thus, not on the next time slot for the PS[1] pseudo channel (i.e., the ‘1’ above signal210directly after CMD PS[0]), but three time slots later. Thus, the host sends CMD PS[1] on signal212at time t3.

At time t1, the RCD generates the command on signal222to PS[0] with CMD PS[0]. In one example, at, or approximately at, the same time as t1, the RCD generates the first transfer of the BCOM command for PS[0] on signal226, indicated as PS[0] BCOM0, which is a Format-1 command. At time t2, the RCD sends the second transfer of the BCOM command to PS[0] on signal226, indicated as PS[0] BCOM1, which is the second transfer of the Format-1 command.

At time t4, in response to CMD PS[1] on signal212at time t3, the RCD sends CMD PS[1] for the second pseudo channel on signal224. At, or approximately at, time t4, the RCD sends the first transfer of the BCOM command to PS[1] on signal226, indicated as PS[1] BCOM0, which is the first transfer of the command. In one example, because the RCD sends PS[1] BCOM0more than two clocks after PS[0] BCOM0, the PS[1] BCOM command is a Format-1 command. At time t5, the RCD the second transfer of the BCOM command to PS[1] on signal226, indicated as PS[1] BCOM1, which is the second transfer of the Format-1 command. The timing of the PS[1] BCOM command is correct, and no delay indication is needed. Thus, the RCD can send a Format-1 command.

FIG. 3Ais a table representation of a traditional BCOM command format. Table310represents a format or protocol/bit indication of a traditional BCOM command for a system that provides two pseudo channels, PS[0] and PS[1]. There can be other bits of the command, which are not illustrated in table310. The BCOM command is assumed to be two clock cycles, with transfer1indicating the transfer on the first clock cycle and transfer2indicating the transfer on the second clock cycle.

Row312, row314, and row316represent BCOM bits [2:0], respectively, of the first clock of the BCOM command. Row312indicates a command (CMD) select bit, where a logic ‘0’ indicates a read or a write command (i.e., a command for which the data buffer will transfer data) and a logic ‘1’ indicates a non-data command. Row314indicates a PS[0] select bit, where a logic ‘0’ indicates PS[0] is not selected and a logic ‘1’ indicates PS[0] is selected. Row316indicates a selection between type of data command, where a logic ‘0’ indicates a write command and a logic ‘1’ indicates a read command.

Row318, row320, and row322represent BCOM bits [2:0], respectively, of the second clock of the BCOM command. Row318indicates a PS[1] select bit, where a logic ‘0’ indicates PS[1] is not selected and a logic ‘1’ indicates PS[1] is selected. Row320indicates selection of a rank or a burst length (BL) for PS[1]. Row322indicates selection of a rank or a burst length (BL) for PS[0].

It will be observed from row320and row322that the traditional BCOM command cannot indicate a rank and a burst length for a pseudo channel. Rather, the system would be configured which feature to use, and then that feature can be enabled or disable with these bits. For row320and row322, the value of the bit indicates a rank selection if rank is configured for use, or a burst length selection if burst length is configured for use. For example, rank can be indicated for a DIMM with x8DRAMs, and burst length can be indicated for x4DRAMs.

FIG. 3Bis a table representation of examples of a first BCOM command format and a second BCOM command format. In contrast to table310, a system can apply a different format for the BCOM command enabling the BCOM command to indicate a rank and a burst length for a pseudo channel. The BCOM command format can be separated into two formats, depending on the timing of the second command relative to the first command.

In contrast to the traditional BCOM command, which can select both PS[0] and PS[1] by the same command, with the new format, the RCD will send separate commands for the different pseudo channels. The new BCOM commands can provide for both the rank and burst length. In one example, the RCD normally uses Format 1 (table330), with Format 2 (Table350) reserved for the case when the RCD will send commands to PS[0] and PS[1] directly consecutive to each other (e.g., two clock cycles apart).

If a BCOM command is limited to one pseudo channel, the timing of the BCOM command for the second pseudo channel would be delayed by at least 2 clocks relative to the first pseudo channel indicated, which would provide an unacceptable limitation on the system. The use of two formats for the BCOM command can address the timing limitation, by having the second command indicate a timing offset relative to the first command, allowing the system to trigger the timing of the second command to be the same as the first command, or offset by one clock.

When the RCD sends a Read/Write BCOM command exactly two clocks after the previous Read/Write BCOM command, the data buffer can infer two things. The first is that the command must be the same command for the data transfer to occur in the same direction as the previous command. Additionally, since the timing offset for commands to the same pseudo channel has not been met, the data buffer can infer the second BCOM command is indicated for the OTHER pseudo channel; thus, whatever pseudo channel the first BCOM command indicates, the second BCOM command must indicate a command for the other pseudo channel.

In one example, to indicate the command timing offset, the second BCOM command format indicates timing offset information. In one example, the timing offset can enable the RCD to indicate that the second BCOM command is intended to have the same start time, a one clock delay relative to, or a two clock delay relative to, the previous command or the first BCOM command. With the same start time, both pseudo channels are to start at the same time. With a one clock delay, the pseudo channel indicated by or inferred from the second BCOM command is to start one clock after the pseudo channel indicated by the first BCOM command. With a two clock delay, the pseudo channel indicated by or inferred from the second BCOM command is to start two clocks after the pseudo channel indicated by the first BCOM command.

Table330represents a format or protocol/bit indication of a new BCOM command for a system that provides two pseudo channels, PS[0] and PS[1]. There can be other bits of the command, which are not illustrated in Table330. Other bits of the command not shown would not be changed from a traditional protocol. The BCOM command is assumed to be two clock cycles, with transfer1indicating the transfer on the first clock cycle and transfer2indicating the transfer on the second clock cycle. In one example, Table330represents a first format to use as a default BCOM command.

Row332, row334, and row336represent BCOM bits [2:0], respectively, of the first clock of the first format BCOM command. Row332indicates a command (CMD) select bit, where a logic ‘0’ indicates a read or a write command (i.e., a command for which the data buffer will transfer data) and a logic ‘1’ indicates a non-data command. Row334is a reserved bit, not used in Format 1. Row336indicates a selection between type of data command, where a logic ‘0’ indicates a write command and a logic ‘1’ indicates a read command.

Row338, row340, and row342represent BCOM bits [2:0], respectively, of the second clock of the first format BCOM command. Row338indicates a pseudo channel select bit, where a logic ‘0’ indicates PS[0] is selected and a logic ‘1’ indicates PS[1] is selected. Row340indicates a burst length (BL) selection for the pseudo channel indicated in row338, where a logic ‘0’ indicates BC8(burst chop8, or burst chop for only 8 transfer cycles) and a logic ‘1’ indicates BL16(full burst of16transfer cycles). Row342indicates a rank selection for the pseudo channel indicated in row338, where a logic ‘0’ indicates Rank[0] and a logic ‘1’ indicates Rank[1].

Table350represents a second format to use as a BCOM command when the BCOM command will be sent directly consecutive to a first BCOM command. The command illustrated is also assumed to be two clock cycles, with transfer1indicating the transfer on the first clock cycle and transfer2indicating the transfer on the second clock cycle.

Row352, row354, and row356represent BCOM bits [2:0], respectively, of the first clock of the second format BCOM command. Row352indicates a command (CMD) select bit, where a logic ‘0’ indicates a read or a write command (i.e., a command for which the data buffer will transfer data) and a logic ‘1’ indicates a non-data command. Row354indicates a first delay bit (Delay[0]), where the value of the bit is the LSB (least significant bit) as a lookup table reference for Table370. Row356indicates a selection between type of data command, where a logic ‘0’ indicates a write command and a logic ‘1’ indicates a read command. It will be understood that row356is redundant information, since the type of command can be inferred based on the type of command indicated in Table330.

Row358, row360, and row362represent BCOM bits [2:0], respectively, of the second clock of the second format BCOM command. Row358indicates a first delay bit (Delay[1]), where the value of the bit is the MSB (most significant bit) as a lookup table reference for Table370. Row360indicates a burst length (BL) selection, where a logic ‘0’ indicates BC8(burst chop8, or burst chop for only 8 transfer cycles) and a logic ‘1’ indicates BL16(full burst of16transfer cycles). Row362indicates a rank selection, where a logic ‘0’ indicates Rank[0] and a logic ‘1’ indicates Rank[1]. The pseudo channel to which row360and row362apply will be the “other” pseudo channel as the one indicated in row338of Table330. The pseudo channel for the Format 2 command is inferred as the other pseudo channel to what is specified in the Format 1 command.

The delay indicated by Delay[1:0] (row354and row358) represents two bits interpreted as in Table370to indicate the delay offset relative to the timing of the Format 1 command sent just prior to the Format 2 command. Thus, Table370can indicate a two bits of delay code. In one example, the encoding of the delay bits can be as indicated in row372('00′), row374('01′), and row376('10′). As indicated, a ‘00’=no delay, indicating the command has the proper timing for the data (e.g., 2 clocks after the previous command); a ‘01’=1 clock delay, indicating the data for the command will come on the data bus one clock earlier than the command timing; a ‘10’=2 clock delay, indicating the data for the command will come on the data bus two clocks earlier than the command timing (e.g., at the same time as the other pseudo channel); and, a ‘11’ is not defined for Table370, but could be used to indicate a different delay offset. It will be understood that the timing offset is relative to the prior (Format 1) command. Thus, if the command would normally have a timing of N clock cycles from receipt of the BCOM command to the receipt of the data on the data bus, and the offset will indicate (N), (N-1), or (N-2) in accordance with Table370. Other offsets could alternatively be used.

The RCD determines how to send the BCOM commands. In example, if both channels have the same data timing, the RCD can set the cycle and burst length of the first pseudo channel with the Format 1 command. In one example, the RCD can send the Format 2 BCOM command two cycles after the first BCOM command to set the configuration for the second pseudo channel with the appropriate timing offset indicated.

FIG. 4is a table representation of examples of a first BCOM command format and a second BCOM command format. Table410and Table430represent a first format and second format BCOM command approach to contrast with the traditional BCOM command illustrated in Table410. Table410represents a Format 1 BCOM command as an alternative to the Format1BCOM command indicated in Table430. Table430represents a Format 2 BCOM command as a companion or corresponding command to the Format 1 BCOM command of Table410, and is an alternative to the Format 2 BCOM command of Table430.

The timing and use of the first format and the second format is the same as indicated previously, with a different protocol. Table410represents a format or protocol/bit indication of a new BCOM command for a system that provides two pseudo channels, PS[0] and PS[1]. There can be other bits of the command, which are not illustrated in table410. Other bits of the command not shown would not be changed from a traditional protocol. The BCOM command is assumed to be two clock cycles, with transfer1indicating the transfer on the first clock cycle and transfer2indicating the transfer on the second clock cycle. In one example, Table410represents a first format to use as a default BCOM command.

Row412, row414, and row416represent BCOM bits [2:0], respectively, of the first clock of the first format BCOM command. Row412indicates a command (CMD) select bit, where a logic ‘0’ indicates a read or a write command (i.e., a command for which the data buffer will transfer data) and a logic ‘1’ indicates a non-data command. Row414indicates a format select bit, where a logic ‘0’ indicates the command is a Format 1 command. Row416indicates a selection between type of data command, where a logic ‘0’ indicates a write command and a logic ‘1’ indicates a read command.

Row418, row420, and row422represent BCOM bits [2:0], respectively, of the second clock of the first format BCOM command. Row418indicates a pseudo channel select bit, where a logic ‘0’ indicates PS[0] is selected and a logic ‘1’ indicates PS[1] is selected. Row420indicates a burst length (BL) selection for the pseudo channel indicated in row418, where a logic ‘0’ indicates BC8(burst chop8, or burst chop for only 8 transfer cycles) and a logic ‘1’ indicates BL16(full burst of16transfer cycles). Row422indicates a rank selection for the pseudo channel indicated in row418, where a logic ‘0’ indicates Rank[0] and a logic ‘1’ indicates Rank[1].

Table430represents a second format to use as a BCOM command when the BCOM command will be sent directly consecutive to the first BCOM command of Table410. The command illustrated is also assumed to be two clock cycles, with transfer1indicating the transfer on the first clock cycle and transfer2indicating the transfer on the second clock cycle.

Row432, row434, and row436represent BCOM bits [2:0], respectively, of the first clock of the second format BCOM command. Row432indicates a command (CMD) select bit, where a logic ‘0’ indicates a read or a write command (i.e., a command for which the data buffer will transfer data) and a logic ‘1’ indicates a non-data command. Row434indicates a format select bit, where a logic ‘1’ indicates the command is a Format 2 command. Row436indicates a selection between type of data command, where a logic ‘0’ indicates a write command and a logic ‘1’ indicates a read command. It will be understood that row436is redundant information, since the type of command can be inferred based on the type of command indicated in Table410.

Row438, row440, and row442represent BCOM bits [2:0], respectively, of the second clock of the second format BCOM command. Row438indicates a delay bit, where a logic ‘0’ indicate a 2 clock delay and a logic ‘1’ indicates a 1 clock delay. In one example, to achieve a zero clock delay, the RCD can send a Format 1 BCOM command again. More detail on this implementation follows below.

Row440indicates a burst length (BL) selection, where a logic ‘0’ indicates BC8(burst chop8, or burst chop for only 8 transfer cycles) and a logic ‘1’ indicates BL16(full burst of 16 transfer cycles). Row442indicates a rank selection, where a logic ‘0’ indicates Rank[0] and a logic ‘1’ indicates Rank[1]. The pseudo channel to which row440and row442apply will be the “other” pseudo channel as the one indicated in row418of Table410. The pseudo channel for the Format 2 command is inferred as the other pseudo channel to what is specified in the Format 1 command.

The RCD determines how to send the BCOM commands. In example, if both channels have the same data timing, the RCD can set the cycle and burst length of the first pseudo channel with the Format 1 command. In one example, the RCD can send the Format 2 BCOM command two cycles after the first BCOM command to set the configuration for the second pseudo channel with the appropriate timing offset indicated.

The application of Format 1 in accordance with Table410and Format 2 in accordance with Table430can eliminate the need for the data buffer to remember state, if there is a rule associated with the format type. Namely, Format 1 indicates the pseudo channel, while Format 2 does not.

Other descriptions herein refer to a system configuration where when a second BCOM command is send directly consecutive to the first BCOM command, the second format (Format 2) is always used. In the alternative of Table410and Table430, such an assumption would not be valid. Rather, the system can select whether to use Format 1 or Format 2, depending on the delay offset desired (where the system uses Format 1 when a delay offset of zero is desired).

FIG. 5is a block diagram of an example of an LRDIMM with two pseudo channels. System500represents a system in accordance with an example of system100. System500specifically illustrates DIMM (dual inline memory module)510, which can be considered an LRDIMM because it includes data buffers. In one example, the control of the BCOM commands described with reference to the DIMM can be applied to a stacked device or stacked module.

System500illustrates one example of DIMM510with RCD (registered clock driver)520, memory devices, and data buffers. RCD520represents a controller for DIMM510. In one example, RCD520receives information from a host or a memory controller, and buffers the command signals to the memory devices over a CA bus to the memory devices.

The memory devices are represented as DRAM devices, with different ranks as indicated by the different select lines (CS[0] and CS[1]) and different pseudo channels (PS[0] and PS[1]). More specifically, DIMM510includes two sub channels, sub channel0or sub channel A, and sub channel1or sub channel B. DRAMs532are part of PS[0] for sub channel A (PS[A0]) and receive command information over CA522, with selection via CS[A0] for the “front” devices and via CS[A1] for the “back” devices. It will be understood that front devices refer to the devices on the same side of the DIMM PCB (printed circuit board) as the RCD, while the back devices refer to the devices on the opposite side of the DIMM PCB on which the RCD is mounted.

DRAMs534are part of PS[1] for sub channel A (PS[A1]) and receive command information over CA524, with selection via CS[A0] for the front devices and via CS[A1] for the back devices. DRAMs536are part of PS[0] for sub channel B (PS[B0]) and receive command information over CA526, with selection via CS[B0] for the front devices and via CS[B1] for the back devices. DRAMs538are part of PS[1] for sub channel B (PS[B1]) and receive command information over CA528, with selection via CS[B0] for the front devices and via CS[B1] for the back devices.

DIMM510includes data buffers (DB)542for sub channel A and data buffers (DB)544for sub channel B. Thus, in accordance with one implementation, a data buffer can be one of multiple data buffers for a pseudo channel. In one example, a data buffer can buffer data for both pseudo channels. In one example, the data buffers buffer data for memory devices that are part of both pseudo channels. The BCOM commands would not need to specify pseudo channel or have the directly consecutive commands referred to above if the data buffers were specific to a pseudo channel.

In one example, DIMM510is a DDRS LRDIMM implementation with a single RCD520and multiple data buffers, data buffers542for sub channel A and data buffers544for sub channel B. RCD520can receive the commands from the host and pass a subset of the commands to the data buffers to trigger them to properly transmit or transfer the data between the DRAMs and the host controller.

DIMM510represents BCOM bus552for data buffers542and BCOM bus554for data buffers544. In one example, the BCOM buses are5wire buses. In one example, RCD520sends Read and Write commands as the primary commands to the data buffers over the BCOM buses. RCD520can send the BCOM commands with very specific timing to ensure the data buffers know exactly when to transfer data.

As illustrated, DIMM510includes two pseudo channels, and can thus be considered an implementation of an MCR DIMM, which is form of LRDIMM that divides the DRAMs into two pseudo channels which can transfer data simultaneously. In one example, the data buffers time multiplex the data from both pseudo channels onto the host bus.

System500includes data bus572for sub channel A and data bus574for sub channel B. System500includes CA bus562to provide commands for sub channel A from the host to RCD520and CA bus564to provide commands for sub channel B from the host to RCD520. RCD520can receive and decode commands on CA bus562to provide commands on CA522and on CA524. RCD520can receive and decode commands on CA bus564to provide command on CA526and on CA528.

DIMM510illustrates different data buses between the DRAMs and the data buffers. To simplify the diagram, not all data buses between the DRAMs and the data buffers are labeled. Instead, only one data bus for each pseudo channel is labeled. More specifically, DRAMs532can couple to data buffers542via data (DQ) buses DQ[A0], DRAMs534can couple to data buffers542via data buses DQ[A1], DRAMs536can couple to data buffers544via data buses DQ[B0], and DRAMs538can couple to data buffers544via data buses DQ[B1].

In one example, the host data bus operates at twice the data rate of the DRAMs to accommodate the two pseudo channels. Thus, for example, the transfer speed of data bus572can be twice the transfer speed of DQ[A0] and DQ[A1]. Similarly, the transfer speed of data bus574can be twice the transfer speed of DQ[B0] and DQ[B1], where the transfer speed of data bus572and data bus574can be equal to each other.

In one example, the host command bus operates at twice the data rate of the DRAMs to accommodate the two pseudo channels. Thus, for example, the transfer speed of CA bus562can be twice the transfer speed of CA522and CA524. Similarly, the transfer speed of CA bus564can be twice the transfer speed of CA526and CA528, where the transfer speed of CA bus562and CA bus564can be equal to each other.

In one example, clock (CLK)566represents a clock or timing signal for the commands from the host to RCD520. The data buses can have their own clock signals (e.g., DQS or data strobe), which are not specifically shown.

In one example, read commands and write commands in system500use 5 transfers on the BCOM bus, and the read and write commands must be at least 8 clocks apart from each other. If data takes 8 clocks to transfer, there is plenty of bandwidth to provide the BCOM commands as described above. System500can ensure that the data will always be sent a specific number of clocks after the BCOM command to ensure that with the BCOM command signaling described, the DRAMs, host, and data buffers can remain in sync for the data transfers.

As illustrated in system500, there can be a logical layout to the groupings of DRAMs. For example, as illustrated, sub channels can be organized as right side versus left side of the RCD, ranks can be organized as front and back of the DIMM, and pseudo channels can be organized as upper row versus lower row. Other configurations are possible. A standard DDRS DIMM has two sub channels. In one example, an MCR DIMM has two sub channels, with 2 pseudo channels per sub channel. In one example, data buffers542time multiplex data from PS[A0] and PS[A1] on data bus572, and data buffers544time multiplex data from PS[B0] and PS[B1] on data bus574. When the pseudo channels share data buffers, the pseudo channels must transfer data in the same direction, as mentioned previously.

FIG. 6is a block diagram of an example of a registered clock driver. System600represents an RCD in accordance with system100or system500. RCD610can be a controller for a DIMM or other memory module having data buffers. RCD610includes I/O (input/output)620, which represents a hardware interface to a command bus, represented by CMD (command)622. I/O620enables RCD610to receive commands from the host or memory controller.

RCD610includes I/O630, which represents a hardware interface to a command bus, represented by CMD (command)632, over which RCD610can send commands to memory devices on the memory module. RCD610includes I/O640, which represents a hardware interface to a BCOM bus, represented by BCOM642, over which RCD610can send commands to data buffers on the memory module. Each I/O hardware interface can include signal line interfaces, transmit and/or receive circuitry, and control logic to manage the interface.

Control logic612represents logic to enable the operation of RCD610. In one example, at least some of control logic612is implemented in hardware. In one example, at least some of control logic612is implemented in firmware/software. In one example, control logic612is implemented in a combination of hardware and software.

In one example, control logic612enables RCD610to determine when to use different BCOM command formats. In one example, RCD610can send BCOM commands of first and second formats, and determines when to send a BCOM command of the first format and when to send a command of the second format. Control logic612can generate BCOM commands to send via I/O640with formatting and timing in accordance with any example described. Control logic612can determine to send BCOM commands based on the use of pseudo channels in system600.

FIG. 7is a block diagram of an example of a data buffer. System700represents a data buffer (DB) in accordance with system100or system500. DB710can buffer data between memory device of a memory module and a host controller. DB710includes I/O730, which represents a host-side or host facing hardware interface to a host data bus, represented by host732. DB includes I/O740, which represents a memory-side or memory facing hardware interface to with memory devices, represented by memory742. Buffer714represents the buffer between I/O730and I/O740.

DB710includes I/O720, which represents a hardware interface to a BCOM bus, represented by BCOM722. BCOM722enables DB710to receive commands from an RCD (not specifically shown). Each I/O hardware interface can include signal line interfaces, transmit and/or receive circuitry, and control logic to manage the interface.

Control logic712represents logic to enable the operation of DB710. In one example, at least some of control logic712is implemented in hardware. In one example, at least some of control logic712is implemented in firmware/software. In one example, control logic712is implemented in a combination of hardware and software.

In one example, control logic712enables DB to receive and decode BCOM commands from an RCD. The BCOM commands indicate the timing of data access operations, which directs DB710which side of the data to receive from and which side to transfer to (e.g., from memory side to host side or from host side to memory side), and what the timing of the transfer is. In one example, control logic712determines the specific timings, which can include decoding a timing offset indicated by a second of two consecutive BCOM commands directed to different pseudo channels. DB710can receive BCOM commands of first and second formats, and determine data transfer timings based on the BCOM commands in accordance with any example described.

FIG. 8is a block diagram of an example of data timing for a system with pseudo channels. System802represents a memory module system in accordance with an example of system100or system500. System802includes memory devices of different pseudo channels coupled to a data buffer.

Pseudochannel[0] (referred to as PS[0] for simplicity) represents DRAM devices (e.g., front and back devices) for Rank[0] and Rank[1] devices of a first pseudo channel. Pseudochannel[1] (referred to as PS[1] for simplicity) represents DRAM devices for Rank[0] and Rank[1] of a second pseudo channel. As illustrated, the DRAM devices are x8 devices, having 8 data interface signals. In an alternate implementation, the system can have x4DRAM devices. PS[0] includes data interfaces PSO[D7:DO] and PS[1] includes data interfaces PS1[D7:D0]. The DRAM devices also include interfaces for clock or timing signals, identified as DS_t (data strobe signal) and DS_c (data strobe complement).

DB810represents a data buffer in accordance with any example herein. In one example, DB810includes interface hardware820with retimer822to manage the synchronization of the clock signal from the host bus with the timing signals on the memory module for data signals D[7:4] of the host data bus with associated data strobe DS1_t and DS1_c. In one example, DB810includes interface hardware840with retimer842to manage the synchronization of the clock signal from the host bus with the timing signals on the memory module for data signals D[3:0] of the host data bus with associated data strobe DS0_t and DS0_c.

Diagram804provides a data timing diagram for system802. Signal860represents a clock (CLK) signal, which can be the combination of DS_t and DS_c. Signal872represents the PS[0] data, which represents a BL16burst for all the data interfaces. Signal874represents the PS[1] data, which represents a BL16burst for all the data interfaces. In one example, as illustrated, the data on the memory module for the two pseudo channels takes two clock cycles for transfer.

Signal880represents the host data, and includes two interleaved BL16bursts of data. Diagram804illustrates that DB810interleaves lower speed communication and puts it back to host at double speed for a read, and de-interleaves data double speed data from the host to lower speed communication for a write. Thus, signal880illustrates the read data bits sent to the host interleaved, where each data bit is transmitted at one clock cycle instead of two clock cycles.

FIG. 9is a flow diagram of an example of a process for BCOM command generation by an RCD. Process900represents an example of a process for an RCD to generate and send BCOM commands, and can be performed by an RCD in accordance with any example herein.

The RCD can receive commands from the host, at902. The RCD can decode the command from the host and identify the memory devices to which the command applies, at904. The identification can include determining how to address the memory devices of different pseudo channels. The RCD generates commands to the memory devices and commands to associated data buffers to prepare to transfer data for the command with the correct timing.

In one example, the RCD determines the timing difference between BCOM commands to different pseudo channels, at906. In one example, the RCD will determine what format of BCOM command to send based on the timing of the incoming host commands. The difference of zero to two clock cycles can trigger the RCD to apply a timing of the BCOM commands where a second BCOM command to one pseudo channel will directly follow a first BCOM to the other pseudo channel, with an offset to indicate the desired timing.

If the difference is more than 2 clock cycles (CLK) or a difference that would align the BCOM command one after the other with intervening clock cycles, at908NO branch, the RCD can send a first BCOM command for the first pseudo channel according to a first BCOM command format, at910. In one example, the RCD sends a second BCOM command after a delay of more than 2 clock cycles for the second pseudo channel according to the first BCOM command format, at912. The first format will not have a delay offset indication.

In one example, if there is a 0-2 CLK difference in BCOM commands, at908YES branch, in one example, the RCD determines if the difference is exactly two clock cycles and whether the two clock cycle difference is the desired timing for the second BCOM command, at914. If a two CLK difference is the correct timing for the second BCOM command, at916YES branch, the RCD can send the first BCOM command for the first pseudo channel according to a first BCOM command format, at918. In one example, the RCD sends a second BCOM command for the second pseudo channel according to a second BCOM command format without a delay indicator, at920. Alternatively, the RCD can send a second BCOM command with a delay indicator of 2 CLK cycles to indicate the 2 CLK delay. Alternatively, depending on the BCOM command format, the RCD can send the second BCOM for the second pseudo channel according to the first BCOM command format, for a format that provides sufficient information for the data buffers to generate the correct timing for the correct data signals.

In one example, if the 2 CLK difference is not the correct timing, there is a 0 or 1 CLK difference, at916NO branch. In one example, the RCD determines what is the desired timing for the second BCOM command and generates the command accordingly. The RCD can then send the first BCOM command for the first pseudo channel according to a first format, at922, and then send the second BCOM command for the second pseudo channel according to a second format, with a delay indicator to indicate a delay relative to the first BCOM command, at924.

FIG. 10is a flow diagram of an example of a process for BCOM command processing by a data buffer. Process1000represents an example of a process for a data buffer to receive and process BCOM commands, and can be performed by a data buffer in accordance with any example herein.

The data buffer can receive a BCOM command from the RCD, at1002. The data buffer can decode the BCOM command determine if the command is directed to a data access command, at1004. The data access commands refer to read commands and write commands, which involve a data transfer, which will trigger the data buffer to transfer data from host to memory (write) or from memory to host (read).

If the command is not directed to data access, at1006NO branch, the data buffer can process the non-data command, at1008. If the command is directed to data access, at1006YES branch, in one example, the data buffer can determine if the command is a read command, a write command, and determine a format type of the BCOM command, at1010. The BCOM command can specify the command type in the command.

In one example, the data buffer can determine from the BCOM command what the format of the BCOM command is based on a field in the command, which indicates its format type (e.g., either Format 1 or Format 2). In one example, the data buffer can determine the BCOM command format type simply by the timing of receipt of the command. For example, an implementation of the BCOM communication can specify that a second BCOM command received directly consecutive to a first BCOM command is a Format 2 command, and all other BCOM commands are Format 1. Thus, the determination can depend on the system configuration and potentially an indicator in the command itself.

If the BCOM command is not a second format command, at1012NO branch, the data buffer can decode the command according to a first format, at1014. As described herein, the second format includes a timing delay indication, while the first format is understood to indicate command timing simply by the timing of when the command itself is sent.

If the BCOM command is a second format command, at1012YES branch, the data buffer can decode the command according to the second format and apply the delay indicated, at1016. The delay can be decoded and applied differently based on the protocol used for the BCOM commands. The data buffer can apply the appropriate delay to ensure correct timing of a data transfer associated with the command.

FIG. 11is a block diagram of an example of a memory subsystem in which BCOM communication can be implemented. System1100includes a processor and elements of a memory subsystem in a computing device. System1100represents a system in accordance with an example of system100, system500, or system802.

In one example, memory module1170includes RCD1190, which represents a registered clock driver in accordance with any example herein. In one example, memory module1170includes data buffers1180, which represent data buffers in accordance with any example herein. Data buffers1180couple to DQ1136to buffer data transfer between memory devices1140and memory controller1120. RCD1190can control the operation of data buffers1180through BCOM1192. In one example, RCD1190manages memory devices1140as multiple pseudo channels and controls the data buffers for data access commands in accordance with first and second BCOM command formats, in accordance with any example herein.

Memory controller1120represents one or more memory controller circuits or devices for system1100. Memory controller1120represents control logic that generates memory access commands in response to the execution of operations by processor1110. Memory controller1120accesses one or more memory devices1140. Memory devices1140can be DRAM devices in accordance with any referred to above. In one example, memory devices1140are organized and managed as different channels, where each channel couples to buses and signal lines that couple to multiple memory devices in parallel. Each channel is independently operable. Thus, each channel is independently accessed and controlled, and the timing, data transfer, command and address exchanges, and other operations are separate for each channel. Coupling can refer to an electrical coupling, communicative coupling, physical coupling, or a combination of these. Physical coupling can include direct contact. Electrical coupling includes an interface or interconnection that allows electrical flow between components, or allows signaling between components, or both. Communicative coupling includes connections, including wired or wireless, that enable components to exchange data.

In one example, settings for each channel are controlled by separate mode registers or other register settings. In one example, each memory controller1120manages a separate memory channel, although system1100can be configured to have multiple channels managed by a single controller, or to have multiple controllers on a single channel. In one example, memory controller1120is part of host processor1110, such as logic implemented on the same die or implemented in the same package space as the processor.

Processor1110represents a processing unit of a computing platform that may execute an operating system (OS) and applications, which can collectively be referred to as the host or the user of the memory. The OS and applications execute operations that result in memory accesses. Processor1110can include one or more separate processors. Each separate processor can include a single processing unit, a multicore processing unit, or a combination. The processing unit can be a primary processor such as a CPU (central processing unit), a peripheral processor such as a GPU (graphics processing unit), or a combination. Memory accesses may also be initiated by devices such as a network controller or hard disk controller. Such devices can be integrated with the processor in some systems or attached to the processer via a bus (e.g., PCI express), or a combination. System1100can be implemented as an SOC (system on a chip), or be implemented with standalone components.

Reference to memory devices can apply to different memory types. Memory devices often refers to volatile memory technologies. Volatile memory is memory whose state (and therefore the data stored on it) is indeterminate if power is interrupted to the device. Nonvolatile memory refers to memory whose state is determinate even if power is interrupted to the device. Dynamic volatile memory requires refreshing the data stored in the device to maintain state. One example of dynamic volatile memory includes DRAM (dynamic random-access memory), or some variant such as synchronous DRAM (SDRAM). A memory subsystem as described herein may be compatible with a number of memory technologies, such as DDR4 (double data rate version 4, JESD79-4, originally published in September 2012 by JEDEC (Joint Electron Device Engineering Council, now the JEDEC Solid State Technology Association), LPDDR4 (low power DDR version 4, JESD209-4, originally published by JEDEC in August 2014), WIO2 (Wide I/O 2 (WideIO2), JESD229-2, originally published by JEDEC in August 2014), HBM (high bandwidth memory DRAM, JESD235A, originally published by JEDEC in November 2015), DDR5 (DDR version 5, JESD79-5, originally published by JEDEC in July 2020), LPDDR5 (LPDDR version 5, JESD209-5, originally published by JEDEC in February 2019), HBM2 ((HBM version 2), currently in discussion by JEDEC), or others or combinations of memory technologies, and technologies based on derivatives or extensions of such specifications.

Memory controller1120includes I/O interface logic1122to couple to a memory bus, such as a memory channel as referred to above. I/O interface logic1122(as well as I/O interface logic1142of memory device1140) can include pins, pads, connectors, signal lines, traces, or wires, or other hardware to connect the devices, or a combination of these. I/O interface logic1122can include a hardware interface. As illustrated, I/O interface logic1122includes at least drivers/transceivers for signal lines. Commonly, wires within an integrated circuit interface couple with a pad, pin, or connector to interface signal lines or traces or other wires between devices. I/O interface logic1122can include drivers, receivers, transceivers, or termination, or other circuitry or combinations of circuitry to exchange signals on the signal lines between the devices. The exchange of signals includes at least one of transmit or receive. While shown as coupling I/O1122from memory controller1120to I/O1142of memory device1140, it will be understood that in an implementation of system1100where groups of memory devices1140are accessed in parallel, multiple memory devices can include I/O interfaces to the same interface of memory controller1120. In an implementation of system1100including one or more memory modules1170, I/O1142can include interface hardware of the memory module in addition to interface hardware on the memory device itself. Other memory controllers1120will include separate interfaces to other memory devices1140.

The bus between memory controller1120and memory devices1140can be implemented as multiple signal lines coupling memory controller1120to memory devices1140. The bus may typically include at least clock (CLK)1132, command/address (CMD)1134, and write data (DQ) and read data (DQ)1136, and zero or more other signal lines1138. In one example, a bus or connection between memory controller1120and memory can be referred to as a memory bus. In one example, the memory bus is a multi-drop bus. The signal lines for CMD can be referred to as a “C/A bus” (or ADD/CMD bus, or some other designation indicating the transfer of commands (C or CMD) and address (A or ADD) information) and the signal lines for write and read DQ can be referred to as a “data bus.” In one example, independent channels have different clock signals, C/A buses, data buses, and other signal lines. Thus, system1100can be considered to have multiple “buses,” in the sense that an independent interface path can be considered a separate bus. It will be understood that in addition to the lines explicitly shown, a bus can include at least one of strobe signaling lines, alert lines, auxiliary lines, or other signal lines, or a combination. It will also be understood that serial bus technologies can be used for the connection between memory controller1120and memory devices1140. An example of a serial bus technology is 8B10B encoding and transmission of high-speed data with embedded clock over a single differential pair of signals in each direction. In one example, CMD1134represents signal lines shared in parallel with multiple memory devices. In one example, multiple memory devices share encoding command signal lines of CMD1134, and each has a separate chip select (CS_n) signal line to select individual memory devices.

It will be understood that in the example of system1100, the bus between memory controller1120and memory devices1140includes a subsidiary command bus CMD1134and a subsidiary bus to carry the write and read data, DQ1136. In one example, the data bus can include bidirectional lines for read data and for write/command data. In another example, the subsidiary bus DQ1136can include unidirectional write signal lines for write and data from the host to memory, and can include unidirectional lines for read data from the memory to the host. In accordance with the chosen memory technology and system design, other signals1138may accompany a bus or sub bus, such as strobe lines DQS. Based on design of system1100, or implementation if a design supports multiple implementations, the data bus can have more or less bandwidth per memory device1140. For example, the data bus can support memory devices that have either a x4interface, a x8 interface, a x16 interface, or other interface. The convention “xW,” where W is an integer that refers to an interface size or width of the interface of memory device1140, which represents a number of signal lines to exchange data with memory controller1120. The interface size of the memory devices is a controlling factor on how many memory devices can be used concurrently per channel in system1100or coupled in parallel to the same signal lines. In one example, high bandwidth memory devices, wide interface devices, or stacked memory configurations, or combinations, can enable wider interfaces, such as a x128 interface, a x256 interface, a x512 interface, a x1024 interface, or other data bus interface width.

In one example, memory devices1140and memory controller1120exchange data over the data bus in a burst, or a sequence of consecutive data transfers. The burst corresponds to a number of transfer cycles, which is related to a bus frequency. In one example, the transfer cycle can be a whole clock cycle for transfers occurring on a same clock or strobe signal edge (e.g., on the rising edge). In one example, every clock cycle, referring to a cycle of the system clock, is separated into multiple unit intervals (Uls), where each UI is a transfer cycle. For example, double data rate transfers trigger on both edges of the clock signal (e.g., rising and falling). A burst can last for a configured number of Uls, which can be a configuration stored in a register, or triggered on the fly. For example, a sequence of eight consecutive transfer periods can be considered a burst length eight (BL8), and each memory device1140can transfer data on each UI. Thus, a ×8 memory device operating on BL8can transfer 64 bits of data (8 data signal lines times 8 data bits transferred per line over the burst). It will be understood that this simple example is merely an illustration and is not limiting.

Memory devices1140represent memory resources for system1100. In one example, each memory device1140is a separate memory die. In one example, each memory device1140can interface with multiple (e.g., 2) channels per device or die. Each memory device1140includes I/O interface logic1142, which has a bandwidth determined by the implementation of the device (e.g., ×16 or ×8 or some other interface bandwidth). I/O interface logic1142enables the memory devices to interface with memory controller1120. I/O interface logic1142can include a hardware interface, and can be in accordance with I/O1122of memory controller, but at the memory device end. In one example, multiple memory devices1140are connected in parallel to the same command and data buses. In another example, multiple memory devices1140are connected in parallel to the same command bus, and are connected to different data buses. For example, system1100can be configured with multiple memory devices1140coupled in parallel, with each memory device responding to a command, and accessing memory resources1160internal to each. For a Write operation, an individual memory device1140can write a portion of the overall data word, and for a Read operation, an individual memory device1140can fetch a portion of the overall data word. The remaining bits of the word will be provided or received by other memory devices in parallel.

In one example, memory devices1140are disposed directly on a motherboard or host system platform (e.g., a PCB (printed circuit board) on which processor1110is disposed) of a computing device. In one example, memory devices1140can be organized into memory modules1170. In one example, memory modules1170represent dual inline memory modules (DIMMs). In one example, memory modules1170represent other organization of multiple memory devices to share at least a portion of access or control circuitry, which can be a separate circuit, a separate device, or a separate board from the host system platform. Memory modules1170can include multiple memory devices1140, and the memory modules can include support for multiple separate channels to the included memory devices disposed on them. In another example, memory devices1140may be incorporated into the same package as memory controller1120, such as by techniques such as multi-chip-module (MCM), package-on-package, through-silicon via (TSV), or other techniques or combinations. Similarly, in one example, multiple memory devices1140may be incorporated into memory modules1170, which themselves may be incorporated into the same package as memory controller1120. It will be appreciated that for these and other implementations, memory controller1120may be part of host processor1110.

Memory devices1140each include one or more memory arrays1160. Memory array1160represents addressable memory locations or storage locations for data. Typically, memory array1160is managed as rows of data, accessed via wordline (rows) and bitline (individual bits within a row) control. Memory array1160can be organized as separate channels, ranks, and banks of memory. Channels may refer to independent control paths to storage locations within memory devices1140. Ranks may refer to common locations across multiple memory devices (e.g., same row addresses within different devices) in parallel. Banks may refer to sub-arrays of memory locations within a memory device1140. In one example, banks of memory are divided into sub-banks with at least a portion of shared circuitry (e.g., drivers, signal lines, control logic) for the sub-banks, allowing separate addressing and access. It will be understood that channels, ranks, banks, sub-banks, bank groups, or other organizations of the memory locations, and combinations of the organizations, can overlap in their application to physical resources. For example, the same physical memory locations can be accessed over a specific channel as a specific bank, which can also belong to a rank. Thus, the organization of memory resources will be understood in an inclusive, rather than exclusive, manner.

In one example, memory devices1140include one or more registers1144. Register1144represents one or more storage devices or storage locations that provide configuration or settings for the operation of the memory device. In one example, register1144can provide a storage location for memory device1140to store data for access by memory controller1120as part of a control or management operation. In one example, register1144includes one or more Mode Registers. In one example, register1144includes one or more multipurpose registers. The configuration of locations within register1144can configure memory device1140to operate in different “modes,” where command information can trigger different operations within memory device1140based on the mode. Additionally or in the alternative, different modes can also trigger different operation from address information or other signal lines depending on the mode. Settings of register1144can indicate configuration for I/O settings (e.g., timing, termination or ODT (on-die termination)1146, driver configuration, or other I/O settings).

In one example, memory device1140includes ODT1146as part of the interface hardware associated with I/O1142. ODT1146can be configured as mentioned above, and provide settings for impedance to be applied to the interface to specified signal lines. In one example, ODT1146is applied to DQ signal lines. In one example, ODT1146is applied to command signal lines. In one example, ODT1146is applied to address signal lines. In one example, ODT1146can be applied to any combination of the preceding. The ODT settings can be changed based on whether a memory device is a selected target of an access operation or a non-target device. ODT1146settings can affect the timing and reflections of signaling on the terminated lines. Careful control over ODT1146can enable higher-speed operation with improved matching of applied impedance and loading. ODT1146can be applied to specific signal lines of I/O interface1142,1122(for example, ODT for DQ lines or ODT for CA lines), and is not necessarily applied to all signal lines.

Memory device1140includes controller1150, which represents control logic within the memory device to control internal operations within the memory device. For example, controller1150decodes commands sent by memory controller1120and generates internal operations to execute or satisfy the commands. Controller1150can be referred to as an internal controller, and is separate from memory controller1120of the host. Controller1150can determine what mode is selected based on register1144, and configure the internal execution of operations for access to memory resources1160or other operations based on the selected mode. Controller1150generates control signals to control the routing of bits within memory device1140to provide a proper interface for the selected mode and direct a command to the proper memory locations or addresses. Controller1150includes command logic1152, which can decode command encoding received on command and address signal lines. Thus, command logic1152can be or include a command decoder. With command logic1152, memory device can identify commands and generate internal operations to execute requested commands.

Referring again to memory controller1120, memory controller1120includes command (CMD) logic1124, which represents logic or circuitry to generate commands to send to memory devices1140. The generation of the commands can refer to the command prior to scheduling, or the preparation of queued commands ready to be sent. Generally, the signaling in memory subsystems includes address information within or accompanying the command to indicate or select one or more memory locations where the memory devices should execute the command. In response to scheduling of transactions for memory device1140, memory controller1120can issue commands via I/O1122to cause memory device1140to execute the commands. In one example, controller1150of memory device1140receives and decodes command and address information received via I/O1142from memory controller1120. Based on the received command and address information, controller1150can control the timing of operations of the logic and circuitry within memory device1140to execute the commands. Controller1150is responsible for compliance with standards or specifications within memory device1140, such as timing and signaling requirements. Memory controller1120can implement compliance with standards or specifications by access scheduling and control.

Memory controller1120includes scheduler1130, which represents logic or circuitry to generate and order transactions to send to memory device1140. From one perspective, the primary function of memory controller1120could be said to schedule memory access and other transactions to memory device1140. Such scheduling can include generating the transactions themselves to implement the requests for data by processor1110and to maintain integrity of the data (e.g., such as with commands related to refresh). Transactions can include one or more commands, and result in the transfer of commands or data or both over one or multiple timing cycles such as clock cycles or unit intervals. Transactions can be for access such as read or write or related commands or a combination, and other transactions can include memory management commands for configuration, settings, data integrity, or other commands or a combination.

Memory controller1120typically includes logic such as scheduler1130to allow selection and ordering of transactions to improve performance of system1100. Thus, memory controller1120can select which of the outstanding transactions should be sent to memory device1140in which order, which is typically achieved with logic much more complex that a simple first-in first-out algorithm. Memory controller1120manages the transmission of the transactions to memory device1140, and manages the timing associated with the transaction. In one example, transactions have deterministic timing, which can be managed by memory controller1120and used in determining how to schedule the transactions with scheduler1130.

In one example, memory controller1120includes refresh (REF) logic1126. Refresh logic1126can be used for memory resources that are volatile and need to be refreshed to retain a deterministic state. In one example, refresh logic1126indicates a location for refresh, and a type of refresh to perform. Refresh logic1126can trigger self-refresh within memory device1140, or execute external refreshes which can be referred to as auto refresh commands) by sending refresh commands, or a combination. In one example, controller1150within memory device1140includes refresh logic1154to apply refresh within memory device1140. In one example, refresh logic1154generates internal operations to perform refresh in accordance with an external refresh received from memory controller1120. Refresh logic1154can determine if a refresh is directed to memory device1140, and what memory resources1160to refresh in response to the command.

FIG. 12is a block diagram of an example of a computing system in which BCOM communication can be implemented. System1200represents a computing device in accordance with any example herein, and can be a laptop computer, a desktop computer, a tablet computer, a server, a gaming or entertainment control system, embedded computing device, or other electronic device.

System1200represents a system in accordance with an example of system100, system500, or system802. In one example, memory subsystem1220includes a memory module with memory1230. Memory1230can represent the memory module, which includes RCD1292, which represents a registered clock driver in accordance with any example herein, and data buffers (DBs)1296, which represent data buffers in accordance with any example herein. DBs1296buffer data transfer between memory devices of memory1230and memory controller1222. RCD1292can control the operation of data buffers1296through BCOM1294. In one example, RCD1292manages memory devices of memory1230as multiple pseudo channels and controls the data buffers for data access commands in accordance with first and second BCOM command formats, in accordance with any example herein.

System1200includes processor1210can include any type of microprocessor, central processing unit (CPU), graphics processing unit (GPU), processing core, or other processing hardware, or a combination, to provide processing or execution of instructions for system1200. Processor1210can be a host processor device. Processor1210controls the overall operation of system1200, and can be or include, one or more programmable general-purpose or special-purpose microprocessors, digital signal processors (DSPs), programmable controllers, application specific integrated circuits (ASICs), programmable logic devices (PLDs), or a combination of such devices.

System1200includes boot/config1216, which represents storage to store boot code (e.g., basic input/output system (BIOS)), configuration settings, security hardware (e.g., trusted platform module (TPM)), or other system level hardware that operates outside of a host OS. Boot/config1216can include a nonvolatile storage device, such as read-only memory (ROM), flash memory, or other memory devices.

In one example, system1200includes interface1212coupled to processor1210, which can represent a higher speed interface or a high throughput interface for system components that need higher bandwidth connections, such as memory subsystem1220or graphics interface components1240. Interface1212represents an interface circuit, which can be a standalone component or integrated onto a processor die. Interface1212can be integrated as a circuit onto the processor die or integrated as a component on a system on a chip. Where present, graphics interface1240interfaces to graphics components for providing a visual display to a user of system1200. Graphics interface1240can be a standalone component or integrated onto the processor die or system on a chip. In one example, graphics interface1240can drive a high definition (HD) display or ultra high definition (UHD) display that provides an output to a user. In one example, the display can include a touchscreen display. In one example, graphics interface1240generates a display based on data stored in memory1230or based on operations executed by processor1210or both.

Memory subsystem1220represents the main memory of system1200, and provides storage for code to be executed by processor1210, or data values to be used in executing a routine. Memory subsystem1220can include one or more varieties of random-access memory (RAM) such as DRAM,3DXP (three-dimensional crosspoint), or other memory devices, or a combination of such devices. Memory1230stores and hosts, among other things, operating system (OS)1232to provide a software platform for execution of instructions in system1200. Additionally, applications1234can execute on the software platform of OS1232from memory1230. Applications1234represent programs that have their own operational logic to perform execution of one or more functions. Processes1236represent agents or routines that provide auxiliary functions to OS1232or one or more applications1234or a combination. OS1232, applications1234, and processes1236provide software logic to provide functions for system1200. In one example, memory subsystem1220includes memory controller1222, which is a memory controller to generate and issue commands to memory1230. It will be understood that memory controller1222could be a physical part of processor1210or a physical part of interface1212. For example, memory controller1222can be an integrated memory controller, integrated onto a circuit with processor1210, such as integrated onto the processor die or a system on a chip.

In one example, system1200includes interface1214, which can be coupled to interface1212. Interface1214can be a lower speed interface than interface1212. In one example, interface1214represents an interface circuit, which can include standalone components and integrated circuitry. In one example, multiple user interface components or peripheral components, or both, couple to interface1214. Network interface1250provides system1200the ability to communicate with remote devices (e.g., servers or other computing devices) over one or more networks. Network interface1250can include an Ethernet adapter, wireless interconnection components, cellular network interconnection components, USB (universal serial bus), or other wired or wireless standards-based or proprietary interfaces. Network interface1250can exchange data with a remote device, which can include sending data stored in memory or receiving data to be stored in memory.

In one example, system1200includes one or more input/output (I/O) interface(s)1260. I/O interface1260can include one or more interface components through which a user interacts with system1200(e.g., audio, alphanumeric, tactile/touch, or other interfacing). Peripheral interface1270can include any hardware interface not specifically mentioned above. Peripherals refer generally to devices that connect dependently to system1200. A dependent connection is one where system1200provides the software platform or hardware platform or both on which operation executes, and with which a user interacts.

In one example, system1200includes storage subsystem1280to store data in a nonvolatile manner. In one example, in certain system implementations, at least certain components of storage1280can overlap with components of memory subsystem1220. Storage subsystem1280includes storage device(s)1284, which can be or include any conventional medium for storing large amounts of data in a nonvolatile manner, such as one or more magnetic, solid state, NAND, 3DXP, or optical based disks, or a combination. Storage1284holds code or instructions and data1286in a persistent state (i.e., the value is retained despite interruption of power to system1200). Storage1284can be generically considered to be a “memory,” although memory1230is typically the executing or operating memory to provide instructions to processor1210. Whereas storage1284is nonvolatile, memory1230can include volatile memory (i.e., the value or state of the data is indeterminate if power is interrupted to system1200). In one example, storage subsystem1280includes controller1282to interface with storage1284. In one example controller1282is a physical part of interface1214or processor1210, or can include circuits or logic in both processor1210and interface1214.

Power source1202provides power to the components of system1200. More specifically, power source1202typically interfaces to one or multiple power supplies1204in system1200to provide power to the components of system1200. In one example, power supply1204includes an AC to DC (alternating current to direct current) adapter to plug into a wall outlet. Such AC power can be renewable energy (e.g., solar power) power source1202. In one example, power source1202includes a DC power source, such as an external AC to DC converter. In one example, power source1202or power supply1204includes wireless charging hardware to charge via proximity to a charging field. In one example, power source1202can include an internal battery or fuel cell source.

FIG. 13is a block diagram of an example of a multi-node network in which BCOM communication can be implemented. System1300represents a network of nodes that can apply adaptive ECC. In one example, system1300represents a data center. In one example, system1300represents a server farm. In one example, system1300represents a data cloud or a processing cloud.

Node1330represents a system in accordance with an example of system100, system500, or system802. In one example, node1330includes LRDIMM1344with memory devices represented by memory1340, data buffers represented by DB1394, and a registered clock driver represented by RCD1392. RCD1392represents a registered clock driver in accordance with any example herein, and DBs1394represent data buffers in accordance with any example herein. DBs1394buffer data transfer between memory1340and memory controller1342. RCD1392can control the operation of DBs1394through a BCOM bus. In one example, RCD1392manages memory1340as multiple pseudo channels and controls the data buffers for data access commands in accordance with first and second BCOM command formats, in accordance with any example herein.

One or more clients1302make requests over network1304to system1300. Network1304represents one or more local networks, or wide area networks, or a combination. Clients1302can be human or machine clients, which generate requests for the execution of operations by system1300. System1300executes applications or data computation tasks requested by clients1302.

In one example, system1300includes one or more racks, which represent structural and interconnect resources to house and interconnect multiple computation nodes. In one example, rack1310includes multiple nodes1330. In one example, rack1310hosts multiple blade components1320. Hosting refers to providing power, structural or mechanical support, and interconnection. Blades1320can refer to computing resources on printed circuit boards (PCBs), where a PCB houses the hardware components for one or more nodes1330. In one example, blades1320do not include a chassis or housing or other “box” other than that provided by rack1310. In one example, blades1320include housing with exposed connector to connect into rack1310. In one example, system1300does not include rack1310, and each blade1320includes a chassis or housing that can stack or otherwise reside in close proximity to other blades and allow interconnection of nodes1330.

System1300includes fabric1370, which represents one or more interconnectors for nodes1330. In one example, fabric1370includes multiple switches1372or routers or other hardware to route signals among nodes1330. Additionally, fabric1370can couple system1300to network1304for access by clients1302. In addition to routing equipment, fabric1370can be considered to include the cables or ports or other hardware equipment to couple nodes1330together. In one example, fabric1370has one or more associated protocols to manage the routing of signals through system1300. In one example, the protocol or protocols is at least partly dependent on the hardware equipment used in system1300.

As illustrated, rack1310includes N blades1320. In one example, in addition to rack1310, system1300includes rack1350. As illustrated, rack1350includes M blades1360. M is not necessarily the same as N; thus, it will be understood that various different hardware equipment components could be used, and coupled together into system1300over fabric1370. Blades1360can be the same or similar to blades1320. Nodes1330can be any type of node and are not necessarily all the same type of node. System1300is not limited to being homogenous, nor is it limited to not being homogenous.

For simplicity, only the node in blade1320[0] is illustrated in detail. However, other nodes in system1300can be the same or similar. At least some nodes1330are computation nodes, with processor (proc)1332and memory1340. A computation node refers to a node with processing resources (e.g., one or more processors) that executes an operating system and can receive and process one or more tasks. In one example, at least some nodes1330are server nodes with a server as processing resources represented by processor1332and memory1340. A storage server refers to a node with more storage resources than a computation node, and rather than having processors for the execution of tasks, a storage server includes processing resources to manage access to the storage nodes within the storage server.

In one example, node1330includes interface controller1334, which represents logic to control access by node1330to fabric1370. The logic can include hardware resources to interconnect to the physical interconnection hardware. The logic can include software or firmware logic to manage the interconnection. In one example, interface controller1334is or includes a host fabric interface, which can be a fabric interface in accordance with any example described herein.

Processor1332can include one or more separate processors. Each separate processor can include a single processing unit, a multicore processing unit, or a combination. The processing unit can be a primary processor such as a CPU (central processing unit), a peripheral processor such as a GPU (graphics processing unit), or a combination. Memory1340can be or include memory devices1340coupled to memory controller1342to control host access to memory devices1340.

In general with respect to the descriptions herein, in one example an apparatus includes: a buffer communication (BCOM) bus interface to couple to a BCOM bus, the BCOM bus to provide commands to a data buffer that is to buffer a data bus for memory devices of a first pseudo channel and a second pseudo channel; a controller to send a first BCOM command on the BCOM bus to the data buffer to control data transfer for the first pseudo channel, the first BCOM command to specify a rank and a burst length for the first pseudo channel; and to send a second BCOM command on the BCOM bus to the data buffer to control data transfer for the second pseudo channel, the second BCOM command to specify a rank and a burst length for the second pseudo channel, and a timing offset relative to the first BCOM command.

In one example of the apparatus, the timing offset of the second BCOM command comprises two bits of delay code. In accordance with any preceding example of the apparatus, in one example, the second BCOM command comprises a command having exactly two clock cycles of separation from the first BCOM command. In accordance with any preceding example of the apparatus, in one example, for a BCOM command to be sent subsequent to the first BCOM command at other than two clock cycles of separation from the first BCOM command, the subsequent BCOM command is to specify a rank and a burst length without including the timing offset. In accordance with any preceding example of the apparatus, in one example, the first BCOM command is to specify a read command, and the data buffer is to infer the second BCOM command to be directed to a read command, based on the read command specified in the first BCOM command. In accordance with any preceding example of the apparatus, in one example, the first BCOM command is to specify a write command, and the data buffer is to infer the second BCOM command to be directed to a write command, based on the write command specified in the first BCOM command. In accordance with any preceding example of the apparatus, in one example, the first BCOM command is to specify either the first pseudo channel or the second pseudo channel, and the data buffer is to infer the second BCOM command to be directed to the pseudo channel specified by the first BCOM command. In accordance with any preceding example of the apparatus, in one example, the data buffer is one of multiple data buffers for the first pseudo channel. In accordance with any preceding example of the apparatus, in one example, the data buffer is one of multiple data buffers for the second pseudo channel.

In general with respect to the descriptions herein, in one example a method for data buffer management includes: sending a first buffer communication (BCOM) command on a BCOM bus to a data buffer, the first BCOM command specifying a rank and a burst length for a first pseudo channel; and sending a second BCOM command on the BCOM bus to the data buffer, the second BCOM command specifying a rank and a burst length for a second pseudo channel, and a timing offset relative to the first BCOM command.

In one example of the method, the timing offset of the second BCOM command comprises two bits of delay code. In accordance with any preceding example of the method, in one example, sending the second BCOM command comprises sending the second BCOM command with exactly two clock cycles of separation from the first BCOM command. In accordance with any preceding example of the method, in one example, the method includes: inferring that the second BCOM command is directed to a same type of data access command specified in the first BCOM command. In accordance with any preceding example of the method, in one example, the method includes: inferring that the second BCOM command is directed to a same pseudo channel indicated in the first BCOM command.

In general with respect to the descriptions herein, in one example a system includes: a registered clock driver (RCD) of a memory module; dynamic random access memory (DRAM) devices on the memory module, the DRAM devices addressed as a first pseudo channel and a second pseudo channel; and a data buffer of the memory module coupled to the RCD on a buffer communication (BCOM) bus, the data buffers to buffer a data bus between the DRAM devices and a host memory controller; wherein the data buffer is to receive a first BCOM command on the BCOM bus specifying a rank and a burst length for the first pseudo channel and to receive a second BCOM command specifying a rank and a burst length for the second pseudo channel, the second BCOM command including a timing offset relative to the first BCOM command.

In one example of the system, the second BCOM command comprises a command having exactly two clock cycles of separation from the first BCOM command. In accordance with any preceding example of the system, in one example, the data buffer is to infer a command type of the second BCOM command based on a type of the first BCOM command. In accordance with any preceding example of the system, in one example, the data buffer is to infer a pseudo channel indicated by the second BCOM command based on a pseudo channel indicated by the first BCOM command. In accordance with any preceding example of the system, in one example, the data buffer is one of multiple data buffers for the first pseudo channel and one of multiple data buffers for the second pseudo channel. In accordance with any preceding example of the system, in one example, the system includes one or more of: a host processor coupled to the memory module; a display communicatively coupled to a host processor; a network interface communicatively coupled to a host processor; or a battery to power the system.

Flow diagrams as illustrated herein provide examples of sequences of various process actions. The flow diagrams can indicate operations to be executed by a software or firmware routine, as well as physical operations. A flow diagram can illustrate an example of the implementation of states of a finite state machine (FSM), which can be implemented in hardware and/or software. Although shown in a particular sequence or order, unless otherwise specified, the order of the actions can be modified. Thus, the illustrated diagrams should be understood only as examples, and the process can be performed in a different order, and some actions can be performed in parallel. Additionally, one or more actions can be omitted; thus, not all implementations will perform all actions.

Besides what is described herein, various modifications can be made to what is disclosed and implementations of the invention without departing from their scope. Therefore, the illustrations and examples herein should be construed in an illustrative, and not a restrictive sense. The scope of the invention should be measured solely by reference to the claims that follow.