Response-based interconnect control

Described apparatuses and methods enable a receiver of requests, such as a memory device, to control the arrival of future requests using a credit-based communication protocol. A transmitter of requests can be authorized to transmit a request across an interconnect responsive to possession of a credit. If the transmitter exhausts its credits, the transmitter waits until a credit is returned before transmitting another request. The receiver can manage credit returns based on how many responses are present in a response queue. The receiver can change a rate at which the credit returns are transmitted by changing a size of an interval of responses that are being transmitted, with one credit being returned per interval. This can slow the rate of credit returns while the response queue is relatively more filled. The rate adjustment can decrease latency by reducing an amount of requests or responses that are pooling in backend components.

BACKGROUND

Computers, smartphones, and other electronic devices operate using processors and memories. A processor executes code based on data to run applications and provide features to a user. The processor obtains the code and the data from a memory that can store information. As a result, like a processor's speed or number of cores, a memory's type or other characteristics can impact the performance of an electronic device. Different types of memory may have different characteristics. Memory types include volatile memory and nonvolatile memory, such as random access memory (RAM) and flash memory, respectively. RAM can include static RAM (SRAM) and dynamic RAM (DRAM), such as Compute Express Link™ (CXL) attached memory. Flash memory can be used to build, for instance, a solid-state drive (SSD).

Demands on the different types of memory continue to evolve and grow. For example, as processors are engineered to execute code faster, such processors can benefit from accessing memories more quickly. Applications may also operate on ever-larger data sets that occupy ever-larger memories. Due to battery-powered electronic devices and power-hungry data centers, energy-usage constraints are becoming more prevalent for memory systems. Further, manufacturers may seek physically smaller memories as the form factors of portable electronic devices continue to shrink. Accommodating these various demands is complicated by the diverse strengths and capabilities of different types of memories.

DETAILED DESCRIPTION

Overview

Processors and memory work in tandem to provide features to users of computers and other electronic devices. Generally, an electronic device can provide enhanced features, such as high-resolution graphics or artificial intelligence, as a processor and memory operate more quickly together in a complementary manner. Some applications, like those for AI analysis and virtual-reality graphics, can also demand increasing amounts of memory. Such applications use increasing amounts of memory to more accurately model and mimic human thinking and the physical world.

Processors and memories can be secured to a printed-circuit board (PCB), such as a rigid or flexible motherboard. The PCB can include sockets for accepting at least one processor and one or more memories. Wiring infrastructure that enables communication between two or more components can also be disposed on at least one layer of the PCB. This PCB, however, provides a finite area for the sockets and the wiring infrastructure. Some PCBs include multiple sockets that are each shaped as a linear slot and designed to accept a double-inline memory module (DIMM). These sockets can be fully occupied by DIMMs while a processor is still able to utilize more memory. In such situations, the system is capable of performing better if additional memory were available to the processor.

Printed circuit boards may also include at least one peripheral component interconnect (PCI) express (PCI Express®) (PCIe or PCI-E) slot. A PCIe slot is designed to provide a common interface for various types of components that may be coupled to a PCB. Compared to some older standards, PCIe can provider higher rates of data transfer or a smaller footprint on the PCB, including both greater speed and smaller size. Accordingly, certain PCBs enable a processor to access a memory device that is connected to the PCB via a PCIe slot.

In some cases, accessing a memory solely using a PCIe protocol may not offer as much functionality, flexibility, or reliability as is desired. In such cases, another protocol may be layered on top of the PCIe protocol. An example of another, higher-level protocol is the Compute Express Link™ (CXL) protocol. The CXL protocol can be implemented over a physical layer that is governed by the PCIe protocol. The CXL protocol can provide a memory-coherent interface that offers high-bandwidth or low-latency data transfers, including data transfers having both higher bandwidth and lower latency.

Various electronic devices, such as a mobile phone having a processor that is part of a system-on-chip (SoC) or a cloud-computing server having dozens of discrete processing units, may employ memory that is coupled to a processor via a CXL-based interconnect. For clarity, consider an apparatus with a host device that is coupled to a memory device via a CXL-based interconnect. The host device can include a processor and a controller (e.g., a host-side controller) that is coupled to the interconnect. The memory device can include another controller (e.g., a memory-side controller) that is coupled to the interconnect and one or more memory arrays to store information in SRAM, DRAM, flash memory, and so forth.

During operation, the host-side controller issues memory requests to the memory-side controller over the interconnect. The memory request may be or may include a read request or a write request. The memory-side controller receives the memory request via the interconnect and directly or indirectly uses the memory arrays to fulfill the memory request with a memory response. Thus, the memory-side controller sends the memory response to the host-side controller over the interconnect. To fulfill a read request, the memory-side controller returns the requested data with the memory response. As part of fulfilling a write request, the memory-side controller can provide notice that the write operation was successfully completed by transmitting an acknowledgement as the memory response (e.g., using a message such as a subordinate-to-master no-data response completion (S2M NDR Cmp) message).

To increase bandwidth and reduce latency, the memory-side controller can include at least one request queue that may accumulate multiple memory requests (e.g., multiple read requests or multiple write requests) received from the host-side controller. In other words, the host-side controller can send a “subsequent” memory request before receiving a memory response corresponding to a “previous” memory request. This can ensure that the memory device is not waiting idly for another memory request that the host-side controller has already prepared. This technique can also better utilize the interconnect by transmitting the subsequent memory request before the memory response for the previous memory request is ready.

The request queue at the memory-side controller may, however, have space for a finite quantity of entries. If the host-side controller overflows the request queue at the memory-side controller, memory accessing can be slowed, and the overflow may even cause data loss. In other words, without a mechanism to control the flow of memory access requests from the host-side controller to the memory-side controller, memory bandwidth or latency can be degraded. Further, an overwhelmed request queue may even cause errors to occur.

The memory-side controller can also include a response queue to store multiple memory responses that have been received from a backend memory component, such as a memory array. The memory-side controller extracts a memory response from the response queue and transmits the memory response to the host-side controller to respond to a corresponding memory request and thereby complete a memory operation. If the memory array is “too fast” or the interconnect is “too slow,” the response queue can overflow or become too full to safely and expeditiously accept new memory responses. Like an oversubscribed forward path of a memory device, a backlog on the return path can also adversely impact memory bandwidth or latency, especially if the host-side controller continues to send new memory requests.

One approach to modulating (e.g., moderating) the flow of memory requests involves using credits. The host-side controller can be granted a particular quantity of credits. A maximum credit quantity may be based, for instance, on a size of the request queue of the memory-side controller. If the host-side controller currently has, or “possesses,” at least one credit, then the host-side controller can issue a memory request to the memory-side controller over the interconnect. On the other hand, if the host-side controller has depleted the granted supply of credits, the host-side controller waits until at least one credit has been replenished before issuing another memory request.

The memory-side controller can be responsible for replenishing credits. The memory-side controller can indicate to the host-side controller that one or more credits have been replenished, or “returned,” using a communication across the interconnect. For example, a memory response that includes read data or a write acknowledgement can also include a credit return indication. In some cases, the memory-side controller returns a credit responsive to a memory request being removed from the request queue at the memory-side controller. This approach to credit-based control of an interconnect can prevent the request queue at the memory-side controller from overflowing and causing an error condition.

This approach may not, however, prevent memory bandwidth from being reduced or latency from increasing due to an oversupply of the total memory requests or memory responses present in the memory device. This is because the memory device includes, in addition to a response queue at the memory-side controller as described above, one or more “downstream” or “backend” memory components that are coupled to the memory-side controller. For example, the memory device includes one or more memory arrays and may include other components to facilitate memory request processing. Other components of the memory device may include at least one “internal” interconnect and one or more memory controllers, which are coupled to the memory arrays to control access thereto. Any of these memory components may include at least one respective queue, such as an additional memory request queue on the forward path or an additional response queue on the return path. For instance, each memory controller of two memory controllers may include a respective memory request queue of two memory request queues.

Responsive to the memory-side controller removing a memory request from its request queue, the memory-side controller forwards the memory request to a downstream or backend component, such as one of the memory controllers. The receiving memory controller may be accumulating memory requests in its respective request queue. This accumulation may occur, for instance, due to a relatively slower memory array that is unable to process requests at the rate at which the requests are being received from the memory-side controller. Thus, the memory-side controller may return a credit, which authorizes another memory request, to the host-side controller even though un-serviced memory requests are “piling up” within the memory device along the forward path. Further, memory responses may also be accumulating within the memory device along the return path, such as in a response queue of a memory controller, once a memory array has serviced a memory request.

Thus, request queues in various parts of the memory device may become saturated with memory requests, and response queues in various parts of the memory device may become saturated with memory responses. Allowing such queues of the memory controllers or of other backend components or the response queue of the memory-side controller to become saturated can lower the bandwidth throughput of the memory system. Moreover, saturating the forward path or the return path can also increase a length of the latency period between when the memory device accepts a memory request from the host device and when the memory device provides the corresponding memory response. Consequently, returning a credit to the host-side controller each time a memory request is removed from the request queue at the memory-side controller may adversely impact memory system performance.

With respect to memory response queues, a response queue can be present at the memory-side controller or any of the backend components of the memory device, like a memory controller or a memory array. Oversaturating the response queues can also decrease bandwidth and increase latency. A response queue can become full or “backed up” if, for instance, an internal interconnect or an external interconnect is too busy or is oversubscribed. For example, the “external” interconnect extending between the host device and the memory device may be oversubscribed by the host device or by other devices (e.g., other PCIe devices) that are coupled to the external interconnect. Additionally or alternatively, a relatively fast memory array may be providing memory responses faster than the memory device can empty them from one or more response queues thereof. In such cases, an unbridled credit-return system can cause at least one response queue of the memory device to become filled. A full response queue can further slow the memory device sufficiently to adversely impact bandwidth and latency.

Decreased processing bandwidth and increased latency for a memory device may be categorized as poor performance. Slow or otherwise poor memory performance can cause system-wide problems and create user dissatisfaction. Especially if the poor performance conflicts with advertised performance capabilities or a published technical specification, such as a quality-of-service (QoS) indication, the user may blame the manufacturer of the memory device. This can happen even if the host device or another device that is coupled to the interconnect is contributing to the bandwidth and latency issues by overusing the shared external interconnect. Further, the misplaced blame can occur if the host device is sending too many memory requests to the memory device due to an inadequate credit-based communications scheme.

To address this situation, and at least partly ameliorate it, this document describes example approaches to managing the flow of memory requests using a credit-based system. In some implementations, a memory-side controller of a memory device can monitor a quantity of memory responses that are present in a response queue of the memory device, including in a memory response queue of the memory-side controller. The memory-side controller can modulate one or more credits being returned to a host-side controller based on the quantity of memory responses that are present in the memory response queue. Thus, credit returns may be conditioned on how many memory responses are stored within the memory response queue, as well as perhaps on the number of memory requests that are within a memory request queue of the memory-side controller.

In other implementations, a memory-side controller of a memory device can include a response queue to store multiple memory responses that have been received from backend memory components. These multiple memory responses can correspond to multiple memory requests (e.g., from a host device), and the memory-side controller is to transmit the memory responses to the host-side controller. The memory-side controller can additionally include credit logic having a counter to store a value. The credit logic can adjust the value of the counter to track a quantity of memory responses that are present in the response queue. In some cases, the credit logic increments the value responsive to receiving a memory response (e.g., from a backend memory component) and decrements the value responsive to transmission of a memory response of the multiple memory responses from the response queue (e.g., to the host device).

The credit logic can manage credit returns to a host-side controller based on the value stored in the counter and a lookup table by changing a rate at which the credit returns are transmitted to the host-side controller. In this context, the rate may relate to a number of memory responses to be transmitted for each credit return to be transmitted. The lookup table can provide the number “N” of memory responses to be transmitted per credit return transmission based on the value of the counter. To change the rate of credit return transmissions, the credit logic may block credit returns for the memory response transmissions in each interval except for one memory response transmission responsive to the value stored in the counter. The credit logic may further permit a credit return a certain number of times (e.g., once) per interval to accompany a memory response being transmitted to the host-side controller.

The lookup table may include multiple entries, with each entry associated with a respective range of quantities that maps to a corresponding number “N” of memory response transmissions. Each range of quantities can relate to a quantity of memory responses stored in the response queue at the memory device. Thus, using the lookup table to obtain the number “N,” the memory-side controller can adjust a rate of credit return transmissions by transmitting a credit return with each “Nth” memory response transmission. In these manners, the value of the counter, which indicates a current quantity of memory responses in the response queue, can at least partially control a rate of reception of new memory requests.

By employing one or more of these implementations, a memory device can obtain greater control over the flow of memory requests received from a host device. The memory device can modulate a quantity of memory responses stored at the memory-side controller and/or a rate at which credit returns are transmitted to the host device and, therefore, a rate at which memory requests are received from the host device over time. By throttling the arrival of the memory requests, a memory device can avoid becoming so saturated with memory requests that bandwidth or latency is adversely impacted. Thus, using the techniques described herein, manufacturers can produce memory devices that are better able to provide some specified quality of service in terms of bandwidth or latency. Although some implementations are described above in terms of a memory request and a memory device performing certain techniques, other device types may alternatively perform the techniques with requests generally. Examples of non-memory implementations are described further herein.

Example Operating Environments

FIG.1illustrates, at100generally, example apparatuses102that can implement response-based interconnect control. The apparatus102can be realized as, for example, at least one electronic device. Example electronic-device implementations include an internet-of-things (IoTs) device102-1, a tablet device102-2, a smartphone102-3, a notebook computer102-4(or a desktop computer), a passenger vehicle102-5(or other vehicle), a server computer102-6, a server cluster102-7that may be part of cloud computing infrastructure or a data center, and any portion thereof (e.g., a printed circuit board (PCB) or module component of a device).

Other examples of the apparatus102include a wearable device, such as a smartwatch or intelligent glasses; an entertainment device, such as a set-top box or streaming dongle, a smart television, a gaming device, or virtual reality (VR) goggles; a motherboard or blade of a server; a consumer appliance; a vehicle or drone, or the electronic components thereof; industrial equipment; a security or other sensor device; and so forth. Each type of electronic device or other apparatus can include one or more components to provide some computing functionality or feature that is enabled or enhanced by the hardware or techniques that are described herein.

In example implementations, the apparatus102can include at least one host device104, at least one interconnect106, and at least one memory device108. The host device104can include at least one processor114, at least one cache memory116, and at least one controller118. The memory device108may include at least one controller110and at least one memory112. The memory112may be realized with one or more memory types.

The memory112may be realized, for example, with a dynamic random-access memory (DRAM) die or module, including with a three-dimensional (3D) stacked DRAM device, such as a high bandwidth memory (HBM) device or a hybrid memory cube (HMC) device. DRAM may include, for instance, synchronous DRAM (SDRAM) or double data rate (DDR) DRAM (DDR DRAM). The memory112may also be realized using static random-access memory (SRAM). Thus, the memory device108may operate as a main memory or a cache memory, including as both. Additionally or alternatively, the memory device108may operate as storage memory. In such cases, the memory112may be realized, for example, with a storage-class memory type, such as one employing 3D XPoint™ or phase-change memory (PCM), flash memory, a magnetic hard disk, or a solid-state drive (e.g., a Non-Volatile Memory Express® (NVMe®) device).

Regarding the host device104, the processor114can be coupled to the cache memory116, and the cache memory116can be coupled to the controller118. The processor114can also be coupled to the controller118directly or indirectly (e.g., via the cache memory116as depicted). The host device104may include other components to form, for instance, a system-on-a-chip or a system-on-chip (SoC). The processor114may include or comprise a general-purpose processor, a central processing unit (CPU), a graphics processing unit (GPU), a neural network engine or accelerator, an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) integrated circuit (IC), a communication processor (e.g., a modem or baseband processor), an SoC, and so forth.

In operation, the controller118can provide a high-level or logical interface between the processor114and at least one memory device, such as a memory that is external to the host device104. The controller118can, for example, receive memory requests from the processor114and provide the memory requests to an external memory with appropriate formatting, packaging, timing, reordering, and so forth. The controller118can forward to the processor114responses to the memory requests that the controller118receives from the external memory.

The controller118may communicate with multiple memory devices, or other types of devices—some of which may include one or more memory components, over one or more interconnects, such as the interconnect106. Regarding connections that are external to the host device104, the host device104can be coupled to the memory device108via the interconnect106. The memory device108may be coupled to, or may include, a main memory or a storage memory, including both in some cases. Another device, such as a cache memory or a switch, may be coupled between the host device104and the memory device108and may be part of or separate from the interconnect106.

The depicted interconnect106, as well as other interconnects (not shown) that communicatively couple together various components, enables data to be transferred between two or more components of the various components. Interconnect examples include a bus, a switching fabric, a crossbar, one or more wires that carry voltage or current signals, and so forth. Each interconnect may be implemented as a unidirectional interconnect or a bidirectional interconnect. The interconnect106can be implemented as a parallel propagation pathway. For example, the interconnect106can include at least one command and address bus and at least one data bus, each of which carries multiple bits of a particular item of information (e.g., a data byte) simultaneously each clock period. Alternatively, the interconnect106can be implemented as a serial propagation pathway that carries one bit of a particular item of information each clock cycle. For instance, the interconnect106can comport with a PCIe standard, such as version 4, 5, 6, or a future version. The interconnect106may include multiple serial propagation pathways, such as multiple lanes in a PCIe implementation.

The components of the apparatus102that are depicted inFIG.1represent an example computing architecture that may include a hierarchical memory system. A hierarchical memory system can include memories at different levels, with each level having a memory with a different speed, capacity, or volatile/nonvolatile characteristic. Thus, the memory device108may be described in terms of forming at least part of a main memory of the apparatus102. The memory device108may, however, form at least part of a cache memory, a storage memory, an SoC, and so forth of an apparatus102.

Although various implementations of the apparatus102are depicted inFIG.1and described herein, an apparatus102can be implemented in alternative manners. For example, the host device104may include multiple cache memories, including multiple levels of cache memory, or may omit a cache memory. A memory, such as the memory device108, may have a respective “internal” or “local” cache memory (not shown). In some cases, the host device104may omit the processor114and/or include other logic. Generally, the illustrated and described components may be implemented in alternative ways, including in distributed or shared memory systems. A given apparatus102may also include more, fewer, or different components than those depicted inFIG.1or described herein.

The host device104and any of the various memories may be realized in multiple manners. In some cases, the host device104and the memory device108may be located on separate blades or racks in a server computing environment. In other cases, the host device104and the memory device108can both be disposed on, or physically supported by, a same printed circuit board (PCB) (e.g., a rigid or flexible motherboard or PCB assembly). The host device104and the memory device108may also be integrated on a same IC or fabricated on separate ICs but packaged together.

A memory device108may also be coupled to multiple host devices104via one or more interconnects106and may be able to respond to memory requests from two or more of the multiple host devices104. Each host device104may include a respective controller118, or the multiple host devices104may share a common controller118. An example computing system architecture with at least one host device104that is coupled to a memory device108is described below with reference toFIG.2.

With continuing reference toFIG.1, however, the interconnect106may propagate one or more communications. The host device104and the memory device108may exchange at least one memory request/memory response120. For example, the controller118may transmit a memory request to the controller110over the interconnect106. Thus, the controller110may transmit a corresponding memory response to the controller118over the interconnect106. In some cases, the interconnect106is operated in accordance with a credit-based protocol. Accordingly, credit-related information122may be exchanged between the host device104and the memory device108. For instance, the controller110may transmit a credit return to the controller118to enable the controller118to transmit another memory request. In some cases, the credit-related information122and a memory request or memory response120may be combined into a joint, shared, or overlapping communication, such as a packet or a flit.

Thus, the memory device108and the host device104can communicate using a credit-based protocol over the interconnect106. The controller110of the memory device108can include credit logic124, and the controller118of the host device104can include credit logic126. In example implementations, the credit logic124and/or the credit logic126can facilitate communication over the interconnect106using at least one protocol that operates based on credits.

A credit-based protocol can use tokens or another quantity-based permissions scheme to authorize an initiator and/or a target to communicate with the target and/or the initiator, respectively. For example, the controller118may transmit a communication (e.g., a memory request) over the interconnect106to the controller110responsive to possessing at least one credit that “authorizes” the transmission. This transmission, however, “consumes” the at least one credit. Examples of credit-based protocols are described below with reference toFIGS.4and5.

In example implementations, the credit logic124of the controller110can moderate the flow of communications from the controller118. To do so, the credit logic124can modulate the frequency or rate at which the credit logic124returns credits to the credit logic126of the controller118. Withholding or delaying credit returns can slow, or even stop, the transmission of memory requests from the host device104to the memory device108if the memory device108becomes oversaturated with in-progress memory requests and/or queued memory responses. Example techniques for modulating such credit returns are described herein to increase the bandwidth for memory-request processing or to decrease memory response latency, including to achieve both. For instance, the credit logic124can delay or slow the return of credits to the credit logic126of the controller118based on a quantity of memory responses that are stored in at least one response queue of the controller110. Example implementations are described further with reference toFIGS.6-11.

In some implementations, the apparatus102operates with one or more protocols over the interconnect106. The apparatus102can operate, for example, a Compute Express Link′ (CXL) protocol across the interconnect106. In at least some of these cases, the apparatus102can overlay the CXL protocol on top of a PCIe protocol for the physical layer. Thus, the controller118can comport with a CXL standard or a PCIe standard, including comporting with both. Similarly, the controller110can comport with a CXL standard or a PCIe standard, including with both. Examples of credit-based aspects of at least one version of a CXL standard are described below with reference toFIGS.4and5. Other circuitry, techniques, and mechanisms are also described below. Next, however, this document describes example computing architectures with one or more processors and a memory device.

FIG.2illustrates an example computing system200that can implement aspects of response-based interconnect control with a memory device. In some implementations, the computing system200includes at least one memory device108, at least one interconnect106, and at least one processor202. The memory device108can include, or be associated with, at least one memory array206, at least one interface204, and at least one controller110. The at least one controller110can be communicatively coupled to the memory array206via at least one interconnect208(e.g., an “internal” interconnect). The memory array206and the controller110may be components that are integrated on a single semiconductor die or that are located on separate semiconductor dies (e.g., but still coupled to or disposed on a same PCB). Each of the memory array206or the controller110may also be distributed across multiple dies (or dices).

The memory device108can correspond, for example, to one or more of a cache memory, main memory, or storage memory of the apparatus102ofFIG.1. Thus, the memory array206can include an array of memory cells. These memory cells can include, but are not limited to, memory cells of Static Random-Access Memory (SRAM), Dynamic Random-Access Memory (DRAM), Synchronous DRAM (SDRAM), three-dimensional (3D) stacked DRAM, Double Data Rate (DDR) memory, low-power Dynamic Random-Access Memory (DRAM), Low-Power Double Data Rate (LPDDR) Synchronous Dynamic Random-Access Memory (SDRAM), phase-change memory (PCM), or flash memory.

The controller110can include any one or more of a number of components that can be used by the memory device108to perform various operations. These operations can include communicating with other devices, managing performance, modulating memory access rates, and performing memory read or write operations. For example, the controller110can include at least one register212, at least one instance of request logic214, at least one instance of response logic216, and at least one instance of credit logic124.

The register212may be implemented, for example, as one or more registers that can store information to be used by the controller110, by another part of the memory device108, or by a part of a host device104, such as a controller118as depicted inFIG.1. A register212may store, for instance, a maximum credit level, a parameter controlling part of a communication-flow modulation process using credits (e.g., a register718ofFIG.7), and so forth. The controller110may also include one or more counters, as is described below. The request logic214can process one or more memory requests, such as by formulating a request, directing a request to a next or final destination, or performing a memory access operation (e.g., a read or a write operation).

The response logic216can prepare at least one memory response, such as by obtaining requested data or generating a write acknowledgement. The credit logic124can modulate the flow of memory requests across the interconnect106using credits, which are described further below, including with reference toFIG.4. Although depicted separately, the components of the controller110may be nested with respect to each other, may be at least partially overlapping with another component, and so forth.

The interface204can couple the controller110or the memory array206directly or indirectly to the interconnect106. As shown inFIG.2, the register212, the request logic214, the response logic216, and the credit logic124can be part of a single component (e.g., the controller110). In other implementations, one or more of the register212, the request logic214, the response logic216, or the credit logic124may be implemented as separate components, which can be provided on a single semiconductor die or disposed across multiple semiconductor dies. These components of the controller110may be individually or jointly coupled to the interconnect106via the interface204.

The interconnect106may be implemented with any one or more of a variety of interconnects that communicatively couple together various components and enable commands, addresses, messages, packets, and/or other information and data to be transferred between two or more of the various components (e.g., between the memory device108and any of the one or more processors202). The information and data may be propagated over the interconnect106“directly” or using some form of encapsulation or packaging, such as with packets, frames, or flits. Although the interconnect106is represented with a single line or arrow inFIG.2, the interconnect106may include at least one bus, at least one switching fabric, at least one crossbar, one or more wires or traces that carry voltage or current signals, at least one switch, one or more buffers, at least one lane, and so forth.

In some aspects, the memory device108may be realized as a “separate” physical component relative to the host device104(ofFIG.1) or any of the processors202. Examples of physical components that may be separate include, but are not limited to, a printed circuit board (PCB), which can be rigid or flexible; a memory card; a memory stick; and a memory module, including a single in-line memory module (SIMM), a dual in-line memory module (DIMM), or a non-volatile memory express (NVMe) module. Thus, separate physical components may be located together within a same housing of an electronic device or a memory product, or such physical components may be distributed over a server rack, a data center, and so forth. Alternatively, the memory device108may be packaged or integrated with other physical components, including a host device104or a processor202, such as by being disposed on a common PCB, combined together in a single device package, or integrated into an SoC of an apparatus.

As shown inFIG.2, the one or more processors202may include a computer processor202-1, a baseband processor202-2, and an application processor202-3, which are coupled to the memory device108through the interconnect106. The processors202may each be, or may form a part of, a CPU, a GPU, an SoC, an ASIC, an FPGA, or the like. In some cases, a single “processor” can comprise multiple processing cores or resources, each dedicated to different functions, such as modem management, applications, graphics, central processing, neural network acceleration, or the like. In some implementations, the baseband processor202-2may include or be coupled to a modem (not shown inFIG.2) and may be referred to as a modem processor. The modem and/or the baseband processor202-2may be coupled wirelessly to a network via, for example, cellular, Wi-Fi®, Bluetooth®, ultra-wideband (UWB), near field, or another technology or protocol for wireless communication.

In various implementations, the processors202may be connected to different memories in different manners. For example, the processors202may be connected directly to the memory device108(e.g., via the interconnect106as shown). Alternatively, one or more of the processors202may be indirectly connected to the memory device108, such as over a network connection, through one or more other devices or components, and/or using at least one other additional interconnect. Each processor202may be realized similarly to the processor114ofFIG.1. Accordingly, a respective processor202can include or be associated with a respective controller, like the controller118depicted inFIG.1. Alternatively, two or more processors202may access the memory device108using a shared or system controller118. In any of such cases, the controller118may include credit logic126(e.g., ofFIG.1). Each processor202may also be separately connected to a respective memory. As shown, the computer processor202-1may be coupled to at least one DIMM210that is inserted into a DIMM slot of a motherboard. The DIMM210can be coupled to a memory controller (not shown), which may be part of the computer processor202-1.

The apparatuses and methods that are described herein may be appropriate for memory that is designed for use with a PCIe bus. Thus, the described principles may be incorporated into a memory device with a PCIe interface. Further, the memory device can communicate over the interconnect106by overlaying a CXL protocol on the physical PCIe interface. An example of a memory standard that relates to CXL is promulgated by the Compute Express Link™ consortium and may include versions 1.0, 1.1, 2.0, and future versions. Thus, the host device104or the memory device108, including both in some cases, may comport with at least one CXL standard. Accordingly, some terminology in this document may draw from one or more of these standards or versions thereof for clarity. The described principles, however, are also applicable to memories that comport with other standards, including earlier versions or future versions of such standards, and to memories that do not adhere to a public standard. Examples of systems that may include a PCIe interface and a CXL protocol overlay are described next with reference toFIG.3.

FIG.3illustrates examples of a system300that can include a host device104and a memory device108that are coupled together via an interconnect106. The system300can implement aspects of response-based interconnect control and may form at least part of an apparatus102as shown inFIG.1. As illustrated inFIG.3, the host device104includes a processor114and a controller118, which can be realized with at least one initiator302. Thus, the initiator302can be coupled to the processor114or to the interconnect106(including to both), and the initiator302can be coupled between the processor114and the interconnect106. Examples of initiators302may include a leader, a primary, a master, a requester or requesting component, a main component, and so forth.

In the illustrated example system300, the memory device108includes a controller110, which can be realized with at least one target304. The target304can be coupled to the interconnect106. Thus, the target304and the initiator302can be coupled to each other via the interconnect106. Examples of targets304may include a follower, a secondary, a slave, a subordinate, a responder or responding component, a subsidiary component, and so forth. The memory device108also includes a memory112. The memory112can be realized with at least one memory module or chip or with a memory array206(ofFIG.2) or another component, such as a DRAM310, as is described below.

In example implementations, the initiator302includes at least one link controller312, and the target304includes at least one link controller314. The link controller312or the link controller314can instigate, coordinate, cause, or otherwise participate in or control signaling across a physical or logical link realized by the interconnect106in accordance with one or more protocols. The link controller312may be coupled to the interconnect106. The link controller314may also be coupled to the interconnect106. Thus, the link controller312can be coupled to the link controller314via the interconnect106. Each link controller312or314may, for instance, control communications over the interconnect106at a link layer or at one or more other layers of a given protocol. Communication signaling may include, for example, a request316, a response318, and so forth.

The memory device108may further include at least one interconnect306and at least one memory controller308(MC308). Within the memory device108, and relative to the target304, the interconnect306, the memory controller308, and/or the DRAM310(or other memory component) may be referred to as a “backend” or “downstream” component or memory component of the memory device108. In some cases, the interconnect306is internal to the memory device108and may operate the same as or differently from the interconnect106or like the interconnect208.

Thus, the memory device108can include at least one memory component. As shown, the memory device108may include multiple memory controllers308-1and308-2and/or multiple DRAMs310-1and310-2. Although two of each are shown, the memory device108may include one or more than two memory controllers and/or one or more than two DRAMs. For example, a memory device108may include 4 memory controllers and 16 DRAMs, such as 4 DRAMs per memory controller. The memory112or memory components of the memory device108are depicted as DRAM as an example only, for one or more of the memory components may be implemented as another type of memory. For instance, the memory components may include nonvolatile memory like flash or PCM. Alternatively, the memory components may include other types of volatile memory like SRAM. Thus, the memory device108may include a dynamic random-access memory (DRAM) array, a static random-access memory (SRAM) array, or a nonvolatile memory array. A memory device108may also include any combination of memory types.

In some cases, the memory device108may include the target304, the interconnect306, the at least one memory controller308, and the at least one DRAM310within a single housing or other enclosure. The enclosure, however, may be omitted or may be merged with one for the host device104, the system300, or an apparatus102(ofFIG.1). In some cases, each of these components can be realized with a separate IC. In some of such cases, the interconnect306can be disposed on a PCB. Each of the target304, the memory controller308, and the DRAM310may be fabricated on at least one IC and packaged together or separately. The packaged IC(s) may be secured to or otherwise supported by the PCB (or PCB assembly) and may be directly or indirectly coupled to the interconnect306. In other cases, the target304of the controller110, the interconnect306, and the one or more memory controllers308may be integrated together into one IC. In some of such cases, this IC may be coupled to a PCB, and one or more modules for the memory components may also be coupled to the same PCB, which can form a CXL memory device108. This memory device108may be enclosed within a housing or may include such a housing. The components of the memory device108may, however, be fabricated, packaged, combined, and/or housed in other manners.

As illustrated inFIG.3, the target304, including the link controller314thereof, can be coupled to the interconnect306. Each memory controller308of the multiple memory controllers308-1and308-2can also be coupled to the interconnect306. Accordingly, the target304and each memory controller308of the multiple memory controllers308-1and308-2can communicate with each other via the interconnect306. Each memory controller308is coupled to at least one DRAM310. As shown, each respective memory controller308of the multiple memory controllers308-1and308-2is coupled to at least one respective DRAM310of the multiple DRAMs310-1and310-2. Each memory controller308of the multiple memory controllers308-1and308-2may, however, be coupled to a respective set of multiple DRAMs or other memory components.

Each memory controller308can access at least one DRAM310by implementing one or more memory access protocols to facilitate reading or writing data based on at least one memory address. The memory controller308can increase bandwidth or reduce latency for the memory accessing based on a type of the memory or an organization of the memory components, such as the multiple DRAMs. The multiple memory controllers308-1and308-2and the multiple DRAMs310-1and310-2can be organized in many different manners. For example, each memory controller308can realize one or more memory channels for accessing the DRAMs. Further, the DRAMs can be manufactured to include one or more ranks, such as a single-rank or a dual-rank memory module. Each DRAM310(e.g., at least one DRAM IC chip) may also include multiple banks, such as 8 or 16 banks.

A forward path of the memory device108may include one or more memory request queues. A return path of the memory device108may include one or more memory response queues. These queues may be present in, for example, the controller110, a memory controller308, a memory array, such as the DRAM310, and so forth. Examples of a forward path and a return path are described next as part of an accessing operation for the memory device108.

This document now describes examples of the host device104accessing the memory device108. The examples are described in terms of a general memory access (e.g., a memory request) which may include a memory read access (e.g., a memory read request for a data retrieval operation) or a memory write access (e.g., a memory write request for a data storage operation). The processor114can provide a memory access request352to the initiator302. The memory access request352may be propagated over a bus or other interconnect that is internal to the host device104. This memory access request352may be or may include a read request or a write request. The initiator302, such as the link controller312thereof, can reformulate the memory access request352into a format that is suitable for the interconnect106. This reformulation may be performed based on a physical protocol or a logical protocol (including both) applicable to the interconnect106. Examples of such protocols are described below.

The initiator302can thus prepare a request316and transmit the request316over the interconnect106to the target304. The target304receives the request316from the initiator302via the interconnect106. The target304, including the link controller314thereof, can process the request316to determine (e.g., extract, decode, or interpret) the memory access request. Based on the determined memory access request, and as part of the forward path of the memory device108, the target304can forward a memory request354over the interconnect306to a memory controller308, which is the first memory controller308-1in this example. For other memory accesses, the targeted data may be accessed with the second DRAM310-2through the second memory controller308-2. Thus, the first memory controller308-1receives the memory request354via the internal interconnect306.

The first memory controller308-1can prepare a memory command356based on the memory request354. The first memory controller308-1can provide the memory command356to the first DRAM310-1over an interface or interconnect appropriate for the type of DRAM or other memory component. The first DRAM310-1receives the memory command356from the first memory controller308-1and can perform the corresponding memory operation. Based on the results of the memory operation, the first DRAM310-1can generate a memory response362. If the memory request316is for a read operation, the memory response362can include the requested data. If the memory request316is for a write operation, the memory response362can include an acknowledgement that the write operation was performed successfully. As part of the return path of the memory device108, the first DRAM310-1can provide the memory response362to the first memory controller308-1.

Continuing the return path of the memory device108, the first memory controller308-1receives the memory response362from the first DRAM310-1. Based on the memory response362, the first memory controller308-1can prepare a memory response364and transmit the memory response364to the target304via the interconnect306. The target304receives the memory response364from the first memory controller308-1via the interconnect306. Based on this memory response364, and responsive to the corresponding memory request316, the target304can formulate a response318for the requested memory operation. The memory response318can include read data or a write acknowledgement and be formulated in accordance with one or more protocols of the interconnect106.

To respond to the memory request316from the host device104, the target304of the memory device108can transmit the memory response318to the initiator302over the interconnect106. Thus, the initiator302receives the response318from the target304via the interconnect106. The initiator302can therefore respond to the “originating” memory access request352, which is from the processor114in this example. To do so, the initiator302prepares a memory access response366using the information from the response318and provides the memory access response366to the processor114. In these manners, the host device104can obtain memory access services from the memory device108using the interconnect106. Example aspects of an interconnect106are described next.

The interconnect106can be implemented in a myriad of manners to enable memory-related communications to be exchanged between the initiator302and the target304. Generally, the interconnect106can carry memory-related information, such as data or a memory address, between the initiator302and the target304. In some cases, the initiator302or the target304(including both) can prepare memory-related information for communication across the interconnect106by encapsulating such information. The memory-related information can be encapsulated or incorporated into, for example, at least one packet (e.g., at least one flit). One or more packets may include at least one header with information indicating or describing the content of each packet.

In example implementations, the interconnect106can support, enforce, or enable memory coherency for a shared memory system, for a cache memory, for combinations thereof, and so forth. Thus, the memory device108can operate in a cache coherent memory domain in some cases. Additionally or alternatively, the interconnect106can be operated based on a credit allocation system. Thus, the initiator302and the target304can communicate using, for example, a credit-based flow control mechanism320. Possession of a credit can enable an entity, such as the initiator302, to transmit another memory request316to the target304. The target304may return credits to “refill” a credit balance at the initiator302. The credit logic124of the target304or the credit logic126of the initiator302(including both instances of credit logic working together in tandem) can implement a credit-based communication scheme across the interconnect106. Example aspects of credit-based communication protocols are described below with reference toFIGS.4and5.

The system300, the initiator302of the host device104, or the target304of the memory device108may operate or interface with the interconnect106in accordance with one or more physical or logical protocols. For example, the interconnect106may be built in accordance with a Peripheral Component Interconnect Express® (PCIe or PCI-E) standard. Applicable versions of the PCIe standard may include 1.x, 2.x, 3.x, 4.0, 5.0, 6.0, and future or alternative versions of the standard.

In some cases, at least one other standard is layered over the physical-oriented PCIe standard. For example, the initiator302or the target304can communicate over the interconnect106in accordance with a Compute Express Link™ (CXL) standard. Applicable versions of the CXL standard may include 1.x, 2.0, and future or alternative versions of the standard. Thus, the initiator302and/or the target304may operate so as to comport with a PCIe standard and/or a CXL standard. A device or component may comprise or operate in accordance with a CXL Type 1, Type 2, or Type 3 device. A CXL standard may operate based on credits, such as request credits, response credits, and data credits. Example aspects of credit types, credit allocation, credit usage, and flow control via credits are described next with reference toFIGS.4and5.

Example Techniques and Hardware

FIG.4illustrates, at400generally, examples of controllers for an initiator302and a target304that can communicate across an interconnect106that employs a credit-based protocol and that can implement aspects of response-based interconnect control. The initiator302can include a link controller312, and the target304can include a link controller314. As also shown inFIGS.1and3, the link controller312can include credit logic126, and the link controller314can include credit logic124. The credit logic124and the credit logic126can support implementations of a credit-based flow control mechanism320that authorizes or permits one or more communications based on possession of at least one credit.

In example implementations, the link controller312or the link controller314, including both link controllers, can communicate across the interconnect106in accordance with a credit-based protocol. A credit-based protocol can be realized using, for instance, the credit-based flow control mechanism320. To do so, the credit logic126can monitor a quantity of one or more credits412, and the credit logic124can monitor one or more credits414. Generally, the credit logic126permits the link controller312to transmit a communication, such as a request316, to the link controller314based on the one or more credits412. Transmitting the request316may use or “consume” one credit of the one or more credits412. Based on the one or more credits414that are to be returned, the credit logic124at the link controller314can modulate the rate of transmission from the link controller312by managing the transmission of at least one credit return420. The credit return420can replenish an indicated quantity of the one or more credits412at the credit logic126. This credit usage is described further below.

As illustrated inFIG.4, the link controller312can include at least one request queue402, at least one arbiter404, at least one response queue406, and at least one instance of the credit logic126. The link controller314can include at least one request queue452, at least one arbiter454, at least one response queue456, and at least one instance of the credit logic124. In some cases, a request queue402or452may be split into a read path and a write path. Thus, the request queue402may include at least one read queue408and at least one write queue410. Similarly, the request queue452may include at least one read queue458and at least one write queue460. Additionally or alternatively, although not shown inFIG.4, a response queue406or456may be split into a read path and a write path. Thus, the response queue406may include at least one read response queue and at least one write response queue for the link controller312. Similarly, the response queue456may include at least one read response queue and at least one write response queue for the link controller314.

In example operations for an initiator302, the link controller312can receive a memory access request352at the request queue402. The request queue402routes the request into the read queue408or the write queue410based on whether the memory access request is for a read operation or a write operation, respectively. The arbiter404controls access to the interconnect106based on instructions or commands from the credit logic126. The credit logic126authorizes the arbiter404to transmit a request316over the interconnect106based on possession of the one or more credits412. For example, the credit logic126may permit the arbiter404to transmit one request316per one available credit412(e.g., a one-to-one ratio of request transmission to credit). If the credit logic126does not currently possess any credits412, the arbiter404can be prevented from transmitting a request316(e.g., by the credit logic126blocking or not authorizing such a transmission).

The response queue406can buffer multiple responses318received from the link controller314via the interconnect106. Each response318may include a least one memory response (e.g., with read data or a write acknowledgment) or at least one credit return420, including the memory response and the credit return in some transmissions. Thus, a response318may include a credit return420bundled with a memory response. For a memory response, the response queue406buffers the response until the response queue406can provide the memory access response366to the processor114(ofFIGS.1and3). For a credit return420, the response queue406, including associated logic, can forward the quantity of credits returned420to the credit logic126. Alternatively, separate circuitry of the link controller312may provide the credit return420to the credit logic126. Based on the credit return420, the credit logic126can replenish at least a portion of the credits412.

Continuing with example operations, but for a target304, the link controller314can receive the request316at the request queue452. The request queue452can then route the request316into the read queue458or the write queue460depending on whether it is a read request or a write request, respectively. The arbiter454can select a read request from the read queue458or a write request from the write queue460for transmission as the memory request354to a backend component, such as a memory controller. Responsive to transmission of a memory request354, which corresponds to a request316that was stored in the request queue452, the arbiter454notifies the credit logic124that the request316has been transmitted to a backend component of the memory device108(e.g., ofFIGS.1-3). Accordingly, the credit logic124can add a credit414to the collection of one or more credits414that are earmarked to be returned to the link controller312.

Thus, the credit logic124can track (e.g., maintain a record of) how many credits414can be returned to the credit logic126because the link controller314has forwarded a corresponding request316from the request queue452to a downstream component of the memory device108. The credit logic124can communicate with the response queue456responsive to the presence of credits414that are to be returned to the credit logic126. When a memory response364is received at the response queue456, the response queue456can store the memory response364. In conjunction with transmitting to the link controller312the memory response364as a response318, the response queue456can include at least one credit return420(e.g., in a same FLIT or other packet). The credit return420can indicate a quantity of one or more credits414that are being returned to the credit logic126to increase the quantity of credits412.

In these manners, the link controller314can use the credit-based protocol to control (e.g., block, gate, modulate, or moderate) the flow of requests316from the link controller312. This can enable the link controller314to prevent the request queue452from overflowing from receiving too many requests316(e.g., from receiving requests316faster than the requests can be forwarded to downstream memory components). Additionally or alternatively, a credit-based protocol can also be used by the link controller312to control the flow of responses318from the link controller314to the link controller312. The response queue456of the link controller314may be blocked from transmitting a response318unless the credit logic124has a “response” credit (not separately shown inFIG.4) to authorize such a response transmission. These response credits may be different from the one or more “request” credits414relating to the requests316. In such scenarios, the credit logic126of the link controller312may return the response credits to the credit logic124responsive to issuances of memory access responses366from the response queue406. Hence, the initiator302and the target304can implement the credit-based flow control mechanism320bidirectionally.

Various approaches can be employed for a credit-based communication protocol. For example, a credit may correspond to a transmission across an interconnect, to a packet, to a flit, or to a request or response. A single credit may correspond to a single instance of any of the preceding examples or to multiple instances of such examples. In some cases a transmission may include multiple requests and/or multiple responses, such as by encapsulating them into a packet or flit. In some systems, a credit may correspond generally to any type of request or response so that, e.g., an initiator can transmit any kind of request or response if the initiator possesses a credit. Additionally or alternatively, a credit may be specific to one or more types of requests or responses or other communications. Examples of communication types can include requests, responses, read-related transmissions, write-related transmissions, read-related requests, write-related requests, read-related responses, and write-related responses. Credits may also be particular to whether or not data is allowed to be included in the corresponding transmission. These and other communication traits may be further combined to create still-more specific types of credits.

By way of example, but not limitation, this document describes some implementations in terms of a credit protocol employed by certain CXL systems. Generally, from the perspective of a memory device, the credit-based flow control mechanism for CXL enables the memory device to employ “backpressure” against a host device if one or more buffers of the memory device are full and therefore cannot receive any more requests on the forward path (or any more responses on the return path). In some example systems, there can be three types of credits on an initiator device or a target device to control the flow of traffic between them. These three credit types can be represented by ReqCrd, DataCrd, and RspCrd. More specifically, these three examples are a request credit (ReqCrd), a data credit (DataCrd), and a response credit (RspCrd).

This document now describes example traffic classifications. For communications from the initiator to the target (e.g., from a host device to a memory device), two traffic classifications are:REQ: Request without Data—generally Read Requests. These can be controlled using ReqCrd.RwD: Request with Data—generally Write Requests. These can be controlled using DataCrd.
For communications from the target to the initiator (e.g., from the memory device to the host device), two traffic classifications are:DRS: Response with Data—generally Read Responses. These can be controlled using DataCrd.NDR: Response without Data—generally Write Acknowledgements.

These can be controlled using RspCrd.

These example CXL terms can be applied to the general system ofFIG.4. At a host device, which can be represented by the initiator302, the credit logic126decrements the ReqCrd value (e.g., a quantity for the one or more credits412) responsive to forwarding a FLIT (e.g., a flit with one read request) across the interconnect106to the target304. If the ReqCrd value reaches zero, the credit logic126causes the arbiter404to cease sending read FLITs (e.g., the credit logic126blocks transmission of further read requests). At a memory device, which can be represented by the target304, the link controller314processes the received FLIT. The arbiter454forwards a request316that was included in the FLIT to backend memory as a read or write memory request354(e.g., a read request for a ReqCrd example). The credit logic124increments the ReqCrd value (e.g., the quantity of the collection of credits414that are to be returned) responsive to the forwarding of the memory request354.

The link controller314return the request credits (ReqCrd) accumulated at the credit logic124to the credit logic126with at least one response318. This credit return420may be associated with a decrement of the ReqCrd at the credit logic124and an increment of the ReqCrd at the credit logic126. In some locations of this document, “credits” and credit-related communications may be described with reference to a CXL standard. Nonetheless, implementations of response-based interconnect control, as described in this document, can apply to and benefit other credit-based systems that operate in a similar or analogous manner. The credit-based flow control mechanism320is described with reference toFIG.4in terms of ReqCrds by way of example only. The principles are applicable to other credit types, which are described further with reference toFIG.5.

FIG.5illustrates, at500generally, examples of credit-based feedback loops to control communication flows between two or more devices in an environment in which aspects of response-based interconnect control can be implemented. In example implementations, the two or more devices can include a host device104and a memory device108. The two or more devices may comport with at least one CXL standard that includes memory requests and memory responses. Four example credit-based feedback loops510,520,530, and540are shown. Each credit-based feedback loop includes an active or affirmative communication, such as a memory request or a memory response, and an associated credit return. The affirmative communication can include a read or write request or a read or write response. The credit return can correspond to a request credit (ReqCrd), a response credit (RspCrd), or a data credit (DataCrd).

In a first example, the credit-based feedback loop510includes a read request512and a request credit514. In operation, the host device104transmits the read request512to the memory device108. In response to the link controller314forwarding the read request512to one or more downstream components of the memory device108, the credit logic124can return the request credit514to the initiator302. Responsive to return of the request credit514, the credit logic126adds another request credit to the request credit repository or count516. While the request credit count516is greater than zero (or the request credit repository516is nonempty), the credit logic126can permit the link controller312to transmit another read request512. In this manner, the link controller314of the target304can provide feedback or backpressure to the link controller312of the initiator302to control (e.g., block, slow, increase/decrease, or otherwise modulate) a flow of the read requests512.

In a second example, the credit-based feedback loop520includes a write request522and a data credit524. In operation, the host device104transmits the write request522to the memory device108. In response to the link controller314forwarding the write request522to one or more downstream components of the memory device108, the credit logic124can return the data credit524to the initiator302. Responsive to return of the data credit524, the credit logic126adds another data credit to the data credit repository or count526. While the data credit count526is greater than zero (or the data credit repository526is nonempty), the credit logic126can permit the link controller312to transmit another write request522. In this manner, the link controller314of the target304can provide feedback or backpressure to the link controller312to control (e.g., block, slow, increase/decrease, or otherwise modulate) a flow of the write requests522.

The first and second examples above relate to the target304controlling a communication flow (e.g., of memory requests) from the initiator302. The credit-based feedback loops can, however, operate in the opposite direction. The third and fourth examples below relate to the initiator302controlling a communication flow (e.g., of memory responses) from the target304.

In a third example, the credit-based feedback loop530includes a read response532and a data credit534. In operation, the memory device108transmits the read response532to the host device104. In response to the link controller312forwarding the read response532to one or more upstream components of the host device104(e.g., to a processor114ofFIG.3), the credit logic126returns the data credit534to the credit logic124. Responsive to return of the data credit534, the credit logic124adds another data credit to the data credit repository or count536. While the data credit count536is greater than zero, the credit logic124can permit the link controller314to transmit another read response532. In this manner, the link controller312of the initiator302can provide feedback or backpressure to the link controller314of the target304to control (e.g., block, slow, increase/decrease, or otherwise modulate) a flow of the read responses532.

In a fourth example, the credit-based feedback loop540includes a write response542and a response credit544. In operation, the memory device108transmits the write response542to the host device104. In response to the link controller312forwarding the write response542to one or more upstream components of the host device104(e.g., to a processor114ofFIG.3), the credit logic126returns the response credit544to the credit logic124. Responsive to return of the response credit544, the credit logic124adds another response credit to the response credit repository or count546. While the response credit count546is greater than zero, the credit logic124can permit the link controller314to transmit another write response542. In this manner, the link controller312of the initiator302can provide feedback or backpressure to the link controller314of the target304to control (e.g., block, slow, increase/decrease, or otherwise modulate) a flow of the write responses542.

The credit-based feedback loops described above enable an initiator302or a target304to control a quantity or rate of received memory responses or memory requests, respectively. For the memory device108, this control may relate to ensuring that a queue at the target304(e.g., the request queue452or the response queue456) does not overflow. If the decision to return a credit to the initiator302is based solely on a memory request being forwarded out of the request queue452, memory requests and/or responses may become too prevalent in backend components, such as an interconnect306, a memory controller308, or a DRAM310(e.g., each ofFIG.3). Likewise, if the decision to return a credit to the initiator302is based solely on a memory response being forwarded out of the response queue456, memory requests and/or responses may become too prevalent in backend components, such as the interconnect306, the memory controller308, or the DRAM310(e.g., each ofFIG.3). For the host device104, this control may relate to ensuring that a queue at the initiator302(e.g., the response queue406) does not overflow. If the decision to return a credit to the target304is based solely on a memory response being forwarded out of the response queue406, memory responses may become too prevalent in upstream components, such as the processor114, a memory controller thereof, or an interconnect of the host device.

To at least alleviate the potential overcrowding of communications (e.g., requests and/or responses) in queues besides those that are identified above in the controller110, such as overcrowding in the backend components of the memory device108, the techniques described herein can be implemented. Certain ones of these techniques monitor memory responses that are present at the memory device. For example, a counter can include a value indicative of a quantity of memory responses that are currently stored in the response queue456of the memory device. The transmission of credit returns (or the return of credits) can be delayed based on the monitoring and/or the value of the counter to at least slow the transmission of additional memory requests to the memory device108. By preventing the buffers in the memory device108from becoming overcrowded, the techniques can decrease the pin latency of the memory device108. Example implementations for response-based interconnect control are described below with reference toFIGS.6-11.

FIG.6illustrates example architectures600to control a communication flow between two or more devices in accordance with certain implementations for response-based interconnect control. As shown on the left (as depicted inFIG.6), a controller118includes at least one request queue402, at least one response queue406, and at least one instance of credit logic126. The credit logic126can include at least one gate602and at least one credit repository or credit counter604. As shown on the right, a controller110includes at least one request queue452, at least one response queue456, and at least one instance of credit logic124. The credit logic124can include at least one gate612, at least one instance of credit return logic614, at least one stored response counter616, and at least one lookup table618(LUT618).

In example implementations, the controller118at an initiator302(e.g., ofFIGS.3-5) can transmit a request316from the request queue402based on a state of the gate602. If the gate602is open (e.g., if a corresponding switch is closed), the controller118can transmit a request316to the controller110. On the other hand, if the gate602is closed (e.g., if a corresponding switch is opened), the credit logic126can block or prevent the controller118from transmitting a request316. The credit logic126can control the state of the gate602based on the condition or value of the credit repository or credit counter604.

If the credit repository604is empty or if the credit counter604has a value that is less than one, the credit logic126closes the gate602using a control signal652to prevent transmission of requests. In contrast, if the credit repository604has at least one credit or if the credit counter604has a value greater than zero, the credit logic126opens the gate602(including keeps the gate602open) using the control signal652to permit or allow the transmission of requests316. As described above with reference toFIG.4, responsive to receipt of a credit return420, a credit is added to the credit repository604, or the value of the credit counter604is incremented. If multiple credits are returned in a single response318, the credit logic126may add multiple credits “jointly” to the credit repository604, or the credit logic126may increment the value of the credit counter604by an amount greater than one.

At a target304(e.g., ofFIGS.3-5), the controller110adds a received request316to the request queue452. To process a memory request, the controller110transmits a request316to a downstream or backend memory component as a memory request354. The credit logic124notifies the credit return logic614of this transmission. The credit return logic614can include a counter (e.g., a second counter (not shown) of the credit logic124) that has a value indicative of a quantity of one or more credit returns that are ready or available to be transmitted to the controller118. Responsive to the controller110removing another request316from the request queue452, the credit return logic614can allocate another credit to be returned as a credit return420. To do so, the credit return logic614can increment the second counter.

By establishing an appropriate quantity of credits for the system, which may be based on a size of the request queue452, this aspect of a credit-based flow control protocol can ensure that a maximum capacity of the request queue452is not exceeded. This aspect may not, however, adequately protect the memory device from oversubscribing backend memory components or the response queue456. For example, the “internal” interconnect306(ofFIG.3) may become too busy, one or more queues of the memory controllers308-1and308-2may become overfilled, and/or the multiple DRAMs310-1and310-2may be unable to fulfill the memory requests as fast as the requests are delivered to the memory arrays.

To protect the backend memory components of the memory device108and/or the response queue456from becoming oversaturated, the credit logic124can operate the illustrated components to manage how quickly and/or how frequently credits are returned at420to the controller118. The credit logic124can condition the transmission of credit returns420at least partially on a fill level of the response queue456. For example, the credit logic124can transmit credit returns420at a rate that is dependent on how many memory requests are currently stored in the response queue456.

To reduce the likelihood that a component, such as the interconnect106(e.g., ofFIGS.1-4) or a downstream memory controller of the memory device108, is rendered idle unnecessarily while still managing the pin latency of the memory device108, the credit logic124can flexibly condition the return of credits on a quantity of memory responses that are present in the response queue456. In some cases, the credit logic124can use the stored response counter616and the lookup table618. The stored response counter616can track a quantity of memory responses that are buffered in the response queue456. The credit logic124can condition the issuance of credit returns420on the quantity tracked by the stored response counter616using the lookup table618as described below. Although not shown inFIG.6, the response queue456may be split into a read path and a write path. Thus, the response queue456may include at least one read response queue and at least one write response queue, and the principles described herein may be applied separately to read responses and write responses (and the corresponding read requests and write requests and the associated credits).

In some implementations, the credit logic124permits a credit return420to be sent to the controller118in conjunction with every “Nth” memory response transmission. The value of “N” can be an integer that is based on the stored response counter616and mappings provided in the lookup table618. In example operations, the credit logic124increments a value620stored or maintained by the stored response counter616at654responsive to the receipt of each memory request364and the corresponding addition of the memory request364to the response queue456. The credit logic124decrements the value620stored or tracked by the stored response counter616at656responsive to transmission of a response318and a corresponding removal of the memory response from the response queue456. Accordingly, the value620can track how many memory requests are stored in the response queue456.

Meanwhile, the credit return logic614can enable a credit to be returned at658responsive to the memory request354being issued from the request queue452and/or responsive to a stored supply of credits that are available to be returned. However, the gate612can block the delivery of the credit return420. A state of the gate612may be open or closed, and the state can be established by a control signal660. If the gate612is open (e.g., a corresponding switch is closed), the credit logic124can permit at least one credit return420to pass for transmission to the controller118(e.g., with a memory response). On the other hand, if the gate612is closed (e.g., a corresponding switch is opened), the credit logic124can block or prevent the controller110from transmitting a credit return420.

The state of the gate612can be controlled responsive to the value620of the stored response counter616using the lookup table618. As described above, the value620can be increased at654responsive to receipt of a memory request364at the response queue456. To enable the value620to represent a quantity of memory requests present at the response queue456, the credit logic124can decrease (e.g., decrement) the value620of the counter616at656responsive to transmission of a memory response318and a removal of the memory response from the response queue456. Thus, the value620of the stored response counter616can track the quantity of memory requests that are stored within the response queue456. As the value620increases, this can indicate, for example, that the backend memory components are producing responses too quickly and/or that the interconnect106(e.g., ofFIGS.3and4) cannot accept the memory responses sufficiently fast. In either case, the associated increasing congestion within the memory device108can appreciably reduce memory responsiveness, including by increasing latency.

To reduce or prevent at least some of these potential avenues of congestion, the credit logic124can modulate the arrival of additional requests316from the controller118by controlling (e.g., delaying or slowing) how credit returns420are transmitted to the controller118. To do so, the credit logic124can close the gate612to block the credit return logic614from providing a credit return. In some implementations, the credit logic124controls a state of the gate612responsive to the value620of the stored response counter616and based on the lookup table618. The credit logic124can access the lookup table618based on the value620, which is indicative of a quantity of memory responses stored in the response queue456. The lookup table618maps ranges of quantities to rates of transmission for credit returns.

Thus, the credit logic124can obtain a rate of transmission for the credit returns from the lookup table618based on the value620of the counter616. The rate can be related to a ratio of memory responses transmitted with a credit return to memory responses transmitted, such as combined memory responses transmitted. The combined memory responses transmitted can include memory responses transmitted with a credit return and memory responses transmitted without a credit return. This ratio can change over time based on changes to the value620. By way of example only, the ratio can be adjusted over time across one more settings, such as 1-to-1, 1-to-2, 1-to-3, 1-to-4, 1-to-5, 1-to-8, and so forth.

The lookup table618can include multiple entries. Each entry can map a range of quantities (e.g., of the value620) of multiple ranges of quantities to a number of memory responses to be transmitted for each credit return to be transmitted. For example, a value620of “X” memory responses stored in the response queue456can be part of a range of quantities that maps to “Y” memory responses being transmitted for each credit return420that is being transmitted. For instance, a value620of “23” may index to a range of “20-32” that maps to “4” memory responses per credit return. In this instance, the controller110transmits a credit return420each four transmissions of a memory response. In other words, each “4th” transmission of a memory response may include or be accompanied by a credit return420. In such instances, there are three memory response transmissions without a credit return between any two consecutive memory response transmissions that include a credit return420. Example implementations of a lookup table618are described below with reference toFIG.7.

The rate may be adjusted over time as the value620changes. Generally, the credit logic124can decrease the rate of the transmitting of the credit returns420responsive to increases of the quantity of the multiple memory responses stored in the response queue456. Conversely, the credit logic124can increase the rate of the transmitting of the credit returns420responsive to decreases of the quantity of the multiple memory responses stored in the response queue456. By changing the rate at which credit returns420are transmitted back to the controller118, the controller110can slow the rate of newly arriving memory requests316. This can smooth request and response traffic within the memory device108and therefore decrease latency.

The credit logic124can use a number obtained from the lookup table618as a basis to provide a control signal660to the gate612to establish a closed state or an open state thereof. For example, if an “Nth” memory response is to be transmitted, the credit logic124can open the gate612to permit a credit return420to flow from the credit return logic614at662to the credit logic126of the controller118with that “Nth” memory response. If, however, a memory response within a cycle or interval is not the “Nth” memory response, the credit logic124can close the gate612to prevent credit returns420from flowing from the credit return logic614to the credit logic126. Over some time period, as memory responses364are received more slowly from the backend memory components (or the interconnect106becomes more available), the value620decreases due to the decrement signal656. Responsive to the value620, which is indicative of the quantity of memory responses within the response queue456, falling into another range of quantities, the credit logic124can open the gate612more frequently or even keep the gate612open.

Using these techniques, the credit logic124can modulate how quickly or how frequently requests316are received from the controller118based on how “busy” the backend memory components of the memory device108are and/or how quickly such backend memory components are providing new memory responses364. These techniques can enable the memory device108to avoid becoming overwhelmed and/or oversubscribed and, therefore, enable the memory device108to provide some specified quality of service. Additional example approaches to implementing the credit logic124, including a lookup table618, are described next with reference toFIG.7.

FIG.7illustrates other example architectures700to control a communication flow between two or more devices in accordance with certain implementations for response-based interconnect control. As illustrated, the architectures700include multiple flit handlers752,754,756, and758that process FLITs, such as by creating or interpreting a FLIT. At the controller118, the credit logic126can include at least one credit counter702, at least one comparator704, and at least one switch706. At the controller110, the credit logic124can include the counter616(e.g., the stored response counter616), at least one credit return counter712, at least one comparator714, at least one switch716, at least one register718, at least one counter720(or other counter720), at least one comparator722, at least one AND gate724, and at least one multiplexer726.

Each of these components can be coupled one to another at least as shown inFIG.7. For instance, the register718and the counter720may be coupled to inputs of the comparator714, which can have an output thereof be coupled to a control input of the switch716. The switch716can be coupled between the lookup table618and the register718. The output of the comparator714may also be coupled to a reset input of the counter720and an input of the AND gate724. An output of the AND gate724can be coupled to a control input of the multiplexer726. The multiplexer726can be provided a “0” or a “1” input and can provide an output to an input of the flit handler756.

In example implementations, the flit handlers752and756produce FLITs, and the flit handlers754and758interpret or unpack FLITs (e.g., a FLow control unIT (FLIT)). Thus, the flit handler752of the controller118can transmit a request FLIT760, and the flit handler754of the controller110can receive the request FLIT760. Analogously, the flit handler756of the controller110can transmit a response FLIT762, and the flit handler758of the controller118can receive the response FLIT762. Responsive to receipt of a response FLIT762, the flit handler758can forward a response to the response queue406and provide one or more credit returns420to the credit counter702of the credit logic126.

The credit logic126can maintain a count of available credits (e.g., request credits516or data credits526ofFIG.5) using the credit counter702. The comparator704can compare a current count from the credit counter702to a set value, such as zero. If the count is greater than zero, the credit logic126can close the switch706to permit requests to flow. If the count is not greater than zero, the credit logic126can open the switch706to block requests from flowing. If requests are flowing, the flit handler752can prepare a request FLIT760and indicate to the credit logic126that the count of the credit counter702is to be decremented responsive to transmission of the request FLIT760. Accordingly, while the controller118possesses at least one credit, the flit handler752can transmit a request FLIT760to the flit handler754of the controller110.

The flit handler754can unpack the request FLIT760and forward at least one memory request to the request queue452. The request queue452, or associated logic, can notify the credit logic124that the count of the credit return counter712is to be incremented in conjunction with the request queue452removing a memory request and forwarding it to a backend memory component as a memory request354. Thus, responsive to a request being forwarded from the request queue452, the credit logic124increments a count of the credit return counter712. The comparator722compares a count of the credit return counter712to a value, such as zero (0). If the count is less than or equal to zero, the comparator722can output a “0.” If the count is greater than zero, the comparator722can output a “1.” These comparison outputs are coupled to an input (e.g. a first input) of the AND gate724.

Another input (e.g., a second input) of the AND gate724receives a signal774from response-related control circuitry, which is described below. An output of the AND gate724is coupled to a control input of the multiplexer726. If the output of the AND gate724is high, then the multiplexer726produces a signal772. The multiplexer726forwards an affirmative signal772to the flit handler756. The affirmative signal772instructs the flit handler756to include a credit return420in the next response FLIT762with one or more memory responses from the response queue456. On the other hand, if the output of the AND gate724is low, then the multiplexer726forwards a negative signal772to the flit handler756. The negative signal772instructs the flit handler756to omit or omit including a credit return420in the next response FLIT762carrying one or more memory responses from the response queue456. The output of the AND gate724depends also on the second input, which receives the signal774from the comparator714. The comparator714operates as described next.

In example operations at the controller110, the credit logic124tracks a quantity of memory responses present at the response queue456using the counter616, which has the value620. The credit logic124increments the counter616(e.g., increases the value620by at least one) responsive to a memory request364being added to the response queue456. The credit logic124decrements the counter616(e.g., decreases the value620by at least one) responsive to a memory request being removed from the response queue456, including responsive to the memory request being transmitted to the controller118as part of at least one response FLIT762.

The credit logic124applies the value620to the lookup table (LUT)618. The credit logic124obtains a number of memory requests to include in each interval based on the value620stored by the counter616. As explained further below, for each interval of some number (e.g., “N,” which can be an integer) of memory request transmissions, the credit logic124authorizes one credit return420for transmission. By adjusting the number “N” of memory requests per interval, the rate of transmission of credit returns420can be changed. The adjustment can be based on the quantity of memory responses in the response queue456using the value620and the lookup table618.

The lookup table618includes multiple entries. Each entry corresponds to a range of quantities of memory requests currently stored in the response queue456. Each entry maps the range to a corresponding number “N” of memory requests per interval. An example of a lookup table618is presented below in Table 1.

The variables A, B, C, and D can represent constants for different quantities of memory requests that may be present in the response queue456. Generally, A<B<C<D. Each range corresponds to a quantity of slots (“X”) in the response queue456that are filled or occupied. These variables can be tuned based on the components of the memory device to achieve particular bandwidth or latency targets. The variables of the LUT618can be hardwired, set (e.g., programmed or fused) during manufacturing or testing, changeable in the field or during assembly of a part, or adjustable in real time during operation of a device. A lookup table618can have more or fewer than the five entries with the five ranges as indicated above.

Each entry maps a range of quantities to a size (“Y”) of an interval of a number of memory requests that are transmitted during or for each interval. The indicated sizes are provided by way of example only, for other sizes for each interval may be used. A lookup table618can additionally or alternatively include an entry (e.g., a range of quantities) that causes no credit returns to be transmitted until the value620is lowered to another entry of the table. Further a lookup table618may additionally or alternatively include an entry having a range that corresponds to unrestrained credit returns such that multiple credit returns can be transmitted with a single memory response, as potentially impacted by other credit-related control loops.

Continuing withFIG.7, the credit logic124obtains a number (“N”) of memory responses (Y) for each interval based on the value620(X) using the lookup table618. The lookup table618outputs the number “N” and provides the number “N” to one side of the switch716. Consider a state of the circuitry in which the comparator714closes the switch716using the signal774. The number “N” is provided to the register718and is loaded therein. The register718establishes the number of memory requests to be transmitted in each interval for a given interval.

The comparator714also provides the signal774to the counter720and the other input (e.g., the second input) of the AND gate724. The signal774resets the counter720—e.g., to zero (0). The counter720holds a number representing how many memory responses have been transmitted in a current interval, which can start with zero (0). With regard to the AND gate724, if the comparator722is providing a positive/high signal to the first input, the AND gate724outputs a high signal at776to cause the multiplexer726to authorize another credit return transmission with the signal772.

With the counter720being reset, a new interval is started that is to have “N” additional memory response transmissions. Each time a memory response is removed from the response queue456, in addition to decrementing the counter616, the credit logic124increments the counter720, which tracks how many memory responses have been transmitted during the current interval. Meanwhile, the comparator714is comparing the number of memory responses that have been transmitted from the counter720to the number “N” of memory responses that are to be transmitted as held by the register718. After one or more increments of the counter720, the two numbers eventually become equal, and the comparator714detects this equivalency.

Thus, the comparator714determines that the number of memory responses that have been transmitted for a current interval has reached the number set for this interval as stored in the register718. The comparator714therefore drives the signal774to activate the switch716, reset the counter720, and indicate to the AND gate724that the interval is being concluded. Accordingly, the AND gate724, the multiplexer726, and the flit handler756can authorize transmission of a credit return420for the current interval. The credit logic124can also update the number “N” stored in the register718using the lookup table618based on the current value620of the counter616responsive to the switch716being closed.

By modulating (e.g., slowing, limiting, or moderating) the rate at which credits at the host device get replenished, described techniques can throttle the host's ability to send traffic to the memory device. This throttling can reduce queuing latency on the memory device response buffer or the memory device request buffer, including both of these buffers and/or backend buffers. This modulation can be accomplished based on how full the response queue is by, for example, adjusting a number “N” of memory requests that are transmitted each interval that is associated with a single credit return transmission. The number “N” can be obtained from a table with multiple ranges of quantities corresponding to an occupied depth of the response queue. The ranges of quantities and the values for the number “N” may be adjusted to account for slower backend memory subsystems or faster backend memory subsystems and/or a crowded or uncrowded interconnect106. The credit logic124may also adjust these ranges and values during operation based on a measured latency or bandwidth/throughput.

FIG.8illustrates, generally at800, multiple example intervals802corresponding to multiple response queue occupation depths in accordance with aspects of response-based interconnect control. The example intervals are described here with reference to Table 1, which is presented above in the description ofFIG.7as an example of a lookup table618. Table 1 includes multiple entries, with each entry associated in the first column with a range of quantities of memory responses that are present in the response queue456. In the second column of Table 1, each range of quantities corresponds to, and can be mapped to, a number “N” of memory responses318to be transmitted each interval. Once each interval, the memory device also transmits a credit return420, which can be transmitted with a memory response318.

As indicated in the legend ofFIG.8, each memory response318is represented by a shorter arrow with a closed point, and each credit return420is represented by a taller arrow with an open point.FIG.8includes example ranges #1, #2, #3, and #4. These ranges are presented in an order of increasing response queue occupation depth from the bottom of the page to the top of the page. In other words, range #1 corresponds to a response queue that has the fewest memory responses stored in it (e.g., the lowest response queue occupation depth) as compared to the other example depicted ranges.

Range #1 corresponds to a first interval802-1having one memory request per interval802-1. Range #2 corresponds to a second interval802-2having two memory requests per interval802-2. Range #3 corresponds to a third interval802-3having three memory requests per interval802-3. Range #4 corresponds to a fourth interval802-4having four memory requests per interval802-4. In each of these first, second, third, and fourth intervals802-1,802-2,802-3, and802-4, the memory device transmits a single credit return420per interval802. In the illustrated examples, each range corresponds to multiple regular intervals. As used herein, a regular interval can relate to intervals that have a same size or length—such as a common number of memory responses318.

As shown, the rate of transmission of credit returns decreases as the values of the range of quantities increases. Thus, the rate of credit return transmission is lower for the range #3 as compared to the range #2. By establishing the number “N” of memory response transmissions that occur per interval802and/or per one credit return transmission, the controller110can change the rate at which credits are replenished at the controller118to control how many requests are sent from the controller118over the interconnect106to the controller110over time.

InFIG.8, the transmissions of memory responses are shown individually and equally-spaced apart. However, this depiction is for clarity and understanding only. First, multiple memory responses can be transmitted together. Second, a series of consecutive memory response transmission may not occur at regular temporal intervals. Further, it should be noted that the intervals802can be defined in terms of a number “N” of memory responses and that the intervals802may not, therefore, correspond to equal lengths of time. Moreover, the ranges may correspond to different values of the number “N,” and other ranges may be included with more memory responses per interval802. Additionally, more, fewer, and/or different ranges (e.g., with wider or narrower value ranges) may be implemented. At other times of operation, the interval-based control may be paused such that other control loops may dominate the determination of how many or when credit returns420are transmitted.

The credits for the example architectures600and700can correspond at least to any of the credits described above, such as request credits, data credits, or response credits. The requests316may correspond, for instance, to read requests or write requests in a memory context. The responses318may correspond, for instance, to read responses or write responses in a memory context. Nonetheless, the principles described with reference toFIGS.6-8, as well as the other figures, are applicable to other types of credits, communications, and/or environments. Also, although certain concepts are described herein in the context of CXL Type 3 devices (“Memory Expanders”), the described techniques can be applied to other CXL device types and/or to non-CXL devices.

Further, the described principles are applicable to environments generally having credit-based communications. For example, a transmitter or initiator component may transmit requests besides memory requests. Similarly, a receiver or target component may receive “general” requests instead of or in addition to memory requests. Accordingly, the credit logic124may monitor the presence of pending responses at the target for non-memory responses, such as a computational response (e.g., for a cryptographic, AI accelerator, or graphics computation), a communications response (e.g., for transmitting or receiving a packet over some interface or network), and so forth. The described techniques can ensure that other types of targets—besides memory devices—do not become oversubscribed if the corresponding requests or responses are pending “too long” in the backend of the other target types while a response queue at a controller is emptying too slowly.

Example Methods

This section describes example methods with reference to the flow chart(s) and flow diagram(s) ofFIGS.9-11for implementing aspects of response-based interconnect control. These descriptions may also refer to components, entities, and other aspects depicted inFIGS.1-8, which reference is made only by way of example.

FIG.9illustrates a flow chart for example processes900that can implement aspects of response-based interconnect control. The processes900can include blocks902to906(e.g., for one or more branching decisions) and blocks912to924. The process900may be performed by, for instance, a target device that is in communication with an initiator device. In a memory environment, for example, the target device may be realized with a memory device, and an initiator device may be realized with a host device. The target device can additionally or alternatively be realized with a communication device (e.g., a modem), an accelerator device (e.g., for AI operations), a graphics device (e.g., a graphics card), and so forth. The initiator can be any device or component that is requesting a service or operation from the target.

At block902, a controller determines if a memory response has been added to a response queue. If so, at block912the controller increments a counter representing an occupied queue depth of the response queue with a value. After the incrementing at block912or responsive to a determination at block902that no memory response has been added, the process900continues with block904.

At block904, the controller determines if a memory response has been removed from the response queue. If so, then at block914the controller decrements the value of the counter representing the occupied queue depth of the response queue. At block916, the controller also increments another counter that tracks a number of transmitted memory responses in a current interval. After the incrementing at block916or responsive to a determination at block904that no memory response has been removed, the process900continues with block906.

At block906, the controller determines if the number of responses transmitted during the current interval (e.g., as maintained in the other counter) is equal to a number “N” of response transmissions per interval (e.g., as read from a register). If the two numbers are not equal, the process900continues with block902. The process900can continue while the response queue is being monitored and credit returns are being throttled based on an occupied depth of the response queue.

If, on the other hand, the controller determines at block906that the number of responses transmitted during the current interval is equal to the number “N” of response transmissions that are established per interval or for the current interval, the process900continues at block918. At block918, the controller accesses a lookup table (LUT) using the occupied queue depth value (e.g., as tracked in the counter) to obtain a next number “N” of memory responses to transmit for the next interval. At block920, the controller loads or reloads the register with the next number “N” of memory response transmissions for the next interval as obtained from the LUT based on the occupied queue depth.

At block922, the controller resets the other counter (e.g., to a zero value) to track anew the number of memory response transmissions in the next interval. At block924, the controller authorizes transmission of a credit return with a memory response for the next interval. Alternatively, the controller can authorize the credit return transmission with a memory response for the “current” interval. In other words, the credit return can be transmitted (e.g., with a memory response) at the start of an interval, at the end of the interval, or otherwise during a given interval. The acts shown inFIG.9may be performed in other orders and/or in partially or fully overlapping manners or in conjunction with any of the acts ofFIG.10or11. The hardware described herein may implement the acts ofFIG.9.

FIG.10illustrates a flow diagram for example processes1000that implement aspects of response-based interconnect control. The example processes1000include blocks1002to1006. At block1002, a memory device stores multiple memory responses in a response queue. For example, a controller110can store multiple memory responses in a response queue456at a memory device108. For instance, the response queue456may receive multiple memory responses364from a backend component of the memory device108and add such memory responses364to the response queue456.

At block1004, the memory device tracks a quantity of the multiple memory responses stored in the response queue. For example, the controller110can track a quantity of the multiple memory responses that are stored in the response queue456. To do so, credit logic124of the controller110may increment a value620of a counter616responsive to additions of memory responses to the response queue456or may decrement the value620of the counter616responsive to removals of memory responses from the response queue456.

At block1006, the memory device transmits credit returns at a rate that is based on the quantity of the multiple memory responses stored in the response queue. For example, the controller110can transmit credit returns420at a rate that is based on the quantity of the multiple memory responses that are stored in the response queue456. In some cases, the credit logic124may increase the rate of the transmission of the credit returns420responsive to decreases in the quantity (e.g., as reflected by the value620) of the multiple memory responses stored in the response queue456and may decrease the rate of the transmission of the credit returns420responsive to increases in the quantity (e.g., as reflected by the value620) of the multiple memory responses stored in the response queue456. The credit logic124may adjust the transmission rate of credit returns by adjusting a ratio of memory response transmissions with a credit return to total memory response transmissions (e.g., combined transmissions of memory response transmissions with a credit return and memory response transmissions without a credit return).

FIG.11illustrates a flow diagram for other example processes1100that implement aspects of response-based interconnect control. The example processes1100include blocks1102to1106. At block1102, the memory device receives multiple memory requests. For example, a memory device108can receive multiple memory requests316. For instance, a controller110of the memory device108may receive the multiple memory requests316from another device (e.g. a host device104) via an interconnect106.

At block1104, the memory device transmits multiple memory responses across multiple intervals of “N” memory responses per interval, with the multiple memory responses corresponding to the multiple memory requests, and with “N” representing a positive integer greater than one. For example, the memory device108can transmit multiple memory responses318across multiple intervals802of “N” memory responses per interval802. The multiple memory responses318can correspond to the multiple memory requests316(e.g., a response may complete or indicate completion of a requested operation), and the variable “N” can represent a positive integer greater than one. In some cases, the controller110may logically separate transmissions of the multiple memory responses318into multiple intervals802. Each interval802may include two (2), three (3), four (4), seven (7), sixteen (16), etc. memory responses318.

At block1106, the memory device transmits a credit return with each “Nth” memory response of each interval. For example, the memory device108can transmit a credit return420with each “Nth” memory response318of each interval802. To do so, the controller110may monitor how many memory responses318have been transmitted in a given interval802and may transmit a single credit return over the interconnect106in the given interval802. The transmission of the credit return420may occur at any point during the interval and may accompany at least one memory response318of the current interval802. Credit logic124of the controller110may adjust the variable “N” over time to change the rate of credit return transmissions, including based on a fullness level of a response queue456.

For the flow chart(s) and flow diagram(s) described above, the orders in which operations are shown and/or described are not intended to be construed as a limitation. Any number or combination of the described process operations can be combined or rearranged in any order to implement a given method or an alternative method. Operations may also be omitted from or added to the described methods. Further, described operations can be implemented in fully or partially overlapping manners.

Aspects of these methods may be implemented in, for example, hardware (e.g., fixed-logic circuitry or a processor in conjunction with a memory), firmware, software, or some combination thereof. The methods may be realized using one or more of the apparatuses, components, or other aspects shown inFIGS.1to8, the components or aspects of which may be further divided, combined, rearranged, and so on. The devices and components of these figures generally represent hardware, such as electronic devices, packaged modules, IC chips, or circuits; firmware or the actions thereof; software; or a combination thereof. Thus, these figures illustrate some of the many possible systems or apparatuses capable of implementing the described methods.

Unless context dictates otherwise, use herein of the word “or” may be considered use of an “inclusive or,” or a term that permits inclusion or application of one or more items that are linked by the word “or” (e.g., a phrase “A or B” may be interpreted as permitting just “A,” as permitting just “B,” or as permitting both “A” and “B”). Also, as used herein, a phrase referring to “at least one of” a list of items refers to any combination of those items, including single members. For instance, “at least one of a, b, or c” can cover a, b, c, a-b, a-c, b-c, and a-b-c, as well as any combination with multiples of the same element (e.g., a-a, a-a-a, a-a-b, a-a-c, a-b-b, a-c-c, b-b, b-b-b, b-b-c, c-c, and c-c-c, or any other ordering of a, b, and c). Further, items represented in the accompanying figures and terms discussed herein may be indicative of one or more items or terms, and thus reference may be made interchangeably to single or plural forms of the items and terms in this written description.

CONCLUSION

Although implementations for response-based interconnect control have been described in language specific to certain features and/or methods, the subject of the appended claims is not necessarily limited to the specific features or methods described. Rather, the specific features and methods are disclosed as example implementations for response-based interconnect control.