Patent Description:
<CIT> relates to methods and systems for dynamically adjusting credits used to distribute available bus bandwidth among multiple virtual channels, based on the workload of each virtual channel.

<CIT> relates to traffic control logic that supports a plurality of channels on a link. A plurality of reserved credit counters are provided to each identify reserved flow control credits for a corresponding one of the plurality of channels. A shared credit counter is provided to identify shared flow control credits to be shared between two or more of a plurality of virtual channels.

<CIT> relates to a memory controller with multiple ports. Each port is dedicated to a different type of traffic.

In accordance with the invention, there is provided: a memory subsystem as recited by claim <NUM>; and a system on chip as recited by claim <NUM>.

This document describes systems and techniques for modulating credit allocations in memory subsystems. The described systems and techniques can provide a feedback mechanism to a credit controller or one or more clients of a memory subsystem to improve the bandwidth at a memory interface. The memory controller monitors statistics associated with transaction requests served to one or more random access memories (RAMs) of the memory subsystem. The memory controller can then provide suggestions to the credit controller or the one or more clients to modulate the number of credits allocated to one or more clients with access to the RAMs. In this way, the described systems and techniques can improve the efficiency of the memory controller in managing the transaction requests and the bandwidth at the memory interface.

For example, a memory subsystem of a system on chip (SoC) includes a credit controller and a memory controller. The credit controller allocates a respective number of credits to one or more clients of the memory subsystem. The memory controller is operably connected to one or more RAMs. The memory controller includes a buffer, which can store transaction requests from the clients to access data in the RAMs. The memory controller can monitor, for each client, statistics of the transaction requests served by the memory controller and generate, based on the statistics, a signal to indicate to the credit controller to modulate the respective number of credits allocated to at least one of the clients.

This document also describes other methods, configurations, and systems for modulating credit allocations in memory subsystems.

This Summary is provided to introduce simplified concepts for modulating credit allocations in memory subsystems, which is further described below in the Detailed Description and Drawings. This Summary is not intended to identify essential features of the claimed subject matter, nor is it intended for use in determining the scope of the claimed subject matter.

The details of one or more aspects of modulating credit allocations in memory subsystems are described in this document with reference to the following drawings. The same numbers are used throughout multiple drawings to reference like features and components.

This document describes systems and techniques to modulate credit allocations in memory subsystems. Memory controllers in SoCs can subdivide an internal buffer into credits. A credit controller can allocate the credits to different clients or traffic classes of clients. In this way, the memory controller can allocate its buffer and schedule transactions to maximize bandwidth at a memory interface.

Memory subsystems can define a traffic class as a set of memory transactions that require a particular treatment to guarantee a certain quality of service (QoS) or obtain a particular system performance. The SoC or credit controller can allocate a different virtual channel identification (VCID) to each traffic class to simplify the credit allocation.

Existing memory controllers may include a relatively large buffer to improve efficiencies in serving transactions to the memory and improve the bandwidth at the memory interface. These memory subsystems generally statically allocate the number of credits assigned to the clients and traffic classes. Such memory subsystems, however, are not able to dynamically adjust the allocated credits to improve the bandwidth at the memory interface in response to actual memory transactions.

In contrast, the described systems and techniques modulate credit allocations to clients, traffic classes, or VCIDs based on real-time statistics of served transactions. In this way, the described systems and techniques can suggest modulation of the allocated credits. The memory controller can also provide a closed-feedback mechanism to a credit controller, or at least some clients, to enable efficient management and delivery of transaction requests. As a result, the described systems and techniques can increase the bandwidth at the memory interface.

As a non-limiting example, a memory subsystem of an SoC includes a credit controller and a memory controller. The credit controller can allocate a respective number of credits to one or more clients. The memory controller is operably connected to one or more RAMs and the credit controller. The memory controller also includes a buffer that can store transaction requests from the clients to access data in the RAMs. The memory controller can monitor, for each client of the one or more clients, statistics of the transaction requests served by the memory controller. The memory controller can then determine, based on the statistics, whether memory throughput would be increased by increasing or decreasing a respective number of credits allocated to at least one client of the one or more clinets. The memory controller can generate, based on a determination that the memory throughput would be increased, an output signal. The output signal can indicate to the credit controller that the respective number of credits allocated to the at least one client should be increased or decreased.

This example is just one illustration of modulating credit allocations in memory subsystems to improve the bandwidth at a memory interface. Other example configurations and methods are described throughout this document. This document now describes additional example methods, configurations, and components for the described modulation of credit allocations in memory subsystems.

<FIG> illustrates an example device diagram <NUM> of a user device <NUM> in which systems and techniques for modulating credit allocations in a memory subsystem can be implemented. The user device <NUM> may include additional components and interfaces omitted from <FIG> for the sake of clarity.

The user device <NUM> can be a variety of consumer electronic devices. As non-limiting examples, the user device <NUM> can be a mobile phone <NUM>-<NUM>, a tablet device <NUM>-<NUM>, a laptop computer <NUM>-<NUM>, a desktop computer <NUM>-<NUM>, a computerized watch <NUM>-<NUM>, a wearable computer <NUM>-<NUM>, a video game console <NUM>-<NUM>, or a voice-assistant system <NUM>-<NUM>.

The user device <NUM> can include one or more radio frequency (RF) transceivers <NUM> for communicating over wireless networks. The user device <NUM> can tune the RF transceivers <NUM> and supporting circuitry (e.g., antennas, front-end modules, amplifiers) to one or more frequency bands defined by various communication standards.

The user device <NUM> also includes the SoC <NUM>. The SoC <NUM> generally integrates several components of the user device <NUM> into a single chip, including a central processing unit, memory, and input and output ports. The SoC <NUM> can include a single core or multiple cores. In the depicted implementation, the SoC <NUM> includes one or more clients <NUM> and a memory subsystem <NUM>. The SoC <NUM> can include other components, including communication units (e.g., modems), input/output controllers, and system interfaces.

The clients <NUM> provide transaction requests to read or write data to random access memory (RAM) <NUM> of the memory subsystem <NUM>. The clients <NUM> can include, as non-limiting examples, a display system, a graphics processing unit, a central processing unit, a communication unit, input/output controllers, and system interfaces of the SoC <NUM>.

The memory subsystem <NUM> includes the RAM <NUM>, a memory controller <NUM>, and a credit controller <NUM>. The RAM <NUM> is a suitable storage device (e.g., static RAM (SRAM), dynamic RAM (DRAM), non-volatile RAM (NVRAM), synchronous dynamic RAM (SDRAM)) to store data accessible by the clients <NUM>. In other implementations, the RAM <NUM> can be located outside the SoC <NUM>.

The memory controller <NUM> manages the transaction requests of the clients <NUM> to the RAM <NUM>. The memory controller <NUM> can buffer and serve the transaction requests to the RAM <NUM> to increase the bandwidth of the memory subsystem <NUM>. In particular, the memory controller can schedule the transaction requests to improve the bandwidth of an interface between the RAM <NUM> and the memory controller <NUM>. The memory controller <NUM> can include hardware, firmware, software, or a combination thereof.

The credit controller <NUM> can allocate a portion of the buffer in the memory controller <NUM> to the clients <NUM>. The allocated portion of the buffer (referred to in this document as "credits") represents a bandwidth guarantee for the respective clients <NUM> at the RAM <NUM>. The credit controller <NUM> can include hardware, firmware, software, or a combination thereof.

A fabric (not illustrated in <FIG>) is operably connected to the clients <NUM> via respective virtual channels. The fabric can forward the transaction requests from the clients <NUM> to the memory controller <NUM>. In some implementations, the fabric is a multiplexer. The credit controller <NUM> can be implemented in the fabric, in any or all of the clients <NUM>, as a stand-alone component in the SoC <NUM>, or as a stand-alone component outside of the SoC <NUM>.

The memory controller <NUM> can also monitor statistics related to transactions served to the RAM <NUM>. Based on the statistics, the memory controller <NUM> can provide feedback to the credit controller <NUM> and/or the clients <NUM> to potentially modulate (e.g., decrease, increase, maintain) the credits allocated to one or more of the clients <NUM>. In this way, the described systems and techniques can dynamically modulate the credit allocations among the clients <NUM> and improve the QoS for the clients <NUM>.

The user device <NUM> also includes computer-readable storage media (CRM) <NUM>. The CRM <NUM> is a suitable storage device (e.g., random-access memory (RAM), static RAM (SRAM), dynamic RAM (DRAM), non-volatile RAM (NVRAM), read-only memory (ROM), Flash memory) to store device data of the user device <NUM>. The device data can include an operating system, one or more applications, user data, and multimedia data. In other implementations, the CRM <NUM> can store the operating system and a subset of the applications, user data, and multimedia data of the SoC <NUM>.

The operating system generally manages hardware and software resources of the user device <NUM> and provides common services. The operating system and the applications are generally executable by the SoC <NUM> to enable communications and user interaction with the user device <NUM>, which may require accessing data in the RAM <NUM> of the memory subsystem <NUM>.

<FIG> illustrates an example device diagram <NUM> of the SoC <NUM> in which systems and techniques for modulating credit allocations in the memory subsystem <NUM> of the SoC <NUM> can be implemented. The SoC <NUM> and the memory subsystem <NUM> can include additional components, which are not illustrated in <FIG>.

The SoC <NUM> includes multiple clients <NUM> and the memory subsystem <NUM>. In the depicted implementation, the clients <NUM> include a Client A <NUM>-<NUM>, a Client B <NUM>-<NUM>, and a Client C <NUM>-<NUM>. The SoC <NUM> can include fewer or additional clients <NUM>. In this example, the clients <NUM> are located outside the memory subsystem <NUM>. In other implementations, the clients <NUM> or a portion of the clients <NUM> can be located in the memory subsystem <NUM>.

As described above, the clients <NUM> can provide transaction requests to read or write data to the RAM <NUM>. One or more clients <NUM> can provide real-time traffic <NUM> and non-real-time traffic <NUM> to the memory subsystem <NUM>. In this example, the transaction requests of the Client A <NUM>-<NUM> include the real-time traffic <NUM> and the non-real-time traffic <NUM>.

The memory subsystem <NUM> includes the RAM <NUM>, the memory controller <NUM>, and the credit controller <NUM>. The RAM <NUM> includes at least one storage device. In the depicted implementation, the RAM <NUM> includes two storage devices: an SDRAM <NUM>-<NUM> and an SDRAM <NUM>-<NUM>. The SDRAM <NUM>-<NUM> and the SDRAM <NUM>-<NUM> can store data for the clients <NUM> or data accessible by the clients <NUM>. The SDRAM <NUM>-<NUM> and the SDRAM <NUM>-<NUM> are operably connected to the memory controller <NUM>.

The memory controller <NUM> can include a buffer <NUM>, a statistic monitoring module <NUM>, and a credit allocation feedback module <NUM>. The buffer <NUM> temporarily stores transaction requests of the clients <NUM>. The buffer <NUM> or another component of the memory controller <NUM> can also send the transaction requests to the SDRAM <NUM>-<NUM> and the SDRAM <NUM>-<NUM> to increase bandwidth at the memory interface.

The statistic monitoring module <NUM> can monitor the transaction requests served to the SDRAM <NUM>-<NUM> and the SDRAM <NUM>-<NUM>. For example, the statistic monitoring module <NUM> can determine statistics on a number of hits (e.g., page hits at the SDRAM <NUM>-<NUM> or the SDRAM <NUM>-<NUM>), conflicts (e.g., the SDRAM <NUM>-<NUM> or the SDRAM <NUM>-<NUM> closing a page before opening a requested page), allocated bandwidth for the clients <NUM> (e.g., the traffic class assigned to the client <NUM> serving the transaction request), an efficiency of the transaction requests, and a desired bandwidth. The statistic monitoring module <NUM> can be implemented in hardware, digital logic, or a combination thereof in the memory controller <NUM>.

The credit allocation feedback module <NUM> can provide feedback to the credit controller <NUM> or the clients <NUM>. The feedback can provide a suggestion to module the credits allocated to one or more clients <NUM>. For example, the credit allocation feedback module <NUM> can suggest that the credit controller <NUM> decreases the number of credits allocated to the Client B <NUM>-<NUM>. The memory controller <NUM> generally does not know the number of credits allocated to a particular traffic class or VCID. As a result, the credit allocation feedback module <NUM> can provide, based on the statistics determined by the statistic monitoring module <NUM>, the feedback while being agnostic to the credit allocations.

This document describes the operation of the memory subsystem <NUM>, specifically the operations of the memory controller <NUM> and the credit controller <NUM>, in greater detail with respect to <FIG>.

This section illustrates an example configuration of a hardware-based memory subsystem to module credit allocations, which may occur separately or together in whole or in part. This section describes the example configuration in relation to a drawing for ease of reading.

<FIG> illustrates an example diagram <NUM> of the memory subsystem <NUM> that can modulate credit allocations. The memory subsystem <NUM> can include additional components, which are not illustrated in <FIG>. The memory subsystem <NUM> provides a hardware implementation to send feedback, based on statistics gathered by the memory controller <NUM>, to upstream clients <NUM> or the credit controller <NUM>. The feedback can suggest modulation of the number of credits allocated to a particular traffic class or a particular client. The feedback loop is completed by the clients <NUM>, or the credit controller <NUM>, acting on the feedback and modulating the credit allocation. In this way, the memory controller <NUM> can increase the performance of the memory subsystem <NUM>.

Similar to <FIG>, the memory subsystem <NUM> includes one more RAMs <NUM> (e.g., the SDRAM <NUM>-<NUM>, the SDRAM <NUM>-<NUM>), the memory controller <NUM>, the credit controller <NUM>, the fabric <NUM>, and one or more clients <NUM> (e.g., the Client A <NUM>-<NUM>, the Client B <NUM>-<NUM>, the Client C <NUM>-<NUM>). The clients <NUM> are operably connected to the fabric <NUM> via internal busses. This document refers to the internal busses as virtual channels <NUM>. Each virtual channel <NUM> is assigned a unique identification, referred to as a virtual channel identification (VCID) in this document. The fabric <NUM>, the credit controller <NUM>, or the clients <NUM> can assign the VCIDs to a particular traffic class. As described above, a traffic class represents a particular treatment required for transaction requests to satisfy a particular QoS for the client <NUM>. For example, real-time traffic <NUM> (e.g., from a display client) may require different QoS treatment than non-real-time traffic <NUM>.

In the depicted implementation, the Client A <NUM>-<NUM> is operably connected to the fabric <NUM> via the virtual channel <NUM>-<NUM>. The Client B <NUM>-<NUM> is operably connected to the fabric <NUM> via the virtual channel <NUM>-<NUM>, and the Client C <NUM>-<NUM> is operably connected to the fabric <NUM> via the virtual channel <NUM>-<NUM>.

The credit controller <NUM> can subdivide the buffer <NUM> (not illustrated in <FIG>) of the memory controller <NUM> into credits and allocate the credits to different traffic classes to manage the bandwidth available to the different traffic classes. The credits allocated to the different traffic classes, and thus the different VCIDs, translates into a bandwidth guarantee for each VCID at the buffer <NUM> of the memory controller <NUM>. In other implementations, the credit controller <NUM> can allocate credits among the clients <NUM> based on a free pool. A free pool allows the credit controller <NUM> to dynamically allocate credits to the clients <NUM> based on feedback from the memory controller <NUM>, a traffic class assignment, a number of free credits available, or a combination thereof.

The fabric <NUM> is operably connected to the memory controller <NUM> via internal bus <NUM>. The fabric <NUM> sends memory transactions from the clients <NUM> to the memory controller <NUM> via the internal bus <NUM>. The memory controller <NUM> temporarily stores the transaction requests in the buffer <NUM>.

The memory controller <NUM> is operably connected to the SDRAMs <NUM> via internal busses <NUM>. In the depicted implementation, the memory controller <NUM> is operably connected to the SDRAM <NUM>-<NUM> and <NUM>-<NUM> via the internal busses <NUM>-<NUM> and <NUM>-<NUM>, respectively. The memory controller <NUM> serves the transaction requests to the SDRAMs <NUM> via the respective internal busses <NUM>. As described with respect to <FIG>, the statistic monitoring module <NUM> (not illustrated in <FIG>) monitors statistics of the memory transactions served to the SDRAMs <NUM>.

The memory controller <NUM> is also operably connected to the credit controller <NUM> via a sideband channel <NUM>. In some implementations, the memory controller <NUM> can also operably connect to one or more clients <NUM> via a sideband channel <NUM>. In other implementations, the memory controller <NUM> can operably connect to the clients <NUM> via the sideband channel <NUM> but does not operably connect to the credit controller <NUM> via the sideband channel <NUM>.

Based on the statistics generated by the statistic monitoring module <NUM>, the credit allocation feedback module <NUM> (not illustrated in <FIG>) can suggest modulating the credit allocation for one or more clients <NUM>. In this way, the memory controller <NUM> can ensure that the buffer <NUM> is maximally efficient. If a particular VCID can achieve the same throughput with a smaller allocation of credits, then the credit controller <NUM> or the clients <NUM> can allocate the spare credits to a different VCID, traffic class, or client <NUM> that would benefit from additional credits.

For example, the memory controller <NUM> can send, via the sideband channel <NUM> or the sideband channel <NUM>, at least one of the suggestion signals listed in Table <NUM>. For example, the suggestion signals r decrease credits and w_decrease_credits suggest decreasing the number of credits allocated to one or more of the clients <NUM>. In this way, the credit controller <NUM> or the clients <NUM> can use the suggestion signals to dynamically modulate a maximum occupancy of the buffer <NUM> for a particular VCID. In a free pool allocation system, the credit controller <NUM> or the clients <NUM> take into account the suggestion signals to modulate credit allocations among VCIDs. In modulating the credit allocations, the credit controller <NUM> also considers minimum credit allocations, QoS specifications, and urgency-based considerations in modulating the credits allocated to a particular VCID.

In operation, the statistic monitoring module <NUM> can monitor several metrics to assist the credit allocation feedback module <NUM>. In particular, the statistic monitoring module <NUM> can determine a disparity metric and an occupancy metric for transaction requests served by the memory controller <NUM> for each clock cycle of the memory subsystem <NUM>. The transaction requests include the VCID or other data identifying the clients <NUM> that served the transaction requests. In this way, the statistic monitoring module <NUM> can determine the disparity metric and the occupancy metric for each of the one or more clients <NUM>. In other implementations, the statistic monitoring module <NUM> can determine and monitor additional metrics. The disparity metric, which represents an efficiency seen by a particular VCID at an interface to the SDRAMs <NUM>, is described in greater detail with respect to <FIG>. The occupancy metric, which infers the number of credits allocated to a certain VCID, is described in greater detail with respect to <FIG>.

As depicted in <FIG>, the memory subsystem <NUM> can implement the management and modulation of credit allocations in hardware. In other implementations, the credit management and re-balancing can be implemented at the kernel level or a driver level.

<FIG> illustrates an example chart <NUM> illustrating the disparity metric monitored by the statistic monitoring module <NUM> of the memory controller <NUM>. In this implementation, the statistic monitoring module <NUM> monitors the disparity metric for a particular VCID of the memory subsystem <NUM>.

The disparity metric represents an efficiency for a VCID at the interface to one or both of the SDRAMs <NUM>. As an example, a VCID with transaction requests that result in a relatively large proportion of hits will have greater efficiency than a VCID with more random transaction requests leading to a higher proportion of conflicts. In this document, a hit refers to a page hit at the SDRAMs <NUM>. A page hit can occur when a required page (e.g., row) of the SDRAM <NUM> for the transaction request is already open. A miss refers to a transaction request for a page that is closed at the SDRAM <NUM>. A conflict refers to an SDRAM <NUM> having a different page open than required for a transaction request, resulting in the open page being closed.

A transaction request at the SDRAMs <NUM> generally results in a particular sequence of commands. For example, a read (RD) or write (WR) command first involves sending an activation (ACT) command. The ACT command loads an entire page of the SDRAM <NUM> into the row buffer. A subsequent RD command addressing a column of that page returns the data. Similarly, a subsequent WR command addressing that page writes the data in the transaction request into the addressed column. After access of that page is complete, the SDRAM <NUM> closes the page by issuing a precharge (PRE) command. In general, a transaction request to the SDRAMs <NUM> can involve opening, accessing, and closing pages. If the transaction request addresses a page or row buffer that is not currently open, the transaction request incurs an additional penalty of closing the already open page.

The memory controller <NUM> tries to maximize the bandwidth at the interface to the SDRAMs <NUM> while also simultaneously satisfying the QoS parameters for the transaction requests. These two requirements can cause the memory controller <NUM> to serve the transaction requests out of order. The client <NUM>, traffic class, or VCID that sends transaction requests that result in a series of hits will have lower latency than one that experiences many conflicts. As a result, the credit allocation feedback module <NUM> can use statistics on hits, conflicts, allocated bandwidth, and desired bandwidth to define performance metrics. The credit allocation feedback module <NUM> can then use the performance metrics to suggest modulating the credits allocated to different clients <NUM>, traffic classes, or VCIDs to improve system performance.

The statistic monitoring module <NUM> can define the disparity metric for a particular VCID as the sum of hits minus the sum of conflicts: <MAT>.

The disparity metric can be an <NUM>-bit unsigned value with some initial offset value (e.g., <NUM>). Each time a transaction request receives a grant in a request scheduler of the memory controller <NUM>, the statistic monitoring module <NUM> updates the number of hits and conflicts belonging to that VCID in the next clock cycle and updates the disparity value. In this way, the statistic monitoring module <NUM> can determine an efficiency associated with a particular VCID using counters while avoiding a need to use division or multiplication to generate the disparity metric. The statistic monitoring module <NUM> generally does not consider misses because each conflict eventually leads to a miss. In another implementation, the statistic monitoring module <NUM> can define the disparity metric in terms of misses instead of conflicts. The statistic monitoring module <NUM> can also saturate the disparity metric at a maximum value <NUM> (e.g., <NUM>) and a minimum value <NUM> (e.g., <NUM>) to avoid the disparity value rolling over (e.g., to avoid an integer overflow or underflow, which can cause an error in the value of the disparity metric).

The chart <NUM> illustrates an example disparity value <NUM> generated by the statistic monitoring module <NUM> for Client A <NUM>-<NUM> for time windows <NUM>, <NUM>, <NUM>, <NUM>, and <NUM>. At the beginning of the time window <NUM>, the disparity value <NUM> for the Client A <NUM>-<NUM> starts at an initial offset value of <NUM>. During the time window <NUM>, the memory controller <NUM> does not serve any transaction requests for the Client A <NUM>-<NUM>, and the disparity value <NUM> remains at a value of <NUM>.

During the time window <NUM>, the memory controller <NUM> serves transaction requests that result in page hits, and the disparity value <NUM> has a positive slope. The disparity value <NUM> saturates at the maximum value <NUM> after a certain number of page hits.

During the time window <NUM>, the statistic monitoring module <NUM> includes a decay factor that slowly brings the disparity value <NUM> back to the initial offset value when there are no transaction requests for that VCID. In this way, the statistic monitoring module <NUM> can avoid the hit rate of an old transaction-request thread from affecting the feedback signal for a current transaction-request thread.

During the time window <NUM>, the disparity value <NUM> oscillates around the initial offset value by going below as well as above the initial offset value. The oscillation of the disparity value <NUM> can result from a series of transaction requests resulting in a conflict and then in a page hit. Consider after a transaction request is sent and results in a conflict, the same transaction request becomes a page hit and results in a positive score that offsets the negative score. During the time window <NUM>, the memory controller <NUM> does not serve any transaction requests for the VCID, and the disparity value <NUM> remains at the initial offset value.

In response to a low disparity value <NUM>, the credit allocation feedback module <NUM> does not necessarily suggest decreasing the credit allocation to a particular VCID. The VCID may have been assigned a small number of credits and expected a small bandwidth at the memory controller <NUM>. As a result, relatively low efficiency or bandwidth at the interface to the SDRAM <NUM> is expected and does not trigger a suggestion to decrease the credits allocated to that VCID. To this end, the credit allocation feedback module <NUM> uses the occupancy metric, which is described with respect to <FIG>, along with the disparity metric, to generate the credit-allocation feedback.

<FIG> illustrates an example chart <NUM> illustrating credits <NUM> allocated to a client <NUM> based on an occupancy metric and the disparity metric and an example chart <NUM> illustrating a feedback signal <NUM> from the memory controller <NUM>. In this implementation, the statistic monitoring module <NUM> monitors the occupancy metric for a particular VCID of the memory subsystem <NUM>.

As described above, the occupancy metric represents an inference by the statistics monitoring module <NUM> of the number of credits <NUM> allocated to a VCID. Because the allocation of credits is managed by the credit controller <NUM> and/or the one or more clients <NUM>, the memory controller <NUM> does not have direct knowledge of the number of credits <NUM> allocated to a particular VCID. The statistics monitoring module <NUM> can use the number of entries in the buffer <NUM> used by a particular VCID to infer the credits <NUM> allocated to the VCID. In other words, the occupancy metric for a particular virtual channel is the number of buffer entries used by that virtual channel.

The credit allocation feedback module <NUM> can determine the feedback signal <NUM> for each VCID based on the occupancy metric and the disparity metric collected by the statistics monitoring module <NUM> as follows: <MAT> where K is a constant scaling factor to get both the occupancy metric and the disparity metric within the same range. If the preceding is true, then the credit allocation feedback module <NUM> suggests that the credits <NUM> allocated to the VCID be decreased. The credit controller <NUM> and/or the client <NUM> can use the feedback signal <NUM> to perform a successive approximation of a maximum number of credits <NUM> to allocate to the VCID.

As an example, the chart <NUM> illustrates the credits <NUM> allocated to a particular client <NUM> (e.g., Client B <NUM>-<NUM>). The chart <NUM> illustrates the feedback signal <NUM> provided by the credit allocation feedback module <NUM> to the credit controller <NUM> about suggestions for modulating the credits <NUM> allocated to the Client B <NUM>-<NUM>. During the time window <NUM>, the credit controller <NUM> allocated a maximum number of credits <NUM> (e.g., <NUM> credits) to the Client B <NUM>-<NUM>. The credit allocation feedback module <NUM> sends a hold suggestion to the credit controller <NUM> for the Client B <NUM>-<NUM> during the time window <NUM>.

In the time window <NUM>, the statistics monitoring module <NUM> determines that the occupancy metric for the VCID associated with the Client B <NUM>-<NUM> is more than a threshold value larger than the disparity metric for this VCID. In response, the credit allocation feedback module <NUM> sends a r decrease credits signal to the memory controller <NUM>. When the credit controller <NUM> sees the decrease suggestion for a T number of clock cycles <NUM>, the credit controller <NUM> can modulate the number of credits <NUM> allocated to the Client B <NUM>-<NUM> downward by a number of credits towards a minimum number of credits <NUM>. The decrease in the number of credits <NUM> allocated to the Client B <NUM>-<NUM> can continue until the feedback signal <NUM> no longer includes a r decrease credits suggestion. The number of credits <NUM> allocated to the Client B <NUM>-<NUM> generally will not drop below the minimum number of credits <NUM>.

During the time windows <NUM> and <NUM>, the feedback signal <NUM> no longer includes a r decrease credits suggestion for the T number of clock cycles <NUM>, and the number of credits <NUM> allocated to the Client B <NUM>-<NUM> is increased back towards the maximum number of credits <NUM>.

The credit controller <NUM> can allocate the credits taken away from the Client B <NUM>-<NUM> to a different client or to a free pool of credits. In this way, the memory subsystem <NUM> can efficiently utilize the buffer <NUM> of the memory controller <NUM> by modulating the credits allocated to different VCIDs. In addition, the credit controller generally tries to allocate the maximum number of credits <NUM> to the clients <NUM> in an absence of a decrease-credits suggestion from the memory controller <NUM>.

<FIG> is a flowchart illustrating example operations <NUM> of modulating credit allocations in memory subsystems. The operations <NUM> are described in the context of the memory subsystem <NUM> of <FIG> and <FIG>. The operations <NUM> may be performed in a different order or with additional or fewer operations.

At <NUM>, a respective number of credits are allocated by a credit controller to one or more clients. For example, the credit controller <NUM> can allocate a respective number of credits to the one or more clients <NUM>.

At <NUM>, transaction requests from the one or more clients to access data in one or more RAMs are stored by a memory controller. The memory controller is operably connected to the one or more RAMs and the credit controller. For example, the memory controller <NUM> is operably connected to the one or more RAMs <NUM> (e.g., the SDRAM <NUM>-<NUM>, the SDRAM <NUM>-<NUM>) and the credit controller <NUM>. The memory controller <NUM> includes the buffer <NUM> to store transaction requests, from the one or more clients <NUM>, to access data in the one or more RAMs <NUM>.

At <NUM>, statistics for transaction requests served to the one or more RAMs are monitored by the memory controller for each client of the one or more clients. For example, the memory controller <NUM> can monitor, for each client <NUM> of the one or more clients <NUM>, statistics for transaction requests served to the one or more RAMs <NUM>.

At <NUM>, the memory controller determines, based on the statistics, whether memory throughput would be increased by increasing or decreasing the respective number of credits allocated to at least one client of the one or more clients. For example, the memory controller <NUM> can determine, based on the statistics, whether memory throughput would be increased by increasing or decreasing the respective number of credits allocated to at least one client <NUM> of the one or more clients <NUM>.

At <NUM>, an output signal is generated by the memory controller to indicate that the respective number of credits should be increased or decreased. The output signal is based on a determination that the memory throughput would be increased. The output signal is sent by the memory controller to the credit controller. For example, the memory controller <NUM> can generate an output signal to indicate that the respective number of credits should be increased or decreased. The output signal is based on a determination that the memory throughput to the RAMs <NUM> would be increased. The memory controller <NUM> can send, via the side channel <NUM>, the output signal to the credit controller <NUM>. The memory controller <NUM> can also send the output signal directly to the at least one of the one or more clients <NUM>.

At <NUM>, the respective number of credit allocated to the at least one client of the one or more clients is modulated by the credit controller and based on the output signal. For example, the credit controller <NUM> can modulate, based on the output signal, the respective number of credits allocated to the at least one client <NUM> of the one or more clients <NUM>.

Claim 1:
A memory subsystem (<NUM>) of a system on chip, SoC (<NUM>), comprising:
a memory controller (<NUM>) operably connected to a credit controller (<NUM>), the memory controller (<NUM>) comprising a buffer (<NUM>) configured to store transaction requests, from one or more clients (<NUM>-<NUM>, <NUM>-<NUM>, <NUM>-<NUM>), to access data in one or more random access memories, RAMs (<NUM>), the memory controller (<NUM>) configured to:
monitor (<NUM>), for each client (<NUM>) of the one or more clients (<NUM>-<NUM>, <NUM>-<NUM>, <NUM>-<NUM>), statistics of the transaction requests served by the memory controller (<NUM>), wherein the statistics comprise at least two of a number of page hits in one or more RAMs (<NUM>), a number of conflicts that require closing a page of the one or more RAMs (<NUM>) before another page can be opened, or an occupancy metric that represents an inference of a respective number of credits allocated to the client (<NUM>) based on a number of entries in the buffer (<NUM>) used by each client of the one or more clients (<NUM>-<NUM>, <NUM>-<NUM>, <NUM>-<NUM>);
determine (<NUM>), based on the statistics, whether memory throughput would be increased by increasing or decreasing a respective number of credits allocated to at least one client (<NUM>) of the one or more clients (<NUM>-<NUM>, <NUM>-<NUM>, <NUM>-<NUM>); and
generate (<NUM>), based on a determination that the memory throughput would be increased, an output signal, the output signal configured to indicate to the credit controller (<NUM>) that the respective number of credits allocated to the at least one client (<NUM>) of the one or more clients (<NUM>-<NUM>, <NUM>-<NUM>, <NUM>-<NUM>) should be increased or decreased.