Providing flexible management of heterogeneous memory systems using spatial quality of service (QoS) tagging in processor-based systems

Providing flexible management of heterogeneous memory systems using spatial Quality of Service (QoS) tagging in processor-based systems is disclosed. In one aspect, a heterogeneous memory system of a processor-based system includes a first memory and a second memory. The heterogeneous memory system is divided into a plurality of memory regions, each associated with a QoS identifier (QoSID), which may be set and updated by software. A memory controller of the heterogeneous memory system provides a QoS policy table, which operates to associate each QoSID with a QoS policy state, and which also may be software-configurable. Upon receiving a memory access request including a memory address of a memory region, the memory controller identifies a software-configurable QoSID associated with the memory address, and associates the QoSID with a QoS policy state using the QoS policy table. The memory controller then applies the QoS policy state to perform the memory access operation.

BACKGROUND

I. Field of the Disclosure

The technology of the disclosure relates generally to memory management, and, in particular, to managing heterogeneous memory systems.

A heterogeneous memory system is a memory system of a processor-based system that incorporates two or more different types of memory having comparatively different performance characteristics (e.g., capacity, bandwidth, access latency, power consumption, and/or the like). As non-limiting examples, a heterogeneous memory system may include a high-bandwidth memory (HBM) that provides atypically wide communication lanes, along with a dynamic random access memory (DRAM) that provides conventionally sized communication lanes. Other aspects of a heterogeneous memory system may include DRAM and phase-change memory, DRAM and a Level 3 (L3) cache on a processor die, and/or other combinations of different memory types known in the art.

Conventional heterogeneous memory systems may present limited options with respect to memory management. In some aspects, memory of one type (e.g., HBM) may be configured to act as a cache for an entire memory address space of the heterogeneous memory system. Such aspects are relatively simple and easy to deploy, and may be transparently managed by hardware. However, because all decisions regarding the placement of data or allocation of memory are handled by hardware, no software input or influence on data placement or allocation memory is utilized.

Alternatively, the heterogeneous memory system may employ disjoint address regions, and may allow dynamic memory management and reallocation operations to be performed by software instead of hardware. This approach is often preferable to exclusive hardware control, as software may have access to data (e.g., workload, program semantics, and/or relative priorities) that is relevant to memory allocation, but that cannot be accessed or transparently inferred by hardware. However, such dynamic memory management may require extremely complex data migration decisions, and software capable of managing such decisions, along with other necessary processor- and time-expensive operations, which may be difficult to develop and maintain. Thus, a memory management mechanism that provides hardware support and software control for flexible management of heterogeneous memory systems is desirable.

SUMMARY OF THE DISCLOSURE

Aspects disclosed in the detailed description include providing flexible management of heterogeneous memory systems using spatial Quality of Service (QoS) tagging in processor-based systems. In one aspect, a heterogeneous memory system of a processor-based system includes a first memory (e.g., a high-bandwidth memory (HBM), as a non-limiting example) and a second memory (e.g., a dynamic random access memory (DRAM), as a non-limiting example). The heterogeneous memory system is divided into a plurality of memory regions, such as memory pages each having a size of four (4) kilobytes, as a non-limiting example. The heterogeneous memory system is configured to employ what is referred to herein as “spatial Quality of Service (QoS) tagging,” in which each of the plurality of memory regions is associated with a QoS identifier (QoSID), which may be set and updated by software. A memory controller of the heterogeneous memory system provides a QoS policy table, which operates to map each QoSID to a QoS policy state (such as a quota, a priority, a proportional-share weight, and/or a probability, as non-limiting examples), and which also may be software-configurable. Upon receiving a memory access request including a memory address of a memory region, the memory controller identifies a software-configurable QoSID associated with the memory address, and associates the QoSID with a QoS policy state using the QoS policy table. The memory controller then applies the QoS policy state to perform a memory access operation. For instance, in aspects in which the first memory is configured as a transparent cache for the second memory, the memory controller may apply the QoS policy state in performing a cache eviction from the first memory or a cache fill of the first memory. In this manner, a software process may indirectly influence memory allocation and data placement by specifying a QoSID for each of the plurality of memory regions, and by associating the QoSID with a QoS policy state that is enforced by the memory controller.

In another aspect, a memory controller for providing flexible management of a heterogeneous memory system of a processor-based system is provided. The memory controller communicatively is coupled to a first memory and a second memory having different performance characteristics. The memory controller includes a software-configurable QoS policy table providing a plurality of QoS policy entries. Each QoS policy entry includes a QoS policy state and is associated with a QoSID of one or more QoSIDs each associated with one of a plurality of memory regions. The plurality of memory regions reside within the first memory, the second memory, or both. The memory controller is configured to receive a memory access request comprising a memory address corresponding to a memory region of the plurality of memory regions. The memory controller is further configured to identify a software-configurable QoSID associated with the memory address. The memory controller is also configured to associate, using the QoS policy table, the QoSID associated with the memory address with a QoS policy state. The memory controller is additionally configured to apply the QoS policy state to perform a memory access operation on one of the first memory and the second memory.

In another aspect, a memory controller for providing flexible management of a heterogeneous memory system of a processor-based system is provided. The memory controller comprises a means for receiving a memory access request comprising a memory address corresponding to a memory region of a plurality of memory regions of one or more of a first memory and a second memory having different performance characteristics. The memory controller further comprises a means for identifying a software-configurable QoSID associated with the memory address. The memory controller also comprises a means for associating, using a software-configurable QoS policy table, the QoSID associated with the memory address with a QoS policy state. The memory controller further comprises a means for applying the QoS policy state to perform a memory access operation on one of the first memory and the second memory.

In another aspect, a method for providing flexible management of a heterogeneous memory system of a processor-based system is provided. The method comprises receiving, by a memory controller, a memory access request comprising a memory address corresponding to a memory region of a plurality of memory regions of one or more of a first memory and a second memory having different performance characteristics. The method further comprises identifying a software-configurable QoSID of a plurality of QoSIDs, wherein the QoSID is associated with the memory address. The method also comprises associating, using a software-configurable QoS policy table of the memory controller, the QoSID associated with the memory address with a QoS policy state. The method additionally comprises applying the QoS policy state to perform a memory access operation on one of the first memory and the second memory.

DETAILED DESCRIPTION

Aspects disclosed in the detailed description include providing flexible management of heterogeneous memory systems using spatial Quality of Service (QoS) tagging in processor-based systems. Before describing a memory controller configured to flexibly manage memory using spatial QoS tagging, exemplary elements and operation of a heterogeneous memory system of a processor-based system are first described.

In this regard,FIG. 1illustrates an exemplary processor-based system100that provides a heterogeneous memory system102. In some aspects, the processor-based system100may encompass any one of known digital logic elements, semiconductor circuits, processing cores, and/or memory structures, among other elements, or combinations thereof. Aspects described herein are not restricted to any particular arrangement of elements, and the disclosed techniques may be easily extended to various structures and layouts on semiconductor dies or packages. It is to be understood that the processor-based system100may include additional elements not illustrated herein for the sake of clarity.

As seen inFIG. 1, the heterogeneous memory system102includes a first memory104and a second memory106. According to some aspects, the first memory104may include a high-bandwidth memory (HBM), and the second memory106may include a dynamic random access memory (DRAM), as non-limiting examples. Access to the first memory104and the second memory106is managed by a memory controller108, which may receive memory access requests from a software process110comprising instructions being executed by the processor-based system100. It is to be understood that the software process110may execute in parallel with one or more other software processes, and may comprise an application, a hypervisor, a virtual machine, an operating system, and/or a container, as non-limiting examples.

Some aspects may provide that the memory controller108may be configured to operate the first memory104as a transparent cache of the second memory106. In such aspects, the second memory106may be used to store a master table112. The master table112contains a plurality of master table entries114(0)-114(M), each of which associates a range of one or more memory addresses116(0)-116(M) of a memory region118(0)-118(X) of the second memory106with a cache way identifier (“WAY ID”)120(0)-120(M). Thus, to determine in which cache way of the first memory104a given memory region118(0)-118(X) of the second memory106may be stored, the memory controller108may perform a lookup operation on the master table112. While the master table112is illustrated as part of the second memory106inFIG. 1, it is to be understood that the master table112may be stored as part of other memory elements that are accessible by the memory controller108including the first memory104for example. In some aspects, the memory regions118(0)-118(X) each may be a memory page (e.g., a four (4) kilobyte (kB) memory page), while some aspects may provide that each of the memory regions118(0)-118(X) is a larger or smaller subdivision of the second memory106. Some aspects may provide that the memory regions118(0)-118(X) may be of non-uniform sizes.

FIG. 2illustrates a processor-based system200that provides a memory controller202for performing flexible management of the heterogeneous memory system102ofFIG. 1using spatial QoS tagging. In the example ofFIG. 2, the memory controller202includes a QoS policy table204that contains a plurality of QoS policy entries206(0)-206(Y). Each of the plurality of QoS policy entries206(0)-206(Y) associates a QoSID208(0)-208(Y) with a corresponding QoS policy state210(0)-210(Y). In some aspects, the QoS policy table204may not explicitly store a value for each QoSID208(0)-208(Y) as shown inFIG. 2, but rather may implicitly define each of the QoSIDs208(0)-208(Y) as the row index in the QoS policy table204of the corresponding QoS policy state210(0)-210(Y). The QoS policy states210(0)-210(Y) may each represent any applicable QoS attribute (such as a quota, a priority, a proportional-share weight, and/or a probability, as non-limiting examples) that may be enforced by the memory controller202. The QoS policy table204may be updated by the software process110. Accordingly, the software process110may specify a QoS attribute for each QoS policy state210(0)-210(Y) associated with each QoSID208(0)-208(Y).

In some aspects, the master table112may be expanded to include QoSIDs212(0)-212(M) as part of the master table entries114(0)-114(M). In this manner, the master table entries114(0)-114(M) may be used to identify the QoSIDs212(0)-212(M) that are associated with a range of one or more of the memory addresses116(0)-116(M) and/or with an entire memory region118(0)-118(X) of the second memory106. In some aspects, the master table112may be organized as two (2) separate structures having different addressing granularities (e.g., one may provide memory address-to-way mapping to provide cache line granularity, while the other may provide memory address-to-QoSID mapping to provide coarser page granularity). Alternatively, some aspects may provide a single master table112in which the contents of the two (2) separate structures are combined or interleaved. According to some aspects, the master table112may not store the memory address116expressly, but rather may use high order bits of the memory address116as an index into the master table112.

The QoSIDs212(0)-212(M) of the master table112may be updated by the software process110, thus enabling the software process110to selectively set the values of the QoSIDs212(0)-212(M) assigned to the memory regions118(0)-118(X). In some aspects, one of the QoSIDs208(0)-208(Y) may be designated as specifying a “default” QoSID208(0)-208(Y) that is used to determine a QoS policy state210(0)-210(Y) to be applied to memory regions118(0)-118(X) that are not explicitly assigned to a QoSID208(0)-208(Y). Some aspects may provide that the QoSIDs208(0)-208(Y) may be associated with memory regions118(0)-118(X) of different sizes representing multiple granularities, such that the QoSID208(0)-208(Y) associated with the most specific granularity is applied. For example, a first QoSID208(0)-208(Y) may be associated with a larger two (2) megabyte memory region118(0)-118(X), while a second QoSID208(0)-208(Y) may be associated with a smaller four (4) kilobyte memory region118(0)-118(X) within the larger memory region118(0)-118(X). In this case, the first QoSID208(0)-208(Y) would be applied to the entire two (2) megabyte memory region118(0)-118(X) except for the smaller four (4) kilobyte memory region118(0)-118(X), which would fall under the second QoSID208(0)-208(Y).

Some aspects may provide that the memory controller202also includes a master table cache214for caching recently read master table entries114(0)-114(M) from the master table112. The master table cache214provides a plurality of master table cache entries216(0)-216(Z) storing cached memory addresses116′(0)-116′(Z), cached way identifiers (“WAY IDs”)120′(0)-120′(Z), and cached QoSIDs212′(0)-212′(Z). When attempting to identify a QoSID associated with a memory address range and/or a memory region of the memory regions118(0)-118(X), the memory controller202may access the master table cache214before performing a lookup in the master table112in the second memory106. If the master table cache214produces a cache hit, the memory controller202may read a cached QoSID212′(0)-212′(Z) from the master table cache214, which reduces memory access latency and conserves memory bandwidth by avoiding an unnecessary read to the master table112. According to some aspects in which the first memory104is used as a transparent cache for the second memory106, accesses to the master table112may also be avoided by storing QoSIDs218(0)-218(Q) within the first memory104(e.g., along with the cache tags (not shown) for each cache line (not shown)). In such aspects, a subset of the cache line granularity data from the master table112may be incorporated into the cache provided by the first memory104.

In exemplary operation, the memory controller202may receive a memory access request220, including a memory address222, from the software process110. The memory controller202identifies a QoSID (e.g., the QoSID212(0)) corresponding to the memory address222(e.g., by performing a lookup into the master table112and/or into the master table cache214using the memory address222or high order bits of the memory address222). Once the QoSID212(0) is identified, the memory controller202uses the QoS policy table204to associate the QoSID212(0) to a QoS policy state (e.g., the QoS policy state210(0)). The memory controller202then applies the QoS policy state210(0) in carrying out the memory access request220. For instance, in aspects in which the first memory104is configured to operate as a transparent cache for the second memory106, the memory access request220may indicate a memory read operation or a memory write operation on the memory address222in the second memory106. Upon receiving the memory access request220, the memory controller202identifies the QoSID212(0) corresponding to the memory address222, and determines the QoS policy state210(0) that applies to that region of the second memory106(e.g., how much of the cache in the first memory104may be occupied, etc.) The memory controller202may perform cache eviction operations or cache fill operations on the first memory104in accordance with the QoS policy state210(0).

In this manner, the memory controller202provides the benefit of transparent addressing and transparent caching, since the software process110does not have to make decisions regarding memory allocation or relocation. However, the software process110is still able to influence data placement and allocation by updating the QoSIDs212(0)-212(M) and/or the QoS policy table204. This aspect provides flexible support for both unmanaged caches and software-managed partitions. For example, when using the first memory104as a cache, the software process110may implement a quota-based memory management scheme, whereby a given one of the memory regions118(0)-118(X) of the second memory106(e.g., the memory region118(0)) is locked in the first memory104by assigning the QoS policy state210(0) to specify a quota larger than the size of the memory regions118(0)-118(X). Consequently, data for the memory region118(0) will never be evicted from the first memory104because the quota will never be met. Similarly, the software process110may implement a cache bypass by assigning the QoS policy state210(0) to specify a quota of zero (0). The software process110could also implement a partial cache scheme by assigning the QoS policy state210(0) to specify a quota less than the size of the first memory104. Note that a quota assigned by the QoS policy state210(0) would apply to the aggregate size of all memory regions118(0)-118(X) of the second memory106that are associated with the corresponding QoSID212(0)-212(M). For example, assume that the QoSID212(0) is associated with four (4) memory regions118(0)-118(3) of the second memory106, with each of the memory regions118(0)-118(3) being four (4) kilobytes in size. As long as the quota specified by the QoS policy state210(0) associated with the QoSID212(0) is at least 16 kilobytes, the memory regions118(0)-118(3) will remain locked in the first memory104.

When applying the QoS policy states210(0)-210(Y) for aspects in which the first memory104is configured to operate as a transparent cache for the second memory106, the memory controller202may employ different strategies for allocation of the first memory104. For example, when determining cache lines to evict from the first memory104, the memory controller202may randomly sample a number of cache lines, and select a cache line associated with a QoSID that is most over quota as the eviction victim. Alternately, if the cache line to be written to the first memory104is the most over quota, the memory controller202may opt to bypass the first memory104entirely, or evict another cache line having the same QoSID. When performing a cache fill, the memory controller202may base the cache fill operation on a quota or a probability indicated by the QoS policy state associated with the QoSID of the new data. Finally, in some aspects, the memory controller202may improve aggregate bandwidth for the first memory104and the second memory106by intentionally leaving some memory capacity of the first memory104unused. This may be accomplished by designating a quota for invalid lines in the first memory104, which allows a specified number of lines to remain idle in the first memory104.

To illustrate exemplary operations of the memory controller202ofFIG. 2for flexibly managing the heterogeneous memory system102,FIGS. 3A-3Care provided. For the sake of clarity, elements ofFIG. 2are referenced in describingFIGS. 3A-3C. InFIG. 3A, operations may begin in different ways depending on the particular implementation of the heterogeneous memory system102. In some aspects, the memory controller202may operate the first memory104as a transparent cache for the second memory106, wherein the plurality of memory regions118(0)-118(X) comprises a plurality of memory regions118(0)-118(X) of the second memory106(block300). The memory controller202receives a memory access request220comprising a memory address222corresponding to a memory region118(0) of the plurality of memory regions118(0)-118(X) of one or more of the first memory104and the second memory106having different performance characteristics (block302). In this regard, the memory controller202may be referred to herein as “a means for receiving a memory access request comprising a memory address corresponding to a memory region of a plurality of memory regions of one or more of a first memory and a second memory having different performance characteristics.” Processing then resumes at block304inFIG. 3B.

Referring now toFIG. 3B, the memory controller202next identifies a software-configurable QoSID208(0) of the plurality of QoSIDS208(0)-208(Y), wherein the QoSID208(0) is associated with the memory address222(block304). Accordingly, the memory controller202may be referred to herein as “a means for identifying a software-configurable QoSID associated with the memory address.” Some aspects, such as those in which the first memory104is configured to operate as a transparent cache for the second memory106, may provide that operations of block304for identifying the QoSID208(0) associated with the memory address222may include determining whether a master table cache entry216(0) of a plurality of master table cache entries216(0)-216(Z) of a master table cache214of the memory controller202corresponds to the memory address222(block308). If so, the memory controller202may access the master table cache entry216(0) (block310). However, if no master table cache entry216(0) of the plurality of master table cache entries216(0)-216(Z) corresponds to the memory address222, the memory controller202may access the master table112in the second memory106, the master table112including a plurality of master table entries114(0)-114(M) each corresponding to the memory address222associated with a QoSID212(0) of the plurality of QoSIDs212(0)-212(M) (block312). After identifying the QoSID212(0) associated with the memory address222, processing resumes at block314ofFIG. 3C.

Turning now toFIG. 3C, the memory controller202associates, using the software-configurable QoS policy table204of the memory controller202, the QoSID208(0) associated with the memory address222with a QoS policy state210(0) (block314). The memory controller202thus may be referred to herein as “a means for associating, using a software-configurable QoS policy table, the QoSID associated with the memory address with a QoS policy state.” The memory controller202then applies the QoS policy state210(0) to perform a memory access operation on one of the first memory104and the second memory106(block316). In this regard, the memory controller202may be referred to herein as “a means for applying the QoS policy state to perform a memory access operation on one of the first memory and the second memory.” In some aspects, such as those in which the first memory104is configured to operate as a transparent cache for the second memory106, operations of block316for applying the QoS policy state210(0) to perform a memory access operation may comprise performing one of a cache eviction from the first memory104and a cache fill of the first memory104based on the QoS policy state210(0) (block318). Some aspects may also provide that the memory controller202may receive, from a software process110executing on the processor-based system200, an update to a QoS policy entry206(0) of the plurality of QoS policy entries206(0)-206(Y) of the QoS policy table204(block320).

Providing flexible management of heterogeneous memory systems using spatial Quality of Service (QoS) tagging in processor-based systems according to aspects disclosed herein may be provided in or integrated into any processor-based device. Examples, without limitation, include a set top box, an entertainment unit, a navigation device, a communications device, a fixed location data unit, a mobile location data unit, a global positioning system (GPS) device, a mobile phone, a cellular phone, a smart phone, a session initiation protocol (SIP) phone, a tablet, a phablet, a server, a computer, a portable computer, a mobile computing device, a wearable computing device (e.g., a smart watch, a health or fitness tracker, eyewear, etc.), a desktop computer, a personal digital assistant (PDA), a monitor, a computer monitor, a television, a tuner, a radio, a satellite radio, a music player, a digital music player, a portable music player, a digital video player, a video player, a digital video disc (DVD) player, a portable digital video player, an automobile, a vehicle component, avionics systems, a drone, and a multicopter.

In this regard,FIG. 4illustrates an example of a processor-based system400that may correspond to the processor-based system200ofFIG. 2, and that can employ the memory controller202illustrated inFIG. 2. In this example, the processor-based system400includes one or more central processing units (CPUs)402, each including one or more processors404. The CPU(s)402may be a master device. The CPU(s)402may have cache memory406coupled to the processor(s)404for rapid access to temporarily stored data. The CPU(s)402is coupled to a system bus408and can intercouple master and slave devices included in the processor-based system400. As is well known, the CPU(s)402communicates with these other devices by exchanging address, control, and data information over the system bus408. For example, the CPU(s)402can communicate bus transaction requests to the memory controller202as an example of a slave device.

Other master and slave devices can be connected to the system bus408. As illustrated inFIG. 4, these devices can include a memory system410, one or more input devices412, one or more output devices414, one or more network interface devices416, and one or more display controllers418, as examples. The input device(s)412can include any type of input device, including but not limited to input keys, switches, voice processors, etc. The output device(s)414can include any type of output device, including but not limited to audio, video, other visual indicators, etc. The network interface device(s)416can be any devices configured to allow exchange of data to and from a network420. The network420can be any type of network, including but not limited to a wired or wireless network, a private or public network, a local area network (LAN), a wide local area network (WLAN), and the Internet. The network interface device(s)416can be configured to support any type of communications protocol desired. The memory system410can include one or more memory units422(0)-422(Z), a subset of which may comprise heterogeneous memory.

The CPU(s)402may also be configured to access the display controller(s)418over the system bus408to control information sent to one or more displays424. The display controller(s)418sends information to the display(s)424to be displayed via one or more video processors426, which process the information to be displayed into a format suitable for the display(s)424. The display(s)424can include any type of display, including but not limited to a cathode ray tube (CRT), a liquid crystal display (LCD), a plasma display, etc.