Dynamic access granularity in a cache media

A method comprising receiving a memory access request comprising an address of data to be accessed and determining an access granularity of the data to be accessed based on the address of the data to be accessed. The method further includes, in response to determining that the data to be accessed has a first access granularity, generating first cache line metadata associated with the first access granularity and in response to determining that the data to be accessed has a second access granularity, generating second cache line metadata associated with the second access granularity. The method further includes storing the first cache line metadata and the second cache line metadata in a single cache memory component.

TECHNICAL FIELD

Embodiments of the disclosure relate generally to memory sub-systems, and more specifically, relate to dynamic access granularity in a cache media.

BACKGROUND

A memory sub-system can include one or more memory components that store data. The memory components can be, for example, non-volatile memory components and volatile memory components. In general, a host system can utilize a memory sub-system to store data at the memory components and to retrieve data from the memory components.

DETAILED DESCRIPTION

A memory module can be accessed by multiple processing components of a host system. In some instances, the processing components (e.g., a central processing unit (CPU), or a graphical processing unit (GPU)), can have different optimal or preferred data access granularities. For example, a CPU can optimally retrieve data from memory in segments that are 64 bytes in size while a GPU can optimally retrieve data from memory in segments that are 32 bytes in size. Conflicts in access granularity management can arise in circumstances in which two processing components with different access granularities are accessing data from the same memory module. Conventional memory modules do not directly provide multiple access granularities to the host system. In conventional memory modules, if two processing components use different access granularities, the memory module provides the data in segments of a single size to the host system. Components of the host system must then identify the different access granularities and format the segments of data according to the correct access granularity of each processing component. Therefore, conventional memory modules do not support multiple access granularities unless the host system includes additional components and/or logic. The additional components and/or logic can increase access latency, increase cost, and reduce available space within the host system, such as on a processor.

Aspects of the present disclosure address the above and other deficiencies by providing dynamic access granularity in a cache media of a memory sub-system. A cache memory component can support cache lines with different access granularities by adjusting status bits of cache line metadata according to the access granularity of each corresponding cache line. Depending on the access granularity of a particular cache line, a cache controller can manage certain status bits of the cache line metadata to identify the status of sectors within the cache line. For example, the cache line metadata can include a mode bit which can indicate the access granularity of the cache line. According to the mode bit, and thus the access granularity, the cache controller can use particular status bits of the cache line metadata to manage the cache line. Each cache line can include the same number of status bits and the cache controller can use the mode bit, or granularity, to determine which status bits to use to manage the cache line. For example, if the access granularity is a smaller size, then the cache controller can manage each status bit of the cache line metadata. If the access granularity is a larger size then the cache controller can manage just half of the status bits, for example. Alternatively, if no mode bit is used, the cache controller can set each bit of the cache line metadata despite the access granularity. Since there are fewer sectors in the larger access granularity cache lines and the number of available status bits is constant, the cache controller can set more status bits for each sector. For example, 2 status bits can be used for a 32 byte sector while 4 status bits can be used for a 64 byte sector. In another example, the cache memory component can be divided into physically separate partitions that each store cache lines with a single access granularity.

Therefore, the ability to provide dynamic access granularity within a memory sub-system could reduce data access latency for host systems that include devices using different granularities of data. In addition, dynamic access granularity in the memory sub-system could reduce the overhead required to provide multiple access granularity and could provide compatibility across many different host devices using either single access granularity or multiple access granularities.

FIG.1illustrates an example computing system100that includes a memory sub-system110in accordance with some embodiments of the present disclosure. The memory sub- system110can include media, such as one or more volatile memory devices (e.g., memory device140), one or more non-volatile memory devices (e.g., memory device130), or a combination of such.

Some examples of non-volatile memory devices (e.g., memory device130) include negative-and (NAND) type flash memory and write-in-place memory, such as three-dimensional cross-point (“3D cross-point”) memory. A cross-point array of non-volatile memory can perform bit storage based on a change of bulk resistance, in conjunction with a stackable cross-gridded data access array. Additionally, in contrast to many flash-based memories, cross-point non-volatile memory can perform a write in-place operation, where a non-volatile memory cell can be programmed without the non-volatile memory cell being previously erased.

Although non-volatile memory components such as 3D cross-point type and NAND type flash memory are described, the memory device130can be based on any other type of non-volatile memory, such as read-only memory (ROM), phase change memory (PCM), self-selecting memory, other chalcogenide based memories, ferroelectric random access memory (FeRAM), magneto random access memory (MRAM), negative-or (NOR) flash memory, electrically erasable programmable read-only memory (EEPROM), and a cross-point array of non-volatile memory cells. A cross-point array of non-volatile memory can perform bit storage based on a change of bulk resistance, in conjunction with a stackable cross-gridded data access array. Additionally, in contrast to many flash-based memories, cross-point non-volatile memory can perform a write in-place operation, where a non-volatile memory cell can be programmed without the non-volatile memory cell being previously erased.

In some embodiments, the local memory119can include memory registers storing memory pointers, fetched data, etc. The local memory119can also include read-only memory (ROM) for storing micro-code. While the example memory sub-system110inFIG.1has been illustrated as including the memory sub-system controller115, in another embodiment of the present disclosure, a memory sub-system110may not include a memory sub-system controller115, and may instead rely upon external control (e.g., provided by an external host, or by a processor or controller separate from the memory sub-system).

The memory sub-system110includes an access granularity management component113that can be used to provide dynamic access granularity in a cache media. In some embodiments, the memory sub-system controller115includes at least a portion of the access granularity management component113. For example, the memory sub-system controller115can include a processor117(processing device) configured to execute instructions stored in local memory119for performing the operations described herein. In some embodiments, the access granularity management component113is part of the host system110, an application, or an operating system.

The access granularity management component113can generate cache line metadata including status bits associated with sectors of a cache line in a cache memory component. The access granularity management component113can manage the status bits of the sectors of a cache line according to the access granularity of the associated cache line. The access granularity management component113can additionally access and read the status bits of the cache line metadata to determine a status of a sector of a cache line. Further details with regards to the operations of access granularity management component113are described below.

At operation210, processing logic of a memory sub-system receives a memory access request including an address of data to be accessed. The address can identify a location in a backend memory such as 3DXP, flash memory, DRAM or any other memory or storage media. The memory access request can be a read operation or a write operation or any other data operation. In addition, the data to be accessed can be stored in the backend memory as well as a separate cache memory component with a lower access latency than the backend memory.

At operation220, the processing logic of a memory sub-system determines an access granularity of the data to be accessed based on the address of the data to be accessed. The processing logic can compare the address of the data to be accessed with address ranges, or regions of memory, identifying different access granularities. For example, the processing logic can query a lookup table, an address map, a set of registers, or the like, to determine which region of memory the address falls within. One region of memory can be associated with one access granularity (e.g., for a CPU) and another region of memory can be associated with another access granularity (e.g., for a GPU).

At operation230, the processing logic generates, in response to determining that the data to be accessed has a first access granularity, first cache line metadata associated with the first access granularity. The first cache line metadata can include one or more bits indicating whether a cache line of the data to be accessed is dirty and/or valid. If the data to be accessed has a first access granularity then the processing logic can manage a defined number of status bits in the cache line metadata to indicate valid and dirty statuses. For example, the first access granularity can be 32 bytes and for each 32 byte sector in a cache memory component, one status bit of the cache line metadata can indicate dirty and one status bit of the cache line metadata can indicate valid. Thus, when accessing data from the cache memory component that is of the first granularity, the processing logic can use the one dirty bit and the one valid bit to determine the status of the cache line

At operation240, the processing logic generates, in response to determining that the data to be accessed has a second access granularity, second cache line metadata associated with the second access granularity. The second cache line metadata can include the same number of status bits as the first cache line metadata, but the processing logic can set and interpret the bits of the second cache line metadata differently according to the second access granularity. For example, the second access granularity can be 64 bytes (i.e., two contiguous 32 byte sections) and the processing logic can set the status bits of the second cache line metadata accordingly.

In one example, the processing logic can identify a mode bit of the cache line metadata. The mode bit can indicate the access granularity of the cache line and which status bits of the cache line metadata to use for the access granularity of the cache line. For example, a mode bit set to 1 can indicate a first access granularity and the mode bit set to 0 can indicate a second access granularity.

In another example, before any access of data, the processing logic can determine the access granularity of the cache line of the data to access using an address map or lookup table. Then the processing logic can manage the status bits of the cache line metadata according to the identified access granularity. For example, for a 32 byte access granularity the processing logic can use a set of bits for each 32 byte sector of data. For a 64 byte access granularity the processing logic can use two pairs of status bits for a 64 byte sector of data, where the two pairs of status bits are set the same (i.e., each 32 byte portion of the 64 byte sector can be set the same).

In yet another example, the cache memory component can be partitioned into physically separate caches with different access granularities. Each partition can additionally include its own corresponding cache line metadata. Thus, the processing logic can switch between access granularities without additional status bits.

At operation250, the processing logic stores the first cache line metadata and the second cache line metadata in a single cache memory component. Although the first cache line metadata and the second cache line metadata are associated with different access granularities of cached data, since the processing logic identifies the access granularities and determines the status of the cache line based on the access granularity, the processing logic can store and access the associated data from the same cache memory component.

At operation310, processing logic of a memory sub-system receives a memory access request including an address of data to be accessed. The address can identify a location in a backend memory such as 3DXP, flash memory, DRAM or any other memory or storage media. The memory access request can be a read operation or a write operation or any other data operation. In addition, the data to be accessed can be stored in the backend memory as well as a separate cache memory component with a lower access latency than the backend memory.

At operation320, the processing logic determines an access granularity of the data to be accessed. To determine the access granularity of the data to be accessed, the processing logic can compare the address of the data with two or more distinct ranges of memory addresses. Each range of memory addresses can store data with a particular access granularity. For example, a first range of memory addresses can be associated with data accessible by a CPU while a second range of memory addressed can be associated with data accessible by a GPU. The data accessible by the CPU can have one accesses granularity (e.g., 64 bytes) while the data accessible by the GPU can have a different access granularity (e.g., 32 bytes). The association between memory ranges and access granularity can be stored in a data structure such as a lookup table or an address mapping table.

At operation330, the processing logic identifies, based on the address of the data to be accessed, a cache line of the data to be accessed and cache line metadata associated with the cache line. The cache line of the data to be accessed can include the data from the address of the backend media which is temporarily stored at the cache memory component. The cache line in the cache memory component can be modified by write operations received from the host system and can be forwarded to the host system in response to a read operation. To identify the cache line and cache line metadata, the processing logic can match the address of the data to be accessed that was received from the memory access request of the host system to a metadata tag included in the cache line metadata. In one example, the processing logic can use just a portion of the address to identify the cache line metadata and the corresponding cache line.

The cache line metadata associated with the cache line can include metadata indicating the status of the cache line, access information about the cache line, and any other data relevant to the cache line. The status metadata of the cache line can include at least a dirty bit and a valid bit. The dirty bit can indicate that the cache line has been modified and thus contains data that is inconsistent with the data in the backend media. When the cache line is indicated as dirty, the cache line can be written back to the backend media prior to removing the cache line from the cache memory component to ensure that the modifications persist in the stored data. The valid bit can indicate that the cache line contains valid data and can be used in a data access operation.

At operation340, the processing logic determines, based on the access granularity of the data to be accessed and one or more bits of the cache line metadata, a status of the cache line. The processing logic can identify which bits of the cache line metadata to use to determine the status of the cache line based on the access granularity of the data. The access granularity can indicate which bits, and how many bits, of the cache line metadata indicate the status of the cache line.

In one example, the processing logic can identify a mode bit of the cache line metadata. The mode bit can indicate the access granularity of the cache line and which status bits of the cache line metadata to use for the access granularity of the cache line. For example, a mode bit set to 1 can indicate a first access granularity and the mode bit set to 0 can indicate a second access granularity.

In another example, before any access of data, the processing logic can determine the access granularity of the cache line of the data to access using an address map or lookup table. The processing logic can then manage the status bits of the cache line metadata according to the identified access granularity. For example, for a 32 byte access granularity the processing logic can use a set of bits for each 32 byte sector of data. For a 64 byte access granularity the processing logic can use two pairs of status bits for a 64 byte sector of data, where the two pairs of status bits are set the same (i.e., each 32 byte portion of the 64 byte sector can be set the same).

In yet another example, the cache memory component can be partitioned into physically separate caches with different access granularities. Each partition can additionally include its own corresponding cache line metadata. Thus, the processing logic can switch between access granularities without additional status bits.

FIG.4depicts an example computing environment including a host system410and a memory sub-system430. The host system includes both a CPU412and a GPU416each with an associated cache, CPU cache414and GPU cache418. The CPU cache414and GPU cache418can each include a distinct cache hierarchy and different data access granularities. The memory controller420of the host system410can send memory access requests, including reads and writes, to the memory sub-system430. The memory access requests can include access requests for the CPU412or the GPU416. In one example, a memory access request from the CPU412can request data of one size while the GPU416requests data of another size (e.g., 64 bytes and 32 bytes respectively).

The memory sub-system430includes a cache controller432, a cache memory component434, and backend memory436A-C. The cache controller432can include cache policies to determine data from the backend memory436A-C to be cached at the cache memory component434for quick access of the data by the host system410. Additionally, the cache controller432can identify data in the cache memory component434that is dirty and that should be evicted from the cache memory component434. The cache controller432can write dirty cache lines of the cache memory component434back to the backend memory436A-C to ensure the data is maintained correctly and remains consistent. The cache memory component434can include both cache lines of data and cache line metadata. Alternatively, the cache line metadata can be included in a separate cache metadata component, such as in SRAM or a CAM, for fast access of the metadata. The metadata can include tags to identify the cache line and status bits to indicate whether each accessible portion of the cache line contains valid data and whether the cache line is dirty (i.e., inconsistent with the backend memory436A-C). The backend memory436A-C can be a type of memory with a larger capacity and higher access latency than the cache memory component.434. The backend memory436A-C can be a form of persistent memory, such as 3DXP, flash or NAND memory, or any other form of persistent memory. The cache controller can include a include an access granularity management component113to manage the metadata of cache lines according to access granularity.

In one example, the access granularity management component113can identify a mode bit of the cache line metadata. The mode bit can indicate the access granularity of the cache line and which status bits of the cache line metadata to use for the access granularity of the cache line. For example, a mode bit set to 1 can indicate a first access granularity and the mode bit set to 0 can indicate a second access granularity.

In another example, before any access of data, the access granularity management component113can determine the access granularity of the cache line of the data to access using an address map or lookup table. The access granularity management component113can manage the status bits of the cache line metadata according to the identified access granularity. For example, for a 32 byte access granularity the access granularity management component113can use a set of bits for each 32 byte sector of data. For a 64 byte access granularity the access granularity management component113can use two pairs of status bits for a 64 byte sector of data, where the two pairs of status bits are set the same (i.e., each 32 byte portion of the 64 byte sector can be set the same).

In yet another example, the cache memory component can be partitioned into physically separate caches with different access granularities. Each partition can additionally include its own corresponding cache line metadata. Thus, the access granularity management component113can switch between access granularities without additional status bits.

FIG.5Aillustrates two example cache lines with two different access granularities. Although the depicted 32 byte granularity cache line has 6 different 32 byte sectors of data, the 32 byte granularity cache line can include any number of sectors of data (e.g., 8 sectors, 16 sectors, etc.). Similarly, although the depicted 64 byte granularity cache line has 3 different 64 byte sectors of data, the 64 byte granularity cache line can include any number of sectors of data (e.g., 4 sectors, 8 sectors, 16 sectors, etc.). As described below with respect toFIGS.5B-D, a cache line can have associated cache line metadata used, for example by the access granularity management component, to indicate a status of each sector of data in a cache line and can include other metadata about the cache line (e.g., access data used for the caching scheme).

FIG.5Billustrates one example of cache line metadata for a 32 byte access granularity cache line and a 64 byte granularity cache line. Because the cache lines with different access granularities are included in the same cache memory component, the access granularity management component can first determine what access granularity the cache line includes. Once the access granularity management component determines the access granularity it can determine which bits of the cache line metadata to use to determine the status, among other information, of the sectors of the cache line.

For example,FIG.5Bdepicts a 32 byte cache line metadata which includes a tag used to identify the metadata and the associated cache line, a mode bit, and 6 sets of status bits (one set for each 32 byte sector ofFIG.5A). The access granularity management component can identify the cache line metadata by matching an address of a data access request (or a portion of the address) to the tag of the cache line metadata. Once the cache line metadata is identified, the cache controller can determine what the access granularity of the cache line according to the mode bit. In one example, the mode bit set to 1 indicates a 32 byte cache granularity, while the mode bit set to 0 indicates a 64 byte cache granularity. It should be noted that any other granularity can be used (e.g., 8 byte and 16 bytes) and additional mode bits can be included to provide support for additional access granularities.

In another example, the access granularity management component can generate the cache line metadata according to the access granularity of the data of the cache line. The cache controller can determine the access granularity of the data based on the address of the data and set the mode bit of the cache line metadata according to the access granularity. Additionally, the access granularity management component can set the status bits for the sectors of the cache line according to the access granularity. For example, as depicted, each status bit in the cache line metadata can be set for the 32 byte cache lines, each set of status bits corresponding to a 32 byte sector of the cache line. For the 64 byte access granularity cache line metadata, every second set of status bits can be set (and later read) by the access granularity management component. Therefore, only half of the status bits of the 64 byte cache line metadata are set. In one example, the cache controller only sets the first set of every two sets of status bits. When accessing the cache line metadata, the access granularity management component can use the first set of every two sets of status bits to determine the status of a 64 byte sector.

FIG.5Cillustrates an example 32 byte access granularity cache line metadata including six sets of status bits. Each set of status bits can correspond to one of the 32 byte sectors of the 32 byte access granularity cache line depicted inFIG.5A. In one example, the status bits ofFIG.5Care set and managed by a cache controller. Rather than setting a mode bit and checking the mode bit when accessing the data, the access granularity management component can determine the access granularity at the time of access using the address of the data to be accessed.

FIG.5Dillustrates an example 64 byte access granularity cache line metadata including six sets of status bits. Each set of status bits can correspond to one of the 64 byte sectors of the 64 byte access granularity cache line depicted inFIG.5A. In one example, the status bits ofFIG.5Dare set and managed by a cache controller. Rather than setting a mode bit and checking the mode bit when accessing the data, the access granularity management component can determine the access granularity at the time of access using the address of the data to be accessed. Therefore, each status bit of the cache line metadata for the 64 byte access granularity cache line metadata is used by the cache controller. For example, as depicted inFIG.5D, if the status of a 64 byte sector is valid and dirty, then four bit can be set to indicate that the 64 bytes are valid and dirty. The first two status bits can be set valid and dirty and the second two status bits can be set in the same way as the first two status bits when the cache line metadata is for data with 64 byte access granularity. In the same or different example, a second 64 byte sector can be valid and not dirty (1-0) and so both the first two status bits and the second two status bits can be set as valid and not dirty (1-0).

FIG.6is a diagram illustrating a cache memory component434partitioned into different access granularities. The cache memory component434can be partitioned into a 64 byte cache granularity partition620and a 32 byte cache granularity partition610. Alternatively, the cache component434can be partitioned into any number of different partitions and can include cache granularities different than 32 bytes and 64 bytes. In the 32 byte cache granularity partition610, the data can be accessed from a cache line in 32 byte portions while the data in the 64 bytes cache granularity partition620can be accessed from a cache line in 64 byte portions. The 32 byte cache granularity partition610can therefore have distinct cache line metadata615associated with each of the cache lines stored in the 32 byte cache granularity partition610. Similarly, the 64 byte cache granularity partition620can therefore have distinct cache line metadata625associated with each of the cache lines stored in the 64 byte cache granularity partition620. Each cache line metadata for a cache line can include a tag to identify the cache line. The metadata for each cache line of the 32 byte cache metadata615can include status bits such as valid and dirty bits for each individual 32 byte portion of the 32 byte partition (i.e., as shown inFIG.6, there can be 6 sets of status bits for each of the 6-32 byte portions of a cache line). The metadata for each cache line of the 64 byte cache metadata625can include status bits such as valid and dirty bits for each individual 64 byte portion of the 64 byte partition (i.e., as shown inFIG.6, there can be 3 sets of status bits for each of the 3-64 byte portions of a cache line). It should be noted that a cache line and the number of bits for each cache line is not limited to the number depicted inFIG.6.

The example computer system700includes a processing device702, a main memory704(e.g., read-only memory (ROM), flash memory, dynamic random access memory (DRAM) such as synchronous DRAM (SDRAM) or Rambus DRAM (RDRAM), etc.), a static memory706(e.g., flash memory, static random access memory (SRAM), etc.), and a data storage system718, which communicate with each other via a bus730.

The data storage system718can include a machine-readable storage medium724(also known as a computer-readable medium) on which is stored one or more sets of instructions726or software embodying any one or more of the methodologies or functions described herein. The instructions726can also reside, completely or at least partially, within the main memory704and/or within the processing device702during execution thereof by the computer system700, the main memory704and the processing device702also constituting machine-readable storage media. The machine-readable storage medium724, data storage system718, and/or main memory704can correspond to the memory sub-system110ofFIG.1.