Apparatuses and methods for an operating system cache in a solid state device

The present disclosure includes apparatuses and methods for an operating system cache in a solid state device (SSD). An example apparatus includes the SSD, which includes an In-SSD volatile memory, a non-volatile memory, and an interconnect that couples the non-volatile memory to the In-SSD volatile memory. The SSD also includes a controller configured to receive a request for performance of an operation and to direct that a result of the performance of the operation is accessible in the In-SSD volatile memory as an In-SSD main memory operating system cache.

TECHNICAL FIELD

The present disclosure relates generally to solid state devices, and more particularly, to apparatuses and methods for an operating system cache in a solid state device.

BACKGROUND

In some implementations, a solid state drive may be utilized only for data storage. As such, the solid state drive may be accessed only when data stored therein is requested by an operating system to enable performance of a function directed by the operating system. However, accessing the data in the solid state drive, accompanied by transfer of the data to a host, may be associated with an access latency and/or transfer time that may contribute to a length of time for performance of the function.

DETAILED DESCRIPTION

The present disclosure includes apparatuses and methods for a main memory (MM) operating system (OS) cache, e.g., as shown at125and225-0,225-1, . . . ,225-N−1 and described in connection withFIGS. 1 and 2A-2B, respectively, in a solid state device (SSD), e.g., an “In-SSD MM OS cache”. An example apparatus includes the SSD, which, in various embodiments, includes a volatile memory (VM), e.g., an In-SSD VM124to distinguish from host VM115, a non-volatile memory (NVM)126, and an interconnect106-4that couples the NVM to the In-SSD VM. The SSD also may include a controller configured to receive a request for performance of an operation and to direct that a result of the performance of the operation is accessible in the In-SSD MM OS cache of the In-SSD VM. An In-host MM OS cache117in a host MM116and the In-SSD MM OS cache125in the In-SSD VM124may each form portions of the host MM116addressable by an OS103in a host system102.

As used herein, designators such as “X”, “Y”, “N”, “M”, etc., particularly with respect to reference numerals in the drawings, indicate that a number of the particular feature so designated can be included. It is also to be understood that the terminology used herein is for the purpose of describing particular embodiments only, and is not intended to be limiting. As used herein, the singular forms “a”, “an”, and “the” can include both singular and plural referents, unless the context clearly dictates otherwise. In addition, “a number of”, “at least one”, and “one or more” (e.g., a number of memory arrays) can refer to one or more memory arrays, whereas a “plurality of” is intended to refer to more than one memory array. Furthermore, the words “can” and “may” are used throughout this application in a permissive sense (i.e., having the potential to, being able to), not in a mandatory sense (i.e., must). The term “include,” and derivations thereof, means “including, but not limited to”. The terms “coupled” and “coupling” mean to be directly or indirectly connected physically or for access to and movement (transmission) of commands and/or data, as appropriate to the context. The terms “data” and “data values” are used interchangeably herein and can have the same meaning, as appropriate to the context.

FIG. 1is a block diagram of a computing system100that includes an In-SSD MM OS cache125in a SSD104in accordance with a number of embodiments of the present disclosure. As illustrated inFIG. 1, the computing system100may include a host system102that includes OS103. In some embodiments, the OS103may be configured as a system software component. The OS103may, in various embodiments, be associated with a central processing unit (CPU)110.

The host system102may include, as illustrated inFIG. 1, the CPU110, the OS103, a number of caches112,114, and a host main memory (MM)116, which may include the host VM115and, in some embodiments, a host NVM (not shown). The host VM115may include an In-host MM OS cache117, which itself may include a number of In-host MM OS page caches119. The host VM115in the host MM116, along with the In-host MM OS cache117and/or the In-host MM OS page caches119included in the host VM115, may be addressable by the OS103and/or the CPU110. For example, a page of data, e.g., 4 kilobytes to 2 megabytes, may be moved107from an In-host MM OS page cache119of the In-host MM OS cache117to CPU internal cache112to enable processing by the CPU110. As such, the CPU internal cache112may be termed a processor cache. The CPU internal cache112may, in various embodiments, include a Level 1 (L1) cache, a L2 cache, and/or a L3 cache. A CPU external cache114may be a L4 cache. In some embodiments, the CPU internal cache112may be static random access memory (SRAM), whereas the host MM116, including the host VM115and the In-host MM OS cache117, may be dynamic random access memory (DRAM).

The host system102may be configured in various embodiments, as described herein. For example, the host system102may include the CPU110, which may execute encoded instructions stored by the OS103. The host system102may further include the number of caches112internal to the CPU110, the number of caches114external to the CPU110, the host MM116, including the host VM115, the In-host MM OS cache117, the In-host MM OS page caches119, and/or a device driver118. The host MM116may function as the main memory for the host system102.

The computing system100also may include at least one SSD104connected to the host system102and/or the OS103by an interconnect106-1, as described herein. The SSD104can, in various embodiments, include a number of components. In some embodiments, the OS103and the interconnect106-1may be operated as part of an interface circuitry108(e.g., a PCIe circuit) with the at least one SSD104representing one of a number of memory devices that may each include at least one In-SSD VM124and at least one NVM126, as described herein. Each SSD104may include a controller120configured to receive requests (commands) for performance of operations issued from the OS103, e.g., via interconnect106-1coupled to interconnect106-2. The controller120also may be configured to direct performance of the operations, e.g., via interconnect106-3and/or interconnect106-4, using data cached and/or stored by the In-SSD VM124and the NVM126, e.g., coupled by interconnect106-4. The controller120also may be configured to output results and/or notification, e.g., confirmation, of performance of the operations to the OS103, e.g., via interconnect106-2coupled to interconnect106-1.

A distributed computing architecture, as described herein, may include the use of interface circuitry108that interconnects distributed components of the computing system100, e.g., as described in connection withFIG. 1. A communication interface, e.g., referred to herein as an interconnect and shown at106-1,106-2,106-3, and106-4, may communicate via a logical connection. An interconnect106may provide a point-to-point communication channel between, for example, a port of a first electronic component and a port of a second electronic component, e.g., among a plurality of ports of the first and second electronic components of the distributed computing architecture. The interconnect106may allow both ports to send and/or receive requests, e.g., for movement of data between components, storing and/or caching of data in one or more of the components, performance of read and/or write operations on data stored by, e.g., stored in, a cache, and/or performance of read and/or write operations on data stored by, e.g., stored in, a memory, among other requests. In some embodiments, the read and/or write operations may be performed on data stored at least temporarily in one or more OS caches in the SSD104, e.g., In-SSD MM OS cache in In-SSD VM124as shown at125and225-0,225-1, . . . ,225-N−1 and described in connection withFIGS. 1 and 2A-2B, respectively. The interconnect106also may allow both ports to send and/or receive results of performance of requested operations, e.g., results of the cache read and/or write operations, the memory read and/or write operations, among results of other operations.

For instance, the computing system100may use a peripheral component interconnect express (PCIe) bus including a number of interconnects106to interconnect the various components into an interface (e.g., PCIe) circuit, as indicated by reference number108inFIG. 1. In order for a PCIe peripheral memory device, e.g., SSD104, to be addressable, the peripheral memory device may first be mapped into an input/output (I/O) port address space or a memory-mapped address space of the computing system100. The computing system's firmware (not shown), device drivers118, SSD controllers120, and/or the OS103, e.g., a part of the computing system100shown and described in connection withFIG. 1, may be used to program the mapping of the peripheral memory device. For example, the mapping may be programmed to a base address register (not shown) to inform the peripheral memory device of its address mapping by writing configuration commands to a PCIe controller (not shown).

For example, a PCIe can be a serial expansion interface circuit (bus) that may provide improvements over, for example, PCI, PCI-X, and AGP (Accelerated Graphics Port) bus standards, among others. Such improvements may include higher bus throughput, lower I/O pin count and a smaller physical footprint, better performance-scaling for bus devices, more detailed error detection and reporting mechanisms, and/or native hot-plug functionality. As used herein, a higher bus throughput is intended to mean using the interface circuitry108to increase a speed and/or rate and/or to reduce a latency of data and/or command movement within and/or between components described herein, e.g., OS103, device driver118, SSD104, controller120, In-SSD MM OS cache125, In-SSD VM124, and/or NVM126, among other components. Embodiments described herein make reference to PCIe for simplicity. However, as one of ordinary skill in the art would appreciate, other interface circuitry and/or interconnects are contemplated, e.g., PCI, among others.

The PCIe interface circuitry108may, for example, be a high-speed serial interconnect bus using shared address/data lines. Accordingly, a PCIe bus may differ from other PCI buses in its bus topology. For instance, a PCI bus may use a shared parallel bus architecture, where the PCI host and all devices share a common set of address/data/control lines. Because of this shared bus topology, access to a PCI bus may be arbitrated in the case of multiple masters and may be limited to one master at a time in a single direction. In contrast, a PCIe bus may be based on a point-to-point topology with separate serial links connecting every device to a root complex (or host).

As used herein, data movement is an inclusive term that includes, for instance, copying, transferring, and/or transporting data values from a component in a source location to a component in a destination location. Data can, for example, be moved from the NVM operating as a data storage resource, e.g., as shown at126and226and described in connection withFIGS. 1 and 2A-2B, respectively, to a host system102and/or an In-SSD MM OS cache125for performance of an operation directed by OS, e.g., as shown at103and203and described in connection withFIGS. 1 and 2A-2B, respectively. The data can, as described herein, be moved via the In-SSD VM124operating as an In-SSD MM OS cache resource, e.g., as shown at224and described in connection withFIGS. 2A and 2B. The In-SSD MM OS cache resource224may, as described herein, also be configured, e.g., formed, to be included in the SSD and coupled to, e.g., via interconnect106-4, the NVM data storage resource226of the SSD204. The NVM126and/or the NVM data storage resource226may, in some embodiments, be NAND memory and/or 3D) XPoint memory operated as secondary storage in relation to primary storage of the host MM116and the In-SSD MM OS cache resource224.

Copying the data values is intended to indicate that the data values at least temporarily stored and/or cached in a source component, e.g., NVM126and/or In-SSD MM OS cache125of In-SSD VM124, are moved to the destination component, e.g., to In-SSD MM OS cache125of In-SSD VM124, to In-host MM OS cache117of host MM116, and/or to CPU internal cache112, and that the original data values stored, e.g., in a row and/or cache of the source component, may remain unchanged. Transferring the data values is intended to indicate that the data values at least temporarily stored and/or cached in a source component are moved to the destination component and that at least one of the original data values stored, e.g., in the row and/or cache of the source component, may be changed, e.g., by being erased and/or by a subsequent write operation, as described herein. Transporting the data values is intended to indicate the process by which the copied and/or transferred data values are moved. For example, the data values can be transported via an interconnect106of the interface circuitry108, as described herein, from being stored and/or cached in the source location to the destination location.

For clarity, description of the computing system100has been simplified to focus on features with particular relevance to the present disclosure. For example, in various embodiments, there may be more than one OS103and/or more than one SSD104operating as an In-SSD MM OS cache resource224coupled to, e.g., via interconnect106-4, an NVM data storage resource226of the SSD204. In some embodiments, the In-SSD MM OS cache resource224may be a dynamic random access memory (DRAM) array operating as a portion of the host MM116, although other configurations of volatile memory are within the scope of the present disclosure, e.g., a static random access memory (SRAM) array, a synchronous dynamic random access memory (SDRAM), and/or thyristor random access memory (TRAM), among others. The data storage resource described herein is intended to be the NVM126of the SSD104. In some embodiments, the NVM126may be a 3D XPoint array, although other configurations of non-volatile memory are within the scope of the present disclosure, e.g., a spin torque transfer random access memory (STT RAM) array, a phase change random access memory (PCRAM) array, a resistive random access memory (RRAM) array, a NAND flash array, and/or a NOR flash array, among others.

As described herein, a 3D XPoint array is intended to mean a three-dimensional cross-point array of memory cells for an SSD that is configured for non-volatile data storage and for reduced latency of data access and/or retrieval. The latency may be reduced relative to other non-volatile memory, e.g., NAND flash memory, among others, to a level approaching the relatively short latency achievable with volatile memory (e.g., DRAM). For example, a 3D XPoint array may, in various embodiments, operate as NVM126in combination with In-SSD VM124, e.g., a DRAM array, via interconnect106-4. Coupling the NVM126to the In-SSD VM124via interconnect106-4may further contribute to increasing the speed and/or rate and/or to reducing the latency of data movement between these components.

Such In-SSD VM and NVM resources in the SSD104each may, in various embodiments, include memory cells arranged in rows coupled by access lines (which may be referred to as word lines or select lines) and columns coupled by sense lines (which may be referred to as data lines or digit lines). In some embodiments, the plurality of memory arrays of the memory resources may be partitioned to enable operations to be performed in parallel in one or more partition.

In at least one embodiment, the host system102may include the OS103running on the CPU110. The OS103may be configured to request, by issuance of a command and/or directing performance of, e.g., via one or more processors, various operations, such as the read operation and/or the write operation described herein. In addition to a L1 cache, the present disclosure may, in various embodiments, utilize combinations of at least one of three other levels of cache, such as L2, L3, and/or L4. The L1, L2, L3, and/or L4 caches each may contribute to bridging a gap between a fast CPU110and a slower secondary memory, e.g., In-SSD VM124and/or NVM126configured without the In-SSD MM OS cache125and/or the interface circuitry108described herein. The host system102can, in some embodiments, include a number of internal caches112as part of the CPU110. For example, L1, L2, and/or L3 caches may be built into the CPU, e.g., as the CPU internal caches112. In various embodiments, an L2 cache or an L3 cache may be a last level cache (LLC) in the CPU internal cache112hierarchy.

Alternatively or in addition, the host system102can include a number of CPU external caches114that are external to the CPU110. The internal and/or external caches may be configured as memory to at least temporarily store data retrieved from the memory resources, e.g., In-SSD VM124and/or NVM126, of the SSD104in order to be used by the CPU110in performance of a number of operations, such as the read operation described herein. For example, a L1, L2, and/or L3 cache hierarchy in the CPU internal caches112may store data that is frequently and/or recently accessed by the CPU110. When data requested by the CPU110are not found in the respective internal cache hierarchy, the CPU external caches114may be checked to determine if the requested data are cached therein. The CPU external cache(s)114may, in various embodiments, include one or more of various types of caches. The CPU external cache(s)114may contribute to a multi-level storage strategy for computer performance. Caches such as L4 may be located external to the CPU (as the CPU external caches114), for example, off-chip and/or built into a motherboard, among other locations.

If the requested data are not cached in the CPU internal caches112, CPU external caches114, and/or the In-host MM OS cache117of the host MM116, a storage access request may be sent (issued) via the OS103to the device driver118. As described herein, the request may be sent (forwarded) by the device driver118to the controller120of the SSD104to retrieve136the data from, for example, NVM126. The controller120may implement the request, e.g., a software routine call, by retrieval of the requested data from an In-SSD MM OS cache125in the In-SSD VM124of the SSD104. The In-SSD MM OS cache125may include a number of In-SSD MM OS page caches127. The In-SSD MM OS cache125and/or the In-SSD MM OS page caches127included in the In-SSD VM124may be addressable by the OS103as a portion of the host MM116. For example, data from the In-SSD MM OS page caches127may be sent directly138to the CPU internal cache112at a granularity of, for example, 64 bytes for processing by the CPU110. By comparison, access of data to be directly sent105from the NVM126to the host system102may have a granularity of 512 bytes or higher. Accordingly, data may be sent directly138with an increased speed, rate, and/or efficiency from the In-SSD MM OS cache125to the CPU internal cache112, rather than by initiating an operation to directly send105the requested data from the NVM126to a cache of the host system102, e.g., to In-host MM OS cache117of the host MM116.

For example, in some pre-existing implementations, such a request may initiate retrieval of such data directly105from the NVM126, e.g., from 3D) XPoint memory, among other types of NVM, for movement of the data via a dual in-line memory module (DIMM) to the In-host MM OS cache117of host MM116for performance of operations. However, such a data movement path has reduced speed and/or rate and/or an increased latency of data movement relative to the OS103directing access of such data for movement from an In-SSD MM OS cache125of the In-SSD VM124rather than directly from the NVM126. For example, the reduced speed and/or rate may be at least partially due to an increased intrinsic latency for the host system102and/or device driver118accessing the NVM126, e.g., the 3D XPoint memory, relative to accessing data having been moved136from the NVM126to the In-SSD MM OS cache125of the In-SSD VM124in the SSD104via the interface circuitry108, e.g., accessing the cached data via interconnects106-1,106-2, and/or106-3.

The host system102may, in some embodiments, include a number of buffers (not shown) as part of, or external to, the CPU110. The buffers may be memory configured to accumulate data, for example, as received from the CPU110, and move the data to the memory resources of the SSD104in performance of a number of operations, such as a write operation. As described herein, the host system102also may include a primary memory resource. The primary memory resource may include the CPU internal caches112, the CPU external cache(s)114, the host VM115and/or the In-host MM OS cache117of the host MM116.

The SSD104includes, for example, a combination of In-SSD VM124, NVM126, and a controller120to direct operations performed using data stored and/or cached by these volatile and non-volatile memories, along with the interconnects106-2,106-3, and/or106-4of the interface circuitry108described herein. The In-SSD MM OS cache125and In-SSD MM OS page caches127described herein may be considered a portion of the primary memory resource.

In some embodiments, the controller120of the SSD104may include a double data rate (DDR) SDRAM controller122. Compared to single data rate (SDR) SDRAM, the DDR SDRAM interface may provide, for example, higher data movement, e.g., transfer, rates by stricter control of the timing of the electrical data and/or clock signals. The DDR SDRAM may use double pumping for transfer of data on both the rising and falling edges of the clock signal to double data bus bandwidth without a corresponding increase in clock frequency. One advantage of keeping the clock frequency down is that it may reduce signal integrity requirements on the circuit board, for example, for connecting memory, e.g., In-SSD VM124and/or NVM126, to the controller120. The “double data rate” refers to the DDR SDRAM operated with a particular clock frequency being able to have nearly twice the bandwidth of a SDR SDRAM running at the same clock frequency due to the double pumping. For example, with data being transferred 64 bits at a time, the DDR SDRAM enables a transfer rate of (memory bus clock rate)×2 (for double data rate)×64 (number of bits transferred)/8 (number of bits/byte). Thus, with a memory bus clock rate, e.g., bus frequency, of 100 megahertz (MHz), the DDR SDRAM may enable a transfer rate of 1600 megabytes per second (MB/s).

The controller120may, in various embodiments, be associated with and/or include logic circuitry123. The logic circuitry123may be configured to include a number of processing resources, e.g., one or more processors, which may retrieve and execute instructions and/or store the results of the executed instructions at a suitable location. A processor can include a number of functional units such as arithmetic logic unit (ALU) circuitry, floating point unit (FPU) circuitry, and/or a combinatorial logic block, for example, which can be used to execute instructions by performing an operation on data, e.g., one or more operands. As used herein, an operation can be, for example, a Boolean operation, such as AND, OR, NOT, NOT, NAND, NOR, and XOR, and/or other operations, e.g., invert, shift, arithmetic, statistics, comparison of operation attributes to a number of thresholds to determine selection of a resultant operation from a number of options, among many other possible operations.

A time and/or power advantage may be realized by increasing a speed, rate, and/or efficiency of data being moved around and stored in a computing system in order to process requested memory array operations. Such operations may include compute operations, such as reads and/or writes, etc., as DRAM operations and/or logical operations, such as logical Boolean operations, determination of data movement operations, etc., among others described herein.

The controller120may be coupled to the logic circuitry123. The controller120and/or the logic circuitry123may be associated with and/or include a number of caches, buffers, sense amplifiers, compute components, extended row address (XRA) latches, and/or registers, which may be associated with arrays of memory cells, e.g., In-SSD VM124and/or NVM126, via control lines and/or data paths, e.g., as shown inFIGS. 2A-2B. The controller120may direct and/or control regular DRAM compute operations for the In-SSD VM array124, such as a read, write, copy, and/or erase operations, etc. Additionally, however, instructions retrieved, stored, and/or executed by the logic circuitry123of controller120may cause performance of additional logical operations, such as, for example, addition, multiplication, threshold comparison, or Boolean operations such as an AND, OR, XOR, etc., which are more complex than regular DRAM read and write operations.

In various embodiments, the computing system100can include a device driver118. The device driver118can be configured to perform at least the functions described herein in connection with sending requests from the OS103to the SSD104and/or receiving results and/or notification, e.g., confirmation, of performance of the operations from the SSD104, e.g., from controller120. In some embodiments, host system102can include the device driver118, although embodiments are not so limited. For example, the device driver118for the SSD104may, in some embodiments, be disposed on the SSD104itself or elsewhere in the computing system100. In embodiments in which the device driver118is disposed on the host system102, the device driver118can be selectably coupled to the CPU110and/or OS103and selectably coupled via bus106to the SSD104. For example, at least one device driver118may be configured to selectably couple to a controller120in each SSD104, e.g., via interconnects106-1and/or106-2of interface circuitry108, to provide instructions related to usage of the CPU internal caches112, the CPU external caches114, VMs124, In-SSD MM OS caches125, NVMs126, and/or cache attributes, as described herein. In addition, the controller120may be configured to control access to memory resource VMs124and/or NVMs126of each SSD104, as described herein, in response to instructions received from the device driver118.

As used herein, host system102, OS103, CPU110, a number of caches112internal to the CPU110, a number of caches114external to the CPU110, host MM116, host VM115, In-host MM OS cache117, In-host MM OS page caches119, device driver118, at least one SSD104, interconnects106, interface circuitry108, controller120, In-SSD VM124, In-SSD MM OS cache125, In-SSD MM OS page caches127, DDR SDRAM controller122, logic component123, and/or NVM126might be separately or collectively considered an “apparatus.”

FIG. 2Ais a diagram illustrating a path230for a data movement operation in accordance with a number of embodiments of the present disclosure. The host system202, OS203, device driver218, SSD204, controller220, In-SSD MM OS cache resource224, In-SSD MM OS caches225-0,225-1, . . . ,225-5, and/or NVM data storage resource226components shown inFIGS. 2A and 2Bmay correspond to and/or be embodiments of the components inFIG. 1that are similarly numbered, except for beginning with the digit “1” inFIG. 1. Moreover, although CPU110, caches112internal and/or caches114external to the CPU110, host MM116, interconnects106, and interface circuitry108, among other components shown or described in connection withFIG. 1are not shown inFIGS. 2A and 2Bfor the interest of clarity, one or more of such components may be included in the circuitry illustrated inFIGS. 2A and 2B. As such, each of these components may, in various embodiments, represent at least a portion of the functionality embodied by and contained in the corresponding components shown and described in connection withFIG. 1and elsewhere herein.

As previously stated, data movement is an inclusive term that includes, for instance, copying, transferring, and/or transporting data values. An example of a path for data movement may, in various embodiments, include a requested block of data being moved236from the NVM data storage resource226to at least one of the In-SSD MM OS caches225-0,225-1, . . . ,225-N−1 in the In-SSD MM OS cache resource224. The request may include addresses of one or more blocks of data stored by the NVM data storage resource226and the one or more blocks of data may be moved from the NVM data storage resource226to one or more of the In-SSD MM OS caches225-0,225-1, . . . ,225-N−1.

Each of the In-SSD MM OS caches225-0,225-1, . . . ,225-N−1 may correspond to a number of memory cells in a row, e.g., rows231-0,231-1, . . . ,231-N−1, and/or a column (not shown) of the In-SSD MM OS cache resource224, e.g., a DRAM array. The size of each In-SSD MM OS cache, e.g., the number of memory cells therein, may be fixed or variable, e.g., depending of the number of bits in the requested data.

When more than one In-SSD MM OS cache is utilized to cache the requested data, the caches may be arranged in various manners in a memory array of the In-SSD MM OS cache resource224. For example, a plurality of In-SSD MM OS caches, e.g.,225-0and225-1, may be positioned in a single row, e.g., row231-0, contiguously and/or with unutilized memory cells between the In-SSD MM OS caches anywhere in row231-0, e.g., as selected by OS203. In some embodiments, the plurality of In-SSD MM OS caches, e.g.,225-2and225-3, may be similarly positioned in two or more rows, e.g., rows231-2and231-3, that are vertically oriented. In some embodiments, the plurality of In-SSD MM OS caches, e.g.,225-4and225-N−1, may be similarly positioned in two or more rows, e.g., rows231-3and231-N−1, that are diagonally oriented. In some embodiments, the In-SSD MM OS caches225may correspond to the In-SSD MM OS page caches127shown and described in connection withFIG. 1. Various combinations of such orientations, among others, of the In-SSD MM OS caches in the In-SSD MM OS cache resource224are contemplated and remain within the scope of the present disclosure.

Being configured to cache requested blocks and/or portions of the requested blocks of data in In-SSD MM OS caches225of various sizes and/or positions, e.g., as determined by controller220based upon the In-SSD MM OS caches selected by OS203, contributes to granularity of moving and/or performing operations on the requested data. For example, a single page of requested data, e.g., four kilobytes, may contain a plurality of requested blocks of data, which may, in some embodiments, be stored by a respective plurality of In-SSD MM OS caches, e.g.,225-0,225-1, . . . ,225-N−1. Each of the requested blocks of data may correspond to a size, e.g., in bits, of one or more respective In-SSD MM OS caches in which the requested block is to be stored. In some embodiments, data stored and/or cached in the In-SSD MM OS caches may be accessed for movement directly138to the CPU internal cache112at a granularity of, for example, a multiple of 64 bytes (512 bits), among other possibilities, which may correspond to a size of a CPU processor cache line. For example, although a page of data, e.g., stored by all memory cells in a row231of In-SSD MM OS cache resource224and which may correspond to thousands of bytes, may be accessed and/or moved238, the data accessed and/or moved238in some situations may have a much finer granularity, e.g., 64 bytes. In some embodiments, the OS203may utilize an In-SSD MM OS cache look-up table (not shown), e.g., a radix tree, interfaced with In-SSD MM OS cache resource224and/or NVM data storage resource226to expedite a look-up routine. In some embodiments, the interface of the look-up table may be via DIMM.

The requested data being moved to and/or cached by particular In-SSD MM OS caches225selected by the OS203may enable a read operation to be directed by the OS203on the data cached in the particular In-SSD MM OS caches225. Alternatively or in addition, the requested data being in the particular selected In-SSD MM OS caches225may enable a write operation to be directed by the OS203on the data cached in the particular In-SSD MM OS caches225. The requested data being in the particular selected In-SSD MM OS caches225may enable the requested data to be moved from the selected In-SSD MM OS caches225to enable performance of a number of operations directed by the OS203.

Moreover, in some embodiments, an operation, e.g., a mathematical operation, a logical operation, etc., may be performed by the SSD204, e.g., the controller220and/or logic circuitry123, to determine whether the requested data moved to and/or cached by the selected In-SSD MM OS caches225is to be moved to the host system202, e.g., for storage in a cache thereof, or whether the requested data is to remain in the selected In-SSD MM OS caches225. For example, a determination whether the requested data is to remain in the selected In-SSD MM OS caches225may be whether the requested data is to remain stored by the selected In-SSD MM OS caches225, e.g., and not to be moved to the host system202, or is to be erased, e.g., equilibrated, from the selected In-SSD MM OS caches225, regardless of whether the selected data is moved to the host system202.

In the path230illustrated inFIG. 2A, a request234for a block of data, e.g., based upon an initiated user application that uses the data for performance of an application operation, may be sent from the OS203, e.g., via device driver218, to open and read the block of data, e.g., a data file, stored by the NVM data storage resource226, e.g., a 3D XPoint array. The OS203may direct creation of a number of In-SSD MM OS caches225in the In-SSD MM OS cache resource224before, essentially simultaneously with, and/or after sending the request234for the block of data to the NVM data storage resource226. The OS203may direct movement236, e.g., copy, transfer, and/or transport operations, of requested blocks of data from the NVM data storage resource226into a number of selected In-SSD MM OS caches225, e.g., a page cache, for caching the requested data in response to a read and/or a write call, e.g., consistent with a particular cache attribute, as described herein. The selected In-SSD MM OS cache225may store and/or cache the requested block of data for access by and/or movement238to the host system202to enable performance of the operation. In various embodiments, sending the request234, creation of the In-SSD MM OS caches225in the In-SSD MM OS cache resource224, movement of the block of data from the NVM data storage resource226, and/or the access by and/or the movement238to the host system202may be controlled and/or directed by the device driver218and/or the controller220.

A routine performed by OS203may determine, e.g., by receiving a cache miss signal, that a requested block of data to enable performance of an operation is not stored by a readily accessible cache, e.g., a CPU internal cache112, a CPU external cache114, and/or In-host MM OS cache117of the host MM116. The routine may then determine, e.g., by receiving a cache hit signal, that the requested block of data is stored and/or cached in an In-SSD MM OS cache225of In-SSD MM OS cache resource224in SSD204, e.g., as a result of previously being moved236from NVM data storage resource226. Hence, OS203may directly access the block of data in and/or direct movement238of the block of data from the In-SSD MM OS cache225, e.g., for use by a number of processors of CPU110. In some embodiments, blocks of data may be sequentially accessed in and/or moved from an In-SSD MM OS cache225and/or a plurality of In-SSD MM OS caches, e.g.,225-0,225-1, . . . ,225-N−1.

Among differences between the present disclosure and transferring requested data to DIMM memory from an SSD storage device is, for example, that an initial destination of the requested block of data is, as described herein, an In-SSD MM OS cache225of SSD204configured as a portion of the host MM116. Unconditionally transferring the requested block of data to the In-host MM OS cache117of the host MM116via the DIMM that is not part of the SSD204, e.g., the DIMM being connected to a motherboard of a computing device, notably contrasts with moving the requested block of data initially to an In-SSD MM OS cache225that is a portion of the host MM116prior to selectably moving the requested block of data to at least one of the additional primary memory resources in the host system202. As described herein, the requested block of data may be moved directly138from the In-SSD MM OS cache125to the CPU internal cache112operating as the processor cache for CPU110. Alternatively or in addition, in various embodiments, the requested block of data may be moved from the In-SSD MM OS cache125to the CPU external cache(s)114and/or the In-host MM OS cache117of the host MM116, as described herein.

For example, there may be a shorter distance between NVM data storage resource226and the In-SSD MM OS cache225in In-SSD MM OS cache resource224, which may contribute to a shorter access latency and/or movement time, than movement directly105between the NVM data storage resource226and the In-host MM OS cache117of the host MM116that is addressable by the OS203, e.g., when connected by the DIMM. Accessing the requested data from an In-SSD MM OS cache225in In-SSD MM OS cache resource224of the SSD204may have a shorter intrinsic latency, e.g., 50 nanoseconds (ns), than accessing the requested data directly from the NVM data storage resource226, e.g., 100-500 ns. Latency of accessing the data directly from the NVM data storage resource226may be further affected by whether a read or a write operation is to be performed on the data. As such, accessing the requested data in the In-SSD MM OS cache225, e.g., for performance of a read operation or a write operation, may further contribute to reducing the intrinsic latency.

FIG. 2Bis a diagram illustrating another path240for a data movement operation in accordance with a number of embodiments of the present disclosure. The host system202, OS203, device driver218, SSD204, controller220, In-SSD MM OS cache resource224, In-SSD MM OS caches225-0,225-1, . . . ,225-5, and/or NVM data storage resource226components shown inFIG. 2Bmay correspond to the components inFIGS. 1 and 2Athat are similarly numbered. As such, each of these components may, in various embodiments, represent at least a portion of the functionality embodied by and contained in the corresponding components shown and described in connection withFIGS. 1 and 2Aand elsewhere herein.

The path240shown inFIG. 2Billustrates an example of a requested block of data not being previously stored and/or cached by an In-SSD MM OS cache225of the In-SSD MM OS cache resource224, e.g., as determined by the OS203and/or device driver218receiving a cache miss signal in response to a request234for the block of data. In response, in some embodiments, the OS203may initiate an I/O routine for the requested block of data with output241of an address of the requested block of data, e.g., block245, in the NVM data storage resource226and/or output of requests and/or instructions (commands) for operations performed on the requested block of data, as directed by the device driver218. Some embodiments may be configured for input248of the requested data to primary memory resources in the host system202, as described herein, e.g., host MM116addressable by the OS203, as directed by the device driver218.

Such commands can direct movement236of a number of blocks245of the requested data from the NVM data storage resource226to a number of selected In-SSD MM OS caches, e.g.,225-0,225-1, . . . ,225-N−1, in the In-SSD MM OS cache resource224of the SSD204. The device driver218may, in some embodiments, direct performance of data movement into, within, and/or from the SSD204via the controller220of the SSD204being selectably coupled to the In-SSD MM OS cache resource224and the NVM data storage resource226. For example, the controller may direct the movement236of the requested data from the NVM data storage resource226to the In-SSD MM OS cache resource224, e.g., rather than directly105to the In-host MM OS cache117of the host MM116addressable by the OS203, as may be done in other implementations.

Movement of the requested data to the selected In-SSD MM OS caches225in the In-SSD MM OS cache resource224may result in the I/O routine updating look-up table information to document a location of the moved data, e.g., in which of In-SSD MM OS caches225-0,225-1, . . . ,225-N−1 the moved data is cached. In various embodiments, as described herein, the requested data may remain in the documented location before, during, and/or after the requested data is moved to a cache of the host system202and/or regardless of whether the requested data is moved to the cache of the host system.

Accordingly, in various embodiments, the present disclosure can be implemented as follows. Note that operations indicated, for example, inFIGS. 2A and 2Bmay be performed sequentially and/or substantially simultaneously because performance of the respective illustrated paths230and240may be individually controlled via the controller220of the SSD204and/or the device driver218associated with the OS203. Accordingly, various operations may be performed in parallel via cooperation of hardware, firmware, and/or software. Among other advantages, such parallel performance of the operations may expedite performance of an associated path, e.g., paths230and240.

In various embodiments, an SSD204, as described herein, can include In-SSD MM OS cache resource224, NVM data storage resource226, and an interconnect106-3, e.g., as shown and described in connection with the interface (e.g., PCIe) circuitry108ofFIG. 1, that couples the NVM to the VM. The SSD also may include the controller220configured to receive a request234for performance of an operation and to direct that a result of the performance of the operation is accessible in the In-SSD MM OS cache resource224as an In-SSD MM OS cache225. The device driver218may send the request234to the controller220. The request234may be received by the controller220from the device driver218associated with the OS203. The request234may include a block address243for a block of data245stored by the NVM data storage resource226.

The request234may, in various embodiments, include a cache attribute. As described herein, a cache attribute is intended to refer to one of a number of caching options for storing a copy of data at least temporarily in a storage medium, e.g., for performance of operations by a processor. In various embodiments, the storage medium may be one or more In-SSD MM OS caches, e.g., as shown at125and225-0,225-1, . . . ,225-N−1 and described in connection withFIGS. 1 and 2A-2B, respectively, and the processor may be a CPU, e.g., as shown at110and described in connection withFIG. 1. The In-SSD MM OS caches may be more readily accessible by the processor than bulk storage of the data, for example, in NVM data storage resource226of the SSD204. Examples of the cache attributes may include write combining (WC), write protected (WP), write back (WB), write through (WT), write around (WA), strong uncacheable (UC), and/or uncacheable (UC−), among other options known in the art.

A cache attribute may be assigned to and/or included in a request234for a read operation. For example, the cache attribute may be set to WP cacheable, e.g., in the CPU internal caches112, the CPU external caches114, the In-host MM OS cache117of the host MM116. and/or in the In-SSD MM OS caches225of the In-SSD MM OS cache resource224. A cache attribute may be assigned to and/or included in a request234for a write operation. In some requests for a write operation, the cache attribute may be set to WC for use with, for example, a write combining buffer (not shown). In some requests for a write operation, the cache attribute may be set to WB for use with, for example, a write operation performed in the In-SSD MM OS caches225of the In-SSD MM OS cache resource224and a write back of the written data to the CPU internal caches112, the CPU external caches114, the In-host MM OS cache117of the host MM116, and/or to the NVM data storage resource226.

When a read operation is requested, the device driver218and/or controller220may determine that a read request has been issued and may relay a command for performance of the read operation based upon the associated cache attribute. In situations where a write operation is requested, the device driver218and/or controller220may determine that a write request has been issued and may relay a command for performance of the write operation based upon the associated cache attribute.

Accordingly, in response to a first assigned cache attribute in a request234, the controller220may be configured to direct that a read operation be performed on the data moved to the In-SSD MM OS cache225in the In-SSD MM OS cache resource224, e.g., DRAM. In response to a second assigned cache attribute in a request234, the controller220may be further configured to direct that a write operation be performed on the data moved to the In-SSD MM OS cache225in the In-SSD MM OS cache resource224, e.g., DRAM, where the first assigned cache attribute is different from the second assigned cache attribute.

Because data for a read operation retrieved from, for example, the NVM data storage resource226of the SSD104may be cached in an In-SSD MM OS cache225associated with the In-SSD MM OS cache resource224, a processing speed of the data may be improved. For example, the processing speed may approximate a speed of the data being provided to a processor of CPU110from host MM116, e.g., DRAM. Similar improvement in processing speed may be achieved for a write operation.

As described herein, the SSD204may, in some embodiments, include DRAM, e.g., a number of DRAM arrays, as the In-SSD MM OS cache resource224and a 3D XPoint memory as the NVM data storage resource226. The controller220may be configured to receive a request234for movement of data stored by the SSD204and to direct that the requested data is moved236to an In-SSD MM OS cache225in the DRAM. The moved data may be stored as a block of data in the In-SSD MM OS cache resource224, e.g., in one or more In-SSD MM OS caches225-0,225-1, . . . ,225-N−1, as described herein. For example, the In-SSD MM OS cache may be positioned in a single row and/or a single column of memory cells in the DRAM or, in some embodiments, be positioned across a plurality of rows and/or columns, e.g., diagonally.

In some embodiments, a PCIe bus may be utilized as a communication interface between OS203, device driver218, controller220, and the DRAM to form a PCIe circuit, e.g., as interface circuitry108including interconnects106-1,106-2, and106-3. The PCIe bus may further be utilized as a communication interface between the DRAM and the 3D XPoint memory, e.g., via interconnect106-4, to form the PCIe circuit.

The controller220may be further configured to direct that a requested block of data245be moved from the block address in the NVM data storage resource226to a selected In-SSD MM OS cache in the In-SSD MM OS cache resource224, e.g., selected by OS203from In-SSD MM OS caches225-0,225-1, . . . ,225-N−1. In some embodiments, performance of the operation can include movement of data stored by the NVM data storage resource226to the In-SSD MM OS cache resource224and storage of the data by the In-SSD MM OS cache225.

The controller220may be further configured to selectably move a result of the operation, e.g., a read and/or write operation, from the In-SSD MM OS cache225of the In-SSD MM OS cache resource224to the host system202. As described herein, to selectably move the result of the operation is intended to include to select between movement of the result for storage in a cache, e.g., In-host MM OS cache117, of the host system202and movement of the result to a cache internal to, e.g., CPU internal cache112, a processing resource, e.g., CPU110, of the host system102, without being stored by the internal cache, to enable performance of a particular operation directed by the OS203.

The controller220may be further configured to notify247the OS203, e.g., via device driver218, that a block of requested data is moved236from the NVM data storage resource226to the In-SSD MM OS cache225in the In-SSD MM OS cache resource224. The controller220may be further configured to notify247the OS203, e.g., via device driver218, that the performance of the operation is completed.

The controller220may be further configured to receive the request234for data stored by the SSD204, to determine from the request that the requested data is stored245at a block address in the NVM data storage resource226, e.g., the 3D XPoint memory, to direct movement236of the requested data from the block address in the 3D XPoint memory to an In-SSD MM OS cache225in the DRAM, and to direct storage of the requested data by the In-SSD MM OS cache225in the DRAM. The controller220may be further configured to receive a request234for data stored by the In-SSD MM OS cache225in the DRAM and to direct movement of the data from the In-SSD MM OS cache225to enable performance of a particular operation, as directed by the OS203that requested the data. The request234for the data stored by the In-SSD MM OS cache225in the DRAM may be received from the OS203and/or CPU110. The requested data may be moved238from the In-SSD MM OS cache225to the host system202, e.g., to a CPU internal cache112and/or a CPU external cache114to enable processing by the CPU110.

Prior to the request234being sent for movement of the requested data stored by the In-SSD MM OS cache225in the DRAM, the OS203may be configured to determine that the requested data is not stored by a cache of the host system202, e.g., by receiving a cache miss signal, and to determine that the requested data is stored by the In-SSD MM OS cache225in the DRAM, e.g., by receiving a cache hit signal. Alternatively or in addition, prior to movement of the requested data from the block address in the 3D XPoint memory, the controller may be configured to determine that the requested data is not presently stored by an In-SSD MM OS cache225. If the requested data is presently stored by the In-SSD MM OS cache225or the requested data is moved236to an In-SSD MM OS cache225, the controller220may be further configured to notify247the OS203that a block of requested data is accessible in the In-SSD MM OS cache225. In some embodiments, the controller220may be further configured, as described herein, to direct movement238of the requested data from the In-SSD MM OS cache225to the host system202to enable processing by execution of instructions stored by the OS203. For example, the requested data may be moved directly238to a processor cache, e.g., CPU internal cache112, of the host system202to enable performance of a particular operation and the requested data is not sent to and/or stored by another cache of the host system202, e.g., CPU external cache114and/or In-host MM OS cache117.

Accordingly, the controller220may be further configured to co-manage the In-SSD MM OS cache resource224, e.g., DRAM, among other types of VM, and the NVM data storage resource226, e.g., 3D XPoint memory, among other types of NVM, for data migration as directed by the OS203. In some embodiments, the controller220may include logic circuitry123, e.g., as shown in and described in connection withFIG. 1. The logic circuitry123may be configured to determine whether a request234, received from the OS203, for particular data stored by the SSD204meets a threshold, e.g., by a threshold comparison operation. The threshold may be applied to an operation attribute, which may be selected from a number of operation attributes. The operation attributes may include, for example, a number of requests, a frequency of requests, e.g., in a particular length of time, a size of a request, e.g., defined by a number of bits, a timing of a request, e.g., in a particular time frame, for the particular data, and/or a free memory capacity of a host system, e.g., a low free memory capacity may affect whether the data is retained in a particular In-SSD MM OS cache225rather than being moved to the host system202, among other possible operation attributes.

The controller220and/or the logic circuitry123may be further configured to determine, if the request does not meet, e.g., is lower than, a first threshold, that the requested data be moved from a particular In-SSD MM OS cache225in the In-SSD MM OS cache resource224to the host system202to enable performance, e.g., by the CPU110connected to the OS203, of the particular operation and that the requested data not be stored by a cache of the host system202, e.g., a number of the CPU internal caches112, CPU external caches114, and/or In-host MM OS cache117during and/or after performance of the operation. In some embodiments, the requested data may remain stored by the particular In-SSD MM OS cache225when the requested data is not stored by a cache of the host system202. Hence, a destination for storage of the requested data may be the particular In-SSD MM OS cache225.

If the request does meet, e.g., is equal to or higher than, the first threshold, the controller220and/or the logic circuitry123may be further configured to determine that the requested data be moved from the particular In-SSD MM OS cache225to be stored by a cache of the host system202, e.g., a number of the CPU internal caches112, CPU external caches114, and/or In-host MM OS cache117. In some embodiments, when the request does meet the first threshold, the requested data may not remain stored by the particular In-SSD MM OS cache225when the requested data is moved to and/or stored by a cache of the host system202. Hence, a destination for storage of the requested data may be a particular cache of the host system202.

Alternatively or in addition, the controller220and/or the logic circuitry123may be further configured to determine, if the request does not meet, e.g., is lower than, a second threshold, that the requested data be moved from a particular In-SSD MM OS cache in the In-SSD MM OS cache resource224to the host system202to enable performance, e.g., as directed by the OS203, of a particular operation and that the requested data not be stored thereafter by the particular In-SSD MM OS cache225. As such, in some embodiments, the requested data may not remain stored by the particular In-SSD MM OS cache225when the request does not meet the second threshold and the requested data is not stored by a cache of the host system202by not meeting the first threshold. Hence, the requested data may not be stored by either the particular In-SSD MM OS cache225or a cache of the host system202, although the requested data remains stored by the NVM data storage resource226.

If the request does meet, e.g., is equal to or higher than, the second threshold, the controller220and/or the logic circuitry123may be further configured to determine that the requested data be stored thereafter by the particular In-SSD MM OS cache225to enable access by the OS203of the host system202to the stored requested data. Hence, a destination for storage of the requested data may be the particular In-SSD MM OS cache225.

The second threshold, among any number of thresholds, may apply to an operation attribute that is the same as or is different from that of the first threshold. In some embodiments, the first threshold for a particular operation attribute may have a higher numerical value or other rating indicative of utilization of the requested data by the OS203than indicated by the second threshold. For example, if a number of requests for the requested data meets the first threshold, the requested data may be stored by a cache of the host system202, rather than remaining stored by the particular In-SSD MM OS cache225when the number does not meet the first threshold. Meeting a lower second threshold for the number of requests may be used to determine that the requested data remain stored by the particular In-SSD MM OS cache225, whereas not meeting second threshold for the number of requests may be used to determine that requested data is not stored by either the particular In-SSD MM OS cache225or the cache of the host system202. In some embodiments, a particular threshold may be determined for each type of request, e.g., for data movement, a read operation, and/or a write operation.

Embodiments described herein provide a method of operating a first apparatus that may be in the form of the computing system100including OS203, In-SSD MM OS cache resource224, In-SSD MM OS caches225, NVM data storage resource226, and interface circuitry108, among other possible components, e.g., as shown in and described in connection withFIGS. 1 and 2A-2B. In various embodiments, the computing system100may include DRAM operating as the In-SSD MM OS cache resource224, coupled to 3D XPoint memory as the NVM data storage resource226, operating as the secondary data storage resource, and the interface circuitry108, e.g., a PCIe bus, operating as the communication interface between the OS203, the DRAM In-SSD MM OS cache resource224, and the 3D) XPoint memory data storage resource226. The method can, as described herein, include utilizing the first apparatus for shortening a latency of data movement, for an operation performed utilizing the first apparatus, relative to a same type of operation performed by a second apparatus.

In various embodiments, the type of operation may be selected from a data movement operation, a read operation, and/or a write operation, among others. Shortening the latency relative to the operation performed by the second apparatus can include utilizing the second apparatus configured such that the DRAM is not interconnected to the 3D XPoint memory via the PCIe bus. Alternatively or in addition, shortening the latency can include utilizing the second apparatus configured to access data in the 3D XPoint memory via a memory bus for movement to a DIMM, e.g., in contrast to the first apparatus being configured for the OS203to access data in the DRAM via the PCIe bus.

In some embodiments, shortening the latency relative to the same type of operation performed by the second apparatus can include shortening a write operation performed on data stored by the DRAM In-SSD MM OS cache resource224. For example, the interface circuitry108described herein may be utilized to enable performance of a write operation on data stored by an In-SSD MM OS cache225, which may contribute to performance of the write operation, and to enable subsequent movement of the result to a cache of the host system202, e.g., a CPU internal cache112, a CPU external cache114, and/or In-host MM OS cache117. Such a WB operation may be performed in approximately half the time utilized by the second apparatus having the NVM data storage resource226, e.g., the 3D) XPoint memory, directly connected by the DIMM for transfer of an equivalent number of data bits to L1, L2, L3, and/or L4 caches and/or the In-host MM OS cache117.

The present disclosure may provide benefits compared to other approaches that use 3D XPoint memory. Utilization of the computing system100described herein may help overcome potential problems with “dirty” cache lines. For example, dirty cache lines, e.g., L1, L2, L3, L4, In-host MM OS cache117, and/or victim caches, may store data values that differ from corresponding data values stored by the 3D XPoint memory. In contrast, the In-SSD MM OS caches225described herein may cache data values retrieved directly from the 3D XPoint memory such that the data values stored by the In-SSD MM OS caches225are the same as the data values stored by the 3D XPoint memory.

Accordingly, the present disclosure describes a SSD204that provides both primary memory functions as an In-SSD MM OS cache resource224and secondary storage functions as an NVM data storage resource226. The SSD204configured as such, in combination with the interface circuitry108described herein, may expedite retrieval of blocks of data requested by the OS203, e.g., when a cache miss signal is received by OS203. The SSD204configured as such enables co-management, of the In-SSD MM OS cache resource224, e.g., DRAM, and the NVM data storage resource226, e.g., 3D XPoint memory. In various embodiments, the devices driver218and/or controller220described herein may contribute to the co-management by directing data migration requested by the OS203.