Patent Description:
Big data applications handle relatively large datasets. SSDs are widely used as a hardware feature in cloud infrastructure for big data services. SSDs are well suited for big data applications because they provide fast storage performance, and are efficient and cost-effective. Specifically, input/output (I/O) intensive operations can be accelerated by using an SSD architecture. The statements above are provided for illustrative purposes only and are not intended to constitute an admission of prior art.

<CIT> discloses apparatuses and methods related to a memory system with cache line data.

<CIT> discloses apparatus, systems, and methods that implement a less-recently-used data eviction mechanism for identifying a memory block of a cache for eviction.

Embodiments enable efficient access to an SSD hardware cache.

Hereinafter, embodiments of the present disclosure are described in detail with reference to the accompanying drawings. It should be noted that the same elements will be designated by the same reference numerals although they are shown in different drawings. In the following description, specific details such as detailed configurations and components are merely provided to assist with the overall understanding of the embodiments of the present disclosure. Therefore, it should be apparent to those skilled in the art that various changes and modifications of the embodiments described herein may be made without departing from the scope of the present disclosure. In addition, descriptions of well-known functions and constructions are omitted for clarity and conciseness. The terms described below are terms defined in consideration of the functions in the present disclosure, and may be different according to users, intentions of the users, or customs. Therefore, the definitions of the terms should be determined based on the contents throughout this specification.

The present disclosure may have various modifications and various embodiments, among which embodiments are described below in detail with reference to the accompanying drawings. However, it should be understood that the present disclosure is not limited to the embodiments, but includes all modifications, equivalents, and alternatives within the scope of the present disclosure.

The electronic device, according to one embodiment, may be one of various types of electronic devices utilizing storage devices. The electronic device may use any suitable storage standard, such as, for example, peripheral component interconnect express (PCIe), nonvolatile memory express (NVMe), NVMe-over-fabric (NVMeoF), advanced extensible interface (AXI), ultra path interconnect (UPI), ethernet, transmission control protocol/Internet protocol (TCP/IP), remote direct memory access (RDMA), RDMA over converged ethernet (ROCE), fibre channel (FC), infiniband (IB), serial advanced technology attachment (SATA), small computer systems interface (SCSI), serial attached SCSI (SAS), Internet wide-area RDMA protocol (iWARP), and/or the like, or any combination thereof. In some embodiments, an interconnect interface may be implemented with one or more memory semantic and/or memory coherent interfaces and/or protocols including one or more compute express link (CXL) protocols such as CXL. io, and/or CXL. cache, Gen-Z, coherent accelerator processor interface (CAPI), cache coherent interconnect for accelerators (CCIX), and/or the like, or any combination thereof. Any of the memory devices may be implemented with one or more of any type of memory device interface including double data rate (DDR), DDR2, DDR3, DDR4, DDR5, low-power DDR (LPDDRX), open memory interface (OMI), NVlink high bandwidth memory (HBM), HBM2, HBM3, and/or the like. The electronic devices may include, for example, a portable communication device (e.g., a smart phone), a computer, a portable multimedia device, a portable medical device, a camera, a wearable device, or a home appliance. However, an electronic device is not limited to those described above.

The terms used in the present disclosure are not intended to limit the present disclosure but are intended to include various changes, equivalents, or replacements for a corresponding embodiment. With regard to the descriptions of the accompanying drawings, similar reference numerals may be used to refer to similar or related elements. A singular form of a noun corresponding to an item may include one or more of the things, unless the relevant context clearly indicates otherwise. As used herein, terms such as "<NUM>st," "2nd," "first," and "second" may be used to distinguish a corresponding component from another component, but are not intended to limit the components in other aspects (e.g., importance or order). It is intended that if an element (e.g., a first element) is referred to, with or without the term "operatively" or "communicatively", as "coupled with," "coupled to," "connected with," or "connected to" another element (e.g., a second element), it indicates that the element may be coupled with the other element directly (e.g., wired), wirelessly, or via a third element.

As used herein, the term "module" may include a unit implemented in hardware, software, firmware, or combination thereof, and may interchangeably be used with other terms, for example, "logic," "logic block," "part," and "circuitry. " A module may be a single integral component, or a minimum unit or part thereof, adapted to perform one or more functions. For example, according to one embodiment, a module may be implemented in a form of an application-specific integrated circuit (ASIC), a co-processor, or field programmable gate arrays (FPGAs).

Machine learning (ML) use-cases, such as, for example, deep learning recommendation model (DLRM) interference and training, involve accessing data at a smaller granularity (e.g., <NUM> bytes versus <NUM> kilobytes) and follow a very skewed Pareto distribution. Latency-sensitive use cases require fast I/O accesses, but reading from a NAND flash memory can be relatively slow.

A standard cache replacement policy in an SSD may lead to pollution of the cache, because every read/write request is provided to the NAND flash memory. Accordingly many small write requests may be detrimental to SSD performance and endurance. As shown in <FIG>, the same number of requests received at an SSD controller are provided to the NAND flash memory.

<FIG> is a diagram illustrating a cache replacement policy. An SSD controller <NUM> of an SSD (or memory device) includes a host interface layer (HIL) <NUM>, which may include a host controller interfaced to a host side, and an SSD interface providing an abstracted application programming interface (API) to the host controller. Within the SSD controller <NUM>, the HIL <NUM> is in communication with a buffer manager <NUM>, a processor <NUM>, and a cache manager <NUM> within the SSD controller <NUM>. The buffer manager <NUM>, the processor <NUM>, and the cache manager <NUM> are also in communication with multiple flash cores <NUM>, which interface with pages of a NAND flash memory <NUM> of the SSD. The buffer manager <NUM> is communication with a write buffer <NUM> of the SSD. The cache manager <NUM> is in communication with a cache <NUM> (e.g., dynamic random access memory (DRAM)) of the SSD.

Every read or write request received from a host application, through the HIL <NUM> of the SSD controller <NUM>, is provided to the NAND flash memory <NUM> of the SSD. For example, when ten write requests received via the HIL <NUM> are provided to the buffer manager <NUM>, each of the ten write requests are provided to the NAND flash memory <NUM>, resulting in pollution of the cache.

Embodiments of the disclosure optimize the cache manager <NUM> such that requested data is read from or written to the SSD more efficiently via the cache <NUM>.

<FIG> is a diagram illustrating an SSD controller, according to an embodiment. An SSD controller <NUM> includes an HIL <NUM>, which is in communication with a processor <NUM> and a cache manager <NUM> within the SSD controller <NUM>. The processor <NUM> and the cache manager <NUM> are in communication with multiple flash cores <NUM>-<NUM> to <NUM>-n, which interface with pages of a NAND flash memory <NUM> of the SSD. The cache manager <NUM> includes a buffer manager <NUM>, a write buffer <NUM>, a data preloader <NUM>, and a list of dirty cache blocks <NUM>. The cache manager <NUM> is in communication with a cache <NUM> of the SSD.

When the SSD controller <NUM> is in a read-only cache mode, the cache is preloaded with data while the SSD is offline. Accordingly, a cache eviction and/or update is not required to be performed during runtime, resulting in less cache pollution. The read-only cache mode is described in greater detail below with respect to <FIG> and <FIG>.

Further, when the SSD controller <NUM> is in a write mode, write requests to the NAND flash memory <NUM> are minimized by writing data to the cache <NUM> at a small granularity (e.g., <NUM> bytes). These data of each write requests to the cache <NUM> has a smaller size than that of a page of the NAND flash memory <NUM>. For example, ten write requests received from a host application, via the HIL <NUM>, are provided to the cache manager <NUM>. Data is written to the cache <NUM> in accordance with the ten write requests. A single write request for multiple cache blocks corresponding to a single page of the NAND flash memory <NUM> may be provided from the cache manager <NUM> to the NAND flash memory <NUM>. The write mode of the SSD controller <NUM> is described in greater detail below with respect to <FIG>.

<FIG> is a diagram illustrating an SSD controller in a read-only cache mode, according to an embodiment. In the read-only cache mode, the data preloader <NUM> of the cache manager <NUM> may be user-programmed and heuristics-based in order to preload the cache <NUM> offline at regular intervals. Faster cache access is enabled when there are no cache evictions and/or updates at runtime. For a given cache size, the preloading of the cache also results in less cache pollution and a higher cache hit rate than a similarly-sized least recently used (LRU) cache or least frequently used (LFU) cache.

For example, a read request may be sent from a host application, to the SSD controller <NUM>. The cache manager <NUM> determines whether data corresponding to the read request is in the preloaded cache <NUM>. If the data is in the cache <NUM>, the cache manager <NUM> retrieves the data from the preloaded cache <NUM>, and the requested data is provided to the host application. When the data is not in the preloaded cache <NUM>, the data is read from the NAND flash memory <NUM>, stored in the cache <NUM>, and then provided to the host application. Accordingly, preloading data to the cache <NUM> offline may prevent latency caused by NAND flash memory data retrieval during runtime.

<FIG> is a chart illustrating preloading of data offline, according to an embodiment. Normalized cumulative accesses are plotted versus cumulative accessed vectors. The normalized cumulative accesses are shown to increase during the first <NUM> cumulative accessed vectors, before leveling off, illustrating the offline preloading of the cache at <NUM>.

<FIG> is a diagram illustrating an SSD controller in a write mode, according to an embodiment. The SSD controller <NUM> minimizes the number of write requests provided to the NAND flash memory <NUM>. For example, N write requests may be provided from a host application to the HIL <NUM> and the processor <NUM> of the SSD controller <NUM>. The cache manager <NUM> writes data corresponding to these write requests only to the cache <NUM>. A dirty bit is set to <NUM> for each block of the cache <NUM> having this newly written data. The list of dirty cache blocks <NUM> in the cache manager <NUM> maintains a list of dirty blocks in the cache <NUM> for each page of the NAND flash memory <NUM>.

Upon filling the cache <NUM> with data corresponding to the N write requests, and receiving a subsequent write request (N+<NUM>), data from a cache block must be evicted in order to store data of write request N+<NUM>. For example, data from cache block <NUM> may be chosen for eviction from the cache <NUM> to the NAND flash memory <NUM>. The cache manager <NUM> refers to the list of dirty cache blocks <NUM>, and determines other dirty cache blocks that belong to a same page of the NAND flash memory <NUM> as cache block <NUM>. For example, as shown in <FIG>, blocks <NUM>, <NUM>, N-<NUM>, and N belong to page <NUM>. Data from all dirty blocks belonging to a same page (e.g., page <NUM>) as block <NUM> is written to the NAND flash memory <NUM> with data from block <NUM> as a single write request, upon eviction of the data from block <NUM>. The cache manager <NUM> sends the single write request to update page <NUM> of the NAND flash memory <NUM>.

A dirty bit for all blocks of page <NUM> is set to <NUM> in the cache <NUM> to indicate that the respective data is in the NAND flash memory <NUM>. For example, as shown in cache details <NUM> of <FIG>, when the blocks of the cache <NUM> are filled with data from write requests <NUM> through N, each block is assigned a valid bit of <NUM> and a dirty bit of <NUM>. After evicting data from block <NUM> and writing data from all dirty blocks of page <NUM> to the NAND flash memory <NUM>, block <NUM> is assigned a valid bit of <NUM>, and blocks <NUM>, N-<NUM>, and N are assigned a dirty bit of <NUM>, as shown in updated cache details <NUM> of <FIG>. Accordingly, the cache <NUM> denotes that data from block <NUM> has been evicted and data from blocks <NUM>, N-<NUM>, and N has been written to the NAND flash memory <NUM> of the SSD.

Accordingly, in writing data corresponding to an evicted block and all dirty blocks of a same page, fewer backend writes are required to the NAND flash memory <NUM>. While these async writes minimize the number of writes to the NAND flash memory <NUM>, they may also result in potential inconsistencies. Accordingly, the SSD controller <NUM> may selectively perform a first mode for async writes and a second mode for sync writes in which every write request is provided to the NAND flash memory.

<FIG> is a diagram illustrating a method for efficiently accessing an SSD hardware cache, according to an embodiment. At <NUM>, a request is received from a host application at an SSD controller. The request may be a read request or a write request. When the request is a read request, data is read from the cache based on the read request, at <NUM>. The cache may be preloaded with data at predefined intervals while the SSD is offline. At <NUM>, the data is provided from the cache to the host application.

When the request is a write request, cache data is evicted from a block of the cache, when the cache is full, at <NUM>. At <NUM>, the cache data and corresponding data is written from the cache to the NAND flash memory as a single write request. A cache manager may maintain a list of dirty blocks of the cache per page of the NAND flash memory. The corresponding data may be from one or more blocks of a same flash memory page as the block from which the cache data is evicted. Specifically, the one or more blocks may be dirty blocks having a dirty bit set to <NUM> in the cache. After writing the cache data and the corresponding data to the NAND flash memory, the dirty bit of the one or more blocks may be set to <NUM> in the cache. At <NUM>, data is written to the block of the cache based on the received write request.

The cache type may be set associate or direct mapped, and the cache replacement policies may include LRU, LFU, etc..

<FIG> illustrates a block diagram of an electronic device <NUM> in a network environment <NUM>, according to one embodiment. The electronic device <NUM> may communicate with the electronic device <NUM> via the server <NUM>. The electronic device <NUM> may include a processor <NUM>, a memory <NUM>, an input device <NUM>, a sound output device <NUM>, a display device <NUM>, an audio module <NUM>, a sensor module <NUM>, an interface <NUM>, a haptic module <NUM>, a camera module <NUM>, a power management module <NUM>, a battery <NUM>, a communication module <NUM>, a subscriber identification module (SIM) <NUM>, or an antenna module <NUM>. In one embodiment, at least one (e.g., the display device <NUM> or the camera module <NUM>) of the components may be omitted from the electronic device <NUM>, or one or more other components may be added to the electronic device <NUM>. In one embodiment, some of the components may be implemented as a single integrated circuit (IC). For example, the sensor module <NUM> (e.g., a fingerprint sensor, an iris sensor, or an illuminance sensor) may be embedded in the display device <NUM> (e.g., a display).

The processor <NUM> may execute, for example, software (e.g., a program <NUM>) to control at least one other component (e.g., a hardware or a software component) of the electronic device <NUM> coupled with the processor <NUM>, and may perform various data processing or computations. The processor may correspond to the HCPU, or a combination of the HCPU, embedded CPUs, and/or NAND CPUs of the SSD. As at least part of the data processing or computations, the processor <NUM> may load a command or data received from a host or another component (e.g., the sensor module <NUM> or the communication module <NUM>) in volatile memory <NUM>, process the command or the data stored in the volatile memory <NUM>, and store resulting data in non-volatile memory <NUM>. The processor <NUM> may include a main processor <NUM> (e.g., a CPU or an application processor (AP)), and an auxiliary processor <NUM> (e.g., a graphics processing unit (GPU), an image signal processor (ISP), a sensor hub processor, or a communication processor (CP)) that is operable independently from, or in conjunction with, the main processor <NUM>. Additionally or alternatively, the auxiliary processor <NUM> may be adapted to consume less power than the main processor <NUM>, or execute a particular function. The auxiliary processor <NUM> may be implemented as being separate from, or a part of, the main processor <NUM>.

The auxiliary processor <NUM> may control at least some of the functions or states related to at least one component (e.g., the display device <NUM>, the sensor module <NUM>, or the communication module <NUM>) among the components of the electronic device <NUM>, instead of the main processor <NUM> while the main processor <NUM> is in an inactive (e.g., sleep) state, or together with the main processor <NUM> while the main processor <NUM> is in an active state (e.g., executing an application). According to one embodiment, the auxiliary processor <NUM> (e.g., an image signal processor or a communication processor) may be implemented as part of another component (e.g., the camera module <NUM> or the communication module <NUM>) functionally related to the auxiliary processor <NUM>.

The communication module <NUM> may include one or more communication processors that are operable independently from the processor <NUM> (e.g., the AP) and supports a direct (e.g., wired) communication or a wireless communication. According to one embodiment, the communication module <NUM> may include a wireless communication module <NUM> (e.g., a cellular communication module, a short-range wireless communication module, or a global navigation satellite system (GNSS) communication module) or a wired communication module <NUM> (e.g., a local area network (LAN) communication module or a power line communication (PLC) module). A corresponding one of these communication modules may communicate with the external electronic device via the first network <NUM> (e.g., a short-range communication network, such as Bluetooth™, wireless-fidelity (Wi-Fi) direct, or a standard of the Infrared Data Association (IrDA)) or the second network <NUM> (e.g., a long-range communication network, such as a cellular network, the Internet, or a computer network (e.g., LAN or wide area network (WAN)). These various types of communication modules may be implemented as a single component (e.g., a single IC), or may be implemented as multiple components (e.g., multiple ICs) that are separate from each other.

According to one embodiment, commands or data may be transmitted or received between the electronic device <NUM> and the external electronic device <NUM> via the server <NUM> coupled with the second network <NUM>.

According to one embodiment, each component (e.g., a module or a program) of the above-described components may include a single entity or multiple entities. One or more of the above-described components may be omitted, or one or more other components may be added. In this case, the integrated component may still perform one or more functions of each of the plurality of components in the same or similar manner as they are performed by a corresponding one of the plurality of components before the integration. Operations performed by the module, the program, or another component may be carried out sequentially, in parallel, repeatedly, or heuristically, or one or more of the operations may be executed in a different order or omitted, or one or more other operations may be added.

Claim 1:
A method, comprising:
receiving a write request from a host device at a memory device;
modifying a block of a cache (<NUM>, <NUM>) of the memory device to remove cache data;
writing, to a flash memory (<NUM>, <NUM>) of the memory device, the cache data and corresponding data from the cache (<NUM>, <NUM>) as a single write request; characterised by
writing first data to the block of the cache (<NUM>, <NUM>) based on the received write request;
loading the cache (<NUM>, <NUM>) with data with the memory device in a first state;
receiving a read request from the host device with the memory device in a second state;
reading second data from the cache (<NUM>, <NUM>) based on the read request; and
outputting the second data to the host device.