META MODEL EXTENSION FOR A MACHINE LEARNING EQUALIZER

Systems and methods of the present disclosure may be used to improve equalization module architectures for NAND cell read information. For example, embodiments of the present disclosure may provide for de-noising of NAND cell read information using a Multiple Shallow Threshold expert Machine Learning Models (MTM) equalizer. An MTM equalizer may include multiple shallow machine learning models. A meta network may generate parameters for each of the shallow machine learning models such that each shallow machine learning model may be able to solve a classification task (e.g., a binary classification task) corresponding to a weak decision range between two possible read information values for a given NAND cell read operation. Accordingly, during inference, each read sample with a read value within a weak decision range may be passed through a corresponding shallow machine learning model (e.g., a corresponding threshold expert) that is associated with the particular weak decision range.

BACKGROUND

The following relates generally to flash memory de-noising, and more specifically to flash memory de-noising using machine learning models.

Memory devices are commonly used electronic components for storing data. NAND flash memory devices allow several bits of data to be stored in each memory cell, providing improvements in manufacturing costs and performance. A memory cell in which multiple bits of data are stored may be referred to as a multi-level memory cell. A multi-level memory cell partitions a threshold voltage range of a memory cell into several voltage states, and data values written to the memory cell are extracted using the memory cell voltage levels.

However, storing multiple bits per memory cell may decrease the dynamic voltage range of each voltage state, making the memory cells more susceptible to noise. Compensating for this noise may require increased computational power, which may hinder performance in a mobile device. Therefore, there is a need in the art for reliable, low power multi-cell memory systems for use in mobile electronic devices.

SUMMARY

A method for reading data from a memory device is described. An embodiment of the method may read a block of data from a memory device (e.g., a NAND cell), process the block of data using an equalizer with multiple shallow threshold expert machine learning models to classify each bit of the block of data with a log likelihood ratio (LLR) value, and decode the block of data based on an error correction code (ECC) coding scheme. A machine learning, meta model may be used to generate parameters for each threshold expert machine learning model, and a threshold expert machine learning model may classify bits based on the parameters generated by the meta model.

A method, apparatus, non-transitory computer readable medium, and system for flash memory de-noising using machine learning models are described. One or more embodiments of the method, apparatus, non-transitory computer readable medium, and system is configured to receive first information for a plurality of memory cells of a memory device and second information for a memory cell of the plurality of memory cells, generate one or more threshold expert parameters for a threshold expert based on the first information, select the threshold expert from a plurality of threshold experts based on the second information, wherein each of the threshold experts correspond to at least one weak decision voltage range from a plurality of weak decision voltage ranges of the memory cell, and generate a memory channel output based on the second information using the selected threshold expert and the one or more threshold expert parameters.

An apparatus, system, and method for flash memory de-noising using machine learning models are described. Embodiments of the apparatus, system, and method include a memory device including a plurality of memory cells arranged in a plurality of wordlines, a meta network configured to generate one or more threshold expert parameters for a threshold expert based on first information for a wordline of the plurality of wordlines, and a plurality of threshold experts configured to generate a memory channel output based on second information from a memory cell in the wordline and the one or more threshold expert parameters.

A non-transitory computer-readable medium, apparatus, system, and method for flash memory de-noising using machine learning models are described. One or more embodiments of the method, apparatus, non-transitory computer readable medium, and system is configured to receive first information for a plurality of memory cells of a memory device and second information for a memory cell of the plurality of memory cells, generate one or more threshold expert parameters for a threshold expert based on the first information, select the threshold expert from a plurality of threshold experts based on the second information, wherein each of the threshold experts correspond to at least one weak decision voltage range from a plurality of weak decision voltage ranges of the memory cell, and generate a memory channel output based on the second information using the selected threshold expert and the one or more threshold expert parameters.

DETAILED DESCRIPTION

The present disclosure relates to systems and methods for reading data from a memory device, and more specifically to de-noising data read from a memory device. Certain embodiments of the disclosure relate to generating parameters for multiple threshold expert machine learning models and de-noising NAND cell read information using an equalizer with the multiple threshold expert machine learning models.

Memory devices are commonly used electronic components for storing data. NAND flash memory devices allow several bits of data to be stored in each memory cell, providing improvements in manufacturing costs and performance. A memory cell in which multiple bits of data are stored may be referred to as a multi-level memory cell. A multi-level memory cell partitions a threshold voltage range of a memory cell into several voltage states, and data values written to the memory cell are extracted using the memory cell voltage levels.

To read the information from a memory device, a voltage of each cell is measured and the voltage level that was stored in the cell is inferred. The bits can then be recovered. There may be a tradeoff between the number of voltage levels and the memory reliability. The larger the number of bits per cell, the more information that can be stored on the device, where there are more bits in every cell. Alternatively, the voltages that represent different levels may be packed more tightly together, because within the same dynamic range, a larger number of distinguishable voltages is used. As a result, noise in the cell programming or cell reading has a larger chance of changing the voltage of a level to another voltage representing a different level, therefore rendering an error upon reading the cell. In other words, when information is read from a cell (e.g., a NAND cell), the read information may be corrupted with noise (e.g., depending on the characteristics of the medium), which may result in an ambiguous value of the read information (e.g., the read voltage level may be within some weak decision voltage range between two voltage values where voltage information for different values of the detected channel output are likely to overlap). In some examples, such noise may cause error correction code (ECC) operations to fail.

Conventional approaches for handling ECC operation failures may include utilization of an equalizer module in an effort to reduce inter-symbol-interference (ISI) across the NAND cells. Such equalization modules may receive sample (memory cell) features (e.g., cell voltage, pillar and wordline neighbors, thresholds, etc.) as input, and equalization modules may output improved soft symbol estimations (e.g., that may allow for more efficient ECC operations on the improved soft symbol estimations). However, equalization modules may pass samples through a same model (e.g., such as a neural network) for soft symbol estimations across different voltage levels. Such approaches may demand relatively sophisticated (e.g., deep, computationally intensive, etc.) machine-learning models with high latency and memory requirements, which may not provide optimal performance. Further, due to high latency/memory requirements, only limited NAND architectures may be able to support such equalization modules.

Systems and methods of the present disclosure may be used to improve equalization module architectures for NAND cell read information. For example, embodiments of the present disclosure may provide for de-noising NAND cell read information using a Multiple Shallow Threshold expert Machine Learning Models (MTM) equalizer. An MTM equalizer may include multiple shallow machine learning models, where each machine learning model is used to specifically solve a classification task (e.g., a binary classification task) corresponding to a weak decision range between two possible read information values for a given NAND cell read operation. Accordingly, during inference, each read sample with a read value within a weak decision range is passed through a corresponding shallow machine learning model (e.g., a corresponding threshold expert) that is associated with the particular weak decision range. The plurality of threshold expert shallow machine learning models may thus output improved soft symbol estimations for their respective weak decision ranges.

A meta network may be trained to generate parameters for each threshold expert of an MTM equalizer, and a threshold expert may classify bits based on parameters generated by the meta network for the threshold expert. The parameters generated for a threshold expert may allow the threshold expert to handle classification tasks associated with a specific weak decision range. Improved soft symbols estimations may be used to generate improved codewords for subsequent ECC operations (e.g., which may improve EEC operation efficiency). Utilization of multiple shallow machine learning models, each including parameters that allow the machine learning model to solve a specific binary classification task associated with a unique weak decision range, may also result in a set of machine learning models that can be operated with low memory usage (e.g., less memory usage than ECC) and low latency. In addition, such equalization models can be implemented in firmware, such that the equalizer models can be used by various NAND flash memory architectures. Further, because a meta network may be trained to generate parameters for a threshold expert that can classify bits in any of multiple weak decision ranges, a machine learning model with multiple threshold experts may be quickly configured to work with memory cells with a variety of weak decision ranges (e.g., without having to train each threshold expert independently).

Embodiments of the present inventive concept will be described more fully hereinafter with reference to the accompanying drawings. Like reference numerals may refer to like elements throughout the accompanying drawings.

It will be understood that the terms “first,” “second,” “third,” etc. are used herein to distinguish one element from another, and the elements are not limited by these terms. Thus, a “first” element in an exemplary embodiment may be described as a “second” element in another exemplary embodiment. It should be understood that descriptions of features or aspects within each exemplary embodiment should typically be considered as available for other similar features or aspects in other exemplary embodiments, unless the context clearly indicates otherwise. As used herein, the singular forms “a”, “an” and “the” are intended to include the plural forms as well, unless the context clearly indicates otherwise.

Herein, when one value is described as being about equal to another value or being substantially the same as or equal to another value, it is to be understood that the values are within a measurement error of each other, or if measurably unequal, are close enough in value to be functionally equal to each other as would be understood by a person having ordinary skill in the art. For example, the term “about” as used herein is inclusive of the stated value and means within an acceptable range of deviation for the particular value as determined by one of ordinary skill in the art, considering the measurement in question and the error associated with measurement of the particular quantity (i.e., the limitations of the measurement system). For example, “about” may mean within one or more standard deviations as understood by one of the ordinary skill in the art. Further, it is to be understood that while parameters may be described herein as having “about” a certain value, according to exemplary embodiments, the parameter may be exactly the certain value or approximately the certain value within a measurement error as would be understood by a person having ordinary skill in the art.

Example Memory Device

FIGS.1-5show an example memory system according to aspects of the present disclosure. In memory systems (e.g., such as solid-state drive (SSD) systems), memory controllers are connected to several NAND channels in parallel to achieve high data throughput. Memory controllers include signal processing and ECC engines that decode the data from the NAND and retrieve the stored data reliably. As described herein, the example memory system ofFIGS.1-5may include a memory controller with a plurality of NAND de-noising threshold experts (e.g., a memory controller with an equalizer including multiple shallow threshold expert machine learning models).

FIG.1is a block diagram illustrating an implementation of a data processing system including a memory system, according to an embodiment of the inventive concept.

Referring toFIG.1, the data processing system10may include a host100and a memory system200. The memory system200shown inFIG.1may be utilized in various systems that include a data processing function. The various systems may be various devices including, for example, mobile devices, such as a smartphone or a tablet computer. However, the various devices are not limited thereto.

The memory system200may include various types of memory devices. Herein, embodiments of the inventive concept will be described as including a memory device that is a non-volatile memory. However, embodiments are not limited thereto. For example, the memory system200may include a memory device that is a volatile memory.

According to embodiments, the memory system200may include a non-volatile memory device such as, for example, a read-only memory (ROM), a magnetic disk, an optical disk, a flash memory, etc. The flash memory may be a memory that stores data according to a change in a threshold voltage of a metal-oxide-semiconductor field-effect transistor (MOSFET), and may include, for example, NAND and NOR flash memories. The memory system200may be implemented using a memory card including a non-volatile memory device such as, for example, an embedded multimedia card (eMMC), a secure digital (SD) card, a micro SD card, or a universal flash storage (UFS), or the memory system200may be implemented using, for example, an SSD including a non-volatile memory device. Herein, the configuration and operation of the memory system200will be described assuming that the memory system200is a non-volatile memory system. However, the memory system200is not limited thereto. The host100may include, for example, a system-on-chip (SoC) application processor (AP) mounted on, for example, a mobile device, or a central processing unit (CPU) included in a computer system.

A processor is an intelligent hardware device, (e.g., a general-purpose processing component, a digital signal processor (DSP), a CPU, a graphics processing unit (GPU), a microcontroller, an application specific integrated circuit (ASIC), a field programmable gate array (FPGA), a programmable logic device, a discrete gate or transistor logic component, a discrete hardware component, or any combination thereof). In some cases, the processor is configured to operate a memory array using a memory controller. In other cases, a memory controller is integrated into the processor. In some cases, the processor is configured to execute computer-readable instructions stored in a memory to perform various functions. In some embodiments, a processor includes special purpose components for modem processing, baseband processing, digital signal processing, or transmission processing.

As described above, the host100may include an AP110. The AP110may include various intellectual property (IP) blocks. For example, the AP110may include a memory device driver111that controls the memory system200. The host100may communicate with the memory system200to transmit a command related to a memory operation and receive a confirm command in response to the transmitted command. The host100may also communicate with the memory system200with regard to an information table related to the memory operation.

The memory system200may include, for example, a memory controller210and a memory device220. The memory controller210may receive a command related to a memory operation from the host100, generate an internal command and an internal clock signal using the received command, and provide the internal command and the internal clock signal to the memory device220. The memory device220may store write data in a memory cell array in response to the internal command, or may provide read data to the memory controller210in response to the internal command. In some examples, a memory device220may be referred to as a channel. The term channel is used because a write and/or send operation can go to and/or through the channel. In some cases, when the information is read, the information may be corrupted with noise, depending on the characteristics of the medium.

As described herein, a memory controller210may include multiple threshold experts (e.g., a memory controller210may de-noise memory device220(e.g., NAND) read information using an equalizer including multiple shallow threshold expert machine learning models). For instance, memory controller210may include a MTM equalizer that includes a plurality of threshold experts, where each threshold expert is trained to specifically solve a NAND read value classification task. In some examples, a meta network may be used to generate parameters for each of the plurality of threshold experts in the memory controller210. Memory controller210is an example of, or includes aspects of, the corresponding element described herein (e.g., with reference toFIG.8).

The memory device220includes a memory cell array that retains data stored therein, even when the memory device220is not powered on. The memory cell array may include as memory cells, for example, a NAND or NOR flash memory, a magneto-resistive random-access memory (MRAM), a resistive random-access memory (RRAM), a ferroelectric access-memory (FRAM), or a phase change memory (PCM). For example, when the memory cell array includes a NAND flash memory, the memory cell array may include a plurality of blocks and a plurality of pages. Data may be programmed and read in units of pages, and data may be erased in units of blocks. An example of memory blocks included in a memory cell array is shown inFIG.4.

FIG.2is a block diagram illustrating the memory system200ofFIG.1, according to an embodiment of the inventive concept.

Referring toFIG.2, the memory system200includes the memory device220and the memory controller210. The memory controller210may also be referred to herein as a controller circuit. The memory device220may perform a write operation, a read operation, or an erase operation under control of the memory controller210.

The memory controller210may control the memory device220depending on a request received from the host100or an internally designated schedule. The memory controller210may include a controller core211, an internal memory214, a host interface block215, and a memory interface block216. The memory controller210may also include a device information storage217configured to provide first device information DI1to the host interface block215and second device information DI2to the controller core211.

In some embodiments, controller core211may include encoder212and equalizer213. In some embodiments, equalizer213may be an example of, may include aspects of, or may implement neural network600, plurality of threshold experts810, plurality of threshold experts905, and plurality of threshold experts1120. As described herein, encoder212may perform an ECC operation on read information (e.g., cell voltage, same-pillar neighbor voltage, etc.). In cases where an ECC operation fails, the read information may be passed to equalizer213, or new read information may be read from memory device220by the equalizer213.

In accordance with the techniques described herein, the equalizer213may, using a plurality of shallow threshold expert machine learning networks, generate an improved memory channel output from the read information using a threshold expert selected for the read information (e.g., a threshold expert trained for a classification operation in a weak decision range corresponding to the read information). That is, an initial memory channel output may be generated, and if an ECC operation fails, the equalizer213may generate an improved memory channel output that represents a better estimation of the originally stored information in one or more memory cells. Accordingly, the controller core211may generate an updated codeword and perform a successful ECC operation based on the estimated channel output (e.g., based on soft symbol estimations from the one or more threshold experts of the equalizer213).

The controller core211may include a memory control core that may control and access the memory device220depending on a request received from the host100or an internally designated schedule. The memory control core may manage and execute various metadata and codes used to manage or operate the memory system200. In some embodiments, the controller core211may include a memory control core and a machine learning core, and each of these cores may be implemented by one or more processors. In some examples where memory controller210includes a machine learning core, the machine learning core may be used to perform training and inference of threshold expert networks (e.g., shallow threshold expert machine learning networks) described with reference toFIGS.6and8-11. For instance, such threshold expert networks may be designed and trained to perform equalization or de-noising operations on read information read from memory device220.

The internal memory214may be used, for example, as a system memory which is used by the controller core211, a cache memory which stores data of the memory device220, or a buffer memory which temporarily stores data between the host100and the memory device220. The internal memory214may store a mapping table (MT) that indicates a relationship between logical addresses assigned to the memory system200and physical addresses of the memory device220. The internal memory214may include, for example, a DRAM or an SRAM.

The host interface block215may include a component for communicating with the host100such as, for example, a physical block. The memory interface block216may include a component for communicating with the memory device220such as, for example, a physical block.

Below, an operation of the memory system200over time will be described. When power is supplied to the memory system200, the memory system200may perform initialization with the host100.

The host interface block215may provide the memory control core with a first request REQ1received from the host100. The first request REQ1may include a command (e.g., a read command or a write command) and a logical address. The memory control core may translate the first request REQ1to a second request REQ2suitable for the memory device220.

For example, the memory control core may translate a format of the command. The memory control core may obtain address information with reference to the mapping table MT stored in the internal memory214. The memory control core may translate a logical address to a physical address of the memory device220by using the address information. The memory control core may provide the second request REQ2suitable for the memory device220to the memory interface block216.

The memory interface block216may register the second request REQ2from the memory control core at a queue. The memory interface block216may transmit a request that is first registered at the queue to the memory device220as a third request REQ3.

When the first request REQ1is a write request, the host interface block215may write data received from the host100to the internal memory214. When the third request REQ3is a write request, the memory interface block216may transmit data stored in the internal memory214to the memory device220.

When data is completely written, the memory device220may transmit a third response RESP3to the memory interface block216. In response to the third response RESP3, the memory interface block216may provide the memory control core with a second response RESP2indicating that the data is completely written.

After the data is stored in the internal memory214or after the second response RESP2is received, the memory control core may transmit a first response RESP1indicating that the request is completed to the host100through the host interface block215.

When the first request REQ1is a read request, the read request may be transmitted to the memory device220through the second request REQ2and the third request REQ3. The memory interface block216may store data received from the memory device220in the internal memory214. When data is completely transmitted, the memory device220may transmit the third response RESP3to the memory interface block216.

As the third response RESP3is received, the memory interface block216may provide the memory control core with the second response RESP2indicating that the data is completely stored. As the second response RESP2is received, the memory control core may transmit the first response RESP1to the host100through the host interface block215.

The host interface block215may transmit data stored in the internal memory214to the host100. In an embodiment, in the case in which data corresponding to the first request REQ1is stored in the internal memory214, the transmission of the second request REQ2and the third request REQ3may be omitted.

The memory device220may also transmit first Serial Peripheral Interface information SPI1to the memory interface block216. The memory interface block216may transmit second Serial Peripheral Interface information SPI2to the controller core211.

FIG.3is a detailed block diagram of the non-volatile memory device220ofFIG.1, according to an embodiment of the inventive concept. Referring toFIG.3, the memory device220may include, for example, a memory cell array221, a control logic222, a voltage generation unit223, a row decoder224, and a page buffer225.

The memory cell array221may be connected to one or more string select lines (SSLs), a plurality of wordlines WLs, one or more ground select lines (GSLs), and a plurality of bit lines BLs. The memory cell array221may include a plurality of memory cells disposed at intersections between the plurality of wordlines and the plurality of bit lines.

The control logic222may receive a command CMD (e.g., an internal command) and an address ADD from the memory controller210and receive a control signal CTRL for controlling various functional blocks within the memory device220from the memory controller210. The control logic222may output various control signals for writing data to the memory cell array221or reading data from the memory cell array221, based on the command CMD, the address ADD, and the control signal CTRL. In this manner, the control logic222may control the overall operation of the memory device220.

The various control signals output by the control logic222may be provided to the voltage generation unit223, the row decoder224, and the page buffer225. For example, the control logic222may provide the voltage generation unit223with a voltage control signal CTRL_vol, provide the row decoder224with a row address X-ADD, and provide the page buffer225with a column address Y-ADD.

The voltage generation unit223may generate various voltages for performing program, read, and erase operations on the memory cell array221based on the voltage control signal CTRL_vol. For example, the voltage generation unit223may generate a first driving voltage VWL for driving the plurality of wordlines, a second driving voltage VSSL for driving the plurality of string select lines SSLs, and a third driving voltage VGSL for driving the plurality of ground select lines GSLs. In this case, the first driving voltage VWL may be a program voltage (e.g., a write voltage), a read voltage, an erase voltage, a pass voltage, or a program verify voltage. In addition, the second driving voltage VSSL may be a string select voltage (e.g., an on voltage or an off voltage). Further, the third driving voltage VGSL may be a ground select voltage (e.g., an on voltage or an off voltage).

The row decoder224may be connected to the memory cell array221through the plurality of wordlines, and may activate a part of the plurality of wordlines in response to the row address X-ADD received from the control logic222. For example, in a read operation, the row decoder224may apply a read voltage to a selected wordline and a pass voltage to unselected wordlines.

In a program operation, the row decoder224may apply a program voltage to a selected wordline and a pass voltage to unselected wordlines. In an embodiment, in at least one of a plurality of program loops, the row decoder224may apply the program voltage to the selected wordline and an additionally selected wordline.

The page buffer225may be connected to the memory cell array221through the plurality of bit lines BL. For example, in a read operation, the page buffer225may operate as a sense amplifier that outputs data stored in the memory cell array221. Alternatively, in a program operation, the page buffer225may operate as a write driver that writes desired data to the memory cell array221.

FIGS.4and5illustrate an example in which the memory system200is implemented using a three-dimensional flash memory. The three-dimensional flash memory may include three-dimensional (e.g., vertical) NAND (e.g., VNAND) memory cells. An implementation of the memory cell array221including three-dimensional memory cells is described below. Each of the memory cells described below may be a NAND memory cell.

FIG.4is a block diagram of the memory cell array221ofFIG.2, according to an embodiment of the inventive concept.

Referring toFIG.4, the memory cell array221according to an embodiment includes a plurality of memory blocks BLK1to BLKz. Each of the memory blocks BLK1to BLKz has a three-dimensional structure (e.g., a vertical structure). For example, each of the memory blocks BLK1to BLKz may include structures extending in first to third directions. For example, each of the memory blocks BLK1to BLKz may include a plurality of NAND strings extending in the second direction. The plurality of NAND strings may be provided, for example, in the first to third directions.

Each of the NAND strings is connected to a bit line BL, a string select line SSL, a ground select line GSL, wordlines WL, and a common source line CSL. That is, each of the memory blocks BLK1to BLKz may be connected to a plurality of bit lines BL, a plurality of string select lines SSL, a plurality of ground select lines GSL, a plurality of wordlines WL, and a common source line CSL. The memory blocks BLK1to BLKz will be described in further detail below with reference toFIG.5.

FIG.5is a circuit diagram of a memory block BLKi according to an embodiment of the inventive concept.FIG.5illustrates an example of one of the memory blocks BLK1to BLKz in the memory cell array221ofFIG.4. AlthoughFIG.5illustrates an example with6wordlines and memory cells, this is just an illustration and any number of wordlines and memory cells may be used.

The memory block BLKi may include a plurality of cell strings CS11to CS41and CS12to CS42. The plurality of cell strings CS11to CS41and CS12to CS42may be arranged in column and row directions to form columns and rows. Each of the cell strings CS11to CS41and CS12to CS42may include a ground select transistor GST, memory cells MC1to MC6, and a string select transistor SST. The ground select transistor GST, the memory cells MC1to MC6, and the string select transistor SST, which are included in each of the cell strings CS11to CS41and CS12to CS42, may be stacked in a height direction substantially perpendicular to a substrate.

The columns of the plurality of cell strings CS11to CS41and CS12to CS42may be connected to different string select lines SSL1to SSL4, respectively. For example, the string select transistors SST of the cell strings CS11and CS12may be commonly connected to the string select line SSL1. The string select transistors SST of the cell strings CS21and CS22may be commonly connected to the string select line SSL2. The string select transistors SST of the cell strings CS31and CS32may be commonly connected to the string select line SSL3. The string select transistors SST of the cell strings CS41and CS42may be commonly connected to the string select line SSL4.

The rows of the plurality of cell strings CS11to CS41and CS12to CS42may be connected to different bit lines BL1and BL2, respectively. For example, the string select transistors SST of the cell strings CS11to CS41may be commonly connected to the bit line BL1. The string select transistors SST of the cell strings CS12to CS42may be commonly connected to the bit line BL2.

The columns of the plurality of cell strings CS11to CS41and CS12to CS42may be connected to different ground select lines GSL1to GSL4, respectively. For example, the ground select transistors GST of the cell strings CS11and CS12may be commonly connected to the ground select line GSL1. The ground select transistors GST of the cell strings CS21and CS22may be commonly connected to the ground select line GSL2. The ground select transistors GST of the cell strings CS31and CS32may be commonly connected to the ground select line GSL3. The ground select transistors GST of the cell strings CS41and CS42may be commonly connected to the ground select line GSL4.

The memory cells disposed at the same height from the substrate (or the ground select transistors GST) may be commonly connected to a single wordline, and the memory cells disposed at different heights from the substrate may be connected to different wordlines WL1to WL6, respectively. For example, the memory cells MC1may be commonly connected to the wordline WL1. The memory cells MC2may be commonly connected to the wordline WL2. The memory cells MC3may be commonly connected to the wordline WL3. The memory cells MC4may be commonly connected to the wordline WL4. The memory cells MC5may be commonly connected to the wordline WL5. The memory cells MC6may be commonly connected to the wordline WL6. The ground select transistors GST of the cell strings CS11to CS41and CS12to CS42may be commonly connected to the common source line CSL.

FIG.6shows an example of an equalizer architecture according to aspects of the present disclosure. The example shown includes neural network600, read information605, and symbol estimations610. Neural network600may be included in, or implemented via, equalizer213as described with reference toFIG.2. Read information605is an example of, or includes aspects of, the corresponding element described with reference toFIG.9. Symbol estimations610is an example of, or includes aspects of, the corresponding element described with reference toFIG.9.

A neural network600is a type of computer algorithm that is capable of learning specific patterns without being explicitly programmed, but through iterations over known data. A neural network600may refer to a cognitive model that includes input nodes, hidden nodes, and output nodes. Nodes in the network may have an activation function that computes whether the node is activated based on the output of previous nodes. Training the system may involve supplying values for the inputs and modifying edge weights and activation functions (algorithmically or randomly) until the result closely approximates a set of desired outputs.

An artificial neural network600is a hardware or a software component that includes a number of connected nodes (i.e., artificial neurons), which loosely correspond to the neurons in a human brain. Each connection, or edge, transmits a signal from one node to another (like the physical synapses in a brain). When a node receives a signal, it processes the signal and then transmits the processed signal to other connected nodes. In some cases, the signals between nodes comprise real numbers, and the output of each node is computed by a function of the sum of its inputs. Each node and edge is associated with one or more node weights that determine how the signal is processed and transmitted. During the training process, these weights are adjusted to improve the accuracy of the result (i.e., by minimizing a loss function which corresponds in some way to the difference between the current result and the target result). The weight of an edge increases or decreases the strength of the signal transmitted between nodes. In some cases, nodes have a threshold below which a signal is not transmitted at all. In some examples, the nodes are aggregated into layers. Different layers perform different transformations on their inputs. The initial layer is known as the input layer and the last layer is known as the output layer. In some cases, signals traverse certain layers multiple times.

As described herein, an equalizer (e.g., an equalizer module) is used to minimize the inter-symbol-interference (ISI) across NAND cells. The equalization module receives sample (memory cell) features. For example, sample features may include cell voltage, pillar and wordline neighbors, thresholds, etc. Some equalization modules assume all samples pass through the same neural network (e.g., such as neural network600). In other words, neural network600may analyze all samples and perform complex classification operations to classify the various samples into a voltage state out of several possible voltage states. The same features are fed to a neural network600(e.g., producing 16 total probabilities that are associated with each level). For example, the equalizer architecture shown in the example ofFIG.6can be used for 3 bits per cell (e.g., triple-level cell (TLC)) flash memory.

In some examples, neural networks such as neural network600may include complex machine-learning models (e.g., that may be associated with high latency and memory parameters for complex classification operations). Due to high latency and memory parameters, specific NAND architectures are used to support the equalization modules. Some NAND architectures may not be able to support some equalization modules (e.g., due to the complexity of some neural network architectures used to classify various samples into one voltage state out of several possible voltage states).

In the example ofFIG.6, an equalizer architecture includes a neural network600that outputs soft symbol estimations610based on input read information605. Read information605may include cell voltage information615, same-pillar neighbors voltage information620, same-wordline neighbors voltage information625, hard decision (HD) thresholds (e.g., all HD thresholds) of a wordline information630, and wordline index information635, among other examples. In some examples, all samples (e.g., all read information605) may be passed through the neural network600(e.g., and the neural network600outputs soft symbol estimations610for all read information605).

In some examples, a scheme for error correcting may be based on multiple shallow threshold expert models (e.g., threshold experts), rather than a single neural network600. Such a scheme may be used for error correcting in an equalizing phase (e.g., before an ECC is applied). A class of equalizer using multiple shallow threshold expert models may be referred to as multi-threshold expert models (MTEMs). In some v-NAND devices, each cell may store N (e.g., 3 or 4) bits of information using a voltage. In a de-noising or equalizing phase, a voltage stored in a cell may be mapped to one of 2Npossible values (e.g., levels) corresponding to all possible combinations of the N bits.

Threshold expert models may split a de-noising problem into single bit decision problems by considering cases where the voltage of a cell is close to a decision boundary between two levels. Cells with voltages close to a decision boundary between two levels may be referred to as weak cells. The soft symbol estimations610output by the neural network600may include an output for a weak cell. The output for the weak cell may be a single LLR value:

where X is a vector of features measured for each cell (e.g., read information605), and Fwis a function with a small number of parameters (w). For N bits per cell, there may be 2Nlevels and 2N−1 threshold models.

In a de-noising phase, an equalizer architecture (e.g., or another error correcting apparatus) may scan all cells (e.g., with parallelism) and may identify weak cells, find a corresponding threshold model for each weak cell, and apply the threshold model on the weak cell. In some examples, however, the equalizer architecture may have two competing performance measures: accuracy and complexity cost. Accuracy may refer to the error correcting capability of the equalizer architecture, and complexity cost may refer to the cost of using models (e.g., in terms of latency or gate count). To improve a model's ability to correct bit-flip errors, it may be appropriate for the read information605to include a larger feature vector and for each threshold expert model to include more parameters. However, larger models may result in an increased computing time (e.g., in firmware) or a larger gate count (e.g., in hardware). Thus, improving accuracy may result in a higher complexity cost.

Shallow models may include a small number of features and may accept fewer feature inputs. Thus, shallow models may favor lowering complexity cost over accuracy. An example architecture for shallow threshold expert models picked for implementation in mobile controllers may be a linear function (Fw) with four features: target cell voltage, upper neighbor along pillar voltage, lower neighbor along pillar voltage, and a threshold reference voltage (e.g., a voltage used for a hard decision without an equalizer). Each threshold expert model may have five parameters (e.g., 4 weights multiplying input features and a bias term). So, all together, there may be 5×(2N−1) parameters in an equalizing phase.

A model including shallow threshold expert models that accepts feature inputs may be referred to as a linear-MTEM. A linear-MTEM may be implemented in hardware or firmware (e.g., depending on a balance between CPU latency and gate cost). In some examples, training a model with multiple threshold expert models may include choosing a single workpoint, such as a number of program-erase (PE) cycles and a retention time (e.g., measured by oven time at 100° C.). During training, random (but known) bits may be written to a chip, brought to the workpoint, and the features of the bits described above may be measured. The data may then be split into corresponding threshold models (e.g., for weak cells), and single threshold models may be trained (e.g., using machine learning techniques) to correctly estimate an LLR.

Despite its simplicity, a model including multiple threshold expert models may achieve close to optimal accuracy. For instance, the model may experience an accuracy gap or a drop of up to 5% in corrected bits compared to high-performing models. The accuracy gap may be measured at the same workpoint used for training (e.g., on a different chip). In some examples, however, although a model including multiple threshold expert models may achieve high accuracy, the performance of the model may be low on workpoints that are far from a training workpoint. Larger models may utilize additional features (e.g., wordline index, pe-cycle count, etc.) to adjust for changes in workpoints. However, increasing the sizes of the threshold expert models may increase the complexity cost associated with these models.

In some embodiments, rather than increasing the sizes of threshold experts, a meta network may be used to generate parameters for threshold expert models to allow for increasing the accuracy of the threshold expert models (e.g., at various workpoints) while keeping the complexity costs of the threshold expert models low. For instance, the meta network may be implemented in a neural network that includes multiple threshold expert models to share performance cost while keeping accuracy close to optimal for many (e.g., all) workpoints.

The meta network may use wordline features (e.g., features or characteristics of a wordline in memory) to generate parameters for each of a set of threshold expert models. One example of a wordline feature is a set of block PE cycles corresponding to a number of PE cycles for a target block. Another example of a wordline feature is a tier index (e.g., within a pillar) corresponding to a page index divided (e.g., without a remainder) by a number of pages in a layer. Another example of a wordline feature is an SSL index corresponding to a layer of a target page (e.g., a layer in which a target page is embedded). Another example of a wordline feature is a very-strong errors ratio corresponding to a percentage of cells in a sector with a very-strong soft decision symbol. Another example of a wordline feature is a normal-strong errors ratio corresponding to a percentage of cells in a sector with a normal-strong soft decision symbol. Another example of a wordline feature is a normal-weak errors ratio corresponding to a percentage of cells in a sector with a normal-weak soft decision symbol. Another example of a wordline feature is a very-weak errors ratio corresponding to a percentage of cells in a sector with a very-weak soft decision symbol. Another example of a wordline feature is an HD threshold voltage (e.g., all HD threshold voltages) corresponding to a reference voltage used to read HD data from cells. Another example of a wordline feature is a soft decision bits histogram.

The process for flash memory de-noising shown inFIG.7may employ machine learning techniques to minimize ISI across NAND cells and reduce bit error rate (BER) and fine frame error rate (FER). A meta network may be used to adjust a shallow threshold expert to changing wordlines and workpoints (e.g., rather than adjusting a model for a single workpoint, having all wordlines share the same model, and determining a working point by a number of PE cycles and retention time). The use of a meta network to generate parameters for threshold experts may allow for improved BER reduction and improved LLR calibration. In addition, the meta network may be trained across multiple workpoints to adjust threshold expert parameters (e.g., rather than operating a threshold expert with fixed parameters and training the threshold expert on a single workpoint).

At operation700, the system receives first information for a plurality of memory cells of a memory device and second information for a memory cell of the plurality of memory cells. In some cases, the operations of this step refer to, or may be performed by, a meta network, a voltage detector, or both as described with reference toFIG.8.

At operation705, the system generates one or more threshold expert parameters for a threshold expert based on the first information. In some cases, the operations of this step refer to, or may be performed by, a meta network as described with reference toFIG.8.

At operation710, the system selects the threshold expert from a plurality of threshold experts based on the second information, wherein each of the threshold experts correspond to at least one weak decision voltage range from a plurality of weak decision voltage ranges of the memory cell. In some cases, the operations of this step refer to, or may be performed by, a selection component as described with reference toFIG.8.

At operation715, the system generates a memory channel output based on the second information using the selected threshold expert and the one or more threshold expert parameters. In some cases, the operations of this step refer to, or may be performed by, a threshold expert as described with reference toFIG.8.

FIG.8shows an example of a block diagram illustrating an implementation of a data processing system according to aspects of the present disclosure. The example shown includes a host device100and a memory system200. The memory system200includes a memory controller210and a memory device220. The memory controller210includes a voltage detector800, selection component805, plurality of threshold experts810, error correction component815, and meta network820. Host device100, memory device220, and memory controller210are examples of, or include aspects of, the corresponding elements described with reference toFIGS.1and2. The plurality of threshold experts810may be included in, or implemented via, equalizer213as described with reference toFIG.2.

Examples of a memory system200include random access memory (RAM), read-only memory (ROM), or a hard disk. Examples of memory systems200include solid state memory and a hard disk drive. In some examples, memory is used to store computer-readable, computer-executable software including instructions that, when executed, cause a processor to perform various functions described herein. In some cases, the memory contains, among other things, a basic input/output system (BIOS) which controls basic hardware or software operation such as the interaction with peripheral components or devices. In some cases, a memory controller210operates memory cells. For example, the memory controller210can include a row decoder, column decoder, or both. In some cases, memory cells within a memory store information in the form of a logical state.

In some embodiments, examples of the memory system200include flash memory. Flash memory is an electronic (solid-state) non-volatile computer storage medium that can be electrically erased and reprogrammed. The two main types of flash memory are named after the NAND and NOR logic gates. The individual flash memory cells exhibit internal characteristics similar to those of the corresponding gates. Where EPROMs had to be completely erased before being rewritten, NAND-type flash memory may be written and read in blocks (or pages) which are generally much smaller than the entire device. NOR-type flash allows a single machine word (byte) to be written—to an erased location—or read independently. The NAND type operates primarily in memory cards, USB flash drives, solid-state drives (those produced in2009or later), and similar products, for general storage and transfer of data. NAND or NOR flash memory is also often used to store configuration data in numerous digital products, a task previously made possible by EEPROM or battery-powered static RAM. One key disadvantage of flash memory is that it can only endure a relatively small number of write cycles in a specific block. Example applications of both types of flash memory include personal computers, PDAs, digital audio players, digital cameras, mobile phones, synthesizers, video games, scientific instrumentation, industrial robotics, and medical electronics.

In addition to being non-volatile, flash memory offers fast read access times, although not as fast as static RAM or ROM. Its mechanical shock resistance helps explain its popularity over hard disks in portable devices, as does its high durability, ability to withstand high pressure, temperature and immersion in water, etc. Although flash memory is technically a type of EEPROM, the term “EEPROM” is generally used to refer specifically to non-flash EEPROM which is erasable in small blocks, typically bytes. Because erase cycles are slow, the large block sizes used in flash memory erasing give it a significant speed advantage over non-flash EEPROM when writing large amounts of data. Flash memory costs much less than byte-programmable EEPROM and had become the dominant memory type wherever a system required a significant amount of non-volatile solid-state storage.

Flash memory stores information in an array of memory cells made from floating-gate transistors. In single-level cell (SLC) devices, each cell stores only one bit of information. In multi-level cell (MLC) devices, including TLC devices, each cell can store more than one bit per cell. The floating gate may be conductive or non-conductive. In NOR flash, each cell has one end connected directly to ground, and the other end connected directly to a bit line. This arrangement is called “NOR flash” because it acts like a NOR gate: when one of the wordlines is brought high, the corresponding storage transistor acts to pull the output bit line low. NAND flash also uses floating-gate transistors, but they are connected in a way that resembles a NAND gate: several transistors are connected in series, and the bit line is pulled low only if all the wordlines are pulled high. These groups are then connected via some additional transistors to a NOR-style bit line array in the same way that single transistors are linked in NOR flash. Compared to NOR flash, replacing single transistors with serial-linked groups adds an extra level of addressing.

In the example ofFIG.8, an equalizer architecture of memory controller210includes a plurality of threshold experts810that each output soft symbol estimations (e.g., generated estimations of channel output of a memory cell) to error correction component815based on input read information from memory device220(e.g., for error correction component815to perform an ECC operation on the output soft symbol estimations). For instance, a voltage detector800may detect read information (e.g., voltage information) from a memory cell, and selection component805may select a threshold expert from the plurality of threshold experts810to perform a classification operation on the read information. In the example ofFIG.8, samples (e.g., read information samples, detected voltage information samples, etc.) may be passed through a specific threshold expert, from the plurality of threshold experts810, where the specific threshold expert for each sample is selected by selection component805based on the sample (e.g., based on a weak decision range the sample resides in, as further described herein, for example, with reference toFIG.11).

Accordingly, each threshold expert of the plurality of threshold experts810may include a shallow machine learning model (e.g., a shallow threshold expert machine learning model) that may be associated with low latency and low memory parameters. That is, the shallow threshold experts described herein may perform binary classification operations for a particular weak decision range (e.g., as described in more detail herein, for example, with reference toFIGS.9-11). Such classification operations may be less complex relative to classification operations for several possible voltage states (e.g., as performed by more complex machine learning models such as, for example, neural network600). Accordingly, due to reduced latency and memory parameters, various NAND architectures may support such equalization architectures implementing a plurality of threshold experts810(e.g., a plurality of shallow threshold expert machine learning models each performing a respective classification operation between two corresponding voltage states).

According to some embodiments, host device100may be configured to transmit a read command to the memory device220(e.g., where read information is read by the memory controller210based on, or in response to, the read command). According to some embodiments, host device100may be configured to receive a response from the memory controller210based at least in part on estimated channel output.

According to some embodiments, meta network820receives first information for a plurality of memory cells of the memory device220, and the meta network820generates one or more threshold expert parameters for a threshold expert based on the first information. In some examples, the plurality of memory cells comprise a wordline or a subset of a wordline of the memory device220. In some examples, the first information includes a number of PE cycles, a layer of a target page, a tier index, a very-strong error ratio, a normal-strong error ratio, a normal-weak error ratio, a very-weak error ratio, one or more threshold voltages, or any combination thereof. In some examples, the one or more threshold expert parameters include a weight of an upper cell voltage feature, a weight of a target cell voltage feature, a weight of a lower cell voltage feature, a weight of a threshold voltage feature, a bias term value, or any combination thereof. In some examples, the meta network820generates threshold expert parameters for each of a plurality of threshold experts.

According to some embodiments, voltage detector800receives second information (e.g., read information) for a memory cell the plurality of memory cells of the memory device220. In some examples, the second information includes one or more of a voltage information of the memory cell, a voltage information of a cell in a neighboring wordline of the memory cell, a voltage of a cell in a same wordline as the memory cell, one or more HD thresholds of a wordline, one or more SD thresholds of a wordline, and a wordline index. In some examples, the second information includes an upper cell voltage, a target cell voltage, a lower cell voltage, a threshold voltage, or any combination thereof. According to some embodiments, voltage detector800may be configured to detect voltage information from a memory cell.

According to some embodiments, selection component805selects the threshold expert from the plurality of threshold experts based on the second information, where each of the threshold experts correspond to at least one weak decision voltage range from a plurality of weak decision voltage ranges of the memory cell. In some examples, selection component805determines that a voltage value of the second information is within the at least one weak decision voltage range corresponding to the threshold expert, where the threshold expert is selected based on the determination. In some examples, each of the weak decision voltage ranges include a region between two hard decision thresholds where voltage information for different values of the channel output are likely to overlap. In some examples, a number of the threshold experts is one less than a number of hard decision thresholds. In some examples, selection component805transmits the read information to the selected threshold expert. In some examples, the weak decision voltage ranges are separated from each other by a set of strong decision voltage ranges, and where each of the strong decision voltage ranges corresponds to a value of the channel output.

According to some embodiments, selection component805may be configured to select a threshold expert from the plurality of threshold experts810based on the detected voltage information. In some examples, the selection component805is further configured to determine that the detected voltage information corresponds to a weak decision voltage range from the set of weak decision voltage ranges, where the threshold expert is selected based on the determination. Selection component805is an example of, or includes aspects of, the corresponding element described with reference toFIG.9.

The plurality of threshold experts810is an example of, or includes aspects of, the corresponding element described with reference toFIGS.9and11. According to some embodiments, the selected threshold expert of the plurality of threshold experts810generates a memory channel output based on the second information and the one or more threshold expert parameters. For instance, the selected threshold expert generates an improved memory channel output from the read information using the selected threshold expert. In some examples, each of the threshold experts is configured with parameters for processing read information corresponding to a specific weak decision voltage range. In some examples, plurality of threshold experts810generates LLRs based on the improved memory channel output.

According to some embodiments, plurality of threshold experts810may be configured to estimate a channel output of the memory cell based on the detected voltage information, wherein each of the threshold experts corresponds to a different weak decision voltage range from a plurality of weak decision voltage ranges. In some examples, each of the set of threshold experts is further configured to generate an output value that indicates whether the detected voltage information corresponds to a first value of the channel output or a second value of the channel output. In some examples, each of the threshold experts is trained to operate on read information (e.g., using input information) corresponding to a specific weak decision voltage range. In some examples, each of the weak decision voltage ranges includes a region surrounding a hard decision threshold where voltage information for different values of the channel output are likely to overlap.

According to some embodiments, error correction component815performs an ECC operation and determines that the ECC operation failed. The memory channel output generated by a threshold expert of the plurality of threshold experts810may be generated based on the determination.

According to some embodiments, memory device220comprises a plurality of memory cells arranged in a plurality of wordlines and a plurality of bitlines. According to some embodiments, memory controller210reads a set of memory cells including the memory cell. In some examples, memory controller210generates a codeword based on the reading. In some examples, memory controller210determines that the ECC operation failed. In some examples, memory controller210generates an updated codeword based on the improved memory channel output. In some examples, memory controller210receives a read command from a host device100. In some examples, memory controller210reads voltage information from the memory cell in response to the read command, where the read information includes the voltage information. In some examples, memory controller210transmits a response to the host device100based on the codeword.

In some examples, memory controller210comprises a processor. According to some embodiments, the memory controller210is configured to receive read information for a memory cell of a memory device220. According to some embodiments, the memory controller210is configured to select a threshold expert (e.g., a threshold expert) from a plurality of threshold experts810based on the read information, where each of the threshold experts810corresponds to a different weak decision voltage range from a plurality of weak decision voltage ranges (e.g., as further described herein, for example, with reference toFIG.11). According to some embodiments, the memory controller210is configured to generate an estimation of a channel output from the read information using the selected threshold expert. As described herein, in some examples, host device100may be configured to receive a response from the memory controller210based at least in part on the estimate of the channel output.

FIG.9shows an example of an equalizer architecture using multiple threshold expert machine learning models according to aspects of the present disclosure. The example shown includes selection component900, plurality of threshold experts905, read information925, symbol estimations955, meta network960, and wordline features965.

Plurality of threshold experts905may be included in, or implemented via, equalizer213as described with reference toFIG.2. Plurality of threshold experts905is an example of, or includes aspects of, the corresponding element described with reference toFIGS.8and10. In one embodiment, plurality of threshold experts905includes first threshold expert910, second threshold expert915, and third threshold expert920. First threshold expert910, second threshold expert915, and third threshold expert920are examples of, or include aspects of, the corresponding elements described with reference toFIG.11.

Selection component900is an example of, or includes aspects of, the corresponding element described with reference toFIG.8. Meta network960is an example of, or includes aspects of, the corresponding element described with reference toFIG.8. Read information925is an example of, or includes aspects of, the corresponding element described with reference toFIG.6. Symbol estimations955is an example of, or includes aspects of, the corresponding element described with reference toFIG.6. Wordline features965is an example of, or includes aspects of, the corresponding element described with reference toFIG.10.

In the example ofFIG.9, an equalizer architecture includes a plurality of neural networks905that each output soft symbol estimations955(e.g., to an error correction component) based on input read information925from memory device (e.g., for an error correction component to perform an ECC operation on the output soft symbol estimations). Read information925may include cell voltage information930, same-pillar neighbors voltage information935, same-wordline neighbors voltage information940, HD thresholds (e.g., all HD thresholds) of a wordline information945, and wordline index information950, among other examples.

In the example ofFIG.9, samples (e.g., read information samples, detected voltage information samples, etc.) may be passed through a specific threshold expert, from the plurality of threshold experts905, where the specific threshold expert for each sample is selected by selection component900based on the sample (e.g., based on a weak decision range the sample resides in, as further described herein, for example, with reference toFIG.11).

As described herein, each threshold expert of the plurality of threshold experts905may include a shallow machine learning model. For example, each of first threshold expert910, second threshold expert915, third threshold expert920, etc., may include a shallow threshold expert machine learning model trained and configured to perform a classification task between two voltage states corresponding to a specific weak decision range. That is, the shallow threshold experts described herein may perform binary classification operations for a particular weak decision range (e.g., as described in more detail herein, for example, with reference toFIG.11). Such classification operations may be less complex relative to classification operations for several possible voltage states (e.g., as performed by more complex machine learning models such as, for example, neural network600). Accordingly, due to reduced latency and memory parameters, various NAND architectures may support such equalization architectures implementing a plurality of threshold experts905(e.g., a plurality of shallow threshold expert machine learning models each performing a respective classification operation between two corresponding voltage states).

In some cases, the example architecture ofFIG.9may be referred to as a multiple threshold architecture. Such architectures may include multiple threshold expert models (e.g., such as first threshold expert910, second threshold expert915, third threshold expert920, etc.). Each threshold expert model may be assigned per HD threshold (e.g., as further described herein, for example, with reference toFIG.11). For example, the multiple threshold architecture may include 15 threshold expert models, but the present disclosure is not limited thereto. Each sample is received by (e.g., analyzed by, classified by, etc.) one of the threshold expert models out of the plurality of threshold experts905depending on a detected voltage of the sample. An equalizer treats the weak samples (e.g., samples within a weak decision range that are not readily classifiable by the equalizer into a voltage state).

The meta network960may determine or generate parameters for each threshold expert model to use to classify relevant samples received from the selection component900based on corresponding HD proximity. In some examples, the meta network960may identify wordline features965, and the meta network960may be trained to generate parameters for each threshold expert model based on the wordline features965. The wordline features965for the meta network960may be based on aggregated properties of a sector (e.g., in memory) and may include features or characteristics of one or more wordlines in memory. In some examples, wordline features965may be based on standard device operations, and a device may identify the wordline features965without additional read operations.

Once parameters are assigned to the plurality of threshold experts905by the meta network960, the plurality of threshold experts905may be used to perform error correction and de-noising. A device may read a corrupted codeword, read additional information describing the likelihood that there was an error in a given cell, and estimate the LLRs for each cell. An ECC decoder may then attempt to decode the codeword. The ECC decoder may search for a most likely codeword given the estimated LLRs. If decoding succeeds, the ECC may output an information word. However, if decoding fails, the device may read additional information corresponding to the read information925. A de-noising algorithm may then perform error correction (e.g., on a weak cell) to change the values of the LLRs, and the ECC decoder may attempt to decode the modified LLRs.

As used herein, a codeword may refer to information that is written to a device and may be a part of an ECC codebook. As used herein, a corrupted codeword may refer to a word with a similar size to a codeword but with some bits flipped (e.g., so the corrupted codeword may not be in the ECC codebook). As used herein, an LLR is generated for each cell and may refer to the natural logarithm of the ratio between the probability that a true bit value is 0 to the probability that the true bit value is 1

FIG.10shows an example of a meta network architecture according to aspects of the present disclosure. Meta networks (or meta models) in machine learning are models that generate parameters for another model. In some examples, a meta network may be implemented in firmware (e.g., based on C code). In the example ofFIG.10, the meta network1000may generate parameters of MTEM models. Unlike threshold experts that may run (e.g., be evaluated) per cell (e.g., per weak cell), the meta network1000may run (e.g., be evaluated) once for each sector in memory (e.g., where there may be approximately 34000 cells in a sector). Thus, a larger model may be used for the meta network1000(e.g., a model with more parameters) with a minimal increase in complexity cost.

Because memory may be read in pages (e.g., a single bit out of N bits in a cell), the meta network1000may be designed to be efficient for single page reads. The input features for the meta network1000may be referred to as wordline features1005and may be based on standard device operations (e.g., rather than being obtained with additional read operations). In some examples, the wordline features1005may not be shared by an entire page. Once the wordline features1005are obtained, the meta network1000may generate parameters for threshold experts using neural networks. Further, once the meta network1000generates (e.g., calculates) parameters for the threshold experts, a device (e.g., an MTEM equalizer) may read from memory as described inFIG.9using the threshold experts for error correction.

In some implementations, the meta network1000may include multiple neural networks1010. A separate neural network may be used to generate parameters for a threshold and a corresponding threshold expert (e.g., a corresponding linear threshold model). For example, a neural model may be used to generate parameters for threshold0and another neural model may be used to generate parameters for threshold1. The inputs to the meta network1000(e.g., wordline features1005) may be shared between all neural networks1010. In addition, the output of a neural network may be the parameters (e.g., 5 parameters) for a corresponding threshold and threshold expert (e.g., used to setup or configure the threshold expert).

In some examples, the parameters generated for a threshold expert may include a weight of an upper cell voltage feature (e.g., a nearest neighbor cell along the pillar with higher wordline index), a weight of a target cell voltage feature, a weight of a lower cell voltage feature (e.g., a nearest neighbor cell along the pillar with lower wordline index), a weight of a threshold voltage feature, and a bias term value. The architecture of a neural network (e.g., of the neural networks1010) used to generate parameters for a threshold expert may include a fully-connected, feed-forward neural network, two hidden layers (e.g., one with 32 neurons and another with 8 neurons), a rectified linear unit (RELU) activation after each hidden layer, and a fully linear layer as the output layer.

Once the parameters are assigned to a threshold expert, the threshold expert may be used for error correction. The threshold expert may accept one or more inputs including a target cell voltage, an upper neighbor along pillar voltage, a lower neighbor along pillar voltage, and a threshold reference voltage (e.g., a voltage used for a hard decision without an equalizer). Based on the inputs, the threshold expert may generate a single output Fw(X)=b+Σiwi· Xi, where b is a bias term and wiis a weight assigned to an input feature i. In some examples, a threshold expert may be trained using standard machine learning techniques such that Fw(X) estimates an LLR.

FIG.11shows an example of a sample de-noising scheme according to aspects of the present disclosure. The example shown includes levels1100, thresholds1105, strong decision voltage ranges1110, weak decision voltage range1115, and plurality of threshold experts1120. Plurality of threshold experts1120is an example of, or includes aspects of, the corresponding element described with reference toFIGS.8and9. In one embodiment, plurality of threshold experts1120includes first threshold expert1125, second threshold expert1130, and third threshold expert1135. Plurality of threshold experts1120may be included in, or implemented via, equalizer213as described with reference toFIG.2. First threshold expert1125is an example of, or includes aspects of, the corresponding element described with reference toFIG.9. Second threshold expert1130is an example of, or includes aspects of, the corresponding element described with reference toFIG.9. Third threshold expert1135is an example of, or includes aspects of, the corresponding element described with reference toFIG.9.

To read the information from a memory device, voltage of each cell is measured and the voltage level that was stored in the cell is inferred. The bits can then be recovered. There may be a tradeoff between the number of voltage levels and the memory reliability. The larger the number of bits per cell, the more information that can be stored on the device, where there are more bits in every cell. Alternatively, the voltages that represent different levels may be packed more tightly together, because within the same dynamic range, a larger number of distinguishable voltages is used. As a result, noise in the cell programming or cell reading has a larger chance of changing the voltage of a level to another voltage representing a different level, therefore rendering an error upon reading the cell.

There are multiple sources of noise in a memory device that can result in erroneous reading of the information, such as writing noise, interference noise, aging, and operation of reading. Writing noise is a function of a programming procedures and is seen when the voltage of a cell immediately after programming differs from the intended voltage. Interference noise is a function of the voltage of a cell changing as a result of programming a different neighboring cell. Programming cells causes a disturbance that affects other cells. Aging is where there is an increase in noise the more times a device is written and read from. Additionally, the more time between programing of a cell, the more noise the cell will produce. Also, the operation of reading a cell can cause noise and disturbances.

In the example ofFIG.11, samples (e.g., read information samples, detected voltage information samples, etc.) may be passed through a threshold expert (e.g., first threshold expert1125, second threshold expert1130, third threshold expert1135, etc.) that is selected by a selection component from the plurality of threshold experts1120. As described herein, a specific threshold expert for each sample is selected (e.g., by selection component) based on characteristics (e.g., a detected value, such as detected voltage) of the sample. For instance, a threshold expert is selected based on a weak decision range1115the sample resides in.

For instance, as described herein, memory devices (e.g., NAND flash memory devices) may allow several bits of data to be stored in each memory cell. A memory cell in which multiple bits of data are stored may be referred to as a multi-level memory cell. A multi-level memory cell partitions a threshold voltage range of a memory cell into several voltage states, and data values written to the memory cell are extracted using the memory cell voltage levels (e.g., such as ‘Level 0,’ ‘Level 1,’ . . . ‘Level 7’).

To read the information from a memory device, voltage of each cell is measured and the voltage level (e.g., such as ‘Level 0,’ ‘Level 1,’ . . . ‘Level 7’) that was stored in the cell is inferred. The bits can then be recovered. The larger the number of bits per cell, the more information that can be stored on the device. However, the voltages that represent different levels (e.g., voltages representing ‘Level 0,’ ‘Level 1,’ . . . ‘Level 7’) for such cells may be packed more tightly together, because within the same dynamic range, a larger number of distinguishable voltages is used. As described herein, when information is read from a cell (e.g., a NAND cell), the read information may be corrupted with noise (e.g., depending on the characteristics of the medium), which may result in an ambiguous value of the read information (e.g., the read voltage level may be within some weak decision voltage range1115between two voltage values).

In the example ofFIG.11, eight levels1100are shown (e.g., ‘Level 0,’ ‘Level 1,’ . . . ‘Level 7’) that may be represented by three bits (e.g., ‘000’= ‘Level 0;’ ‘001’= ‘Level 1;’ . . . ‘111’= ‘Level 7’). HD thresholds1105may exist between two neighboring levels1100. Moreover, within a dynamic range spanning ‘Level 0’ to ‘Level 7,’ strong decision ranges1110and weak decision ranges1115may exist. Strong decision ranges1110may refer to ranges (e.g., voltage ranges) where the level of a sample is readily determined (e.g., by a voltage detector). Weak decision ranges1115may include ranges surrounding a HD threshold1105, where the level of a sample is not readily determined (e.g., by a voltage detector). For instance, a weak decision range1115may include values (e.g., voltage values) closely preceding and closely subsequent to HD thresholds1105.

Generally, levels1100may be distinguishable based on HD thresholds1105, and levels1100may include strong decision ranges1110and portions of weak decision ranges1115. Read information (e.g., detected voltages) within a strong decision range1110may be readily classified as a corresponding level1100(e.g., by a voltage detector without the use of a threshold expert). Read information (e.g., detected voltages) within a weak decision range1115may be passed to a selection component, where the selection component may select a threshold expert to classify the read information as one of the two neighboring levels1100. In the example ofFIG.11, a HD threshold1105-bmay reside between a level1100-aand a level1100-b. A weak decision range1115may include the range between the level1100-aand a level1100-bwhere further analysis on the read information is implemented via a corresponding threshold expert from the plurality of threshold experts1120. As described herein, a weak decision range1115may include a range where read information for different values of the channel output are likely to overlap (e.g., such that a threshold expert may be used to classify the read information as one of the two associated neighboring values, for improved accuracy and reliability).

Accordingly, strong samples, or samples within a strong decision range1110may have channel output estimated based on the voltage detector (e.g., without the use of a threshold expert, as the sample is withing a strong decision range1110where accuracy of the estimated channel output is relatively high). However, weak samples, or samples within a weak decision range1115may have channel output estimated by a threshold expert selected by a selection component (e.g., where the threshold expert is selected based on a HD threshold classification operation associated with the weak decision range1115the detected read information resides in).

Further, as described herein, such equalization operations may be performed subsequent to ECC operation failure in order to generate an updated codeword for a subsequent ECC operation attempt. In cases where an ECC operation is successful, equalization operations need not necessarily be performed.

In an example scenario, first threshold expert1125trains based on samples in the proximity of HD Threshold0, second threshold expert1130trains based on samples in the proximity of HD Threshold1, and so on (e.g., though third threshold expert1135that trains based on samples in the proximity of HD Threshold6). Each threshold expert model solves a binary classification task. During inference, each sample passes through one of the threshold expert models. As an example, pseudocode for single threshold mode is described below.

Single Threshold Mode Pseudocode:

Let MLEθ- the threshold model network= MLEθ(f)info - the information bits mini-batch per network use- the estimated inforamtion bits mini-batch at the read network outputf - feature vectorλθ- learning rateInitialize parameter vector θfor each iteration doθ ← θ - λθ∇θCrossEntropy(info,)end for

The neural network nature of the Threshold Models is just an example, any other types of machine-learning models could be used (decision trees, random forests, etc.). Embodiments of the present disclosure are based on multiple shallow threshold expert machine learning models. Multiple threshold architectures include several machine-learning models, one per each HD threshold (e.g., or in some cases, as described herein, one threshold expert per subset of total HD thresholds spanning a given dynamic range). Each threshold expert is a shallow machine-learning model, and each threshold expert may solve binary classification tasks. During inference, each sample passes through one of the threshold models. The threshold expert is selected by a selection component (e.g., a threshold model selection module), according to the memory cell's voltage and/or the closest HD threshold.

The Multiple Threshold architecture of the present disclosure results in a set of shallow machine learning models that may be operated with low memory usage (memory usage less than ECC operation memory usage) and low latency. The equalization model may be implemented in a firmware version. Therefore, the equalization model may be used by any NAND flash memory architectures.

The present disclosure reduces latency and improves accuracy, where each threshold expert model is shallow and includes a binary output. Additionally, or alternatively, relevant samples are used for each training model, improving accuracy. For example, Table 1 shows an example examination of TLC and quadruple-level cell (QLC) data sets. Table 1 summarize the bit error rate (BER) reduction achieved for each data set.

FIG.11illustrates aspects of a selection component (e.g., a threshold model selection module, such as selection component805and selection component900described with reference toFIGS.8and9). For instance, in a selection component, a threshold expert is selected for each sample within a weak decision range1115(e.g., within a range1115between two states or between two ‘Levels.’ A threshold expert may be selected that classifies between two states or levels (e.g., between two voltage states). In some cases, a weak decision range may be referred to as a range around a HD threshold (e.g., weak decision ranges1115may include and surround one of ‘Threshold0,’ ‘Threshold1,’ . . . ‘Threshold6’).

In an example scenario, the voltage of each sample is determined (e.g., via voltage detector800). Then, for samples with voltage within a weak decision range1115, an HD is determined based on the closest voltage (e.g., the closes voltage state) to the determined voltage of a sample. The features of the sample are then provided to the corresponding threshold module (e.g., the threshold expert trained or configured to perform a classification operation for the weak decision range1115associated with the voltage of the sample determined by the voltage detector). An improved parameter of each sample may then be determined, as each sample may be classified by a threshold expert trained and configured to classify samples within the specific weak decision range1115that includes the detected voltage of the sample. Moreover, due to reduced latency and memory parameters associated with the plurality of shallow threshold experts1120, various NAND architectures may support such equalization architectures implementing the plurality of threshold experts1120.

Moreover, the example ofFIG.11is provided for illustrative purposes and is not intended to be limiting in terms of the scope of the present disclosure.FIG.11shows an example where a parameter such as level estimation may be determined, however other estimations may be used (e.g., such as like-bits, soft symbols, etc.). In another example embodiment of the present disclosure, the number of threshold experts may be reduced (e.g., as a threshold expert may perform classification operations for two weak decision ranges1115, for example). For instance, samples within an HD threshold behave similarly. Therefore, a same threshold model may be used for samples in different HD (e.g., weak decision range1115) neighbors, reducing the number of threshold models in an equalization module. For example, a first threshold expert1125may be used for classification operations associated with both weak decision range1115between ‘Level0’ and ‘Level1,’ as well as for weak decision range1115between ‘Level1’ and ‘Level2.’ In such an example, a second threshold expert1130may be used for classification operations associated with both weak decision range1115between ‘Level2’ and ‘Level3,’ as well as for weak decision range1115between ‘Level3’ and ‘Level4.’

Each threshold expert may be a shallow neural network (e.g. where each shallow neural network may be trained or configured for a classification operation for different HD thresholds or different weak decision ranges1115). Some embodiments of the present disclosure use a logistic regression model, but other architectures may be used. An accuracy-model simplicity trade-off may be obtained.

Accordingly, embodiments of the present disclosure can reduce the overall number of threshold experts and still reduce latency and memory usage (e.g., via usage of multiple shallow threshold expert networks trained or configured for one or more certain classification operations, rather than usage of a single complex machine learning network that performs classification operations for a wide range of levels or states.

As described herein, the present disclosure includes the following embodiments.

A method for flash memory de-noising using machine learning models is described. Embodiments of the method are configured to receive first information for a plurality of memory cells of a memory device and second information for a memory cell of the plurality of memory cells, generate one or more threshold expert parameters for a threshold expert based on the first information, select the threshold expert from a plurality of threshold experts based on the second information, wherein each of the threshold experts correspond to at least one weak decision voltage range from a plurality of weak decision voltage ranges of the memory cell, and generate a memory channel output based on the second information using the selected threshold expert and the one or more threshold expert parameters.

Some examples of the method described above further include determining that a voltage value of the second information is within the at least one weak decision voltage range corresponding to the threshold expert, wherein the threshold expert is selected based on the determination.

In some examples, each of the weak decision voltage ranges comprises a region between two hard decision thresholds where voltage information for different values of the channel output are likely to overlap.

In some examples, each of the threshold experts is trained to operate using input information from a corresponding weak decision voltage range from the plurality of weak decision voltage ranges.

Some examples of the method described above further include performing an ECC operation and determining that the ECC operation failed, wherein the memory channel output is generated based on the determination.

In some examples, the plurality of memory cells comprise a wordline or a subset of a wordline of the memory device.

In some examples, the first information comprises a number of PE cycles, a layer of a target page, a tier index, a very-strong error ratio, a normal-strong error ratio, a normal-weak error ratio, a very-weak error ratio, one or more threshold voltages, or any combination thereof.

In some examples, the second information includes an upper cell voltage, a target cell voltage, a lower cell voltage, a threshold voltage, or any combination thereof.

In some examples, the one or more threshold expert parameters include a weight of an upper cell voltage feature, a weight of a target cell voltage feature, a weight of a lower cell voltage feature, a weight of a threshold voltage feature, a bias term value, or any combination thereof.

Some examples of the method described above further include generating threshold expert parameters for each of the plurality of threshold experts.

An apparatus for flash memory de-noising using machine learning models is described. The apparatus includes a memory device including a plurality of memory cells arranged in a plurality of wordlines, a meta network configured to generate one or more threshold expert parameters for a threshold expert based on first information for a wordline of the plurality of wordlines, and a plurality of threshold experts configured to generate a memory channel output based on second information from a memory cell in the wordline and the one or more threshold expert parameters.

In some examples, the meta network comprises a plurality of models corresponding to the plurality of threshold experts, respectively.

Some examples of the apparatus described above further include a threshold expert selection component configured to select the threshold expert from the plurality of threshold experts based on the second information.

In some examples, each of the plurality of threshold experts comprises a linear machine learning model.

In some examples, the meta network comprises a fully-connected feedforward neural network.

Some examples of the apparatus described above further include a decoder configured to perform an ECC operation, wherein the memory channel output is generated based on a result of the ECC operation.

A non-transitory computer readable medium storing code for flash memory de-noising using machine learning models is described. In some examples, the code comprises instructions executable by a processor to: receive first information for a plurality of memory cells of a memory device and second information for a memory cell of the plurality of memory cells, generate one or more threshold expert parameters for a threshold expert based on the first information, select the threshold expert from a plurality of threshold experts based on the second information, wherein each of the threshold experts correspond to at least one weak decision voltage range from a plurality of weak decision voltage ranges of the memory cell, and generate a memory channel output based on the second information using the selected threshold expert and the one or more threshold expert parameters.

In some examples, the code comprises instructions executable by a processor to determine that a voltage value of the second information is within the at least one weak decision voltage range corresponding to the threshold expert, wherein the threshold expert is selected based on the determination.

In some examples, the code comprises instructions executable by a processor to perform an ECC operation and determine that the ECC operation failed, wherein the memory channel output is generated based on the determination.

In some examples, the code comprises instructions executable by a processor to generate threshold expert parameters for each of the plurality of threshold experts.