Nonvolatile memory with on-chip encoding for foggy-fine programming

A non-volatile storage apparatus includes a plurality of non-volatile memory cells formed on a memory die, each non-volatile memory cell configured to hold a plurality of bits of data, and a control circuit formed on the memory die. The control circuit is configured to calculate parity data for data to be stored in the memory cells and program the memory cells to first distributions. The control circuit is also configured to read memory cells in the first distributions, recover the data from results of reading the memory cells in the first distributions combined with the parity data, and further program the memory cells from the first distributions to second distributions to store the data.

BACKGROUND

The present technology relates to the operation of memory devices.

Semiconductor memory devices have become more popular for use in various electronic devices. For example, non-volatile semiconductor memory is used in cellular telephones, digital cameras, personal digital assistants, mobile computing devices, non-mobile computing devices and other devices.

A charge-storing material such as a floating gate or a charge-trapping material can be used in such memory devices to store a charge which represents a data state. A charge-trapping material can be arranged vertically in a three-dimensional (3D) stacked memory structure, or horizontally in a two-dimensional (2D) memory structure. One example of a 3D memory structure is the Bit Cost Scalable (BiCS) architecture which comprises a stack of alternating conductive and dielectric layers.

A memory device includes memory cells which may be arranged in series, in NAND strings, for instance, where select gate transistors are provided at the ends of the NAND string to selectively connect a channel of the NAND string to a source line or bit line. However, various challenges are presented in operating such memory devices.

In some memory structures (e.g. NAND structures) that use charge-storing or charge-trapping material, programming of memory cells may disturb previously-programmed memory cells (e.g. charge added to a memory cell may affect nearby memory cells). To mitigate such program disturb effects, programming may occur in two or more program operations, with neighboring memory cells programmed (at least partially) between program operations. While this approach may mitigate program disturb effects, implementation may require significant data storage capacity to maintain data between program operations (e.g. before data is fully programmed in a NAND structure) and may incur significant overhead (e.g. significant movement of data between components for multiple program operations). Efficiently programming memory cells in multiple program operations may be challenging.

DETAILED DESCRIPTION

Apparatuses and techniques are described for programming non-volatile memory cells, for example, non-volatile memory cells in a NAND structure. In some cases, non-volatile memory cells are affected by programming of other non-volatile memory cells (e.g. nearby non-volatile memory cells along neighboring word lines). One technique to deal with such effects is to program non-volatile memory cells in two or more program operations, e.g. a first “foggy” program operation that leaves non-volatile memory cells in first distributions that are approximate, followed by a second “fine” program operation that brings the non-volatile memory cells to second distributions that are more accurate (e.g. narrower distributions), which may be used for long term storage. Between such foggy programming and subsequent fine programming, neighboring non-volatile memory cells may be programmed (at least foggy programmed) so that when fine programming occurs, charge is present in neighboring non-volatile memory cells and little or no further disturbance is caused by any further programming of neighboring non-volatile memory cells (e.g. fine programming may add relatively little additional charge so that effects of fine programming are not significant). A safe copy of data (e.g. in DRAM) may be maintained while the data is foggy programmed and not yet fine programmed. However, maintaining such data may require significant space in DRAM or other storage location. Also, accessing such data (to save and subsequently retrieve it) may use significant resources (e.g. may result in significant traffic on a bus between a memory die and DRAM)

In an example, data may be encoded prior to being foggy programmed (e.g. by generating parity data) so that when a read of foggy-programmed memory cells (in first distributions) is performed, the results of that read (a foggy read) may be decoded (e.g. combined with parity data) to recover the original data. In some cases, no safe copy may be needed when data is encoded and is recoverable from foggy-programmed memory cells and parity data. On-chip circuits may be provided to perform such encoding (e.g. XOR circuits) and decoding so that resources are efficiently used and excessive transfer of data (e.g. through memory bus) is avoided. Parity data may be stored on-chip in some cases (e.g. in a portion of a non-volatile memory structure that is configured for Single Level Cell (SLC) storage). In some cases, parity data may be corrected (decoded) by Error Correction Code (ECC) circuits prior to being used for recovery of data. Parity data may also be stored off-chip (e.g. in DRAM).

FIG.1is a block diagram of an example memory device comprising memory cells arranged in different planes. The memory device100, such as a non-volatile storage system, may include one or more memory die108. The memory die108includes a memory structure126of memory cells, such as an array of memory cells, control circuit110, and read/write circuits128. The memory structure126is addressable by word lines via a row decoder124and by bit lines via a column decoder132. The memory structure126may comprise multiple planes, such as neighbor planes P0and P1. Each plane may include one or more blocks of memory cells. For example, P0includes blocks B0a-B0dand P1includes blocks B1a-B1d.

The read/write circuits128include multiple sense blocks51,52, . . .53(sensing circuit) and allow a page of memory cells to be read or programmed in parallel. Typically, a controller122is included in the same memory device100(e.g., a removable storage card or other non-volatile storage apparatus) as the one or more memory die108. The controller may be separate from the memory die. Commands and data are transferred between the host140and controller122via an interface such as data bus120, and between the controller and the one or more memory die108via lines118.

The control circuit110cooperates with the read/write circuits128to perform memory operations on the memory structure126including read, write and erase, and includes a state machine112, an on-chip address decoder114, a temperature-sensing circuit115, a power control module116, a power on detection circuit117and a timer119.

The state machine112provides chip-level control of memory operations. A storage region113may be provided, e.g., for operational parameters and software/code. In one embodiment, the state machine is programmable by the software. In other embodiments, the state machine does not use software and is completely implemented in hardware (e.g., electrical circuits). The on-chip address decoder114provides an address interface between that used by the host or a memory controller to the hardware address used by the decoders124and132. An indication of temperature which is obtained by the temperature-sensing circuit115may be used to adjust a read operation, as described further below.

The power control module116controls the power and voltages supplied to the word lines, select gate lines, bit lines and source lines during memory operations. It can include drivers for data and dummy word lines, SGS and SGD transistors and source lines. The sense blocks51-53can include bit line drivers, in one approach. The power on detection circuit may be used to detect when the memory device has been powered on. The detection circuit117may comprise an event handler which may be a software or firmware routine, or a routine implemented in hardware. The timer119may be used to determine a time which has elapsed since a last operation (e.g. read or write operation). The timer119may increment based on a clock signal used in the memory device.

In some implementations, some of the components can be combined. In various designs, one or more of the components (alone or in combination), other than memory structure126, can be thought of as at least one control circuit which is configured to perform the techniques described herein including the steps of the processes described herein. For example, a control circuit may include any one of, or a combination of, control circuit110, state machine112, on-chip address decoder114, power control module116, sense blocks51,52, . . . ,53, read/write circuits128, controller122, and so forth.

The off-chip controller122(which in one embodiment is an electrical circuit) may comprise a processor122c, storage devices (memory) such as ROM122aand RAM122band an error correction code (ECC) engine245. The ECC engine can correct a number of read errors.

A memory interface122dmay also be provided. The memory interface122d, in communication with ROM122a, RAM122band processor122c, is an electrical circuit that provides an electrical interface between controller122and memory die108. For example, the memory interface can change the format or timing of signals, provide a buffer, isolate from surges, latch I/O and so forth. The processor122ccan issue commands to the control circuit110(or any other component of the memory die) via the memory interface122d.

The storage device comprises code such as a set of instructions, and the processor is operable to execute the set of instructions to provide the functionality described herein. Alternatively, or additionally, the processor can access code from a storage device of the memory structure, such as a reserved area of memory cells in one or more word lines.

For example, code can be used by the controller to access the memory structure such as for programming, read and erase operations. The code can include boot code and control code (e.g., a set of instructions). The boot code is software that initializes the controller during a booting or startup process and enables the controller to access the memory structure. The code can be used by the controller to control one or more memory structures. Upon being powered up, the processor122cfetches the boot code from the ROM122aor memory structure126for execution, and the boot code initializes the system components and loads the control code into the RAM122b. Once the control code is loaded into the RAM, it is executed by the processor. The control code includes drivers to perform basic tasks such as controlling and allocating memory, prioritizing the processing of instructions, and controlling input and output ports.

A NAND memory array may be configured so that the array is composed of multiple strings of memory in which a string is composed of multiple memory elements sharing a single bit line and accessed as a group. Alternatively, memory elements may be configured so that each element is individually accessible, e.g., a NOR memory array. NAND and NOR memory configurations are examples, and memory elements may be otherwise configured.

The semiconductor memory elements located within and/or over a substrate may be arranged in two or three dimensions, such as a 2D memory structure or a 3D memory structure. In a 2D memory structure, the semiconductor memory elements are arranged in a single plane or a single memory device level. Typically, in a 2D memory structure, memory elements are arranged in a plane (e.g., in an x-y direction plane) which extends substantially parallel to a major surface of a substrate that supports the memory elements. The substrate may be a wafer over or in which the layer of the memory elements is formed, or it may be a carrier substrate which is attached to the memory elements after they are formed. As a non-limiting example, the substrate may include a semiconductor such as silicon.

As a non-limiting example, a 3D memory structure may be vertically arranged as a stack of multiple 2D memory device levels. As another non-limiting example, a 3D memory array may be arranged as multiple vertical columns (e.g., columns extending substantially perpendicular to the major surface of the substrate, i.e., in the y direction) with each column having multiple memory elements. The columns may be arranged in a 2D configuration, e.g., in an x-y plane, resulting in a 3D arrangement of memory elements with elements on multiple vertically stacked memory planes. Other configurations of memory elements in three dimensions can also constitute a 3D memory array.

Typically, in a monolithic 3D memory array, one or more memory device levels are formed above a single substrate. Optionally, the monolithic 3D memory array may also have one or more memory layers at least partially within the single substrate. As a non-limiting example, the substrate may include a semiconductor such as silicon. In a monolithic 3D array, the layers constituting each memory device level of the array are typically formed on the layers of the underlying memory device levels of the array. However, layers of adjacent memory device levels of a monolithic 3D memory array may be shared or have intervening layers between memory device levels.

2D arrays may be formed separately and then packaged together to form a non-monolithic memory device having multiple layers of memory. For example, non-monolithic stacked memories can be constructed by forming memory levels on separate substrates and then stacking the memory levels atop each other. The substrates may be thinned or removed from the memory device levels before stacking, but as the memory device levels are initially formed over separate substrates, the resulting memory arrays are not monolithic 3D memory arrays. Further, multiple 2D memory arrays or 3D memory arrays (monolithic or non-monolithic) may be formed on separate chips and then packaged together to form a stacked-chip memory device.

One of skill in the art will recognize that this technology is not limited to the 2D and 3D exemplary structures described but covers all relevant memory structures within the spirit and scope of the technology as described herein and as understood by one of skill in the art.

While memory device100ofFIG.1is one example of a non-volatile storage apparatus, other examples of non-volatile storage apparatuses may be implemented using the present technology.

FIG.1Ais a block diagram of one embodiment of a memory device101connected to host140that can implement the technology proposed herein. Memory device101may be considered another example of a non-volatile storage apparatus. Many different types of memory devices can be used with the technology proposed herein. One example memory device is a solid-state drive (“SSD”); however, other types of memory devices can also be used. Memory device101comprises a controller102, non-volatile memory104for storing data, and local memory106(e.g. DRAM. SRAM or ReRAM). In one embodiment, controller102comprises a Front End Processor (FEP) circuit141and one or more Back End Processor (BEP) circuits142. In one embodiment FEP circuit141is implemented on an ASIC. In one embodiment, each BEP circuit142is implemented on a separate ASIC. In one embodiment, the ASICs for each of the BEP circuits142and the FEP circuit141are implemented on the same semiconductor such that the controller102is manufactured as a System on a Chip (“SoC”). FEP circuit141and BEP circuit142both include their own processors. In one embodiment, FEP circuit141and BEP circuit142work as a master slave configuration where the FEP circuit141is the master and each BEP circuit142is a slave. For example, FEP circuit141implements a flash translation layer that performs memory management (e.g., garbage collection, wear leveling, etc.), logical to physical address translation, communication with the host, management of DRAM (local volatile memory) and management of the overall operation of the SSD (or other non-volatile storage system). BEP circuit142manages memory operations in the memory packages/die at the request of FEP circuit141. For example, the BEP circuit142can carry out the read, erase and programming processes. Additionally, the BEP circuit142can perform buffer management, set specific voltage levels required by the FEP circuit141, perform error correction (e.g., generate error correction code (ECC)), control the Toggle Mode interfaces to the memory packages, etc. In one embodiment, each BEP circuit142is responsible for its own set of memory packages. Controller102is one example of a control circuit.

In one embodiment, non-volatile memory104comprises a plurality of memory packages. Each memory package includes one or more memory die. Therefore, controller102is connected to one or more non-volatile memory die. In one embodiment, each memory die in the memory packages104utilize NAND flash memory (including two-dimensional NAND flash memory and/or three-dimensional NAND flash memory). In other embodiments, the memory package can include other types of memory.

Controller102communicates with host140via an interface (e.g. a data bus120). In one embodiment, data bus120implements a Universal Flash Storage (“UFS”) interface.

Host140is configured to run a software application143that needs to access (e.g., write to and read from) memory device101. To access memory device101, application143communicates with driver144, which is software for enabling communication between application143and memory device101. The software implementing driver144can be executed by a microprocessor in host140. Driver144is in communication with a host controller146(e.g., a microprocessor and software, or other type of processor) that communicates with memory device101via device interface129. In one embodiment, device interface129includes a series of connectors, ports capacitors, etc. for physically connecting to memory device101. Host controller146is also connected to host memory130, which is the host's physical memory and can be DRAM, SRAM, non-volatile memory or another type of storage. Host140is external to and separate from memory device101. In one embodiment, memory device101is embedded in host140. In some embodiments, memory device101is not embedded in host140, but is connected to host140.

Host140is one example of an entity that is external to memory device101. Other examples of an entity that is external to memory device101include other computing devices (e.g., computers, servers, smart appliances, smart phones, etc.) that are connected to memory device101and other computing systems that are in communication with memory device101via any communication means (e.g., LAN, WAN, WiFi, wired connection, wireless connection, direct connection, indirect connection, etc.)

FIG.1Bis a block diagram of one embodiment of FEP circuit141.FIG.1Bshows a PCIe interface150to communicate with host140and a host processor152in communication with that PCIe interface. The host processor152can be any type of processor known in the art that is suitable for the implementation. Host processor152is in communication with a network-on-chip (NOC)154. A NOC is a communication subsystem on an integrated circuit, typically between cores in a SoC. NOCs can span synchronous and asynchronous clock domains or use unclocked asynchronous logic. NOC technology applies networking theory and methods to on-chip communications and brings notable improvements over conventional bus and crossbar interconnections. NOC improves the scalability of SoCs and the power efficiency of complex SoCs compared to other designs. The wires and the links of the NOC are shared by many signals. A high level of parallelism is achieved because all links in the NOC can operate simultaneously on different data packets. Therefore, as the complexity of integrated subsystems keep growing, a NOC provides enhanced performance (such as throughput) and scalability in comparison with previous communication architectures (e.g., dedicated point-to-point signal wires, shared buses, or segmented buses with bridges). Connected to and in communication with NOC154is memory processor156, SRAM160and DRAM controller162. The DRAM controller162is used to operate and communicate with the DRAM (e.g., local memory106). SRAM160is local RAM memory used by memory processor156. Memory processor156is used to run the FEP circuit and perform the various memory operations. Also, in communication with the NOC are two PCIe Interfaces164and166. In the embodiment ofFIG.1B, the SSD controller will include two BEP circuits142; therefore, there are two PCIe Interfaces164/166. Each PCIe Interface communicates with one of the BEP circuits142. In other embodiments, there can be more or less than two BEP circuits142; therefore, there can be more than two PCIe Interfaces.

FIG.1Cis a block diagram of one embodiment of the BEP circuit142.FIG.1Cshows a PCIe Interface200for communicating with the FEP circuit141(e.g., communicating with one of PCIe Interfaces164and166ofFIG.2). PCIe Interface200is in communication with two NOCs202and204. In one embodiment the two NOCs can be combined to one large NOC. Each NOC (202/204) is connected to SRAM (230/260), a buffer (232/262), processor (220/250), and a data path controller (222/252) via an XOR engine (224/254) and an ECC engine (226/256). The ECC engines226/256are used to perform error correction, as known in the art (e.g., encoding data to be written and decoding data that is read). The XOR engines224/254are used to XOR the data so that data can be combined (e.g. combine data) and stored in a manner that can be recovered in case there is a programming error. Data path controller22is connected to an interface module for communicating via four channels with memory packages. Thus, the top NOC202is associated with an interface228for four channels for communicating with memory packages and the bottom NOC204is associated with an interface258for four additional channels for communicating with memory packages. Each interface228/258includes four Toggle Mode interfaces (TM Interface), four buffers and four schedulers. There is one scheduler, buffer and TM Interface for each of the channels. The processor can be any standard processor known in the art. The data path controllers222/252can be a processor, FPGA, microprocessor or other type of controller. The XOR engines224/254and ECC engines226/256are dedicated hardware circuits, known as hardware accelerators. In other embodiments, the XOR engines224/254and ECC engines226/256can be implemented in software. The scheduler, buffer, and TM Interfaces are hardware circuits.

FIG.1Dis a block diagram of one embodiment of a memory package104that includes a plurality of memory die300connected to a memory bus294(command lines, data lines and chip enable lines). The memory bus294connects to a Toggle Mode Interface296for communicating with the TM Interface of a BEP circuit142(see e.g.FIG.1C). In some embodiments, the memory package can include a small controller connected to the memory bus and the TM Interface. The memory package can have one or more memory die. In one embodiment, each memory package includes eight or sixteen memory die; however, other numbers of memory die can also be implemented. The technology described herein is not limited to any particular number of memory die.

FIGS.1A-Dprovide one example architecture of a controller. However, the technology described herein is not limited to any specific form of the controller. Therefore, other architectures can be utilized for the controller. For example, other embodiments of a controller include microprocessors, microcontrollers, state machine, etc. in other configurations. In some cases, the controller can be inside the host. In other cases, the controller can be implemented on the memory die. Other options/configurations can also be used. A controller can also be referred to as a processor, even if it includes multiple processing cores, as the controller operates as a processor for the memory device.

FIG.2is a functional block diagram of one embodiment of a memory die300. Each of the one or more memory die300ofFIG.1Dcan be implemented as memory die300ofFIG.2. The components depicted inFIG.2are electrical circuits. In one embodiment, each memory die300includes a memory structure326, control circuit310, and read/write circuits328, all of which are electrical circuits. Memory structure326is addressable by word lines via a row decoder324and by bit lines via a column decoder332. The read/write circuits328include multiple sense blocks350including SB1, SB2, SBp (sensing circuit) and allow a page (or multiple pages) of data in multiple memory cells to be read or programmed in parallel. In one embodiment, each sense block includes a sense amplifier and a set of latches connected to the bit line. The latches store data to be written and/or data that has been read. The sense blocks include bit line drivers.

Commands and data are transferred between the controller102and the memory die300via lines319. In one embodiment, memory die300includes a set of input and/or output (I/O) pins that connect to lines118.

Control circuit310cooperates with the read/write circuits328to perform memory operations (e.g., write, read, erase, and others) on memory structure326. In one embodiment, control circuit310includes a state machine312, an on-chip address decoder314, a power control circuit316and a temperature detection circuit318. State machine312provides die-level control of memory operations. In one embodiment, state machine312is programmable by software. In other embodiments, state machine312does not use software and is completely implemented in hardware (e.g., electrical circuits). In some embodiments, state machine312can be replaced or augmented by a microcontroller or microprocessor. In one embodiment, control circuit310includes buffers such as registers, ROM fuses and other storage devices for storing default values such as base voltages and other parameters.

The on-chip address decoder314provides an address interface between addresses used by controller102to the hardware address used by the decoders324and332. Power control module316controls the power and voltages supplied to the word lines and bit lines during memory operations. Power control module316may include charge pumps for creating voltages.

For purposes of this document, control circuit310, alone or in combination with read/write circuits328and decoders324/332, comprise a control circuit connected to memory structure326. This control circuit is an electrical circuit that performs the functions described below in the flow charts. In other embodiments, the control circuit can consist only of controller102, which is an electrical circuit in combination with software (e.g., firmware), that performs the functions described below in the flow charts. In another alternative, the control circuit comprises controller102and control circuit310performing the functions described below in the flow charts. In another embodiment, the control circuit comprises state machine312(and/or a microcontroller and/or microprocessor) alone or in combination with controller102. In another alternative, the control circuit comprises controller102, control circuit310, read/write circuits328and decoders324/332performing the functions described below in the flow charts. In other embodiments, the control circuit comprises one or more electrical circuits that operate non-volatile memory.

In one embodiment, memory structure326comprises a monolithic three-dimensional memory array of non-volatile memory cells in which multiple memory levels are formed above a single substrate. The memory structure may comprise any type of non-volatile memory that is monolithically formed in one or more physical levels of arrays of memory cells having an active area disposed above a silicon (or other type of) substrate. In one example, the non-volatile memory cells of memory structure326comprise vertical NAND strings with charge-trapping material such as described, for example, in U.S. Pat. No. 9,721,662, incorporated herein by reference in its entirety. In another embodiment, memory structure326comprises a two-dimensional memory array of non-volatile memory cells. In one example, the non-volatile memory cells are NAND flash memory cells utilizing floating gates such as described, for example, in U.S. Pat. No. 9,082,502, incorporated herein by reference in its entirety. Other types of memory cells (e.g., NOR-type flash memory) can also be used.

In one embodiment, the control circuit(s) are formed on a first die, referred to as a control die, and the memory array is formed on a second die, referred to as a memory die. For example, some or all control circuits (e.g. control circuit310, row decoder324, column decoder332, and read/write circuits328) associated with a memory may be formed on the same control die. A control die may be bonded to one or more corresponding memory die to form an integrated memory assembly. The control die and the memory die may have bond pads arranged for electrical connection to each other. Bond pads of the control die and the memory die may be aligned and bonded together by any of a variety of bonding techniques, depending in part on bond pad size and bond pad spacing (i.e., bond pad pitch). In some embodiments, the bond pads are bonded directly to each other, without solder or other added material, in a so-called Cu-to-Cu bonding process. In some examples, dies are bonded in a one-to-one arrangement (e.g. one control die to one memory die). In some examples, there may be more than one control die and/or more than one memory die in an integrated memory assembly. In some embodiments, an integrated memory assembly includes a stack of multiple control die and/or multiple memory die. In some embodiments, the control die is connected to, or otherwise in communication with, a memory controller. For example, a memory controller may receive data to be programmed into a memory array. The memory controller will forward that data to the control die so that the control die can program that data into the memory array on the memory die.

FIG.3Ais a block diagram depicting one embodiment of the sense block51ofFIG.1or sense block350ofFIG.2. An individual sense block51is partitioned into one or more core portions, referred to as sense circuits180-183or sense amplifiers, and a common portion, referred to as a managing circuit190. In one embodiment, there will be a separate sense circuit for each bit line/NAND string and one common managing circuit190for a set of multiple, e.g., four or eight, sense circuits. Each of the sense circuits in a group communicates with the associated managing circuit via data bus172. Thus, there are one or more managing circuits which communicate with the sense circuits of a set of storage elements.

The sense circuit180, as an example, comprises sense circuit170that performs sensing by determining whether a conduction current in a connected bit line is above or below a predetermined threshold level. Sense circuit180also includes a bit line latch184that is used to set a voltage condition on the connected bit line. For example, a predetermined state latched in the bit line latch will result in the connected bit line being pulled to a state designating program inhibit (e.g., 1.5-3 V). As an example, a flag=0 can inhibit programming, while flag=1 does not inhibit programming.

The managing circuit190comprises a processor192, four example sets of data latches194-197and an I/O interface196coupled between the set of data latches194and data bus120. One set of data latches, e.g., T, C, B, A, can be provided for each sense circuit. In some cases, additional data latches may be used. Each set of latches T, C, B, A may correspond to a different logical page of data for programming together to a set of non-volatile memory cells. This is in a sixteen-level or four-bit per storage element memory device. One additional data latch per bit line can be provided for each additional data bit per storage element.

The processor192performs computations, such as to determine the data stored in the sensed storage element and store the determined data in the set of data latches. Each set of data latches194-197is used to store data bits determined by processor192during a read operation, and to store data bits imported from the data bus120during a program operation which represent write data meant to be programmed into the memory. I/O interface196provides an interface between data latches194-197and the data bus120.

During reading and verifying, the operation of the system is under the control of state machine112that controls the supply of different control gate voltages to the addressed storage element. As it steps through the various predefined control gate voltages corresponding to the various memory states supported by the memory, the sense circuit180may trip at one of these voltages and a corresponding output will be provided from sense circuit180to processor192via bus172. At that point, processor192determines the resultant memory state by consideration of the tripping event(s) of the sense circuit and the information about the applied control gate voltage from the state machine via input lines193. It then computes a binary encoding for the memory state and stores the resultant data bits into data latches194-197. In another embodiment of the managing circuit190, bit line latch serves double duty, both as a latch for latching the output of the sense circuit180and also as a bit line latch as described above.

During program or verify operations for memory cells, the data to be programmed (write data) is stored in the set of data latches194-197from the data bus120. In a four-bit per storage element implementation (Quad-Level Cell or QLC), four data latches (T, C, B, A) may be used. The program operation, under the control of the state machine, comprises a series of programming voltage pulses applied to the control gates of the addressed storage elements. Each program voltage is followed by a read back (verify) to determine if the storage element has been programmed to the desired memory state. In some cases, processor192monitors the read back memory state relative to the desired memory state. When the two are in agreement, the processor192sets the bit line latch so as to cause the bit line to be pulled to a state designating program inhibit. This inhibits the storage element coupled to the bit line from further programming even if program pulses appear on its control gate. In other embodiments the processor initially loads the bit line latch and the sense circuit sets it to an inhibit value during the verify process.

Each set of data latches194-197may be implemented as a stack of data latches for each sense circuit. In one embodiment, there are four or more data latches per sense circuit180. In some implementations, the data latches are implemented as a shift register so that the parallel data stored therein is converted to serial data for data bus120, and vice versa. All the data latches corresponding to the read/write block of storage elements can be linked together to form a block shift register so that a block of data can be input or output by serial transfer. In particular, the bank of read/write circuits is adapted so that each of its set of data latches will shift data in to or out of the data bus in sequence as if they are part of a shift register for the entire read/write block.

The data latches identify when an associated storage element has reached certain mileposts in a program operation. For example, latches may identify that a storage element's Vth is below a particular verify level. The data latches indicate whether a storage element currently stores one or more bits from a page of data.

FIG.3Bdepicts another example block diagram of the sense block51ofFIG.1or sense block350ofFIG.2. The column control circuitry can include multiple sense blocks, where each sense block performs sensing, e.g., read, program verify, or erase verify operations for multiple memory cells via respective bit lines. In one approach, a sense block comprises multiple sense circuits, also referred to as sense amplifiers. Each sense circuit is associated with data latches and caches. For example, the example sense circuits350a,351a,352aand353aare associated with caches350c,351c,352cand353c, respectively.

In one approach, different subsets of bit lines can be sensed using different respective sense blocks. This allows the processing load which is associated with the sense circuits to be divided up and handled by a respective processor in each sense block. For example, a sense circuit controller360can communicate with the set, e.g., sixteen, of sense circuits and latches. The sense circuit controller may include a pre-charge circuit361which provides a voltage to each sense circuit for setting a pre-charge voltage. The sense circuit controller may also include a memory362and a processor363.

FIG.4depicts an example circuit for providing voltages to blocks of memory cells. In this example, a row decoder401provides voltages to word lines and select gates of each block in set of blocks410. The blocks could be in a plane and includes blocks BLK_0to BLK_7. The row decoder provides a control signal to pass transistors422which connect the blocks to the row decoder. Typically, program or read operations are performed on one selected block at a time and on one selected sub-block of the block. An erase operation may be performed on a selected block or sub-block. The row decoder and pass gates can connect global control lines402to local control lines403. The control lines represent conductive paths. Voltages are provided on the global control lines from voltage sources420to pass transistors422. The voltage sources may provide voltages to switches421which connect to the global control lines. Pass transistors424are controlled to pass voltages from the voltage sources420to the switches421.

The voltage sources420can provide voltages on word lines (WL), SGS control gates and SGD control gates, for example. The voltage sources can include a selected word line (WL) driver447, which provides a voltage on a word line selected during a program or read operation, a driver447afor unselected data word lines, and a dummy word line driver447bwhich provides voltages on dummy word lines.

The voltage sources can also include a common SGS driver445and separate SGD drivers for each sub-block. For example, SGD drivers446,446a,446band446ccan be provided for SB0, SB1, SB2and SB3, respectively. In another option, a separate SGS driver is provided for each sub-block.

The various components, including the row decoder, may receive commands from a controller such as the state machine112or the controller122to perform the functions described herein.

A source line voltage source430provides the voltage Vsl to the source lines/diffusion region in the substrate via control lines432. In one approach, the source diffusion region433is common to the blocks. A set of bit lines442is also shared by the blocks. A bit line voltage source440provides voltages to the bit lines.

FIG.5is a perspective view of a memory device500comprising a set of blocks in an example 3D configuration of the memory structure126ofFIG.1or memory structure326ofFIG.2. On the substrate are example blocks BLK0, BLK1, BLK2and BLK3of memory cells (storage elements) and peripheral areas with circuitry for use by the blocks. The peripheral area504runs along an edge of each block while the peripheral area505is at an end of the set of blocks. The pass transistors for a voltage driver of the SGS transistors may be located in this peripheral area505, in one approach. In this case, the blocks BLK0, BLK1, BLK2and BLK3are at progressively further distances from the pass transistors. The circuitry can include voltage drivers which can be connected to control gate layers, bit lines and source lines of the blocks. In one approach, control gate layers at a common height in the blocks are commonly driven. The substrate501can also carry circuitry under the blocks, and one or more lower metal layers which are patterned in conductive paths to carry signals of the circuitry. The blocks are formed in an intermediate region502of the memory device. In an upper region503of the memory device, one or more upper metal layers are patterned in conductive paths to carry signals of the circuitry.

In one possible approach, the blocks are in a plane, and the length of the plane, in the x-direction, represents a direction in which signal paths to word lines extend in the one or more upper metal layers (a word line or SGD line direction), and the width of the plane, in the y-direction, represents a direction in which signal paths to bit lines extend in the one or more upper metal layers (a bit line direction). The z-direction represents a height of the memory device. The blocks could also be arranged in multiple planes.

FIG.6Adepicts an example cross-sectional view of a portion of one of the blocks ofFIG.5. The block comprises a stack610of alternating conductive and dielectric layers. In this example, the conductive layers comprise two SGD layers, one SGS layer, two source-side dummy word line layers (or word lines) WLS1and WLS0, two drain-side dummy word line layers WLD1and WLD0, and eleven data word line layers (or data word lines) WL0-WL10. WL0is a source-side data word line and WLS1is a dummy word line layer which is adjacent to the source-side data word line. WLS0is another dummy word line layer which is adjacent to WLS1. WL10is a drain-side data word line and WLD1is a dummy word line layer which is adjacent to the drain-side data word line. WLD0is another dummy word line layer which is adjacent to WLD1. The dielectric layers are labelled as DL1-DL19. Further, regions of the stack which comprise NAND strings NS1and NS2are depicted. Each NAND string encompasses a memory hole618or619which is filled with materials which form memory cells adjacent to the word lines. Region622of the stack is shown in greater detail inFIG.6C.

The stack includes a substrate611. In one approach, a portion of the source line SL comprises an n-type source diffusion layer611ain the substrate which is in contact with a source end of each string of memory cells in a block. An erase voltage may be applied to this layer in an erase operation The n-type source diffusion layer611ais formed in a p-type well region611b, which in turn is formed in an n-type well region611c, which in turn is formed in a p-type semiconductor substrate611d, in one possible implementation. The n-type source diffusion layer may be shared by all of the blocks in a plane, in one approach.

NS1has a source-end613at a bottom616bof the stack616and a drain-end615at a top616aof the stack. Metal-filled slits617and620may be provided periodically across the stack as interconnects which extend through the stack, such as to connect the source line to a line above the stack. The slits may be used during the formation of the word lines and subsequently filled with metal. A portion of a bit line BL0is also depicted. A conductive via621connects the drain-end615to BL0.

In one approach, the block of memory cells comprises a stack of alternating control gate and dielectric layers, and the memory cells are arranged in vertically extending memory holes in the stack.

In one approach, each block comprises a terraced edge in which vertical interconnects connect to each layer, including the SGS, WL and SGD layers, and extend upward to horizontal paths to voltage drivers.

FIG.6Bdepicts an example variation in the width of the memory holes ofFIG.6Aalong their height. Due to the etching process used to create the memory holes, and the very high aspect ratio, the cross-sectional width, e.g., diameter, of a memory hole can vary along its height. Typically, the diameter becomes progressively smaller from the top to the bottom of the memory hole. In some case, a slight narrowing occurs at the top of the hole, as depicted, so that the diameter becomes slight wider before becoming progressively smaller from the top to the bottom of the memory hole.

FIG.6Cdepicts a close-up view of the region622of the stack ofFIG.6A. Memory cells are formed at the different levels of the stack at the intersection of a word line layer and a memory hole. In this example, SGD transistors718and717are provided above dummy memory cells716and715and a data memory cell714. A number of layers can be deposited along the sidewall (SW) of the memory hole630and/or within each word line layer, e.g., using atomic layer deposition. For example, each pillar699or column which is formed by the materials within a memory hole can include a blocking oxide layer663, a charge-trapping layer664or film such as silicon nitride (Si3N4) or other nitride, a tunneling layer665, a channel660(e.g., comprising polysilicon), and a dielectric core666. A word line layer can include a metal barrier661and a conductive metal662such as Tungsten as a control gate. For example, control gates690-694are provided. In this example, all of the layers except the metal are provided in the memory hole. In other approaches, some of the layers can be in the control gate layer. Additional pillars are similarly formed in the different memory holes. A pillar can form a columnar active area (AA) of a NAND string.

Each NAND string comprises a channel which extends continuously from one or more source-end select gate transistors to one or more drain-end select gate transistors.

FIG.7depicts a voltage signal used in a series of program loops in an example program operation. The horizontal axis denotes a program loop (PL) number, ranging from1-22, and the vertical axis denotes voltage. During a program operation, program loops are performed for a selected word line in a selected block in each plane. A program loop comprises a program portion in which a program voltage or pulse is applied to the selected word line followed by a verify portion in which a verify signal is applied to the selected word line while one or more verify tests are performed for the associated memory cells. Other than the erased state, each assigned state has a verify voltage which is used in the verify test for the state in a program operation.

The voltage signal700includes a series of program voltages, including an initial program voltage701, which are applied to a word line selected for programming. In this example, the voltage signal includes program voltages which increase stepwise in amplitude in one or more program loops of a programming pass using a fixed or varying step size. This is referred to as incremental step pulse programming, where the program voltage starts at an initial level Vpgm_int (see initial program voltage701) and increases in a step in each successive program loop, for instance, until the program operation is completed. A successful completion occurs when the threshold voltages of the selected memory cells reach the verify voltages of the assigned data states.

A program operation can include a single programming pass or multiple programming passes, where each pass uses incremental step pulse programming, for instance.

The verify signal in each program loop, including example verify signal702, can include one or more verify voltages, based on the assigned data states which are being verified for the program loop. The verify tests can encompass lower assigned data states and then midrange assigned data states and then higher assigned data states as the program operations proceeds. The example verify signals depict three verify voltages as a simplification.

All memory cells may initially be in the erased state at the beginning of the program operation, for instance. After the program operation is completed, the data can be read from the memory cells using read voltages which are between the Vth distributions. At the same time, a read pass voltage, Vpass (e.g., 8-10 V), also referred to as pass voltage, is applied to the remaining word lines. By testing whether the Vth of a given memory cell is above or below one or more of the read reference voltages, the system can determine the data state which is represented by a memory cell. These voltages are demarcation voltages because they demarcate between Vth ranges of different data states.

Moreover, the data which is programmed or read can be arranged in pages. For example, with four data states, or two bits per cell, two logical pages of data can be stored together in a page. An example encoding of bits for the Er, A, B and C states is 11, 10, 00 and 01, respectively, in the format of upper page (UP) bit/lower page (LP) bit. A lower page read may use VrA and VrC and an upper page read may use VrB.

With eight data states, or three bits per cell, three pages of data can be stored. An example encoding of bits for the Er, A, B, C, D, E, F and G states is 111, 110, 100, 000, 010, 011, 001 and 101, respectively. Memories that store more than one bit per cell may be referred to as Multi-Level Cell (MLC) memory, which includes Three Level Cell (TLC) memory (storing three bits per cell using eight data states) and Quad Level Cell (QLC) memory (storing four bits per cell using sixteen data states). Memories that store one bit per cell using two data states may be referred to as Single Level Cell (SLC) memory.

FIG.8depicts an example Vth distribution of sets of memory cells after a program operation in a set of memory cells with eight data states, or three bits per cell. The vertical axis depicts a number of memory cells, on a logarithmic scale, and the horizontal axis depicts a Vth of the memory cells on a linear scale. In one approach, at a start of the program operation, the memory cells are all initially in the erased state 1000. After the program operation is successfully completed, the memory cells assigned to the Er state may be upshifted due to some amount of program disturb which normally occurs.

The memory cells which are to be programmed to the A, B, C, D, E, F and G states using verify voltages of VvA, VvB, VvC, VvD, VvE, VvF and VvG, respectively, are represented by the Vth distributions 1001, 1002, 1003, 1004, 1005, 1006 and 1007, respectively. Read voltages VrA, VrB, VrC, VrD, VrE, VrF and VrG can be used for reading the states of the memory cells in a read operation. These verify voltages and read voltages are examples of control gate read levels of the selected word line voltage.

After data is stored for some time in memory cells, the data may become obsolete, may be copied to another location, or for some other reason it may be desirable to erase the memory cells. In many non-volatile memory designs, erase is performed on a block-by-block basis. A subset of memory cells within a block may not be separately erased in such a design (block-erasable memory) so that a block may be considered the minimum unit of erase.

FIG.9shows another example Vth distribution of memory cells after a program operation in a set of memory cells with sixteen data states, or four bits per cell (QLC memory). The vertical axis depicts a number of memory cells, on a logarithmic scale, and the horizontal axis depicts a Vth of the memory cells on a linear scale. In one approach, at a start of the program operation, the memory cells are all initially in the erased distribution S0. Read voltages VrS1to VrS15are illustrated for corresponding distributions S1to S15(corresponding verify voltages are not shown inFIG.9for clarity).

In the examples ofFIGS.8-9, programming may be performed in a single continuous operation starting with all memory cells in the erased distribution (Er or S0) and ending with all or substantially all memory cells verified as being in their target distributions according to data to be stored. In other examples, programming to the states illustrated inFIGS.8-9may occur in two or more separate programming operations that may be performed at different times. Other operations may be performed between such programming operations.

In some cases, programming of one or more memory cells may affect threshold voltage distributions of previously programmed memory cells. For example, programmed memory cells along a word line of a NAND structure may be affected by programming of subsequent word lines of the NAND structure (e.g. programming of memory cells along an adjacent word line in the same block). When charge is added to nearby memory cells, the threshold voltages of previously programmed memory cells may increase so that threshold voltage distributions change in what may be referred to as “program disturb.” This may cause misreading of data. In order to reduce such program disturb effects, programming may be performed in two or more operations to allow programming of adjacent memory cells before programming is finalized. For example, a first programming operation may program a group of memory cells to first distributions that are close to the final distributions in what may be referred to as a foggy programming operation. Then, memory cells of one or more neighboring word line may be programmed. Subsequently, after the neighboring cells are programmed, another program operation (fine programming operation) may program the group of memory cells to second distributions (e.g. final distributions like those shown inFIGS.8-9). Programming to first (approximate or foggy) distributions and subsequently programming to second (accurate or fine) distributions in this way may be referred to as foggy-fine programming.

FIG.10illustrates an example of foggy-fine programming of a group of QLC memory cells using sixteen distributions corresponding to sixteen data states. All memory cells may be in an erased distribution (e.g. distribution S0) prior to foggy programming. Foggy programming programs memory cells to the first distributions S1′ to S15′ shown on the top ofFIG.10. Fine programming subsequently programs the memory cells to the second distributions S1to S15shown on the bottom. There may be some time between these two program operations and one or more other groups of memory cells may be programmed during that time. It will be understood that memory cells of a given first distribution are subsequently further programmed to a corresponding second distribution without erasing memory cells between so that fine programming does not start from the erased distribution. For example, memory cells that are programmed to the S1′ distribution in a foggy program operation are subsequently further programmed to the S1distribution in a fine program operation, memory cells that are programmed to the S2′ distribution in a foggy program operation are subsequently further programmed to the S2distribution in a fine program operation, memory cells that are programmed to the S3′ distribution in a foggy program operation are subsequently further programmed to the S3distribution in a fine program operation, and so on.

First distributions S1′ to S15′ are generally wider than second distributions S1-S15and there is significant overlap between adjacent distributions (e.g. distribution S1′ overlaps distribution S2′, distribution S2′ overlaps distribution S3′ and so on). Programming to first distributions may use the same programming steps as used for second distributions or may use different steps (e.g. programming pulses of the same voltage and time or different voltage and/or time). Reading memory cells that are in the first distributions S1′ to S15′ using read voltages as shown inFIG.9may provide a large number of errors (e.g. more errors that can be corrected by ECC) because of such overlaps so that another source may be used for fine programming. A safe copy of the data may be maintained in another location between foggy program and fine program operations so that a good copy is available for fine programming. For example, a copy of the data may be written in additional non-volatile memory cells (e.g. in SLC memory cells, which may be in the same memory structure as the QLC memory cells, or elsewhere) or in a volatile memory for subsequent fine programming.

Between foggy programming and fine programming of a given word line, one or more other word lines (e.g. nearby word lines that may have some coupling with the given word line) may be programmed.FIG.11shows an example of a programming scheme that may be used to program word lines of four strings of a 3D NAND structure (e.g. NAND strings such as NS1and NS2illustrated inFIG.6A). The first column on the left indicates that word line (WL) and the next column (STAGE) indicates the programming stage as either FOGGY or FINE for each of the four strings STR0to STR3. Numbers at each entry indicate the order of foggy and fine programming of different strings and word lines. Arrows are used to indicate order of some steps.

Programming starts with foggy programming of WL0of STR0, STR1, STR2and STR3, followed by foggy programming of WL1, STR0, and then fine programming of WL0, STR0. This is followed by foggy programming WL1, STR1, then fine programming WL0, STR1, foggy programming WL1, STR2, fine programming WL0, STR2, foggy programming WL1, STR3, fine programming WL0, STR3, followed by foggy programming of WL2, STR0, and so on. It can be seen that between foggy programming and fine programming of a given group of memory cells, other memory cells are programmed (e.g. foggy programmed) so that fine programming occurs after coupling effects of neighboring cells are already in place and thus program disturb effects are reduced. For example, while foggy programming of WL1, STR0is the fourth program operation inFIG.11, fine programming of WL1, STR0is the thirteenth program operation so that nine other program operations occur in between. A copy of data that is foggy programmed and has not yet been fine programmed may be saved in a safe location until fine programming is complete (e.g. a safe copy of data for WL1, STR0may be maintained between the fourth and thirteenth operations ofFIG.11). Maintaining a safe copy may take significant resources both in terms of storage space (e.g. in volatile or non-volatile memory) and in terms of bus usage to transfer the data.

In an example of the present technology, parity data may be calculated for data to be stored in MLC memory cells and this parity data may allow data that has been foggy programmed to be recovered (e.g. recovered by combining the results of reading foggy data with the parity data to obtain the original data without requiring a safe copy).

FIG.12illustrates an example of a gray code memory state encoding scheme used with parity data to facilitate recovery of data from memory cells that are foggy programmed. The encoding scheme illustrated may be applied to distributions shown in any ofFIGS.8-10. Distributions S0to S15(column1) are each assigned a different four-bit digital value (column2), or memory state, according to a grey code assignment scheme so that digital values assigned to any adjacent distributions differ by only one bit. For example, the digital values assigned to distribution S4(1100) and neighboring distribution S3(1101) have the first three bits the same (110) and only differ in the last bit. Similarly, the digital values assigned to distribution S4(1100) and neighboring distribution S5(0100) have the last three bits the same (100) and only differ in the first bit. Parity data (column3) consists of one parity bit for each assigned digital value, which in this case is simply obtained by an Exclusive OR (XOR) operation on the bits of the digital value. Because a grey code is used for assignment of memory states, parity bits alternate 0-1-0-1 as shown. In other examples, other parity schemes (or different encoding schemes) may be used. While the digital values are shown mapped to distributions S0-S15, which may be achieved after fine programming, it will be understood that the same digital values are similarly mapped to first distributions S0-S15′, which may result from foggy programming. Thus, for example, both S3and S3′ distributions are mapped to digital value 1101, both S12and S12′ distributions are mapped to digital value 1010, etc. Use of a gray code assignment scheme with parity data may facilitate recovery of data from memory cells that are foggy programmed (e.g. memory cells that have not yet been fine programmed and are in data states S1′-S15′, not in states S1-S15).

FIG.13Aillustrates an example of how a suitable gray code assignment with parity data (e.g. as illustrated inFIG.12or using a different gray code) may be used to facilitate accurate recovery of data from memory cells that are foggy programmed.FIG.13Ashows distributions S1′-S15′ (in addition to erased distribution S0), with one-bit parity data indicated above each distribution (either a 1 or 0 value). This corresponds to memory cells that are in a foggy programmed condition (e.g. while neighboring memory cells are being programmed) with parity data generated e.g. as shown inFIG.12. A series of read steps may be performed at read levels RL1to RL14as illustrated inFIG.13A. Read levels RL1to RL14are not located between distributions as in previous examples (e.g. not like VrA-VrG ofFIG.8or VrS1-VrS15ofFIG.9). Instead, each read level RL1to RL14is located at or near the middle (midpoint) of a distribution, or about halfway between distributions on either side. For example, RL3is about halfway between distribution S2′ and distribution S4′, which is in the middle of distribution S3′. Thus, RL3is between two distributions (S2′ and S4′) with even parity values (parity value=0) at or near the middle of a distribution (S3′) with an odd parity value (parity value=1). Read levels RL1to RL14are located at or near the middle of distributions S1to S14(i.e. at or near the middle of all distributions except the lowest distribution, S0, and highest distribution, S15).

Reading foggy-programmed data states S1′ to S15′ at the fourteen read levels RL1-RL14in combination with parity values shown (which alternate according to the gray code used) may be sufficient to accurately recover data programmed in a foggy programming operation. In some cases, this may allow data that is foggy programmed to be directly recovered from foggy programmed memory cells so that a safe copy does not have to be maintained elsewhere (e.g. may not require a safe copy in SLC or in volatile memory).

Memory cells that turn on (become conductive or undergo a tripping event, e.g. as detected by a sense block) when RL1is used (e.g. when a voltage at RL1is applied to a corresponding word line) may be in either the S0or S1′ distributions. Which of these distribution a given cell is in may be determined from the parity bit associated with the cell (e.g. any cell that turns on with RL1and has parity=0 is in distribution S0, while any cell that turns on with RL1and has parity=1 is in distribution S1′). In this way, all memory cells in the S0distribution (and some memory cells in the S1′ distribution) may be found from a read at RL1. Memory cells that do not turn on with RL1and turn on with RL2(tripping event detected between RL1and RL2) may be in either S1′ or S2′ distributions. Which of these distribution a given cell is in may be determined from the parity bit associated with the cell (e.g. any cell that first turns on with RL2and has parity=1 is in distribution S1′, while any cell that first turns on with RL2and has parity=0 is in distribution S2′). In this way, remaining memory cells in the S1′ distribution (and some memory cells in the S2′ distribution) may be found from a read at RL2. Memory cells that do not turn on with RL1or RL2and turn on with RL3may be in either S2′ or S3′ distributions. Which of these distribution a given cell is in may be determined from the parity bit associated with the cell (e.g. any cell that first turns on with RL3and has parity=0 is in distribution S2′, while any cell that first turns on with RL3and has parity=1 is in distribution S3′). In this way, remaining memory cells in the S2′ distribution (and some memory cells in the S3′ distribution) may be found from a read at RL3. This approach may extend through RL14to identify all cells in S0and S1′-S13′ and some cells in S14′. Memory cells that do not turn on (do not experience a tripping event) when RL14is applied are either in distribution S14′ or S15′ and these may be distinguished by respective parity bits (memory cells that do not turn on with RL14and have parity=0 are in distribution S14′ while memory cells that do not turn on with RL14and have parity=1 are in distribution S15′).

FIGS.13B-Dillustrates implementations of reading QLC cells that have been foggy programmed as illustrated inFIG.13A(e.g. memory cells in a NAND structure as illustrated inFIGS.6A-Cor other memory structure).FIG.13Billustrates an assignment scheme that assigns digital values to sixteen distributions S0-S15using a gray code to ensure that adjacent distributions differ by only one bit. Distributions S0-S15are listed in the first column with digital values (or memory states) listed in the second column and parity values corresponding to each digital value listed in the third column. Parity values may be obtained by XORing the four bits of the corresponding digital value. The scheme illustrated inFIG.13Bmay be an alternative to the scheme illustrated inFIG.12(other encoding schemes, including different parity schemes, may also be used). Encoding data using the parity values shown inFIG.13Bmay allow subsequent recovery of data that is foggy programmed.

FIG.13Cillustrates how reading memory cells at read levels RL1-RL14as illustrated inFIG.13Amay be combined with parity bits to recover data that has been foggy programmed. Read levels are listed in the first column with initial values (four-bit digital values) listed in the second column. For each cell with an initial value in the second column, the recovered value (decoded value) depends on the parity bit for the cell. Parity bits are listed in the third column and corresponding recovered values are listed in the fourth column. Decoding foggy-read data (e.g. initial values) using parity values to obtain recovered values may make maintaining a safe copy of foggy-programmed data unnecessary in some cases.

Memory cells that turn on at RL1may be assigned an initial value 1111 (corresponding to distribution S0in the table ofFIG.13B). If the corresponding parity bit=0 then the recovered value is also 1111 (e.g. memory cell is in distribution S0) and if the parity bit=1 then the recovered value is 1110 (e.g. memory cell is in distribution S1′ and the fourth bit is flipped). Memory cells that do not turn on at RL1and turn on at RL2may be assigned an initial value 1110 (corresponding to distribution S1or S1′ in the table ofFIG.13B). If the corresponding parity bit=1 then the recovered value is also 1110 (e.g. memory cell is in distribution S1′ and the initial value is used as the recovered value) and if the parity bit=0 then the recovered value is 1010 (e.g. memory cell is in distribution S2′ and the second bit is flipped). Memory cells that do not turn on at RL1or RL2and turn on at RL3may be assigned an initial value 1010 (corresponding to distribution S2or S2′ in the table ofFIG.3B). If the corresponding parity bit=0 then the recovered value is also 1010 (e.g. memory cell is in distribution S2′ and the initial value is used as the recovered value) and if the parity bit=1 then the recovered value is 1000 (e.g. memory cell is in distribution S3′ and the third bit is flipped). This approach continues as shown inFIG.13Cuntil RL14, where memory cells that first turn on at RL14are assigned an initial value of 0111 (corresponding to distribution S13or S13′ in the table ofFIG.13B) and, depending on the corresponding parity values, are assigned recovered values of either 0111 (if parity=1) or 0011 (if parity=0). Remaining memory cells that do not turn on at RL14(indicated by “Remaining” inFIG.13C) are assigned an initial value of 0011 (corresponding to distribution S14or S14′) and, depending on the corresponding parity values, are assigned recovered values of either 0011 (if parity=0), corresponding to distribution S14, or 1011 (if parity=1), corresponding to distribution S15.

Foggy reading with parity data to recover values may be implemented in various ways.FIG.13Dillustrates an example that applies logic operations to the initial value (bits T, C, B, and A) and parity bit (S) to flip a bit when indicated by the corresponding logic operation. For example, in the first line, the initial value 1111 may occur with parity bit0or1. If the parity bit is 1 (T&C&B&A&S), then bit A is flipped to 0 (A=0) and otherwise the initial value 1111 is used as the recovered value. In the second line, the initial value 1110 may occur with parity bit0or1. If the parity bit is 0 (T&C&B&!A&! S, where “!” indicates the inverse so !A is the inverse of A, i.e. if A=0, !A=1 and if A=1, !A=0), then bit C is flipped to a 0 (C=0) and otherwise the initial value 1110 is used as the recovered value. In the third line, the initial value 1010 may occur with parity bit0or1. If the parity bit is 1 (T&!C&B&!A&S), then bit B is flipped to a 0 (B=0) and otherwise the initial value 1010 is used as the recovered value. It can be seen that for the sixteen values illustrated for a QLC memory, fifteen logic operations may be sufficient to obtain the recovered values. Because a grey code is used, for any given initial value, there is only one bit to flip depending on the parity bit. The bits to be flipped are shaded for illustration.

Simple logic operations such as AND operations may be implemented on a memory die. In some cases, such logic operations may be applied while data is held in data latches such as data latches194-197(e.g. with additional latches for parity data). Bits may be flipped according to the scheme while data remains in latches (e.g. prior to being used to write the recovered values in a fine write operation). Reading foggy programmed data as illustrated inFIG.13Amay result in initial values being latched in data latches194-197. When this data is combined with parity values (e.g. using logic operations illustrated inFIG.13D) the recovered values may be obtained by flipping bits while the data remains in data latches194-194. This recovered data may then be written back from data latches194-197to the same memory cells in a fine write operation so that transfer of initial values and recovered values to other circuits (e.g. outside a memory die) is unnecessary.

FIG.13Eillustrates an operation to recover data from memory cells that are foggy programmed by using parity data (e.g. as in the example ofFIGS.13A-Dabove). The method includes performing a read with mid-distribution read levels1302(e.g. RL1-RL14as illustrated inFIG.13A), assigning initial values according to read results1304(e.g. initial values corresponding to RL1-RL14as illustrated inFIG.13C), obtaining parity data corresponding with read data1306(parity data may be generated prior to foggy programming and may be saved for subsequent use), and combining parity data with read results to obtain recovered data1308(e.g. by flipping a bit of an initial value when indicated by a parity bit as illustrated inFIGS.13C-D).

Recovery of data from foggy programmed memory cells (decoding) as illustrated inFIGS.13A-Duses parity data in combination with results of a foggy read operation. Parity data may be calculated when the data is written, or before it is written, and may be stored in an appropriate location. For example, parity data may be calculated in a memory controller before it is sent to a memory die and may be stored in a volatile memory (e.g. DRAM) until it is needed to recover foggy programmed data.

FIG.14illustrates an example in which memory device101(e.g. as shown inFIG.1A) is modified to implement encoding and decoding of data from memory cells that are foggy programmed in QLC storage1412in a non-volatile memory die1414(e.g. an individual non-volatile memory die of non-volatile memory104). Controller102includes FEP circuit141and BEP circuit142including SRAM1416. FEP circuit141is connected to BEP circuit142via a bus1418(e.g. PCIe interface). BEP circuit142includes an ECC circuit1420(e.g. ECC engine226/256), which may implement Low Density Parity Check (LDPC) encoding and decoding), a parity circuit, which in this case is an XOR circuit1413(e.g. XOR circuit1413may be implemented by XOR engine224/254) and a decoder circuit1422(e.g. implemented using a processor220/250). Controller102is connected to non-volatile memory die1414through bus1424(e.g. memory bus294).

Data is received from a host at input1428and is transferred from SRAM1416, through bus1418to ECC circuit1420which encodes the data and transfers it1430, through bus1424for foggy programming in QLC storage1412and also sends it to XOR circuit1413, which calculates XOR bits that are saved in local memory (DRAM106in this example). Subsequently, other writes may occur while the data remains foggy programmed and at some later point in a programming sequence (e.g. sequence illustrated inFIG.11) the data may be fine programmed. At the time of fine programming, a foggy read (e.g. as illustrated inFIGS.13A-D) may be performed and the results transferred1434to decoder1422(e.g. initial values illustrated inFIG.13Cmay sent from read/write circuits of non-volatile memory die1414to decoder1422). Decoder1422also receives XOR bits from DRAM106and combines the foggy read results with the XOR bits to obtain recovered data (e.g. as illustrated inFIG.13C). These results are then transferred1434and written in the same memory cells in a fine program operation (e.g. same memory cells that were foggy programmed are further programmed).

It can be seen that this example includes transfer of significant data between components (e.g. between controller102and memory die1414over bus1424and between FEP circuit141and BEP circuit142over bus1418). For example, in QLC storage, four logical pages of data may be stored together and the results of XORing four such pages is a page of parity data. In the scheme illustrated inFIG.14, programming such data includes transfer1430of the four logical pages of data, subsequent transfer1432(through bus1424) of four pages obtained from a foggy read to controller102, and transfer1434of four pages of recovered data back to memory die1414for fine programming. This may result in significant traffic on bus1424especially where the bus is shared between multiple memory dies (e.g. as illustrated inFIG.1D). Furthermore, significant space may be occupied in local memory106by XOR data (e.g. one logical page of XOR data for every four logical pages of data that are foggy programmed). Accessing XOR data in DRAM106may produce significant traffic on bus1418.

FIG.15Ashows an example of a memory device1530, which is configured to perform on-chip encoding and decoding for foggy-fine programming. Memory die1532includes parity data generating circuits, XOR circuits1540in this example, which are configured to generate parity data (XOR data) in memory die1532. XOR circuits1540may be considered as means for calculating parity data for data to be stored in non-volatile memory cells of QLC storage1412. For example, when data is transferred1536to memory device1530, the data may be transferred1538(along with any ECC data) from controller102, through bus1424to QLC storage1412and also to XOR circuit1540of memory die1532. The data may be foggy programmed in QLC storage1412and may also be used to generate parity data (XOR data in this example). The parity data may be saved while the data is foggy programmed until fine programming is initiated. Memory die1532also includes on-chip decoder1542, which is configured to receive foggy read data from QLC storage1412(transfer1544) and to decode the foggy read data using XOR data previously generated by XOR circuit1540. The decoded data that is recovered in this way is transferred1546to QLC storage1412in a fine programming operation. Generating parity data on-chip and performing decoding on-chip in this way may allow foggy-fine programming to be performed without a safe copy of data (e.g. using the combination of foggy programmed data and parity data instead of a safe copy) and with relatively little traffic on bus1424. For example, when four logical pages of data are to be written, the four logical pages are transferred1538to memory die1532. Subsequently, four logical pages of foggy read data are transferred to on-chip decoder1542, which does not generate traffic on bus1424, and recovered data is transferred1546back to QLC storage1412also without generating traffic on bus1424. In some cases, XOR data may be saved in SLC storage in memory die1532so that parity data may be saved and recovered without generating traffic on bus1424(in other examples, XOR data from XOR circuit1540may be saved off-chip, e.g. in memory controller102or in a volatile memory connected to memory controller102).

FIG.15Billustrates an example implementation of on-chip XOR circuits1540, which includes four sets of data latches1550-1553corresponding to four logical pages of data to be programmed in a physical page of QLC storage1412(logical pages labeled “T”, “C”, “B”, and “A” respectively). For example, each set of latches may have capacity for one logical page of data from a host along with ECC data (e.g. calculated by ECC circuits of memory controller102) and any other overhead data that may be stored with data from a host. XOR circuits1540includes XOR logic circuit1554, which is configured to perform an XOR logic operation on each set of bits corresponding to a memory cell of QLC storage1412and generate a corresponding bit of XOR data (parity data). XOR logic circuit1554may be considered an example of means for calculating parity data for data to be stored in the plurality of non-volatile memory cells. Parity data is stored in a set of data latches1556. An example set of bits1558are to be programmed to a memory cell of QLC storage1412. XOR logic circuit1554calculates a corresponding parity bit (in this case “1”) corresponding to bits1101of set of bits1558. For example, modulo 2 addition or other technique may be used to generate XOR bits (e.g. where there is an odd number of “1” bits, the XOR result is “1”). Where parity bits are generated in other ways (not XOR) different logic may be applied to generate parity bits. Data latches1550-1553may be implemented by data latches194-197ofFIG.3Aand encoding (calculation of XOR bits in this example) may be performed when data is in latches before or during foggy programming. Data latches1550-1553(e.g. data latches194-197) in combination with additional circuits of sense block51, may be considered an example of a means for programming the plurality of non-volatile memory cells to first distributions.

FIG.15Cshows an example of on-chip decoder1542, which includes four sets of data latches1560-1563corresponding to four logical pages of data obtained from a foggy read of a page of QLC storage1412. For example, each set of latches may have capacity for one logical page of data along with ECC data and any other overhead data that may be stored with data received from a host. In some examples, the same physical data latches may be used as XOR data latches1550-1553and for decode data latches1560-1563(e.g. data latches194-197). When a foggy read is performed (e.g. as illustrated inFIG.13A) initial data states may be loaded in data latches1560-1563(e.g. initial values listed inFIG.13C). Data latches1560-1563(e.g. data latches194-197) in combination with additional circuits of sense block51, may be considered an example of a means for reading the plurality of non-volatile memory cells in the first distributions. On-chip decoder1542include decode logic circuit1564, which combines the initial values obtained from such a foggy read with parity data (e.g. XOR data generated by XOR circuits1540prior to foggy programming) in a set of data latches1566. Decode logic circuit1564may combine foggy read data and parity data as illustrated inFIGS.13C-Dso that an initial value from a memory cell may be used as a recovered value, or one bit may be flipped, depending on the parity value associated with the memory cell. For example, decode logic circuit1564may include AND logic and NOR logic circuits to perform the logic operations illustrated inFIG.13D. In a first example1568, foggy reading of a memory cell provides an initial value 1101 (e.g. memory cell becomes conductive at RL12) and the corresponding XOR bit for the memory cell is 1. Decode logic circuit1564generates a recovered value of 1101 in this case (as shown in the corresponding entries ofFIG.13C-D). In a second example1570, foggy reading of another memory cell provides an initial value 1101 and the corresponding XOR bit for the memory cell is 0. Decode logic circuit1564generates a recovered value of 0101 in this case (as shown in the corresponding entries ofFIG.13C-D). Decode logic circuit1564may simply flip bit T (shaded) to 0 according to the logic operation illustrated inFIG.13D(If T&C&!B&A&!S then set T=0) so that the initial value is replaced with the recovered value while in latches1560-1563. Decode logic circuit1564is configured to generate the recovered values ofFIGS.13C-Dfrom corresponding initial values obtained by foggy reading of memory cells of QLC storage1412in combination with XOR bits. These recovered values may then be used to perform fine programming of memory cells (e.g. the same memory cells that were foggy programmed are further programmed as illustrated inFIG.10based on recovered values). Data may be fine programmed from data latches1560-1563(e.g. data latches194-197). On-chip decode1542may be considered as an example of a means for recovering the data from results of reading the plurality of non-volatile memory cells in the first distributions combined with the parity data and further programing the plurality of non-volatile memory cells from the first distributions to second distributions to store the data.

FIG.16illustrates an example of a series of steps used in programming. In Step1, four logical pages of data are transferred to a memory die (e.g. memory die1532) and are loaded in corresponding sets of data latches (e.g. latches T, C, B, A). In step2, the four logical pages are subject to an XOR operation to generate an XOR bit (or flag “A”) for each memory cell to be programmed. In the example ofFIG.16, this takes 25 us. In step3, the XOR bits (one logical page of XOR data generated from four logical pages of transferred data) are written in SLC memory (e.g. written in a portion of NAND memory on memory die1532that is designated as SLC). In the example ofFIG.16, this takes 130 us. Thus, on-chip encoding (by XORing) and storing of XOR data in SLC may be rapidly performed. It will be understood that SLC portions and QLC portions may be separate structures or may both be formed in a common structure (e.g. some blocks of a structure may be designated as QLC blocks and others as SLC blocks, and these designations may change over time).

On-chip encoding and decoding for foggy-fine programming as described in any of the examples above may be implemented in various ways including, but limited to, the examples ofFIGS.17-19.FIG.17shows an example in which SLC memory cells are used to store parity data (e.g. XOR data) for later use in fine programming. A data transfer1770(e.g. from a host) is received by memory controller102and the data is ECC encoded by ECC circuit1772. Encoded data is then transferred1774, via bus1424, to QLC storage1412where it is foggy programmed, and also transferred to XOR circuits1540, which generate XOR bits that are then stored in SLC cache1776. SLC cache1776is a portion of structure1778(e.g. 3-D NAND memory structure) that is configured for SLC data storage. Structure1778also includes QLC storage1412and these portions may be reconfigurable. After foggy programming, while the data is in QLC storage1412and corresponding XOR data is in SLC cache1776, other data may be programmed (e.g. as illustrated inFIG.11). Subsequently, fine programming may be scheduled, and a foggy read may be performed to read initial values from QLC storage1412to decoder1524(transfer1780) and XOR data may be read from SLC cache1776(transfer1782). Decoder1542then generates recovered values from the combination of foggy read data and XOR data (e.g. decoding as illustrated inFIG.13C). These recovered values are then used to perform fine programming of the memory cells that were previously foggy programmed (transfer1784). Thus, in this example, the only transfer of data across bus1424is when the initial data (e.g. four logical pages of user data and any overhead) is transferred to memory die1532. Subsequent transfers of XOR data, foggy read data, and recovered data occur internally in memory die1532.

FIG.18illustrates another example in which SLC memory cells are used to store parity data (XOR data) for later use in fine programming. A data transfer1770(e.g. from a host) is received by memory controller102and the data is ECC encoded by ECC circuit1772. Encoded data is then transferred1774, via bus1424, to QLC storage1412where it is foggy programmed, and also transferred to XOR circuits1540, which generate XOR bits that are then stored in SLC cache1776. Subsequently, fine programming may be scheduled, and a foggy read may be performed to read initial values from QLC storage1412to decoder1524(transfer1780). In this example, XOR data is not directly transferred from SLC cache1776to decoder1542. XOR data from SLC cache1776is transferred to ECC circuit1772of memory controller102, via bus1424(transfer1880) where error correction is performed. Corrected XOR data (decoded parity data) is transferred, via bus1424, to decoder1542(transfer1882). ECC circuit1772may be considered an example of a means for Error Correction Code (ECC) decoding parity data and sending decoded parity data to decoder1542. Decoder1542then generates recovered values from the combination of foggy read data and corrected XOR data (e.g. as illustrated inFIG.13C). These recovered values are then used to perform fine programming of the memory cells that were previously foggy programmed (transfer1784). Thus, in this example, in addition to the initial transfer1774(e.g. four logical pages) across bus1424, there is also transfer of XOR data (e.g. one logical page for the four logical pages of data) from memory die1532to memory controller102and transfer of corrected XOR data back from memory controller102to memory die1532on bus1424. Because errors in XOR data may result in errors in recovered data and therefore errors in fine programmed data, reduction or elimination of such errors using ECC may justify the additional traffic on bus1424.

While parity data may be separately encoded by ECC circuit1772prior to storage in SLC cache1776, in some cases no separate encoding is required where, for example, the parity data is an XOR product of encoded data (e.g. XOR product of codewords may provide a codeword so that separate encoding is not required). ECC correction of parity data may be applied to all data or may be selectively used. For example, ECC correction may be used for selected blocks (blocks with high errors), selected portions of a given block (e.g. lower levels of a 3-D block), or in response to a triggering event (e.g. after a certain number of write-erase cycles, a temperature above a threshold, or other such event). Thus, memory device1530may use both the scheme ofFIG.17and the scheme ofFIG.18at different times and/or for data in different portions of QLC storage1412.

FIG.19illustrates an example in which local memory, DRAM106, is used to store parity data (XOR data) for later use in fine programming. A data transfer1770(e.g. from a host) is received by memory controller102and the data is ECC encoded by ECC circuit1772. Encoded data is then transferred1774, via bus1424, to QLC storage1412where it is foggy programmed, and also transferred to XOR circuits1540, which generate XOR bits that are then transferred to DRAM106(transfer1990). Subsequently, fine programming may be scheduled, and a foggy read may be performed to read initial values from QLC storage1412to decoder1524(transfer1780). XOR data is transferred from DRAM106to decoder1542(transfer1992). Decoder1542then generates recovered values from the combination of foggy read data and XOR data (e.g. as illustrated inFIG.13C). These recovered values are then used to perform fine programming of the memory cells that were previously foggy programmed (transfer1784). Thus, in this example, in addition to the initial transfer1774(e.g. four logical pages) across bus1424, there is also transfer of XOR data (e.g. one logical page for the four logical pages of data) from memory die1532to DRAM106and transfer of XOR data back from DRAM106to memory die1532on bus1424.

While in some examples described, encoding and decoding for foggy-fine programming is performed on-chip (e.g. by encoding and decoding circuits formed on the same die as the memory structure in which data is programmed), in other examples, encoding and decoding circuits may be located in a control die that is bonded to a memory die in an integrated memory assembly. For example, XOR circuits and decoder circuits ofFIGS.15A, and17-19may be located in such a control die.

FIG.20is a flowchart illustrating an example of a method that implements aspects of the present technology. The method includes receiving, by a memory die, a plurality of logical pages of data to be stored in a page of Multi-Level Cell (MLC) memory2002, calculating, by circuits of the memory die, parity data for the plurality of logical pages2004(e.g. calculating a page of XOR bits for four logical pages of user data to be stored), and programming the plurality of logical pages of data in the page of MLC memory in a foggy program operation such that memory cells of the page of MLC memory are programmed to first distributions2006. The method further includes reading the page of MLC memory while the memory cells are in the first distributions in a foggy read operation2008(e.g. as illustrated inFIG.13A) calculating, by circuits of the memory die, the plurality of logical pages of data from results of the foggy read operation and the parity data2010(e.g. calculating recovered values ofFIG.13C); and further programming the page of MLC memory from the first distributions to second distributions in a fine programming operation (e.g. as illustrated inFIG.10), the second distributions representing the plurality of logical pages calculated from the parity data2012. The method may include storing the parity data in the memory die in additional memory cells that are configured as Single Level Cell (SLC) cells and may further include reading the parity data from the SLC cells and performing Error Correction Code (ECC) decoding of the parity data prior to calculating the plurality of logical pages from the parity data, or may include sending the parity data from the memory die to be stored in a volatile memory and subsequently receiving the parity data from the volatile memory for the calculating (not illustrated inFIG.20).

An example of a non-volatile storage apparatus, includes a plurality of non-volatile memory cells formed on a memory die, each non-volatile memory cell configured to hold a plurality of bits of data and a control circuit formed on the memory die, the control circuit configured to calculate parity data for data to be stored in the plurality of non-volatile memory cells, program the plurality of non-volatile memory cells to first distributions, read the plurality of non-volatile memory cells in the first distributions, recover the data from results of reading the non-volatile memory cells in the first distributions combined with the parity data, and further program the plurality of non-volatile memory cells from the first distributions to second distributions to store the data.

The control circuit may be further configured to program the parity data in additional non-volatile memory cells formed on the memory die, each additional non-volatile memory cell configured to hold one bit of data. The control circuit may be further configured to read the parity data from the additional non-volatile memory cells and send the parity data for Error Correction Code (ECC) decoding. The control circuit may be further configured to send the parity data for storage in volatile memory on a control die that is connected to the memory die and receive the parity data from volatile memory. The plurality of non-volatile memory cells may be located along a first word line of a NAND structure of the memory die, the control circuit further configured to program at least a second word line of the NAND structure between programming the plurality of non-volatile memory cells to the first distributions and further programming the plurality of non-volatile memory cells from the first distributions to the second distributions. The first distributions may consist of sixteen distributions representing four bits of data and the control circuit may be further configured to read the plurality of non-volatile memory cells in the first distributions using fourteen read voltages located at or near the middle of first distributions other than a lowest and a highest first distributions. The control circuit may be configured to program the plurality of non-volatile memory cells according to a Gray code such that neighboring distributions are assigned digital values that differ by one and only one bit. The control circuit may be configured to program the plurality of non-volatile memory cells to the first distributions in a foggy program operation such that first distributions have significant overlap and program the plurality of non-volatile memory cells from the first distributions to the second distributions in a fine programming such that the second distributions are narrower than the first distributions and have less overlap than the first distributions. The control circuit may be configured to calculate one logical page of parity data for four logical pages of data to be stored in the plurality of non-volatile memory cells, write the logical page of parity data in additional non-volatile memory cells formed on the memory die or send the logical page of parity data for storage outside the memory die, and subsequently read the logical page of parity data from the additional non-volatile memory cells or receive the logical page of parity data from outside the memory die for recovery of the data.

An example of a method includes receiving, by a memory die, a plurality of logical pages of data to be stored in a page of Multi-Level Cell (MLC) memory; calculating, by circuits of the memory die, parity data for the plurality of logical pages; programming the plurality of logical pages of data in the page of MLC memory in a foggy program operation such that memory cells of the page of MLC memory are programmed to first distributions; reading the page of MLC memory while the memory cells are in the first distributions in a foggy read operation; calculating, by circuits of the memory die, the plurality of logical pages of data from results of the foggy read operation and the parity data; and further programming the page of MLC memory from the first distributions to second distributions in a fine programming operation, the second distributions representing the plurality of logical pages calculated from the parity data.

Calculating the parity data for the plurality of logical pages may include calculating one parity bit for each memory cell of the page of MLC memory. Calculating the parity data may include performing an Exclusive OR (XOR) operation on the plurality of logical pages. The method may include storing the parity data in the memory die in additional memory cells that are configured as Single Level Cell (SLC) cells. The method may further include reading the parity data from the SLC cells and performing Error Correction Code (ECC) decoding of the parity data prior to calculating the plurality of logical pages from the parity data. The method may further include sending the parity data from the memory die to be stored in a volatile memory and subsequently receiving the parity data from the volatile memory for the calculating. The method may further include, subsequent to the foggy programming operation and prior to the foggy read operation, programming at least one other page of the MLC memory. The page of MLC memory may be along a first word line of a NAND memory structure and the at least one other page of the MLC memory may be along at least a second word line of the NAND memory structure. Reading the page of MLC memory while the memory cells are in the first distributions may include using read levels at or near midpoints of the first distributions.

An example of a non-volatile storage apparatus includes a memory die including a plurality of non-volatile memory cells and further including: means for calculating parity data for data to be stored in the plurality of non-volatile memory cells; means for programming the plurality of non-volatile memory cells to first distributions; means for reading the plurality of non-volatile memory cells in the first distributions; and means for recovering the data from results of reading the plurality of non-volatile memory cells in the first distributions combined with the parity data and further programing the plurality of non-volatile memory cells from the first distributions to second distributions to store the data.

The non-volatile storage apparatus may include a memory controller die connected to the memory die, the memory controller die including means for Error Correction Code (ECC) decoding the parity data and sending decoded parity data to the means for recovering the data.

The foregoing detailed description of the invention has been presented for purposes of illustration and description. It is not intended to be exhaustive or to limit the invention to the precise form disclosed. Many modifications and variations are possible in light of the above teachings. The described embodiments were chosen in order to best explain the principles of the invention and its practical application, to thereby enable others skilled in the art to best utilize the invention in various embodiments and with various modifications as are suited to the particular use contemplated. It is intended that the scope of the invention be defined by the claims appended hereto.