# HAIPO: Hybrid AI Algorithm-Based Post-Fabrication Optimization for Modern 3D NAND Flash Memory
## 3 Overview of NAND Post-Fabrication Optimization
### Limitation of Conventional Post-Fabrication Optimization  
Traditionally, post-fabrication optimization has been done _manually_ by experienced engineers, but doing so becomes highly challenging to modern NAND flash memory for three reasons: (_i_) increasing device parameters to optimize, (_ii_) strong process variability, and (_iii_) limited TAT.  
**Increasing Parameters.** The number of device parameters to optimize for a single NAND flash chip has significantly increased. As NAND flash memory technologies (e.g., 3D stacking or multi-level cell technologies) become more advanced, NAND manufacturers put more device parameters into a NAND flash chip to precisely control each read/program/erase operation. For example, the number of program-related parameters is around one hundred in triple-level cell (TLC) NAND flash memory, but it increases to more than 2000 in modern quad-level cell (QLC) NAND flash memory. Moreover, these device parameters significantly affect flash reliability and performance in a complex trad-off way. For example, using a smaller \(\Delta\)V\({}_{\text{PGM}}\)value enables wider \(V_{th}\) margins between adjacent \(V_{th}\) states (i.e., higher reliability), but it leads to a longer program latency (i.e., poor performance).  
**Process Variability.** The strong process variability among 3D flash blocks, which comes from their 3D manufacturing process [21], makes the post-fabrication optimization process more difficult and complex. To evaluate how serious process variability is in 3D NAND flash memory, we performed comprehensive characterization studies using real 160 3D TLC NAND flash memory. In our evaluation, we followed the standard industry practice (i.e., the JEDEC standard [22]), and reliability is measured using an accelerated lifetime test. For example, to emulate a 1-year retention time condition, we baked the flash chips at 85 \({}^{\circ}\)C for 13 h, which is equivalent to one year at 30\({}^{\circ}\)C based on [PERSON]'s law [23].  
As shown in Figure 6a, our characterization study reveals that there are considerable variations in lifetime over different blocks. Furthermore, as shown in Figure 6b, even within a single block, there are large variations in the bit error rates (RBER, Row Bit Error Rates) among different WLs in the block. For example, even in the same operating condition, the RBER of the worst WL exceeds 2.25 times more than that of the best WL. Therefore, manual-based post-fabrication optimization for all different chips, blocks, and WLs is no longer practical in modern 3D NAND flash memory.  
Figure 5: ISPP mechanism and related operating parameters.  
**Limited TAT.** The post-fabrication optimization is a time-critical process. The _turnaround time (TAT)_, i.e., the total amount of time taken to _complete_ the production of new NAND flash chips, is quite limited for developing a new generation of NAND flash memory. As the TAT includes the time for not only post-fabrication optimization but also many other tasks, e.g., the layout design, fabrication, and verification of chips, only a week or even fewer days are given for post-fabrication optimization in many cases. Manually optimizing a large number of parameters within such a limited time requires significant human effort and fails to achieve near-optimal results.