# Assessing the Role of Program Suspend Operation in 3D NAND Flash Based Solid State Drives
## 3 Background and Methods Applied
### Characterization and Simulation Tools  
The program suspend operation has been characterized on an off-the-shelf sub-100 layer TLC 3D NAND Flash chip including this feature in its command set. The experimental setup described in [31; 32] was exploited in this work. Such an electrical characterization tool interfaces with the memory at a data rate of 400 MT/s and is capable of extracting timing information with 1 us precision about the on-going program operation by monitoring a dedicated pin of the memory chip (the Ready/Busy pin as the one described in [16]). The overall program time \(t_{PGM}\) is the main parameter that we investigated in all the characterizations. As shown in Figure 3a, 3D NAND Flash devices exhibit an intrinsic variability of this parameter. We also performed a measurement to retrieve the inherent variability of \(t_{PGM}\) by re-programming the same wordline multiple times. Tests demonstrated that, if the number of re-programming cycles was in a range that did not alter the programming characteristics of the wordline, the \(t_{PGM}\) stayed almost constant. This setup can also extract the number of Fail Bits Count (FBC) on 16 kB pages and, therefore, retrieve reliability information from the memory. Finally, the characterization system exploits two current probes to infer the memory power consumption (at a given supply voltage) from the \(VDD_{core}\) (i.e., the power supply for the 3D NAND Flash peripheral circuitry) and from the \(VDD_{Q}\) (i.e., the power supply from the memory I/O pins) supplies. For confidentiality reasons, we can only state that \(VDD_{core}\) is between 2.7 V and 3.6 V, whereas \(VDD_{Q}\) is between 1.2 V and 1.8 V. Figure 3b shows an example (a zoom) of an extracted power trace from a 3D NAND Flash chip during a program suspend operation. In the figure, we can appreciate the single iterations of the program algorithm and the idle consumption during the suspension time since, in this example, no read operations were preempted. As we considered a TLC memory in this study, we collected different power traces for each page type and operation. Figure 3: (**a**) \(t_{PGM}\) measurements performed on 50 randomly sampled wordlines in a 3D NAND Flash block. (**b**) Power consumption measured during a program suspend operation. To explore the QoS, performance, and power consumption features of SSD architectures integrating 3D NAND Flash memories with program suspend capabilities, we exploited the SSDExplorer co-simulator [33]. The tool allows a fine-grained design space exploration of a drive by allowing modifications of its micro-architectural parameters, including the command queues, interaction mechanisms with the host system, and error recovery policies. The simulations performed considered the 3D NAND Flash timing characteristics as well as the power consumption during the serviced program and read operations through a back annotation process as shown in Figure 4.  
The assessment of the program suspend role was performed through a microbenchmark that is common in SSDs latency and QoS evaluations [34], namely a 4-kB-aligned workload. A queue depth (QD) either equal to 16 or to 64 was considered with a different mixture of read and write transactions. All the SSD architectural parameters considered in this work are presented in Table 1 and mimic those of an enterprise-class SSD [35]. \begin{table}
\begin{tabular}{c c} \hline \hline
**Parameter** & **Value** \\ \hline Host interface & PCIe gen3 X4 \\ Frame buffer size & 1024 \\ QD & 16–32–64 \\ Channels & 16 \\ Targets (number of Flash die per chip) & 8 \\ DRAM cache & 64–512 MB \\ Error Correction Code (ECC) & Low Density Parity Check (LDPC) \\ Over-Provisioning & 30\% \\ Write Amplification Factor & 2.4 \\ GC threshold & 10\% \\
3D NAND Flash Storage Medium & Charge trapping \\
3D NAND Flash Storage paradigm & TLC \\
3D NAND Flash page & 16 kB + parity \\ Layers & [48–100] \\ Pages per block & 256 \\ Blocks per die & 2048 \\ \hline \hline \end{tabular}
\end{table}