# Check-QZP: A Lightweight Checkpoint Mechanism for Deep Learning Frameworks
## 3 Design and Implementation  
To address the issues of the checkpoint mechanism in ML models, we carefully revisited the processes of training and checkpoint operations. We identified several processes that could be enhanced for optimization in terms of storage and performance. Based on these observations, we propose two novel design principles to mitigate the overhead caused by ML models.  
**Parallel Step:** We first argue whether it is necessary to process all steps in ML in a sequential manner. The checkpoint step in the ML model spends time performing I/O requests, which may stall other steps in multi-tenant environments because running applications share I/O interfaces simultaneously. Therefore, we inject parallelism into the basic rules of the traditional checkpoint operation to overlap the time for I/O requests with the learning process; we call this the Check-P operation.  
**Data Reliability:** We also argue that the data stored by the checkpoint operations are rarely reused. In other words, the stored data are valuable in just two cases: (1) The data are required after a crash during the training step to recover the last parameters of an ML model. (2) The data become accessible to run transfer learning or deploy a model to other systems. In general, data loss may be tolerated in ML models because the data can be recovered using fine-tuning mechanisms [44; 45; 46]. Therefore, we design Check-Q and Check-Z operations that reduce the amount of data by adopting quantization and compression mechanisms. Note that the Check-Q and Check-Z operations require extra CPU cycles to run, but these cycles can be hidden because of the benefits of the Check-P operation. As a result, they will not affect the overall performance of ML steps.  
Basically, Check-Q2P follows the basic rules of the traditional steps in ML models. Algorithm 1 describes how Check-Q2P works in terms of the checkpoint operation. First, when the checkpoint operation is triggered, Check-Q2P creates a new thread to overlap the checkpoint operation with the training process. In order to make a snapshot, the Check-P operation first copies the model parameters from the memory on the GPU to the host memory. This snapshot is then transferred to the Check-Q operation to perform quantization; we describe this in more detail in the following section. Finally, Check-Q2P compresses the data delivered from the Check-Q operation and stores the compressed data on the underlying storage device.  
```
1:if triggered a checkpoint in main script
2:create_subprocess(Check-P)
3:
4:Function Check-P():
5:snapshot \(\leftarrow\) Copy_parameters()
6:quantized_parameters \(\leftarrow\) Check-Q(snqshot)
7:Check-QZ \(\leftarrow\) Compression(quantized_parameters)
8:Write_storage(Check-QZ)
9:return true
```  
**Function** Check-Q(snapshot):
10:for parameters in snapshot:
11:\(x_{\text{min}}\leftarrow\) Min_update(parameters)
12:\(x_{\text{max}}\leftarrow\) Max_update(parameters)
13://N means quantization bit-width
14:scale \(\leftarrow\frac{x_{\text{max}}-x_{\text{min}}}{2^{N}-1}\)
15:for parameters in snapshot:
16:for parameter in parameters:
17:\(x_{q}\leftarrow\text{Round}\Big{(}\frac{parameter-x_{\text{min}}}{scale}\Big{)}\)
18: Store \(x_{q}\) in quantized_parameters
19:return a set of quantized_parameters ```  
**Algorithm 1** Actions in Check-Q2P.