## 4 Experimental Results on DNNs
### In-Depth Evaluation on Customized Models
By ensuring that the accumulation in matrix calculation remains the same, we alter only the usage frequency to observe its effect on the training bit-width during mixed-precision training, as depicted in Figure 9. We denote this modified model as ResNet-F.  
Figure 8: Customized I confirm that the boldface is necessary. models for isolating the influence of single factors. Illustration of ResNet-D. \begin{table}
\begin{tabular}{c c c c} \hline
**\#Bit** & **Strategy** & **Acc-D** & **Variance** \\ \hline
4 to 8 & R & 92.63 & 0.20 \\
6 & U & 92.62 & 0.34 \\
4 to 8 & C+ & 92.83 & 0.31 \\
4 to 8 & C\(-\) & 92.73 & 0.17 \\ \hline
8 to 16 & R & 92.82 & 0.22 \\
12 & U & 92.94 & 0.22 \\
8 to 16 & C+ & 92.92 & 0.23 \\
8 to 16 & C\(-\) & 92.87 & 0.19 \\ \hline \end{tabular}
\end{table} 
Table 3: Training ResNet-D on CIFAR-10: Test accuracy and variance based on depth-configured layer bit-Widths. Table 4 reveals several crucial insights about the relationship between matrix usage frequency and quantization strategies during model training. Our analysis demonstrates that when maintaining fixed accumulation in matrix calculations while varying usage frequencies, Strategy U (which employs constant training bit-width) consistently outperforms other approaches. This superior performance can be attributed to its ability to maintain consistent numerical precision throughout the training process. A particularly noteworthy observation from our experiments is the strong correlation between model performance and matrix usage frequency [66; 67]. Matrices with higher usage frequencies demonstrate heightened sensitivity to low-precision noise, as evidenced by both the performance metrics and variance measurements. This finding aligns with recent studies in Transformer model quantization [68; 69], where activation quantization has proven to be particularly challenging. The correlation between usage frequency and quantization sensitivity is further substantiated by Figure 3, which illustrates the exceptionally high usage frequency of activations within Transformer architectures. This high frequency of use makes these matrices particularly susceptible to quantization effects, explaining why activation quantization often presents a significant challenge in practice. These findings lead to an important conclusion: matrices with high usage frequency during DNN training play a pivotal role in maintaining training stability. Their increased vulnerability to low-precision noise suggests that special attention should be paid to these matrices when designing quantization strategies. This understanding has significant implications for developing more effective quantization approaches, particularly in the context of large-scale Transformer models. **Factor of accumulations in matrix calculations only. ** We next analyze the impact of accumulation in matrix calculation separately. We build ResNet-A, a model that utilizes the BottleNeck structure. Within the same group, layers maintain uniform usage frequency and include both \(3\times 3\) conv and \(1\times 1\) conv, which have different accumulation counts in their computations, as shown in Figure 10. We set the training bit-widths according to the accumulation in matrix calculation and observe the experimental results. \begin{table}
\begin{tabular}{c c c c} \hline
**\#Bit** & **Strategy** & **Acc-F** & **Variance** \\ \hline
4 to 8 & R & 90.56 & 0.41 \\
6 & U & 90.88 & 0.32 \\
4 to 8 & C+ & 90.71 & 0.41 \\
4 to 8 & C\(-\) & 90.54 & 0.60 \\ \hline
8 to 16 & R & 91.77 & 0.36 \\
12 & U & 92.02 & 0.33 \\
8 to 16 & C+ & 91.99 & 0.31 \\
8 to 16 & C\(-\) & 91.79 & 0.60 \\ \hline \end{tabular}
\end{table} 
Table 4: Training ResNet-F on CIFAR-10: Test accuracy and variance based on frequency-configured layer bit-Widths. Figure 9: Customized models for isolating the influence of single factors. Illustration of ResNet-F.  
Table 5 reveals that strategy C\(-\) achieves the highest accuracy while significantly enhancing the stability of training.