## 4 Experimental Results on DNNs
### Performance on Commonly Used Models  
We conduct comparative experiments, focusing on three factors and training DNNs using four different training strategies based on each factor, recording the final model's test accuracy/perplexity. Table 1 presents our experimental results, from which the following can be observed:  
(1) There is a strong correlation between the accumulation in matrix computations and the capability to withstand quantization noise. Layers with a higher number of accumulations consistently achieved the best test performance when employing a lower training bit-width. This finding validates the analysis from Section 3.1, which suggests that layers with a larger number of accumulations can tolerate greater quantization noise, meaning they can effectively use lower training bit-widths, whereas layers with fewer accumulations require higher training bit-widths. Furthermore, in the case of MobileNet-V2, when the network width is reduced to half of its original size, the impact of correlation becomes more pronounced. As MobileNet-V2 is inherently a parameter-efficient model, it engages a relatively small number of parameters in computation. Once the model is narrowed, this leads to a further reduction in the aggregate quantity of parameters, consequently resulting in a significant decrease in accuracy. Additionally, the model becomes more sensitive to the effects of correlation. (2) The frequency of elements in a matrix are used also correlates with the ability to resist quantization noise. Identical levels of quantization noise added to layers with lower usage frequencies often lead to superior training results. Even when optimal outcomes are not achieved for some tasks, the performance is still significantly better than with the opposite bit-width configuration strategy. This corroborates the analysis from Section 3.2, indicating that quantization noise in layers with higher usage frequencies has a more detrimental effect on training. (3) In the context of CNN models, a decrement in the training bit-width at each layer with increasing depth yields improved outcomes. In contrast, for Transformer and LSTM models, such a trend is not evidently correlated. However, for CNN models, the trends in depth and usage frequency are same (as shown in Figure 2), resulting in completely consistent final outcomes, indicating a need for further analysis. Reducing model size is also a significant benefit of low-precision quantization. Table 2 lists the sizes of some experimental models from Table 1. It can be observed from Table 2 that for CNN models (ResNet-20/ResNet-50 on CIFAR-10/ImageNet), higher test accuracy does not require an increased model size. In fact, an appropriate bit-width allocation strategy can achieve greater accuracy with a reduced model size. In the case of the Transformer model, the sizes of the models with different bit-width allocation strategies do not differ significantly; however, a well-considered allocation strategy can lead to improved accuracy. Combining the observations from Tables 1 and 2, it is clear that different bit-width settings have a discernible impact on training. A judicious configuration strategy can effectively balance the reduction of model size while maintaining high accuracy.