## 3 Low Precision in Matrix Calculations
### Frequency of Elements in the Matrix Used  
When calculating \(r_{3}=r_{1}r_{2}\), the elements of \(r_{3}\) are obtained by multiplying the rows of matrix \(r_{1}\) with the columns of matrix \(r_{2}\) and then summing the results. In the process of computing \(r_{3}\), it is noted that the elements in matrices \(r_{1}\) and \(r_{2}\) are utilized with different frequencies, which is related to the dimensions of the matrices being multiplied, e.g., when the number of columns in \(r_{2}\) is less than the number of rows in \(r_{1}\), matrix \(r_{2}\) is accessed more frequently.  
For the convenience of analysis, we assume that matrix \(r_{1}\) is an \(M\times N\) matrix, and \(r_{2}\) is an \(N\times 1\) matrix. In this case, the elements obtained from the low-precision matrix multiplication \(r_{3}\) in Equation (2) can be represented as  
\[r_{q,3}^{(i,1)}=\sum_{j=1}^{N}r_{1}^{(i,j)}r_{2}^{(j,1)}+\sum_{j=1}^{N}r_{1}^{ (i,j)}e_{2}^{(j,1)}+\sum_{j=1}^{N}e_{1}^{(i,j)}r_{2}^{(j,1)}+\sum_{j=1}^{N}e_{ 1}^{(i,j)}e_{2}^{(j,1)} \tag{3}\]  
In Equation (3), it is evident that the quantization noise from the two matrices has distinct impacts: the quantization noise in \(r_{2}^{(i,1)}\) affects all elements in \(r_{3}\) (for any \(i\)), while the quantization noise in \(r_{1}^{(i,j)}\) only influences the \(i\)-th row of elements in \(r_{3}\).  
We adopt a methodology similar to that described in Section 3.1. In our matrix multiplication operations (Figure 1), we incrementally increase the size of one matrix (from the same distribution) to reduce the usage frequency of each element within the multiplication (from 512 to 9). We utilize the results computed in full precision as the ground truth and compare the average distance of each element between the low-precision computation results and the ground truth under different usage frequencies. Additionally, we analyze the fluctuation of these distances across multiple experiments.  
As shown in Figure 5, the graph demonstrates the impact of quantization as matrix usage frequency decreases. For both 4-bit and 2-bit quantization, while the distance  
Figure 4: Distance and variance with increasing number of accumulation.  
metrics (solid lines) remain relatively stable, there are notable differences in their absolute values, with 2-bit quantization showing consistently higher distances. This indicates a greater loss in precision with lower-bit quantization. The most striking observation is the significant downward trend in variance (dashed lines) as frequency decreases. This trend is particularly pronounced in the logarithmic scale of variance, suggesting that matrices with lower usage frequencies exhibit better tolerance to quantization noise. This enhanced tolerance contributes positively to the numerical stability during model training, making these matrices more suitable candidates for aggressive quantization strategies.  
In DNN computations, due to the significant difference in the usage frequency of different matrices (as depicted in Figures 2 and 3), the impact of quantization on each matrix also varies. Matrices that are frequently used have a more extensive error impact. Consequently, the quantization errors are amplified with the increased usage of these matrices.