# Optimization of Linear Quantization for General and Effective Low Bit-Width Network Compression
## 4 Experiment
### Accuracy Comparison between Existing Post Quantization Methods Based on Clipping  
In Tables 5 and 6, this paper compares the clipping-based method with our clustering-based method for two models, Resnet-50 and Inception-v3 on ImageNet, using the 4-bit precision. For ResNet-50, our methods increase the top-1 accuracy by 4.7% compared with the other post-quantization methods. For Inception-v3, our method increases the top-1 accuracy by 30.8% compared with the other post-quantization methods. This is strong evidence that the problem with linear quantization is that it cannot be reasonably clustered weights at low bit-widths.  
\begin{table}
\begin{tabular}{c c c c c c} \hline \hline \multirow{2}{*}{**Network**} & \multirow{2}{*}{**Bit-Width**} & \multirow{2}{*}{\begin{tabular}{c} **Random** \\ **(Before Pruning)** \\ \end{tabular} } & \multirow{2}{*}{\begin{tabular}{c} **Random** \\ **(After Pruning)** \\ \end{tabular} } & \multirow{2}{*}{\
\begin{tabular}{c} **Uniform-Partition** \\ **(After Pruning)** \\ \end{tabular} } & \multirow{2}{*}{
\begin{tabular}{c} **PSO** \\ **(After Pruning)** \\ \end{tabular} } \\ \hline  & 8 & 69.560 & 69.432 & 69.312 & 69.422 \\ ResNet-18 [33] & 7 & 69.178 & 68.758 & 68.848 & 66.398 \\ (69.758) & 6 & 67.906 & 68.540 & 67.660 & 68.114 \\  & 5 & 61.782 & 64.936 & 60.288 & 66.426 \\  & 4 & 49.134 & 55.328 & 50.720 & 59.594 \\ \hline  & 8 & 75.906 & 75.146 & 75.146 & 75.354 \\ ResNet-50 [33] & 7 & 75.336 & 75.086 & 74.256 & 75.316 \\ (76.13) & 6 & 75.280 & 74.620 & 74.182 & 75.050 \\  & 5 & 70.784 & 71.708 & 71.120 & 72.496 \\  & 4 & 60.736 & 65.208 & 64.630 & 67.622 \\ \hline  & 8 & 74.138 & 74.530 & 74.140 & 74.064 \\ DenseNet-121 [34] & 7 & 73.876 & 73.138 & 73.330 & 74.006 \\ (74.433) & 6 & 72.468 & 72.992 & 72.402 & 73.786 \\  & 5 & 70.856 & 70.712 & 68.130 & 72.392 \\  & 4 & 54.250 & 54.838 & 60.858 & 60.918 \\ \hline  & 8 & 68.382 & 66.916 & 66.656 & 67.220 \\ Inception-v3 [35] & 7 & 68.048 & 66.43 & 67.070 & 67.364 \\ (69.538) & 6 & 64.358 & 66.582 & 65.628 & 65.774 \\  & 5 & 56.420 & 57.200 & 55.514 & 60.958 \\  & 4 & 18.112 & 27.560 & 18.916 & 33.076 \\ \hline \hline \end{tabular}
\end{table}
Table 4: ImageNet top-1 validation accuracy with weight quantization—the float accuracy is displayed under the model name. Results include different initialization methods, random, uniform partition, and the PSO algorithm at each bit-width.  
\begin{table}
\begin{tabular}{c c c c} \hline \hline \multirow{2}{*}{**Method**} & \multirow{2}{*}{**Approach**} & \multicolumn{2}{c}{**Bit-Width**} & \multirow{2}{*}{**Top-1 (\%)**} \\  & & **(Weight/Activation)** & \\ \hline Baseline & - & W32/A32 & 76.13 \\ none & Linear & W4/A4 & 0.1 \\ MSE [38] & Clip-Linear & W4/A4 & 45.0 \\ ACIQ [22] & Clip-Linear & W4/A4 & 33.2 \\ KL [15] & Clip-Linear & W4/A8 & 62.9 \\ OCS [23] & OCS-Linear & W4/A8 & 63.8 \\ PSO-K-means & Cluster-Map & W4/A4 & 67.622 \\ \hline \hline \end{tabular}
\end{table}
Table 5: Comparisons with existing quantization work based on clipping methods for ResNet-50 on ImageNet, using the 4-bit precision. The quantization methods include linear, clip-linear, OCS-linear, and our two-stage quantization with PSO-K-means and mapping.