# SNA: Sensitivity-Aware Neural Architecture Adaptation for Uniform Quantization
## 4 Experimental Results
### Results  
To obtain the results in Table 1, we first set the target bitwidth \(b_{t_{1}}\) to 8 bit and the scaled bitwidth \(b_{1}\) to 4 bit to conduct the ResNet-50-SANA and EfficientNet-B2-SANA experiments. The multiplier \(r_{1}\) was calculated as \(\sqrt{\frac{8}{4}}\approx 1.414\). We used **Halving** for initialization. As can be seen in Table 1, by applying channel splitting, we observed a 0.454% and 0.104% performance improvement for W4A8 quantization on ResNet-50 and EfficientNet-B2, respectively. For ResNet-50, we compared our results with other popular methodssuch as LQ-Net [49], PACT [2], and HAWQ-V3 [50]. For the compressed ResNet-50 model, which has a size of 12.2 MB, we managed to compare our results with both the baseline and the most-popular quantization algorithms available. In our observations, the utilization of LQ-Net and PACT on ResNet-50 led to improvements of either 0.19 or 0.29, whereas our approach realized a more-substantial improvement of 0.42. Furthermore, both LQ-Net and PACT need training phases to find the optimal quantization parameters, consequently extending the duration of the quantization process. In contrast, SANA only requires fine-tuning subsequent to architecture adaptation, simplifying the process significantly. Moreover, the quantization process in SANA relies only on uniform quantization, while LQ-Net and PACT require more-complex designs such as a learnable quantizer, adding to their implementation complexity. The results indicated that channel splitting can benefit both insensitive and sensitive networks, achieving better outcomes compared to W8A8 quantization without channel splitting. \begin{table}
\begin{tabular}{c c c c c} \hline \hline
**Method** & **Accuracy** & **Size (MB)** & **Mul** & **Precision** \\ \hline \multicolumn{5}{c}{(**a**)} \\ \hline  & 77.610 & 97.8 & 1 & FP \\ Naive & 77.318 & 24.5 & 1 & W8A8 \\  & 76.210 & 12.2 & 1 & W4A8 \\ \hline LQ-Net [49] & 76.400 & 12.2 & 1 & W4A32 \\ \hline PACT [2] & 76.700 & 15.3 & 1 & W5A5 \\  & 76.500 & 12.2 & 1 & W4A4 \\ \hline HAWQ-V3 [50] & 77.580 & 24.5 & 1 & W8A8 \\  & 75.390 & 18.7 & 1 & MP4/8 \\ \hline SANA & 77.772 & 24.4 & 1.414 & W4A8 \\  & **76.630** & 12.2 & 1.155 & W3A8 \\ \hline  & & **(b)** & & \\ \hline  & 79.800 & 35.2 & 1 & FP \\ Naive & 78.006 & 8.8 & 1 & W8A8 \\  & 77.542 & 4.4 & 1 & W4A8 \\ \hline B0-HMQ [51] & 76.400 & 7.3 & 1 & W8A8 \\ \hline B3-QN [52] & 67.800 & 5.8 & 1 & W4A8 \\ \hline  & **78.120** & **8.8** & **1.414** & W4A8 \\ \hline SANA & 74.132 & 4.4 & 1.155 & W3A8 \\ \hline \hline \end{tabular}
\end{table} 
Table 1: SANA results with ResNet-50 and EfficientNet-B2 on the ImageNet dataset. We used **Halving** for initialization. Note that SANA with W3A8 has the same model size as the original network in W4A8, while SANA W4A8 has the same size as the original network in W8A8. (**a**) ResNet-50 results. For redundant networks such as ResNet-50, using W3A8 and a 1.155 multiplier obtained a substantial improvement over the original W4A8 methods. Moreover, using W4A8 and a 1.414 multiplier exceeded even the full-precision model. (**b**) EfficientNet-B2 results. For sensitive networks such as EfficientNet-B2, using a high quantization bitwidth and large scaling factor, specifically W4A8 and a 1.414 multiplier, can effectively increase the capacity of the network and maintain performance. \begin{table}
\begin{tabular}{c c c c c} \hline \hline
**Method** & **Accuracy** & **Size (MB)** & **Mul** & **Precision** \\ \hline
**ResNet-50-SANA** & **76.630** & **12.2** & **1.155** & W3A8 \\ \hline ResNet-50-SANA & 75.458 & 12.2 & 1.414 & W2A8 \\ ResNet-50-MP & 76.534 & 12.2 & 1 & MP4 \\ ResNet-50 & 76.210 & 12.2 & 1 & W4A8 \\ \hline \hline \end{tabular}
\end{table} 
Table 2: Ablation results for ResNet-50-SANA on ImageNet. Here, ResNet-50-MP denotes the mixed-precision quantization model, while MP4 refers to the average 4 bit of mixed precisions. For another experimental setting, we set a lower target bitwidth \(b_{1_{2}}\) to 4 bit and a scaled bitwidth \(b_{2}\) to 3 bit.