Joint Format and Max Value Search
The objective of post-training quantization is to minimize the perturbation ( \delta\mathbf{X}=\mathbf{X}_{\rm FP}-\mathbf{X}_{\rm R} ) introduced by quantization to the pre-trained real-valued network:

{\rm min}\ \mathbb{E}[\mathcal{L}(\mathbf{X}_{\rm R}+\delta\mathbf{X})-\mathcal{L}(\mathbf{X}_{\rm R})] (13)

In this study, we adopt the setting presented in Choukroun et al. ( 2019 ); Wu et al. ( 2020 ) , which assumes a positive correlation between the change in the intermediate output of the quantized model and Eq. 13 . Therefore, minimizing the distance between the intermediate output of the quantized layer ( \hat{\mathbf{O}} ) and the output of the original layer ( \mathbf{O} ) leads to minimize Eq. 13 . Hence, the objective loss metric is formulated as:

{\rm min}\ (\hat{\mathbf{O}}-\mathbf{O})^{2} (14)

which is used to search for the optimal FP quantization function in the following proposed framework.

The challenges in FP quantization arise from its sensitivity to the quantization format and clipping range. Undesirable format selection will result in a catastrophic error rate. In addition, we observe that the optimal clipping range varies depending on the format used. Previous work Kuzmin et al. ( 2022 ) on floating-point (FP) quantization-aware training (QAT) proposed to learn both the FP format and maximum value with gradients. However, we find this method suffers from over-fitting in PTQ, with accuracy being even worse than naïve MinMax method, details can be found in Appendix E . Instead, we propose a search-based algorithm that jointly determines the optimal format and its associated clipping range to address this challenge.

The searching process is conducted layer by layer with the metric of minimizing Eq. 14 . The output of matrix multiplication corresponding to each sub-module is denoted as \mathbf{O}=\mathbf{X}\mathbf{Y} , where \mathbf{Y} can be either a weight tensor \mathbf{W} or another activation tensor.

The search space of q -bit FP format includes all formats except for the format with an exponent bit equal to 0 , as the quantization of the format with an exponent bit equal to 1 already degenerates to INT quantization. We search for the real-valued exponent bias \tilde{b} , which equals to the logarithm of the scaling factor. We initialize \tilde{b}_{{}_{\mathbf{X}}} and \tilde{b}_{{}_{\mathbf{Y}}} from Eq. 7 with Q_{max} equals the maximum value of |\mathbf{X}_{\rm R}| and |\mathbf{Y}_{\rm R}| , respectively. We then define the search space of \tilde{b}_{{}_{\mathbf{X}}} and \tilde{b}_{{}_{\mathbf{Y}}} by linearly dividing [\gamma_{{}_{1}}\tilde{b}^{init}_{{}_{\mathbf{X}}},\gamma_{{}_{2}}\tilde{b}^{init}_{{}_{\mathbf{X}}}] and [\gamma_{{}_{1}}\tilde{b}^{init}_{{}_{\mathbf{Y}}},\gamma_{{}_{2}}\tilde{b}^{init}_{{}_{\mathbf{Y}}}] into k intervals, where \gamma_{{}_{1}} and \gamma_{{}_{2}} are empirically set to 0.01 and 1.2 , and k = 100 .

The search process is outlined in Alg. 1 . We search the quantization scheme in all the matrix multiplication layers in parallel following Yuan et al. ( 2022 ); Bai et al. ( 2022 ) . The algorithm can be divided into two parts. (1) Do forward propagation to store the intermediate raw output of each layer l . (2) Iteratively update the optimal format and biases for each layer for three rounds by minimizing the reconstruction metric (Eq. 14 ).
We name this search-based framework as Floating Point Quantization Baseline ( FPQ baseline ), and it can already achieve state-of-the-art results on both 8-bit and 6-bit settings.