Learning from Loss Landscape: Generalizable Mixed-Precision Quantization via Adaptive Sharpness-Aware Gradient Aligning
Authors: Lianbo Ma
Jianlun Ma
Yuee Zhou
Guoyang Xie
Qiang He
Zhichao Lu
Venue: ICML 2025 poster (2025)
Keywords: model quantization
model compression
efficient neural network
Primary area: deep_learning->algorithms
TL;DR: The study introduces GMPQ-ASGA, linking sharpness to MPQ generalization, cutting search costs via small proxy datasets for large-scale accuracy.

Abstract: Mixed Precision Quantization (MPQ) has become an essential technique for optimizing neural network by determining the optimal bitwidth per layer. Existing MPQ methods, however, face a major hurdle: they require a computationally expensive search for quantization strategies on large-scale datasets. To resolve this issue, we introduce a novel approach that first searches for quantization strategies on small datasets and then generalizes them to large-scale datasets. This approach simplifies the process, eliminating the need for large-scale quantization fine-tuning and only necessitating model weight adjustment. Our method is characterized by three key techniques: sharpness-aware minimization for enhanced quantized model generalization, implicit gradient direction alignment to handle gradient conflicts among different optimization objectives, and an adaptive perturbation radius to accelerate optimization. It offers advantages such as no intricate computation of feature maps and high search efficiency. Both theoretical analysis and experimental results validate our approach. Using the CIFAR10 dataset (just 0.5\% the size of ImageNet training data) for MPQ policy search, we achieved equivalent accuracy on ImageNet with a significantly lower computational cost, while improving efficiency by up to 150\% over the baselines.

Learning from Loss Landscape: Adaptive Sharpness-Aware Gradient Aligning
Generalizable Mixed-Precision Quantization via
Lianbo Ma Jianlun Ma
Yuee Zhou'
Guoyang Xie 23 Qiang He* Zhichao Lu²
Abstract
one effective way is to utilize set of small bitwidths to quantize the entire network for reducing model redundancy (Deng et al., 2020), termed as mixed-precision quantization (MPQ). Different from fxed-precision quantization (FPQ) (Esser et al., 2019), MPQ works in a fine-grained way, allow- ing different bitwidths for different layers (Elthakeb et al., 2020; Guo et al., 2020). That is, the layers can be quantized using much smaller bitwidths than the quantization-sensitiyy layers, which can naturally obtain more optimal accuracy-complet
Mixed Precision Quantization (MPQ) has become an essential technique for optimizing neural net- work by determining the optimal bitwidth per layer. Existing MPQ methods, however, face a major hurdle: they require a computationally ex- pensive search for quantization policies on large- scale datasets. To resolve this issue, we introduce a novel approach that first searches for quanti- zation policies on small datasets and then gen- eralizes them to large-scale datasets. This ap- proach simplifies the process, eliminating the need for large-scale quantization fine-tuning and only necessitating model weight adjustment. Our method is characterized by three key techniques: sharpness-awan minimization for enhanced quan- tization generalization, implicit gradient direc- tion alignment to handle gradient conflicts among different optimization objectives, and an adap- tive perturbation radius to accelerate optimiza- tion. Both theoretical analysis and experimental results validate approach. Using the CIFARIO dataset (just 0.5% the size of ImageNet training data) for MPQ policy search, we achieved equiv- alent accuracy on ImageNet with a significantly lower computational cost, while improving effi- ciency by up to 150% over the baselines.
ietizaatiioonisnssiiii
trade-off than FPQ.
Limitations. Most existing MPQ methods require the con- sistency of datasets for bitwidth search and network deploy- ment to guarantee policy optimality, which results in heavy search burden on large-scale datasets (Zhao et al., 2024). For example, HAQ (Wang et al., 2019) requires approxi- mately 72 GPU hours to search for the optimal quantization policy for ResNet50 on ImageNet. A natural solution to solve the above issue is to decouple the dataset used in MPQ search stage and model inference stage. In this way, the search efficiency can be largely improved since the MPQ search can be performed with small size of proxy dataset. Unfortunately, it inevitably suffers from intractable chal- lenges incurred by shifted data distributions and small data volume of the disparate datasets. For example, when CI- FAR10 is used to search for MPQ policy for MobileNet- V2 trained on ImageNet, PACT (Choi et al., 2018) suffers from substantial loss of nearly 10% in Top-1 accuracy.
To bridge such gap, we aim search transferable policy using small proxy datasets via enhancing generalization of quantization search (as shown in Figure 7 in Supplementary Material A. Several studies demonstrate that utilizing the discriminative nature of feature representations (e.g., the attribution rank of image's 's features (Wang et al., 2021), the class-level information of feature maps (Tang al., 2023)) can find transferable policy. However, such disentangled representation learning methods just enhance the discrimina- tion ability of target quanitzed models, which is empirically helpful to improve generalization, but still suffers from lack of theoretical support. Moreover, they entail intricate cal- culation of feature maps (e.g., attribution rank calculation and alignment, classification margin computation), which computationally complex.
1. Introduction
With the fast development of edge intelligence applications, it is desirable to compress DNNs with minimal/no perfor- mance deterioration according to specific hardware configu- rations (Shuvo et al., 2022; Ma et al., 2024). To realize this,
'College Software Northeastern University, Shenyang, China 'The Department of Computer Science, City University of Hong Kong, Hong Kong, China 'The Department of Intelligent Manufacturing, CATL, Ningde, China 'College of Computer Sci- ence and Engineering. Northeastern University, Shenyang, China. Correspondence to: Guoyang Xie <guoyang.xin xie@ ore..org>.
Proceedings of the 42nd International Conference Machine Learning. Vancouver, Canada. PMLR 267, 2025. Copyright 2025 by the author(s).

---

Learning from Loss Landscape: Generalizable
Quantization via Adaptive Sharpness-Aware Gradient Aligning
A60)=0.51 0.51 p 0.1
4 0:45
Ardh - 0.39
Ate) - 0.32 0.1 2010y
0.1
o 0.1
Avo
Top-1 CI
2000gp' - 102
4,99
Top-1 CIFARIO:
o-2h00pp - 90
Top-1 CIFARIO: 90.2%
2MP 7%
400
Aspy
Top- on CIEARIO
Top-1 on ImageNet 65.6 83.
Top- oc ImageNet: 66, 1%
Top-1 on ImageNec: 65, 4%
Top-1 on ImageNet 66.2%
Generafinstooo 25, 1%
Genoruitatiio: 24.7%
Generniizatoo 24 8%
Generslttation 24 6%
E))0)
50(0)
(9)
h(B))
h(())
h(O)
h(())
P
EXO)
p
L(
p
((O)
ALQ
b. ALQUAAGA
EdMIPS
d. EdMIPS+ASGA
Figure 1. Comparison of the generalization performance between the baseline MPQ methods and ASGA on 8 8 with CIFARIO. serves as a measure of the sharpness of the loss landscape (defined in Section 2.2) and Generalization mcans the change in terms of Top-1 accuracy the model on CIFARIO and ImageNet. Compared to the baselines (a and c). ASGA significantly reduces surrogate gap (i.e.. the difference between the perturbed loss C, (0) and the experience loss (())), smoothing the sharpness of the loss landscape, thereby enhancing the model's generalization on the target dataset (b and d).
Motivation, Our idea is motivated by the observation that there exists the strong relationship between the sharpness of loss landscape in MPQ training and the generalization of target quantized model over unseen datasets (see Figure 1). Moreover, the quantization noise could exacerbate the sharpness of loss curvature and blur the minimum of loss landscape. Since minimizing the sharpness of the loss landscape is significantly conducive to generalization en- hancement, we seek the MPQ policy that can lead to flatter loss minima of quantized model, making the loss landscape more robust to quantization noise. Note that the calculation of sharpness is relatively simple and cheap with no need of intricate computation of feature maps.
our method gets 150%, 127% and 113% speedup compared to state-of-the-an (SOTA) MPQs, respectively.
2. Approach
2.1. Problem Formulation
General Formulation. The main goal of MPQ is to quan- tize the model using suitable bitwidths from the quantization policy Q under sssource-constriigg conditions Sg. which can be formulated as the following bi-level optimization problem:
min Lval (0", Q)
(I)
Contributions. In this paper, we present an adaptive sharpness-aware gradient aligning (ASGA) method to learn generalizable MPQ policy, which exploits the sharpness information of loss landscape for generalization improve- ment on the proxy datasets, shown in Figure 2. To incorporate the loss sharpness into the optimization of MPQ training, we handle gradient conflicts between dif- ferent optimization objectives by implicitly aligning gradi- ent directions, and then accelerate the optimization pro- cess by setting an adaptive perturbation radius. To the best of out knowledge, our design is the first attempt to apply sharpness-based generalization to MPQ policy search, which offers advantages such as no intricate compu- tation of feature maps and high search efficiency.
S.L. 0" = arg min Cirain (0, 2(2)= & So:
where L and Lirain represent the task loss on the valida- tion dataset and the training dataset, respectively, and 0" is the optimal weights set of quantized network under Q. By alternatively updating Q and until convergence, we can obtain the optimal mixed-precision model.
Due to differences in data distribution, most existing MPQ methods need consistency between the datasets used in pol- icy search stage and the dataset used in model inference stage, which requires significant computation costs in large datasets. It is also fragile in privacy-sensitive MPQ tasks, where the training data is not allowed to directly come from the validation dataset. To break above limitations, we aim to directly learn optimal MPQ policy on small proxy dataset and then generalize it target large dataset for inference, and then reformulate the search objective of MPQ as:
Experimental results demonstrate that a small sharpness measure learned from proxy dataset helps seek a transfer- able MPQ policy for quantizing the model trained on large datasets. Our method acquires promising performance when searching very small proxy datasets versus directly large datasets, where the size of the former is only 0.5% of the latter. As a result, we accomplish impressive improve- ment of MPQ search efficiency. For ResNet18, ResNet50 and MobileNet- V2, by using CIFARIO as the proxy dataset,
min E (Lval (0", Q. x)) x E Deals Q
s.t. 0 = arg min E (Lirain (0, 2))),, Dirain-prozy) 2(Q) < So.
where E denotes the expectation, Deat is the dataset con- taining all validation data in model inference stage and
2

---

Learning from Loss Landscape: Generalizable Mixed-Precishee Precision Quantization via Adaptive Sharpness-Aware Gradient Aligning
Layer bitwidth policy sourching Comali-scale pruny dataset)
Depeloymest (brgge-scale barget datasert)
2-th layer
Eage
-6
Dauser
Σ
Surgodte Gay (Eg. 6
Leumable biswidth weight
A-th feature
(k-1)-th
Classifier
Fiowidtt Anigement
orgmax
Prony Danasct
Model
feature
Bitwidth
candidate
Compieity
xL-layer
Low
Leorablee
(Eq
Nowith
Wall Cenenilaed
Figure 2. The illustration of our approach. We aim to search for an optimal quantization policy with a flat loss landscape on a proxy dataset, which can be applied to large-scale target datasets. In the policy searching stage, we seek a MPQ policy with flat loss landscape by minimizing the empirical loss, complexity loss, and surrogate gap. In the deployment stage, the searched MPQ policy be directly applied to model inference on large target datasets.
Dtrain -anin--pae prozy is the proxy dataset in MPQ search stage.
2.2. Exploiting the Loss-Sharpness Information
Differentiable Formulation. We consider a differentiable MPQ (DMPQ) search process (Cai & Vasconcelos, 2020; Yu al.. 2020), which formulates the whole searching space as a supernet (as Directed Acyclic Graph) where the nodes represent candidate quantization bitwidths, and the edges denote learnable weights of the bitwidths. The optimization objective of DMPQ is defined as
From the perspective of the loss landscape information in a well-preforming MPQ policy, their sharpness in MPQ train- ing should be minimized, which has the following benefits: a) It helps the training of quantization to escape sharp region, and also alleviates the sharpness aggravation incurred by quantization noise well. b) It has desirable and dataset- independent property, recent research (Foret et al., 2020; Wang et al., 2023) recognizes small loss sharpness con- tributes to model generalization. c) The calculation cost of sharpness measure is relatively simple and cheap.
Lerain (0) L(0)- + ALcomp (0),
(3)
where À is coefficient controlling the accuracy-complexity trade-off, L()) denotes the accuracy loss, and (0) is the loss of target model's 's complexity, i.e.,
Motivated by this, we aim to search the MPQ policy that guarantees the loss landscape of the proxy data distribution flat possible. As discussed above, such general property in MPQ search can ensure usability across the data distributions. However, the sharpness of the loss landscape is not explicitly incorporated in the current cross- cross-entropy loss formulation. Consequently, the objective is not only to optimize accuracy and complexity, but also to seek an MPQ policy that minimizes the sharpness of the loss landscape.
18a
Lcomp =
Σ
Σ
Σ h=0
Pk
(
comp',
1=0
1=0
1.0
cxp
(a²)
Laa
exp
(3))
s.l.
Pj
|B'l Σ k exp (a4)
Pk
B Σ
exp (3)
(4)
where Dd and 8a denote the candidate bitwidth sets for weights and activations, respectively, a' and Se denote the learnable weights vectors of their corresponding bitwidth candidates layer I, and comp' is the BOPs (Billions of Operations Per Second) constraint of layer I, i.e.,
Preliminary. In conventional model training. Sharpness- A ware Minimization (SAM) (Foret et al., 2020) aims to search for a flatter region around the minimum with low loss values. To realize this, SAM introduces perturbations to model weights 8, and solves the following min-max optimization optimization problems:
comp' cha dout x kg x kg, x Nout Wonts
min Cp((),
where Gon and Cout denote the number of input channels and output channels, respectively, Ka and kú are the kernel size, Wout and hout denote the width and height of the output feature map, respectively.
s.t. (,( 2 max C(O + 8),
(6)
1
S~ 8 R²
One may argue that we can directly utilize subset of tar- get dataset to search MPQ policy rather than proxy dataset transfer, but this would result in serious performance deteri- oration, as demonstrated in Section 3.3.3.
where is the perturbation radius, k is the dimension of and b is the scaling factor influenced by the candidate bitwidth. For a well-trained model, when the perturbed loss is minimized, the neighborhood corresponds to low losses
3

---

Learning from Loss Landscape: Generalizable
Quantization via Adaptive Sharpness-Aware Gradient Aligning
(below the perturbed loss). Given p. through Taylor expan- sion around 0, the inner maximization of Eq. (6) becomes
(O))
linear constrained optimization with solution:
(9)
k(0,)
h(0))
((0.)
arg max (p)
((0
arg max C(0) + 8 VC(0)+
(a)
(b)
V (())
Figure 3. (b) exhibits a flatter landscape compared to (a), as
indicated by its smaller h(2)). Nevertheless, SAM favors (a) due to its lower local minimum ((1)).
P C(0)
Then, the optimization problem of SAM reduces to:
(0) and the angles between their gradients:
mjn C0) min L (0p),
0
min (mnn(C(00.CC,)))...)))
V (()
(9)
where 0p a 0
-0
[V
where is the scaling factor. In this way, the training loss can converge to flat region with small loss value. More details of gradient alignment are provided Supplementary Material A.2.2.
where 8, denotes the perturbation weights corresponding to the maximum perturbation loss within the neighborhood. In this way, the goal of SAM is converted to seek an optimal solution on the perturbed loss landscape of (0).
Then, we suggest an adaptive strategy gradually adjust p according h()), i.e., p min(Pmoz: (m](([+])) aiming to obtain optimal gradient direction for sharpness optimiza- tion. By integrating above strategies, we get the design of adaptive sharpness-aware gradient aligning (ASGA):
Adapiie Sharpness-Aware Gradient Aligning. SAM suffers from following drawbacks when applied in MPQ:
(1) For fixed p. there is no linear relationship between the perturbed loss (0) and the sharpness loss maz (the
dominant eigenvalue of the Hessian). Here, Tmax
2h(0)
(0),
where h(0) (0) L(A) denotes the surrogate gap.
min((((), +
VVC() V())))
(10)
We can see a smaller (0) does not necessarily guaran- tee convergence to a flat region, as shown in Figure 3. When ((0))= < C(2)), even if (01) ) is much smaller than C,(02), SAM may still favor the minimum on the left side.
s.t. = min(Pman' In(h(0) +
More details about the merits of our design are provided in Supplementary Material A.3. Note that we do not need to recalculate [()), which has been determined when calculat- ing VL (0), and thus ASGA has an equivalent computation complexity SAM.
(2) When optimizing (0) and h())) simultaneously, there exists conflict between VL(() and V (O) (Zhuang et al.). This is because when minimizing h()), the orthogonal com- ponent VL(() will increase L(0), thereby making the de-
crease of (0) difficult since h(0) is always non-negative, which hurts the model's generalization.
2.3. Generalizable MPQ via Adaptive Sharpness-Aware Gradient Aligning
(3) Since the loss landscape evolves towards being flatter as the search progresses, it is hard to capture surrogate gap by fixed p. slowing down the speed of optimizing sharpness.
Then, we add the optimization objective of our proposed ASGA into the loss design of DMPQ the regularization of quantization generalization, i.e.,
Fortunately, from basic principles of vector operations, we find that when V (()) and (., are consistent, Vh(t)= (0, v (0)) is also consistent with them, which makes feasible to apply the gradient descent to all three losses effectively Hence, as shown in Eq. (9), reformulate the optimization objective of MPQ as seeking a balance between [()), (0), and h(() by implicitly aligning the gradient directions between (()) and (0) while minimizing (()),
min (C(0), (C0,
(11)
0
Using Eq. (10), the above optimization objective becomes: mnj,(C()) (0), C(B +
[CC() VC())),
P
the correpanndngg loss
(12)
'Regarding to this point, we provide more explanations in Supplementary Material A.2.1.
where e denotes the papee--rrseeter for weighting the corresponding loss in the optimization process.

---

Learning from Loss Landscape: Generalizable Mixed-Precision Quantization via Adaptive Sharpness-Aware Gradient Aligning
In essence, the goal of our method is seek a balance be- tween prediction accuracy (via minimizing (()) and (,))) model complexity (via minimizing Loomp and general- ization capability (via minimizing h())).
gradient estimator V Lg) (L()) is bounded by M for all 0 R i..., E V (B) ( S M.
From Assumption 1, with the definition of perturbed weight (Eq.. (8)), we can derive the following expression via Taylor expansion:
2.4. Theoretical Analysis
2.4.1. GENERALIZATION PERFORMANCE ANALYSIS
-0)) _ 11 -011
((0=))
First, we theoretically analyze the impact of loss sharpness on the upper bound of quantization generalization emor.
=L( 79 + 220 (H [V.)) + +29),
and g (15)
Lemma 1. Suppose the training set contains m elements drawn i.idd from the true distribution and the average loss in MPQ search is C(0) VI -E Lz (0, Us) (L:(0) for short), where (Ie, Us) the input-target pair of i-th element. For the prior distribution (independent of training) S. we have:
where H = [
V[(0,)]VVLL (L) for short, then we have:
L ((0++)) <C(O.) 723
[V.(
(16)
223
+ 2
H - 121-11111 71-
E Rx, L.(0) S (0) + R.
(13)
According to the fact that 1 b11aa b|1 < 1|a - +|c 1-bcc|bl 612
with probability at least (1 a) 1- (v vT)²
and Assumption 2, by taking the expectation on both sides of the inequality simultancously, we have:
where 10 denotes confidence level, and R
IEC (0.+1) S (E,) 728 |(())
4v / (KI(F log 2Tx) /m. This shows that ASGA effectively reduces the upper bound of the generaliza- tion error during the MPQ process.
(17)
-1- 7BG 72 BMM.
We set cancellation term for the last term of the above equation. By using the Cauchy-Schwar inequality and reor- ganizing the Eq. (17) we have:
Proof. The detailed proof is provided in Supplementary Material A.4.
2.4.2. CONVERGENCE ANALYSIS
(.(+1+1) SEL( (By² (8y2- 3
Then, we analyze the convergence property of ASGA -based DMPQ under non-convex setting, as shown below.
(18)
1 + 0 32 1) 1)+ BM.
Lemma 2. Under the condition of employing SGD with a learning rate of y = optimizer, we have:
< as the base
By summing Eq. (18) over T iterations, we can obtain:
vT
Σ EIVL(0)) (C) (0*))
T
2TA
θ
(3vT 28
E ( <
(14)
(19)
7o 2A
1=1
p (33% vT) 7M
where A E (0)) L [0-) (0- is the optimal
GVT 4,8%0
solution), θ (3830 VT) + +0,3MM and
From Eq. (19), we can infer that Eq. (14) holds, i.e., Lemma 2 is proven.
A= T- 2,3
Proof. We follow two basic assumptions widely used in convergence analysis of stochastic optimization (Jiang et al., 2023). Assumption 1: Suppose there exists a constant such that for all 0, v E Rdd 1 (L())-L(())l) < ||0 - 2/|2 holds. Assumption 2: For any data batch B, there exists positive constant M such that the variance of the
3. Experiments
3.1. Datasets and Implementation Details
Datasets. The proxy datasets Divain proay for MPQ search include CIFARIO (Krizhevsky et al., 2009). Flowers (Nils- back & Zisserman, 2008), and Food (Bossard et al., 2014).
5

---

Learning from Loss Landscape: Generalizable Mixed-Precisioe Quantization via Adaptive Sharpness-Aware Gradient Aligning
For image classification, the target large dataset Deat for model inference is ImageNet. For object detection, the target dataset is VOC (Everingham et al., 2010).
Table 1. Accuracy and efficiency results for ResNet and MobileNet- V2. "Top-1/5 1/5" represents the Top- 1/5 accuracy of quantized model. W/A denotes the bitwidths of weights and activations. "MP" means mixed-precision quantization. "Con-epoch" denotes the number of epochs used for convergence. (.) A denotes the haseline method
We search for the MPQ policy using the training samples of small proxy datasets. Afterwards, we quantize and fine- tune the model with the searched policy on the target large datasets, where the basic data augmentation methods are adopted. The final performance of the model is then evalu- ated on the ImageNet and VOC validation sets.
with ASGA.
Methods
W/A
con-epoch ResNet18
Top-1
Top-5
ALQ
3/3
115
65.6
87,0
A+A
3/3
92(23.)
66.1(0.61)
87.1(0.11)
EWGS
3/3
123
53.85
77,9
E+A
3/3
98(25L)
53.86(0.011)
18.12/0.22")
Models. We employ four representative network architec- tures for experiments, including: (I) ResNet-18, 50 (He et al., 2016), MobileNet-V2 (Sandler et al., 2018) for image classification, and (2) ResNet-18 18 and VGGI6 (Simonyan & Zisserman, 2014) for object detection.
EdMIPS
3MP/3MP
110
66.4
86.1
E+A
3MP/3MP
75(35L)
(7.9(1.5))
87.7(1.11)
GMPQ
3MP/3MP
96
66.3
85.4
G+A
3MP/3MP
80(16))
66.4(0.11)
10.7
SEAM
3MP/3MP
96
65.1
S*+A
3MP/MMP
85(11
65.8(0.71)
ResNet50
Parameters. For MobileNet-Vi V2 and VGG16, the bitwidth candidates of weights and activations are B B {2.3, 4,5,,,,, 5, 6}. For ResNet, the candidate bitwidths are B = Ba = {2, 3, 4, 6). Following the previous arts (Esser et al., 2019; Wang et al., 2019), the first and last layers are fixed to 8 bits.
HAWQ
4MPAMP
98
74.6
H+A
4MP/MM
92(6()
74.9(0.31)
HAQ
4MP/8
125
72.6
91.4
HAQ+A
4MP/8
101(24L)
73.5(0.91)
91.6(0.21)
EdMIPS
4MP/MM
119
71.1
90.4
E+A
4MP/MM
108(11L)
719(0.81)
91.0(0.61)
GMPQ
4MP/MM
104
71.0
90.1
G+A
4MP/4MP
82(22.)
715(0.55))
90.9(0.81)
For searching, we set the initial perturbation radius as po = 0.1, and finetune it in the ablation study. We finetune the
MobileNet-V V2
DQ
4MP/4MP
94
65.1
hyper-parameter X in Eq. (3) according to prior works on
D+A
4MP/4MP
87(7.))
65.3(0.21)
HMQ
4MP/MMP
102
68.9
DMPQ (Cai & Vasconcelos, 2020; Wang et al., 2021), where a higher À value indicates a less computation complexity policy to seek.
H+A
4MP/4MP
90(12.))
69.1(0.21)
GMPQ
4MP/4MP
100
68.8
87.4
G+A
4MP/4MP
91(9.)
69.1(0.31)
88.4(1.0g
SEAM: regenduction of uspublished code based on (lang al.,, 2023)
For finetuning (quantizing), we follow the basic quantization-aware training settings in ALQ (Qu et al., 2020) and EWGS (Lee et al., 2021).
For ResNe150, we can observe that ASGA achieves best trade-off between prediction accuracy and search efficiency. That is, ASGA acquires quite similar accuracy performance, but using much less search time (e.g.. than gradient- maadee methods HAWQ and HAQ). These results again demonstrate the effectiveness of our method in enhancing quantization generalization.
More implementation details are provided in Supplementary Material A.5.
3.2. Comparison with State-Of-The-Art
We compare our ASGA with the SOTA quantization meth- ods on the classification task, including ALQ (Qu et al., 2020), EWGS (Lee et al., 20 HAWQ (Dong et al., 2019), HAQ (Wang et al., 2019), DQ (Uhlich et al., 2019), HMQ (Habi al., 2020), GMPQ (Wang et al., 2021) and EdMIPS (Cai& & Vasconcelos, 2020)
also reports the results on MobileNet-V V2 under mixed 4-bits level. For mixed 4-bits results on CIFARIO, it can be found that ASGA performs better than all the compared SOTA mixed-precision meth- ods (DQ, HMQ and GMPQ). More specifically, our ASGA arises a 0.2% absolute gain on Top-1 accuracy over HMQ, and 0.3% higher accuracy than GMPQ. For mixed 4-bits searched on CIFARIO 10, ASGA obtains up to 1.13X searching efficiency improvement over HMQ.
MobileNet-V2. Table 1
3.2. 1. RESULTS ON IMAGENET
ResNet. The comparison results for ResNet- (18, 50) on ImageNet are listed in Table 1. For ResNet18, under mixed 3-bits BOPs constraints, our ASGA consistently achieves higher accuracy (in terms of both Top-1 and Top-5) than the baseline methods, while it obtains about 1.5X policy search speedup compared to EWGS, showing its advantage in re- ducing search costs. Especially, ASGA finds the optimal MPQ policy using only 8189 training samples, which may be owing to the small data amounts of Flowers.
3.2.2. RESULTS ON VOC
VGG16. We adopt the SSD detection framework with VGG16 backbone to evaluate ASGA in object detection task. Table 2 reports the results of VGGI6 without and with ASGA over multiple baseline methods. For the the accuracy-complextn trade-off, our ASGA performs power- fully, achieving better accuracy results with less search cost
6

---

Learning from Loss Landscape: Generalizable
Quantization via Adaptive Sharpness-Aware Gradient Aligning
95
89
ResNet18
85
90.62
90.57
90.52
90.23
90.16
RosiNoc50
90
91.65
91.75
91.74
91.89
91:75
MobieNNr--V -22
84
85.99
56.01
$5.78
86:12
80
85
79
68
30
74
70
0.05
0.2
adayiic
0.05
0.1
adaptive
adaytive
rcar-ppodh
Top-l
Figure 4. Comparison of Top-1 accuracy -)) and convergence epochs (- -) of ResNet18, ResNet50, and MobileNet- V2 CIFARIO with different p. The result shows that adaptive p can significantly reduce the search cost without performance deterioration.
Table 2. The mAP (%) on VOC of VGGI6 and ResNet18. W/A denotes the bitwidths of weights and activations. "MP" means mixee-precision quantization. "Con-epoch" represents epochs used by the method to achieve convergence. Param. means the model storage cost, and (+) + A denotes the bascline method with ASGA.
Table 3. Comparison of MPQ policies search results for ResNet18 different datasets. σMA and AMAAA represent the average men for the baselines without/with ASGA, respectively.
Datasets
UMA
OMA-A
Top-1
Top-5
CIFARIO
13.5
(2.1.)
90.86
99.62
Flowers
48.4
44.4 (4.0.)
87.32
98.75
Methods
W/A
Param. VGGI6 with SSD
con-epoch
mAP
Food
25.8
23.1 (2.7L)
84.60
HAQ
4MP/
47.6
87
69.4
H+A
4MP/8
46.2(1.4))
79(8.))
69.4(-)
EdMIPS
4MP/4MP
39.4
94
67.7
3.3.2. EFFECTIVENESS OF PROXY DATASETS
E+A
4MP/MMP
39.1(0.31)
73(21J)
67.8(0.11)
ResNet18 with Faster R-CNN
There is no relevant literature to study the effect of proxy subsets. The searched results of our method on different small-scale proxy datasets, including CIFARIO, Flowers and Food, are shown in Table 3, where for the baseline, we set Po = 0.1 for calculating sharpness. Compared to the baseline without sharpness regularization, the one with ASGA achieves flatter loss landscape across all datasets, and gets the best accuracy on CIFAR10. As shown in Figure 6, using CIFARIO as proxy dataset, we can find more well- performing MPQ policies than other datasets. This may be because the categories of CIFAR 10 more similar to those of target ImageNet compared to other datasets. In this sense, proxy dataset with more class-level similarity to target dataset could be considered to improve the performance better. Figure 5 shows the weight and activation bitwidth assignment for ResNet18, ResNet50 and MobileNet- V2.
HAQ
ЗMP3MP
22.3
82
69.0
HAQ+A
3MP/3MP
21.8(0.5L)
76(5.L)
69.3(0.31)
HAWQ
3MP/3MP
26.3
85
66.4
11+A
3MP/3MP
25.9(0.41)
81(4))
66.6(0.21)
than SOTA methods on VOC. Moreover, directly using the policy found by HAQ and EdMIPS on CIFAR10 10 to quantize target model suffers from significant performance degrada- tion on VOC. Since the mixed- peeeed-preiiioon models needs to be pretrained on ImageNet, the search cost decrease on VOC more sizable than that on ImageNet.
ResNet. Table 2 also shows the results of ResNet18 with Faster R-CNN in object detection task. After applying ASGA, EdMIPS can achieve the same accuracy as before with only 77% of the search cost, which significantly im- proves the search efficiency.
3.3.3. EFFECTIVENESS OF TARGET DATASET'S ' SUBSET
3.3. Ablation Study
We randomly select 60K images from target dataset Ima- geNet (i.e., 0.45% of target ImageNet training data) to form a subset, whose size is similar to that of CIFAR1O. We use them to search for a 3MP policy for ResNet18, and the results are shown in Table 4. From this table, we obtain following observations: (1) the subset of ImageNet without proposed ASGA suffers about 1.6% performance degrada- tion compared to CIFARIO with ASGA. This demonstrates that the data distribution in the subset is still different from the full set, which necessitates the adoption of our ASGA. (2) For both subset and CIFARIO, the use of our ASGA leads to obvious accuracy improvement over the baseline. These results again validate the effectiveness of our ASGA method.
3.3. 1. EFFECTIVENESS OF p
In fact, sharpness-awareee awareness method is sensitive to the per- turbation radius Especially, an excessively large would incur a sharp decline generalization performance, while an excessively small one may cause the model to learn the noise or high-frequency details in the images instead of generalized features. Figure shows the comparison of MPQ policy search accuracy and the convergence epochs of ResNet18, ResNet50, and MobileNet-V2 with different fixed and adaptive p on CIFARIO, 10, respectively. We observe that adaptivity P achieves faster convergence compared to a fixed p without accuracy degradation.
7

---

Learning from Loss Landscape: Generalizable
Quantization via Adaptive Sharpness-Aware Gradient Aligning
axigement R
Rinwitth sesignment ReaNas50
Wrighe Sut
MAniiaaii Bi.
Biraidh
Biowilth exvigmment MubileNaa-V2
Biiggle Bistsaaiii
Index
index
Figure 5. Bitwidth assignment for each layer of 88 ResNet50, and MobileNet-V2
Table 4. Results of ablation study on the subset of ImageNet. Subset refers to the subset extracted from ImageNet. "Top- 1/5" represents 1op1/5 accuracy respectively. / and X represent the baseline method (FdMIPS) with/without ASGA, respectively.
80
Pertorbation Loss
70
CIFARIO
4.65
Flowees Food.
Perturhettom aeteed m
60
6.0
50
6/8
30
608
Dutasets
ASGA
Top-1
Top-5
20
10
600
Subset
X
66.3
85.6
Subset
67.3 (1.01)
86,9 (1.31)
6/6
15
30
45
90
4137
CIFARIO
X
66.4
86.1
epocias
Pertufhation added 6:
CIFARIO
67.9 (1.5t)
87.7(1.61)
(a) Varying datasets
(b) Loss landscape of ResNet18
rank consistency, and SEAM (Tang et al., 2023) introduces the class margin regularization to enhance MPQ search capability. However, both of them suffer from intricate calculation of feature maps, increasing search costs.
Top-1 Top-1 Accuracy (%)
67
65
63.8
63.1
63
61
CIFARIO
Flowers
Food
Sharpness and Generalization. Recent studies (Keskar et al., 2016; Cha al., 2021; 2022) have revealed direct relationship between sharpness and generalization capability of DNNs. Keskar et al. (Keskar et al., 2016) make the first attempt to offer a sharpness measure and demonstrate its correlation with the generalization ability. Dinh et al. (Dinh et al., 2017) further argue that the sharpness measure bears a relation with the spectrum of the Hessian. SWAD (Cha et al., 2021) provides a theoretical insight that a flatter minimum can lead to smaller generalization gap under some out-of- distribution conditions. Then, SAM (Foret et al., 2020) aims to identify parameter regions that exhibit relative "flatness" in the vicinity of the low loss landscape, leading to a set of SAM improvements, e-g., ASAM (Kwon et al., 2021), Fisher SAM (Kim et al., 2022), and GSAM (Zhuang et al.). Several studies (Dung al., 2024) have utilized sharpness to enhance data generalization for quantization. However, no existing works attempt to apply the concept of sharpness for realizing generalization of MPQ.
Dalasets
(c) Top-1 accuracy on ImageNet
Figure 6. (a) Comparison of dmazz variations for ResNet18 three datasets. (b) The heatmap partial loss landscape R on CIFARIO. White denotes a lower perturbed loss and blue denotes a higher onc. (c) Comparison of Top-1 accuracy on ImageNet for MPQ policy searched on three proxy datascts. The MPQ CIFAR 10 obtains the lowest maa and highest Top-1 on ImageNet.
4. Related Work
Mixed-Precision Quantization. Unlike FPQ (Zhou et al., 2016) assigning uniform bitwidth for layers, MPQ aims to allocate different bitwidths to different layers (Elthakeb al., 2020). The key challenge realize this is how to find the optimal bitwidth for each layer from the exponential discrete bitwidth space. A series of intelligent methods have been developed for solving the above issue, c.g., HAQ (Wang et al., 2019). SPOS (Guo et al., 2020), and EdMIPS (Cai & Vasconcelos, 2020). Especially, DNAS (Wu al., 2018) and BP-NAS (Yu al., 2020) formulate the MPQ process as a Neural Architecture Search (NAS) problem to search for optimal MPQ policy. Also, there exist several once quantization-awann
5. Conclusion
In this work, we propose ASGA- DMPQ to search for transferable quantization policy with small amount of data. To bridge the generalization gap, we not only center around enhancing the accuracy on small dataset, but also minimizing the sharpness measure of the loss landscape in MPQ search. We consider this as a loss-sharpness infor- mation exploitation on small proxy dataset, which shows a
methods, e.g.. HAWQ (Dong et al., 2019) (Yao et al., 2021) and MPQCO (Chen et al., 2021), which exploit the second-order quantization information to learn optimal bitwidths. Recently, GMPQ (Wang et al., 2021) proposes to seek optimal policy by using attribution
8

---

Learning from Loss Landscape: Generalizable
Quantization via Adaptive Sharpness-Aware Gradient Aligning
dataset-independent attribute. Then, we design an adaptive sharpness-awang gradient aligning (ASGA) method and in- troduce it into differentiable MPQ search paradigm, aiming to acquire transferable MPQ policy from small dataset to target large dataset. The theoretical analysis and experimen- tal results validate our idea, and we use only small dataset with 0.5% the size of large one to search for quantization policy, achieving equivalent accuracy on large datasets with speeding up the search efficiency by up to 150%.
Choi, J., Wang, Z., Venkataramani, S.,, Chuang, P. L.J., Srini- vasan, V., and Gopalakrishnan, K. Pact: Parameterized clipping activation for quantized neural networks. arXiv preprint 10Xvv1805.00085, 2018.
Deng. J., Dong, W., Socher, R., Li, I..-J,, Li, K., and Fei-Fei, L. Imagenet: A large-scale hierarchical image database. In 2009 IEEE conference on computer vision and pattern recognition, pp. 248 248-255, Icee, 2009.
Deng. L.,, Li, G., Han, S., Shi, L.., and Xie, Y. Model com- pression and hardware acceleration for neural networks: A comprehensive survey. Proceedings of the IEEE. 108 (4):485-532, 2020.
Acknowledgements
This work is supported in part by the National Natural Sci- ence Foundation of China under Grant 62472079 and the Fundamental Research Funds for the Central Universities (No.N2417003).
2020..
Dinh, L., Pascanu, R., Bengio, S., and Bengio, Y. Sharp min- ima can generalize for deep nets. In International Con- ference Machine Learning, pp. 1019-1028. PMLR, 2017.
Impact Statement
Mixed Precision Quantization (MPQ) becomes a promising technique for compressing deep model, but suffers from high search burden. This paper solves this issue via intro- ducing a generalizable quantization paradigm, motivated by learning from loss landscape. Our approach provides fresh insights for advancing MPQ. This work has no ethical aspects as well as negative social consequences.
Dong, Z., Yao, Z., Gholami, A., Mahoney, M. W., and Keutzer, K. Hawq: Hessian aware quantization of neural networks with mixed sexxedpprecisoon. In Proceedings of the IEEE/CVF international conference on computer vision, pp. 293-302, 2019.
Dung. H. A., Pham, Lc, T., Cai, J., and Do, T.-T. Sharpness-awan aware data generation for zero-shot quantiza- tion. In Forty-first International Conference on Machine Learning, 2024. URL https: /ppp//veeneeeeeeeeee/ee/ee .nt// forum? forum?id-al (adddmmKKKMmn......
References
Bossard, L., Guillaumin, M., and Van Gool, L. food-101- mining discriminative components with random forests. In Computer vision-ECCV 2014: 13th European con- ference, zurich, Switzerland, September 6-12, 2014, pro- ceedings. part VI 13, pp. 446-461. Springer, 2014.
Elthakeb, A. T., Pilligundla, P., Mireshghallah, F., Yazdan- bakhsh, A., and Esmaeilzadeh, H. Releq: A reinforce- ment learning approach for automatic deep quantization of neural networks. JEEE micro, 40(5):37-45, 2020.
Cai, Z. and Vasconcelos, N. Rethinking differentiable search for mixed-precision neural networks. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 2349-2358. 2020.
Esser, S. K., McKinstry, J. L., Bablani, D., Appuswamy, R., and Modha, D. S. Learned step size quantization. arXiv 2019.
preprint
Everingham, M., Van Gool, L., Williams, C. K., Winn, J., and Zisserman, A. The pascal visual object classes (voc) challenge. International journal of computer vision, 88: 303 133-338,1 2010.
Cha, J., Chun, S., Lee, K., Cho, H.-C,, Park, S., Lee, Y., and Park, S. Swad: Domain generalization by seeking flat minima. Advances in Neural Information Processing Systems, 34:22405 24422405-22418,, 2021.
Foret, P., Kleiner, A., Mobahi, H., and Neyshabur, B. Sharpness-aware minimization for efficiently improving generalization. arXiv preprint arXiv:2010.01412. 2020.
Cha, J., Lee, K., Park, S., and Chun, S. Domain gener- alization by informatinooomttionn regularization with pre- trained models. In European conference on computer vision, pp. 450- Springer, 2022.
Guo, Z., Zhang, X., Mu, H., Heng, W., Liu, Z., Wei, Y., and Sun, J. Single path one-shot neural architecture search with uniform sampling. In Computer Vision-ECCV 2020: 16th European Conference, Glasgow, UK, August 23-28, 2020, Proceedings, Part XVI 16, pp. 544-560. Springer,
Chen, W., Wang, P., and Cheng, J. Towards preccisinn quantization of neural networks via constrained optimiza- tion. In Proceedings of the IEEE/CVF International Con- ference on Computer Vision, pp. 5350-5359, 2021.
2020.

---

Learning from Loss Landscape: Generalizable
Quantization via Adaptive Sharpness-Aware Gradient Aligning
Habi, H. V., Jennings, R. H., and Netzer, A. Hmq: Hardware friendly mixed precision quantization block for enns. In Computer Vision-ECCV 2020: 16th European Confer- ence, Glasgow, UK, August 23-28, 2020. Proceedings, Springer, 2020.
Sandler, M., Howard, A., Zhu, M., Zhmoginov, A., and Chen, L.-C. Mobilenetv2: Inverted residuals and linear bottlenecks. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 4510-4520, 2018
Part XXVI 16, pp.
He, K., Zhang, X., Ren, S., and Sun, J. Deep residual leamn ing for image recognition. In Proceedings of the IEEF conference on computer vision and pattern recognition, pp. 770-778, 2016.
Shuvo, M. M. H., Islam, S. K., Cheng, J., and Morshed, Efficient acceleration of deep learning inference on resource-constraine edge devices: A review. Proceed- ings of the IEEE, 111(1):42-90) 2022.
Jiang, W., Yang, H., Zhang, Y., and Kwok, J. An adaptive policy to employ sharpness-awes aware minimization. arXiv preprint ar arviv:2304. 14647, 2023.
Simonyan, K. and Zisserman, A. Very deep convolu- tional networks for large-scale image recognition. arXiv preprint arXiv: 1409. 1556, 2014.
Keskar, N. S., Mudigere, D., Nocedal, J., Smelyanskiy, M., and Tang, P. T. P. On large- training for deep learning: Generalization gap and sharp minima. arXiv preprint arXiv: 160X9 04836, 2016.
Sutskever, I., Martens, J., Dahl, G., and Hinton, G. On the importance of initialization and momentum in deep learn- ing. In International conference on machine learning, pP. 1139-1147. PMLR, 2013.
Kim, M., Li, D., Hu, S. X., and Hospedales, T. Fisher sam: Information geometry and sharpness aware minimisation. In International Conference on Machine Learning, pp. 11148-11161. PMLR, 2022.
Tang, C., Ouyang, K., Chai, Z., Bai, Y., Meng, Y., Wang, Z., and Zhu, W. Seam: Searching transferable mixed- precision quantization policy through large margin regu- larization. In Proceedings of the 31st ACM International Conference on Multimedia, pp. 7971-7980, 2023.
Krizhevsky, A., Hinton, G., et al. Learning multiple layers of features from tiny images. 2009.
Uhlich, S., Mauch, L., Yoshiyama, K., Cardinaux, F., Garcia, J. A., Tiedemann, S., Kemp, T., and Nakamura, A. Dif- ferentiable quantization of deep neural networks. arXiv preprint arXiv: /905. 1452, 2(8), 2019.
Kwon, J., Kim, J., Park, H., and Choi, I. K. Asam: Adaptive sharpness- aware minimization for scale-invariant learn- ing of deep neural networks. In International Conference on Machine Learning, pp. 5905-5914 14. PMLR, 2021.
Wang, K., Liu, Z., Lin, Y., Lin, J., and Han, S. Haq: Hardware- automated quantization with mixed pre- cision. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), June 2019.
Lee, J., Kim, D., and Ham, B. Network quantization with element-wise gradient scaling. In Proceedings of the IEEE/CVF conference on computer vision and pattern Adamon, pp. 6448 6457, 2021.
Ma, L., Zhou, Y., Ma. J., Yu, G., and Li, Q. One-step forward and backtrack: overcoming zig-7agging in loss- aware quantization training. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pp. 14246-14254, 2024.
Wang. P., Zhang, Z., Lei, Z., and Zhang, L. Sharpness- aware gradient matching for domain generalization. In Proceedings of the IEEE/CVF Conference on Computer Vision and Puttern Recognition, pp. 3769-3778, 2023.
Wang. Z., Xiao, H., Lu, J., and Zhou, J. Generalizable mixed-precision quantization via attribution rank preser- vation. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 5291-5300, 2021.
McAllester, D. Simplified pac-bayesian margin bounds. In Learning Theory and Kernel Machines: 16th Annual Conference on Learning Theory and 7th Kernel Workshop, COLT/Kernel 2003, Washington, DC, USA, August 24-27, 2003. Proceedings, pp. 200-2022222 215. Springer, 2003.
Wu, B., Wang, Y., Zhang, P., Tian, Y., Vajda, P., and Keutzer, K. Mixed precision quantization of convnets via dif- ferentiable neural architecture search. arXiv preprint arXiv: 182200009, 2018
Nilsback, M.-E. and Zisserman, A. Automated flower clas- sification over a large number of classes. In 2008 Sixth Indian conference on computer vision, graphics & image processing, pp. 722-729. IEEE, 2008
Yao, Z., Dong, Z.,, Zheng, Z., Gholami, A., Yu, J., Tan, E., Wang, L., Huang, Q., Wang, Y., Mahoney, M., et al. Hawq-v3: Dyadic neural network quantization. In Inter- national Conference on Machine Learning, pp. 1 11875- 11886. PMLR, 2021.
Qu, Z.. Zhou, Z., Cheng, Y., and Thiele, L. Adaptive loss- aware quantization for multi-bit networks. In Proceedings of the JEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 7988-7997, 2020.
10

---

Learning from Loss Landscape: Generalizable Mixed-Precision Quantization via Adaptive Sharpness-Aware Gradient Aligning
Yu, H., Han, Q., Li, J., Shi, J., Cheng, G., and Fan, B. Search what you want: Barrier panelty for mixed precision quantization. In Computer Vision-ECCV 2020: 16th European Conference, Glasgow, UK, August 23-28, 2020, Proceedings, Part IX 16, pp. 1-16. Springer, 2020.
Zhao, X., Xu, R., Gao, Y., Verma, V., Stan, M. R., and Guo, X. Edge-mpq: Layer-wise mixed -xxxedprecision quantization with tightly integrated versatile inference units for edge computing. IEEE Transactions on Computers, 2024.
Zhou, S., Wu, Y., Ni, Z., Zhou, X., Wen, H., and Zou, Y. Dorefa-net: Training low bitwidth convolutional neural networks with low bitwidth gradients. arXiv preprint arXiv: 2016.
Zhuang, J., Gong, B., Yuan, L., Cui, Y., Adam, H., Dvornek, N. C., Duncan, J., Liu, T., et al. Surrogate gap minimiza- tion improves sharpness-aware training. In International Conference on Learning Representations.

---

Learning from Loss Landscape: Generalizable Sixxe-Prrccisoon Quantization via Adaptive Sharpness-Aware Gradient Aligning
Overview of the Supplementary Material
The main contents of this Supplementary Material are as follows:
Section A.1 Differences between Our Method and Conventional MPQ Methods
Section A.2 Details of Adaptive Charpness-Aware Gradient Aligning.
Section 1 Details of the directional consistency among Vh(O), VL()), and (C,
Section A.2.2 Details of Gradient Alignment.
Section A.3 Details of adaptive strategy of controlling
Section A.4 Proof of Lamma 1.
Section A.5 Datasets and Implementation Details.
A. Supplementary Material
A.1. Differences between Our Method and Conventional MPQ Methods
Figure 7 demonstrates the difference between our proposed method and conventional MPQ mehohds. More specifically, our proposed ASGA-bbase GMPQ method (termed as GMPQ-ASGA) aims lear transferable MPQ policy via loss sharpness optimization for efficient inference. Unlike existing methods that require the dataset consistency between quantization policy search and model deployment, our approach enables the acquired MPQ policy to be generalizable from proxy datasets to target large-scale datasets. The MPQ policy found small-scale datasets achieves promising performance on challenging large-scale datasets, so that the MPQ search burden is largely alleviated.
ImageNet
ImageNet
leyar
Search
Finetune
layer
leyar i+1
Conventional MPQ
CIFAR10
leyar i-1
Search
Finetune Generlizable
ImageNer
Iayer itl
GMPQ-ASGA
Figure 7. Conventional methods require the consistency of datasets for bitwidth search and model deployment, while our GMPQ searches the optimal quantization policy on small datasets and generalizes it to large-scale datasets.
A.2. Details of Adaptive Sharpness-Aware Gradient Aligning
1. DETAILS OF THE DIRECTIONAL CONSISTENCY AMONG Vh(O), VL((), AND ())
In Section 2.2, we know that:
Vh(0) = (O, VL(0).
(20)
According to Eq. (20), when the directions of V L and V (0) are the same, there exists a scalar v such that:
(h(0)= = VC, V = VZ) v)V - VL(0) = (1 1)V
(21)
When 77 > 1. the direction of (h()) is exactly the same as that of (C))), and its length is expanded by (1- 1). When 0 < 77 < 1, (h(f) and (())) are still in the same direction, but the length of Vh(d) is reduced. This means that, regardless of the value of n, (h(@) always maintains the same direction as VC()). Therefore, we need to align the gradients (()) and (0) so that h() can be effectively optimized under the above-mentiones circumstances.
12

---

Learning from Loss Landscape: Generalizable
Quantization via Adaptive Sharpness-Aware Gradient Aligning
A.2.2. DETAILS OF GRADIENT ALIGNMENT
In this section, we present the detailed design of the gradient alignment in ASGA. Following the work (Wang et al., 2023). perform Taylor expansion of (0) Eq. (9) and obtain:
min (pp( ~V2(())
-V VC(() + +(
0
(22)
~ min((,(() - (4),())))--(,,00) VZ(()),
where (()))()) represents a higher-order infinitesimal term that becomes negligible as fr approaches Intuitively, we can see that minimizing Eq. (22) is actually equivalent to minimizing (0) while maximizing the inner product of (0) and VZ(
More specifically, minimizing (0) is conductive to searching for sufficiently low minimum, and minimizing the inner product can ensure that the loss landscape near the minimum is sufficiently flat. If the gradient direction of (O) is substantially similar to that of V VC))), their inner product will be greater than O, and achieves the maximum value when the directions are completely aligned.
As stated above, we can clearly see that the gradient direction of h()) is influenced by both VC(O) and (O, VL and (0) deviate significantly, severe gradient conflict will prevent the simultaneous optimization of all the three loss objectives. Conversely, when (Z) and (0) are aligned, the three will be optimized efficiently and consistently, In this sense, aligning VL( and V (0) facilitates the achievement of the optimization objectives in the given formula.
A.3. Details of adaptive strategy of controlling
According to Section 2.2, we know that p should be gradually increased effectively continue optimizing the sharpness of the model. Therefore, the value of p should meet the following requirements:
It must always be greater than O;
There must be an upper limit Pmawi
It should adaptively increase as the h(0)'s increase rate decreases.
Based on the aforementioned considerations, we formulate an adaptive strategy to adjust the value of h:(0) according to the status of sharpness of the loss landscape, which is defined as
minmpmaas
(23)
In(h(f) + 1)
Our design has the following merits:
(1) The characteristies of the logarithmic function in Eq. (23) ensure that when h(()) (which is also the case in reality). is greater than 0.
(2) The term Pmaz in Eq. (23) ensures that p will not increase indefinitely, thus preventing instability issues in the optimization process.
(3) In the early stages of policy search, the value of h(0) is relatively large, while the value Ia(h(P)+1) is small. As the loss landscape becomes increasingly flatter, the value (0) gradually decreasing, and [(( is increasing accordingly. This dynamic characteristic allows the value of p to be adaptively adjusted according to the changes in h(O), avoiding the issue of a fixed being unable to sensitively capture changes when h(0) is small.
Morcover, the scaling factor ò in Eq. (23) is able to freely adjust the value range of to adapt to different tasks, as shown in Figure 8, which illustrates the curves of p varying with h(0) under different combinations of Pmaz and Φ.
13

---

Learning from Loss Landscape: Generalizable Preed-Precision Quantization via Adaptive Sharpness-Aware Gradient Aligning
p min (Paaa: ln(
0.35
Priz 0.30, 0.035
Prose 0.25, 0.040
Prozz 0.20, 0.045
0.25
0.2
0.15
0.05
0.2
0.3
2.4
h(()
Figure 8. The curves of p varying with h()) under different combinations of pmar and Φ.
A.4. Proof of Lamma 1.
Lemma 1. Suppose the training set contains vn elements drawn i..dd from the true distribution and the average loss in MPQ
search is C(0)= = 3 En= Li (0, 04, U.) (0) for short), where (Is, 3i) is the input-target pair of i-th element. Suppose 0
is drawn from posterior distribution T. For the prior distribution (independent of training) S, we have:
BonTEB Lo( S (0) + R,,
(24)
with probability at least (1 - a) - (.20 v where a E (0,1) denotes confidence level, and R
/V (KL(TIIS) + log 2m)) /m. This shows that ASGA effectively reduces the upper bound of the generalization error during the MPQ process.
Proof. According the PAC-Bayeing theory (McAllester, 2003), we can employ Probably Approximately Correct (PAC) inequalities to establish the generalization bounds for various machine learning models. Here, the generalization bound reefers to the upper limit on the divergence between the model's empirical loss (estimated from training data) and the population loss (which is the loss incurred on the true, underlying data distribution). The optimization of these bounds fosters the emergence of self-certified learning algorithms. According to the Bayesian theory, for a well-trained quantized model derived from the optimal MPQ policy, the upper bound of its expected loss convergence can be expressed as:
Ex, (0) < F + R.
(25)
Here, L()) can be considered as a local minimum at this point, and then (p) which is defined in Eq. (6), is always greater than ((0). As a result, the following inequality holds:
Eg Ez, (,) I h(0) + [()) + R,
(26)
with a probability of (1 -1-a) a) 1 - e
- Vis
vk) 27 v
less than (1 - a), which implies that minimizing h(0) is expected to achieve a tighter upper bound of the generalization performance. Therefore, Lemma is proven. Moreover, R in Eq. 26 is typically hard to analyze and often simplified to L2 regularization. Note that (p) h(0) + L(0) only holds when p equals Pture (the ground truth value determined by underlying data distribution); when p * Ptures min h())) is more effective than min (0) in terms of minimizing generalization loss.
A.5. Datasets and Implementation Details
Details of datasets. CIFAR10 consists of 60K images of 32 X 32, divided into 10 categories. Flowers has 102 categories and each category involves 40 to 258 images. Food contains 32135 high-resolution physical images from 6 restaurants. For image classification, the target large dataset D Dynt for model inference is ImageNet (Deng et al., 2009) with 1000 categories, containing 1.28M training samples and 50K validation samples. For object detection, the target dataset is VOC (Everingham et al., 2010) with 20 categories, containing about 1.6K training samples and SK validation samples.
14

---

Learning from Loss Landscape: Generalizable
Quantization via Adaptive Sharpness-Aware Gradient Aligning
Details of models. We employ four representative network architectures for experiments, including: (1) ResNet-18, 50 (He al., 2016), MobileNet-V2 (Sandler et al., 2018) for image classification, and (2) ResNet-118 and VGG16 (Simonyan & Zisserman, 2014) for object detection.
Details of parameters. For policy searching, we adopt the SGD as base optimizer, and the initial learning rate is set to 0.01 for 90 epochs. Empirically, we find the sharpness of loss landscapes is not sensitive to the hyper-parameter and thus set e 0.1 for all proxy datasets. We use the full-precision model (trained on Divain) as the initialization and adopt the SDG optimizer with Nesterov momentum (Sutskever al., 2013) and the initial learning rate is 0.04. We use the cosine learning rate scheduler and finetune the model until convergence and the first 5 finetune-epochs are used as warm-up.
The key notations and experimental parameters used in this paper are listed in Table 5 and Table 6, respectively.
Table 5. Key notations in this paper.
Notation
Description
the loss value of the model the perturbation loss value of the model
Cp Eval Earain Ccomp
the task loss the validation dataset task on the train dataset the complexity loss
0
the weights set of quantized network the optimal weights set of quantized network the proxy dataset in MPQ search stage
8"
Crassn- prozy Deal
the dataset containing all validation data in model inference stage the dataset containing all training data in model inference stage the quantization policy
Dtrain
the consrae-conttr conditions the perturbations to model weights the perturbations radius the scaling factor to control p the surrogate gap
0 max
upper limit the value of p the scaling factor to control the ascent step the scaling factor for weighting the corresponding loss the optimization process the posterior distribution of & under the optimal MPQ policy
j
the prior distribution of confidence level
B
the predefined bitwidth candidate sets for weights the predefined bitwidth candidate sets for activations
Table 6. Key experimental parameters in the paper.
Parameter
Scope of application
Setting
epochs for scarching
all
90
base optimizer
all searching stage inference stage
SGD
0.01
learning rate
0.04
all all MobileNet-V2 V2, VGG16
0.1
Pa
0.1
(,,3,4,5,6)
B
ResNet18. ResNet50
(2,3,4,6)
MobileNet-V V2, VGG16
(2,3,4,5,0)
Ba
ResNet18, ResNet50
(2,3,4,6)
15


Reviews:
[{"review_id": "tbMug4n6tY", "reviewer": "Reviewer_XyLq", "summary": "This paper proposes an Adaptive Sharpness-Aware Gradient Aligning (ASGA) method for generalizable mixed-precision quantization. ASGA aims to address the issue of excessive search burden in MPQ through quantization generalization, i.e., searching for quantization policies on small proxy datasets and then generalizing them to the large-scale dataset tasks. Specifically, authors consider a series of sharpness improvement strategies, such as SAM, gradient alignment, and adaptive perturbation radius strategy, which can balance accuracy, complexity, and generalization in the MPQ search. The paper conducts both theoretical analysis and experiments on a set of datasets, which have validated the effectiveness of the proposed method. Ablation studies confirm the effectiveness of its components. In summary, this work provides a new way to address MPQ's high search burden.\n\n## update after rebuttal", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "## Q1 The authors should briefly explain why σmax can measure the sharpness of the loss landscape.\n\nA1: Many thanks for your comments! First, reference [1] has proven that $\\sigma_{max}$ is positively correlated with the sharpness of the loss landscape. Moreover, $\\sigma_{max}$ is the eigenvalue with the largest absolute value of Hessian, which reflects the curvature information of the loss landscape near a local minimum. In other words, a larger curvature indicates a sharper loss surface, leading to poorer model generalization. Conversely, smaller $\\sigma_{max}$ corresponds to flatter regions where the loss changes gradually, leading to better generalization. Therefore, using $\\sigma_{max}$ as a measure of the sharpness of the loss surface is appropriate.\n\n---\n\n[1]: Surrogate Gap Minimization Improves Sharpness-Aware Training, ICLR 2022.\n\n---\n\n## Q2 Authors enhance model generalization by minimizing the loss sharpness. However, during actual training, the model may get stuck in locally flat regions, leading to convergence to suboptimal solutions. How can we ensure that the ASGA method, while pursuing a flat loss landscape, does not excessively get trapped in locally flat regions and miss the global optimum solution?\n\nA2: Thanks for the insightful comment! Indeed, avoiding convergence to flat regions while pursuing a flat loss landscape is a critical challenge. We recognize this issue, and thus incorporates both empirical loss and perturbed loss into the design of the ASGA loss function. This aims to maintain low loss value while seeking a flat landscape. In addition, the implicit gradient alignment mechanism helps minimize sharpness while preventing convergence stagnation caused by gradient conflict. This ensures that the model converges along a flat yet globally optimal direction guided by the loss landscape. From an experimental perspective, we validate our strategy across multiple datasets, demonstrating that our method can **obtain superior quantization policies to direct optimization on the target dataset while simultaneously improving generalization performance**.\n\n## Q3 Has the author considered extending the ASGA method to other fields? For example, applying the ASGA method to other types of quantization methods, such as binary quantization?\n\nA3: Many thanks. We are indeed considering applying ASGA to other quantization methods. The core idea of ASGA - improving model generalization by optimizing the sharpness of the loss landscape - **is fundamentally generalizable**.  For instance, the binary quantization you mentioned represents an extreme form of low-bit quantization. However, this method has very strong constraints, which may lead to a drastic change in the loss landscape. As a result, the model optimization process becomes extremely difficult, and it is challenging to find the optimal solution. **ASGA's sharpness optimization and adaptive gradient alignment mechanism can effectively mitigate this issue**. Therefore, we plan to further investigate and validate the effectiveness of ASGA in other quantization approaches in our future research. We are confident that ASGA can also demonstrate excellent performance in other quantization methods.\n\n## Q4 In the paper, authors claim that the ASGA method can be applied to edge intelligence applications. A natural question is that whether the ASGA can be applied to other kinds of hardware platforms. For example, deploying this method on a Raspberry Pi?\n\nA4: Thanks for the comment! In edge intelligence applications, the primary challenges are to reduce computational overhead and enhance model generalization—objectives that align closely with ASGA’s design goals. While this paper focuses on ASGA’s application to conventional edge devices, we believe our method can be applicable to resource-constrained platforms such as Raspberry Pi. As pointed out in Section 3 of our paper, unlike the traditional model training process, ASGA only needs to conduct policy search on a small-scale proxy dataset and then finetune it on the large-scale target dataset. This approach can **significantly reduce a large amount of computational overhead for resource-constrained devices**. Thus, even for low-power devices like the Raspberry Pi, ASGA can be practically deployed by first performing offline quantization policy searches before on-device deployment, making it feasible for energy-efficient scenarios.", "claims_and_evidence": "In this work, all the statements are strongly backed by theoretical demonstrations or experimental findings, and all of them are without problems.", "methods_and_evaluation_criteria": "ASGA aims to solve the high search cost in MPQ. Using proxy datasets like CIFAR10 and benchmark datasets such as ImageNet and VOC helps evaluate the generalization and performance of quantization strategies for various tasks.", "theoretical_claims": "I have carefully reviewed the theoretical proof sections in the article and believe they are correct.", "experimental_designs_or_analyses": "The datasets and baseline methods used in the article are representative and can be used to evaluate the effectiveness of the method.", "relation_to_broader_scientific_literature": "The paper innovatively uses loss landscape sharpness for enhancing MPQ policy search, which is built on neural network sharpness-generalization research. Unlike existing MPQ methods with complex feature map calculations, it uses simple sharpness measures for a more efficient approach. This work is the first attempt to apply the sharpness of the loss landscape to the MPQ generalization.", "essential_references_not_discussed": "The related works have discussed the most relevant works about MPQ and Sharpness.", "other_strengths_and_weaknesses": "Strengths \n\n1. The paper has a clear structure and its idea is well illustrated, which is very easy to understand.\n\n2. The utilization of the sharpness of the loss landscape for MPQ policy search is novel. \n\n3. This design has sufficient theoretical proofs as support.\n\n4. The experimental design of the work is reasonable and highly persuasive, which can verify the effectiveness of the proposed method.\n\n5. The proposed method is computationally efficient since it does not introduce additional computational resource overhead.\n\n6. The design about how to exploit the loss-sharpness information is clear.\n\n7. The experimental results show the effectiveness of the proposed method in tackling various MPQ tasks.\n\nWeaknesses \n\n1. The authors should briefly explain why σmax can measure the sharpness of the loss landscape.\n\n2. Authors enhance model generalization by minimizing the loss sharpness. However, during actual training, the model may get stuck in locally flat regions, leading to convergence to suboptimal solutions. How can we ensure that the ASGA method, while pursuing a flat loss landscape, does not excessively get trapped in locally flat regions and miss the global optimum solution?\n\n3. Has the author considered extending the ASGA method to other fields? For example, applying the ASGA method to other types of quantization methods, such as binary quantization?\n\n4. In the paper, authors  claim that the ASGA method  can be applied to edge intelligence applications. A natural question is that whether  the ASGA can be applied to other kinds of hardware platforms. For example, deploying this method on a Raspberry Pi?", "other_comments_or_suggestions": "Please refer to the weaknesses presented in Section “Other Strengths and Weaknesses”.", "questions_for_authors": "Please refer to the weaknesses presented in Section “Other Strengths and Weaknesses”.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "UDOrm1SCXI", "reviewer": "Reviewer_WDfQ", "summary": "In the paper, authors propose a novel mixed-precision quantization method (ASGA) via learning the sharpness of  loss landscapes, which improves the quantization generalization across datasets, thereby reducing the search cost. Particularly, the idea of introducing sharpness measure into quantization is interesting, and shows an obvious merit: the quantization policy searched from the small dataset can be directly implemented in the large one. To realize this, authors propose an enhanced sharpness measure, which aligns gradient directions, and dynamically adjusts the perturbation radius during training. This way can well deal with conflicts between different optimization objectives, and accelerate the optimization convergence. Also, authors provide theoretical analysis demonstrating the effectiveness of ASGA in reducing the upper bound of quantization generalization error and ensuring convergence. Experimental results show that ASGA achieves significant speedup in policy search compared to state-of-the-art methods while keeping comparable accuracy on large datasets.", "questions": "", "limitations": "", "rating": 5, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "## Q1 The rationality of the theoretical assumptions in the article.\n\nA1: Many thanks for your insightful comments! As you mentioned, our theoretical analysis employs certain assumptions, including the PAC-Bayesian framework and convergence analysis under stochastic optimization. We adopt PAC-Bayesian theory primarily because it provides a rigorous theoretical foundation for model generalization capability. Building upon this framework, we theoretically demonstrate that reduced loss sharpness in quantized models leads to tighter upper bounds on generalization error, thereby enhancing the generalization of MPQ policies. Our experimental results validate this theoretical insight: when applying ASGA for MPQ search, we observe a **1.5% improvement in Top-1 accuracy on ImageNet compared to conventional methods**. This empirically confirms that PAC-Bayesian error bounds effectively measure MPQ generalization capability in practice. Furthermore, **Figure 6 in our paper** illustrates the sharpness reduction during MPQ search, demonstrating ASGA's effectiveness in minimizing $\\sigma_{max}$. These experimental findings align precisely with our theoretical predictions, reinforcing confidence in the validity of our analysis. Additionally, we analyze ASGA's convergence properties and derive its convergence upper bound under non-convex optimization conditions. As evidenced in **Figure 4**, our method demonstrates **stable convergence**, where the adaptive $\\rho$ parameter **enhances training stability** for MPQ.\n\n## Q2 Is the dataset used representative?\n\nA2: Many thanks. We selected multiple proxy and target datasets based on their widespread use and representativeness in image classification tasks. These datasets cover diverse image categories, styles, and difficulty levels, providing preliminary validation of ASGA's effectiveness and generalization capability. Experimental results demonstrate stable improvements in MPQ transfer performance across all scenarios. Although these rigorously selected datasets can, to a certain extent, already reflect the performance of ASGA quite well, it is undeniable that they still have certain limitations. For example, the data coverage in some specific fields may not be comprehensive enough, and so on, we believe ASGA's theoretical foundation and design approach possess universal applicability. To further evaluate ASGA's broad applicability, we plan to test it on larger-scale datasets, non-vision tasks (speech classification, autonomous driving detection), and extremely low-bit quantization tasks to ensure its effectiveness in more complex quantization environments.\n\n## Q3 The adaptability and scalability of the proposed method to models?\n\nA3: Thanks for your comment! ASGA **requires no substantial modifications when applied to different models**. The core idea of the ASGA method lies in smoothing and optimizing the loss landscape. Through this approach, it effectively reduces the undesirable characteristics of the loss landscape, such as sharp local minima, and thereby significantly enhances the model's generalization ability, enabling it to better handle different input data and task scenarios. Our experimental validation in **Table 1 in Section 3 of our paper** across diverse model architectures, including ResNet18 and others, has consistently demonstrated improved performance, confirming ASGA's applicability to various structural designs. While minor parameter adjustments might be necessary when applying ASGA to more complex or specialized model architectures, such potential modifications would not compromise the method's general applicability. The architectural independence of our approach stems from its operation at the fundamental level of loss landscape optimization rather than specific network topologies.\n\n## Q4 The scalability of the proposed method in other deep learning fields.\n\nA4: Thanks. As previously mentioned, ASGA demonstrates considerable versatility and can be applied to other deep learning domains without requiring substantial modifications. We plan to not only further investigate ASGA's performance in areas such as natural language processing, where its ability to handle sparse gradients and non-convex landscapes could offer distinct advantages, but also to deploy ASGA across diverse hardware scenarios, including both low-power devices and high-throughput systems, to explore its broader potential applications. These efforts will be complemented by rigorous benchmarking against state-of-the-art optimizers to obtain trade-offs in convergence speed, memory efficiency, and generalization.", "claims_and_evidence": "The paper conducts both theoretical analysis and extensive experiments on a variety of datasets and models. The results demonstrate that the proposed method can achieve high accuracy with less search cost, validating the claims.", "methods_and_evaluation_criteria": "The Adaptive Sharpness-Aware Gradient Aligning (ASGA) method is designed to tackle the challenge of heavy search overhead in Mixed-Precision Quantization (MPQ). It is reasonable to utilize proxy datasets such as CIFAR10, along with benchmark datasets like ImageNet and VOC, to assess the generalization capabilities and performance of quantization strategies across different tasks.", "theoretical_claims": "The proof of generalization performance in Lemma 1 is clear, which is based on PAC–Bayesian theory. The convergence analysis in Lemma 2 is correct, with clear steps.", "experimental_designs_or_analyses": "The use of multiple proxy and target datasets in the experiments is valid as it helps assess generalization.", "relation_to_broader_scientific_literature": "The paper's key contribution of using loss landscape sharpness for MPQ policy search builds on prior work on sharpness and generalization in neural networks. It differentiates from existing MPQ methods which rely on complex feature map calculations. By leveraging simple sharpness measures, it offers a more efficient approach, filling a gap in the literature where no prior work applied sharpness for MPQ generalization.", "essential_references_not_discussed": "The paper comprehensively covers relevant MPQ and sharpness-related literature.", "other_strengths_and_weaknesses": "Strengths \n1. The idea of the work is interesting. The use of loss-sharpness information in transferable quantization search (which can be trained with small proxy datasets) is well-motivated, showing great potential to deal with the efficiency issue suffered by current MPQ methods.\n2. The paper is well-organized and the presentation is generally clear.\n3. The thermotical analysis from Lemma 1 is important, showing the  impact of loss sharpness on the upper bound of quantization generalization error.\n4. The observation about the relationship between sharpness and MPQ transferability is important for understanding the motivation of the work.\n5. The strategies tailored for gradient direction alignment is clear and also effective.\n6. The experiments are convincing, which demonstrates that a small sharpness measure learned from proxy dataset helps seek a transferable MPQ policy for quantizing the model trained on large datasets. The experiment results are good both on the accuracy and efficiency.\n7. The experimental results for ResNet18 and ResNet50 on CIFAR10 are promising. The proposed method achieves considerable speedup compared  to SOTA methods.\n8. Also, the method has great potential for practical applications of neural network quantization.\n\n\nWeaknesses \n1. The paper employs certain assumptions in its theoretical analysis, such as those related to PAC-Bayesian theory and assumptions in the convergence analysis of stochastic optimization. These assumptions may not always hold true in practical application scenarios. Authors should explain how the theoretical conclusions drawn from these assumptions effectively guide the actual mixed-precision quantization (MPQ) tasks.\n2. In the experimental section, you selected CIFAR10, Flowers, and Food as proxy datasets, and used ImageNet and VOC as target datasets. Authors should ensure that this choice of datasets adequately represents a wide range of scenarios and confirm that the performance of the ASGA method still holds for other types of datasets. Additionally, authors should have plans to conduct experiments on a broader range of datasets for validation.\n3. The paper mentions that the ASGA method has been tested on different models, such as ResNet-18, ResNet-50, and MobileNet-V2. However, these models have significantly different structural characteristics. The authors should clarify the specific performance of the ASGA method in terms of its adaptability to model structures. Additionally, the authors should also consider whether the method needs to be adjusted accordingly for more complex or specialized model structures.\n4. This paper mainly focuses on image classification and object detection tasks in the theoretical analysis and experimental verification. It is recommended that the authors clarify the applicability of the ASGA method in other deep learning fields, such as natural language processing and speech recognition. The authors should also determine whether this method requires significant modification to be extended to these fields.", "other_comments_or_suggestions": "Please check Other Strengths And Weaknesses.", "questions_for_authors": "Please check Other Strengths And Weaknesses.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "ynQmlbjubp", "reviewer": "Reviewer_NNU4", "summary": "This paper aims to reduce the mixed-precision quantization search costs by decoupling the policy search and model deployment dataset. In this way, the mixed-precision quantization policy of a model can be searched on a small-scale dataset and then the policy can be transferred to a large-scale one for deployment. To this end, the authors introduce sharpness-aware minimization (SAM) to ensure generalization, as well as a gradient alignment method to handle gradient conflicts and an adaptive perturbation radius to accelerate optimization.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "## Q1 The novelty of the work.\n\nA1: Many thanks for your comment! This is a valuable question. The method SAQ [1] you mentioned appears similar to our work, but **they are actually different**. First, the goal of our work is **fundamentally different** from such work. SAQ quantizes and trains target model **on the same dataset**, aiming to enhance model's anti-interference ability by smoothing the loss function to prevent the occurrence of overfitting. In contrast, our ASGA aims to significantly boost the search efficiency on large dataset via searching for a generalizable MPQ policy on a small-scale proxy dataset, enabling it to achieve SOTA accuracy with fewer computational resources on a large-scale target dataset (**Table 1 in Section 3 of our paper**). The advantage of ASGA lies in the improvement of **cross-dataset generalization ability**. Second, SAQ focuses on **non-differentiable fixed-precision quantization**, particularly uniform quantization scenarios, whereas ASGA is designed for **mixed-precision quantization**. Comparative studies[2][3] demonstrate that MPQ **achieves better complexity-accuracy** balance in practical applications. Moreover, we emphasize that **ASGA is not merely an \"A+B\" combination**. Beyond separating search and deployment datasets, ASGA's primary contribution lies in optimizing the existing MPQ training paradigm, **offering novel solutions for cross-dataset generalization, convergence efficiency, and computational resource utilization**.\n\n---\n\n[1]: Sharpness-aware quantization for deep neural networks[J]. arXiv preprint arXiv:2111.12273, 2021.\n\n[2]: Haq: Hardware-aware automated quantization with mixed precision, CVPR 2019.\n\n[3]: Towards mixed-precision quantization of neural networks via constrained optimization, ICCV 2021.\n\n---\n\n## Q2 Literature supplementation and old baselines. \n\nA2: Thanks for the advice. We'll cite these papers in revision. Regarding outdated baselines, according to the available source code in the papers you provided, we add comparative experiments on ResNet18 (GH means GPU hours). The results show that with ASGA-searched MPQ, RFQuant obtains up to **1.11X searching efficiency improvement**, further validating our method's effectiveness.  Due to time limitation, the comparison experiments with more new baselines will be conducted in our subsequent research.\n\n|  Methods|   W/A   | con-epoch |  Top-1   | Top-5 | GH of ASGA | GH/epoch | Total GH  |\n| :-----------: | :---------: | :-------: | :------: | :------: | :--------: | :------: | :-------: |\n|  RFQuant[4]  |   4MP/4MP   |94 | 65.8 | 84.4 | - |1.30| 122.4 |\n| **R[4] + A** | **4MP/4MP** |**85** | **65.9** | **84.9** |  **2.1**   | **1.31** | **113.4** |\n\n---\n\n[4]: Retraining-free model quantization via one-shot weight-coupling learning, CVPR 2024.\n\n---\n\n## Q3 The differences between ASGA, SEAM and GMPQ.\n\nA3: Thanks for your comment! In fact, both ASGA and GMPQ have similar goal, i.e., find generalizable policy across datasets, and thus share similar quantization process like Fig.7. Regarding Figure. 2 and Eqs.4-5, both SEAM and ASGA build upon the differentiable MPQ framework, and thus share several differentiable search formulations. However, our method is **fundamentally distinct from SEAM and GMPQ**: SEAM seeks generalizable MPQ policy by minimizing intra-class compactness and maximizing inter-class separation, and GMPQ exploits attribution consistency and it relies on a pre-trained full-precision model for quantized model alignment, requiring extra training costs; Different from them, ASGA focuses on loss sharpness optimization with **no need pre-trained model**, which is more efficient.  For sake of clarity, we will modify the related figures and equations located by reviewers in the revision.\n\n## Q4 Performance improvement of ASGA compared with GMPQ and SEAM.\n\nA4: Many thanks. First, many studies[5][6] have theoretically proven that minimizing loss landscape sharpness can well enhance model’s generalization ability. Then, we conducted extensive experiments to validate the superiority of ASGA over SEAM and GMPQ. Experimental results in **Table 1** show  the quantized model derived from MPQ policy found by ASGA (for ResNet18) **achieves a Top-1 improvement of 0.7% and 0.1% on ImageNet over those of SEAM and GMPQ**, respectively, and **save training costs by 17% and 11.5%**, respectively. Moreover, the comparative experiments on a set of baselines in **Table 1** show that by using a small dataset of **only 0.5% the size of the large dataset** to search for quantization policies, ASGA **achieves the same accuracy on the large dataset and improves the search efficiency by up to 150%**. These validate **the superiority of ASGA in improving  generalization of quantization policies over both SEAM and GMPQ**.\n\n---\n\n[5]: Fisher sam: Information geometry and sharpness aware minimisation, ICML 2022.\n\n[6]: Surrogate Gap Minimization Improves Sharpness-Aware Training, ICLR 2022.\n\n---", "claims_and_evidence": "Yes.", "methods_and_evaluation_criteria": "I appreciate the authors' efforts, however, [1] already explored the feasibility of combining SAM with quantization. While I see the authors aim to organize this submission from another perspective (i.e., decoupling the search/deployment datasets), the perspective itself is still not new. Therefore, this submission looks like a bit A+B for me now. \n\nMoreover, I think at least some discussion and primary experiments should be provided in this submission, to further distinguish differences between the proposed method and [1]. \n\n\n[1] Sharpness-aware Quantization for Deep Neural Networks", "theoretical_claims": "Checked.", "experimental_designs_or_analyses": "I think the authors could provide more comparisons about recent published papers.", "relation_to_broader_scientific_literature": "This work extends the previous works (GMPQ, SEAM).", "essential_references_not_discussed": "While this submission provides basic discussion for mixed-precision quantization, it lacks of published papers in recent 3 years, e.g., [1][2][3][4]. Moreover, baselines are really old, e.g., EdMIPS (CVPR'20), GMPQ (ICCV'21), HAQ (CVPR'19). \n\n[1] Retraining-free Model Quantization via One-Shot Weight-Coupling Learning, CVPR 2024\n\n[2] One-shot Model for Mixed-Precision Quantization, CVPR 2023\n\n[3] SDQ: Stochastic Differentiable Quantization with Mixed Precision, ICML 2022 \n\n[4] Mixed-Precision Network Quantization via Learned Layer-wise Importance, ECCV 2022", "other_strengths_and_weaknesses": "I checked the two most related works, GMPQ and SEAM, and found the Fig.2, and Equ (4)(5) of this paper are quite similar to SEAM, and Fig.7 is quite similar to GMPQ.", "other_comments_or_suggestions": "N/A", "questions_for_authors": "Combining quantization with SAM seems to be promising but leads to an additional question, can they improve the generalization performance? In other words, does the searched mixed-precision quantization policy provide better generalization performance compared to existing works (SEAM, GMPQ)?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "kGb7rTjcNB", "reviewer": "Reviewer_WvMG", "summary": "This paper propose a search-based method to find mixed-precision quantization scheme that can work with a much smaller dataset. To enable the generalization from the small search set to the large validation set, the paper propose to seek for quantization scheme that can lead to flatter loss minima of the quantized model. This leads to the proposal of using an adaptive sharpness-aware gradient aligning objective in the search process.\n\n## update after rebuttal\n\nPost rebuttal, I agree with the author that the proposed method can help on calibrating the model with different datasets, no matter which policy search method is used. To this end, I increase my score to weak accept.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "## Q1 The claims of ASGA advantages and search cost.\n\nA1: Many thanks. We think the ambiguity of claim's presentation may cause your confusion of this work. Our experiments (**Table 1 in Section 3 of our paper**) show that **using ASGA with ResNet50 saves 16 GPU hours with better Top-1 compared to HAWQ alone, which validates our motivation**. We explain this as below. HAWQ's second-order information of the Hessian matrix is data sensitive[1]. It works well if the proxy subset distribution matches target domain; otherwise, performance degradation occurs since distribution shift may disrupt Hessian estimation. In fact, proxy dataset is often distribution-discrepant, especially for privacy-sensitive scenarios where directly getting subset is not allowed. Conversely, ASGA’s generalizable policy—searched on proxy dataset—can obtain satisfactory performance with large reduction of search time on target datasets (**Table 1**). We will improve related presentations (related work).  As for search cost, we have added comparison experiments with GPU hours evaluation (**see A2**).\n\n---\n\n[1]: Hawq: Hessian aware quantization of neural networks with mixed-precision, ICCV 2019.\n\n---\n\n## Q2 Is the evaluation criterion fair?\n\nA2: We have added experimental results of real training time as below (GH means GPU hours). However, we still believe con-epoch can be used for our scenarios. Rethink our two-step workflow: “X+A” means that ASGA first finds generalizable MPQ policy with small proxy data, and then baseline X finetunes this policy on target dataset with several epochs, where ASGA is not involved, and thus the search cost of each baseline at each epoch is **not increased significantly by ASGA**. Note that, the generalizable policy of model found by ASGA can be suitable to the baselines that require the same model structure, which **largely reduces search burden**. Without ASGA, baselines should train from scratch on large dataset, requiring much search time (**Table 1**).\n\n|Model| Methods | W/A | con-epoch |Top-1 | GH of ASGA | GH/epoch | Total GH  |\n| :------: | :-----: | :---------: | :-------: | :-----: | :--------: | :------: | :-------: |\n| ResNet50 |  HAWQ   |   4MP/4MP   |    98     |   74.6   |     -     |   3.04   |   298.2   |\n|          | **H+A** | **4MP/4MP** |  **92**   | **74.9** |  **2.4**   | **3.05** | **283.1** |\n| ResNet18 |  GMPQ   |   3MP/3MP   |    96     |   66.3   |   -      |   1.24   |  119.05   |\n|          | **G+A** | **3MP/3MP** |  **80**   | **66.4** |  **2.1**   | **1.24** | **101.5** |\n| ResNet18 |  SEAM   |   3MP/3MP   |    96   |   65.1   | - |   1.11   |  106.53   |\n|          | **S+A** | **3MP/3MP** |  **85**  | **65.8** | **2.1** | **1.11** | **96.48** |\n## Q3 Compare with HAWQ as baseline in other settings. \n\nA3: We need to clarify that \"H+A\" refers to using HAWQ to finetune the MPQ policy searched by ASGA, as presented in **A2**, while \"con-epoch\" denotes the number of iterations required for convergence. In this sense, both DNAS and HAWQ can be used as baseline to utilize the policy found by ASGA. Then, we have added experimental results under ResNet18 model, which show that by fine-tuning the policy of ASGA, HAWQ **saves 2.7 GH** (GH means GPU hours). Due to time limitation, the experiments on MobileNet-V2 will be provided in the future.\n\n|  Model  | Methods |   W/A   | con-epoch |  Top-1   | GH of ASGA | GH/epoch | Total GH  |\n| :-----: | :-----: | :---------: | :-------: | :------: | :--------: | :------: | :-------: |\n| Reset18 |  HAWQ   |   4MP/4MP   |  95   |   67.3   |     -      |   1.14   |   108.3   |\n|         | **H+A** | **4MP/4MP** |  **90**   | **67.4** |  **2.1**   | **1.15** | **105.6** |\n\n\n## Q4 The significance of ASGA.\n\nA4: Notice that both DNAS- and ILP-based MPQs can employ ASGA for reducing search cost without sacrificing performance, which demonstrate the significance of the proposed method. In fact, our ASGA is **suitable to the enhancement of ILP-based MPQs**, for instance, HAWQ-V3 can quickly finetune ASGA’s policies for stable performance across different datasets and hardware. Besides of DNAS-based methods, **ASGA offers a flexible and efficient solution for enhancing various MPQs**, especially for resource-limited, cross-dataset scenarios.\n\n## Q5 The scalability of ASGA on Transformer.\n\nA5: **ASGA is applicable to various models including Transformer**, since its core design—minimizing loss landscape sharpness — is not CNN-specific. In fact, [2] have demonstrated the scalability of SAM, which is applied to Transformers (BERT and ViT). Note that current MPQ researches (including SOTA works like EdMIPS and HAQ) primarily focus on CNNs, and thus our experiments also central on CNN models rather than Transformers. We will extend our work to Transformers in future research—a direction that is theoretically justified and critical for advancing practical MPQ deployments.\n\n---\n\n[2]: Towards efficient and scalable sharpness-aware minimization, CVPR 2022.\n\n---", "claims_and_evidence": "The claim that using sharpness information can help the search of a more generalizable quantization scheme is supported by both theortical analysis and experiment results. However, the claimed advantage of the proposed method is not justified. For example: \n1. The paper claims that searching MPQ policy with subset would reduce performance, yer this is only a drawback of differentiable search-based methods, not for other more advanced mixed precision quantization methods like HAWQ. This claim is inaccurate and hinders the motivation of the proposed method.\n2. In related work, the paper mentions that previous method suffer from intricate calculation of feature maps, increasing search costs. However, it seems apparent that the gradient computation required for the proposed ASAM is more costly than the feature map computation. The improved seearch cost claimed by the proposed method is only supported by the reduced numbre of epochs, yet ignoring the potential higher cost in each epoch.", "methods_and_evaluation_criteria": "The evaluation is conducted cross multiple models and datasets. The setting is standard for quantization literature. However, the paper uses \"con-epochs\" as a key criteria in the experiments, which is not a fair criteria. The proposed method relies on SAM, which requires at least two back propagations per epoch, effectively increasing the convergence time over baseline methods even with a reduced epoch counts. Real training time should be reported here as a better criteria.", "theoretical_claims": "I checked the method derivation from Equ.(1) to (12). However, I did not check the theortical analysis in Sec. 2.4.", "experimental_designs_or_analyses": "As reported in Table 1, HAWQ appears to be a significantly stronger baseline than other DNAS-based methods, yet HAWQ is not used as baseline in other settings. I believe HAWQ shoudl be compared with in all settings to prove the contribution of the proposed method is effective. Moreover, HAWQ performs a single-shot ILP-based search followed by finetuning. It is unclear how the H+A is performed in Table 1 and what con-epoch stands for here.", "relation_to_broader_scientific_literature": "This paper aims to improve over previous DNAS-based mixed precision quantization method by including SAM-based objective in the search process. As both DNAS-based methods and SAM objectives are well studied, this specific usage appears to be novel. However, it shoudl be noted that DNAS-based methods are not promising compred with ILP-based methods like HAWQ in inducing mixed-preciison quantization scheme. This fundamentally hinders the significance of the proposed method.", "essential_references_not_discussed": "References are adequate.", "other_strengths_and_weaknesses": "Besides the weakness mentioned prevously on the motivation, baseline, and evaluation metrics, SAM-based optimization has a fundamental drawback of complexity, which makes the convergence challenging on more complicated architectures like transformers. The exploreation of the proposed method is limited to simple CNN models in this paper. The scalability of the search and the SAM objective to transformer models is questionable.", "other_comments_or_suggestions": "Post rebuttal, I agree with the author that the proposed method can help on calibrating the model with different datasets, no matter which policy search method is used. To this end, I increase my score.", "questions_for_authors": "Please address the weaknesses mentioned in previous parts.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Meta-review: This paper tackles the high computational cost of Mixed-Precision Quantization (MPQ) policy search for compressing DNNs, which traditionally requires searching on the large target dataset. The authors propose searching on a small proxy dataset instead, but this usually suffers from poor generalization due to data distribution shifts. To fix that, they leverage the link between flatter loss landscapes and better generalization. Specifically, they introduce a method that optimizes the MPQ policy search on the proxy dataset to favor solutions leading to flatter loss minima for the quantized model. This enhances policy transferability without needing complex feature analysis like prior methods. Experiments confirm finding effective and transferable MPQ policies using very small proxy datasets (e.g., 0.5% of target size), significantly accelerating the search process (113-150% speedups over SOTA) while maintaining good performance on the final model.

The reviews favor acceptance due to a successful author rebuttal addressing initial concerns. Reviewers generally recognized the value and novelty of the proposed method to drastically reduce MPQ policy search costs. Highlighted strengths include the intriguing use of loss sharpness ideas for MPQ transferability, the method's significant potential for improving search efficiency, its theoretical underpinnings, and strong experimental results demonstrating speedups and competitive accuracy. However, initial reviews also raised several important questions regarding the distinctiveness from prior sharpness-aware work (like SAQ), the completeness and fairness of experimental comparisons (especially against baselines like HAWQ and the use of convergence epochs versus wall-clock time), the practical scope across different datasets and models, and clarity on some technical aspects.

The authors provided a detailed rebuttal that effectively tackled these critical points. They successfully differentiated ASGA from related work by clarifying its specific goals within the cross-dataset MPQ context. The authors also provided additional experimental evidence, including comparisons using GPU hours which validated the claimed efficiency gains, and demonstrated how ASGA could complement strong baselines like HAWQ, particularly when dealing with dataset distribution shifts. They also justified their evaluation metrics within their proposed two-stage workflow and provided requested technical clarifications regarding the sharpness measure and optimization details. This comprehensive response alleviated the main reservations by the reviewers, leading to positive final recommendations, with one reviewer explicitly increasing their score citing the clarified benefits of ASGA for model calibration across datasets.

In agreement with the reviewers, I find that this submission successfully demonstrates the benefits of leveraging loss sharpness in the novel application domain of Mixed-Precision Quantization (MPQ) to propose a new algorithm, and supports that by strong and promising empirical results. I recommend acceptance.
Decision: Accept (poster)