Merge-Friendly Post-Training Quantization for Multi-Target Domain Adaptation
Authors: Juncheol Shin
Minsang Seok
Seonggon Kim
Eunhyeok Park
Venue: ICML 2025 poster (2025)
Keywords: Post-Training Quatnization
Multi-target Domain Adaptation
Model Merging
Primary area: deep_learning->algorithms
TL;DR: We analyze the impact of quantization on model merging through the lens of error barriers and highlight key considerations for quantization in the context of multi-target domain adaptation via model merging.

Abstract: Model merging has emerged as a powerful technique for combining task-specific weights, achieving superior performance in multi-target domain adaptation. However, when applied to practical scenarios, such as quantized models, new challenges arise. In practical scenarios, quantization is often applied to target-specific data, but this process restricts the domain of interest and introduces discretization effects, making model merging highly non-trivial. In this study, we analyze the impact of quantization on model merging through the lens of error barriers. Leveraging these insights, we propose a novel post-training quantization, HDRQ - Hessian and distant regularizing quantization - that is designed to consider model merging for multi-target domain adaptation. Our approach ensures that the quantization process incurs minimal deviation from the source pre-trained model while flattening the loss surface to facilitate smooth model merging. To our knowledge, this is the first study on this challenge, and extensive experiments confirm its effectiveness.

Large-scale models have driven breakthroughs across vari- domains, particularly in generative AI, enabling efficient adaptation multiple tasks and user-specific data. However, deploying such models on conuree-onntrrinedd devices re- mains a significant challenge due to their computational de- mands. this perspective, model merging has emerged as promising technique that enables multi-target domain adap- tation without additional training. A recent study on multi- Merge-Friendly Post-Training Quantization for Multi-Target Domain Adaptation
Juncheol Shin
Minsang Seok
Seonggon Kim
Eunhyeok Park 2.2
Abstract
Ouantization
Model merging has emerged as powerful tech- nique for combining task-specific weights, achiev- ing superior performance in nulti-target domain adaptation. However, when applied to practical scenarios, such quantized models, new chal- lenges arise. In practical scenarios, quantization is often applied to target-specific data, but this process restricts the domain of interest and intro- duces discretization effects, making model merg- ing highly non-trivial. In this study, we analyze the impact of quantization on model merging through the lens of error barriers. Leveraging these insights, propose novel post-training quantization, HDRQ Hessian and distant regu- larizing quantization that is designed to consider model merging for multi-target domain adapta- tion. Our approach ensures that the quantization process incurs minimal deviation from the source pre- pre-trained model while flattening the loss surface to facilitate smooth model merging. To knowl- edge, this is the first study on this challenge, and extensive experiments confirm its effectiveness.
Thrget Domain
Marging
Target Domain
Nalve Quantization
HDRO
Domain
Model Merging
Target Domain
HORQ
Figure I. We propose a quantization scheme designed with future merging in mind. Our method ensures that networks are quantized to more merge-friendly state, reducing the degradation induced by merging.
target domain adaptation (Li et al., 2024) demonstrated that models fine-tuned for different target domains can be fused into single general model via simple weight averaging, even in training-frre manner. This discovery highlights the potential of real-time adaptive AI.
1. Introduction
Despite its promise, a major obstacle in achieving practical multi-target domain adaptation the effect of quantization. Quantization essential for reducing memory and compu- tational costs for efficiency, but it introduces discretization that is not well aligned with the merging idea, leading to sub- optimal merging and degraded performance. While previous works have extensively explored quantization and domain adaptation separately, little attention has been given to their interplay. In particular, no existing research systematically investigates how quantization influences model merging or proposes solutions to mitigate its impact.
"Graduate School of Artificial Intelligence, Pohang Univer- sity of Science and Technology (POSTECH), Pohang. Repub- lic of Korea *Department of Computer Science and Engineer- ing, Pohang University of Science and Technology (POSTECI), Pohang, Republic Korea. Correspondence to: Eunhyeok Park <ch, park @ postech.ac.kr>.
To address this challenge. we introduce HDRQ (Hessian and Distance Regularizing Quantization), the first post-training quantization (PTQ) method designed to preserve merging compatibility in multi-target domain adaptation. Our key contributions are as follows:
Proceedings of the 42nd International Conference Machine Learning, Vancouver, Canada. PMLR 267, 2025. Copyright 2025 by the author(s).

---

Merge-Friendly Post-Training Quantization for Multi-Target Domain Adaptation
Theorefical Analysis of Quantization's 's Impact on Model Merging: We extend the concept of the error barrier (Frankle et al., 2020) to analyze how weight perturbations from quantization affect merging quality. Our study reveals that quantivation-induced misalign- ment reduces merging effectiveness across different adaptation scenarios.
practical for resource-constraineed environments. Research
quantization can be broadly categorized into two ap- proaches: Quantization-Aware Training (QAT) and Post- Training Quantization (PTQ).
The first category, Quantization-Aware Training (QAT) (Esser et al., 2020; Baskin et al., 2021; Défossez et al., 2022; Shin et al., 2023), fine-tunes full full-precision mod- els using fake quantization operators and straight- estimators (STE) (Bengio et al., 2013) to approximate gradients. These methods leverage the full dataset to compensate for errors induced by quantization. A notable subfield within QAT is noise-based quantization (Baskin et al., 2021; Défossez et al., 2022; Shin et al., 2023), which models quantization noise as an additive perturbation to weights, eliminating the need for STE and improving stability.
Regularization for Merge-Friendly Quantization: Based on our analysis, HDRQ incorporates two key regularization techniques ensure that quantized mod- els remain merge- ceregpcompa
Hessian Regularization:
By controlling sensitivity perturbations, mitigate the adverse effects of quantization on merging stability.
Distance Regularization: By reducing weight di- vergence among quantized models, enhance their ability to merge effectively.
The second category, Post- Training Quantization (PTQ), enables quantization without full retraining, making it more efficient but often at the cost of performance degradation. Various techniques have been developed to enhance PTQ: (Nagel al., 2020) proposed a theoretically justified layer- wise reconstruction method to optimize rounding policies, while (Li et al., 2021) extended this idea to block-wise reconstruction, addressing cross-layer dependencies. (Wei et al., 2022) further introduced selective activation quan- tization and dropout strategies to enhance robustness. A more recent advancement, Bit-shrinking (Lin et al., 2023), leveraged basee quantization with sharpness-awe aware scheduling to minimize degradation. Our proposed method, HDRQ, falls within this category, introducing compact noise-based quantization strategy specifically designed to maintain merging compatibility across models.
Noise-Sampling-Bann Based Rounding: We introduce an advanced rounding mechanism that resolves the round- ing ambiguity problem in conventional quantization, ensuring more stable weight updates.
We evaluate HDRQ across multiple datasets and compare it against conventional PTQ methods. Our key findings are:
Comparable or Superior Single-Model Perfor- mance: HDRQ maintains accuracy individual quan- tized models, performing on par with or better than existing PTQ methods.
Significantly Improved Merging Performance: Un- like standard PTQ, which degrades merging quality, HDRQ ensures that merged models achieve higher ac- curacy and better generalization. For example, HDRQ improves the performance of merged model by 4.21 mloU compared to conventional PTQ in the multi- target domain adaptation task for semantic segmenta- tion.
2.2. Domain Adaptation
Domain adaptation enables models to generalize to tar- get domain by leveraging knowledge from a related source domain. One of the most widely studied areas is Unsu- pervised Domain Adaptation (UDA) (Long et al., 2016; Zou et al., 2018), which aligns domain distributions using labeled source data and unlabeled target data. However, con- ventional UDA techniques become impractical when source data is unavailable due to privacy constraints. To address this, Source-Free Domain Adaptation (SFDA) (Liang et al., 2020: Liu et al., 2021; Hou & Zheng, 2021) has emerged, where adaptation relies solely on a pre-trained model and target domain data, eliminating the need for di- rect source data access.
Robust Multi- Target Domain Adaptation: HDRQ consistently improves merging outcomes across differ- ent adaptation settings, confirming its effectiveness in real-world scenarios.
By addressing the impact of quantization through theoretical analysis and targeted regularization, HDRQ ensures that quantized models remain merge- compatbll,,,, paving the way for real-time adaptive AI on resource-constraing devices.
2. Related Work
Most domain adaptation approaches focus on Single-Tare Domain Adaptation (STDA), where model adapts to one specific target domain. However, real-world applications often require adaptation to multiple distinct target domains. Multi- MarieT Domain Adaptation (MTDA) (Yu et al.,
2.1. Quantization
Quantization is a crucial technique for reducing model size and computational cost, making deep learning models more
2

---

Merge-Friendly Post-Training Quantization for Multi-Target Domain Adaptation
2018; Gholami al., 2020; Ngayen-Meidine et al., 2021; Li et al., 2024) addresses this by training a model capable of handling diverse target domains. Many MTDA methods em- ploy multiple student models "guyen--Meidng et al., 2021), which significantly increases computational overhead.
the two points. It serves as an indicator of convexity (Frankle et al., 2020; Ainsworth et al., 2023). Specificallyy, a zero error barrier implies linear mode connectivity, indicating that the loss remains flat or exhibits positive curvature along the path. In other words, it provides insight into merging feasibility. A low error barrier indicates a smooth loss landscape, while high barrier suggests weight misalignment.
recent alternative, training-free MTDA via model merg- ing (Li al., 2024), leverages the observation that models fine-tuned from the same initialization often reside within similar optimization basin. This enables weight merging via simple averaging, provided that normalization statis- ties are properly handled. While effective, this approach has largely overlooked the impact of quantization, which disrupts weight alignment and hinders merging quality.
Since the error induced by quantization can be approxi- mated as the addition of uniform noise to the original val- ues (Baskin et al., 2021; Défossez et al., 2022; Shin et al., 2023), we can derive the error barrier for the quantized weights as follows:
max [C(0) + Ex) / (c(0)+ + C(O, + +2)),, (3) xE(O.1]
Our work bridges this gap by proposing HDRQ, quan- tization method explicitly designed preserve merging compatibility in mulli-targee domain adaptation. By address- ing quantivationinndueed misalignment, HDRQ unlocks new possibilities for real-time, adaptive AI on resource- constrained devices.
where c1; 62 represent quantization noise sampled from uniform distribution 61 - 2 and 62
U UI 2 ] respectively, with quantization step sizes s1 and S2.
Applying second-order Taylor expansion, we obtain:
3. Analysis
max [[(02)- (c(0)))+ [(02]]+ 1E
Both quantization and model merging have been extensively explored, yet rigorous understanding of how quantization noise affects model merging remains unexplored. To address this gap, we provide a theoretical analysis of quantization- induced misalignment and its impact on the merging pro- cess. Inspired by prior works (Ainsworth et al., 2023; Xu et al., 2024; Stoica et al., 2024) on model merging, we extend the concept of the error barrier to explicitly incor- porate quantization effects. While previous studies have examined merging under full-precision settings, we ana- lyze how muamization-innuuee landscape and merging quality. This analysis reveals key factors that degrade merging performance and motivates our proposed guantization regularization techniques.
max [ex VoL(O>) - 1 LT, XE
v/L(0,) -22
2 -²
2 (c1 Vol(O)) + 7
v2L(0)) .1t
1 c2 VoC(O,) + / Vi,((2) c2)].
(4)
This yields the sum of the original error barrier and the maximum of terms involving the first- and second-order derivatives at the two points and their merged point. Given that both 0n and 02 have converged to the same loss L, all terms involving the first-order derivatives can be ignored. Furthermore, assuming a zero loss barrier for the original points for simplicity, we obtain:
perturbations affect the loss
3.1. General error barrier case
max [cx VoL(O>) 2 VZL(O>) -a- AE[O,1]
An ideal merging process should ensure that interpolated models maintain low error without introducing sharp in- creases in loss. This motivates us to analyze the merging process through the lens of the error barrier, which quanti- fies the degree of interpolationiinduee performance degra- dation. Given two converged weight points 01 and fg. we define the interpolated model as:
1
V2C(()) & V3C(0)) . 6.)l
(5)
To minimize this term, we consider two approaches. The first approach maximizes the sum of the second-order terms. Since both weights are at local minima, their Hessians are positive semi-definite, ensuring that these terms remain non- negative regardless of E and eg. However, maximizing this term is undesirable, an increased Hessian implies reduced robustness. This approach deliberately increases the loss of the quantized models to reduce the maximum difference between their mean and the interpolated point.
0x = (1 x)01 + A82.
E [0, 1].
(1)
The error barrier (Frankle al., 2020) is then given by: 1
max ((xx) XE 1
2 + ((2)))
The error barrier quantifies the maximum increase in error relative to the average loss along the linear path connecting
An alternative, but promising approach is to minimize the term related to the merged point 0x. Assuming that the
3

---

Merge-Friendly Post-Training Quantization for Multi-Target Domain Adaptation
Hessian of the loss C is M-Lipschitz continuous between I and 0z. we can bound the Hessian the merged point using the original points as follows:
L and L2, respectively. For simplicity, we analyze the case from the perspective of one domain with loss L1, though the same reasoning applies symmetrically to the other domain.
V)C((s)- V2C(0,)
+ ()
M M - 011l (6)
When the Hessian is M-Lipschitz continuous, as assumed in the general error barrier case, the gradient also becomes Lipschitz continuous with some constant L. Since 0, and 8g lie within the same basin, their linear interpolation 0 also resides within this basin. Consequently, the Jacobian term VoL1(82) in Equation (7) becomes scaled version of Val2. proportional to the distance between points. There- fore, Equation (7) can be reformulated as:
2
2
This result indicates that the Hessian at the merged point can be effectively regularized by controlling the Hessians at the original points and minimizing the distance between them. Since the Hessian is MM M--Lpschitzz continuous, the gradient of the loss also becomes Lipschitz continuous with some constant L. Given that the Hessian is closely related to the rate of change of the gradient, regularizing both the Hessians and the distance between the points implicitly regularizes the first-order terms at the two points and the merged point.
1 (c1( + 21(00))
max (1x) - 1
[EE + k E2) VoLi + 1 = (Ox)(0x) EX-
max 1
3.2. Regularization for Merge- Friendly Quantization
1
Our theoretical analysis identifies two key contributors to the emror barrier: (1) increased sensitivity quantization noise due to sharp loss landscapes, and (2) excessive divergence between quantized weights that disrupt interpolation. Based on these insights, we introduce two regularization:
4 (cf (2L)(01) .4.08)).ttt + (02) x2),, (7)
where k is a scalar proportional to the distance between A and 02. Although it may not be feasible for each weight to di- rectly account for the Hessians of both losses, regularizing it within single domain bound of the error at the merged points. This insight ensures that the model remains more robust to quantization-induced errors and facilitates smoother merging across domains.
still indirectly regulate the upper
Hessian Regularization: To reduce sensitivity to per- turbations, we minimize the second-order term in (7) by encouraging smooth Hessian spectra during quan- tization. This prevents excessive local curvature that amplifies quantization noise effects.
4. HDRQ: Hessian and Distance Regularizing Quantization
Distance Regularization: To control weight diver- gence, we minimize |10, - -02ll during quantization, ensuring better alignment between the merged models.
Building upon our previous analysis, we propose novel quantization method called Hessian and Distance Regular- izing Quantization (HDRQ), Our method incorporates two key strategies to enable merge-friendly quantized networks. First, employs noise-based quantization to regularize the Hessian, reducing the sensitivity of the loss landscape to weight perturbations. Second, it introduces weight distance regularization, encouraging quantized models to converge that are inherently more compatible for merging.
By integrating these regularization techniques, we signifi- cantly mitigate the impact of quantization on merging per- formance. Figure 2 illustrates the effectiveness of HDRQ in flattening the loss landscape, leading to improved merging.
3.3. Domain Adaptation case
In domain adaptation, the losses not necessarily equal, i.., L(0:) & L(f2), the models are optimized with re- spect to different domain-specific objectives. This discrep- ancy shifts the lower bound of the error barrier from 0 to 2 (2()) - ((2))) Despite this shift, minimizing the error barrier remains essential for effective model merging.
Additionally, to address the rounding ambiguity problem that occurs during the merging of quantized models, we pro- pose an advanced noise sampling-based rounding technique. This approach effectively mitigates ambiguity in rounding policies, ensuring stable weight merging and enhancing the overall quality of the adapted models.
The key implication of this change is that one of the first- order terms the original points in Equation (7) does not vanish. However, by leveraging the fact that domain adapta- tion from the same source weight results in weights located within a single basin (Li et al., 2024), the remaining first- order term can be absorbed into that of the merged point. Let us assume 02 and 9g are obtained through domain adapta- tion from the same source weight og. optimized with losses
4.1. Noise-based hessian regularization
To regularize the Hessian of the network, HDRQ simulates quantization by introducing additive sampled quantization noise. For each optimization step involving weight w and quantization step size A, we first compute the quantized weight v under uniform quantization as follows:
4

---

Frrge-Friendly Post-Training Quantization for Multi- Target Domain Adaptation
5
-0.4
-0.4
-0.-
0.2
0.2
0.2
0.0
0.2
V2
0.0
0.2
V2
0.0
0.2
V2
v1
v1
-0.2
0.00.4
-0.2
0.40.4
(a) BRECQ
(b) Qrrop
(c) HDRQ(Ours)
Figure 2. Visualization of loss surfaces quantized with each method is shown. ResNet-50 adapted from Real domain to Clipart domain (R
+ C) is quantized to W4A8. HDRQ effectively regularize hessian with noise-based quantization, leading weights to flatter surface.
Harmonic mean of accuracy
73.8
à clamp(L 11 1,- 12001120-1 -1). A
(8)
73.6
73.4
where b denotes the bit width. We then sample the quanti- zation noise from the quantization eгror w and train using instead of the deterministic quantized value w. Since the quantization noise follows uniform distribution UI- A 1 the modified loss function inherently regular- izes the Hessian as follows:
73.2
73.0
Advanced
Naive
Figure 3. The distribution of harmonic mean accuracy for merging W4A8 quantized C-> and C-+ A models on Office-Home dataset is presented. Our simple yet effective cosine similarity- based method, denoted as Advanced, successfully filters out low- quality weights, stabilizing merging outcomes.
E[C(()) Ne E[[(w+ [ e)]
= E[[(w) + v. + +/²²² r. (ww)
1 ~ F(((w) + Nz (ew
(9)
4.2. Weight distance regularization
Measuring the distance between separately adapted weights for each domain is challenging, as prior information about the target domain is assumed. Instead of directly min- imizing this distance, HDRQ ensures that the distances be- tween domain -adated weights remain small by regularizing their distances from the source weight. The upper bound of the distance between the domain-adapted weights, Wtar1 and Wtar2, can be derived using the triangular inequal- ity:
where the first-order term of the Taylor expansion vanishes because E(c] 0. As a result, the expected loss function penalizes sharp curvature in the loss landscape, leading to implicit Hessian regularization.
While nosse-based quantization is not novel concept its effectiveness has been established in prior works (Baskin al., 2021; Défossez al., 2022; Shin et al., 2023; Lin et al., 20r approach is the first to integrate this tech- nique within a model merging framework, motivated by theoretical analysis. This integration results smoother loss surfaces and improved compatibility between quantized models during merging.
|Wtar Wtar2 < |aree Wtarl + | - Wtare]- (10)
Since both the source and domain aadaie- weights reside within the same basin, enforcing this regularization during quantization is unlikely to significantly degrade weight qual- ity. To implement this, we introduce regularization term based the /2-norm between the source and target weights.
The effect of noise-based Hessian regularization is demon- strated in Figure 2. When this regularization technique is applied, the network converges to a smoother loss surface compared to existing methods. This smoother convergence not only enhances the robustness of the quantized model but also facilitates better merging compatibility by reducing sharp loss barriers between models.
It is important to note that assuming access to the original weights reasonable. In most deployment scenarios, mod- els are initially pretrained on source data and subsequently adapted on user devices using user-specific data.
5

---

Merge-Friendly Post-Training Quantization for Multi-Target Domain Adaptation
Table 1. Quantization and multi-target domain adaptation results semantic segmentation task
#ddwaa
Soffe Kigh
Method
BirW/A)
Dosain
8
miou
FP
52/32
95.71
72.37
58.52
27.85
22.65
49.62
59.37
69.97
19.47
44.69
63
77.37
49.
91.29
54,30
56.
58.22
36.91
58.25
61.
G
91.25
4,41
50.91
28.86
11.03
34,710
21.50
44.15
50.83
29.00 .60
95.71
72.36
63.62
80.25
58.56
60.16
0.00
78.63
26.13
$2.06
BRESO)
-+
96.85
97.15
71.73
5,40
58.44
80.55
25.
38.57
23.77
13.90
35.44
57.46
16.95
43.55
47
50.51
39.00
43.65
95.51
30
72.36
63.35
47.82
80.09
.0
58.52
54.53
59.2
55.48
0.00
65
77.97
33.76
57.17
25.92
$1.80
60.86
QDoo
616
G
95.54
91.13
71.43
4.61
19.19
28.33
39.01
23.82
11.00
48.87
34.05
58.38
17.87
44.21
70.06
89.52
80.66
39, 69
44.65
89.73
A2
76. 79
71.99
63.37
48.04
91.19
79:95
53.13
$7.66
35.94
78.18
35.89
23.82
57.62
61.32
51.65
Ours
615
G
97.20
95.64
71,83
7.34
68.
80
60.60
29.45 46
38.80
23.11
13.30
49.3
14. 68
$7.98
13,55
43.53
96
80.
90.75
44.22
19.97
89.57
95.65
76 76
71.34
62.69
47.32
70 66
11.30
55, 12
58.95
56.
59.61
35 60
0.00
78.00
39.25
23.41
58.61
61.54
$1.55
BREEQ)
90.36
94,90
41.90
2,01
71.59
27.91
12.16
24.78
19.49
42.92
28.61
43.57
2.53
57.77
35.70
77.04
57.96
34.97
40.64
55.05
91.49
72.89
60.66
58.42
40.65
75.41
7M
55.01
47.64
48.
56.81
0.00
38.63
68.97
52.27
9.05
45.50
$3.67
QDoop
4/4
G
G-+
94,94
96.08
69.01
2.48
87.85
79.38
24.99
36.29
24.55
12.31
46.99
32.32
99
53.66
10.40
42.19
68:27
58.86
39,97
43.07
95
88.94
74.53
61.52
43.87
78.48
89:95
56.80
19.29
54.87
$4,44
27.51
0.000
75.98
32.76
14.90
56.04
49.44
58.92
Ours
4/4
G
06.74
95.23
69.78
3,52
79:03
57.63
26.60
35.56
20.52
11.54
46.93
53.04
51.65
7.22
67.90
40.18
79.71
89.04
38.76
42.25
95.41
89.15 12
73.63
66.29
59.74
38.86
90.12
78.23
47.50
56.89
52.09
55.09
0.00
88
34,97
75.67
56.46
12.29
58.25
48.68
(a) Quantization results on cach target domain
Method
Bix(WA)
Moric
riod
bolldnng
Phlee
Godl::
Niyol
mloU
91
49.98
86.83
36.35
38.83
46.82
54.05
55.10
89.31
42.77
90.18
75.37
41.86
99
50.51
58.0 04
22,46
35
48.86
12
FP
32/32
97.21
33.64
80.12
35.85
24.95
32.95
24.76
41.86
79.45
34.70
94.54
73.88
00:97
78.01
51.37
57.71
0.00
76.96
37.65
$3.50
94.36
85.34
36.05
84.09
38.33
74.62
49.64
57.87
4%,
42.55
C
87.51
37.85
84.99
23.90
31,28
42.44
48.39
49.52
87.78
36.48
87.56
73.02
39.78
78.03
45.
53.45
16.66
29
44.35
52.52
BRECQ
6/6
96.90
32.24
78.21
29.63
21.66
31.33
24
38.85
78.86
35. 62.62
93.76
71.12
58.71
74.51
45.0
53.68
0.00
75.00
33,85
51.12
34.55
81.45
26
36.01
83.07
35. 7M
72.05
53.52
41. 11.74
38.37
12
85.05
36,04
31,94
49.95
52.07
88.
60
88.60
73.
38.
76.99
47.67
55.27
19.81
25.
43.40
53.23
QDrop
6/6
1
96.72
32.46
78.58
30.58
23.90
31.57
26. 10
41.38
78.99
33.11
94.16
72.00 72.06
59. 50
74.41
46.3 93
54.44
0.00
75. 15.09
34.64
51.82
1
91.83
35:77
81.09
25.04
21.25
35.95
33.93
46.10
K3:34
34.84
91.29
72
46.81
15.50
47:25
54.84
0.00
41.14
38.51
52.51
C
80 18
47.97
85.19
70 36
34.05
50.32
54.24
88.57
37
88.75
73.66
37.97
82.34
55.45
19.20
50.00
44.58
54,98
Ours
6/6
1
97.48
33.35
76.82
32.82
23.84
31.56
25.15
42.09
75.83
36.55
94.39
72.42
59.65
76.45
45. 35
54.81
0.00
76.06
36.18
52.58
93.25
39,25
30.95
28.02
36.36
N3.41
37.88
73.01
46.34
19.24
43.45
0.00
42
39.94
53.70
60:65
5.48
67.05
4.87
8.51
13.56
23.32
17,46
70.92
13.59
71.41
43.65 43.65
8.59
59.05
17 17.26
61
8:14
16.01
17.93
29:21
BRECQ
4/4
1
81.71
18.07
$6.39
12.74
13.03
18.91
8.46
23.55
70.04
24.72
74,94
45.00
38, 98.98
54.66
27.55
25. 96
0.00
53.86
16.80
35.34
70.06
8.09
60.55
6.90
9.90
15.58
12.
19.37
70.40
16.70
72.51
45.5 53
13.74
56,43
20
26.76
0.00
24.25
16.9K
31.95
90.75
10.68
77.57
12.62
19.34
25.25
29.52
37.63
80.47
21.73
79.26
61. 61.99
21.18
74.64
$4.81
41.43
6,45
18:77
25.25
39.91
@Diep
414
91.54
17.59
67.78
22. 80
18.99
23.79
11. 66
36.53
75.005
07
86.68
62. 78
018
68.91
54.67
41.01
0.00
65 66
22.87
43.37
[[
65,70
12:45
72.13
16.08
18.85
24.41
16:50
37.00
77.64
23.54
82.07
62.08
29.25
71.32
34.57
41.05
0.00
28.81
23.89
41.54
83.36
18.63
16.81
35.01
43.11
28.75
81.18
25.
77.21
42.72
20.91
30.04
Ous
4/4
94.78
25.67
75:27
85
19.49
26.28
17.94
38.15
78:35
30.46
91.85
67.13
53.48
20.81
59.
92
43.66
0.00
069 09:90
26.25
47.17
H
85.67
21.08
76.49
20.78
22.50
30.82
23.60
30
40.45
81.51
29.33
86.12
67.29
34.65
73.69
39.30
30
44.00
0.00
51.95
27.95
45.75
(b) Multi- target domain adaptation via model merging results. and I represents CityScapes and Indian Driving Dataset, respectively. H
denotes the harmonic mean of performance on C and I. HDRQ demonstrates comparable superior performance across all settings.
4.3. Handling ambiguity in rounding policy
approach again degenerates into an ambiguous case, as the A terms effectively cancel out. This issue is particularly pronounced in domain adaptation scenarios, where weights are fine-tuned from shared source weights with a small learning rate, making their step sizes likely to be similar.
While HDRQ optimizes the network a more merge- friendly state, naive merging may still lead to quality degra- dation due to ambiguity in the rounding policy.
Consider two quantized values being merged with integer representations I and Iz, and step sizes A and Ag, respec- tively. Using the midpoint merging technique described in (Li et al., 2024), if the sum of 1 and 12 is an odd number, the merged integer value falls between two adjacent integers, leading to ambiguity in the rounding direction. A potential solution is to merge in the floating-poin point domain as follows:
To handle the ambiguity problem in rounding policies, HDRQ employs noise sampling during the merging pro- cess. Instead of merging weights directly in the integer or Hoating-poiit domain, we sample noise ~ UI - + and add it to the weights prior to merging. This maintains the same quantized representation as the original values while mitigating ambiguity:
I1 A + 12 A2
Imeegeed =
(11)
A + 2₂
(11 Ai + E1) + (12 A + E2)
Imerged
(12)
However, when the step sizes A1 and Ag are similar, this
A + A2
6

---

Merge-Friendly Post-Training Quantization for Multi-Target Domain Adaptation
Table Multi-target domain adaptation results the Office-Home dataset. The harmonic mean of accuracy across three target domains for the merged model is reported as the main metric, with individual domain performance of each quantized models provided in parentheses. The best accuracy for each configuration is highlighted in bold, with red text indicating a gap of more than 1% compared to the second-best result. Blue text highlights cases where previous methods failed to converge to flat loss surface. Note that sampling is not applied in this setting, as there is no ambiguity in merging across the three-task scenario.
Domain
Methods
W8A8
W8A4
W4A8
W444
W3A3
67.74
64.79
64.15
60.95
43.66
BRECQ
(73.55 (155559.1888338 18883.85
(73.09/55.62/82..))
(73.38 19.11)83.87)
(72.89 56.56 (2.70)
(67 (67.00:47.724 (7.0447774775 73,449
R A.C.P
67.78 (73.59) (9.22)83.877
QDiop
67.91
67.4
64.85
66.26
62,99
(73 51 (59 (99/20883883)
(73.34
(73.71 59, 19/3992)
(73.47 58.99/83.99
(73.14 14.68.69 83.62)
67.46
66.71
66.74
66.41
64.70
HDRQ
(73. (4/59.06)883885
(73.34/59. 18/83,71)
(73.42 59:20/83.89)
(73.42 (9.04)83.87)
(72.81 (8.77,883.333
68.75
65.4
66.06
62.53
48.04
BRECQ
(81.71 (81.71|56.01/78.00
(80.06,54.91
(81.73 55.99/77.99)
(80.35 17.11)
(71.43 48.84488747 71.32)
A RRR.C
68.80
QDrop
68.75
68.24
66.83
66.04
64.22
(81.78 55. 78.00)
(81 (81.71/55.97//88044
(81.75 (5.855588888881 15)
(81. 66 55. (8.88))884)
(81. 55.78 (88. 18.13)
(81 41 (5.655 (7.88)
68.
68.15 18
67.50
67.58
65.29
HDRQ
(81.7 (81.73/55.92/77.99
(81.75 (75555.78))) (777977)
(81 68 65.92))7804)
(81.89 (5.88/77.88)
(80.97,55 (5.755540070 78.04)
74.75
73.51
73.22
71.31
56.
BREQQ
(79 (99.077668.89//99.999
(78.70/68.8) (1.007
(79.11 69.06/79.39)
(78.49 68.8 (78.91)
(72.69 64.69 70.40)
C-+ R.A.P
75.07
QDrap
74.79
74.58
73.81
73.25
71.01
(79.11688.9779..6
(79 (9 69/7999))445)
(79.23 (6. 85/79994)
(79.11 68.89/79.38
(79.23 69 (9.399
(79.07 07 68.46664400 79.25)
74.58
73.70
74.26
73.38
71.63
HDRQ
(o..1466.141/79.38
(79.21 68.6669999144
(79.23 68.77 79..399
(79.11 68.60) 79.18)
(78.77 68.07 (8896)
65.14
63.27
64.09
61.92
45.09
BRECQ
(81.96688.8 19(55,90)
(80.88 (4.97))4777)
(82.03 68. 15(56066
(81.25 67.82)
(72.11 62.75 (1,90)
P-RRAC
65:25
QDrap
64.8
64,52
62,52
63.22
61:24
(82.03 55.95)
(81. (81.98/68 (81.98)68.2((55.97)
(82.05 67.866
(82. 68. 19)55.88))
(81. 56.008
(81 55.51)
64.51
64.09
63.93
61.55
HDRQ
(81.91/68.11 (5/6.8)
(81.96 15)56.01)
(82.01 68. 15)55.83)
(82.10 68. 55,99)
(81.23 67.45 55.33)
Since noise sampling introduces randomness, it is crucial to reject noise samples that could degrade the quality of the merged network. To achieve this, we evaluate the cosine sim- ilarity between the vector connecting the merged sampled weights to the target domain weights and the original inter- polation vector connecting the target domain weights. The sample with the highest similarity is selected, ensuring high- quality merging. As shown in Figure 3, this straightforward yet effective approach successfully filters out low-quality weight samples and stabilizes the merging quality.
primary evaluation metric. Unless otherwise specified, ad- vanced noise-based sampling is used during the merging step after quantization. We report the mean results from 30 sampled weights for semantic segmentation.
Semantic Segmentation For semantic segmentation, we use the GTA synthetic dataset (Richter et al., 2016) as the source domain and two real-world datasets, Cityscapes (Cordts et al., 2016) and Indian Driving datasets (Varma et al., 2019), as the target domains. We adopt HRDA (Hoyer et al., 2022) as the single-target domain adaptation method, us- ing ResNet-101 (He et al., 2016) as the backbone model and simple convolution head. All other settings are kept consistent with the original paper.
5. Experimental Results
To validate the effectiveness of our proposed HDRQ method, we conduct experiments on multi-target domain adaptation tasks for semantic segmentation and image classification. Following our initial objective, we adopt the merging- multi-target domain adaptation approach (Li et al., 2024), where models are first adapted independently to each target domain and then merged for unified target adaptation. Quantization is applied between the single-targer domain adaptation and the merging process. The harmonic mean of performances across target domains is reported as the
Image Classification For image classification, we use the Office-Homee dataset (Venkateswara et al., 2017), which con- sists of four domains (Real, Art, Clipart, and Product). We select one of the four domains as the source domain and treat the other three as target domains. ResNet-50 (He et al., 2016) is used as the backbone architecture, and SHOT (Liang et al., 2020) is adopted as the single-target domain adaptation method. Adaptation settings follow the original work.
7

---

Merge-Friendly Post-Training Quantization for Multi-Target Domain Adaptation
Quantization Details Quantization is applied after folding batch normalization layers. Thus, batch normalization lay- ers do not need to be considered and only mid-point weight averaging is used for the merging process. The hyperparam- eter X for weight distance regularization is set to 5e-2 for all tasks. HDRQ conducts 20,000 iterations of block-wise re- construction, including activation quantization with partial dropout. We use the Adam optimizer with an initial learning rate of 0.001 and a cosine annealing with warmup. For the last 3,500 iterations of reconstruction, we switch from noise- based quantization simulation to actual fake quantization, which stabilizes training and improves the final results by decreasing the gap between simulated and actual quanti- zation. important to note that most reconstruction is conducted with noise-based quantization, and the learning rate becomes very small the final stage, ensuring this switch does not undermine our objectives.
Table 3. Incremental ablation study cach component of HDRQ
Method
Accuracy
62.99
Baseline
(73.14/58.69|85. (8366)
64.21
Noise-based quantization
(..42)58665883667)
64.70
Distance regularization
(72.81 (58.7//88)))/83333
block-wise wise reconstruction and partial dropping may regular- ize overall hessian of loss landscape, may not effective to handle lumps around actual converged points. In contrast, HDRQ effectively guides the network smoother loss sur- faces by direct injection of noise to weights, as visualized in Figure 2. This ability to find flatter minima is particularly evident in the P - R, A, C task, where QDrop struggles to merge the models successfully. HDRQ, however, demon- strates robust and consistent performance, achieving either comparable or superior results across all evaluated settings.
5.1. Semantic Segmentation
The results for multi-target domain adaptation on semantic segmentation are presented in Section 4.1. HDRQ demon- strates comparable performance to QDrop for the quanti- zation task itself, while the merged quantized models sig- nificantly outperform other methods after multi-target do- main adaptation. The benefits of HDRQ become more pro- nounced as the bit precision decreases, where the impact of quantization on model merging becomes more severe. While HDRQ surpasses QDrop by a modest 1.19 mloU in the W6A6 setting, it achieves much more substantial improvement of 4.21 mIoU in the W4A4 setting.
5.3. Ablation Study
To evaluate the effectiveness of each component in HDRQ. we conducted an incremental ablation study on the Office- Home dataset under the W3A3 precision, A.C.P setting, The results are presented in Table 3.
When neither based quantization nor weight dis- tance regularization was applied, our method degenerated to QDrop, which only employs block-wise reconstruction and partial dropping during quantization. Introducing the noise-based quantization scheme yielded a significant per- formance gain of 1.22% highlighting the importance of direct regularization around the converged solution surface.
These results highlight the importance of robust quantization strategies for successful multi-target domain adaptation via model merging and validate the effectiveness of the Hessian and distance regularizations proposed by HDRQ.
Further incorporating weight distance regularization brought an additional 0.49% accuracy gain, showing the effective- ness of keeping target domain weights close to each other for improved merging stability. These results underscore the combined value of noise-based quantization and distance regularization in HDRQ for enhancing model merging.
5.2. Office Home
The experimental results on the Office-Home dataset (Venkateswara al., 2017) are summarized in Table 2. We report the harmonic mean of accuracies across target domains, with the best results highlighted in bold, and red-colored results indicating accuracy gains of over > 1% compared to the second-best result.
6. Conclusion
For the relatively easy-to casy-to-quantize Office-Home task, all methods achieve comparable performance when weight pre- cision is high. However, in specific settings such R - A.C.P, when weights are quantized into lower bit- width pre- vious methods exhibit significant performance degradation compared to HDRQ.
In this paper, we analyze the impact of quantization on model merging, particularly its effect on the error barrier. Building on this, we propose HDRQ. post-training quanti- zation scheme tailored for multi-target domain adaptation. HDRQ leverages noise-based quantization to regularize the Hessian and applies weight distance regularization to facili- tate better merging. It also mitigates the rounding ambiguity inherent in naive merging methods through noise sampling. Experimental results demonstrate that HDRQ consistently outperforms previous approaches, validating its superiority.
We conjecture that this discrepancy arises because previous methods occasionally fail to converge to flat minima, which are critical for successful merging after quantization. Since
8

---

Merge-Friendly, Post-Training Quantization for Multi-Target Domain Adaptation
Acknowledgements
Gholami, B., Sahu, P., Rudovic, O., Bousmalis, K., and Pavlovic, V. Unsupervised multi-target domain adapta- tion: An information theoretic approach. IEEE Trans- actions on Image Processing. 29:3993 4002, 2020. doi: 10. 00.109/71P220112296333
This work was supported by IITP and NRF grant funded by the Korea government(MSTT) (No. RS-2019-11111906 RS- 2024-00396013, RS-2023-00228970, RS-2024-00457882),
He, K., Zhang, X., Ren, S., and Sun, J. Deep residual learn- ing for image recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2016.
Impact Statement
This paper addresses the novel problem of combining quan- tization with multi multi-target domain adaptation and proposes an effective solution. Our work holds potential societal con- tributions by advancing the practical deployment of machine learning models with reduced computational and memory footprints.
Hou, Y. and Zheng, L. Source free domain adaptation with image translation, 2021.
Hoyer, I.., Dai, D., and Van Gool, I.. Hrda: Context-aware high-resolution domain-adaptive semantic segmentation. In European conference on computer vision, pp. 372-391. Springer, 2022.
References
Ainsworth, S., Hayase, J., and Srinivasa, S. Git re-basin: Merging models modulo permutation symmetries. In The Eleventh International Conference on Learning Repre- sentations, 2023.
Li, W. Gao, H.-a., Gao, M., Tian, B., Zhi, R., and Zhao, H. Training free model merging for multi-target domain adaptation. In European Conference Computer Vision Springer, 2024.
Baskin, C., Zheltonozhki,, E., Rozen, T., Liss, N., Chai, Y., Schwartz, E., Giryes, R., Bronstein, A. M., and Mendel- son, A. Nice: Noise injection and clamping estimation for neural network quantization. Mathematics, 9(17):2144, 2021.
Li, Y., Gong, R., Tan, X.,, Yang. Y., Hu, P., Zhang, Q., Yu, F. Wang, W., and Gu, S. (BRECQ): Pushing the limit of post-training quantization by block reconstruction. In International Conference on Learning Representations, 2021.
Bengio, Y., Léonard, N., and Courville, A. C. Estimating or propagating gradients through stochastic neurons for conditional computation. CoRR, abs/1308.3443 2013. URL /ttptii.///axiv...// www.xxivv.oorraaooo abs///b////0 1308.10880 3432.
Liang, J., Hu, D., and Feng, Do we really need to access the source data? Source hypothesis transfer for unsuper- vised domain adaptation. In III, H. D. and Singh, A. (eds.), Proceedings of the 37th International Conference on Machine Learning, volume 119 of Proceedings of Ma- chine Learning Research, pp. 6028-6039. PMLR, 13-18 Jul 2020.
Cordts, M., Omran, M., Ramos, S., Rehfeld, T., Enzweiler, M., Benenson, R., Franke, U., Roth, S., and Schiele, B. The cityscapes dataset for semantic urban scene under- standing. In Proc. of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2016.
Lin, C., Peng, B., Li, Tan, Ren, Y., Xiao, J., and Pu, S. Bit-shrinking: Limiting instantaneous sharpness for im- proving post-training quantization. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition (CVPR), pp. 16196-16205, June 2023.
Défossez, A., Adi, Y., and Synnaeve, G. Differentiable model compression via pseudo quantization noise. Trans- actions on Machine Learning Research, 2022. ISSN 2835-8856.
Liu, Y., Zhang, W., and Wang, J. Source-free domain adap- tation for semantic segmentation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1215-1224, June 2021.
Esser, S. K., McKinstry, J. I.., Bablani, D., Appuswamy, R., and Modha, D. S. Learned step size quantization. In International Conference on Learning Representations, 2020.
Long, M., Zhu, H., Wang, J., and Jordan, M. Unsupervised domain adaptation with residual transfer networks. In Lee, D., Sugiyama, M., Luxburg, U., Guyon, I., and Garnett, R. (eds.), Advances in Neural Information Processing Systems, volume 29, Curran Associates, Inc., 2016.
Frankle, J., Dziugaite, G. K., Roy, D., and Carbin, M. Linear mode connectivity and the lottery ticket hypothesis. In III. H. D. and Singh, A. (eds.), Proceedings of the 37th International Conference on Machine Learning, volume 119 of Proceedings of Machine Learning Research, pp. 9259-3269 PMLR, 13-18 Jul 2020.
Nagel, M., Amjad, R. A., Van Baalen, M., Louizos, C., and Blankevoort, T. Up or down? adaptive rounding for post- training quantization. In International Conference on Machine Learning, pp. 7197-7206. PMIR, 2020.

---

Merge-Friendly Post-Training Quantization for Multi-Target Domain Adaptation
Ngayen-Meidine, L. T., Belal, A., Kiran, M., Dolz, J., Blais- Morin, L.-A., and Granger, E. Unsupervised multi-target domain adaptation through knowledge distillation. In Proceedings of the IEEE/CVF Winter Conference on Ap- plications of Computer Vision (WACV), pp. 1339-1347, January 2021.
class-balanced self-training. In Proceedings of the Euro- pean Conference Computer Vision (ECCV), September 2018.
Richter, S. R., Vineet, V., Roth, S., and Koltun, Playing for data: Ground truth from computer games. In Leibe, Matas, J., Sebe, N., and Welling, M. (eds.), European Conference on Computer Vision (ECCV), volume 9906 of LNCS. pp. 118. Springer International Publishing, 2016.
Shin, J., So, J., Park, S., Kang. S., Yoo, S.,, and Park, E. Nipq: In
Noise proxy-based integrated sseudo-quantizann Proceedings of the IEFF/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 3852-3861, June 2023.
Stoica, G., Bolya, D., Bjorner, B., Ramesh, P., Hearn, T., and Hoffman, J. Zipit! merging models from different tasks without training. In The Twelfth In- ternational Conference Learning Representations, 2024. URL id Ed-LEYUkvdUhq.
Varma, G., Subramanian, A., Namboodiri, A., Chandraker, M., and Jawahar, C. IDD: A Dataset for Exploring
Problems of Autonomous Navigation in Unconstrained Environments In 2019 IEEE Winter Conference on Applications of Computer Vision (WACV), pp. 1743-1751, Los Alamitos, CA, USA, January 2019. IEEE Computer Society. doi: 10.1 1099//AAV.2299.00099...
Venkateswara, H., Eusebio, J., Chakraborty, S., and Pan- chanathan. S. Deep hashing network for unsupervised domain adaptation. In Proceedings of the IEEE Confer- ence on Computer Vision and Puttern Recognition, pp. 5018-5027, 2017.
Wei, X., Gong, R., Li, Y., Liu, X., and Yu, F. Qrrop: Ran- domly dropping quantization for extremely low-bit post- training guantization. In International Conference Learning Representations, 2022.
Xu, Z., Yuan, K., Wang. H., Wang, Y., Song, M., and Song. J. Training- free pretrained model merging. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 5915-5925, June 2024.
Yu, H., Hu, M., and Chen, S. Multi-target unsupervised do- main adaptation without exactly shared categories, 2018.
Zou, Y., Yu, Z., Kumar, B. V., and Wang. Unsuper- vised domain adaptation for semantic segmentation via
10


Reviews:
[{"review_id": "LxHpd1WFY1", "reviewer": "Reviewer_UftN", "summary": "This paper investigates the impact of quantization on model merging in multi-target domain adaptation. The key insight is that prior approaches, which quantize the model before merging, degrade merging quality. To address this, the paper introduces HDRQ, a merge-friendly quantization method that incorporates two regularization terms: one to control sensitivity to perturbations and another to minimize weight divergence between models. The authors evaluate the proposed method on two tasks: semantic segmentation and image classification. Experimental results demonstrate that the proposed method HDRQ enhances model merging performance across various adaptation settings.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We sincerely thank you for thoughtful feedback and constructive comments.\n\n## Weakness 1 : Main motivation\nTo begin with, we would like to emphasize that our work deliberately addresses the most challenging scenario for enabling domain adaptation on resource-constrained edge devices. Specifically, we design our setting to be highly demanding in the following four aspects: \n1.  The edge device lacks sufficient resources to support fine-tuning operations. \n2.  State-of-the-art PTQ algorithms are also too computationally expensive for such devices. \n3.  To minimize communication costs, only quantized models can be deployed. \n4. Model merging operations must remain lightweight.\n\nFrom this perspective, as some reviewers pointed out, an alternative approach—merging two unquantized models first and then applying PTQ—could be a viable solution. However, this approach becomes less practical when progressive adaptation is considered.\nIn the case of progressive adaptation, additional merging must be performed on top of the previously merged weights. In our proposed scenario, the target model is expected to accumulate updated weights locally over multiple rounds. Since the server discards the personalized model after each fine-tuning round, this approach imposes a strong constraint. Nonetheless, it provides significant security advantages, as personalized models are not stored centrally. By contrast, in the alternative approach, the server must maintain a separate customized model for each device, which leads to increased storage overhead and raises potential concerns regarding information leakage. Please note that our proposed scheme works really well on our challenging scenario, showing the outstanding performance of the proposed idea. \n\n\n\n## Weakness 2 : Results vary with changes in the hyperparameter λ\nHere, we present the results of model merging with varying values of $\\lambda$. As shown in the table below, increasing $\\lambda$ initially improves performance due to the regularization effect. However, beyond a certain point, excessive regularization starts to harm the quality of reconstruction during the quantization process, leading to accuracy degradation. Based on empirical observations, we selected $\\lambda = 5 \\times 10^{-2}$ as a balanced choice that consistently performs well across different settings. We adopt this value uniformly in all experiments throughout our paper.\n\n| λ   | 5e-2  | 7e-2  | 9e-2  | 1.1e-1 | 1.5e-1 |\n|---------|-------|-------|-------|--------|--------|\n| Harmonic Mean of Accuracy (%) | 66.75 | 66.78 | 66.84 | 66.98  | 66.86  |\n\n\n## Weakness 3 : Discrepancy in highlighted number \nSorry for the confusion, and thank you for pointing that out. As you correctly noted—particularly in Table 1(a)—the performance of QDrop should be more clearly emphasized. Our main intention was to highlight the quality improvement achieved after model merging, as demonstrated in Table 1(b). To avoid misunderstanding, we will revise Table 1(a) to clearly indicate that QDrop is the best-performing method for standalone quantization.\n\n\n## Other comments : Comparison with AWQ, SmoothQuant, SpinQuant, or GPTQ\nIn our experiments, we apply per-tensor, round-to-nearest quantization with truncation. Notably, the truncation range is learnable, following the formulation used in QDrop.\n\nAWQ, SmoothQuant, SpinQuant, and GPTQ are designed specifically for LLMs, and thus differ fundamentally from gradient-based methods like QDrop. These algorithms perform progressive, layer-wise quantization without iterative updates to reduce the computational burden—an essential property given the vast number of parameters in LLMs. However, in the case of vision models, QDrop often outperforms these LLM-oriented techniques. For this reason, our primary focus is on vision-oriented PTQ approaches.\n\nThat said, we agree it is also valuable to assess the effectiveness of our proposed method under more advanced PTQ schemes, even if they were originally developed for LLMs. To this end, we conducted 4W4A quantization experiments on semantic segmentation using SmoothQuant, and compared our method against relevant baselines:\n\n|  | SmoothQuant | Qdrop  | HDRQ  |\n|:------:|:----------:|:------:|:-----:|\n| **6W6A** |  44.69  |  52.62 | **53.95** |\n| **4W4A** |  5.43   |  42.5  | **46.21** |\n\nAs shown in the table, ours performs significantly better than SmoothQuant, especially in the low-bit W4A4 case. We will update the results in the camera-ready version.", "claims_and_evidence": "Yes. It looks so. The insight that quantization induces misalignment and affects the model merging process makes sense to me.", "methods_and_evaluation_criteria": "The proposed method appears reasonable; however, since the quantization approach is limited to vanilla round-to-nearest, it remains unclear whether more advanced quantization methods could further improve its effectiveness.", "theoretical_claims": "This theoretical claim in Section 3.1 makes sense.", "experimental_designs_or_analyses": "The experimental design is well-structured and supports the main claim of this work.", "relation_to_broader_scientific_literature": "- This work analyzes the interplay between quantization and model merging, offering a valuable contribution to the model merging literature.", "essential_references_not_discussed": "- The quantization method relies solely on vanilla round-to-nearest. A discussion on more advanced methods, such as AWQ, would be beneficial.", "other_strengths_and_weaknesses": "- The main motivation is unclear. If the goal is to obtain a quantized merged model for efficient deployment, why not merge two unquantized models first and then apply post-training quantization? Could the authors elaborate on the practical applications to provide a clearer motivation?\n\n- In Table 3, the improvements from distance regularization are not significant. How do the results vary with changes in the hyperparameter λ?\n\n- In Table 1 (W4A4), the best results come from QDrop, yet the highlighted number corresponds to the proposed method. Could the authors clarify this discrepancy?", "other_comments_or_suggestions": "- Some implementation details of quantization are unclear, such as the quantization granularity and whether the method is based on RTE. Additionally, how would more advanced approaches like AWQ, SmoothQuant, SpinQuant, or GPTQ compare to the proposed method?", "questions_for_authors": "N/A", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "zos8MMlZsI", "reviewer": "Reviewer_PqA3", "summary": "This paper introduces HDRQ (Hessian and Distance Regularizing Quantization), a novel post-training quantization (PTQ) method designed to improve merge-friendly quantization for multi-target domain adaptation. Model merging has been shown to be an effective way to adapt models across multiple target domains, but quantization introduces discretization effects that degrade merging performance. This work systematically analyzes the impact of quantization on model merging through the lens of error barriers, showing that quantization-induced perturbations disrupt weight alignment, leading to suboptimal merging.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your valuable feedback. We've covered your key points and clarified our work.\n## Weakness 1 : Comparison with QAT\nQuantization-aware training (QAT) typically yields higher quantization quality compared to post-training quantization (PTQ), but it demands full access to training data and significantly more computational resources. In our setting, which centers on per-device domain adaptation, the available target data is often too limited to support end-to-end fine-tuning without risking overfitting. To ensure practical applicability, we focus on PTQ and compare our approach against baseline PTQ methods.\n## Weakness 2 : latency/runtime comparisons\nIn our setup, fine-tuning and PTQ are done on a server, while only model merging and inference run on the edge device. Since the quantized models retain the same bit configuration and granularity, their latency and runtime remain unchanged after merging. \n\nAs a result, the only remaining computational component on the edge device is the model merging process. To clarify its cost, we measured the time on a Raspberry Pi 5 using MobileNet-V2. The process took 35.53 seconds with 100 noise sampling iterations on an ARM Cortex-A76 CPU—about 3.60 seconds per 10 iterations—showing that merging is feasible even on low-power devices.\n## Weakness 3 : Highly heterogenous domains\nDomain adaptation typically uses a lower learning rate and fewer epochs than training from scratch, so we assume that the resulting weight deviations are not large enough to escape a single basin—even under significant domain shifts.\n\nHowever, we acknowledge that if such a case occurs, additional treatments may be required to maintain the quality of quantized networks.\n\nWe conducted a partial experiment on DomainNet (Real, Clipart, Sketch, Painting), a more complex dataset, and observed consistent improvements over baselines. Please refer to our response to Reviewer 5KmE’s Weakness 1.\n## Question 1 : Sharpness measure\nAs a measure of sharpness, we report the Hessian trace of the Real→Clipart domain-adapted model quantized to W4A8 on the Office-Home benchmark. The table below shows that HDRQ exhibits a lower Hessian trace, indicating better flatness:\n| |QDrop|HDRQ|\n|:-:|:-:|:-:|\n| **Hessian Trace** |42171|38348|\n## Question 2 : Multiple runs\nWe conducted five additional experiments with different seeds and report the mean and standard deviation:\n| |QDrop|HDRQ|\n|:-:|:-:|:-:|\n| **4W4A results (Harmonic mean)**|41.97 (±1.27)|46.14 (±0.74)|\n\nWe would like to sincerely apologize for an error identified during the revision process. Specifically, we discovered that in our earlier experiments, the quantization scale in our method was not properly updated, which led to the reporting of degraded accuracy. In the updated results presented in this table, we have corrected this implementation bug. As a result, the overall accuracy has improved significantly. While we deeply regret this oversight, we also hope for your understanding that this issue introduced a negative bias in evaluating the performance of HDRQ.\n## Question 3 : Other architectures\nSince our analysis is agnostic to model architecture, we expect it to generalize well to vision-oriented models. However, architectures like transformers—which often exhibit unique characteristics such as outlier channels—may require additional techniques to preserve quantization quality. That said, existing baseline methods are also likely to experience similar degradation in such cases. We plan to include results on ViTs and other architectures in the final version.\n## Question 4 : Runtime, memory comparison\nHere is a table comparing the maximum memory consumption and quantization time of baseline methods and ours:\n| |BRECQ|QDROP|HDRQ|\n|:-:|:-:|:-:|:-:|\n| **Peak Memory Consumption (MB)** |9394|14643|14586|\n| **Quantization Time (hours)** |2.45|2.12|2.5|\n\nAs shown, quantization time slightly increases over QDrop due to noise sampling at each iteration. Theoretically, memory usage may also rise slightly, as sampled noise must be stored for each gradient step. However, actual measurements via torch.cuda.max memory allocated show similar memory consumption due to PyTorch's caching mechanism. From the result, this allows us to assume that the memory overhead of HDRQ is minimal.\n\nIn summary, these increases in memory consumption and overall time are minimal and represent an affordable cost.\n\nThe inference speed of models quantized with each method remains the same, as the quantized models have the same bit configuration and quantization granularity.\n## Question 5 : Ablation of noise-sampling-based rounding technique\nThe ablation study is provided in Figure 3 of the main paper, where \"Naive\" denotes merging without advanced sampling. For the merging of ambiguous cases with multiple valid quantization levels, the Naive method assigns one at random. Advanced sampling filters out low-quality samples, improving average performance and reducing distribution tail size.", "claims_and_evidence": "The paper presents a well-structured argument for HDRQ, and its claims are largely supported by theoretical analysis and empirical results. The paper provides a theoretical analysis using the error barrier concept, showing that quantization noise misaligns weights and increases merging degradation. Experiments on both semantic segmentation and classification tasks demonstrate that HDRQ achieves comparable or superior accuracy to standard PTQ methods in single-model settings while significantly improving merged model accuracy.", "methods_and_evaluation_criteria": "The proposed methods and evaluation criteria generally make sense for the problem of post-training quantization (PTQ) in multi-target domain adaptation (MTDA) via model merging.  The use of semantic segmentation datasets (GTA → Cityscapes, Indian Driving Dataset) is appropriate since segmentation is a resource-intensive task where quantization is highly relevant. The Office-Home dataset (Real, Art, Clipart, Product) is a well-established benchmark in domain adaptation, making it a reasonable choice for testing multi-target domain adaptation in image classification. The paper correctly evaluates the harmonic mean of accuracy across target domains, which is a standard metric for multi-target domain adaptation to ensure balanced performance across different domains.", "theoretical_claims": "Yes, I checked the proof and found no issues.", "experimental_designs_or_analyses": "I examined the experimental design and analysis in the paper, particularly focusing on the validity of the experimental setup, evaluation metrics, and statistical rigor. The use of GTA → Cityscapes, Indian Driving Dataset for segmentation and Office-Home dataset for classification aligns well with multi-target domain adaptation (MTDA). Quantization is applied after domain adaptation but before merging, which mirrors a realistic deployment scenario where models are adapted to different target domains and then quantized for efficiency. The paper incrementally removes components (noise-based quantization, distance regularization) and shows their impact on performance.", "relation_to_broader_scientific_literature": "The key contributions of this paper, HDRQ (Hessian and Distance Regularizing Quantization), are closely related to several existing areas in the broader scientific literature, including model merging, post-training quantization (PTQ), multi-target domain adaptation (MTDA), and loss landscape analysis.", "essential_references_not_discussed": "No", "other_strengths_and_weaknesses": "Strengths: 1. The paper introduces HDRQ, a merge-friendly post-training quantization (PTQ) method, which is a novel contribution. While quantization and model merging have been studied separately, this work is the first to systematically analyze how quantization affects merging and propose a quantization method explicitly designed for multi-target domain adaptation (MTDA) via model merging. 2. The use of error barriers to analyze quantization effects on merging adds a theoretical foundation to a practical problem, which is an underexplored area in PTQ. 3. It enhances the practicality of training-free multi-target domain adaptation, making real-time adaptive AI more feasible without retraining.\n\nWeakness: 1. The paper claims HDRQ is the first to study quantization-aware model merging, but it does not compare against quantization-aware fine-tuning methods (e.g., QAT approaches that may mitigate merging issues). 2. Since edge deployment efficiency is a key motivation, including latency/runtime comparisons would improve the practical significance. 3. in highly heterogeneous domains, where domain shifts are large, does HDRQ still work? A failure case analysis would help clarify the method’s limitations.", "other_comments_or_suggestions": "No", "questions_for_authors": "1. In Section 3.2, you state that HDRQ flattens the loss surface using Hessian regularization, and Figure 2 visually supports this claim. However, you do not provide a quantitative metric for Hessian smoothness. Could you report the Hessian eigenvalue distribution or a sharpness measure (e.g., Spectral Norm of Hessian, Fisher Information Matrix, or Sharpness-Aware Minimization (SAM) curvature metrics) to validate this claim?\n\n2. The results in Tables 1 and 2 report performance improvements, particularly in low-bit settings (e.g., W4A4, where HDRQ achieves +1.69 mIoU over QDrop). However, standard deviations and statistical significance tests are not reported. Could you provide standard deviations over multiple runs and conduct a paired significance test (e.g., t-test or Wilcoxon signed-rank test) to ensure that the improvements are statistically meaningful?\n\n3. HDRQ is tested on ResNet-based architectures (ResNet-50, ResNet-101), but modern resource-efficient models (e.g., Vision Transformers, MobileNets, EfficientNet) are widely used in real-world low-power applications. Have you tested HDRQ on non-ResNet architectures? If not, do you anticipate challenges in applying HDRQ to architectures with different weight distributions (e.g., ViTs)?\n\n4. HDRQ introduces Hessian-based smoothing, distance regularization, and noise-sampling-based rounding. How does this impact quantization time, inference speed, and memory overhead compared to standard PTQ methods (e.g., QDrop, BRECQ)? Can you provide runtime comparisons (in milliseconds per sample) and memory usage measurements?\n\n5. HDRQ includes a noise-sampling-based rounding technique to improve merging stability, but no direct ablation is provided for this component. Could you report results comparing HDRQ with and without noise-based rounding to show its direct impact on merging performance?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "6EMMqJneeA", "reviewer": "Reviewer_5KmE", "summary": "This paper proposes a merge-friendly post-training quantization method for multi-target domain adaptation. It considers quantization and domain adaptation simultaneously. They propose HDRQ (Hessian and Distance Regularization Quantization), a post-training quantization method to preserve merging compatibility in multi-target domain adaptation. Hessian Regularization controls sensitivity to perturbations, while distance regularization reduces weight divergence among models. A noise-sampling-based rounding mechanism is proposed to alleviate rounding ambiguity. Experiment results prove the effectiveness of the proposed method.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your insightful review and constructive feedback on our submission.\n\n## Weakness 1 : Complicated data and tasks\n\nWe agree that experiments on more complicated tasks are truly helpful. We appreciate your suggestion and will consider additional experimental results.\n\nDue to the limited resource and time, we are not able to produce full experimental results for domainnet, but we here provide partial merging results of 3W3A quantized DA models from Real → Clipart, Sketch, Painting:\n\n|                                  | BRECQ | QDROP | HDRQ  |\n|:--------------------------------:|:-----:|:-----:|:-----:|\n| **Harmonic Mean (R → C, S, P)** |  34.07 |  41.51 | **42.08** |\n\nIn this moment, we hypothesize HDRQ to work well with larger domain shift, since domain adapted weights are expected to still remain in single basin, where our theoretical analysis are valid.  We expect our method to either surpass baseline methods or show comparable performance in other settings, as observed in the Office-Home dataset. We will include the full experimental results in the camera-ready version. Thank you for your invaluable suggestion.", "claims_and_evidence": "I think the claims in this paper are supported by convincing evidence.", "methods_and_evaluation_criteria": "I think the proposed method is reasonable. The evaluation criteria are also reasonable, but I think the authors can include more complicated datasets or tasks, eg. for image classification, I am curious about the method performance on larger datasets such as DomainNet.", "theoretical_claims": "I have checked the theoretical claims in Chapter 3. They are clear and can support the method design.", "experimental_designs_or_analyses": "I think the model design is reasonable and supported by theoretical claims. The experiment designs and analyses are also reasonable and clear. My concern is about the experiment results on more complicated tasks.", "relation_to_broader_scientific_literature": "I think proposing merge-friendly quantization method for a specific downstream task is beneficial to the community and application of deep learning.", "essential_references_not_discussed": "N/A", "other_strengths_and_weaknesses": "Strength:\n1. The paper is overall well-written, with a very clear organization. \n2. The theoretical insights of this paper are demonstrated clearly, and it naturally supports the model design. \n3. The proposed method is reasonable and interesting to me. \n\nWeakness:\n1. As mentioned, I think the authors can provide more results on more complicated datasets or tasks.", "other_comments_or_suggestions": "N/A", "questions_for_authors": "See weaknesses.\n## After Rebuttal\nAfter reading the rebuttal and other reviews, I still tend to accept this paper.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "TaHIS6k7zS", "reviewer": "Reviewer_zjyC", "summary": "This paper proposes a post-training quantization (PTQ) method tailored specifically for the model merging scenario in multi-target domain adaptation. In particular, the paper argues that existing PTQ methods quantize each individual model into a state that makes them difficult to merge effectively into a single multi-domain model. To address this issue, the proposed method, HDRQ (Hessian and Distance Regularizing Quantization), introduces three specific design elements aimed at mitigating the degradation in merging quality. These include novel regularization strategies based on Hessian and weight-distance measures, as well as an advanced noise-sampling-based rounding technique.", "questions": "", "limitations": "", "rating": 2, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "First and foremost, we sincerely appreciate your thoughtful comments.\n   \n## Concern 1 : Limited merging setting\n\nWe emphasize that our work deliberately targets the most challenging scenario for enabling domain adaptation on resource-constrained edge devices. Specifically, we design our setting to be demanding in the following four aspects: \n1. Edge devices lack sufficient resources for fine-tuning. \n2. State-of-the-art PTQ algorithms are also too costly for such devices. \n3. To minimize communication costs, only quantized models can be deployed. \n4. Merging must remain lightweight.\n\nWhile exploring boundary-breaking methods is important, we also believe addressing practical and challenging real-world deployment scenarios is equally critical. From this view, complex, data-intensive merging methods fall outside our scope. Given the strict resource constraints and latency sensitivity of edge environments, merging must remain lightweight.\n\nWith this in mind, we adopt the interpolation-based merging strategy proposed in [1]. Notably, our baseline outperforms prior MTDA techniques, demonstrating its strength as a representative of state-of-the-art trends in model merging.\n\n[1] Wenyi Li, Huan-ang Gao et al, “Training-Free Model Merging for Multi-target Domain Adaptation”, ECCV 2024\n    \n## Concern 2 : Attribution of accuracy degradation\n\nAs shown in Table 1 (a) and (b), quantization degrades each model's quality and negatively impacts merged performance. However, when quantization is done with merging in mind, quality is better preserved. In Table 1 (a), HDRQ matches QDrop in sole quantization but significantly outperforms it in harmonic mean(1.69 mIoU / 4.14 mIoU, before/after correction).  This highlights the importance of merging-aware quantization.\n\n## Concern 3 : Baseline performance\n\nWe acknowledge that HDRQ initially underperformed QDrop and have devoted significant effort to address this. We identified a minor bug in our implementation of HDRQ—the quantization step size was not updated during training—leading to negatively biased results.\n\nCorrected results are shown below:\n\n|    |  | W8A8    | W8A4    | W4A8    | W4A4    | W3A3    |\n|:---:|:----:|:----:|:---:|:---:|:----:|:---:|\n| **R → ACP**  | QDrop | **67.91** | **67.4**  | 64.85  | 66.26  | 62.99  |\n|       | HDRQ  | 67.46 | 66.71 | **66.75** | **66.41** | **64.7**  |\n| **A → RCP**  | QDrop | **68.75** | **68.24** | 66.83  | 66.04  | 64.22  |\n|        | HDRQ  | 68.1  | 68.15 | **67.8**  | **67.58** | **65.29** |\n| **C → RAP**  | QDrop | **74.79** | **74.58** | 73.81  | 73.25  | 71.01  |\n|        | HDRQ  | 74.58 | 73.7  | **74.26** | **73.57** | **71.63** |\n| **P → RAC**  | QDrop | **64.8**  | **64.52** | 62.52  | **63.22**  | 61.24  |\n|         | HDRQ  | 64.51 | 64.09 | **63.93** | 63.19  | **61.55** |\n\n\nHDRQ generally outperforms QDrop at lower bit-widths of weight (W4A8, W4A4, W3A3). In 11 wins, HDRQ’s average margin is 0.95—nearly double the 0.41 margin across 9 losses. These results demonstrate  HDRQ’s consistent superiority over QDrop in many scenarios. We will revise all results in the final paper. Thank you again for the feedback led to this correction.\n\n    \n## Concern 4 : Comparison with SoTA PTQ method\n\nWhile we recognize the importance of comparing with recent methods, we note that BRECQ and QDrop remain strong and reproducible baselines despite earlier publication. Still, we agree on the importance of comparison with recent methods and conducted further experiments.\n\nSpecifically, we included FlexRound as a recent baseline, but excluded PTMQ due to its multi-bit focus and weak single-bit performance. We also tried to reproduce Shang et al.’s method, but the lack of official code and limited time prevented inclusion.\n\nWe validated implementation using 4W4A quantization on ResNet-18 with the official PyTorch checkpoint (top-1: 69.76%). FlexRound achieved 67.63%, and QDrop 67.87%, consistent with original reports (~0.2%p difference).\n\nThen we compared our method and FlexRound on semantic segmentation under 4W4A setting:\n\n| | FlexRound | HDRQ  |\n|:------:|:-----:|:----:|\n| **Sole - CS**  |   60.01   | 59.76 |\n| **Sole - IDD**  |   48.94   | 48.51 |\n| **Merged - HMean** |   46.44   | **46.73** |\n\nWhile FlexRound is slightly better standalone, HDRQ achieves a higher harmonic mean-highlighting the advantage of merging-aware quantization.\n\n## Concern 5 : Model merging strategy\n\nAs detailed in our motivation, our goal is real-time model merging on edge devices. While advanced methods could improve merging quality, edge devices often lack resources to handle complex techniques. Thus, we adopt interpolation-based merging as our default, given the limited research on quantization-merging interplay. We hope this work offers valuable insights in this direction.\n\nWe also acknowledge the missing key references and will include the suggested citations in the final version.", "claims_and_evidence": "The authors claim that conventional PTQ methods lead to failure in model merging due to uncontrolled weight perturbations, and they propose a “merge-friendly” PTQ method supported by both theoretical analysis and empirical evidence. However, several issues arise:\n\n- Limited Merging Setting:\nBoth the theoretical analysis and experiments consider only interpolated models—the most naïve strategy for model merging. In reality, many state-of-the-art (SOTA) merging methods exist and should be included for a more robust comparison. (See the “Essential References Not Discussed” section for relevant literature.) This limitation undermines the motivation for HDRQ, as it remains unclear whether the observed degradation is inherent to PTQ or specific to the merging strategy chosen.\n\n- Attribution of Accuracy Degradation:\nIn the experimental section (e.g., Table 1(b) and Table 2), it is observed that merged quantized models yield slightly lower performance (e.g., in terms of mIoU) compared to merged full-precision models. However, it is not clear whether this drop is due to the quantization process itself or because the quantized weights are less amenable to merging. For instance, Table 1(a) shows that quantization alone degrades performance even without merging. Hence, the authors should clarify whether the accuracy drop after merging quantized models is a consequence of quantization per se or of the incompatibility of the quantized weights with the merging process.\n\n- Baseline Performance:\nMore critically, the experimental results indicate that the proposed HDRQ method does not consistently outperform the compared baseline; in particular, Table 2 suggests that QDrop yields superior performance. This raises concerns regarding the superiority of HDRQ over existing methods.", "methods_and_evaluation_criteria": "The proposed HDRQ method is derived through theoretical analysis and is based on a well-reasoned process. However, as mentioned above, the theoretical derivation is based on a setting where model merging is confined to interpolated models—a scenario that is overly simplistic and not representative of more advanced merging cases. Moreover, the experimental evaluation does not incorporate SOTA merging methods.\n\nFurthermore, since the paper’s goal is to develop a PTQ method that is “merge-friendly,” the benchmarks should be drawn from established model merging benchmarks. The authors are encouraged to evaluate HDRQ across a variety of vision tasks, NLP tasks, and different numbers of target tasks, as well as on models fine-tuned via parameter-efficient fine-tuning (PEFT). The authors may want to learn from the following papers:\n\n[1] Yadav, Prateek, et al. \"Ties-merging: Resolving interference when merging models.\" NeurIPS 2023.    \n[2] Huang, Chenyu, et al. \"Emr-merging: Tuning-free high-performance model merging.\" NeurIPS 2024.", "theoretical_claims": "As mentioned in the Claims and Evidence section, the theoretical claims in this paper have a major issue: the basic setting is not entirely realistic because it only considers interpolated models. Apart from this initial setting, the subsequent derivations are straightforward and easily understandable. Consequently, the novelty of the theoretical contributions is limited, and the analysis does not convincingly establish that HDRQ can resolve merging issues beyond what standard quantization errors would introduce.", "experimental_designs_or_analyses": "As noted in the Methods and Evaluation Criteria section, the benchmarks employed in the paper require a more formal treatment. In addition, there are two major concerns regarding the experimental design:\n\n- The PTQ methods used for comparison are not state-of-the-art. For instance, BRECQ (from 2021) and QDrop (from 2022) are used, yet these methods are relatively outdated—and they have not been properly cited in the main text. More recent approaches (see the Essential References Not Discussed section) should be considered for a fair comparison.\n\n- The model merging strategy evaluated in the experiments is limited solely to model interpolation—a very naïve merging approach. There exist many advanced, state-of-the-art merging methods that should be included in the experiments to thoroughly validate the proposed approach.\n\nMoreover, the reported experimental results are not statistically significant. As mentioned in the Claims and Evidence section, the results in Table 1(a) and Table 2 do not clearly demonstrate HDRQ’s superiority; in fact, the baseline QDrop appears to outperform HDRQ.", "relation_to_broader_scientific_literature": "N/A", "essential_references_not_discussed": "Representative papers about model merging:    \n[1] Yadav, Prateek, et al. \"Ties-merging: Resolving interference when merging models.\" NeurIPS 2023.    \n[2] Huang, Chenyu, et al. \"Emr-merging: Tuning-free high-performance model merging.\" NeurIPS 2024.    \n[3] Ilharco, Gabriel, et al. \"Editing models with task arithmetic.\" ICLR 2023.    \n[4] Wang, Ke, et al. \"Localizing task information for improved model merging and compression.\" ICML 2024.\n\n\nSOTA papers about PTQ:    \n[1] Lee, Jung Hyun, et al. \"Flexround: Learnable rounding based on element-wise division for post-training quantization.\" ICML 2023.    \n[2] Shang, Yuzhang, et al. \"Enhancing post-training quantization calibration through contrastive learning.\" CVPR 2024.    \n[3] Xu, Ke, et al. \"Ptmq: Post-training multi-bit quantization of neural networks.\" AAAI 2024.", "other_strengths_and_weaknesses": "Please refer to the other sections. Thanks.", "other_comments_or_suggestions": "Please refer to the other sections. Thanks.", "questions_for_authors": "Please refer to the other sections. Thanks.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Meta-review: This paper proposes ​​HDRQ​​, a merge-friendly post-training quantization method for multi-target domain adaptation, addressing how standard quantization disrupts weight alignment during model merging. While reviewers initially raised concerns about motivation clarity, baseline comparisons, and marginal gains over QDrop, the authors clarified the edge-device use case, corrected implementation bugs (improving results), and added comparisons with recent methods eg. FlexRound. Allthough benefits remain modest and advanced merging techniques were untested, the paper’s theoretical contribution and practical relevance for resource-constrained scenarios justify acceptance. Thus the AC recommends acceptance. Congratulations to the authors!
Decision: Accept (poster)