Minimax-Optimal Aggregation for Density Ratio Estimation
Authors: Lukas Gruber
Markus Holzleitner
Sepp Hochreiter
Werner Zellinger
Venue: ICLR 2026 Poster (2026)
Keywords: Density Ratio Estimation
Self-Concordance
Kernel Methods
Domain Adaptation
Primary area: other topics in machine learning (i.e., none of the above)
TL;DR: An aggregation algorithm for minimax-optimal hyperparameter optimization in density ratio estimation.

Abstract: Density ratio estimation (DRE) is fundamental in machine learning and statistics, with applications in domain adaptation and two-sample testing. However, DRE methods are highly sensitive to hyperparameter selection, with suboptimal choices often resulting in poor convergence rates and empirical performance. To address this issue, we propose a novel model aggregation algorithm for DRE that trains multiple models with different hyperparameter settings and aggregates them. Our aggregation provably achieves minimax-optimal error convergence without requiring prior knowledge of the smoothness of the unknown density ratio. Our method surpasses cross-validation-based model selection and model averaging baselines for DRE on standard benchmarks for DRE and large-scale domain adaptation tasks, setting a new state of the art on image and text data.

Published as a conference paper at ICLR 2026
MINIMAX-OPTIMAS AGGREGATION FOR DENSITY RATIO ESTIMATION
Lukas Gruber Markus Holzleitner Sepp Hochreiter
Werner Zellinger
ELLIS Unit Linz and LIT AI Lab, Institute for Machine Learning Johannes Kepler University
Linz, Austria
(gruber, zell zellinger) jku. at
ABSTRACT
Density ratio estimation (DRE) is fundamental in machine learning and statisties, with applications in domain adaptation and two-sample testing. However, DRE methods are highly sensitive to hyperparameter selection, with suboptimal choices often resulting in poor convergence rates and empirical performance. To address this issue, we propose a novel model aggregation algorithm for DRE that trains multiple models with different hyperparameter settings and aggregates them. Our aggregation provably achieves minimax-optimg error convergence without re- quiring prior knowledge of the smoothness of the unknown density ratio. Our method surpasses coos-valldattion-ausee baselines for DRE on standard benehmarks for DRE and large-scale domain adap- tation tasks, setting new state of the art on image and text data.
model selection and model averaging
INTRODUCTION
Let {Xi) 1Md and (ar J. with M, N E N, be two sets of samples independently drawn from probability density functions p and y, respectively. The goal of Density Ratio Estimation (DRE)
to learn the density ratio 3(z) == p(a) from these samples. According to the fundamental lemma
q(z)
of mathematical statistics, statistical test with maximal power has a unique representation
threshold decision on the density ratio (Neyman Pearson (933). is therefore natural that DRE plays central role in machine learning and statistics (Sugiyama al. (012a)) DRE is used in a wide range of applications, including distribution shift detection (Shimodaira 2000; Dinu et al.
2023 Della Vecchia et al.. 22025), divergence estimation (Nguyen al.) 2007 |010 Rhodes al.,
2020 anomaly detection (Smola et al., 2009 Hido et al. 2011), energy based modeling (Ceylan &
Gutmann 2018 Gutmann & Hyvärinen] [2012] Tu 2007 Grover Brmon 2018). and generative
modeling (Mohamed & Lakshminarayanani 2016]] Grover et al. 20199 Kim et al.) [2024] Sanokowski
al. 2025). Unlike supervised learning, DRE performed without access the true density ratio
values [(z) for the combined sample set (um I U {>/
However, the accuracy of density ratio estimators is highly sensitive to hyperparameter choices, which has already been observed in KKanamori et al. (2009a]b). For example, a series of works (Sugiyama et al. 2007| 2008 2010) investigates kernel bandwidth selection for Kernel Mean Matching (KMM) (Huang et al. (006)., which cannot be done by straight-forwart cross- cosss-alidation (CV). In practice, one often ends up with a sequence of models trained with distinct hyperparameter settings, with the goal of selecting a model that achieves fast convergence rates.
In this work, we address the problem of optimally aggregating models in such sequence. In contrast to related approaches, c.g., gross-validation used in (Kanamori al. [209aa] Bickel al.) 2009 Menon & Ong 2016 Nguyen et al., 2010 Rhodes et al., 20200] Choi et al.) [02 Srivastava
al. 2023 Zellinger 2025 Kato & Teshima 2021), aggregation can outperform the best model in such a sequence of models, see Figure
Informally, our method trains model fa for each hyperparameter setting, and subsequently op- timizes the aggregation Ee 04 fk s.t. minimax-optimat emror rates are achieved. This is done by optimizing upper bound the Bregman divergence between the aggregated estimator and the

---

Published as a conference paper at ICLR 2026
25
Gaursian Q- N:0.1:
True Density Ratio P
PDF Gaursian p- N20.0.5) Yroe Dennity Balio
Model
Moddel 2
1.5
Our Aggrogation
1.0
0.0
-2.0
-1.5
-1.0
-0.5
0.0
0.5
1.3
3.5
2.0
-2.0
-1.5
-1.0
-0.5
0.0
0.5
1.0
1.5
2.0
Figure 1: Leftt Density ratio between Normal p p= (0, 0.5) and q = |0 1). Right: Procedures for selecting a single Model 1-3 (dashed) cannot approximate 3 well. In contrast, aggregation method (solid) achieves good approximation.
true density ratio 3. More details in Eq. 5 that follows (Zellinger al. 2023). combining (Menon
& Ong 2016) with (Marteau-Ferss FFrey et al. 2019 Bach 2010). Our approach is summarized in Algo-
rithm I that conveniently computes an analytic solution of this upper bound. This solution satisfies minimax-optimal convergence rates for optimization in reproducing kernel Hilbert spaces (RKHS),
following the theory for self-concordant loss functions (Bach) 2010) Marteau- Ferey al., 2019
Beugnot et al. 202Tp. Our approach can be applied to DRE methods which don't it allow parameter tuning by CV (such as KMM), and is computationally more efficient than standard n-fold CV (see Ablation 2 in Table [ and Figure (Left)). We conduct experiments on DRE benchmarks, well as on large-scale model selection benchmarks for deep domain adaptation (text, image, time-series), training over 9,000 neural networks and performing approximately 500 hyperparameter selection tasks. Results underpin the empirical effectiveness our approach, which sets a new state of the art for model selection Nominoominn and Amazon Reviews.
Our main contributions are:
We propose novel theory-grounde aggregation method for approaching hyperparameter choice issues in DRE.
We prove moniimax-ppimme error convergence (for M,N + oo) for a broad class of DRE meth- ods (Sugiyama & Kawanabe, 2012) when applied in combination with our method.
We conduct experiments on benchmarks for DRE and large-scale domain adaptation tasks on diverse data modalities (text, image and sensory data) involving over 9000 neural networks and 500 hyperparameter selection tasks; validating outperformance of related approaches.
Our method improves cross-validation in density ratio estimation in terms of (a) computational efficiency (see Figure 2 (Left), (b) the possibility of recovering density ratios not present in the given sequence of models (e.g., Figure [) and (c) applicability (e.g., to parameter choice issues of KMM, Ablation 2 in Table
Our method achieves better performance than recent and advanced model averaging and ensem- bling baselines for DRE.
BACKGROUND
was first observed by Sugiyama et al. (2012b) that a broad class of methods for DRE use the objective
fH arg min (r(3,g(f))
fEH
and derive the desired density ratio estimator by 3/ == g(xx, where R + IR is strictly increasing function (depending on the specific DRE method as in Example and
BF (3. ==
-(((()))(() - (())]]
denotes the Bregman divergence with prescribed generator + R. For our theoretical results, assume H to be in RKHS, which is not required for algorithm and experiments.
2

---

Published as a conference paper at ICLR 2026
For example, as be seen in Table [ experiments conducted for both, methods in RKHS and neural networks in a given model class. Consider the following example (cf. Sugiyama et al.
(2012b); Menon & Ong (2016); Kato & Teshima (2021) Zellinger et al. (2023): Kim et al. (2024);
Zellinger 2025)):
Example
As shown in Kanamori et al. (2012b) the kernel unconstrained least squares importance fitting
procedure (KuLSIF) KKanamori al. 2009a) is realized by using F(x) (x - 1) and
g(x) = in equation such that (3 (())
1|3 g(f) I1
2. Menon & Ong 2016) use F(x)= x²/² and g(x) = 22 in equation
to obtain the exponential
function approach (Exp) as applied for AdaBoost (Freund et al. 1996).
3. The square loss approach (SQ) in Menon & Ong 2016) uses F(x) 2042 and g(x:) =2-2- 22-xx
-1+2r
4. To realize the logistic regression (Nelder Wedderburn 1972 (LR) approach as used Bickel
et al. (2009) F(x) = x log(æ) + 2)) log(1 x) and g(x)= = e are set in equation
3 RELATED WORK
This work is along the lines of research that investigates methods that achieve sinite-sample error rates for approximations of equation [ Such enror rates were shown for KuI.IFF in Kanamori et al. (2012b): Gizewski et al. (2022); Nguyen et al.] (2023) and extended to class of DRE methods with self-concordant loss functions by Zellinger et al. (2023); Gruber et al. (2024). Their work demonstrates that fast error rates can be achieved when the optimal hyperparameter configuration employed in Eq. 1 However, to achieve fast error rates when estimating the finit--saplle ap- proximation in Eq. i crucial have appropriately chosen hyperparameter setting. DRE is a complex problem and different studies highlight its complexity. For example, problems such as density-chasm (Rhodes al. 2020) arise when the underlying densities exhibit significant dissimi- larity Srivastava et al. (2023) Zellinger (2025). To mitigate these issues, Choi et al. (202T) propose leveraging normalizing flows to achieve simpler and closer approximations of the target densities.
Many applications exist. For example, recent research has explored deep learning- based DRE meth- ods in positive-unlabeled learning (Kiryo al., 2017: Kato & Teshima 2021). Another active area of research focuses on integrating DRE techniques into deep generative models. For exam- ple, Kim et al.] (2023) utilize DRE to enhance diffusion- basied generative models. Subsequently, Kim et al. (2024) extend the Bregman framework to incorporate time-dependent corrections and in- troduce novel Bregman divergences. DRE methods have also been applied to diverse tasks beyond generative modeling. These include conditional density estimation (Schuster et al.] [2020), mutual information estimation (Poole et al. 2019: Belghazi et al. 2018 Song Ermon 2020), importance weighting (Sugiyama et al.] 2012a Fang et al.) 2020 Zhang al 2023), model-based enrichment
estimation (Busia & Listgarten 2023) and unsupervised representation learning (Thomas et al.
2020).
Although aggregation methods for averaging and ensembling density ratios have been used (Zhu & Tan 2013 Chandra al. 2018: Rhodes al.) 2020 Fragoso et al. 2018 [ushhhyn & Ustyuzhanin, 2021 Yu et al. 2021 Wu & Benkeser 22024), none of these methods can guarantee fast convergence
rates for finite sequence of suboptimal models, even when the sample size M + N grows infinity,
see Ablation 3 in Table I and Fig, 2 (Right) for practical consequences.
4 PROBLEM
Notation Following Menon & Ongl 2016)), equation I can be equivalently formulated as binary classification with loss function
x R + I and y 1), such that ((1, fx)) and {( (-11.(()) measure the error of a classifier f(x) predicting whether x is originating from or q- For this, we only need the assumption that the combined sample
z = {(.m 1) ((=/,, -1)].
is an i.i.d. sample from the distribution P constructed as follows Zellinger al. 2023): is defined
as a probability measure on x y' with conditionals p(zly = 1) = p(z), p( = 1) := (()), and, marginal By defined as Bernoulli measure such that the probability for both events y 1,
3

---

Published as a conference paper at ICLR 2026
Algorithm DRE Aggregation
Input: K different hyperparameter setups, dataset z = {(Tm. 1) U Ui - 1] - see 2
Output: Model ((x) 9 E axf((z)) with optimal coefficients = (ā....., kx) E I K
Step 1: Train one model 3k for each hyperparameter setup (e.g., using method in Example I
and denote by fx = g (3,) the associated binary classifier (see Example 2)
Compute aggregation weights ā G 'f with empirical Gram matrix G and inner product vector F defined by
Step 2:
K
G= M+N
Σ (Vill, ((()f(())) (2.8)
f=
E (V+tl. fa(2))
M+N (R.a)
km
with second derivative ( (x)) w.r.t. f (computed analytically) as in equation 11 Ap- pendix B for an example.
Return: Aggregation 3(x) - و (Etn (x))(
-1 is 2. Then equation]] 1 is equivalent to the minimization (Menon & Ong. 2016)
JH == arg min R()) fEH
with expected risk
R()) = / E(y. ((x))dp((,.))
JxXY
and the density ratio estimator 34 := g(fx) can be recovered from the classifier fx. For simplicity, assume that the problem well-specified. i.e., fx arg min f:x=>R R(f). For instance, all the
methods discussed in Example [ can be optimized through risk minimization equation 3 by using the loss functions in the following example (Menon & Ong 2016).
Example
1. KuI.SFF satisfies equation 3 by ((-1, = ( I.
Exp can be obtained by 1, = (11,x) =
SQ realize equation with ((-1,a) = +1+z) ((1,)) =(1 -1-r))² x)2
4. For LR equation 3 holds with ((-1. 2) = log(l (e) e² (11,)) = log +e
Problem Given density ratio models ,,,...,Br : X + R, each trained with different hyperpa- rameter setting, the goal is to find a model X + IR with minimal error (3,, 3).
APPROACH
We approach this problem by an aggregation of models
B == 9 Σ anfk
(4)
with fs := g- ((3k being the binary classifier associated with the density ratio estimator 3k. and the aggregation weights .1...., a, which we compute as follows.
(2019) bound the Bregman divergence in equation b norm (cf. Zellinger et al. (2023) for this strategy) s.t.
First, we follow Menon & Ong (2016) and Marteau- Ferey
Br (3.0) 2(((()) R(fx)) < ((3) fr
=
(5)
4

---

Published as a conference paper at ICLR 2026
where the norm ||.|| depends F and g. a1...., ag that minimize this upper bound
Our approach is to choose the aggregation weights
min ERR
Σ an - SH
(6)
which has three immediate advantages: (a) an analytic solution, (b) computational improvement compared to n-fold CV (see Fig. 2 Left) and (c) minimax-optimal rates (see Theorem I Analytic solution necessary optimality condition for (6) leads to the solution
K
K
L(a) =
Σ
aufi fH =
Σ
anfi - Sn. Σ anfa - f'H
K
K
Σ
Onej (Jk: -2 - ank (sk, IHs) + (s)
=
k,j=1
k=1
OL(a) 2
K
Ok (Jk, - (Sk, (JesJ)) =0
dak
k=1
r= ((k,ff)) After discretization and using binary labels 1} as specified in equation 2 for minimizer fx we arrive at Algorithm [
with G G= ((fer)
Relation to model selection methods In continuous case (i.e., with access to p, p. our aggre- gation strategy is at least as good as model selection:
K
K Σ k=1
min
Σ
cn
SH1
S
min an E (0.1)
SH
S min (l(.....KK
[[
szell (7)
0100000
ke
1=1
However, as we have only access to finite samples from p. we need to investigate the accuracy of the discretization more detail.
RESULTS
6.1 THEORETICAL RESULT
In the following we show that our aggregation approach Eq. 4 achieves minimax- optimal error rates when Eq. is optimized RKHS with norm ||-I|x and Tikhonov penalty ||/ s.t.
arg min FEH
By Br(3,g())
(8)
where À 0 is the regularization parameter which aim to choose. All DRE methods in Exam-
[ fit into the optimization problem in equation 8 see Zellingee et al.) (2023) Gruber et al. 2024 We fix sequence (> of regularization parameters and refine aggregation equation by
K
3 =
Σ
To prove that our Algorithm [ achieves minimax-optimal convergence rates, we introduce typical assumptions in learning theory, such on data, the regularity of the underlying problem and the previously introduced loss function e.
The loss function E y x R which is used for separating {(rm) 1)] M from {(*%, ~ has an associated link function vb [0, + R with the following properties:
Assumption 1.
vb is invertible and the associated conditional Bayes risk G(u) = u E(1, vu)) + -11
u) ((-1, (u)) is twice differentiable.
5

---

Published as a conference paper at ICLR 2026
the minimizer fu satiifies fu(2) = pp(l 1/2)). Assumption (source condition). There exist some r E (0, 1b v E H. satisfying fx H(f+)' for the expected Hessian H(f) := B(e.a)~ (x)))
Assumption (capacity condition). There exist 2 and S such that dfx S SX-= with the degrees of freedom
dfx E (s.y)~)
- vn(w)f((=))
and Ha( = H() XL
Both, source and capacity condition, are typically used in learning theory (Caponnetto & De Vito, 2007 Bauer et al. 2007; Marteau-Ferey
al.] 2019] Pereverzyev] (022) to encode the regularity of the underlying problem. We also need the following self-concordance assumption (Bach 2010 Ostrovskii & Bach 2021) on our loss function, which is known to satisfied for the examples we discussed (cf. Zellinger al. (2023)). Assumption 4 (Pseudo self-concordance). For any y E V. the function ly R. R defined by (T) == E(y, 7) is convex, three times differentiable and satisfies
1/" (n)) S ())
Note that by Lemma in (Grbber et al. 2024), Assumption [ implies generalized self concor- dance (Bach 2010). which is also employed in Marteau- Ferey et al. (2019); Zellinger et al. (023)) As a next step we discuss the specific norms that occur in (6) and in more detail. From Menon & Ong (2016); Marteau- al. (2019) know that
BF ( 8 Br(8, x(H)) S 2 2
SH HHA (3)
for fu. Hf(fH) as in Assumption 3 and
(10)
However, the norm values || ( are not directly accessible, as we have only observation from the measure p. That is, we have to estimate the norm to find an approximate solution to the opti- mization problem (6). In the following, consider some small regularization parameter value À > 0 (as precisely specified in Section and the empirical Hessian and empirical risk minimizer
H*() M N
Σ v't(y.S(x)) + AI, f == arg JEN
1 M N
X ((v.((a)) + II (R.N)ER
Σ
(2.g)
(11)
with combined sample z as defined in equation 2 It is worth noting that, for practical applications, the Hessian terms can be obtained analytically and do not ineur any additional computational cost. We are now in the position of proving our main convergence rates result, the proof and constants can be found in Appendix Section A
Theorem 1. Let assumptions
7 and technical assumptions from Appendix satisfied. Consider K > 1, 8 > 0. {Ae) as defined in Appendix and associated fhr as in equation 11 Then we have that for B of Algorithm I applied with 3y. ;= p(P))):
BF (8.3)
(r(B,g((x)) S C(M+ + N) Sro-to
(12)
with probability at least 1 (9 + 2k)5, for sufficiently large sample sizes M, N and some C >>>>> 0
that is independent of M.
Remark 1. To the best of our knowledge, Theorem 7 provides the first provably minimax-optimal convergence rates for a parameter choice procedure in DRE. Zellinger al. (2023) provide similar rates but they rely on a heuristically fixed constant for their implementations of the balancing prin- ciple. For proofs of the minimax optimality see Caponnetto & De Vito (2007) v; Blanchard & Mücke 2018 Steinwart el al.) (2009); Marteau-Ferey et al. (2019); Beugnol et al. (2027); Zellinger el al. 2023 While we focus on the regularization parameter, our algorithm can be applied to any hy- perparameter setting of given DRE method, see Ablations and 2 in Table 7 For a detailed discussion about how noise affects this bound, we refer to Appendix Section C
In the following, we go beyond theoretical convergence results and conduct a comprehensive empir- ical evaluation of our algorithm across multiple benchmark datasets.

---

Published as a conference paper at ICLR 2026
6.2 EXPERIMENTS
We evaluate benchmarks with known density ratios (Kanamori al. 2012b) where we generate ten distinct geometric datasets using Gaussian Mixture distributions in a 50-dimensional space, see Appendix D for more details.
To evaluate the effect of our algorithm in large-scale real-world scenarios we conduct extensive ex- periments involving the training of over 9000 deep neural networks on the task of domain adaptation
(DA). Given a source dataset {>',, ) n= ~ with labels (vn J and a target dataset { {Imm M iid
iid
without labels, the goal is to learn a model ø' with low expected risk R(Ø') := E( [
[L y. [x))
on the target domain without having sampled labels from the target domain. Motivated by Sugiyama
al. (007)) this task can be approached by approximating the density ratio 2 with estimator / and empirical risk minimization such that
= arg min 6E
Σ 3(xr)) ((x)))
(13)
In our case, ø' is a downstream predictor constrained by a DA method as in Table MMDA) and optimized separately for each DA method. Our aggregation approach provides improved density ratio estimator 3(.), see App. D for further details. To comprehensively test our aggregation approach, we use the four popular DRE methods presented in Example in all our main experiments.
Results for known density ratios In Table we compare cross cooss-vllidttion-biee tion (left part of the table) to aggregation approach (right part). Across all 10 geometrically constructed datasets, the performance of all evaluated DRE methods is clearly better for aggrega- tion than for cross-validation. All aggregated methods consistently outperform their non-aggregated counterparts where models are selected by cross-validation. In Fig. ([.ft)) we demonstrate that our method scales computationally more favorable than 10-fold cross validation when increasing the number of hyperparameter settings.
model selec-
We further conduct two ablation studies on geometric datasets to demonstrate that our approach re- mains broadly applicable under settings with relaxed theoretical assumptions, All other experiments conducted fall fully within the scope of our theoretical analysis. For Ablation we train several deep neural network-basee logistic regression models (DeepI.R) with different hyperparameter settings beyond regularization parameters (see Appendix [D) and optimize objective (6) as done for the other DRE methods. In Table T Ablation we can see that our aggregation approach improves experi- mental results on all datasets compared to its baseline model that was selected by cross-validation (CV). Similarly, in Ablation 2 we use our approach to tune the kernel bandwidth parameter in Kernel Mean Matching where CV cannot be used. As baseline we set the bandwidth parameter according to the popular median heuristic (Schölkopf & Smola 2002). In Table [ Ablation 2 it can be seen that our aggregation method improves upon this on all datasets.
In Ablations 3 and 4, we evaluate method geometric datasets against aggregation baselines employing model averaging and ensembling strategies for SuI. As shown Tab. ( (Ablation 3), our aggregation method consistently outperforms model averaging as proposed by Raza SSamoth rakis (2019). Ablation 4 investigates how the performance changes as we increase the number of trained models. Fig. 2 (Right) demonstrates that our approach remains robust as the number of (po- tentially suboptimal) models increases. In contrast, advanced and recent model averaging and en- sembling techniques like Raza & Samothrakis (2019), Bayesian Model Averaging (BMA) (Fragoso) et al. 2018 Yeung et al.) 2005). and Super Learner (SL. (Wu & Benkeser 2024) exhibit instabilities and suffer from severe performance degradation. These findings highlight the theoretical guarantees of our method, which ensure fast error convergence when optimizing the aggregation weights. This is further illustrated in Fig. 3 (Left) in the Appendix, where we examine the weight assignments to KuLSIF estimators with different hyperparameters dataset "c3,d1.70." More accurate models are assigned larger weights.
In Ablation 5 we test the scalability of our method using the KuLSIF and LR DRE methods by grad- ually increasing the dimensionality of the Gaussian mixture datasets. As shown in Fig. 3 (Right) of the Appendix, our aggregation method (dashed lines) maintains robust performance across di-
7

---

Published as a conference paper at ICLR 2026
Grometric Figurss
Croee Validation for Binary Chosehee
Agoneaation
Datase
KII.SIF
Exp
LR
SQ
Kul.SFF
Exp
LR
9Q
c3.11.0
10.01)
10..00100000
S,500 100021)
9.1
9,320:10.00 004
8.151(10.011)
8.573(-00010
0.001
c2,11.2
03.001::0 1:001::0000000
12.904 100015)
13,255 L.0 0133
13,537 -0.002)
12.854 01.854:11000 007)
12,360 10.0011
13:250;100000
c2.41.59
22.220 (
10.765400055
12,829 25 0144
18,056
(13
12.422 :00.03)
12.441 (20.042)
=0.12)
12615(=0011)
cl,d1.55
11 11.11::00007 001)
14.477(60.110
12,0001
200 0233
12.175
-0.002)
11.025:2:000 004)
11.824.200015
00.011:0.0151
100410)
62,41.78
18.0082200000
40083)
9,000
20.0066
3.425:1.0.033
17.043: 10.015)
9. =0002)
0.623
c2,01.55
10 20..07
9.774/10:191
10,5301 2.0 0099
10.757 -4, (22
10.001
002)
9.520/10.0111 10.010
10:415|10.000 0391
00011
c3,d1.77
100000)
15000/110112
12,2141 10 057)
14,048
L00 20.002)
12.003
1.0.0.0..0000 007)
12.0211 (10. 008)
11 202201000001
12.940
c2.11.61
11 210004000004
11.801 0083
12 12,2121
(07)
11.365:+10 003)
10.787
10 00.920: :0 008)
11.891(=0000)
c3,01.45
12 12.013:3:00009
(2.01((200088)
22.771
18.150 -1
9.421 0.22::200000
12,025
12 12:152 004)
12.970[=00004
c1,01.63
(.1.7(2110000)
9.704
9,732 (00014)
9.900
0.307:10.008)
90.01
(.001(
0..09
Avg
11 200
13.00((00000)
11.000
11. 10,015
10 10.683
11 11.5529 1000010)
11 (0.11
00.34)(00022)
Dep Leumingg
Mathods
<3,t m
224123
2MI 9M
cidl.95
<MI.75
cAd 55
CMI.5
276161
CVI
DeetUR
14 -0.011
738) -0000)
8.560
0081
12 18.738--000
0011)
11.22:100.110
Mehheds
2301.00
62.1.72
C341.9
4841 55
CAIL35
COBLAL
Mel Bosr
(97)
KMH Apg
15.223) 10 (71)7))
18:155H AASSSEMAO (917
16.530 20 (MS) 20 35.319
13 33960) 90W...0.000 0..068)
10 10.033
17.02 L0.334)
16.12 07T))
14.50N 2I (0A5)
Ablation
CHULST
NaLSUr
11 (OAEE)
BAND LIERIE
HOLDOY BATARAS
ArE
KeL 8.320 (004))
12 U000
13 422 (10)
12.6920 100. QU6))
U 423 LO001S)
122 OUS
B. 6. (113)
Table 1: Mean and standard deviation (after +) of twice the Bregman divergence error on the geo- metrically constructed datasets following Kanamori et al. (2012b) over ten different sample draws from P and Q
mensions, whereas the cross-validation approach (solid lines) degrades due to growing difficulty in hyperparameter selection.
These results confirm that, compared to other averaging and ensembling methods for DRE, our approach exhibits fast error convergence and remains stable when increasing the number of mod- els. For completeness, we also include a BMA method discussed in (Fragoso et al. 2018) and based on (Yeung et al.j (005), and Super Learner (Wu & Benkeser 2024) as model averaging and ensembling baselines in our experimental evaluation. In Table 2 we show the performance of all compared averaging methods on the geometric datasets, results are averaged over datasets, for indi- vidual results see Table in the Appendix. Consistent with prior findings, approach consistently outperforms the baseline averaging and ensembling methods across all geometric datasets.
1200
Agd comp
26
1000
CV
Ago
tranod modes
800
22
fgg
5L
600
10
16
14
200
12
10 10-11-11011
25
Figure 2: Left: Computational complexity, x-axis with the number of models with distinct hyperpa- rameter settings, on y-axis the number models that have to trained. Our method (orange) needs to train by factor of n-folds (here 10) less models compared to CV (blue). Right: Our aggregation method (orange) remains robust to models with suboptimal hyperparameters. Other averaging and ensembling methods (red, green, purple) severely degrade when increasing the number of models.
Results for domain adaptation We evaluate the performance of our method on three DA modal- ities (image, text, time-series) as conceptually introduced in Eq. [3] for more details see App. D The results in Tab. 3 (due to space limitation disaggregated by DA methods only for MiniDoman- Net) demonstrate that combining established DRE methods with our aggregation approach consis- tently outperforms their non-aggregated counterparts for which model selection is done by cross-
8

---

Published as a conference paper at ICLR 2026
Geometric Figures (Avg.)
Method
KuLSIF
Exp
LR
SQ
Cross Validation
(000000000000))
(..399((+0009)
11.400(+0.0233)
11.810(++0015)
Bayesian Model Averaging
11.119(##0008
12.972(+0.017) +0.017)
01.99)(+00019)
11.785(=0.013)
Super Leamer
11.009(+0.007)
12.386 +0.015)
11.320(=0.016)
11.658(+0.012)
Aggregation (Ours)
10.683 100.006)
(10.011
11.002 (00.014)
11.348 (10.012)
Table 2: Mean and standard deviation (after =) of twice the Bregman divergence enror the geo- metrically constructed datasets following Kanamori et al. (2012b) over ten different sample draws from P and Q and averaged over datasets.
validation. The observed improvement is consistent across the three domain adaptation modalities and all evaluated domain adaptation methods. Notably, our approach also achieves state-of-the-ae art performance, surpassing the results Dinu al. (2023) on MiniDomainNNt and Amazon Reviews. Analogous to the experiments on the geometric datasets, we include model averaging and ensem- bling baselines for completeness in Tab. 3 As previously, our approach consistently outperforms the competing methods. More detailed results, including the performance of individual model averaging approaches and DRE methods across specific DA scenarios, are presented Tab.
Domain Adaplation: MiniDomsinNit
Cruss Validation Binary Clasifier
Aggregation
DA-Method
Kol.SIF
Exp
LR
SQ
Kul.SIF
Exp
SQ
MMDA
L..009)
0.525:
10.011)
0.528(10.0:3
0.008(10.010)
0.536
10.0077
0.539/ 10.0011
0.536: 006)
0
009)
CODATS
0.542(
20.00
0.530li 20.02))
8.527(=10.003
0.542;000010
0..84))
0.540
0.589:+0.013
DANN
0.581(-
0.522;-00 +5.02-
0.500;+000..0
0.506)00006)
0.534]-0.007 000007
-0.011)
0.519:+0.015
CDAN
2..022
0550111000111
0.523110.023)
0.512 10.021)
0.587
0.544
0.005)10011)
0.526:10 10.020)
DSAN
0.53%
10.010.0011 00.01)
0,532 00.015
0.513/10.013 0.527(100015)
10.013)
0.544 16444_00000 0..00
[.566_00011)
0.535;10.008
0.525:10- 012
DIKT
-.017(=111155
0.386 177;
0.520)2200023
0.50K =000
0. 0.523;=0.002
0.895(001175)
0.526(=00017)
0.52G 020
Ad-SKM
0.516 0.000 00%)
0.515;-0.000 000
0512(++00:2)
0.500-00.00
5222 00.00000000 00.005
0.529:-0 02.00-0.008
0.521;+0.000 008)
0.517:+0 008
HoMM
0.531/10.008
0.529 10000000000
0.521(10.015)
0.305(1.0.00
53.599
0.544|-0.011
0.329:10
0.528:10 012
DEC
00.100
0.51(/200:2)
0.500=00.010
0.529(=0.013
0.616:#0
DeepCoral
1.005:
0.000:100022
0.514;
0.540
0.527.10
0.0000
20001
20000))
0.000
0.3 0.538|=0.002
0.527 015)
0.521/20
Arg.
(..26(=0.0113
0.518(=0009))
000077)
0.500(=000033
0. 0.005
(0.26()
0.330 01101)
0.626
Domain Adaptation: MiniDomainNet (Avg.)
Method
KuLSIF
Exp
LR
SQ
Cross Validation
0.528(100001)
0.513(=0.029)
0.523(+0.011)
0.510(=0.015)
Bayesian Model Averaging
0.528( +0.008)
0.514(+0.027)
0.523(+0.015)
0.507(-0.015)
Super Learner
0.530 +0.010)
0.514(+0.027)
0.523(+0.011)
0.511(-0.013)
Aggregation (Ours)
0.535 10.000)
0.526(10.028)
(..00(1
0.525 0.014)
Domain Adaptation: Amazon Reviews (Avg.)
Method
SIF
Exp
I.R
SQ
Cross Validation
0.787(10.011)
0.771(10.010)
0.785(11.001)
0.781(10.009)
Bayesian Model Averaging
0.787(10.012)
0.772(10.010)
0.785(10.011
0.781 0.782(=0.008)
Super Learner
0.788(+0.010)
0.709)=
0.785
Aggregation (Ours)
0.795(110.09)
0.783
0.789 0.789 +0.011)
0.789 -
Domain Adaptation: HHAR (Avg.)
Method
KuLSIF
Exp
LR
SQ
Cross Validation
0.737(+00080)
0.604((=0.085)
0.772(+00019)
0.731(=0.011)
Bayesian Model Averaging
0.738(++0.077
0.711(=0.084)
0.772(+0.018)
0.738(=0.013)
Super Learner
0.740(10.076)
0.701(=0.077)
0.703(20.015)
0.743(=0.010)
Aggregation (Ours)
0.780(10.082)
0.707(=0.075)
0.759 0.785(10.014)
Table 3: Mean and standard deviation (after +) of target classification accuracy on MiniDomainNNt, Amazon Reviews and HHAR datasets over three different random initialization of model weights and several domain adaptation scenarios.

---

Published as a conference paper at ICLR 2026
7 CONCLUSION
To approach hyperparameter choice issues in DRE, we proposed a novel model aggregation algo- rithm that trains models with distinct hyperparameter settings and subsequently aggregates them. The aggregation coefficients are optimized to achieve minimax-optimal convergence rates without requiring prior knowledge of density ratio smoothness. Empirical evaluations on DRE benchmarks and scale domain adaptation tasks demonstrate that our method consistently outperforms es- tablished DRE methods, and model averaging and ensembling baselines, achieving new state-of- the-art results MininDmminNNe and Amazon Reviews.
REPRODUCIBILITY STATEMENT
We have taken several steps ensure the reproducibility of our results. All theoretical assumptions and complete proofs our main results are provided in Appendix A Details on the derivation of loss functions and Hessians used in our algorithms are included in Appendix For empirical repro- ducibility, Appendix D provides comprehensive information on experimental protocols, including dataset generation for benchmarks with known density ratios, preprocessing steps for MiniDomain- Net, HHAR, and Amazon Reviews datasets, as well as details on compared methods and hyper- parameter ranges We also report full ablation studies, experimental setups, and results tables in the appendix to facilitate verification. A reference implementation of the methods presented this paper available at: https//gittuh..oo//uggbbheee/
LLM USAGE
An LLM was used solely to aid with polishing the writing. It did not contribute research ideation, analysis, or content creation.
ACKNOWLEDGMENTS
The ELLIS Unit Linz, the LIT AI Lab, the Institute for Machine Learning, are supported by the Federal State Upper Austria. We thank the projects FWF AIRI FG 9-N (10.55776/FG9), Al4GreenHeatinGrrids (FFG- 899943), Stars4Waters (HORIZON-CL6- CL6-2021-CLIMATEE CLIMATE-01-01), FWF Bilateral Artificial Intelligence (10.55776/COE12) We thank NXAI GmbH, Audi AG, Sil- icon Austria Labs (SAL), Merck Healthcare KGaA, GLS (Univ. Waterloo), TÜV Holding GmbH, Software Competence Center Hagenberg GmbH, dSPACE GmbH, TRUMPF SE + Co. KG. We acknowledge EuroHPC Joint Undertaking for awarding us access to Karolina at IT4nnnovations, Czech Republic, Leonardo at CINECA, Italy, MeluXina at LuxProvide, Luxembourg, LUMI at CSC, Finland and MareNostrumS as BSC, Spain.
10

---

Published as a conference paper at ICLR 2026
REFERENCES
Francis Bach. Self-concordant analysis for logistic regression. Electronic Journal of Statistics, 4 (none):384 414, 2010.
Frank Bauer, Sergei Pereverzev, and Lorenzo Rosasco. On regularization algorithms in learning theory. Journal of Complexity, 23(1):52-72,22 2007.
Mohamed Ishmael Belghazi, Aristide Baratin, Sai Rajeshwar, Sherjil Ozair, Yoshua Bengio, Aaron Courville, and Devon Hjelm. Mutual information neural estimation. In Jennifer Dy and Andreas Krause (eds.), Proceedings the 35th International Conference on Machine Learning, volume 80 of Proceedings of Machine Learning Research, pp. 531-540. PMLR, 10-15 Jul 2018. URL wtw.anisssm.....r..e.....eeeee.e
the
mlr press Setress/v0//bee httpesss//0//eecaaaiill8110 htm
Gaspard Beugnot, Julien Mairal, and Alessandro Rudi. Beyond tikhonov: faster learning with self- concordant losses, via iterative regularization. Advances in Neural Information Processing Sys- tems, 34:28196-28207 2021.
Steffen Bickel, Michael Brückner, and Tobias Scheffer. Discriminative learning under covariate shift. Journal of Machine Learning Research, 10(9), 2009.
Lucien Birgé, An alternative point of view on lepski's method. Lecture Notes-Monograph Series, pp. 113-133, 2001.
Gilles Blanchard and Nicole Mücke. Optimal rates for regularization of statistical inverse learning problems. Foundations of Computational Mathematics, 18:971-1013, 2018.
Gilles Blanchard, Peter Mathé, and Nicole Mücke. Lepskii principle in supervised learning. arXiv preprint 1077444905.00764 2019.
John Blitzer, Ryan McDonald, and Fernando Pereira. Domain adaptation with structural correspon- dence learning. In Proceedings of the 2006 conference on empirical methods in natural language processing, pp. 128, 2006.
Leo Breiman. Bias, variance and arcing classifiers. Technical report, Statistics Department, Univer- sity of California, 1996.
Akosua Busia and Jennifer Listgarten. Mbe: model-based enrichment estimation and prediction for differential sequencing data. Genome Biology, 24(1):218, 2023.
Andrea Caponnetto and Ernesto De Vito. Optimal rates for the regularized least-squares algorithm. Foundations of Computational Mathematics. 7(3):331-368, 2007.
Ciwan Ceylan and Michael U Gutmann. Conditional noise-contrastive estimation of unnormalised models. In International Conference Machine Learning, pp. 726-734. PMLR, 2018.
Swarup Chandra, Ahsanul Haque, Hemeng Tao, Jie Liu, Latifur Khan, and Charu Aggarwal. Ensem- ble direct density ratio estimation for multistream classification. In 2018 IEEE 34th International Conference on Data Engineering (ICDE), pp. 1364-1367. IEFE, 2018
C. Chen, Z. Fu, Z. Chen, S. Jin, Z. Cheng, X. Jin, and X.-S. Hua. Homm: Higher-order moment matching for unsupervised domain adaptation. Association for the Advancement of Artificial Intelligence (AAAI), 2020.
Kristy Choi, Madeline Liao, and Stefano Ermon. Featurized density ratio estimation. In Proceedings of the Seirth- Conference on Uncertainty in Artificial Intelligence, pp. 172-182, 2021.
Ernesto De Vito, Sergei V Pereverzyev, and Lorenzo Rosasco. Adaptive kernel methods using the balancing principle. Foundations of Computational Mathematics, 10(4):455-479 2010.
Andrea Della Vecchia, Arnaud Mavakala Watusadisi, Ernesto De Vito, and Lorenzo Rosasco. Computational efficiency under covariate shift in kemel ridge regression. arXiv preprint arXiv:2505.
/4083, /408885 2025.
11

---

Published as a conference paper at ICLR 2026
M.-C. Dinu, M. Holzleimeer M. Beck, D. H. Nguyen, A. Huber, H. Eghbal-zadeh, B. A. Moser, S. V. Pereverzyev, S. Hochreiter, and W. Zellinger. Addressing parameter choice issues unsupervised domain adaptation by aggregation. International Conference on Learning Representations, 2023.
Tongtong Fang, Nan Lu, Gang Niu, and Masashi Sugiyama. Rethinking importance weighting for deep learning under distribution shift. Advances in neural information processing systems, 33: 12097,12007, 2020.
Tiago M Fragoso, Wesley Bertoli, and Francisco Louzada. Bayesian model averaging: A systematic review and conceptual classification. International Statistical Review, 86(1):1-22 2018.
Yoav Freund, Robert E Schapire, al. Experiments with new boosting algorithm. In ieml, volume 96, pp. 148-156. Citeseer, 1996.
Yaroslav Ganin, Bvgeniya Ustinova, Hana Ajakan, Paseal Germain, Hugo Larochelle, François Laviolette, Mario Marchand, and Victor Lempitsky. Domain-adversarial training of neural net- works. The journal of machine learning research, 17(1):2096-0388 2016.
Elke R Gizewski, Lukas Mayer, Bernhard A Moser, Duc Hoan Nguyen, Sergiy Pereverzyev Jr, Sergei V Pereverzyev, Natalia Shepeleva, and Werner Zellinger. On regularization of unsuper- vised domain adaptation in RKHS. Applied and Computational Harmonic Analysis, 57:201-20 -227, 2022.
Alexander Goldenshluger and Sergei Pereverzev. Adaptive estimation of linear functionals in Hilbert scales from indirect white noise observations. Probability Theory and Related Fields, 118 12):169-186, 2000.
Aditya Grover and Stefano Ermon. Boosted generative models. In Proceedings of the AAAI Confer- ence on Artificial Intelligence, volume 32, 2018.
Aditya Grover, Jiaming Song, Ashish Kapoor, Kenneth Tran, Alekh Agarwal, Eric Horvitz, and Stefano Ermon. Bias correction of learned generative models using likelihood-fre importance weighting. Advances in neural information processing systems, 32, 2019.
Lukas Gruber, Markus Holzleitner, Johannes Lehner, Sepp Hochreiter, and Werner Zellinger. Over- coming saturation in density ratio estimation by iterated regularization. In Proceedings of the 41st International Conference on Machine Learning, pp. 16502-16529, 2024.
Michael U Gutmann and Aapo Hyvärinen. Noise-contrastive estimation of unnormalized statistical models, with applications natural image statisties. Journal of machine learning research, 13 (2), 2012.
William W Hager and Hongchao Zhang, A survey of nonlinear conjugate gradient methods. Pacific journal of Optimization, 2(1):35-58, 2006.
K. He, X. Zhang, S. Ren, and J. Sun. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pultern recognition, pp. 770-778, 2016.
Shohei Hido, Yuta Tsuboi, Hisashi Kashima, Masashi Sugiyama, and Takafumi Kanamori. Statis- tical outlier detection using direct density ratio estimation. Knowledge and Information Systems, 33:309-336, 2011.
Jiayuan Huang, Arthur Gretton, Karsten Borgwardt, Bernhard Schölkopf, and Alex Smola. Correct- ing sample selection bias by unlabeled data. Advances in Neural Information Processing Systems, 19, 2006.
Mikhail Hushchyn and Andrey Ustyuzhanin. Generalization of change-point detection in time series data based on direct density ratio estimation. Journal of Computational Science, 53:101385, 2021.
Takafumi Kanamori, Shohei Hido, and Masashi Sugiyama. A least-squares approach direct im- portance estimation. The Journal of Machine Learning Research, 10: 10:1391-1445. 2009a.
Takafumi Kanamori, Taiji Suzuki, and Masashi Sugiyama. Condition number analysis of kemel- based density ratio estimation. arXiv preprint arXiv:09/2.2800 2009b.
12

---

Published as a conference paper at ICLR 2026
Takafumi Kanamori, Taiji Suzuki, and Masashi Sugiyama. Statistical analysis of kernel-based least- squares density-ratio estimation. Machine Learning, 86:335-367, 2012a.
Takafumi Kanamori, Taiji Suzuki, and Masashi Sugiyama. Statistical analysis of kernel-based least- squares density-ratio estimation. Machine Learning, 86(3):335- 0673--37--50 2012b.
Masahiro Kato and Takeshi Teshima. Non-negative bregman divergence minimization for deep direct density ratio estimation. In International Conference on Machine Learning, pp. 5320- 5333. PMLR, 2021.
Dongjun Kim, Yeongmin Kim, Se Jung Kwon, Wanmo Kang, and Il-Chul Moon. Refining genera- tive process with discriminator guidance in score-based diffusion models. In Andreas Krause, Emma Brunskill, Kyunghyun Cho, Barbara Engelhardt, Sivan Sabato, and Jonathan Scarlett (eds.), Proceedings of the 40th International Conference on Machine Learning, volume 202 of Proceedings of Machine Learning Research, pp. 16567-16598. PMLR, 23-29 Jul 2023. URL pettps:/prroees igg. ngs m weigggs......rer. press
html
Yeongmin Kim, Byeonghu Na, Minsang Park, JoonHo Jang, Dongjun Kim, Wanmo Kang, and chul Moon. Training unbiased diffusion models from biased dataset. The Twelfth Interna- lional Conference Learning Representations, 2024. URL http::::///pprreeeeeee n
serum?id=39cPPe iBed.
Ryuichi Kiryo, Gang Niu, Marthinus du Plessis, and Masashi Sugiyama. Positive- unlabeled learning with non-negative risk estimator. In Guyon, Von Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Garnett (eds.), Ad- vances Neural Information Processing Systems, volume 30. Curran Associates, Inc., 2017. URL htttps://proceddd http://ewweerr.eeeeeeeeeeeeeeeee File/7ce333000774277770000000000
neurips htcppp.cc///ee cc/paper fi files/paperr20117
A. Krizhevsky, I. Sutskever, and G. E. Hinton. Imagenet classification with deep convolutional neural networks. Advances in Neural Information Processing Systems, pp. 1097-1105, 2012.
OV Lepskii. On a problem adaptive estimation gaussian white noise. Theory of Probability & Its Applications, 35(3):454-466 1991.
Q Liu and H. Xue. Adversarial spectral kernel matching for unsupervised time series domain adaptation. Proceedings of the International Joint Conference on Artificial Intelligence (IJCAI), 30, 2021.
M. Long, Z. Cao, J. Wang, and M. Jordan. Conditional adversarial domain adaptation. Advances in Neural Information Processing Systems (NeurlPS), 31, 2018.
Shuai Lu, Peter Mathé, and Sergei Pereverzev. Balancing principle in supervised learning for general regularization scheme. Applied and Computational Harmonic Analysis, 48(1):123-148, 2020.
Ulysse Marteau-Ferey, Dmitrii Ostrovskii, Francis Bach, and Alessandro Rudi. Beyond least- squares: Fast rates for regularized empirical risk minimization through self-concordance. In Conference on Learning Theory, pp. 2294-2340 PMLR, 2019.
Peter Mathé. The Lepskii principle revisited. Inverse problems. 22(3):L11, 2006.
Aditya Menon and Cheng Soon Ong. Linking losses for density ratio and class- probabiittt estima- tion. In International Conference on Machine Learning, pp. 304-313, PMLR, 2016.
Shakir Mohamed and Balaji Lakshminarayanan. Learning in implicit generative models. arXiv preprint 1611v:1610.03483 2016.
Nicole Mücke. Adaptivity for regularized kernel methods by Lepskii's principle. arXiv preprint arXiv: 1804. 2018.
John Ashworth Nelder and Robert WM Wedderburn. Generalized linear models. Journal of the Royal Statistical Society Series A: Statistics in Society, 135(3):370-384, 1972.
13

---

Published as a conference paper at ICLR 2026
J. Neyman and E. S. Pearson. On the problem of the most efficient tests of statistical hypotheses. Philosophical Transactions of the Royal Society of London. Series A, Containing Papers of a Mathematical or Physical Character, 231:289- -2777337,, 1933.
DH Nguyen, Zellinger, and S Pereverzyev. On regularized Radon-Nikodym differentiation. RICAM-Reporl 2023-13, 2023.
XuanLong Nguyen, Martin Wainwright, and Michael Jordan. Estimating divergence functionals and the likelihood ratio by penalized convex risk minimization. Advances in Neural Information Processing Systems, 20, 2007.
XuanLong Nguyen, Martin J Wainwright, and Michael Jordan. Estimating divergence functionals and the likelihood ratio by convex risk minimization. IEEE Transactions on Information Theory, 56(11):5847-5811 2010.
Dmitrii M. Ostrovskii and Francis Bach. Finite-sample analysis of m-estimators using self- concordance. EJSTAT, 15:326-399 2021.
X. Peng. Q. Bai, X. Xia, Z. Huang, K. Saenko, and B. Wang. Moment matching for multi source domain adaptation. In Proceedings of the IEEE International Conference on Computer Vision, pp. 1406-1415, 2019.
Sergei Pereverzyev. An introduction artificial intelligence based on reproducing kernel Hilbert spaces. Springer Nature, 2022.
Ben Poole, Sherjil Ozair, Aaron Van Den Oord, Alex Alemi, and George Tucker. On variational bounds mutual information. In Kamalika Chaudhuri and Ruslan Salakhutdinov (eds.), Pro- ceedings of the 36th International Conference Machine Learning, volume 97 of Proceed- ings of Machine Learning Research, pp. 5171-5180. PMLR, 09-15 Jun 2019. URL https: //proceddings.me www.s.ml..prees.. / poo e html
Mohamed Ragab, Emadeldeen Eldele, Wee Ling Tan, Chuan-Sheng Foo, Zhenghua Chen, Min Wu, Chee-Keong Kwoh, and Xiaoli Li. Adatime: A benchmarking suite for domain adaptation time series data, ACM Transactions on Knowledge Discovery from Data, 17(8):1-18, 2023.
M. M. Rahman, C. Fookes, M. Baktashmotlagh, and S. Sridharan. On minimum discrepancy esti- mation for deep domain adaptation. Domain Adaptation for Visual Understanding, 2020.
Haider Raza and Spyridon Samothrakis. Bagging adversarial neural networks for domain adaptation in non-stationary eeg. In 2019 International Joint Conference on Neural Networks (IJNN), pp. 1-7. IEEE, 2019.
Benjamin Rhodes, Kai Xu, and Michael U Gutmann. Telescoping density-ratio estimation. Ad- vances neural information processing systems, 33:4905-4991 2020.
Alessandro Rudi and Lorenzo Rosasco. Generalization properties of learning with random features. Advances in Neural Information Processing Systems, 30, 2017.
Sebastian Sanokowski, Lukas Gruber, Christoph Bartmann, Sepp Hochreiter, and Sebastian Lehner. Rethinking losses for diffusion bridge samplers. arXiv preprint arXiv:2506. 10982, 2025.
Bernhard Schölkopf and Alexander Smola. Learning with kernels. Learning with kernels. MIT Press, 2002.
Ingmar Schuster, Mattes Mollenhauer, Stefan Klus, and Krikamol Muandet. Kernel conditional density operators. In International Conference on Artificial Intelligence and Statistics, pp. 993- 1004. PMLR, 2020.
Hidetoshi Shimodaira. Improving predictive inference under covariate shift by weighting the log- likelihood function. Journal of Statistical Planning and Inference, 90(2):27-2444 2000.
R. Shu, H. Bui, H. Narui, and S. Enmon. A dirt-t approach to unsupervised domain adaptation. International Conference Learning Representations (ICLR), 2018.
14

---

Published as a conference paper at ICLR 2026
Alex Smola, Le Song, and Choon Hui Teo. Relative novelty detection. In Artificial Intelligence and Statistics, pp. 236-543. PMLR, 2009.
Jiaming Song and Stefano Ermon. Understanding the limitations of variational mutual information
estimators. In International Conference on Learning Representations, 2020. URL https://
openreview nwwpwweeeeeeeee
X
Akash Srivastava, Seungwook Han, Kai Xu, Benjamin Rhodes, and Michael U. Gutmann. Esti- mating the density ratio between distributions with high discrepancy using multinomial logis- tic regression. Transactions on Machine Learning Research, 2023. ISSN 2835-8856. URL ittps://openreviews
forum ?id- jM8nzüzBör
Ingo Steinwart, Don R Hush, Clint Scovel, et al. Optimal rates for regularized least squares regres- sion. In Conference on Learning Theory, pp. 79-93, 2009.
A. Stisen, H. Blunck, S. Bhattacharya, T. S. Prentow, M. B. Kjærrgardd, A. Dey, T. Sonne, and M. M. Jensen. Smart devices are different: Assessing and mitigatingmobile sensing heterogeneities for activity recognition. In Proceedings of the 13th ACM Conference on Embedded Networked Sensor Systems, pp. 127-140, 2015.
Masashi Sugiyama and Motoaki Kawanabe. Machine learning in non-stationary environments: Introduction to covariate shift adaptation. MIT press, 2012.
Masashi Sugiyama, Matthias Krauledat, and Klaus-Robert Müller. Covariate shift adaptation by importance weighted cross validation. Journal of Machine Learning Research, 8(5), 2007.
Masashi Sugiyama, Shinichi Nakajima, Hisashi Kashima, Paul V Buenau, and Motoaki Kawanabe. Direct importance estimation with model selection and its application to covariate shift adaptation. In Advances in Neural Information Processing Systems, pp. 1433-1444 2008
Masashi Sugiyama, Taiji Suzuki, and Takafumi Kanamori. Density ratio estimation: A comprehen- sive review (statistical experiment and its related topics). STAT, 1703:10-31 01000,11200000 2010.
Masashi Sugiyama, Taiji Suzuki, and Takafumi Kanamori. Density ratio estimation in machine learning. Cambridge University Press, 2012a.
Masashi Sugiyama, Taiji Suzuki, and Takafumi Kanamori. Density-ratio matching under the breg- man divergence: a unified framework of density-ratio estimation. Annals of the Institute of Sta- ristical Mathematics, 64(5):1009-1044 2012b.
B. Sun, Feng, and K. Saenko. Correlation alignment for unsupervised domain adaptation. Domain Adaptation in Computer Vision Applications, pp. 153-171, 2017.
Owen Thomas, Ritabrata Dutta, Jukka Corander, Samuel Kaski, and Michael U. Gutmann. Likelihood inference by ratio estimation, 2020.
Zhuowen Tu. Learning generative models via discriminative approaches. 2007 IEEE Conference on Computer Vision and Puttern Recognition, pp. 1-8. IEEE, 2007.
E. Tzeng, J. Hoffman, Zhang, K. Saenko, and T. Darrell. Deep domain confusion: Maximizing for domain invariance. arXiv preprint arXiv: 14Xx:::412.3474 2014.
Pauli Virtanen, Ralf Gommers, Travis Oliphant, Matt Haberland, Tyler Reddy, David Cournapeau, Evgeni Burovski, Pearu Peterson, Warren Weckesser, Jonathan Bright, et al. Scipy 1.0: funda- mental algorithms for scientific computing in python. Nature methods, 17(3):261-272 2020.
G. Wilson, J. R. Doppa, and D. J. Cook. Multi- deep domain adaptation with weak supervision for time-serics sensor data. Special Interest Group on Knowledge Discovery and Data Mining (SIGKDD), 2020.
Wencheng Wu and David Benkeser. A density ratio super learner. arXiv preprint N41::2408004796 2024.
15

---

Published as a conference paper at ICLR 2026
Ka Yee Yeung, Roger E Bumgarner, and Adrian E Raftery. Bayesian model averaging: development of an improved multi-class, gene selection and classification tool for microarray data. Bioinfor- matics, 21(10):2394-230
2005.
Lantao Yu, Yujia Jin, and Stefano Ermon. A unified framework for multi-distribution density ratio estimation. arXiv preprint 12.44.2/, 2021.
Werner Zellinger. Binary losses for density ratio estimation. International Conference on Learning Representations, 2025.
Werner Zellinger. Thomas Grubinger, Edwin Lughofer, Thomas Natschläger, and Susanne Saminger-Platz Central moment discrepancy (emd) for domain-invarian representation learn- ing. International Conference on Learning Representations, 2017.
Werner Zellinger, Natalia Shepeleva, Marius-Cove Constantin Dinu, Hamid Eghbal-zadeh, Hoan Duc Nguyen, Bernhard Nessler, Sergei Pereverzyev, and Bernhard Moser. The balancing principle for parameter choice in iistance-regularized domain adaptation. Advances in Neural Information Processing Systems, 34, 2021.
Werner Zellinger, Stefan Kindermann, and Sergei Pereverzyev. Adaptive learning of density ratios in RKHS. Journal of Machine Learning Research, 24:1-28, 2023.,
Jie Zhang, Zhen- Yu Zhang, Peng Zhao, and Masashi Sugiyama. Adapting continuous covari- ate shift via online density ratio estimation. Advances in Neural Information Processing Systems, 16:29074-29113. 2023.
Kaiyang Zhou, Yongxin Yang, Yu Qiao, and Tao Xiang. Domain adaptive ensemble learning. IEEE Transactions on Image Processing, 30:8008-800 8018, 2021.
Xiaojin Zhu and Yimin Tan. Dragging: Density-ratio bagging. Technical report, University of Wisconsin-Madiset 2013.
16

---

Published as a conference paper at ICLR 2026
A
PROOF OF THEOREM ]
Sketch of the proof:
First we bound the excess risk R(P)) R(fx) of the empirical estimator by 11 fH (( using results from Marteau-Ferey et al. (2019).
To obtain bounds for minj
fn H., (fx) we construct an admissible sequence of HA,
parameters
Ax A=) and use some techniques developed for analyzing Lepskii's balancing prin-
K
ciple (Lepskii, 1991, Goldenshluger & Pereverzev 2000; Birgé 2001: Mathé 2006 De Vito
al.
2010 Mücke 2018 Blanchard et al.) 2019 Zellinger et al. 2021 Pereverzyev 2022).
Finally show (12) by using several concentration bounds the Hessian-weighted norms to
conclude the proof.
To prove main theorem, will need to recall several concentration bounds weighted norms, which were to a large extend established by Marteau-Feree et al. 2019). We will need the following technical assumption:
Assumption 5 (technical assumption).
The kernel k is continuous and bounded supnex [K((2,) 1l S R.
Lel (x,y) E X y and denote (z( [C= Ve2 Tr(V2(e(()) are almost surely (wrt. p) bounded.
Then the quantities
The first estimate deals with estimates for empirical risk minimizers pa weighted norms.
Lemma (Marteau- Ferey et (2119), Theorem 38). Let Assumptions 1 be satisfied, (0, 2 and define Bi, B32 and L by
B
sup =Esupp(p)
(rx)),(fx)
B sup Tr(V2(s
L I1fxell
zESUDp(p)
Whenever O < X < min B2z (2LRlog 2 S2a (B5)a M + N is larger than
2a 2a)
with S as in Assumption 3 and
B
8 4142
1296S2
max 5184
log
A8
T2
12x142r+1/a
then minimizer pa of equation
satisfies, with probability at least
25.
R(P*) RR(H)) 1 || SH 2
||
SH
(14)
H((N)
(x(x))
5
= 2 + = S(M
< 414
log
N, A()).
(M + N)AI/a
Next similar arguments as in Zellinger al. 2023 construct admissible sequence of regularization parameters and associated regularized estimators, which we want to aggregate in the end. The same reasoning used in the well known Lepskii balancing principle (Lepskii) 1991
Goldenshluger & Pereverzev 2000 Birgél 2001 Mathe| 22066] De Vito et al. 2010 Mücke 2018 Blanchard al. 2019 Lu al. 2020 Zellinger al. 2021] Pereverzyey 2022) will give us, that the minimal (over all admissible values for X) risk difference to the target even achieves the optimal rate:
Lemma 2. Let Assumptions 7- be satisfied and let
A = >o E", [I.....K]
with & 1, Ao S K min B3 (2 -)
( (BB)
1286 2 and K
1 | Moreover, let 1j = 1296 log -(2) and X* be the solution of TISM + N, 8,
A. Then Xg,..., AR and X satisfy the assumptions of the previous Lemma and we have for k 1,... K:
17

---

Published as a conference paper at ICLR 2026
pr
FH
SMM + N,) A(Ae).
(15)
and even more:
min ISkCK
pru Ph Fu
- (5M + A(X)) = CM + N)
2razen
(16)
HN, (fx)
with probability at least 1 (4+ 2K)6, ,, for large enough sample size M N greater than
B
88444
129655
max
5184
log
(17)
Ao
Agd
L 1+2r+1/a
2ro
and C 16500gmax(12211/001
I
12965 L
Proof. It has been shown in the proof of Zellinger et al. (2023 Theorem 1) that under the given conditions, As. for = I..., K well as X are admissible according to Lemma [ which estab- lishes (14). Let us now show (T6). First note that S is decreasing and A an increasing function a minimizer of + A. Due to the structure of A, is also clear that there is an index 1 s K with Xy X I À--1 so that
with respect to X, so that
A arg min S(M + N, (, A(A) E (Ag; xe(AL...kkk)
We now distinguish two cases: Let us first consider X = >3. Then x 2 x² which on the other hand implies S(A²) = x)"(M++N) & ((6)(x-)1//ggll))
- NN g1/a5(2) (for
B = 414.2 log (3)) and A(A²) S A(X) S 5/
In the second case we assume A₃ Ay+1, which yields X, S X and therefore A(X₂) S A(EX))
A(X*)ç++22 and also S(2)) I S(x*) < e S(x*). Thus in both cases: 5(2)) + A(A)) S
(S) + A()*)), implying 16
As next step, let us state some concentration bounds weighted norms, that relate the one hand the Hessian norms Hf(f) evaluated for different f E H. where on the other hand, the connections between empirical and non-empirical Hessian norms are explored.
To do so, we will also need to introduce further notation: we write x A iff A B is positive semi-definite. Moreover, we define t(f) SUP(a.w) SUPpe( gf.
Lemma 3 Marteau-Ferey et al. 2019 Proposition 15). Let Assumptions and fú, f2 E H. Then, we have
75 be satisfied, À 2
(iif) x M(h (6(f--))) (.)))..
(18)
Lemma Marteau-Feree al. (2019 Zellinger al. (2023)) we have, with probability at least
Under the conditions of Lemmal 7
26.
v(H < l2g(2),
t(p* P) < l2g
t(fH - -P) S log(2).
(19)
Lemma Martteau- Fartey et al. (2019, Rudi & Rosasco (017)). ). Let the conditions of Lemma be
satisfied. Then, it holds, with probability least 0.
Hx(f) & (f).
(20)
If. in addition, 0 < [H( then it holds, for all
16B;
2
M
log
5
(21)
with probability at least 1 8,
3 Hx < HA(f). 2
(22)
18

---

Published as a conference paper at ICLR 2026
We will need another technical assumption in order to apply (22) to sequence of models: Assumption min.g( K3
6. For the sequence (Au)
there
exists
b*
with 6*
H H PM))
We are now in the position to formulate and prove detailed version of our finite sample bounds for aggregation of empirical DR-estimators:
Theorem 2.
Let assumptions 7 be fulfilled. Consider K >> 1 and sequence 2E of regularization
parameters and associated empirical risk minimizers fa for S k S K, defined in Lemma 2
Let moreover 8
[ 9+2K Then we have that for B of Algorithml appliid with 5x := g( g({²));
Bø [. (a(339(ffu) S C(M N)
(23)
with probability at least 1 (9 + 2K)6 for M N larger than 26 and C given by equation 25
Proof. If the inequalities that we are going to state hold with high probability (under conditions mentioned in the theorem), we will write Sp. Let us start by following Menon & Ong 2016):
K
C )) Br (r( = R
K
Br
8,g
Σ
Σ
R (fH)
Next we apply
14). combine it with 18) and 199 to obtain:
K
aiePi )) ))
k
2
-B
BF
8,9
Σ
BF((,
Sp
2
Σ
-f#
k=1
Hap (fx)
(A)
K
2
p
8
0
Ar
k=1
Hap (f20)
(B)
An application of (20) then yields
K
K
8
ak
Ni. fm
Sp 16
ak
fH
k=
Hap (P)
HAg (Pa)
(0)
K fH
16
min t
Σ
anp
Fr fH
S
16
min
(24)
kE
Hr (p:o)'
where the last inequality follows from
7 Next we use that do < d to bound-
24 by:
min
Pr..
fm
min kE(l....K)
| fm
kE(L....K) K}
Hs, (P))
kE
H
(Po)
We can further apply (22 which is valid due to Assumption 6 and moreover, we use and 19 so that we end up with:
16
min
|
fx
S, 24
min
|| - f+
ke K)
Fsn (P:0
ke kE.....,
HA, (P
(D)
| - f# 2
Sp 96
min
kg (E...K,,K
H (fx)
(8)
S,
C(M + N)
2rato 30040T
(F)
19

---

Published as a conference paper at ICLR 2026
where (F) follows from invoking equation 16 and the constant is given by
o+2ro
1296S
C 96C* 96 16560gmux(112:)1/))) 1
(25)
L
so that equation 23 is established. Note that by Lemma 5 (F) holds with probability 1 (4 2K)
and (A)-E) all hold with probability
so that 23 is valid with probability 1 - (9 + 2K)
yielding the mentioned admissible range for 8. Moreover, taking into account the requirements (17)
and 21 we obtain that M + N needs to be at least as large as
B log
8 4-414BB
1296S
16Ba
2
max
5184
Ago
L2 1
2+2r+1/a
b
log
8 8
(26)
B DERIVATION OF LOSS AND HESSIAN FOR LR
Starting with F,g of LR from Example D we have
F(x) == x log(()) - (1 + x) log(1 + x) g
e
From Theorem from Zellinger 2025 we have
g(x)
(x) + g(x)
$ (x)
Ex
1 - 1/ (xx)
1
7(1) = F
(1 - 1))
1 -
log
17
(1+ 1
log
1 7) 17 )) (1 17)
1 17
Using these terms leads after simplification to
(+b-((x) (z)) = - (e*)) (1 2b (x:)
4
+ log (1 + - -x))-x x
=
1 e
(1) Ōn
a
!1 log
[- (1 1 1-n
1 log 1+ [ -+ (1-n)
log
1
Taking the result for loss function ((y,a:) from Zellinger 2025 gives us further after simplifying
((-1,x) = >( (x)) (x))
+ -
I+ + log (oog(++=")) x Ite (-x) log + e*)
(1,xx) (1)-1 (x)
+ log(1 + e²)) + (1 - e
-
)-x) =log(1 e- (-x)=
1 cz
i+
20

---

Published as a conference paper at ICLR 2026
which is the loss function for LR in Example 2 In the next step we derive the second derivative
Ely f(x)) w.r.t f as used in Algorithm Taking LR loss we get after simplification
ef(x)
v((-1,f(z))
+ ef(x)'
e
f((()1,((x)) = (1 e/x))22
v((1,((x))) ef(x)²
ef(r)
v<)( (1 (1+eele)))
Other loss functions with respective Hessians can be derived analogously.
C DISCUSSION ABOUT MODEL AND DATA NOISE
Model output variance noise is handled by the least squares construction of the weights ā = G Consider noisy model f^x. Its self- correlorreltion GAR, i.e., the k-th diagonal element of the empirical Gram matrix, becomes large, while its aligmment with the target function, Fg = (fe- fx), remains small because noise does not correlate with the true signal. This is desirable because the optimal weights satisfy á G and, in particular, the contribution of model k is governed by the ratio Fr/GAR (in the diagonal approximation, intuitively ãu F Pr/Grsa) Hence, a large denominator and small numerator force āy. to be small, ensuring that noisy or unstable models are down-weighted in the aggregation.
Data noise appears in two places: First, in Lemma 5 the deviation between the population Hessian H and the empirical Hessian H is controlled exactly in classical uniform per-sample operator-trace bounds (e.g., matrix Bernstein), through the constant Bÿ, see Eq. (00 (2 Second, in Lemma 4
the deviation measure t(xx - fro) is controlled in the standard way used in self-concordant and kernel-learning analyses, via the bounded kernel norm R, the bounded derivative (O)], and the Hessian trace bounds B and B². Summarizing, these quantities affect the concentration bounds and therefore the multiplicative constant C' in the final bound (1)). Larger values increase the deviation terms controlling the guarantees to hold. Importantly, however, they do not influence the non-asymptotic rate in M + N, which remains unchanged.
differences and enlarge the sample size required for the
D DETAILS ON EXPERIMENTS
D.1 DETAILS ON DATASETS WITH KNOWN DENSITY RATIOS
To evaluate the accuracy of our aggregation algorithm, we build upon the methodologies proposed by Kanamori et al. (2012b); Nguyen et al. (2010). Specifically, we construct high-dimensional data with precisely known density ratios, enabling systematic analysis of the estimation accu- racy. Our approach extends and generalizes the settings used in prior studies. For instance, widely used datasets such as Ringnorm and Twonorm (Breiman] 1996) represent specific cases of Gaus- sian mixture models, where the mean and covariance parameters are set to predefined values for each mixture component. Similarly, the experiments described in [nguyen et al.) (2010) can also be framed within Gaussian mixture model structure. To introduce greater complexity, we generate distributions by randomly sampling the number, weights, and covariances of the mixture compo- nents from 50-dimensional space, which exceeds the dimensionality considered existing studies. More concretely, the means are sampled uniformly from [0, 0.5,555 while the weights of the mix- ture components are sampled from [0, 1] and subsequently normalized to sum to 1. Each distribution (source and target) is assigned a distinct Gaussian Mixture distribution. We use the following naming convention for the Gaussian miitaammmitttttttaaaae datasets. The notation "c3, d1.70" serves as an identifier for one of the 10 Gaussian-mixe miittnn-iettetetteeee based datasets used in our evaluation. Here, "c3" indicates that
21

---

Published as a conference paper at ICLR 2026
the denominator distribution of the density ratio has 3 mixture components, and "d1.70" denotes the minimum separation (1.70) between any pair of modes across both the numerator and denominator distributions. The remaining datasets follow the same naming convention, and each contains 2,500 samples.
D.2 DETAILS ON DATASETS FOR DOMAIN ADAPTATION
We use the MiniDomainNet (Peng et al. 2019 Zellinger et al. 2017 Heterogeneity Activity Recog- nition (Stisen et al. 2015), and Amazon Reviews (Blitzer et al. 2006) dataset to cover image, time series (sensory), and language modalities respectively for evaluating the performance of our al- gorithm. Each dataset consists of a number of domains which induce several domain adaptation scenarios with the goal of adapting prediction models from a source to target domain as described in equation 13
Image data for domain adaptation. The DomainNet-2019 dataset (Peng et al. 2019) includes the six distinct image domains "Real", "Clipart", "Quickdraw"", "Sketch", "Paintinn" and "Infograph". Following the approach of [Zellinger et al. (2017) we utilize a simplified version of this dataset known as MiniDomainNet. This reduced version narrows the focus to the five largest classes in the training set across all six domains. To optimize computational efficiency we use a ResNet-18 18 (He et al. 2016) trained ImageNet (Krizhevsky al. 2012). This pre-trained backbone is assumed to have already learned low-level filters effective for the "Real" image domain requiring adaptation only for the remaining five domains resulting in five domain adaptation tasks.
Time series (sensory) data for domain adaptation. The Heterogeneity Activity Recognition dataset (Stisen et al. 2015) explores the variations specific to sensors, devices, and workloads for human activity recording. It utilizes data collected from 36 different smartphones and smartwatches comprising 13 different device models from four manufacturers. Our experimental setup incorpo- rates all five domain adaptation scenarios analyzed in Ragab et al. (2023).
Text data for domain adaptation. The Amazon Reviews dataset (Blitzer et al. 2006) contains bag-of-words representations of textual reviews across four categories: books, DVDs, kitchen, and electronics with binary labels indicating the class of review. Each category represents different domain from which a total of twelve domain adaptation tasks are constructed by pairing each domain as source domain with every other domain target domain.
D.3 DETAILS ON COMPARED METHODS
To test our aggregation approach we pick four popular representatives of the large class of widely used DRE methods that can be modeled as Bregman divergences as in equation For this we use the methods presented in Example 1 KuLSIF (Kanamori et al., 2009a), Exp (Menon & Ong. 2016). SQ (Menon & Ong] 2016). and L.R (Bickel et al.) 2009) with pross-validation parameter selection procedure and Bayesian Model Averaging FFagoso et al., 2018) and Super Learner (Wu & Benkeser] 2024) baselines and compare these for each of the four DRE methods against our aggregation approach. For KuLSIF, which offers a closed-form solution, no numerical optimization is required. For the other methods, utilize the CG algorithm from Python's SciPY library (Virtanen et al. (2020), employing the Polak-Ribière line search strategy (Hager & Zhang 2006). In our domain adaptation experiments, we adhere to the evaluation protocol established by Dinu et al. ) Gruber et al. (2024). Specifically, we calculate an ensemble of various deep neural networks using the importance- weighhted functional regression approach. We use the density ratio estimates from the aggregated and -gggagggeet versions of KuLSIF, Exp. SQ, and LR as importance weights. To comprehensively assess the impact on different domain adaptation tech- niques, we generate ensemble model candidates for 11 domain adaptation methods. These include Minimum Discrepancy Estimation for Deep Domain Adaptation (MMDA) (Rahman et al. 2020). the Convolutional deep Domanin Adaptation model for Time-Series data (CoDATS) (Wilson al.) 2020). Coominn-Aveesaaan Neural Networks (DANN) (Ganin et al.) 2016). Conditional Adversar-
hyper-
ial Domain Adaptation (CDAN) (ongg al. 2018). Deep Subdomain Adaptation (DSAN) (Zhou ct al. 202T), the DIRT-T approach to Unsupervised Domain Adaptation (DIRT) (Shu et al. 2018), Adversarial Spectral Kernel Matching (AdvSKM) (Liu & Xue 2021), Higher-order Moment Match-
22

---

Published as a conference paper at ICLR 2026
ing (HoMM) (Chen et al. 2020). Deep Domain Confusion (DDC) ( Tzeng et al. 2014), Correlation Aligmment via Deep Neural Networks (Deep Coral) (Sun et al. 2017). J. and Central Moment Dis- crepancy (CMD) Zellinger al. 2017). Altogether, this benchmark involved training over 9,000 deep neural network models and evaluating the complete ensembling benchmark proposed by Dinu et al. (2023) for Amazon Reviews, MiniDomainNNe, and HHAR. The experiments these datasets were conducted to compare 16 DRE methods (with and without aggregation).
D.3.1 KNOWN DENSITY RATIOS
For each Gaussian mixture dataset 5,000 samples are drawn from the underlying distributions. The selection and evaluation of the baseline DRE methods are conducted using a standard train/validation/test split approach with split ratios 64/16/20 respectively. The corresponding ag- gregated DRE methods are trained and evaluated on the same splits while the aggregation weights are computed on the respective validation sets. For model selection, the regularization parameter À selected by validation using grid-search from the range {10 S .10*). whereas for model aggregation the models corresponding to distinct À values are aggregated. Each experiment is repeated 10 times to ensure statistical robustness. Consistent with Kanamori et al. (2012a). a Gaus- sian kernel is employed for all density ratio estimation methods, with the kernel width determined using the median heuristic (Schölkopf Smola 2002).
D.3.2 ABLATION STUDIES
Ablation I: We use deep neural network based logistic regression model trained with binary cross- entropy loss, different regularization parameters in {10-5 10-5
..... 5........, 1001 number of hidden layers in (1, 2, 4, 6, 8). different layer width settings, and learning rates in {10- 10-3 10-4, 10-b). The model is optimized by gradient descent for 125 epochs. We evaluate this model on geometrically- constructed datasets by using the exact same data splits as for the other DRE methods.
In the remaining ablation studies we follow the experimental protocol from main experiments on the geometric Gaussian mixture datasets. For Ablations 4 and we illustrate median error and 50% confidence intervals.
Crror of Individual Nodel
RULSIF CV
CV RULSIF
Agg
14-
12
10-4
Blog scale)
Dimenstons
Figure 3: Left: Value of the aggregation weights against twice the Bregman diveregence error on the geometrically constructed dataset "c3,d1.70". More accurate models get assigned higher weights. Right: Our method (dotted lines) scales favorably when increasing the dimensionality of the data. The error computed by averaging over all geometric datasets, shaded confidence intervals.
D.3.3 DOMAIN ADAPTATION
The results of the domain adaptation benchmark experiment were computed using gradient- bassdd training across a total of 9,174 models. The implementation of certain components is based on the codebase by Dinu et al. (2023); Gruber et al. (2024). The specifies of the experimental setup are as follows:
MiniDomainNet (image data): 11 methods parameters x 5 domain adaptation tasks 3 seeds 1320 trained models
23

---

Published as a conference paper at ICLR 2026
AmazonReviews (text data): 11 methods 14 parameters 12 domain adaptation tasks X 3 seeds = 5544 trained models
HHAR (sensory data): 11 methods 14 parameters x 5 domain adaptation tasks x 3 seeds 2310 trained models
Consistent with Dinu et al. (2023), we utilized 11 domain adaptation methods from the AdaTimme benchmark (Ragab al., 2023). For each method and domain adaptation modality (text, image, sensory), we evaluated all 16 density ratio estimation approaches across 22 domain adaptation sce- narios. The model implementation for domain adaptation methods and experimental configurations also followed Dinu et al. (2023). Specifically, fully connected networks were used for Amazon Reviews, while a pretrained ResNet-18 backbone was employed for MiniDomainNet. For training and selecting/aggregating the density ratio estimation methods within this pipeline we perform an additional train/val split of 80/20 20 on the datasets that are used for training the domain adaption meth- 10 as hyperparameter space and
ods. For the regularization parameter À we use {10-5 10 use the same selection/aggreeatioo procedure as for known density ratios. Following Kanamori et al. (2012a).
Gaussian kernel with kernel width set according to the median heuristic (Schölkopi & Smola (002) for all DRE methods. In the results tables, we report the classification accuracy on the respective test sets of the target distribution for all compared DRE methods.
D.4 DETAILED EXPERIMENTAL RESULTS
Geometric Figures
Baycsian Model Averaging
Super Learner
Dataset
Kil SIF
Exp
I.R
SQ
Kul
Eap
1R
SQ
<3,l1.70 70
00.00)
5.5000000.000)
9. 00.011 )
8. 8.552 0000
8.257(-0.010
6.850:0.016)
900001000011;
c2.41.12
13.001 (00006)
12.80(110001)
13 13.25511:0000 01.3
13.536 Li (.1.)121)
12. 12.907 2.1.008)
121.30/1111110
13.4022 08.255(-0.010)
c2,01.59
12.621
(0000))
16.600 +0.00+00005) 032)
12.8084-00 01:31
13. 13.050k +-0 +-0.0-4)
12.532 +00000000 +0.0000
14.507(=00.00
12.790(660112
12.961 (-60015)
cl.81.55
11.8111100 (10000)
13.562 1.0 (23)
12:011:10.000 (21)
12,0006.10..
11.80111.0.00
12.015(11..0
(2.00](-0.019)
11885| 0.010)
<2,1178
9.022/200000
17.99011000220
9.801(231.1255
2-1.00)
9.509X (132)
17.57%
9.792(=0.024)
10:3600
0-7036+0.0151
10.555:+0.00
10.7541 00.021)
10.2911-000005
00.011
10.49%
-0.019)
<3,1.57
12.010
16.9791 16.9791100012)
15.022
c2di.6l
12.2416.0
63,81.46
(000))))
12.610 12.666::000077
12.901/20.000
13.150N (000000)
11.372 =11.000
12.523
12.822
c1,81.33
9. 9.500(+0.000
00.000+00000)
(+0.00)
0.964(+0.013)
9.4088 0.0007
0.6000
00.000000000
9.661
0.0083
82.000
Avg:
(1:110(00000)
12.972(00017))
11.307(00.019)
11.783(+0.013)
11.000 0.007)
12.399 0.003)
11.320
01.058
Table 4: Mean and standard deviation (after =) of twice the Bregman divergence error the geo- metrically constructed datasets following Kanamori et al. (2012b) over ten different sample draws from P and Q
24

---

Published as a conference paper at ICLR 2026
Domain Adaptation: Amaron Reviews
Cross Validation for Binary Classifer
Aggregation
DA-Mod
Koi-Si
LR
5Q
Ka
Exp
LR
MMDA
0.786=10.1100
(-
0. 0.784)20.010)
0.Tex 0. +11_
0.794]=0.008
0. 0.782
0.750 17000=00.000
0.780:#0
006
CoDATS
0.705 00001)
0 0.201:+0.012) 0.717:-0.011:
2)
0. 0.758,+0 +00003)
0. 0.803) -0.0100
0. -0.000))
0.707:+0.00
0. 0.793:10 008
DANN
0.794
0.250:10.01 0..111
0.79X 100.010
0.804 0000:-000000
0,800 00.80)00000)
0.792|
0.790
008
CDAN
0..07)
00.010
0.753 0.787:+00000)
0.792:00.01)
0.792 0.000
0.704
0,778:10.0015
0.790 0.200;10.0.3)
8030
0.798,10:13;
0,708.10
DIRT
0.787
0.78% 00.01:
8.754/20.002)
0. 00000)
0.007)
0.001
0.794:20
008
AdVSKM
07.01-
-0.00
0.777++0.000)
0.772;+0.0038
-0.000)
-0.000
000000
0.782
006)
HoMM
0.777 0.0009
0,780'
10.000:
0.706110.001)
0.77XX 10.000)
0. 766 0..007
0.772: 00.000
00011
0.778:10
008
DEX
0.764
=0.011
=1.010 0.77
0. 7.777:00000 00.00)
0.777 =0.030)
0.782
0.789:
008
Decp
8.784
0.767
20.0110
0.78112:0011
B.77G (11
0. 78:0::200000 -0.000
0.779 00.005)
0.786
0.784:20 907)
CMD
0.700% 0.0013)
0.778) +0.011
0.780)+0.01.1
2)
0.781 (+0.011)
0. 7797--00111
0. 0.763) -0.011)
0.7900+0.015
0.787(+0 008)
Agg
0.701)
0.701:
0 00705(+0001)
0.781 (+00000))
0. 79.75 00000)
0. 0.763) 0033:-00.000 0.000)
0.789;
0.789:+0 007)
Domain Adaptation: HHAR
Cross Validation for Binary Classtler
Aggregation
DA DA-Method
Kol-
Exp
LR
5Q
Kai.Sur
Exp
LR
S)
MMDD
0.77d
U.TaG==0.100
O 0.826|=00022
0.713(=0008G)
0-786:20.000
0.769:#0 006)
CaDATT
0.723(=0.199)
0. 0,776:: 021
0.770++0.021
0.74 (+0.01
0. 0..100
0.818:-0.018
0.783;+0.01
0.767/+0 +0000)
DANN
0.097220.119
0.785(210000 024
0.756120.0211
0.757
0. 0.118
0.828
0.807 0.01)
0.750:10.010
CDAN
0.706/ 0.792(000029)
106
0.788)+0.022 (29)
0.751(+-0.0 (22)
0. 0.884)
-0.14))
0.804:+0.022 024)
0.776:+0.022
DSAN
0.754/.00 128)
0.528; 10 225
0.202:10.0.5
0.754(1:0.011 0144
0. 0.791100.155
0.572:-0.211
0.805:10 015)
0. 0.775:10 013)
DIKT
(.783(20.084)
(.750.2:1.0144 0.200:200020
0.774[
0.825[=0.028]
0.801
0.779:10 013:
MdSSKM
0.752(-0.017)
00.00:
0.740:+0.008)
0.706/+0.08
00.993-0.000
-0.019)
0.7000-00011)
0.755-+0 006)
HoMM
0.002 -0.00)
0.764)+0.021)
0.716/=0.011
0. 0.749'=0
DDC
0..017)
0.706/ 0.740:10.000)
(06)
0.001100.0118
2.496,100 19166)
0.759:10.00
0.752.10 009)
DeepCooral
0.700(=00.11)
B.T2X 0.708 =0000
0.115
0.709:=00011 0..03)
0.743:20 005)
CMD
0.67)((-01100)
0.700
0.770(+0.006)
0.7052
0.71 071141-01144
0.814)-0.017)
000000000100
0.754/+0 007)
Avg:
0:707(
00.00:000055;
0.772(+0.0.9) 9)
0.751(+0.011
0.760[-0.082)
0.707))00075)
0.785;+0.014)
0.750
Table 5: Mean and standard deviation (after =) of target classification accuracy Amazon Reviews and HHAR datasets over three different random initialization of model weights and several domain adaptation scenarios.
Domain Adaptation: MiniDomainnNet
Bayesian Model Averiging
Super Learner
DA MA-Method
Kul
Exp
Log
SQ
Kul
Exp
Log
SQ
MMDA
0.525 )0.005
0.529 -0.000
00,0000
0.512(10.010)
0.50
0.590 0:300:0.000 009)
0.529( 006)
0.516:10.006
CoDATS
0.5371 1-0009)
0.533 -0.013)
0..00)20.018)
10.01)
0.00)-00011
0.534
0.531 (100014)
0.000
DANN
0.522
0.503 0.624)=00021
0..00
0.538/
0.522
0.520( 10.011)
0.307:10.014
CDAN
8.5211=0.020)
0.531 =0.005)
8.505/
2..21(+
=0.09
DSAN
8.548i20.0008
0.532 20.033)
0.527(=0.0133
0..13)
0.540[=00010]
20.03
20.00)
=.001)=0.0011
DIRT
+0.06)
0.387(-0.1755
0.520)-0.021)
+..03)+0.021
0.516[-0.022
-099(-001155)
0.522) +0.011)
0. -..07100.019
AdvSKM
0.514(+000005) 005)
0.510
0.512(-00000)
0.000)
0.506(=0.007]
0.514 (0T)
0.513(
=0.04)
HcMM
0.520i 1.0 06-1)
0.501(=00010)
0.521;:000155
0.506(:0.013)
(0.30(00.006)
0.590;
0.522 0002(1000000 008)
0.000:100111
DDC
0.604)+0.007)
-0007
0.514[-0.010)
0..010
0.516(
00.000
0.519/-0.011
0.506/-0.000) 0.500(+0.005)
DeepCoral
0.507)10.008)
10.011)
0..00
8.513(10.010)
0.530/ -0000)
0.531
0.517/10.008 (..00)
CMD
0.500(10.004)
0.525(10.019)
0.001)
0.504(10.000
0.535
0.525
0.522 1.00015)
10.06
Aug.
0.528)+00.088
0.523(=0.015) 0.024()00227)
0.507(+00016)
0.530(-00000)
0.514(=0.027)
0.623(+0.011)
0.511(=0.003)
Table 6: Mean and standard deviation (after +) of target classification accuracy on Dini over three different random initialization of model weights and several domain adaptation scenarios.
Domain Adaplation: Amazon Reviews
Bayeslan Model Averaging
Super Learner
DA-Method
Exp
Log
Exp
Log
SQ
MMDA
B.7861100000
0.772 10.009)
0.784[20.010)
0.701(20.007)
0.775 0.0000[0.008)
(0.0000)
0.784(10.000)
10.000
CaDATS
0,786i 10.012)
0.781 0.781[10001]
0.704[10.012)
0,785 10.009
0.782; 0.800_00011] _0011]
10.0000
0.780(10.011)
0. 10.000
DANN
8.789220.00
0.781=20011)
0.792(=9.012)
20.00)
0.792[(0001]
0.782[=0.010)
0.792)
0.701;=0.000)
CDAN
07777110.00
0.775L
0.717[20.010)
0.01)
0.776
(..07(10.088)
0.004)
DSAN
1..01110.012
10011)
0..05110.015)
0.788 10.010
0.791_00111
0.781
0.799/
10.000
DIKT
0.7891
0.788)=0.012)
0.782
20.009)
(.701=000000
0.770[=0.010
0.78%(
Ac-SKM
20.00)
0.763 20000)
0.717
20.027
0.70L
0.764[20.008
0.775 2..07)
HcMM
+0.007
0.761(=0.010)
0.774(-0.011)
0.769)
+0.009)
0.778
-0.000
0.702) 009)
0.776 +0.000
0. =0.000
DDC
008)
0.7677 -0.010)
0.778(=00011)
0.782/=0.007)
=0.0077
0.705 0099
0.773 +0.01)
+0.00)
DecpComl
0.787(
0.766
=0.010
0.7011:0.0113
0.776 :00 008)
0.707;
0.782
1:0 010)
0.778:20.000
0.784)+0.020)
0.772
-0.012)
0.796(-0.012)
+0.00))
0 -0.01))
0.773(+0.011)
0.701)+
0.704)00.006)
Avg.
0.707)20.0122
0,722 20.000 0.785(200011
0.781(20.009)
0.788[=00010)
0.773(20.008)
0.785(20.010)
0.782
Table 7: Mean and standard deviation (after +)) of target classification accuracy on Amazon Reviews over three different random initialization of model weights and several domain adaptation scenarios.
25

---

Published as a conference paper at ICLR 2026
Domain Aduptation: HHAR
Bayesian Model Averaging
Super Learner
DA-Method
Kul
Exp
Log
Kul
Exp
Log
SQ
MMDA
0.701)+000044)
0.712
0.241(=0.008)
0.704()0.00)
0.696 088;
0.773| +0.000
0.747(=0.00G)
CoDATS
0.720 126)
0.790 20020)
0.70
0.722 20.03))
0.503 01111
0.779[ 20.015
0.751;=0.010]
DANN
+0.177
0.872 -0.03)
0.705:-0.020
0.761 +0.013)
0.201 (-0.15)
0. 0.900+0.017;
0.797
+0.016)
0.765
CDAN
0.793
0.724
0.758(=0.028)
0002)+00024)
0.206(-0.022)
0.735[=0.157
(0.004)
0.003
DSAN
0.755
0.545(.0
0..02)
00.0610
0.730[-0.122
0.555, 224
0.705(10.015)
100033
DIRT
0.732 +0.0811
0,502 -0000)
0.700)-000022
0.753L +0.016)
0.734(-00000
0.817|+0.00 830
+.701)+0.018)
+0001)
Ad+SKM
0.7521 10.004)
0.762/
0.786,20.011)
10,010)
0.707]000000
0.771
0.708(10.0133
00.07)
H&MM
0,759t (0.005
0.679 10.095)
0.751)_0.020)
0.7231 10.013)
0.761_000000
0.890) 086
0.755)10.0117
0.721
DDC
0.749
0.754[=0.018) 0.42(
00.11
(.47)[00.208)
[.741(+0.014)
0.703
DeepCoral
0.7011 10.113) 113)
0,768/ 20,000)
0.727110.000 0.705)
0.111(
0.779
0.730 40.01)
CMD
106)
0.770(10.016) 0.005(10.014)
0.616(001165
0.797;10.0055
0.772
1..02
Avg.
0.738(+0.077)
07701--0004)
0.772(=0.018)
0.738(+
0.740(-0.076)
0.721
0.773(+0.015)
0.743(=0.010
Table 8: Mean and standard deviation (after +) of target classification accuracy on HHAR over three different random initialization of model weights and several domain adaptation scenarios.
Cous-Validation for Binary Chasilier
Aggregationn
Somnario
KTI
Exp
LR
SQ
Kuf.STF
Exp
1.R
SQ
R
- C
10.05)
0.569(10:004) 20.06))
0.578/10.022:
0.595 1005:100.000 10.011)
10.011
10.06:000117)
0.592/20.021
R
0.877(200000)
(20000)
0.378 (10,007
0.365
0.399 3.0. 000007
0.385/2200000
0.394;20.007 0011
0.388/ 084;
R
0. 0,722) 10000000000 094)
0.729 +0.06))
0 0.725 +0 006)
+0.044
0.730 +0 00.002)
0.745 +0.006
0.72)(-0.000)
0.728(++0.0 088
(.356(100.000
(..0011
0:349.10.011)
0528:100015)
006) 0.356
0.356/
100016
0.358 000008)
0.351(1100012
0.5006
400000 20000)))0000)
0.606/20.000 -0.007)
(.611(==0000))
0.610
0.618(= 0006
Avg.
000000
0.528(100001)
0.508(10:012)
0.538/10.007) 0.08[1000000
0.009(10:011:
0.536 0060)
0.535(10:009)
Table 9: Mean and standard deviation (after +) of target classification accuracy on MiniDomainNet computed with domain adaptation method MMDA.
Cross-Validation for Binary Classitier
Aggregation
Scenario
Kol
Exp
SQ
KuI.SIF
Eup
SQ
R
C
0.004/20.0100)
0.576/211.100
0.602) 1: (OS)
0.987(=0.0008
0.601:20 005)
0.593 OU6;
R
+0.01))
0.988(0.022))
0.349)+0.035)
1.3788-0.001
0.585 +0.028
0.357
0.354i +0.008
R
0.739
10.011)
0.737 10.05
0.726 10.015
10.0010
10.017
17.001100111
0.742
R
Q
0.361
0334(-00014)
0.342; 0.855(=0.1127)
0.370 =0.000)
=0807
0.368:=0.0223
0.364; 028
R
0.019(=0.008)
06166;00008)) 008)
0.616(+0.000)
0 0.601:+0.008 005)
0.617(=0.006)
0.624(=0.008)
0.622|-0.000 002)
0.007(=000000 008)
Avg
0.536(10.012)
0.550:00.115)
0.530(10.030)
0.007:10.015:
0.542(10.010)
0.003;10.014;
00.00))00014)
0.005(10:017)
Table 10: Mean and standard deviation (after =) of target classification accuracy on MiniDomainNet computed with domain adaptation method CoDATS.
Cross- Validation
Binary Classifier
Aggregation
Scenurio
KoLSIP
Exp
SQ
KoLSIF
Exp
LR
0.501 (-0.01))
0.576(+0.0133)
0.578(+..012)
8.560[ 00.00)
-.607(-0.013)
0.683(-0.011
0.587(=0.000
0.568(-0.007)
R
0..04(
0:3721 10.0111)
0.360 10.028)
0.354| (0.024)
0.379/_0.00 0.007)
0.387 000009
0.375 10.021)
0.368 13388:10.022 022)
R
(0.00=000077)
0.716
0.692
(00000
0.724
1.727(00.000
0.702 00.06))
0.699) 029)
R
Q
0.337(200009) 009)
0.300(000000
21.0061
0,322 40.0014)
0.005)
00.055
0.348 20.009
0.326
R
06613(-0.007)
0.607( +0.0100
0.604(+0.007)
+0000))
0.619 00.005)
0.624(-0.010)
1.621(=0.00
0.633|=00000)
Avg
(-0.00))
0.622(+0.311)
0.520(+0.019
0.506(+00016)
0.536 0.007)
0. 0.526|-0.00 0.536(-0.011)
0.519/-0.015)
Table 11: Mean and standard deviation (after =) of target classification accuracy on MiniDomainNet computed with domain adaptation method DANN.
Cross- Validation Hinary Classifier
Aggregation
Scenarão
KuL.SPP
Exp
SQ
KuL
Exp
SQ
0.5022(=0.018)
0.505(+0.029)
+.204(+).022)
0.62%(=0.02))
0.004
0.616/=0.022
0.608:=0.017
0.608[=0.019)
R
0.372 0103)
03.01(20.012)
0..07(10.056)
8.524[10.63)
1.381-0.011
0.384|=0001)
0.346)
0.343).00055) 10.052
R
0.729(-0.00)
0.720 +0.0155
0.725+0.0011
0.715
0.736/-0.001
-0.04)
0.751(-0.055)
0.722|-0.006
R
Q
1..05))
0,352
10.000)
0.354/10.011)
0.351
10.014)
0.354/ 0.014)
0.369 _0.020)
0.364: 10.011
0.363;
R
S
(0.00)
+0.012))
0.611(+1.000)
0.618
0.618(=0.010)
(623)=0.0003
00.06);00006)
Avg
0.53(()0001)
0.531(10.017)
0.520(10.025
0.512(10001)
0.657[00.000
0.541
0.005/10.017)
0.526;10.0200
Table 12: Mean and standard deviation (after =) of target classification accuracy on MiniDomainNet computed with domain adaptation method CDAN.
26

---

Published as a conference paper at ICLR 2026
Crus-Valiaation for Binary Clasalier
Aggregation
Scenarto
KuL
Exp
LR
SQ
KoLSIP
Exp
LR
SQ
R
C
0..0);0.002)
0.509(+0.019)
0 00.60:
+.003)+0.015)
0.614;+0.022
0.608:+0.013)
-..07)
R
L...11)
(0.008)
0. 0.338 0.300))
0.383
10.008)
1.391/10.000
(3663100.02
0.344/10.011)
R
0.720 +
0.721/=0.005
0.72X=00000 D08)
0. 00.00
11726:=0.00 =0.004)
0.738)
007
0.729:=0.001 001)
0.725)=0.00l0 0014
0.0012
0,338 018)
0.554
0.096:10.016;
0.366.100 10.010)
0.564;10.011
100016:
0.359 :0.00:00.01)
0.3477 0..47:2000155 015)
0.05
40.001
011)
0.632
0.618:20.010) 0100)
0.612/=0.013
Avg.
0..17)
0.544: 40..33
0.536:20.00p 5.006(
000)
0.025(=0.012)
Table 13: Mean and standard deviation (after +f target classification accuracy on MiniDomainNNet computed with domain adaptation method DSAN.
of
Cruss- Validaaali for Binary Classifier
Aggregation
Secnarto
SIF
Exp
LR
5Q
KTF
Exp
1.R
SQ
R
C
0:572(+0.021) 021)
0.474[+0.11 176)
+0.00+00023)
0.535:+0.020)
0.580(-0.019)
0.478(+0.176:
0.582 1.582-00010)
0.075(=0.011
R
0:3761 100
6.236| L10 135)
0,406i 1........00 L0.0225
0.391;10.022
0.387 0.455
0.246/
10:133
0.407.10.011
0.402) 022:
0.714/=00000 (tas)
0.642 271
+0 026)
0.718 024)
0.550) 20.270
0.711;=0.020)
0.724)=0
0:336
0.248[10.066)
0.5201110.021
0.5:5;10.019:
0.340
00008)
0.336
0.325;:0.018
R
0.588)
(.387(29)
0.385/20.0199
0.501(10.019)
0.021(20.2444
0..05
0.008
Avg
0.517(+0.026)
0.380(+0.177)
0.520(+0.023)
0.300;00.031
0.523/+0.025)
0.895[+0.117)
0.626(+0.017)
0..25(+0.020)
Table 14: Mean and standard deviation (after +) of target classification accuracy on MiniDomminNNet computed with domain adaptation method DIRT.
Cross- Validation for Binary Clussiller
Asgregation
Scenario
KUI
Exp
SQ
Kul.SIF
0.608(-0.008)
0.501:+0.013)
0.562(+00000
0.635(-0.004
0.580
0.579/+0.0)
0.584:+0
0.548:+0
R-11
0.382| -00000)
0.309/10.0077
0.383/
0.3691-00022
0.381;10.008)
0.415/10
0.381;10.019)
0.384/10
0.7122 =00000
0.7055000008
0.7177=01.844
0.726:=0.013 01.8)
0.724;=0
0.728:=0
Q
0.002)
0.525/+0.0077
0.324(+0.011)
0.315
0.342 008)
0.329:
006)
0.331;:00
064)
0:352;:0 007)
R
(..84))).08))
1000000
0.5881 4:0005)
0.006)
0.587/10 003)
0.602/10.000 0000))
0.602/10 0.01
0.598/100 082)
Avg.
0.516(=0.006)
0.515/200.009
0.312(=0.002)
0.000
0.522 005)
0.529; 008)
0.521/220.0 006)
0.517/200.0 008)
Table 15: Mean and standard deviation (after =) of target classification accuracy on MiniDomainNet computed with domain adaptation method AdvSKM.
Cross- Validation
Binars Classifier
Aggregation
Scenurio
KILSIP
Exp
SQ
KoLSIF
Exp
LRR
0.500(-0.020)
0..85(+0.020)
0.507(+
(..008))
0.604(-0.018)
(.604(-0.019)
0.589(=0.013)
0.501
R
0.396 008)
0.1077 1.0 .09)
0.378 10.025)
0.565] 10.024)
[.000)_0.004]
0.419i 0.0008)
0.386/10.018
0.385; 023)
R
P
0.72x/=000002 (022)
0.726 =00000)
0.704 0.720220..010
(0000)
0.736(-0.000)
0.744[=0.008)
0.797(=0.004)
0.726 OUT)
R
Q
0,331 (200000000 0
0,333 20.013)
0.333/ 20.013)
0,318 20.012)
0.356
0.345|20.011
0.341 00641:20.000)
0.345(20.011]
R
(.088(-0.008)
0.693] +0.011)
+0.000)
00000)
0.599-000077
0.610
0.694;=0.00m -0.002)
0.586]=0.0083
Avg.
(.0.0(00000))
0.620(13.012)
0.321(10.015)
0.505 10.003)
0.544/-0.011) 0.000
0.528[10.012)
Table 16: Mean and standard deviation (after of of classification accuracy on MiniDomainNet computed with domain adaptation method HoMM.
Croes- Validation for Binary Classifier
Aggregation
Scenario
KoL.IF
Esp
LR
SQ
KoL.IFF
Exp
1.R
SQ
R
C
0.528 (27)
0.671(=0.1129)
0.503(+0.027)
0.584/=0.027)
0.5900=0.029)
0.677
0.673(=0.027)
R
0.389 011)
0.301[
13.0116
0.390 10.011;
0.0000
0.391/ 00.00
0.380
0.398:10.00 00.008:
0.388) 009
0.721(20.000) 000
2.705)210.011)
0.727(0.000) (20000)0000)
0.726
0.729
0.711[20.000)
Q
10.00))
0.323| 1..013)
0.313 0..01
0.337/
0.335(10.012)
0.336
0.331[10.000)
0.500(=0.006)
0.570, +.678(+00.000)
#0.03))
0..04
0.586[=0.004)
0.686 00))
=.604(=0.001)
Avg
0.01((00.0L)
0.517(10.013)
0.51010.012)
0.500[1000000
0.507(_0.008)
0.329[L0.012]
0.524[10.088]
0.515/10.009)
Table 17: Mean and standard deviation (after =) of target classification accuracy on MiniDomainNet computed with domain adaptation method DDC.
Cross- Validation for Hinary Classifter
Aggregation
Scenario
KaI.SIF
Esp
CR
KoL.IIF
Exp
1.R
0.5881
0.567(10.012) 0.500(100011)
0.543;10.010
0.593/10
0.578[10.015
0.578
0.548i_10.0m
R
0.009
0.3811220000 200001)
018)
0.371 020)
0.387[=0.008
0.380
0.379
R
10.011
0.7371 10.010)
0.730010.000)
0.702)10..00
0.748 01.00)
0.749/10.0100
0.737[10.001)
0.732f 003
R
Q
=0008)200011
0.340)-40.016
0.354++1.014
1.363
+0.00)
D 0.869) =0.002
0.366: 011
0.365|+0.0115
R
S
0.621(10.006) 006)
10000(10.000)
06617(100000)
0 0.604,10.006
0.637 1 003)
0 0.629:10.00 10.008
0.619 10.00)
0.620 0.000
Avg
0.528(110005) (0.012)
0.500(110011)
0.543 0.5.4.10.011
10.00(
0.536:10.007 10.007)
0.027(00.009
Table 18: Mean and standard deviation (after =) of target classification accuracy on MiniDomainNNe computed with domain adaptation method DeepCoral.
27

---

Published as a conference paper at ICLR 2026
Cross-Validation for Hinary Classiter
Aggregation
Scenriio
Kol
KoL.SIF
Esp
1.R
0.578(10.015)
0.560
0.552):0.017)
0.005:
0.585/100000 01.))
0.574;:0.0225
0.580:10.011
0.541[:00011]
R
0.91
0.361(20)0133)
0.386/ 0.3002
0..00:20.019
0.0 0.364;20.266
R
0.742210.011) 1..061
10.000
0.753/ 1.0 006)
1..01
0.732/ 0.707:10.005)
006
R
Q
0.302-1.000)
0.355(+0.022)
0. 0.362 =0 011)
0.362 #0.029
0.341
(22)
0.855)+00000 026
R
S
0.595(110.000 0000
0.001;000110)
0.507(=0.014)
0 +..015
0.5 20.000)
0..17):0.010
0.005
Avg
0.00901
0..01)
0.521(10.021)
0.536 1.538810.008) 0..00
0..00010.020
0..05)
0.521(10.018)
Table 19: Mean and standard deviation (after =) of target classification accuracy on MiniDomainNet computed with domain adaptation method CMD.
Bayesian Model Averaging
Super Learner
Scenario
KII
Esp
LR
SQ
SuI
Exp
LR
5Q
R
C
0.583(+00.008)
0.589(+0.0222) 0.015()0.015)
0.565;=0.023
0.000(=00000)
0.688(=0.014)
0.588(=000019)
0.509;=00021)
(.321(200:001
0.374(=0.008)
4.305; a.308)
0.973 0.00000
0.377( 4.370)
0011)
0.309)
R
P
0.721 (.0.003)
0.005
00.00))
10.02))
0.723t 20.004
0.733) 10.008)
0.727(100 001)
0.716;
R 2-00 Q
0.3901 1.0.
0.319) 0.011)
110.011)
0.338 10.001)
0.502 1.0000
0.348(1000013
0.351 008)
0.312,10.010
R
S
0.301(++00007)
(-0.000)0
0.600/+0.000
0.680
0. 0.5051 600000
0.602(++0.00) +00000)
0002
0.584(=0000-)
Avg.
(.005()
0.000))
0.528(+0.010)
0.612
000007
0.52M(+000066 0.500()0.000)
0.306(+0.008)
Table 20: Mean and standard deviation (after =) of target classification accuracy on MiniDomainNet computed with domain adaptation method MMDA.
Bayesian Model Averazing
Super
Scomario
SII
Exp
I.R
8Q
SII
Exp
LR
SQ
R
C
00.000
0.800(=0.008)
0.594(+0.008)
0.604[ #0:013
0.577(=00008)
0.593/ 0044)
0.576;=0.006)
R
(.355((10.000)
0.370
0.329(10.0885
0.332 20.000)
0,360[ 20.005
1.300110.0221)
0.308(10.081)
0.536;10.663)
R
P
0.741(110.013
0.735(
0.757(10.012)
0.723 10.015)
0.7451 10.016;
10.0010
0.739.10.00) 10.011)
0.727,10.0155
0.364]
0.363
0.306)10.02)
0.342
(25)
0.368/
10.012)
0.354( 020)
0.346(20.0123
(0.00((00000)
06.1((00006)
0.616[+0.000)
0.608/ +0.000)
0.622)
0.600++0.007
0.618(+0.004)
0.602[+-00004
Avg.
0.507(+00000))
0.533(-0.013
0.590(+0.018)
0.614(+0.018)
0 0.5000+
0.534(-0.013)
0.531(+0.014)
0.308(+0.016)
Table 21: Mean and standard deviation (after =) of target classification accuracy on MiniDomainNet computed with domain adaptation method CoDATS.
Bayesian Model Averaging
Super Learner
Secnario
SIF
Exp
I.R
8Q
Kul SIF
Exp
IR
SQ
R
C
0.605(200.007
0.579 0.012)
0.570
0.807(
(.575(20.011)
0.579 10000)
0.574;;00.08)
0.372 10.000)
0.389 -.000)
0,352 10.02))
0.
0.309/ 10,080
0.356(10.022)
R
0.719) (000006
0.699 0.705(
0.681
00.08 200008
0.688 0266)
0.683/10.0228
R
Q
0.372) +0.001)
0.342(-0.01)
0.597(++.013)
0.314/+0.00
0.374[ 000003
0:33((+0.013)
0.339)+00 000)
0.318(+0.011)
R
S
(.015(+000033)
0.606(00.00)
0.614[+0.006)
+0.00)
0.617( +0.005;
0.605(+0.009)
0.616[+0.001)
0.002(=00003)
Aug.
(.507(+
0.522(
0.520(+0.016)
+0.00)+0.017)
0..00(
0.622((00000)
0.520(+0.012)
(.307(+0.014)
Table 22: Mean and standard deviation (after +) of target classification accuracy on MiniDomainNet computed with domain adaptation method DANN.
Rayesian Model Averaging
Super Learner
Sccnario
SII
Exp
L.K
8Q
KaI.SIF
Exp
LR
SQ
R
C
0.596 (20)
0.596( 00007)
0.594(10.020)
0.574)10.022)
0.508( 10.022)
0.590
(.883/10:017)
10..20)
R
0.3691 10.014)
0.382 0.010)
(1.00(1
0.313 10.054)
0.3711 10.011;
0.585/10.011
0.539;10 050)
0.317
R
P
0.729) +00002)
0.712()00012)
0.725(+0.010)
0.708;=0.000)
0.731 (.0004
0.724(+0.00
005)
0.709/=0.0077
0.354[+0000
(-.519)))1))
0.354[+0.016)
0.333/+0.011
00.00)
0.354(+0.020)
0.356(+0.011)
0.337[+0.013)
S
(00000)00000))
0.608(00.00)
0.611(+0.007)
0.589;=0.006)
=0000(=0.008)
(.611(+0.011)
0.611(+00005)
0.000];000044
Avg.
(.531(+0.0100
(-0.01))))
0.524(+0.021)
=.505;=0.021)
0.533 +0.001)
0.533(=0.016)
0.524(+0.017)
0.500(=0.019]
Table 23: Mean and standard deviation (after =) of target classification accuracy on MiniNemt computed with domain adaptation method CDAN.
Bayesian Model Averaging
Super Learner
Scenario
KII
Exp
LR
SQ
KII
Exp
LR
SQ
R
C
0,657(
L.B.
0.
0.596 110.017)
0.583 10.007)
0. 0.038K 10.008:
0.601
0.588(10.013
0.3877110011)
0.368 (.668(=0000)
0.550)
0.329 (12)
0.376 2..072
(:0.0()))
0.352(1:0000 008)
0.333)20.011)
+..00)
0.723/ -0000)
0.714/+0.007 0.364[+1.016)
0.720) +0.000)
+..007
0.724/+0.006
0.722(+0.00))
0.718(-0.005)
R
Q S
(13700+000000
0.314(-0.11)
0.336 +0.015)
0.372] #0.011)
0.350
0.355/+0.011)
0.340(=000113)
R
+0.011
(.611=00013)
0.6155(+0.013)
00.00)
0.822( 20.013)
0.613(=0.011)
0.617(=00.009)
0.004(=0.011)
Avg.
0.544(#000088
0.502(=00013
0.527(40.013)
0.513;=0.013)
0.000(=0.010)
0.633(+0.013)
0.529(200009)
0.517(=0.011
Table 24: Mean and standard deviation (after +) of target classification accuracy on MiniDomainNNt computed with domain adaptation method DSAN.
28

---

Published as a conference paper at ICLR 2026
Bayesian Model Averaging
Super Learner
Scenario
KaL
Exp
SQ
KaLSIF
Exp
LR
SQ
R
C
(.305())
0.470/20.112
0.586/
0.553;20.00200
0.577 (0002)
0.478(20.174)
0.588/
0.357;=00008)
R
(..00)
0.237 -0.133)
(20.022)
0.365;10.022
0.377 (9.084
0.239 10..33)
0.389;10.0223 00.05)
R
P
0,711 (0.007)
0.542
-02200
0,713(10 120.
0.694, 025)
0.715110.0000)
10.2661))200)
0.715r 020)
0.698;10.022
0.323) +0.004
0.250X -0.53)
0..09)
0.317/=0.020
0.006
0.346/ (0.66))
0.351(+0.015)
0-521;=+0019)
(.5.1()
0.430 -.264)
0.587(40.022))
0.565 20.0000
0.5831 2..006
(.035))02248)
0.589(10:0155
0.0091
Avg.
0.513(2n.000)
0.357(00.175)
0.520(20.021)
(.000)
(..05()
(.389(20.175)
0.522()00017)
0.307;4000119
Table 25: Mean and standard deviation (after +) of target classification accuracy on MiniDDomainNet computed with domain adaptation method DIRT.
Bayesian Model Averaging
Super Learner
Seemario
SIF
Exp
I.R
SQ
Kul SIF
Exp
ER
SQ
R
--+ C
0.505 2.0. (077))
1.569/20.009)
0.502(
0.538:20.00551
0.505 200009)
(.555((20.012)
0.550(+0.0001)
0.542;
R
0,575 (10.011
0.38% -0 0.005)
0.385 )
0.372 10.026)
0.377 10.013:
0.399/10.000
0.387 019)
0.376;
0.719/
0.011))
0.712(10.009)
0,704 10
0.721 0.0008
0.710/
0.71 005)
0.00001000000
Q
0.331[+8.000)
0.327
0.524;+0.0000
0.308 +0.008)
0.333) 000005)
0.3254+0.000
0.323/+00000 004)
0.307;-0.006)
R
8
+0.00(
0. 0.585(-0.001)
0.588(+0.000)
0.684 +0.004)
0.582) +0.0044
0.584(+0.001)
(.587((00.000)
0. 0.588(+0.003)
Avg.
0.514(+0.005
0.516(-0.007)
0.512(+0.000)
0.500
0.516((0.007)
0.614(-0.007)
0.513(+00000))
0.504(-0.00)
Table 26: Mean and standard deviation (after of target classification accuracy on MiniDomainNet computed with domain adaptation method AdvSKM.
Bayesian Model Averaging
Super Learner
Scenario
KaLSIF
Exp
LR
SQ
KaLSIF
Exp
LR
C
0.504(+0.000)
15890-0.011)
+..067-0.017) 0.577(++0.018)
0.506 +0.000
0.686(+0.018)
0.520,+0.015) 0.670)+0.011)
0.000[+00000)
0.408(00.006)
0.378|+0.020
0.359:=0.023
0.3921 +0.007
0.380 +.000)
0.363;=0.021)
R
P
0.728) (01000)
0.728(=0.000
0.710;=0.008 0.726)
0.730| #0.000
0.725/20.006)
0.726 003)
0.714;=0.005)
R
2-00 Q
(10.00)
0.318;20.011) 0.303))
0,3531 20.0003
0.335/100000 (.330(10.0111
0.322;-0.009)
R
S
0.584(1.0.000
0.512
0.592 1000066)
0.077)100000)
(.586(100.000
10.00(10.011)
0.591 003)
(..00(1
Aug.
(..00(+00004)
(-631(-)
0.521(+0.013)
0.506(+0.013)
(-0000))
0.530(+0.010)
0.522(+00000)
0.510(+0.011
Table 27: Mean and standard deviation (after =) of target classification accuracy on MiniDomainNet computed with domain adaptation method HoMM.
Bayesian Model Averaging
Super Leurner
Scenario
KuLSUFF
Exp
SQ
KuLSIF
Exp
LR
R
C
0.568/110000)
0.573(_0.028)
0.583/10.0227
0.558/10.028)
0.588( 10.022)
0.575(100022)
0.582 023)
0.362
R
0.583] 10.006)
0.393| 0.013))
0.580,11.000
0.369 0.369(10.000 10.009
0.385 10.00)
0.580
0.502(110 005)
0.573,1000000
R
P
0.713)
6.720-00.000
0.715
0.701 0000
0.715(
00002)
(+0.000))
0.717
0. 0.705,+0.000 0.04)
0.333( (..33((+00002)
0.302(-0.011)
0.308/-0.011) 0.324(+0.011
0.335(=0.003))
0.326/ 0.327(+0.011)
0.312(+0.008)
(.007(+00.006)
0.670(=0.003)
+.570(+0.000)
0.560;=0.0022
0.579[=0.0077
(.682(
0.501(+0.001)
0.573(=0.001)
Avg.
0.514(10.007)
0.517
0.514(10.010
(.501(10.010)
0.516(10.009)
0.519(10.011)
(0.00)
0.305;10.009)
Table 28: Mean and standard deviation (after =) of target classification accuracy on MiniDomainNet computed with domain adaptation method DDC.
Bayesian Model Averaging
Super Learner
Scenario
KII
Exp
LR
SQ
KoI.SIF
Exp
LR
SQ
R - C
0.58a| (..020)
00.01
0.507(11..00)
1.554,10.0110
0.590)11.000
1.503)
10.007)
0.558,10.0008
R
0.365 1:0. 0100
0.581(1000113 0.000
0.370 20.016
0.367 0.0012
0.370 20.000 (06)
0.383 012)
0.374;
P
0.736
0.787(-0.008)
0.734(+0.004
0.717 00008)
(-0.000)
0.736
0.721[+0.009)
Q
+0.00(+
0.348-00013
+.320)+0.015)
0.336
0.3800
+0.000)
0.300+0.019)
0.3500(000010
0.340(=0.0123
R
S
(0.000
0.610[00.006)
0.617(+0.007)
0.500|00000
0.622 #0.004
0.618[=0.006]
0.616 0040))
0.204[=000066
Avg.
(+0.00)
00.00()00011)
0.530)+0.010)
0.513
=0.0000
(=0.011)
0.531(+0.007))
0.517(=00008)
Table 29: Mean and standard deviation (after of classification accuracy on MiniDomainNNe computed with domain adaptation method DeepCoral.
Bayesian Model Averaging
Super Learner
Scemario
KnI.IF
Exp
LR
SQ
KII
Exp
LR
5Q
R
C
0.501 (5001++000000
0..02((0.002))
0.630;=0.0113 0.002(+0.016))
0.500 #0,00)
0.502((000024)
0.05-4(+0.011)
0.543)
0.377
0.019)
0.364;=0.0aii (..81(20.03L)
0.382) 0.390(20.011)
0.305)
R
P
0.738| 0020
0..11()
0.702(10.008)
0.723..00000
0.740f 10.004
10.011)
0.741 005)
10.07)
R
Q
0.362) 1.0.00000
0.338/ - 0.028)
0.535 010.025)
13155,10.228
0.56 10.007
0.331
0.53 022)
0.319
R
S
0.3000 (00.0000000))
00000)
+0.012)
0.561(=0.002)
0 0.502( (.030)
0.5922 20.000))
0.5096200000 008)
Avg. 0.525(=0.019) (+000))
0.521(40.019)
0.504;=0.018)
0.525((0.019) =0.00(=
0.522(+0.015)
0.308(00.016)
Table 30: Mean and standard deviation (after +)) of target classification accuracy on MiniDomainnNt computed with domain adaptation method CMD.
29

---

Published as a conference paper at ICLR 2026
Cross Validation
Hinary Classierr
Aggregation
Scenario
KaLS1F
KuLSIF
Exp
0.716/ 20
0.736 (.784
0. 220001
0.799 001
0.794[=0.002
0.802: 001)
0. 0.796 0000
0.770 +0.01
0.0055
0.200--0.016
0.784 +0 016)
-0.015)
0.770:+0.010
0.773:-0
0.750)
0.791(10.029)
0.294110.00
0.806.10 017
0.797 -0.018
0.804:10
0.800:10
0.700) +0.007)
-1100
0.708)=0.000)
+3004
5000 004
0.790 -0.0044)
0.789:+0.000
0-80G;=0 001
0.701 (60000)
0
0.700:=0.0113
=:0000)
0.797 0U7
0.783[=0.010
0.800:20.00
0.80G 008
0.502 +0.010)
0.70];=0.008]
0.001-000000
0.7071
+0/07)
0.810 006
0 803) -0.007)
0,808;10.00 000)
0.812:-0
0.7001 10.005)
0.000/10.0100
0.709:
0.7051 1.0010)
0.720 1.0 014
0. 0.708|_55 -0000))
0.708/.10.011
0.718/10 907)
E
0.746 +0.003)
0.787)=00009
0.7271 #:000)
0.761 +00 002)
0.791
0.744: 009)
0. 0.781/=0 004)
E K
(LSP +5.000)
000
[18811 #:0006
0.889 2:0 00G
00.00)
0.879 20 008
0 =0 804)
K R
0,720]+0.000
00.055
0:782--0-006
0 +0.000)
0. 0.7.40 +0 003)
0 0.734-00000
0.742 006
0. 0.727)
K D
0.757) 1.00055)
0.706)20.012)
0.7000 10.010)
0.767 013)
0.001
0.766:10.01
0. 0.758 008
K K-
0.857 +00008)
0.808(000007)
0.548 +3005)
(1.00)))
0.800
0.860 +0.0
0.864;=0.000 004)
Avg.
0.780(+H0000)
0.784(+0.010) 0.771(+0.008)
0. 0.794(+0.008)
0.782(-0.008)
0.700:+00010
0.789'=0.00 006)
Table 31: Mean and standard deviation (after +) of target classification accuracy on Amazon Re- views computed with domain adaptation method MMDA.
Cross-Validatin for Hinary Classifier
Aggregation
Scenario
KII.SIF
Esp
LR
SQ
KuLSIFF
Exp
1.R
SQ
B
D
0.804/=0000 (08)
8.775(+00000)
0 0.806 +00 007)
0.706:+0.0044
0.8077 -00000)
0.787 0.787;=00.014
+000044
0.504:00.007
0..00(=000002
B
E
0: 0.785 1:0 012)
0.7651 10.000)
0. 0.782(11000 12:2::1..110
0.773:100.0 00:00
0.789:10.00 1..011)
0.777:10.007
0.777)
0.777 :10 005
0.8077 1:0
0.784 20000))
0.007 (00,0077
0.800 10 9061
0.799
30799;00.004
0.811 007)
0.806i 1:0
0.801--0.000
0.7901+0.000
+-0.005
0.817( 000075+
0.798:
-0.003:
0.806/-0
0.812;-0.001
10.011
0.787 (100115)
0.800 1000015
0.815 0..01:10.013:
10.011
0.606:10.0155
0.805
0.02))000022
()0001))
+00000)
00.00
0.830 020)
0.824[=0.013,
0.812) 0.020;0000.6
E
0.719)=00000 (25)
+0.00))
(-.20(+0.03)
0.707:
0. 0.752:=0 024)
0.716:=0.000
0.728;
0.718(
020
E D
0.7881
0.729 10.0017
0.743) 1.00077
0.743/
0.755 022)
0.707
0.748 0..00
E-+K K
0.883)
+0.00)
0.8811 (+0.000)
0.874;+11.007
0.886:-0 -0 03:0)
0.860(=00006
0.888:=0
0.877 086)
K C-+BB B
0.741(20.006)
0,730 10.007
0.70001100005
0.730; 10.006
0. 0.7.77 007)
0.735
0.757 008)
0.736 000004
K D
0.706)2000007
6.755
0.702 (0000077
0.778 2.0 0051
0.762
006
0.705 000)
0.766 002)
K
E
0,806/
0.849 10.000)
0.305110.007
0.855
11881110 006)
0.86((15 1.00055
0.873
0,862| 1.0
Avg.
0.700(2000022
0.790(20012) (27001)
0. 0.803 1.00:00.000 -0.000)
0.700
0.793(10.008 0.797:20.012)
Table 32: Mcan and standard deviation (after =) of target classification accuracy on Amazon Re- views computed with domain adaptation method CoDATS.
Cras-Valiifaion for Hinary Classiffer
Aggrepation
Scenario
KILSIF
Exp
LR
KuL
Exp
1R
SQ
0,8022 1. (00)
10.008)
0 0,802( 11.010000000
0.814/10.004 10.057)
0.812(10.007
1..00110.011)
0.818(10.000 005)
0.784 +0 012
0.743
0.763) +0.013)
0.755:+1.010
0.798:
0.749) =0.010
0.770:
0.762)
0.801::00000
0.785
0.503110.000
0.812 0..00:0.0055
0.795)10.000
0.813:10
0.818
0.804 (00)
0.788 2000022)
0.7991
0.738/21..003
0.812 0010
0.797/20.001
0.802 0031
0.805
D E
0,808+00000
2.787(+0.000
0.811
0.007
0.814;-0.004
0.801
00.00)
0.819)
D K
0.114)+00013)
20007))
0.822:=0.033
=0.11
0.822:=0 018)
0.818
E B
0:715
6,706 10
07717(1000111
0.710:10.012:
0.724:
0.722 10001
00
0.725
E++ D
0.7481220000)
20.00(20.005
0.708(2
0.751
0.708
0.702 003:
0.758 0061
0.758 083
E K 0887)(+0.01))
+0.013)
0.87((-0015)
0.571:+0.012
(..04(-0.01.))
0.877
(.844(-00015) +0.012
K B 6:7391
6,738 10.000)
10.000
0.740
10.0000000
0.749%100.
10..10..000 10.012)
0.745 1.. 0.2
0.760 003)
O.TA7(L0
001
2.770))20:221
0.752 20007)
0.7066
0.708 11.
00722220000 00.0
0.767
0.776
0.764(
0.863(+0.011)
0.653 +0.007
0.605)+0.0007
0.000
(08))
0.864: +0.006)
0.874;-0.007
0.878(
Avg
0.704(:0001)
0:770(10001)
0.703(+0.012)
0.700
0.804/-0.000 008)
0.792
00000:00012)
0.799(:0.008
Table 33: Mean and standard deviation (after =) of target classification accuracy on Amazon Re- views computed with domain adaptation method DANN.
Cross- Validation for Binary Classifler
Asgregation
Scemario
Kuf
Exp
SQ
Kol.SIF
Esp
0.802(-0.000)
0.780:+0.0101
0.800(+00011)
0.800
0.819/+0
0.793/+0 009)
0.808:+0
0.805;+0
0.779
0.709:10.007)
0.7771 1.0.07)
0.707(-00006)
0.775/10.006)
0.770'10.00
0.779
0.784/10
0.797 -0.0077
0.776;+0.017)
0.789) =0.088)
0.800:
005)
0.788:#0.00
0.794:
0.791:=0.014:
6.207(-6.008)
0.287 006)
0.706[1:0000 000007
0.771-60000
0.813 006)
0.809.11
006)
0.706 007]
0.814;10 063)
D
B
1.798|
0.790.20.004)
0.500/ 130000
0.798/20.005)
0.799
006
0.801.1000 003)
0.802:20 005)
0.804/20 082)
D
K
0.804 (8.00-60015))
0.7945+0.011
0.802 +0.066)
0.803|-0:014
0.818 01.5)
0.796:+0.014
0.806:
+0.00+ 016)
0.805:+0.00 012)
E
10.014
0.705 10.016)
0.605 0.014)
0.7 0.711 018)
0.685 01.4)
0.708: 0006:10.013 016:
012)
E D
0.724:+-0.0122
0.737 230033)
00100)
0.744:20.00 008
0.792 010)
0.744 013:
(.700X211.100)
E K
(1079(=000111
0.208:2000012
0.87b 200001)
20005
0.880'20
009)
0.877.20.013
0.885/20 013:
0.877 009
K B
0.727(-60014)
0.717:+0.006)
0.736 +0-007)
0.733
- (06)
0. 0.741:+0.00
0.752 1.. 01.3)
0. 7.700000 006)
0. 0.7:77+0 067)
0. 0.744:+0.00 084)
D
0.754| 026)
0..000
0.7651 10.000
7651
0.007)
0.758:10 006)
0.760:
009)
0. 0.770:10 005)
0.858(=0.000 010014
(0
0.8822
0.850::0
0.865/ 0.854/+11.1KG)
Arg.
(.787((000122)
0.772/+00110
0.787(
0.792 (00000
0100
0.788:+0.000
0.790;+0.010
0.792;#0.00 000)
Table 34: Mean and standard deviation (after +) of target classification accuracy on Amazon Re- views computed with domain adaptation method CDAN.
30

---

Published as a conference paper at ICLR 2026
Cross Validation for Binury Classifer
Ageregation
Scenario
KaLSIF
Exp
LR
SQ
KuLSIF
Exp
01000 20.015)
(.800 1.1001101:011
(.216200.019)
0.816:10 013)
0.795(=0.009
0.818:
0.804 007)
0.758(
0.725-0-00111
0.79%
0.711-00011)
0.794 003)
0.792 -0.014)
0.793:+0.00
+0000++000000
0.796;-0
0.201:+0.01.4)
0.807( +0.011
05003-00.03)
0.812 009)
-.0021)
0.800:+0.015
0.805:-0 01
0.000010.000
0,802
0.005)
0.813 003)
0.704)-0.002
0.810.10.00 10.00
0. 001
01500) 260000)
0200
0.8 0U6
0.791=00007)
0.805:
0.821 005)
K
0.5031 +0.02)
0.700:+0.0151 (3
0.805
0810
0.813:+0 028)
0 8090 -0.044)
0,807:+0.01
0.812:00.0
0.7191 10.021)
0.719X 0.200)1000021
-.022)_00020
0. 0.733 020
0.717|00.020
0.725 10.000..022 022)
0.724:10 018)
0.7401 10.000)
100008
0.706%
0.708(_0000)
0. 0.764 002)
0,754| 0.008
0.753: 1.0 009
0.740; 905)
E K
(Lae3)) +0.000))
01854;200.007
0.873 0007)
0.898 008)
0 86262 -..007
1.800:20.000 0088
(=0.14))
K R
0.750 (.7391000010)
0.729)+003117
0. 0..051
++000))
0 -0001)
0.759;+0.0188
0. 0.732: 0114)
K D
0.767 1.0.005)
0.770C 0.753)110:344
0.7012
0.778 005)
0. 0.704)
1.7000-10.015)
0. 0.782 010:
K
K-EE E
10.006)
0.047(100006)
0..00)
0.008()0000)
0,870:10.00 004)
0 0.000)
0.870:10.007
0. 0.870 002)
Aog.
(.700(100001)
0.778(20:312)
0.750%
0.790(20011)
10.00010.000)
0.700
0.709(10.013)
0.708 009)
Table 35: Mean and standard deviation (after +) of target classification accuracy on Amazon Re- views computed with domain adaptation method DSAN.
CrossValidation for Binary Chasiller
Aggregation
Scenario
KII
Exp
LR
SQ
KuI
Exp
1.R
SQ
B
0-D D
0,812110 10.004)
0,795 10.00))
0.811(10.000 005)
0.800,10.00
0.825 1.. 003)
0.804
10.001000003)
10.005)
0.817 000)
B
0.780 =00 (07)
20.00))
0.773) +0U003))
0.79G DOS
0.784
00.844;000077
0.786
-00008
0.784(==0.00
=01008
B
K
0.8001
006)
20.06)
0.797(10.007)
0.794;20.0005
0.818 00.0)
20.04:
0.797
0.805(:00000 002)
0,8066 1:0
0.801 210.00
0.811
0.807.10.8007
0.809
(0.)
0.812(10.008
0.809
0.812
0.8044-0.012)
=0.021)
+0.002)
0.900:+0.023
0.810 -0.000
+0.0000
0,806:
=0.000
0.6300_10.012)
0.600 10.017)
0.623110.0055
0.844; 0..17:10.016
0.826:10.017
0.652
0.829;10000
E- 20.000)
0.065(3:00019) 200001)
0.696:=0 0.000
=0.07:
0.089:
00006
F D
(.7011-0.017)
6.706[+0.016 +0.010)
0.706:+0.013 +0001)+
0.727(-0.010)
-0.015
0.715%-0.016)
0.726)-0.00
E K
0.8851100000
10.000)
0.580 10.003
10.005
0 898 1.0 00.0
10.08)
0.886: 008)
0.885 002)
K
B
0.720)=000011
#0.000
0.7:31 (+00001)
0.730 =0 0100
0.723 #0.008
0.72%
0.757 007)
K D 0.720)
+0.000)
0. 0.748; 010)
+.0000
0. 0.742 -0.011)
0.7355 00.008
0.766;
0.764[ 0.07
K F.
0.863)+0.013)
0.857(+0.000)
0.670(
0.567
1..72--0.011
0.872)
0.680:+0.000
00.06)-
Ave
0:787(+0.011)
H.776(+0.011
0 0.780 0.788(H0.012)
0. 0.797(+0.000) 000)
0.767 +0.0100
0.780:+0.013
0.794(=0.008 008)
Table 36: Mean and standard deviation (after +) of target classification accuracy on Amazon Re- views computed with domain adaptation method DIRT.
Cros-Valiffaion for Binary Classifir
Aggregation
Scenario
KuL
Exp
LR
SQ
KuLSUP
Exp
1R
SQ
B
D
0.794 (1
21700(20.008) 01.55
0.778/21.100
0,801 (10 (.0000)
00.07
0.793:20.00 006
0.784((20.0) 002)
0.780 (0.012)
0..4110.010)
0.756.10.011
0.755 10.0000.0000 0.00
0.764 1764/10.000 01000
0.750 10.010
0.761 0.011
0.762/ 007:
0.782 )11.118)
0.765=000115)
0.781(20.117)
01.06
0.784 01.6
0.778 =0.015
0.786
0.785
0.001000.010)
0.780 0.000)
0.004)
0..65:00.005
0.767:=0.004
0.795
0.795(
0.700))10.000)
0.708/20.06)
20.005
0.700,10.004
0.790: 10.004)
10.00
0.779 005
0.792,10.00
0.788)
0.771-000011
0.78%
0.779:
-0.011100011)
0.780 +0.010
0.796:=0.011
0.785|=0000
E
0.701(1000021)
0.691:10.017)
0.200:10.015 0.706.10.018)
0.721
0.709
1..0011
0.117
E D
0.722(=1.0077
0. 0.728 0.75:(201.007)
0.746 804)
0.729)
0.734;=0 20.007)
007)
0.728)
083)
R K
0.876(+0.011)
(-8.0(-0001))
0..04)-0.012)
0..00
0 0.882:-0 -0.808)
0.885
-0.009-
0.880 0.880;-0.01 -0000
0.876
+0.000 00K
K K-+B B
0.723) 1.0.007)
0.701
0.723 L0.005
0. 6.726 10.003
0. 0.726/ 10. 00.00
0.706110.000 10.003
0.729) L0 005)
0.742 00.01.0.001 001
K D K K-F F.
0.74% (1704
1.748 -0.013)
0.740
0.767:=00124 =0.013
0.704)
0.858(+0.008)
0.833 006)
0.85A( 007)
0,549: 0.0005
0 16867(000000 0000)
0.838[10.000
0.658 007)
0.858( 00.003 003
Avg.
0.773)(2110013
0.737(
0.784/=0.00P 000000
0.774(=0.008)
0.782 0.781(=0.010)
Table 37: Mean and standard deviation (after =) of target classification accuracy on Amazon Re- views computed with domain adaptation method AdvSKM.
Cross- Crnss-Validation for Binary Classifier
Aggregation
Somnario
Kal SIF
Exp
I.
5Q
KoI.SFF
Exp
1.R
SQ
B
D
0. 0:786(:10 007)
(0077(+0.008)
0.792(+0.010)
0. 1.780:10.000 005:
0,794(200000)
0.768(:0.007)
0.798/=0.018
0.794[::0.00 006:
0.7611
0.748 (100088)
0.706(1000115
0.105)
0.770
0.766(10.017)
0.756/10.018)
0.773
0..78(10.022) 0.004(10.021)
0.776,10.020
0.792/ 015)
0.781 0.761[10.020
0.167(10.017)
0.7800
8.776(=40022
0.7800
0.780/=0.800
0.796/=0.
0.792|=0.0U2
=.700:=0.008)
0.793
D E
0.7811=00000 004)
27550-0.0000
00.02(1000031
0.777:
0. 1.7900 00)0)
0.767 +0.002
0.778:+0.00
0.763
000
D K
2..00)
0.788i 10.015)
0.785
0. 0.800 0. 01:31
0.783/10.0133
0.798:10.015)
0.792
E =00012)
#0.08))
+-0.014)
0.580: +1.01.5
0.710 =0 011
0.696| =0.012
0.701:=0.014)
0.699)+0000
E D
00.011
10.000)
1.00010
0.722: 10.009
0.7.4 008)
(.724[10.005
0.734;:00010)
0.727(10.007)
E K
0.073)2000000)
4.807(200077
0.52 (10.000)
0.802; 2902112111865
0.884 00442-0.000
0 00007
0.881:20.008 0088
0.863
K B
6.722(+0.000 006)
+0.012)
0.717(++0.11
0.701:+0.013
0. 1.722 -0 -0.00%
0.709)
+0.011
0.713/-0.015)
0.724(
77.44+0.0000 009
D 0.7488L10.008
10.000)
0.752(10.005)
0.753 10.000
0. 1.760 0.. 006)
0.742 1.0.000000000 000006
0.762
009)
0,759 005)
E
20006)
0.864/22
=00000
0.860 (07)
0.867[
Avg.
(.700)100000)
0.700(100010)
0.770110011
0.770
0.780 0077
0.772(10.009)
0.700(10.011)
0.779(10.008)
Table 38: Mean and standard deviation (after +) of target classification accuracy on Amazon Re- views computed with domain adaptation method HoMM.
31

---

Published as a conference paper at ICLR 2026
Cros-Valiiaaion for Binary Classifeer
Aggrepation
Scenario
KILSIFF
LR
KoL
Exp
1R
0.790 (00)
0.775 (10000)
0,7500 100,00)
0.783:10.800
0.794 000007
0.797
0.796:10.009 0.0008)
0.789 0.04
0.763+=0.00
0.742) -0.01))
+0.016]
0.700:+0.011
0.769) +0.013:
0.763 03.
0.762) 00.022
0.783110.0011
10.02))
6.770:10.002)
0.722;10,020
0.767):0.022
0.786
0,760
8.72M)
8.780(
0.791[200000)
0.796: 6.783)
2::9::2:0000 0.709
0.793:20.008
0.790
D
E 0.7841110000
0.757(10.0007
0.278
0.7.2
0.790 1:0 001
0.776/ 10.007
0.784:100008
10.004
D
K
0.789)=00022
0.7500 +-0.01))
0.781;=1.001;
0.796 =0 018)
0.787
#0.014
0.792:=0.0135
E B
0.7051:00000 0241
06680]10.022
0:000:1000021
00.02:10.022
0.713: :0 0.0022
0.692) 10.021
0700::0.002 02.044)
0.713: 0.00
E D
0.7301220.007
(.727(200007)
0.740:
0.739%
17746:00.005
20.00
U.T577200.1117)
0.742 003
E K
0.877(+0.001)
0.802) +0.0077
0.874(+00008)
+0.0077
0.886(=0.000 009)
0.850 +0.006
0.584/+0.000 009)
0.890
+0.00+0.000 004
B
100000))0000)
0.714 10.000)
0.719: 10. 005
0.754 0..001)
0.721
10.006
0.731
00007)
0. 004
0-7611=00000)
0.7266 2000077
0.750
0.760
0.760
0.759
0.850)+0.008)
0.607(+0.000
+.600)
+0.005
0,502
0.807)+00000)
0.605/+0.000
Avg
0.780E=00011)
0.701(10.011)
0.778(10.012)
0.7:4;+0.010
0.787(+00000 009)
0.777
00002))
0.785(=0.008)
Table 39: Mcan and standard deviation (after +) of target classification accuracy on Amazon Re- views computed with domain adaptation method DDC.
CrsssValidhatin for Binary Clasileer
Aggreegttion
Scenario
Exp
LR
SQ
KuL
Exp
LR
SQ
B
D
0 03))
(2705(
0.759 +01011)
0.790;+00.M00
0.814 000000
0.812;+0.008
0.804:000011
D.801(=00001
B
E
0.775 1:0 008)
07.55100.012)
0.705110..112
0.756:
0.781 0.000
0.762::0.011
0.772::0.012)
0.765(:0000 008
0.785
017)
0.765 2000077
0.781(100015)
0.797
0.759:20.013 20.0711
00.09
0.7971
0,783 10.000))
0.790
0.789
0.794|10.000
0.604 0.00)
0.803
0.789) 00.00))
0.769 20.00))
0.779)++0:013.
0.700:
0.793
002)
0.778 +0.008
0.777:=0.011)
0.775(=0.0077
D
K
0.782))0.014)
0:001[1000116 6.774)
0.785;10.013
0. 7999: :0.012)
0.7871 10.015
0.792:
0.785(=0.000)
0,7100 (24)
0.095 10.02)
0.709)
0.706
0.719 1: ..0.022
0.704
10.022:
0.709 0..07
020)
E-+ D
=0.04)
=.724)
0.739) +00.000)
0.720
0.747 802)
0.731 +0.002
0.748;=0 808)
73738++0.00
E K
0.678(10.007)
0.663 10.007)
0.878 10.000)
0.575 100007
0.894; 10 005)
0 10.006
0.878
1.0 009)
0.866 100.00000 005
K B
0.729(200000
8,720(200077
0. 0.735 2-0 088)
0. 0.754/=000077
0.735 :00.0T7
U.T.5:=0.01S)
0.736 013)
K D 0.7664+0.015)
+0.066)
(0749(+0.0017
+0.014
0. -0.013)
0.745 +0.0115
00.04))
K
E
0..00110.010)
0,630 10.004)
0 0.85 005)
0.551100000
0. 8686611..0 00.0
0.842(10.00m 10.0033
L..005
0.863 1..0010..022 0.00
Ave.
0:784(+0.010)
0.767(+0.01))
0.781(+0.011)
0.776;+0.000
0.794(-0.008)
0.779(+0.000)
0.786(-0.011)
0.784(+0.007)
Table 40: Mean and standard deviation (after +) of target classification accuracy on Amazon Re- views computed with domain adaptation method DeepCoral.
Cross Validation for Binary Classitier
Aggregation
Scenario
KaLSIP
Exp
LR
SQ
KuLSIF
Exp
LR
SQ
D
(.788)
20000))
0.785
0.793 1092:=0.000 (07)
0.809: 1:0 002)
0. 0.796 -0.007)
0.808: 009)
B
F
0.773 +0.00 4)
0.247(+0-350)
(+0.01)
(0000)-000))
0.787 +0.011
0. -0.018)
0.736/+0.020
0. 0.76K(-0.017
B
K
0.7811 (10.
0. 0.782 1.0 1.0000
10.001)
0.783 10.09)
0.788 100 026
0.701_00000)
0.790 10.011
0.701 007)
D
8
(17710 +0.00.))
(0000000000))
1.8002/-111.000
0. 0.799 +0 010)
0 0.796-000 =0.005
0.806: 00.00:00.007
0.798 0633
D
(1716) 2000000)
(.277)231.000)
20.000)
1:011::00000 007
0. 0.796 000008
0.812:20.010)
0.800'=0 006:
D
K
0.7981 +0.01
0.786.+00.0 +0-0130
-0.015)
-0.01))
0.812 016
0 797 -0.015)
+0.00
0.804;-0.011
0.710l 1..011)
0.716:10.016)
0.72:(1.0.017
10.016)
0.732 012)
0. 0.727 00005)
0.727
0.732 013)
0.750
0.744(==0.003 006
00.02/
1.746/
0.788 -0.006)
0.752 0018
0.762 066)
20.015)
0.876
0367(=00007)
0.
0.875=0.0080
00.0012000000
00880
0.745(+0.004)
0.709)+0.0041
0.000)
07.00-00005
0.733
0.775 (.0.01))
0.720/1100000
0.7
0.020)
0.783/10.00 018)
0.732 0.030)
0.749
0.740
0.564| +0.00.0)
(.000))00000)
00000)
-000(=00004)
118688 008
(666000004)
0.870) 006
0.806 002)
Avg.
0.700(+0.003)
0.773/+0.011)
0.78%(+0.012)
0781(-0011)
0.797(+00011)
0.783(-0001)
0.790;+0.022)
0.787 0.008)
Table 41: Mean and standard deviation (after +) of target classification accuracy on Amazon Re- views computed with domain adaptation method CMD.
32

---

Published as a conference paper at ICLR 2026
Bayesian Model Averasing
Super Learner
Scenario
KIL
Exp
LR
SQ
KoL.SIF
Exp
LR
SQ
B
D
0.000)
0.783(=0.002)
(0796/+0.000)
0.794;=0.0011
0.799 #.000))
0.796(+00.00 0.788(=0.001)
002)
0. 0.752(=0.001)
B
E
0.772[ 022028)
0.76%%=0.000
0.706/
0.764:20.0131
0.777( 20.026)
0.70 0.706:00.011) 0.705(=000015) 2.730)20.011)
B
(..994
0..20-00022)
0.794(10.00 021
20.018
0.796 10.018
0.795,10.015
0.793 (20)
10.017)
10.001
1.772--0.005
0.793,10.00
10.80100000 (012)
0,803 10.002
0.708
10.001)
0.705
10.0022
D
0..01))
0.784(-0.010)
0.700)20.011
8.794
0099)
0. 0.700 000000
0.789
+0.000)
0.702(+00001)
0. +0.0088
D K
+0.003)
0.792(
+0.000)
0.750
+0.006
0.8151 +0.001)
0.803)+00.000) +0.00(7)
0.800.000006
B B
(..05)+
(.096(=0.010)
0.707 +0.01
0.717[ =0.009
(.000(000010)
0.711(+0.011)
0.700:=0.008)
E D
0.747(=+0.006)
0.737[ 0.725(=0.008)
0,720 0006)
0.745[= -0.004
0.736 +.727)
(08)
0.719;=0.005)
(.007)11
(0.07)=0.008)
0,870/
10.000)
0.855
0.8801 10.004
0.875 (.87L(20.000)
2.8061
K B
0.7.9[
0.723%_0.000
0,752 10.000)
0.726 1.0 003))
0.719 10.0022
10.0001
0.791(110 006)
0. 0.721120.0011
K K
D
(0.00)
00011)
0.756 +0.012))
0.750 27.000000000
0.76 000007
0.739%
20.000)
0.757 011)
0. 0.756,
F.
00.02(+0.010)
0.834-00.000
0.858) +000077)
0.848;+0.005)
0.901 (0.000
0.83 +0.005)
0.800(+0.000)
0.851(-0.003)
Avg.
0.780(20.000)
0.784(40.010) 0.772(00.008)
0.781;20.007)
00000(
0.778(40.008)
0.786(200009)
0.782(=0.006)
Table 42: Mean and standard deviation (after +) of target classification accuracy on Amazon Re- views computed with domain adaptation method MMDA.
Bayesian Model Averaging
Super
Scenario
SII
Exp
8Q
Kul
Exp
LR
8Q
B
D
(0800[+00000)
1.798(-0.006)
+0.007)
0.800+00.000
0.9000 +0.000)
0.800(+0.005)
1.807(+00000)
(=00000
B
E
0.785 +00000
0.780 -0.009)
0.782
0.781
=0.088)
0.778(=0.008)
0.784(=0.010)
0. 0.781[=000055
B
0.815(200.000
0.792[=0.000
0.8071
0807740.0077
0.802;
+0.005
+0007))
0.808 006)
0.804;=0.004)
10:004)
0.789 -0.00))
1..000)
0.801:
0.8001 10.0002
0.790
0.801610:000
1.790110.001)
D
0.802
1.00111
0.001(-00011)
0.000(00.015)
0,802 1.0 013)
0. 0,785(1100 11.000:
0.796
0.808:1000033
0.800;10.0111
D K
0.815| +0.004)
0.801
0.819
0.829 00.01
0.825 =0.012
0.800(=0.011
0.818:00.014)
0.815[=0.013)
E 0-+B
B
20.007)
-0000)
0.721(2000081
0.726 1.0 029
0.731 2..0055
2..00)
0.729:10.029)
0.730;20.028)
E D
0.755/110.018
0.716(-0.015)
0.7.5(10.011)
0,744 10. 0155
0.758( 10.0016:
0.736t1000017 0.728
0.739:10.013
K
00.00)
00000)
0.001(100000)
0.881
00.077
0.868
0.883 10.007)
008)
0.867;-0.006]
0.700(+0.000)
0.723(-0.0077
0.734 +0.008)
0.731
000
0. 0.750) 0.0008
(-0.00-
0.736(+000077
0. 0.782(-0.005)
K K
D 0.708)++0.010)
(-701(-
0..02)+0.007)
0.762 -0. +0.003)
0.773) =0.000
0.747
0.761 (00000)))))
0. 0.767(=0.003)
E
(.003(
(00.007
0.865(++0.000)
0.802 005)
0.870 -0.005
0.867(=000000 0.845(200006)
0 0.809(=(0.004)
Avg.
0.796(#0:012)
0.781(=0.011)
0.794(+0.012)
0.750:000099
0.000(
0.782(+0.010)
0.794(=0.011)
0. 0.792;=0.008)
Table 43: Mean and standard deviation (after +) of target classification accuracy on Amazon Re- views computed with domain adaptation method CoDATS.
Bayesian Model Averaging
Super
Scemario
SIF
Exp
I.R
8Q
Kul SIF
Exp
I.R
8Q
(.0040
00,00)00.008)
0.802(00.010)
0.804::0.0077
0,806 #0.003;
0.780(20.008)
0.804 009)
0.800;000006
B
E
0.774[10.018 11.001
0.757 -0.011)
0.763
10..013
0.759:10.0100
0.776 10.015)
0.762
0.705(10.011)
0. 0.765;10.009
B
0.796 1.0.09
0.798K-0 -0.00)
0.803
11..00000 008
0.802 10.0000
0.797 10.005
0.798 10.00)
0.802(10..0 1.0007
0. 00.0001000001
0.802( 001001
0.77X
0.709, -0.00)
0000
0.811
0.777(+0.001)
0.801(+0.007)
0.7800+0.000)
D
+0.065+
0,802
-0.005)
0.801(+00000)
0.806
0.812(-0.002
(-0.0000
0.811(+0.000)
-.813(-0.005)
D
K
+0.01)
0.706(-0.016)
0.818; 0.114++0.018)
0.9171 #0.014
+.000
0.813(+0.016)
0.816(
E B
0.716 02440
00015)
0.711 0.217(20.014)
0.722( 20.022
0.710(20.012)
0.719)=000033
0.714(=0.0111
E D
1..00))
00.05))
0.708/10.000
0.749 0055
0.752( 10.004)
10.00))
0.750r 005)
00.01)
0.878(100004)
0,869(
-0.015)
0.506100.015
0.864
0.8801 10.01.5
0.876 100011)
0.859(10.009)
K
B
+..071
0.732 00000))
20.003)
0.745 0011
0.713)
000028)
(-0.002)
0.748 003)
0. 0..43
K K
D
0.768[
+0.033
0.76XX 00.05)
0.766 +0.000)
0.768 +0.95)
0.75% +0.000
0.761
(+0.036)
0.765/+0 (37)
0.775(-0.055)
E
(.805(+000100
-0.006)
+0.007
0.864(000005)
0.960 +0.008
+0858(+0.005)
0.867(=00000))
0.809:00000-)
Avg.
(.780
0.781(-0001)
0.790,10.009) 0.703(10.012)
0. 11.0012
0.782)1100110
00.031
0.791910.009)
Table 44: Mean and standard deviation (after =) of target classification accuracy on Amazon Re- views computed with domain adaptation method DANN.
Bayesian Model Averaging
Super Learner
Scenario
KII
Exp
SQ
KaLSIF
Exp
LR
SQ
B
D
0.805(=00.000
(.780/=0.000)
0.800)+00.011
0.796;
0.906 #00004
0.788(=0.008)
0.800 (09)
0.750[=0.006)
B
E
0.784[[00.00)
0.708=00.000
0.771;
0.782( 20.005
0.700(200.015)
0.778/1000000
0.766;20.003)
B
0,801 0.00001
0.785: -0001)
0.789(10.03
0.789 10.001)
0.805/ 10.004
0,788) 10.811
0.788L10.00g
0.790
0.787000000)
0.700710.007
0.791 (01)
0.790
0.705 006)
0.700(10.0001
D
0.7066 +0.014)
0.78% -0.00)
0.800 +0.006)
0.797( 0014)
0. 0.703) 00.012
+0.004)
0.800;+0.0001
0.801(-0.003)
D
K
00.05)+0.018)
00.015-0.015)
0.006)+
0.758
-0.014
0.9177 +0.016)
0.702(-0.014)
0.801(+0.015)
0.705[=0.012]
E
B
+0.020)
(.086/-0.001
0.705(+0.016)
0.029:
0.704[= =0.020)
(.685(=0.014)
0.707(=0.011)
0.706.00.112)
B D
0.737)+0.009)
0.757) 0.707))00012)
0.732
0. 0.730| 200016
0.730)20.011)
0.739;20.011)
0. 0.728.=0.010)
K
n.885(+0.008)
0.862(-0.01
0.875(+0.013)
0.872 -0.000000000
0.882[ +0.005)
0.864(-0.011)
0.870(+0.011)
0.877[+0.008)
0.720[+0.0000
0.720(=0.006)
+0.07)
0.735 00.04)
0.723
0.73-(+0.006) =.710)=0.005)
+.703)
K D (0.42(
0.785)20.000) 0..00)=0.008)
0.708: 200007
0.744 #0.022
0.702)
0.764 000)
0.001;
K
E
(10.1[)
20.004)
0.858(10..00)
10.002)
0.865 20.002
0.647 (10.003))
1.000010.000
0.852(10.012
Avg.
0.707(20.016)
0.715(-00110)
0.787(10.010)
0.704)
0.776(10.009)
0.787(10.009)
0.784(10.007)
Table 45: Mean and standard deviation (after +) of target classification accuracy on Amazon Re- views computed with domain adaptation method CDAN.
33

---

Published as a conference paper at ICLR 2026
Bayesian Model Averasing
Super Learner
Scenario
KIL
Exp
LR
SQ
KoL.IFF
Exp
LR
SQ
B
D
0.780100.011)
0.801:=0.00% 0.003)+0.0111
0.904 20
(=0.000)
0.806(+0.010)
0.806(=00008)
B
E
0.790] 000000))
(0.05))
0.788::00.014 0.706)
0.789 200003
0.770(20.011)
0. 0.789; 0.785)=20.015)
B
0,805| 1..012)
0.7922-0.000
10.01)
0.804 40.011
0.812 10.010
0.79% (0.011)
0.806.
0.808
10.001
00000))
0,802,100000
(012)
0.8021 10.002
0.796 10.002)
0.801110.000 10000)
10.02110.001
D
28.006)
0.788(-0.000
0.0010+0.000)
0.789: =0 0606
0. 7.70M 0.003:
0. 0.789 =0.007)
0.803; 008)
0. 00.000)
D K
(0801(+0.007)
0.788(-0.013
+..0177
0.805; 0.805 -0.014
0. 0.905) +0.056)
0.768(+0.011
0.8004(+0.016)
-.800--00014
B B
(0.01)+
0.708(-00021)
0.708: 0.712)+0.0222)
0. 0.7231 =0.0200
0.713(=0.000)
(.721(+0.001)
0. 0.704(00.018)
E D
(.709)
0.730/=0.008)
40.00)
0.7477 006)
0.749| -00005
0.702)
0.746 (.09)
0.747.=00005)
10,00)
0.878 2.0000 00.05)=0.007)
0.892| 20.004
0.880) 2.564)
0.876110.005)
K
B
0.736/1180112
0,7521 0..11
0.718
0. 0.729 1000000
0.791(100011) 10.010
0.719
K K
D
0.756
10.0111
0.7611000015
(0.701)
0.762 00.013
0.758 0.000
+0.0122
0.772) 013)
0. 0.762
F
0.860 +00000
0.852
10552-00.000 00.005)
0.863) +000007
0.809(000044
0. 0.808[+-18 -0.001)
0.853 +0.000))
0.865(+0.000)
0.859:-0.002)
Avg.
0.794(20.012)
0.710(
0.793(40.013)
0.768:40.010)
0.794(40.0022
0.78((20.010)
0.702)20.012)
0.789;
Table 46: Mean and standard deviation (after +) of target classification accuracy on Amazon Re- views computed with domain adaptation method DSAN.
Bayesian Model Averaging
Super
Scenario
SII
Exp
8Q
SII
Exp
LR
SQ
B
D
0.814[+0000
00100)
1.790X-0.005
+0.972= (.811(+00005)
0.797
(.804(00009)
0.811(+0.000)
0.800(000001
B
E
0.783
0030
(00007)
0.779) 008)
0.779:
#0.0061
+0.00)
0.702)
0.781(=0.007)
0. 0.780(=0.004)
B
0.805(200.000
0.782(=0.000
0.297[40.000
0.753 -0. 0005
20.007
0.799 0.78%/20.007)
007)
0.701:000004)
10:004)
0.800/00.008)
0.8001 10.0003
0.813 1.000)
10005))
D
0.787
18.015)
1.792|00.0200
0.8021 10.02)
0.753: 10 020)
0.788/ 10.004:
0,700
10.020)
0.801 021)
0. 0.790;10.019)
D K
0.826 +1.019
0. 0.8000000010
0.825
0.824 =0.044
0.8301 #0.017:
0.822;=00017
0.820
E 0-+B
B
0.7022
11.0077
0.685(10.019)
0.679 10.0061
0.701 22.0014
1.600(10.018)
0.687(20.017)
0.678
E D
0..00010.000)
0.0155-00115
0.715(10.011)
0,707 10.00
0.731 10.0044
0.698 10.011)
0.705110.012) 0.717(10.015)
K
0.001:20:006)
0.580 0.809()00007)
0.868 (01)
20.000000000
0.870 (:0.00))
0.874(20000-)
0.744(+0.016)
0.710K-0.0100
0.7311 +0.011)
0.729
0
0.746(-10 00.033
(-0.00-
0.730(+0.0100
0. +0.860
K K E
D +..00)
0.708(-0.009)
+..08)+
0.741 +0.0077
0.742] +0.022
0.749)+0.0100 0.707)
0. 0.742[=0.006]
0.808(+#0.011)
00.00)00.007)
0.876 +..00(+000000
0.85 (0.015
0.500(20.006)
(.809(=000009)
0. 0.882(=000055
Avg.
0.783(#0.011)
0.775(=0.011)
0.788(+0.012)
0.782:00.009)
0.770(=0.010) 0.701000:010;
0.788(=0.011)
0 0.784(=0.008)
Table 47: Mean and standard deviation (after +) of target classification accuracy on Amazon Re- views computed with domain adaptation method DIRT.
Bayesian Model Averaging
Super
Scemario
SIF
Exp
I.R
8Q
SIF
Exp
IR
8Q
0.798[220.000
0.207()
0.7861
006)
0.752:=0.000
0.790 #0.0000
0.787(210.00) 0.778(20.004)
000)
0.705
B
E
0.761 (10.010)
0.756 0.0100)
10.011)
0.750:10.009)
0.759 10.009
(.0.00))
0.708(10.010)
0. 0.757:10.007
B
0.785 1..015)
1.76%-0.000
0,781 (
0. 0.785( 10.012
0.782(100013) 10.011)
0..71;
00000)
0.78%/ -0.00))
0.704)+0.006)
0000000.000
0.788 00.005
0.786/
+0.00)
0.706[+0.005)
0. +0.062)
D
0.787 +0.006)
1.753(-0.007)
+0.00)
8.765:+0.000
0.781 -0.000
+0.003)
0.770(+0.0055)
-.7061-0.0011
D K
+..000+000100
0.212(=0.010)
0.789)+0.0011
0.781
0.797 =0.000)
0.778(=0.008)
0.788(+00011)
0.784(=0.008)
E 6-BB
B
(..05)
0.206 (.701(=0.011)
0.716;
0. 0.708( 20.016
0.700(20.0115
0.705(200.018)
0.723;
E D
1.0.00)
0.707)-0.00)
00.00)
0.732'
0.740l 10.002
0,722
1000005)
0.730 006)
0..01
0.874
-0.01)
0.801110.013)
008)
0.8831 10.006
0.800 10.0000
0.876L
0.872(10.008)
K B 0..07
0.700K=0.005
0.723) 10.006)
0.722 0033
0. 0.7201 00.004
=0.005)
0.725 004)
0. 0.726:000011
K K
D +0.01)
0.731 (-0.11))
0.750)
0.741(-0.011) +0.044)
0.749| +0.015)
0.730 (0.012)
0.749)+ +0.013)
0.743(-0.010)
E
(00.0(+
0.845 -0.006)
00004) +.004(+000007)
0. 0.860(= -0.003;
0.840(+0.005)
0.853(
+0.007
0.863(=0.003)
Avg.
0.780(18:000)
00.08))
0.777(10.010)
0.772,
(1.000)
0.777(10.009) 0.761(10.008)
0.775(10.006)
Table 48: Mean and standard deviation (after +) of target classification accuracy on Amazon Re- views computed with domain adaptation method AdvSKM.
Bayesian Model Averaging
Super Learner
Scenario
KII
Exp
SQ
KILSIF
Exp
LR
SQ
B
D
0.800[200.000
0.792(+0.010) 0.205(=0.000)
0.753:=0.007
0.796 #0.002)
0.774(=0.008)
0.794[#0.000
0.787;=0.006)
B
E
(.738(20.0000
0.788=00003
0.750 (..0018)
0.756;:000115)
0.767( 20.007
0.700(10.011)
0.758(20.0118)
0.758;=0.015)
B
0,785/ 0150
0.75% (725100021
0.776 1.0 022)
0.774'
0.781 10.014)
0.780(10.022
0.778 021)
1..01.1
002)
0.776 0.002)
0.789) 10.000)
0.791 (001)
0.792) 10.004
0,778
10.8010
0.788 002)
0.790(10.000)
D
+8.000
0.767(00.005
0.772| +0.004)
0.765 (02)
0. 0.784400.002)
0.754(
0.774[+0.004)
0. 0.765(=0.001]
D
K
+0.012)
0.760x
0.786/ +1.05)+
0.787 +0.000
0.774(-0.013)
0.789)+0.0115
0.780(=0.011
E B
+0.011
-0.01)
0.603(+0.011)
0.685
0.704[ 0.704[=0.000 =0.008
(.62)(=0.012)
(.605(=0.013)
0.001(=0.0111
B D
0.738)+00.008
00.17
0.752) 10.000)
1/726:=0.000 0077
0.737 2.0005
0.718/
0.73-(20.0000
0. 0.724,:0.000
K
+0.03(+
0.867
0.872|+0.000
0.873 1022000.005
0.881 -0.003
0.863/
0.873(+0.005)
(.671(-00004)
0..20(+000088
0.701(00011)
0..07)+0.013)
0.700 =0.0100
0.721 =0.000
0.70122
00.10(=0.012)
0.700)=0.009)
K D
0.705)+00000)
00.00)
0.742: 0.752)20.000)
0.743| 00.007
0.738[20.007)
0.754
(0000
0.740;=0.006)
K
E
0.862 (100033
00.007
0.854(10..07)
00.49000.006)
0.852( 10.001
0.840(10.000)
0.853 10.00)
0.847;20.005)
Avg.
0.780(20.007)
0.701(=0.11)
0.774(10.011)
0.7009
0.762(10.008) 0.700(1
0.775(10.010)
0.771)
Table 49: Mean and standard deviation (after +) of target classification accuracy on Amazon Re- views computed with domain adaptation method HoMM.
34

---

Published as a conference paper at ICLR 2026
Bayesian Model Averasing
Super Learner
Sconario
KII
Exp
I.R
8Q
KII
Exp
ER
5Q
B
D
0.796
0.700) 00000)
00768(20.005)
0.000
2.775(20.006)
0.789; 008)
0. 0,792;
F
0.7601 +0.011
0.740--0.000 (0.05)
0.700 +0.016)
0.761(-0.013)
0.764 +00000
0.735/+0.013 +0.013)
0.750.+0.016)
0. 0.762:+0.013)
0.7900 +0.0011
0.770(-0.020)
0.779;+0.022
0.768
0.790
+0.000
00.099
0.791(+0.021)
0.708.00.018)
(00000
0.277(=00000)
0.0005
0.757
0.793]
0.773/40.000
0.792)
D
0.7801 +8.001
0.7655 -0.007)
0.778 +0.008
0.766
+0 0055
0.701 -0.0033
0.708,
+0.006
0.780,+0.007
0. +..04))
D
K
0.784 +0.025)
0.771((0.01-4
+0.00))
0.779 00012
0.7821 +0.023
0.778(+0.014)
0.792)(+0.018
0.786(=0.013)
E
B
(.097(=00.223 +0.008)
0.004: +..05)+0.0024
0.714 =0.016;
0.701 (20.002)
0.698(
0.006;=00000)
E D
0.000)
0.700)-0.001)
0.700)40.0007
0.737: 0005)
0.735 20.001
40.06))
0.739 (06)
0.738
0.878|110:007
0.87L(-0.00))
10.000
1.. .0000)
0.8811
0.873) 10.000
0.870: (07)
1.865(10.015)
K B 18.000
0.7226=0.000
0.722(10.007)
0.718 (934)
0. 0333 0.0002
0. 0.728) 10.004)
0.726(1.0 006)
0.718
K K
D 0.761[+8.011)
0.7388 00000))
0.752)+H0000
0.750
+0.006)
0.755) +0.010
0.7.9% +0.007)
0.752) 008)
0. 00.65(+
F
00.00)
0.848 -0.007)
0.850)
0.857(+0.007) +0.007)
0. 0.855 +0.004
+0.0006)
0.867(+00007))
+0800(+00003)
Avg.
0.781(+0.008)
0.778(+0.012) 0.701))
0.771(+0.009)
00000)+
(.708(+0000))
0.778(+00011)
00773(+0.008)
Table 50: Mean and standard deviation (after +) of target classification accuracy on Amazon Re- views computed with domain adaptation method DDC.
Bayesian Model Averaging
Super Learner
Scenario
SII
Exp
I.R
8Q
SII
Exp
ER
5Q
B
D
0,805 0070
0. 0.788l 00.00)
0.709)+00011 +1.001)
0.789 +0.008)
0. 0.802[ #0.006
0.767(=0.000) (0.000))
(.801(+0.010)
0. 0.787:=0.007)
8
0.778 +0.004)
0.763) +0.011)
0.761
0.777( +0.002)
0.742
0.762 +0.001)
0. 0.763.-0.80)
0.790 (..900(000112)
0.711
0.701[+0.018)
0.770==0.00
(..00))0.000)
0.711)
0.793(+0.017)
0.773(=0.013)
(.000
0.778(=0.002)
0.753 4.700
0.905 00.001
0.793 +.002)
0.087:
D
0.790 )
0.701(-0.010
0.779(10.011)
0.776 10.008)
0.792 1.00000
0.76L (0.008))
0.778(110000 0000
10.807
D
K
0.791 11.0013
(10.000))
0.788 10.015)
0.7951 10.012
0.77781100011
0.783 10.016)
0 0.768
E
B
00.022)000022)
0.202)-000222
0.709) H0.024)
0.711
0.71512200000 0000000
0,706 =0.022)
0.711(200 023)
0.718;
+0.001)
0.75% 0.717(-0.002)
-0.00
0.746( +0.0000
0.722
0.743,-0.000 0.741(+0.002)
0.882 10.004)
000008)
0.000(10.0000
0.864 0077
0.880 10.002
0.857
0.878:1000000
0.802710.007)
K
0.737 1000008
0.722(=0.007)
10.0000
043-) 0.725
0. 0.743 0.0001
0. 728) 10.006
0.784(1::0 006)
0. 0.727.20.00-1
K
D
0.763) +0.003)
00.01)
+0.017)
0.751 (00014)
0. 0.750) +0.01 1]
0.741 (-0.004)
0.751(++0.016)
0. 0.768(-0.013
K
E
+00002(000000)
0.837 (0.004)
0.855 +0.000)
+0.02
0.861 (00.005
0.807((00.00) -00000)
0.856(=0.000)
Avg.
0.787(+0.008)
0.706(-0.010
0.791(+0.011)
0.776(+0.008)
0.767(+00010) +0.00)
0.782(+0.010)
0.778(+08008)
Table 51: Mean and standard deviation (after =) of target classification accuracy on Amazon Re- views computed with domain adaptation method DeepCoral.
Bayesian Model Averaging
Super Learner
Scenario
KaL.IFP
Exp
LR
SQ
KaLSIF
Exp
LR
SQ
B
D>
0.800
0.800(+000000 +000
0.776(-0.000
0.704)+0.000)
0.791 00000+00000)
0.708 +0000)
0.776(+00000)
0.796)+00000 008)
0. (.708(+0.006)
B
E
0.780 +0.019)
1.746-0.022
0.7611+00002)
0.764/00000
0.770 +0.017
+0.019))
0.761(+0.020)
0. 0.761[=0.017]
0.748(=00.00)
0.010)
0.793
0.796 00862=0.0000
0.749 0.007)
0.780
0.796
0.206,
10.007
0.802 0.207(=00008)
0.756
0.797[
200055
0.806(1000006
0.006))
D
E
0.779| 1.0015)
0.790_
0.802(10.000
0.794
0.787 10.013:
0.792 10.008
0.801(1100000
0.753
D
K 0.796 10.028
0. 0.786( -0001)
0.804(1100015
0.808 10.012)
0. 0.795 10.025
1.791(10.013
0.8066(000014
0.000
E B
+0.012)
0.712(-0.016
0.701)++0.01))
0.726 -0.014
0. 0.725[ :000000
0.717(-0.015)
0.720;+000116
0. 0.726.=0.013)
+.746(+
0..07(00.005
0.746 0.744[+0.008)
0.747 +0.001)
0.746. (-7.000)
0.750(-0.005)
0.075(+0.020)
0.852 00.08)
0.865 +0.05(+
0 =0.0233
0.874(+0.0088 0.807(+0.007)
0.803;
K
B
0.741[20.004)
00004)
0.731(20.004
0.723 =0.0022
0.747 =0.000
0.733 0.713)
000)
0.020
K
D
0.782[ 120.02)
0.700 (0.20)
0,732 1.0
0.780[ 10.019
0,718 (0.380)
0.739 10000000
0.737:20.12T)
K
E
0..00(10..000
0.807(10.006) 0.000))
0.85%
0. 10.008
0,846 10.0000
0.809,10.000 006)
0.860[10.003)
Aug.
0.784(+0.120)
0.772(=0.012)
0.796(+0.012)
0.782(#0.009)
0.780(=0.0188
0.778(+0.011)
0.787(+0.011)
0.784(+0.00)
Table 52: Mean and standard deviation (after +) of target classification accuracy on Amazon Re- views computed with domain adaptation method CMD.
Cress- Validation
Binory Classifier
Aggregation
Scenario
Kol SIF
Exp
IR
SQ
Kol
Exp
1.8
SQ
0.746/=0.008 008)
0.701
0.716(+0.026)
+0003))
0.793.-0.022)
0.740/=0.00)
0.728(=0.010)
0.702|=0.011
6
0.897 (10.012
0.864 1:00009
0.867 10.006)
0.829| 10.008)
0.953/0.020)
0.906|10.022
0.801(20.000)
0.858'10000F
++7
0.488
10000010.0009
0.490/ 10.009
0.360] 10:005
0.552) 0.021)
0.528 00.022)
0.494: 10.003)
0.475
8
(8.2(=000122)
0846(
0.61
20002)
0.877(000011
0.883
0.877
0.859/==0.001 001)
5
0.926/-0.006)
0.4561+0.400
0.923)++0.012)
-0.05))
60.86-000060
=0.001
Avg.
0.700((0.000)
0.670((20:5585
0.826(=00023) 20708))0000)
0.711(=0.08))
0.786/=0.007)
0.703
Table 53: Mean and standard deviation (after +) of target classification accuracy on HHAR com- puted with domain adaptation method MMDA.
35

---

Published as a conference paper at ICLR 2026
Cros- Validation for Binary Classifer
Aggregation
Scenario
KoLSuP
Exp
LR
SQ
KoLSUP
Exp
LR
SQ
0 6
+0654(+).045
(660((+
0.505(+0.009)
0.658
0.674(-00012
0.648 +0.888
0.620/=0.028|
0.539 (0000))
10.0111
0.500/
0.565| 10.001)
0.962/
1.955__0.08
0.919,10.000
0.806;10.000
7
0,472(=0.005)
0.476;+0.008)
0.006
0.511(=00024
0. 0.498 =0.0012
0.461(=00.000
0,050 (30)
0.021[13.0225
0.901:10.090)
00.061)
0.002)
0.954 0.904
00.01:
0.646(-0.662)
0.9-7(+0.011)
0.947(+0.012)
+0.002)
0.682 0.530)
0.964[-0.018)
1.958[+0.00m)
0.925[+0.002)
Avg
0.723/ 130)
0.700(+0.021)
0.779(+0.02))
0.765 0.701()00011
0.0
0.703|#0.015: 0.008000018)
0.000
Table 54: Mean and standard deviation (after =) of target classification accuracy on HHAR com- puted with domain adaptation method CoDATS.
Cress- Validation for Binary Classifie
Aggregation
Secnarão
SIF.SIF
Exp
IR
SQ
KII
Esp
1.8
SQ
0 6
0.601(00.006)
+0.023+)
06.07)+
0.619(+0.013)
(.647(-0.006)
0.690(-0.010
(0.00))
0.637/-0.012
0.936/100000
0.500( 13.058)
0.506/10.010
0.005)100004
0.977
0.935|L0.000
0.906]10.004
0.001(10.002)
0.527(
10.000)
1.,00
(0
0.387/ 0.233
0.541 036
0.003:00.050
1.005/ 2.000
0.964/=0.028
0.929/20.000
=0.00
(.654-00500))
(-000+0)
0.857(+0.010)
0.819(+0004)
0.696/00.550
0.908[-0.026)
-0.09)-0.004)
-0.000-
Avg.
(20.17)))
0.785(40.124)
0.790(1110211
0.707(3000011
0.738(=0.170)
0.828(=0.015)
(.007
0.780[=0.010]
Table 55: Mean and standard deviation (after =) of target classification accuracy on HHAR com- puted with domain adaptation method DANN
Cross Validation for Binary Classifier
Aggregation
Scenario
SII.SIF
Exp
LR
KII.SIF
Esp
1.8
0.622/ 0099)
0.5077
281)
0.000/10.001
1.660-00.022
0.538[L0.248
0.685
1000088
0.636,100000
00.03)
0.000]
0.500/+0.000 -0.006)
8.870 +0.08)
0.982 -0.081)
0.940/-0.00 -0.000-
0.922/=01 =0.00000000
0.899;=0.007
0.530 061)
0.547
1:000000 061)
0.550 066)
0.502/ 0.030)
0.502 020
0.562::0
10.060
0.533;10.005
8
0.874(-0.076)
0.863) +0.059)
0,8927 (0.006)
0.820| +0.022
0.911(-0.0445
0.878 -0.0000
0.850)-0.041
5
(00000)
0,7077 110 423)
0.004/10.006)
0,016 10.000)
1.025/ 0.031)
0.749
0..02]10.001]
0.005)10.007)
Ave
0.792(-0.028)
0.706 +0.165)
0.788(+0.029)
0.751
0.884(-0.881)
0.750(-0.144)
0 0.804;-0.024)
0.776(-0.021)
Table 56: Mean and standard deviation (after +) of target classification accuracy on HHAR com- puted with domain adaptation method CDAN.
Cross- Validation Binary Classifier
Aggregation
Scenario
KIL.IF
Exp
LR
SQ
KoLSIF
Exp
1.R
SQ
0-+6
0.710(-0.070)
0.350[ +0022000 280)
0.667(+00.08) +.205(+0.048)
0.75%(
0.891(-0.257)
0.720(=0.088)
(.686[=0.028)
I -+6
8.929.10.000 000)
0:110 (29)
0,890 10.02))
0.855) 10.000)
0.461|00.396)
00.01
0.870 10.008)
2 7
0.490 O0I)
0. 0.495 2:
0.450 211001)))
1.457 (20003))
0.005(
0.538 =0.000)
0.509 200000)
0..02(20.012]
8
0.971 (-0 005)
0.826) +0.005)
0.927)+00.005
00000)
1.009-00.22
0.965[-0.025
-.907[-0.001)
0.913;-0.000
0.460( (.654(=0.566)
0.930 +0.000)
(=000))000))
0.694[-0.536)
0.508 (0.6))
0.947(=0.0011
0.926[=0.008)
Avg.
0.754(-0.128)
0.625(+0.228)
0.702(+0.011)
0.754(+0.014)
0.794(-0.138)
0.672(-0.219)
0.806(=0.013)
0.776(-0.018)
Table 57: Mean and standard deviation (after +) of target classification accuracy on HHAR com- puted with domain adaptation method DSAN.
Croes- Validation for Hinary Classifier
Aggregation
Secnario
KIL
Exp
LR
SQ
KuI.SIF
Exp
L8
SQ
0-3 6
0.003=000722) 072)
+0.06)
0.671 (00002)
0.747(=0.041
0.72%(=0.018)
0.729 08887)
0.701 027]
1.NN(230122
0.668 20004)
0.985
0.926/=0.000
0.909 20004)
0.887[=00004]
0.194)
0.582[+0.000
0.507 0.545(+0.041)
0.254 00.06)
0.620(-0.058
0.548| -0.88)
0.541;-0.026)
(0000)
0.829
00.0110
(0000)
0.887_0.024
0.856 0000 1.0011
0..02)
00004(00000)
+0.005
(.850(+00089)
#0,00)
1.026
0.980(=0.024)
-..64(=0.002)
Avg:
(.731(10:084)
0.790(10.040)
0.790(10.023)
0.753(10001)
0.774[_0.066)
0.828[10.023]
0.801(10.017)
0.779
Table 58: Mean and standard deviation (after +) of target classification accuracy on HHAR com- puted with domain adaptation method DIRT.
Cross Validation for Binury Classifier Exp
Aggregation
Scenario
KuL.S
KuL.SU
Exp
1.K
0.718[
0.711010.0100
0.705 0.000)
(10
0.766|_
0.755(L000022
0.721(10.014
0.000
0.857 (=0.000)
0.330 (0,05))
8.798[ (011)
0.899/00.022
0.894/200007
0.840
0.823;=0.010]
0.00)
0.497
1000100)
0.502 10.024)
0.364[ 10.0000
0.526 0.017)
0.5411-000 0.002
0.518
0.492:10.000 008
8
0.810( )
0.806 #0.005)
0,807 (.007+000000
0.760
40000)
0.856
-0.099)
0.840)
-0.027)
0.819 -0.01)
0.736)==00000 008
5
0.851)_0.0065
006)
10.0014)
0.578/ 10.07)
0.8400
0.924/ 0.026)
0.901
0.697|
Avg.
0007)
0.740((00.013)
0.746(+0.0188
(0708)+00088)
0.793)=0.024)
0. 0.769(=0.012) 0.790(=0.019)
0.746[=0.006)
Table 59: Mean and standard deviation (after +) of target classification accuracy on HHAR com- puted with domain adaptation method AdvSKM.
36

---

Published as a conference paper at ICLR 2026
Cross Validation Blnary Classitter
Aggregation
Scenario
Kul
Fap
1R
SQ
STF
Exp
1.
SQ
0.002
0.710) 025)
0.725(10.024)
0.007)
0.782: 000021
0.764[=0.000 007)
00.01:
0.721/:0.008
0.879(-6.011) 1)
0.850 +3.019)
+0.094)
0.815(+0.010)
0.917/
0.899 -0.011)
0.866
0.8-66/+0.009)
L0.00))
0.366| 10.029)
0..04
0.436| 10.010)
0.499 0.016)
0.515/_0.002 10.002
0.491 10.018)
1..06:10.000
8
U.SI(=00.005) (MG)
0.200 +:000))
0,822) +10.004
(2784
0.838 00006)
16.00=0.0012
0.820/=0.000)
5
0.91 005)
0.440 509)
0.800)+0.051)
0..017)
0.958-0.0020
0.496|0.336)
+0.00)
=0.0111
Aug.
0.002(230.595 0070)))00000))
0.7542+000221
0.80%( (.718(200011
0.707(=0.083)
0.708(=0.016)
0.7000=000100
Table 60: Mean and standard deviation (after +) of target classification accuracy on HHAR com- puted with domain adaptation method HoMM.
Cross Validation Bluary Classifler
Aggregation
Scenario
STF
Fxp
IR
SQ
KoI
Exp
1.8
SQ
0-> 6
(=0.00=
0.300( 286)
+1.01)
(0000))
0.705(=0.018)
0.397((00255))
0.709/20.000)
0.680 000)
1 6
0.852/-0.017 7)
0.639) +0.413)
0.835 +..03)
40/00)
0.924
0.677
0.886 -0.009)
-0.04)-0.001
2
0, 0.439(110.04)
0.3300 10.039
0 0.439 10.007)
0.001 (0.000)
0.452 10.002
0.422;
0,818 008)
0.621 133.51)
0.020:10.006)
10000)
0.858/ 0.022)
0.663
0.840:30.001
0.813:10.007 007:
0.00)(00.00))
0.200(23.000)
0.905)) (2007
0.260(=00032)
0.908(=0.019)
0.879[=00010)
Ave
0778((00:117)
0.164140.111
0.740/0.0100;
0.706(10.000
0.791]000188
0.496[-0.199)
0.750
0.709):0.009
Table 61: Mean and standard deviation (after +) of target classification accuracy on HHAR com- puted with domain adaptation method DDC.
Cross- Validation for Binury Classlier
Aggregation
Scenario
KuL.SPP
Exp
LR
SQ
KuL.SI
Exp
L.
SQ
0-+6 6
0.008
0.700[+0.035)
0.017
+0001)
0.766[-0.012
0.739|-0.000
0.733 =0.019:
0.604;:00010
6
00006(20000)
0.60 0.470(+0.011)
0.823 10000)
0.822)
0.890/_0.000
0.8422(20.001 20.00)
7
0.400=00020)
0.509(-0.019) +0000))
0.512
0.488--0.01
0.462
0.002(000000
0.810(13.000)
0.1101000011
1.005110.013)
0.852 0.009)
0.832|00.001
0.604;10.011
0.616(-0.536)
0.910(+0.020)
0.923/+0.016)
0.805)+
0.672/0..50
0.002(-0.012
0.920| 0020/=0.000)
0.911;=0.000)
Ave
(.701(:00.115)
0.700(100000)
0.75((10.015)
0.718 0.700)
0.701(=0.013)
0.700
0..08
Table 62: Mean and standard deviation (after =) of target classification accuracy on HHAR com- puted with domain adaptation method Deep Coral.
Cross Validation for Binury Classifer
Aggregation
Scenario
KuL.SI
Exp
LR
SQ
KuL
Exp
1.K
SQ
0 6
00002)
00.00 00.03)
0.005))0000)
0.730/-0.010
0.707)=00020)
00.004
0.607:20.002)
6
(=0.7(200000) 006)
0.885 20.018)
20.0)
0.859 20000)
0.947
0.904(20.001) 0.004
0.891[20.005]
7
0.5 (77)
0.5051 10 (27)
0.513/ (.800)230000)
10:026
0,370 13370/0..247 0.247)
0.542 00.003)
0.523
10.055
0.483, 0244
0.811 (=0
0.856 (.775(20008)
0.860
0.826/ 2326//00.000 006)
0..07)=0.001)
00.55)
+000111
0.001;00.015)
10.00)
0.6601-0.511)
0.980
0.945|
0077
0.001;
Avg.
(=0.17)))
0.7700(60.0033)
0.770(+10006)
(.732(
0.714/=0.144)
0.814(=0.017)
0.780[=0.010)
0.764[=0.007)
Table 63: Mean and standard deviation (after =) of target classification accuracy on HHAR com- puted with domain adaptation method CMD.
Bayesian Model Averaging
Super Learner
Scenario
KIL
Exp
LR
SQ
KaLSIF
Exp
LR
SQ
D
0.748(110.000
0.716 0.33)
0.081::000011
0.727(=0.0220
0.7156100000
(20)
0.686;
0.899 1.0.000000 002)
=0,009)
0.887
0.837,10.010
0,899
0.871 0.891(10.001)
0.835(10.000)
0.4591 1.0. 006
0.502) 1.002/000000 0.000)
0.000000.008)
0:456 10.007)
0.0001 10.005
0.611(10.001)
0.483/10 004)
0.473
3 8
0.838]
+ 008)
0 0187(-00051
0.864(+0.011)
0884(=0.0000
0.842) +0.007
0.876 +0.00)
0.866 007)
0.841[=0.001]
4
+0.002))
(-0.100)))
0.023(+0.0111
+0.044
0.9355 +00002)
0.474(+0.288)
0.921(+00008))
0.805[-0.000)
Avg.
(.781(20.004)
0.005(20.008)
0.715(10.012)
0.741:20.008)
0.784(20:004)
0.696(10.088)
0.773(10.008)
0.747)
Table 64: Mean and standard deviation (after +) of target classification accuracy on HHAR com- puted with domain adaptation method MMDA.
Rayesian Model Averaging
Super Learner
Scenario
KII
Exp
LR
SQ
KoI
Exp
LR
SQ
D 6
0.708(+00.00
0.650X=0.044
0.631(+00001)
0.506:00.331)
0.593 +0.04)
0.665(+0.034)
0.610"=0.022% (.629)(00.088)
6
0.9.1((00.001]
0.012(
0.906/
0.870:=00003))
0.942[ #0,000)
0.500[=0.000)
0.910 008)
0.008(=0.000)
(.001(1
0.476/ 2.008)
0.436; 10.00000008
0.008[
0.475 2.097))0
0.419/20.005)
0.962 1.0. 026)
0.935/ 00026)
0.904(10.028)
0.900/10.012
0.858/ 10.024)
0.938 10.001)
0.932 023)
1.900)10.015)
4 5
0.645|
10.558)
0 0.962[=0.013
0.907(110.01)
10.05
0.656) 1 10.556)
0.0500 0.008(20.002)
006)
0.919(10.001]
Avg.
0.700(=0.000) (..24
0.779)+0.020)
0.744
0.723 +0.
0.800(00.011)
0.779(+000115)
0.701[=00110
Table 65: Mean and standard deviation (after +) of target classification accuracy on HHAR com- puted with domain adaptation method CoDATS.
37

---

Published as a conference paper at ICLR 2026
Bayesian Model Averaging
Super Learner
Scenario
KaI
Exp
SQ
KaLSIF
Exp
LR
SQ
0
0.606[200.000
0.063/20.0422 0.
0.657(20.028)
0.001:20.0155
0.000)
2.673(20.001)
0.658(220 (22)
0.628(=0.019)
1 +
0.938| 1.0.02)
0.906 (00.000)
0.941 0.878;10.006)
0.508/10.0127
0.910; 003)
0.850,10.0031
0.526,, 1.0. (
0.527-00022
(.555100.044)
0.500 0.000,1..000 0.505)
0.332 100 278)
10.011)
0.596. 040)
0.500;10.050)
0.9633 +: 001)
0.941 00000)
0.003)+00013)
0.885
(US))
0.9644 00.00000000000
0.946/+0.000
0.926: 009)
0.800:=0.000)
(10.56))
0.9722 -.0004)
0.957
0.923) 20.000)
0.8531 (..561)
(.908))000000
0.005(10.004)
0.925(20.0004
Avg.
0.025) (10.77))))775)
0.795(10.020)
0.701
0. 0.7001(10.113
0.80%(20.017)
0.797(10.016)
0.765;400010)
Table 66: Mean and standard deviation (after +) of target classification accuracy on HHAR com- puted with domain adaptation method DANN.
Bayesian Model Averaging
Super Learner
Scemario
SIF
Exp
I.R
SQ
Kul SIF
Exp
ER
SQ
D 6
0.624(200.005)
0.530 -0.2000)))0 280)
0.638:2000012 0.006))0.0133
20.000
2.530))02270)
20:00)
0.636;
6
(.0.2)10.000
0.920 1000000.000 0.000)
0,905| 10
10.010)
0.941( 10.006
0.925(110.00)
0.910 000)
0.879;10.0008
0.5491
0.57M
0.552
0,510 10.054)
00.06)
0.5P0/10.050)
0.556 060)
0.501;20.050)
++8
0,8701 +8.072)
0.870-00.058
0.862 +0.036)
0.831++0.044)
0,870 0.007)
0.800++00049
0.806(+0.051)
+.8352+
5
(+0.00)
0.722(-0.422)
0.954(+0..00)
00000
0.987 +0.006
0.741 (+0.11))))
0.958(+00.000
0.003(=0.006)
Avg.
007033(+00002))
0.724(-0.166)
+.798(+0.028)
(0.02))
0.705 +0.028)
0.735(+0.157)
0.701(+0.0024
D. 0.763)+0.021)
Table 67: Mean and standard deviation (after =) of target classification accuracy on HHAR com- puted with domain adaptation method CDAN.
Bayesian Model Averaging
Super Learner
Scenario
KaLSIF
Exp
LR
SQ
KaLSIF
Exp
LR
0.721 (06606)
0.371 (-
0.705
0.6677-0.031)
0.727( +0.065)
0.383(+0.27))
0.709;+0.000
0.679(+0.090)
6
0.000(+0.004))
0.427(-0.222
+0.02)
0.863:000011
0.931 00.005
0.897(+0.018) 0.43)
0.808(=0.008]
2 7
0.498[[#00000
0.515(=0.001)
0.895
0.402;
0.502
#00000
0.555/20.008)
0.497 004)
0.461;=0.011)
3 8
0.972 (00.000)
0.943/
0.927(100.04)
2..01;
0.972( 20.000
0.982
0.901(200:001
0.901;20.009]
1+5
100.566)
0.170
0.939(10.000
0..00(10.011)
0.002 (.0.561)
0.192(11.000
0.9306
0.900)100.008
Avg.
(1775(+00127))
0.545(-0.227)
0.792(+0.014)
0.768(+00016
0.650(+0.224)
0.796(+0.013)
0.763(+0.013)
Table 68: Mean and standard deviation (after =) of target classification accuracy on HHAR com- puted with domain adaptation method DSAN.
Bayesian Model Averaging
Super Leurner
Scenario
KuLSIFF
Exp
SQ
KuLSIF
Exp
LR
D
0
(68.)68)
0.707(
0.667( 0.709(10.0441
0.095( 10.067)
0.712(10 10.03)10.033)
0.679(10.027)
I +
0.936| 1.0.0000 003)
0 0.902 0.029)
0.906
0.864
0.831 10000)
0.908[10.0115
0.902[10 001)
0.870(1000022
2 7
1966 +- 333)
0. 0.590 -0001)
3.545(+0.040)
03066-000200
0.202( (.3330
06617(+0.880)
0.543(+0.037
0.513;+0.026)
0.847(+0.000
0.855(-0.020
0.8424+00011
0.812;-0.000,
0.852[ +0.000)
0.808
0.846/+0.000
(-801(+0.002)
4-
+00000)
0.007(=0.007)
0.901(+0.007)
=.919;=0.007]
0.983]=0.000
0.565(=0.002)
0.000(+00004)
0.021(=000055
Avg.
(10.2()
0.806(20.030)
0.790(10.023)
10.031
0.731(10:051)
0.87(100080)
0.701(10:018)
0.759(10.013)
Table 69: Mean and standard deviation (after +) of target classification accuracy on HHAR com- puted with domain adaptation method DIRT.
Bayesian Model Averaging
Super Learner
Scenario
KII
Exp
LR
SQ
KoI.SIF
Exp
LR
SQ
D 6
0.720
10400)
0.735 0.008)
0.705 100.11)
1.679,10.000
0.720l 10.004
(.745(10.000)
0.703 10.015)
0.671
1 6
0.850 1:0. 068)
0.255
0.836 100 024)
0.800:00003
0 0.862(21 200002
20.015)
0.838 020)
0.801;20.010)
(00000)
0.510(-0.110)
0.5022(+0.022)
0.460
0.485 +0.000
0.634(+0.001
0.504(+0.018)
0.478(+0.008
3 8
+0.002
0.20(=00004)
00.0744
0.770
+.000)
+.80070.0077
0.811(00.000
0.779[=0.008]
4 4
(0.00)
0.81)
0.878(+0.015)
01848:0.0000
0.992 #0.000
0.800(00.00)
0.882(+0.012)
0.854(=0.002)
Avg.
0. 01702()00012)
0.706)(+00011)
0.713
+0.000
(.770(=)0005)
0.748(+00.013)
0.717;=0.007)
Table 70: Mean and standard deviation (after +) of target classification accuracy on HHAR com- puted with domain adaptation method AdvSKM.
Bayesian Model Averaging
Super Learner
Scemario
KnLSIF
Exp
LR
SQ
KII
Exp
LR
5Q
D 6
0.733(+#0.000
0.738(=0.022)
0.725(+0.022)
0.085/=0.012)
=..40(=
0.744[=0.014)
0.729(
0.005
(.000)
0.853 0.012
00.23;
20.0077
0.884[20.008)
0.851(20.018)
0.821;=000088
0.454 10.011)
0.477(
0.874(10.024)
10.011)
0.451 10.011
0.477(10.019) 0,192/
0.442;10.009
3 ++8
0.817|
1000011
0.807(00.00))
0.822) (10.000)
0.792 10.012)
10:000
0.847
0.820) 1.00:0000000 001)
0.75 0.763,10.000)
4 +
0.9100
0. 0.467(00.005)
0.500)20.0030
0.862
0200022
0.451 (+60.388
0.807 (27)
0.869(=0.016)
Avg.
(..00)
0.695)
0.754(+0.020)
0.723;=0.013)
0.600(00.086)
0.766(+0.017)
0.721(=0.010)
Table 71: Mean and standard deviation (after +) of target classification accuracy on HHAR com- puted with domain adaptation method HOMM.
38

---

Published as a conference paper at ICLR 2026
Bayesian Model Averaging
Super Learner
Scenario
KII
Exp
LR
3Q
KaLSIF
Exp
LR
0.699,
-
0.648/10.005
0.700(10.0077
0.389/10.270)
0.69
0.600.1..002
#0.004)
0.112
0..03)+0.013)
=0.02:
=.001)00.001)
0.877(=0:000)
0.805)
0.4411 (500077)
0.0005
0.410 0.439(10.035)
0.4511 20.000)
20.255
0.439/ 002)
0.001:
3 +
0.817 10.004]
0.803(10..06)
0.783 10.0010
0.8201 10.000
(.64)
0.827 10.001
0. 0.797.10.006]
+
0.907 (:0.00))
0.000))
10.05))
20.63)
0.011 (.0000
20.011)
0.803)..0000) 023)
0.8091000011)
Avg.
0.700(2000044)
0.175(-02155)
0.764(10.018)
0.710(30.011)
0..00
0.478(302000)
0.706(10.014)
0.713)20.000)
Table 72: Mean and standard deviation (after =) of target classification accuracy on HHAR com- puted with domain adaptation method DDC.
Bayeslan
Averaging
Super
Scenario
KIL
Exp
LR
SQ
KaLSIF
Exp
LR
SQ
D 6
0.728(+40.000)
00:10))0.333
0.687;=000131 0.717(=60.024)
0.730 #00014
0.725(=0.020)
0.719)=00.019
0.688(=0.010)
0.855( (.0.00)
0.507
0.881(20.001.
0.827;
0,889/ 10,003
20.05)
0.859r..0000 0099
0.836;
0.186[ 00005)
0:450
0.457
0.497
0.078(10.015)
0.446(10.000)
0.812
10.00)
18800=00002
0.000010.000)
10.015
0.820 00.004
0.846
0.809/10 004)
0. 20.8812
0.620) +0.6311
00001))
0.923(+0.016)
0.800 +0.004)
0.625] +..500
0.057(+0.009)
(.027(+00011)
0.800(000011
Avg.
133)
(.705(20.020)
0.758(40.014)
0.727:20.008)
0.77%(00.014)
0.758(+000112)
0.730;=0.006)
Table 73: Mean and standard deviation (after +)) of target classification accuracy on HHAR com- puted with domain adaptation method DeepCoral.
Bayesian Model Averaging
Super Learner
Scemario
KaLSIF
Exp
SQ
KaLSIF
Exp
0603(+00000
0.72% 00011)
+0.010)
8.663/ 8.663/+0.000
0.805 8.000)
0.743,
0.697(++0.005)
0.663(+0.08)
1 1-66
0.909] +0.002)
0.016
0..00)+
0.850 +0.00'000008
0.909] +0.001)
0.897(+0.008 0..03/+
0.80((00005)
2 2
0.320]
0.02)
0.513(+0.040)
0.462 (0.028)
0.322 = 2.22
(20.117))177)
0.511(=0.055
0. 0.489;00.025)
2..00
0.005
08.1(440.010
0.7752
0.815/20.007
0.817 20007))
006)
0.788
100.47)
0.010(
0.934(10.012)
0.904
0.8330 10.536
0.963/10.0000
0.938[100000
0.002110.001)
Avg.
(.671(10.166)
0.785(00.014)
0.770610.016)
0.736;10.0100
0.674(10.00 100.16))
0.797(100005)
0.772(10:011)
0.742
Table 74: Mean and standard deviation (after +) of target classification accuracy on HHAR com- puted with domain adaptation method CMD.
39


Reviews:
[{"review_id": "v4Rf2Yo8X6", "reviewer": "Reviewer_kUnW", "summary": "The paper propose a new algorithm for Density Ratio Estimation (DRE) that aggregates multiple models with different hyper parameter settings. The empirical experiments focus on domain adaptation, as well as comparisons with cross validation for several different loss objectives. The authors prove that the aggregation achieves minimax-optimal error convergence under very mild conditions on the density ratio.", "questions": "Can the method also be applied when instead of having different hyperparameters in the same objective, we have multiple models from different objectives (KuLSIF, Exp, etc.)?", "limitations": "", "rating": 8, "confidence": 2, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "Overall I think this is a good paper. It's written well and proposes a novel method for DRE estimation that is more computationally efficient than simple cross validation, and improves performance over several other baselines, on several relevant tasks. \nThe method is supported by a provably minimax-optimal error convergence. \nSeveral ablations are done to show the robustness of the method.", "weaknesses": "Slightly more detail about the dataset for the top part of Table 1 would have been useful. What does \"c3,d1.70\" stand for instance, and how many samples were used in the dataset \"c3,d1.70\". \n\nMinor:\nI think the left part of figure two can be improved. If my understanding is correct, the number of models needed is reduced by a factor of the number of k-folds. By plotting it out instead of just stating the mechanical relationship, people may get confused and look for some patterns in the figure.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "ll3TtHOBAL", "reviewer": "Reviewer_96xS", "summary": "The paper find suitable weights that is used to aggregate results from multiple models trained with different hyperparameter to predict density ratio estimation(DRE). The weights are obtained by minimizing the upper bound on the Bregman divergence.", "questions": "1. Does the variance of the models output affect the bounds?\n2. How would noise within the data affect the results analytical?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. There is a strong mathemtical proof\n2. There a many benchmarks across various task and domains", "weaknesses": "1. The paper could provide analysis on the number of models trained and results obtained through this method?\n2. The paper could also show analysis of the distribution of the hyper-parameters used.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "y6mIM8WcMa", "reviewer": "Reviewer_HR3j", "summary": "This paper proposes a method for training multiple models under different hyperparameter configurations and optimally aggregating their outputs. The proposed approach demonstrates the ability to achieve minimax-optimal convergence rates without requiring any prior knowledge about the smoothness of the density ratios. Experimental results also demons\ttrate that the proposed method outperforms cross-validation, Bayesian model averaging, and Super Learner.", "questions": "* How did you choose the regularization parameter value, lambda?  \n* Could you discuss how the lambda selection affected the computational complexity in Fig. 2? In my understanding, the trained models and empirical Hessians can be shared across different lambda values, so there is little impact on overall training time. When you tried 10 different lambda values, $\\{10^{−6},10^{−5},...,10^{4}\\}$ in the experiments, how much did the whole training time increase compared with the single lambda value?", "limitations": "", "rating": 8, "confidence": 3, "soundness": 3, "presentation": 2, "contribution": 2, "strengths": "* The method achieves minimax-optimal convergence rates under reasonable assumptions.\n* The proposed algorithm is theoretically guaranteed and practically computable.  The proposed algorithm is a simple least-squares method that is intuitively easy to understand.\n* Through various density ratio estimation methods, they demonstrate the superiority of the proposed approach over the baseline methods.", "weaknesses": "* The following descriptions of the experiments are insufficient.\n  * The tuning method for the hyperparameter, lambda\n  * The details of BMA:  It is better to write which algorithm in Fragoso et al. (2018) is used.\n\n* Since cross-validation requires only inference from a single selected density ratio estimator during inference, it has the computational advantage of requiring less processing time than the proposed method. Therefore, I consider the superiority of the proposed method to be limited to only certain applications.\n\n* Using $f$ both for DRE models in equation (1) and for DA models in equation (13).   The use of the symbol may confuse readers.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "X5XPhbGbSk", "reviewer": "Reviewer_aJPm", "summary": "This paper provides an aggregation method for solving density ratio estimation (DRE). Through this, it tackles both the issue of method selection as well as hyperparameter tuning by instead training multiple models with multiple hyperparameter setups and aggregating them instead of just selecting one. The authors provide minimax-optimal error bounds without smoothness assumptions for RKHS and convex settings. Thorough empirical validation of the method against cross-validation baselines is done on models beyond just RKHS settings including neural networks.", "questions": "As per the weakness section,\n\n1. Does the theory extend to all your empirical evaluations?\n2. Could the authors comment briefly on the numerical stability of computing the Gram matrix in real applications, and also comment on the costs of computing Hessians in Alg 1?\n3. Are there guarantees for the aggregated DRE to always be non-negative?", "limitations": "", "rating": 8, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "* Proposes a novel and effective strategy for dealing with model and hyperparameter aggregation for density ratio estimation.\n* The proposed closed form solution beats every candidate model instead of just selecting a single one.\n* Theoretical minimax guarantees for RKHS and convex settings.\n* Empirical results also show significant improvements over existing selection baselines in different data settings and using models like neural networks.", "weaknesses": "* It is unclear if the theory covers all the empirical settings like the case where models are neural networks, though this is a tough ask.\n* The method requires empirical computation of a large Gram matrix of size ~ number of models X hyperparameter settings. This could be numerically unstable for various real world applications.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)