Binary Losses for Density Ratio Estimation
Authors: Werner Zellinger
Venue: ICLR 2025 Poster (2025)
Keywords: density ratio estimation
domain adaptation
composite binary losses
class probability estimation
Primary area: learning theory
TL;DR: We propose novel loss functions for classifier-based density ratio estimation, that are characterized by minimizing a prescribed Bregman divergence between the density ratio and the constructed estimator.

Abstract: Estimating the ratio of two probability densities from a finite number of observations is a central machine learning problem. A common approach is to construct estimators using binary classifiers that distinguish observations from the two densities. However, the accuracy of these estimators depends on the choice of the binary loss function, raising the question of which loss function to choose based on desired error properties. For example, traditional loss functions, such as logistic or boosting loss, prioritize accurate estimation of small density ratio values over large ones, even though the latter are more critical in many applications.

In this work, we start with prescribed error measures in a class of Bregman divergences and characterize all loss functions that result in density ratio estimators with small error. Our characterization extends results on composite binary losses from Reid & Williamson (2010) and their connection to density ratio estimation as identified by Menon & Ong (2016). As a result, we obtain a simple recipe for constructing loss functions with certain properties, such as those that prioritize an accurate estimation of large density ratio values. Our novel loss functions outperform related approaches for resolving parameter choice issues of 11 deep domain adaptation algorithms in average performance across 484 real-world tasks including sensor signals, texts, and images.

Published as a conference paper at ICLR 2025
BINARY LOSSES FOR DENSITY RATIO ESTIMATION
Werner Zellinger LIT AI Lab, Institute for Machine Learning Johannes Kepler University Linz, Austria zellinger@ai aaaaaabrraiiabb...... ....j juu.at at
ABSTRACT
Estimating the ratio of probability densities from finite number of observa- tions is a central machine learning problem. A common approach is to construct estimators using binary classifiers that distinguish observations from the two den- sities. However, the accuracy of these estimators depends on the choice of the binary loss function, raising the question which loss function choose based on desired error properties. For example, traditional loss functions, such as lo- gistic or boosting loss, prioritize accurate estimation of small density ratio values over large ones, even though the latter are more critical in many applications.
In this work, we start with prescribed error measures in class of Bregman di- vergences and characterize all loss functions that result in density ratio estimators with small error. Our characterization extends results on composite binary losses from Reid & Williamson (2010) and their connection to density ratio estimation as identified by Menon & Ong (2016). As result, we obtain a simple recipe for constructing loss functions with certain properties, such as those that priori- tize an accurate estimation of large density ratio values. Our novel loss functions outperform related approaches for resolving parameter choice issues of 11 deep domain adaptation algorithms in average performance across 484 real-world tasks including sensor signals, texts, and images.
INTRODUCTION
Estimating the Radon-Nikodým derivative 3 := dP X R of some probability measure P dQ
with respect to another probability measure Q, from iid observations (z;) and (a/) of the two measures, respectively, is common machine learning task; with applications in two-sample testing (Neyman Pearson 1933 Kanamori et al.] 2011: Hagrass et al. (2022), anomaly detection (Smola ctt al. 2009; Hido et al. (011), divergence estimation (N (Nguyen al. 2010; Rhodes et al. 2020
covariate shift adaptation (Shimodairal 2000; Dinu et al. 2023| Gruber et al. (024)) energy-
modeling (Gutmann & Hyvärinen| 2012 Tu 2007), generative modeling (Mohamed & Lakshmi- narayanan 20overrovver classification from positive and unlabeled data (Kato al. 2019). When both measures P and Q have densities w.r.t. the Lebesgue reference measure, then 3 equals the ratio of their densities.
Grover et al. 2019), conditional density estimation (Schuster et al. (202)). and
A large class of methods for density ratio estimation follows Algorithm T to construct an estimator
3 of 3 from binary classifiers which separate the observations (z, (x,),_, P from the observations
( of Q (Qinl 1998 Bickel et al. 2007 [Ceng & Chu 2004 Sugiyama et al. 2012a Menon
& Ong 2016 Kato & Teshimal 2021 Zellinger et al. 2023 The estimator constructed by Algo-
rithm
converges (for m, nl + (c) to the minimizer of Bregman divergence"
B3 3) = E (x)())) - ((3(z)) -6 o' (3(z)) - 3(z))
(1)
with generator Φ R R Sugiyama et al. 2012b
Menon & Ong
2016 Zellinger et al. 2023
Assuming 8 L (Q) and defining Fe L' (Q) R by (b) fx o(h(x)) dQ(x) identifies Eq.
Bregman divergence (Bregman 1967) of the form Bo 8) Fo(0) - (o (3) 3.

---

Published as a conference paper at ICLR 2025
Algorithm 1: Density ratio estimation by binary classification
Setup : Losss function & {-1, x R + R. invertible probability link function
y=1 IR + [0, 1] and function class F c {f I I.).
Input : Observations (x: ! ~ P and (x+) ~ Q.
Output: Estimator 3 X + R of Radon-Nikodým derivative 3 = dP
aQ
Step 1 : Compute binary classifier
i== arg min fer
1
Σ
Ely, [())
(2)
m (x.0)e 1 U(r 1)
and estimate by & (f()) the probability
p1y 1xx)
that æ is drawn from P.
v (f(r))
Step 2 : Construct 3 using Bayes theorem B(x)
PLU p(=-112) -(ar).
cly 1|2)
1- (f(x))
However, the error measure Bg in Eq. is only implicitly defined by the choice of the binary loss funcciin l used to instantiate Algorihmm [ That is, choosing the wrong loss function might lead to unexpected behavior of the density ratio estimator. For example, typical loss functions for Algorithm are the logistic loss used in (Bickel et al. 2009, Section 7), the boosting loss applied in (Menon & Ong 2016,, Section 7), the square loss (Menon Ong 2016 Table 1) and logarithm- based loss functions (Nguyen et al., 2007 2010 Sugiyama et al.) 2008a). All of the loss functions above lead to error measures in Eq. that put higher weight on smaller values the density ratio, despite the fact that large values are crucial in many applications, see (Menon & Ong. 2016).
propose to start from an error measure B& in Eq. [ Our first contribution is, given such an error measure, to combine and extend results from & Williamson (2010) 2011) and Menon &
In this work, instead of choosing classical loss function for Algorithm
for characterizing all loss functions that lead to minimization of this measure B& by Algorithm | More precisely, given some (strictly convex) Bregman generator
[0, oc) R and some (monotonically increasing) density ratio link R + R. we provide the unique form of (strictly proper composite) loss function e: {-1, 1} x R + E such that Algorithm minimizes (3,3, 3) with := go oo f. The link
function is given by V 1-1(2)) (xx) = I+g(#) Following Reid & Williamson 2010), we can show that
g(m)
the choice ( (Ø') ( leads to convex loss functions.
Our characterizations provide with a simple recipe for constructing convex loss functions that allow Algorithm to to find estimators with certain properties. Our second contribution is to provide novel loss functions that (in contrast to classical losses) prioritize an accurate estimation of large density ratio values over smaller ones, see Figure
One important application of density ratio estimation is importance weighting. Our third contribu- tion to show that our novel loss functions can lead to improvements for the parameter selection procedures of Sugiyama et al.) (2007); Dinu et al. (2023) when their importance weights are com- puted with our novel loss function instead of the ones of Example T More precisely, importance weighted validation (Sugiyama et al.) (007) (respectively importance weighted aggregation (Dinul (ct al.] 2202)) selects better parameters on two (respectively three) out of three datasets when it is based on our loss function instead of the ones in Example where "better" refers to higher accu- racy on average over 11 deep learning algorithms and all domain adaptation tasks of three datasets for text data (Blitzer al. 2006), human body sensor signals (Stisen al. 2015 Ragab 2023) and images (Peng et al. 2019 Zellinger et al. 2021).
2 Denote by y = the event "drawn from P" and by = -1 the event "drawn from Q", Then, the probability measure X' x {-1, 1} is defined by p(zly = 1) == P(x), p(zly = -1) = Q(x) and the marginal (w.r.t. y) Bernoulli measure py realizing y with probability and y 1 otherwise. It follows that ((x) plzly- pla.w (1)) pla.u--1)
y( plr.y=1)yy(v=->
p(z.v=1) e(z.v=1)ex()))
e(v=1(z)
pla.s D)x(() p(v--1[3)
2

---

Published as a conference paper at ICLR 2025
AxedPidQ
LR: ((())-t/(((ts))
KoLSIF. ("(0]=1
Ours: (f(cjo
Ours: (/())066
Ours: ((()==
Figure 1: Left: Two piecewise constant probability measures P and Q on [-1, Right: Estimators B(x) == kx + of density ratio B dP with k, d > computed by Algorithm (for m + n
di
00). Compared to LR (Bickel et al., 2009) and KuLSIF (Kanamori et al. 2009 our estimators originate from error measures B& in Eq. with increasing weight functions
(c)
c, co 22 and
consequently obtain better estimates for large values 3(x), x E [-1, 0.9] u [0.9, 1], see Section
Summarizing above, we provide complete set techniques for designing binary loss functions for Algorithm that (a) prioritize the estimation of large density ratio values and (b) allow high performance in practice. Our three key contributions are:
Characterization (Section 4 Given an error measure (3 3), we extend results from Reid & Williamson (2010) 2011): Menon Ong (2016) and characterize all loss
functions that lead to minimization (3, 3) by Algorithm
Loss Functions (Section
5): We design novel loss functions with increasing weight func-
tions ø" (cf. Figure
Experiments (Section 6 We obtain state-of-the-art average performance in benchmark experiments for re-solving parameter choice issues in deep domain adaptation; involving 9174 neural networks, 484 real-world tasks from three datasets for texts, images and human body sensor signals.
RELATED WORK
has been first observed by [Sugiyama et al. (2012b) that many density ratio estimation methods minimize Bregman divergence of the form of Eq. see Example
Example 1.
Following (Sugiyama al. 2012b
Section 3.2.1). the kernel unconstrained least- impor- tance fitting procedure (KuLSIF) of Kanamori et al. (2009) uses E(1, y) := yy {(-1,y) := hny² and the link function t
(v) := 1+v in Algorithm
Eq. is realized with c(x) = (x - 12
and B(x) = f(x) (cf. Kanamori et al. 2012 Eq. (4))) such that B3 3) - -
1972) approach (LR) applied in (Bickel et al. log(1 e- ((-1,y) v) log(1 cv) and link - == (1 +1++ +e e----- The implicit error measure Eq. with p(x) = x log(z)
The logistic regression (Nelder & Wedderburn
2009 Section 7) realizes Algorithm with e(1, y)
x) log(1 x)) and ((x)) e(()) see (Sugiyama al. 2012h Section 3.2.3).
The Kullback-Leibler estimation procedure (KLest) of Nguyen et al. (010 uses E(1, y) y
log(x), E(-1.y) == y and v (y) = I+y' leading to Eq. with x log(z) x and
3 = F: see (Sugiyama et al. 2012b Section 3.2.4) and cf. (Sugiyama et al. 2008b
The exponential loss (Boost) defined by E(1,3) : e- and f(-1. 1. e", as applied in
AdaBoost (Freund & Schapire 1993), is used in iMenon & Ong 2016 Section 7) with link 2y) Eq. used with 6(x) 3/2
V -
3/2 and ((x) 2f(x)
A general formula of the Bregman divergence for given (strictly proper composite) loss functions has been derived by Menon & Ong (2016) based on characterizations of Reid Williamson (2010). We extend their result by showing that the constructions in (Menon & Ong 2016 Appendix B) are
3

---

Published as a conference paper at ICLR 2025
in fact necessary if their loss representation (Menon & Ong 2016 Proposition 3) holds. Our proofs are based on the theoretical works (Shuford Jr et al. 1966); Savage (1971): Gneiting Raftery
2007) : Buja et al. (2005); Reid & Williamson (2010); Bao (2023) on proper loss functions.
There is a line of research concerned with the sampling behavior of algorithms following Eq. Specifically, for the special case of KuL.SIF, error rates (for +
oc) have been discovered by Kanamorri et al.] (2012); Gizewski et al.] (2022): Nguyen et al. (2024). These results have been extended by Zellinger et (2023); Gruber al. (2024) the general case of Eq. [ for loss func- tions that satisfy a self-concordance property. Although, our (continuous) analysis relies on similar theoretical arguments, except for the polynomial case of k O, our novel loss functions are not self- cofcorantt..... That is, these results do not apply without modification.
Situations of quite dis-similar distributions have been recently discussed by [Rhodes et al. (2020) (density-chasm problem), Kato & Teshima (202T): Kiryo et al. (2017) (PU-learning) and Srivastava jt al. (2023). Our work can be seen as analysis of sub-problem of these works, where P is large and Q is small. [Choi et al. (202T) propose an approach by training normalizing flows to obtain closer and simpler densities.
Another recent line of research applies density ratio estimation for correcting deep generative diffu- sion models (Kim et al., (203). In (Kim et al. (024), the Bregman divergence approach Eq. [ extended and further interesting Bregman divergences are identified.
NOTATION AND PROBLEM
Let P,Q E M1 be two probability measures from the set M2(X') of probability measures on a compact space X' CR. d N such that P is absolutely continuous with respect to Q. P & Q- Sugiyama al. 2012b) observed that many methods for estimating the density ratio / aQ are minimizations of estimated Bregman divergences Eq. Menon & Ong (2016) further proved that there is in fact a general mathematical structure underlying this observation, which we review in the following.
We start with the data generation model of Reid & Williamson (2010); Menon & Ongi 2016
probability measure p on X' x y for y := (-1,1) with conditional probability measure defined by p(xlly 1) P(x) and p(zly -1) Q ()) and marginal (w.rt. y) probability measure
Py defined as Bernoulli measure with probability for both events = 1 ("æ belongs to P") and
y -1 ("xx belongs Q""). Our analysis based on the following underlying assumption.
Assumption
(Zellinger et al. (203)). The data
(I,, 1)
x
is an i.i.d. sample of p.
Assumption D allows us to introduce binary classification problem for the accessible data as follows. For a fixed loss function é; 1}, x IR + R, the goal of binary classification is to find, based on the given the data z, a classifier f X + R with small expected risk
fr Jxxy
R(F) E(ra)~ [((,,
(()) dp(z,y).
The loss function é is a central object in our study. In particular, we consider composite loss func- tions l 11, x R + R such that Ø) A(y, v - (77) composed of loss function A: 1,, x [0, tions allow to estimate p(y 1|x) by functions X + R mapping outside of [o, 1] (Reid & Williamson 2010). A central object in the study of composite loss functions is the conditional risk defined by
IR and an invertible link function V [0, 1] + R. Composite loss func-
0,
L(n 0) = (82( + (11-))(-1(0))
'Existence follows from X' x y heing Polish (cf. DDulley 2002 Theorem 10.2.1), the product case).
*Our analysis can be generalized probabilities # using Menon & Ong 2016 Appendix A).
4

---

Published as a conference paper at ICLR 2025
where (1))) ((1. and (-1 ((-1.0) denote the partial losses of E. A composite loss function with invertible link v/ [0.1] - R is called proper, if the conditional risk is minimized at (n):
T(n, ((n) L(n)
inf T. VER
and it is strictly proper, if the Bayes risk L(n) is achieved at a unique We denote by f A' + y the function defined by (x) == V o ply = [rr. Then, the theoretical starting point of our work is the following key lemma for density ratio estimation.
Lemma 1 (Menon & Ong (206)) Proposition 3)). Let e: (-1,1} x R - R be a strictly proper composite loss function with invertible link function V [0, 1] + R and twice difserentiable negative Bayes risk- - L: [0, - R. Then
1 R(S) - -R(S" B_ (3,00
(5)
with
gly) V (ty)
and - L -(1 (1+ -1
L
2
(6)
1+
For convenience of the reader, a proof of Lemma D is provided in Subsection A.3 Lemma speci- fies, for a given loss function, the asymptotic enror L- 3) of the model However, the error L_ defined by the loss function while one might be
g f computed
by Algorithm T
interested in specifying the error measure BB( g o). i.e., specifying @ and g. The problem of this work is to answer the following question:
Which convex loss functions satisfy [B R(()- R((f) for given & and g?
Using the characterization of such loss functions, we are interested in designing loss functions that lead to error measures B3 which put higher weight to larger density ratio values than smaller values, and, to test their effect in importance weighted parameter selection of deep neural networks.
4 CHARACTERIZATION OF LOSSES FOR DENSITY RATIO ESTIMATION
The following Theorem see Subsection B.2 for proof, answers our research question Section above for general (not necessarily convex) strietly proper composite loss functions.
Theorem 1. Let o [0, oc) + R be strictly convex and twice differentiable and let R + R be strictly monotonically increasing. Then, E 1,,,1] x + R is strictly proper composite and satisfies
R(f) = Bo(
(7)
if and only if
(7(v- vー
ff
C(v. ()
(-) 1
if =
with v v-1
g(t) I+9
and (1))
(1 1 7J02 + e1 for some Cı C2 E R.
Remark 1 (on Novelty). The novelty of [eTTThoormm[ssseee
lies in the fact that any strictly proper
composite loss function satisfying Eq. is necessarily of the form of Eq. 8 Our proof has four main
components: (a) the inversion of the constructions developed in (Menon & Ong 2016 Appendix B)
done in Lemma 5 (b) the application of (Reid & Williamson
2010
Corollary 13), (c) Savage
theorem (Savage 1971) and (d) our technical Lemma 4
Remark 2. The sufficiency of Eq. 8 can be easily seen from Menon & Ong (0166) Reid &
Williamson (2010): Bao (2023) as follows: In Menon Ongl (2016) Proposition 1) it is proven that any strictly proper composite loss satisfies Eq. M and Reid & Williamson (2010) Theorem 7)
(ef. Reid Williamson (201T); Bao (2023)) proves that the loss in Eq. B construcced by any strictly concave ry strictly proper composite.
5

---

Published as a conference paper at ICLR 2025
Theorem expresses the expected Bregman divergence excess risk. This allows to estimate the Bregman divergence in a natural way by empirical risk minimization. However, the minimization of non-convex functions is hard. We therefore characterize convex loss functions by corollary of Theorem D combined with Theorem
provided by RRid & Williamson (2010).
Corollary 1. Let Φ : (0.00) - R. be three times differentiable such that d'(x) 0 for all x
[0, 0o) and let g IR. + R be strictly monotonically increasing. Then, the loss l in Eq. is convex in
its second argument for all y (-1,1)
1 if and only if
1
(x)
(0 (x))))
S 00,
Vx E (0)
(9)
x
(g ())
Corollary Tis proven in Subsection B.3 Corollary suggests a canonical
Jcan (
(10)
for constructing strictly convex loss functions by Eq.
g
of Buja al. (2005) ing loss of Kivinen & Warmuth (1997;; Helmbold et al. 1999 as discussed in AReid & Williamson 2010 Section 6) and the canonical Jean do not lead to the same loss function. To see this, note 1 that the equation can ( = 1+9(3)
Remark The canonical link y can w & V
(i.e., the match-
9( in Theorem leads to
2 (0) V can 1+
Buja et al.
(0055) show that differentiating second time Eg. 20ggives w((i) -I" and therefore
1 + -- 1 1+ (1+2)= (²) 2)
(=))
-
--"
which satisfies Eq. o but differs from (grah) (z) by a factor (1 + 2)
Thanks to Theorem 1 and Corollary 1 given strictly convex Bregman generator é, we are now able to express the Bregman divergence (B Jean î) excess risk of canonical convex loss
function lean defined by Eq. 8 and 9ean Instantiating Algorithm 1 by fean and
gran (
1 Dean
we arrive at minimizing (8, B) for = Ican o f.
5 LOSS FUNCTIONS FOR ESTIMATING LARGE DENSITY RATIOS
has been observed by Menon & Ong 2016) that no loss function in Example prioritizes an accurate estimation of large density ratio values smaller values. We show this behavior in whichh
CubsectioonA.I) which uses the fact that the second derivative d" of the Bregman generator & satisfies the relation
Bo(3,3) Eane [ ('(c)),(((()) de
(11)
for the piecewise linear function de defined in Eq. 16 that does not depend on Φ. It can be seen that the weight functions (c) <(I++c) (LR), (c(c) = c-1 (KLest), ("(c) = c 3/2 (Exp) and (c) = 1 (KuLSIF) of Example are non-increasing and therefore do not realize prioritization of estimating large values. In contrast, we propose the following novel Bregman divergences as error measures for Algorithm 1 which prioritize accurate estimation of large values, see Figure I for an illustration.
The exponential weight (EW) (c) == 22 and g:nh ø' realize the novel Bregman divergence
(B 33) R 220()) + 20(m) 2,((x) + 23()))
(12)
Plugging & and Jcan in Eq. gives the loss functions ((1,y) = -y, ((-1,y) = #u(og(()y) 11)
and 8(x) = log(2/(z)), which satisfy Eq. according Theorem

---

Published as a conference paper at ICLR 2025
0.6
5.0
exact
0.5
2.5
2.5
2.5
2.5
KuLSiF #%cc)
ours 911)
0.4
2
8:
0
16
0.0001
-- 0.01
1.0
0.3
5.0
5.0-
5.0
5.0
0.1
0.0
0.0
0.0
0.0
0.0
Estimation of density ratio (left, black) in Gaussian RKHS with Tikhonov penalty weighted by for sample sizes m 10 (top right) and + 100 (lower right). Our loss function (blue) prioritizes accurate estimation of larger values and consequently produces flat- ter curves then KuLSIF (green).
Figure 2:
The polynomial weight functions (c) == 22 for E {0,1,2,3,... and 9cau == ø' in Eq. 10 realize the Bregman divergences
(3, (1 (2 + k))
R B(x)2tk 1)3(x) +1) (13)
Plugging é and Jcan in Eq. gives the loss functions {(1, = y and (-11,)) (1++)vl 21E
2+
and the estimators 3(xx) = ((1 + 1)) fz)( Note that the class of divergences in Eq. 13 can be
interpreted extend KuLSIF, which realizes Eq. [3 with k 0, to larger exponents.
6
EMPIRICAL EVALUATIONS
6.1 NUMERICAL ILLUSTRATIONS
Before showing the potential of our novel loss functions on real world problems in Subsection 6.2 we illustrate the behavior of our methods two experiments: Experiment (a) on the form of kernel estimators for a one-dimensional Gaussian density ratio and Experiment (b) on the behavior of (analytically computed) density ratio estimators used as importance weights in kernel regression.
Experiment (a): Figure 2 shows the behavior Algorithm [ instantiated with the loss functions for KuLSIF (see Example 1 and our novel loss function corresponding to Eq. 12 for the numer- ical illustration of Zellinger et al. (2023). The function class was fixed to a Gaussian RKHS with bandwidth parameter computed by the median heuristic. We also added Tikhonov penalty +a of to Eq. 2 and computed the solution for different values of a E (10-9 10 10 .1) by (BFGS) algorithm, see, e.g., (Nocedal & Wright 1999..
the Flttceee-eeeeeeeefereet
The equal weighting of all density ratio values KuI.IF leads to exploding estimates for small 10 with small regularization parameter a E {10-6, 10-4). This problem does not appear for our loss function, which comes, however, at the cost of "flatter" estimates due to higher penalization of errors in larger density ratio values.
sample size
Experiment (b): Figure 3 extends the setting of Figure to unsupervised domain adaptation under covariate shift (Shimodairal (2000): given iid source observations (z/) from Q together with labels y! ff()))) + & corrupted by iid Gaussian noise observations (c))= and unlabeled iid target observations (a:) from P. the goal to estimate the unknown Bayes predictor fr(z) sin (3 This is done by importance weighted kernel least squares regression
m
f arg min
B(x)) 1l f(x())1 + +
(14)
JEF
m
+ n
with 3 of Figure for LR and our method with exponential weight (c) 20 We used degree- five polynomial RKHS F. a small a = 10 -2 and computed the solution by BFGS optimizing the weights of the solution granted by the representer theorem Kimeldorf & Wahha 1970).

---

Published as a conference paper at ICLR 2025
Sayer mndetter
bariing Me0 Dayas prediter
1.-
Parning fram
1.0
no woglling d-l
estinate sidigg
0.5
attimate Loto PTC:
extimate Eoto
0.00
-022
-0.2
0.5
-0.50
-1.00-2.75 0..75--55------- -.25--22 0.00 025
0.75
1.50
0.54
0.00
0.50
3.00
Figure 3: Using density ratio estimators from Figure I sample weights for polynomial kernel least squares regression. Left: Bayes predictor (black, dashed) and observations (x) from Q. Middle: regressors (blue: uniform weighting 3 = 1, green: exact 3, red: our estimate with d' = 220
from Figure [ yellow: LR estimate from Figure Right: Pointwise errors for target observations from P. Exponential weight (ours) is higher for larger density ratio values; consequently achieves smaller errors [0.9, 1].
Our weight (c) = e² is increasing, which leads to an error smaller than 0.040 in squared (P)) norm. This be explained by the fact that the (P)) assigns higher weight regions where the density ratio is large and that these regions are exactly the ones where our weight leads to more accurate estimates. In contrast, the weight function of LR decreasing, which leads an error larger than 0.067 in squared (P((--oor. The higher performance of our method in squared LP)) norm comes at the cost of higher squared (Q))))nor -norm (0.798) than for LR (0.778). However, we note that in unsupervised domain adaptation we are interested in the target excess risk on P which is nothing but the squared L (P)))norm The corresponding pointwise errors are depicted in Figure
6.2 REAL-WORLD DATASETS
In the following, test the performance of our exponential weighted Bregman divergence in Eq. compared to the performace of LR, KuLSIF and Boost (see Example [ for calculating importance weights for parameter selection methods in unsupervised domain adaptation. In total, we test the performance on 484 parameter selection tasks; one for each combination of a parameter selection method, domain adaptation method (from three datasets) and a domain adaptation tasks (originat- ing from three datasets). We trained 9174 neural networks in total, see Section for details.
The unsupervised domain adaptation setting: given iid source observations (z/) from Q to- gether with labels v/4 = fp(x() + corrupted by general iid noise observations (e:) and unla- beled iid target observations (x,)²_, from P. the goal is to estimate the generally unknown Bayes predictor fp: X Rdv, dy N.
The goal is to select parameters from the methods Adversarial Spectral Kernel Matching (AdvSKM) (Liu & Xue (021), Deep Domain Confusion (DDC) (Tzeng
Domain adaptation methods:
et al., 2014), Correlation Alignment (DCoral) (Sun et al. 2017 Niizzad-Langerodi et al. 2019).
Central Moment Discrepancy (CMD) (Zellinger et al. 2017 2019)). Higher-order Moment Match-
ing (HoMM) (Chen et al.] 2020). Minimum Discrepancy Estimation for Deep Domain Adaptation
(MMDA) (Rahman et al. 2020), Deep Subdomain Adaptation (DSAN) (Zhu et al.) 2021). Domain-
Adversarial Neural Networks (DANN) (Ganin et al. 2016), Conditional Adversarial Domain Adap-
tation (CDAN) (onng et al., 2018). DIRT-T (DIRT) (Shu et al., 2018) and Convolutional Deep
Domain Adaptation specifically suitable for time-series (CoDATS) (Wilson et al. 2020).
Parameter selection methods: We select these parameters by Importance Weighted (Cross- Validation (IWV) (Sugiyama et al. (007) and Importance Weighted Aggregation (IWA) (Dinu et al. 2023). Both methods, IWV and IWA, first compute several models f1....,ft X' > Rd one for each parameter setting to try, and second compute the final model f : X' + Rdd by Eq. 14 for F == [Ji..... fi} and F = (Σ Ck fe|ei,...,, Ck E R} for IWV and IWA, respectively. Since Eq. reliis on accurate estimate of ao an estimate.
dr also IWV and IWA crucially depend on such
Datasets and tasks: We rely on three domain adaptation datasets with several domains and tasks. The first dataset is the AmazonReviews dataset from Blitzer et al. (2006) which consists of text
8

---

Published as a conference paper at ICLR 2025
views from four domains: books (B), DVDs (D), electronics (E), and kitchen appliances (K). The reviews are encoded in feature vectors of bag-of of-g-of-oodds unigrams and bigrams with binary labels indicating the sentiment of the rankings. From the four domains we obtain twelve domain adapta- tion tasks (B-D, D D->E...) where each domain serves once as source domain and once as target domain. The second dataset is the Heterogeneity Human Activity Recognition dataset (HHAR) from Stisen et al. (2015) which investigates sensor-, device- and workload-specitn heterogeneities from 36 smartphones and smartwatches, consisting of 13 different device models from four manu- facturers. We follow Ragab et al. 2023) and obtain five domain adaptation tasks. Our third dataset is the MiniDomainNNe dataset from Zellinger et al. (2021) which is reduced version of the large scale DumainNet-2019 from Peng et al. (2019) consisting of six different image domains (Quickdraw, Real, Clipart, Sketch, Infograph, and Painting). MiniDomainNNt reduces the number of classes of 10 to the top-five largest representatives in the training set of each class across all six domains. MiniDomainNet is a multi-source single target dataset for which we obtain five tasks, one task for each domain chosen target domain (and all others used as source domains).
Density ratio estimation methods: We estimate the density ratio by Algorithm D initialized by our loss function corresponding to Eq. 12 and the other loss functions from Example T excluding SQ which did not perform well in our experiments and also not in the one of Gruber et al. (2024). Eq. 2 optimized with the BFGS algorithm (100 iterations) in a Gaussian RKHS with bandwidth chosen by the median heuristic and Tikhonov regularization with parameter a E {10,0.A 0.1, 10-3) chosen by valsdavniiationn following Gruber et al. (2024).
Our framework extends the Pytorch framework of Gruber ct al. (2024), by our exponential weight method in Eq. 12] The Pytorch framework of Gruber al. (024 itself extends the ond (Dinu et al. (2023) which relies on the AdaTime benchmark suite" of Ragab et al. (2023). ). all our remaining parameters, we exactly follow the statistical setup and data splits of Dinu et al. (2023) together with the cross-validation based parameter choice for density
Framework and benchmark protocol:
ratio Gruber et al. (2024), see Section C for details.
6.3 RESULTS OF EMPIRICAL EVALUATIONS
Beyond the obvious advantages of our novel methods illustrated in Figure 1 Experiment (a) and Ex- periment (b), we obtain the following results (see Subsections C.3 C.2 and C.3 for the full results):
EW outperforms average LR. KuLSIF and Boost for IWV and IWA all three datasets, excluding IWV on MiniDomainNet for which all methods (Boost, SuI, LR and EW) perform similarly.
EW is among the best methods in 56% of parameter selection problems, i.e., among all cells in all tables in Subsections C.3 and clearly outperforming KuLSIF (23%), LR (13%) and Boost (17%).
7 CONCLUSION AND FUTURE WORK
A large class of methods for density ratio estimation minimizes a Bregman divergence between the true density ratio and an estimator. However, the Bregman divergence is only implicitly determined and depends on the choice of a binary loss function used to instantiate the algorithm that underlies this class of methods. convex) loss function for explicitly given Bregman divergence that we want to minimize. Our formula allows us to construct novel loss functions with the property of focusing on estimating large density ratio values, which is in contrast to related approaches. We provide numerical illustrations and applications in deep domain adaptation. The sample complexity of our methods is, to the best of our knowledge, an open problem.
this work, we derived the unique formula of a (strictly proper composite
"http://gituub..o//uuubeee/t wotp//gg//ommuee/b/rrb... iter reg
"https://github.. wwwm//iitft///////////.... http::///gthuuu.commmmaeleeeeee Time
9

---

Published as a conference paper at ICLR 2025
AmazonResten
Impertane Woightal Valibrioon
Ingustacc Wophtol Aprrestion
S0
Bisel
Kol.IFF
1R
EW (oun))
Bore
KalSUF
EW (ours)
TR
ABRSKM
0,16711000)
0.770=.010)
07736E.009
0.766
6.778150100
0.7XXE000
CBAN
0.262L.012)
0.723.1.013
0.7211-013
0.77k (09)
0,088 025)
026228.0141
0.715
0.789
0.7776.000 012)
CMD
0.26211 K09)
0.7MM-0011 (117)
0.734:1.015)
07:4(--019)
0.77% .011
0,745 0460
0762:1-0115
0.723-00233
0.791(1.010)
0.765 (09)
CUDATS
0.783--011 0180
0.783(2.017)
0.7812-025)
0.784220117
0.727 020)
0.70062:012
0.784:20030
0.79712.010
0.79162.013
DANN
0.767L3.012)
0.790=0915)
0.781(2.017)
0.777440013
0.785
0.772/4 024)
0.7892.011)
0.779
0.79%1. 968)
0.79812 0099
DDC
0.706:2.013
0.220
0.770K..010)
0.77012.111 (01))001))
0.769
0.705 022)
0.7213.0141
0.255
277911.111
0.770X1.00 011)
DIRT
0.764610013
0,7251-0000 0200
0.28001.016
0.7721-017 017
0.78X..017)
0,769:3 022)
0.76)2.010)
0.734:20006)
0.794.1.009 809)
0.780 (1.5)
DSAN
0.709
004)
0.778..0155 0130
0.7721-015
0.783
0013)
0,06K: 955)
0.788:200141
0.787
0.798:1 908
0. 0.789 (U.))
DOml
0 9133
D. 0.5000 0120
0 012)
0
0.20 010
0.771:
-012)
0,26K 020)
0122
0.723
0.783 118
0.2 77 AL1)
HaMM
0033
B 1-7654-11 0100
0.
06266 012)
0. 0.661 (40)
0.7671
.81
0.7632
+ 0233
0.221(+-000
01
0.7654+-0155
0.776 (099)
0. 76R (12)
MMDA
0.26714.009) 0099
-014)
0.77214.011 014)
0.209120022
0.775L..112
0121
0.76%
018)
0.778:4.0121
0.772.-0:88
0.78% (48)
0,752 (012)
02 0.773
01774
0.776
20
0.766
0244
0.7816420021
0.744:50075
0.759
1. (0-1
Impoctance Weighted Validaiinn
Importance Wagleed Apgregatio
Roost
SEF
1R
coura)
Bons
Kel SIP
lours)
A
1.724
07726++ 033)
0.22 .28)
0.684
0 (03)
CDAN
0.738:+-043)
0.789
0.765 082)
0.785 033)
0.781=-001
0.780 051)
0.27(+-051)
0.79k=0047)
0.788
0 (25)
CMD
0.748
0.760-
0.757(=-043
0:757 043)
0.768=-000 027)
0.11:40033
0.72% (40)
0723:+003)
0.794 (15)
CuDATS
0.700+.063)
0741
0
0.741:: (52)
0.739 949
0.724:: 042)
0.310+.089)
0.72k
0.732
7.75 (2T)
DANN
0.757
0.73311081
0 827)
0.798 0271
0.75212 014)
0.7962.077)
0.785
0.7711.040)
180771 023)
0:238/+-090
0.814))000)
0.71% 062)
0770++
0.725
0.747(+-44)
DSAN
0.731++-049
0.717
0.755+
0.723 062)
0,731 050)
0.736+-8377
0.728c=0034)
0.791:+003)
0.82<+ (05)
DOanl
0.745(2-089)
0.74%:: 038)
0.74%
0.673/ 054)
0.67
0.62):00022)
0.657:--131)
0.776c 0d6)
B&MM
0.739120087
0.066
0.728 037)
0.729
037)
0.671:
061)
01690(2.048)
0.53))
0.703:1.137)
0.764: (20)
MMDA
0
(77611-044)
0.7221
(.69711-089
0,69%
0.710:1.029)
0.78511 (46)
0.745 033
0.31
0.72311
MiniDomainNN
Impertane Weightal Valibriin
Ingustane Woplast Apmegatio
50
Brael
Kul.UFF
LR
ER (oun))
Bord
Kel.SlI
LX
EW tours)
AdvSKM
(.509110015)
0.515(=.014)
0.514c1.014 014)
0515(1-014)
014)
0.516 0015
0,512 012)
0.513:1.813)
0,5tic2:0:5)
0.512(4.013
0.52243-000
CDAN
0.514(2.015)
0.513(=0015)
0.513/2.019)
0.510/2.021
0.514
0.519:20311
0.530(2.014)
0.59
0221
(1.42(2.007)
CMD
0.509
0.525 0.535(_0222 022)
0.523(1.023
0.522 023)
0.52X2.0211
0.517 025)
0.516(2.025
0.51%
0.518:1.000
1.533K1.02 020
CoDATS
0.50211.022 032)
0.114(=-0255)
1.209
0.51211.023 023)
0.515 (20)
0.527:4 023)
0.526:2.015)
0.525
0.527L1.018 0166
1.52911.019
DANN
0.996110119
0,113(=-0000
0.514(1.018)
0.1211--021 021)
0.515 018)
0515:10113
0.11612.013)
0.5166.000)
0.15.1.014 014)
0.552 (24)
DDC
0.10111021
(113))_010))
0.51115 0.51
(10)
0.512.5.0122 01
0.516: 015)
031512.018)
0.1152.0.66
0.518(1.018)
0.521 (29)
DIKT
0.899111100 100
8 D-4P[[-0001 2035)
0.69% 052)
0-484
(2:4)
0.51K 0313
0.523:1.251
0,51
020)
0.599: 021
0.525 (42)
DSAN
0.509++1122 1223
D 504 (220
0.
0.510 033)
0,00 020)
0.516
01.)
0.51* 0185
0:52++0014)
0.523) 0:55
0.524(-.818)
0. 565 (04)
DCond
0.505:1.023) 025)
0.522) 012)
0.522)1.014)
0.52112.0133
0.52215.015
0122
0.2212.013
0.52)
0.521:1.0113
0.535
BaMM
0.508(1.023
0.1181
021)
0.521(1.0025
0.516110020)
0.514(2.019)
0199
0.2261.013
0.52N-1.016
0.557LL 0.527(1.014)
MMDA
0.0911-022 022)
0.522 6200
0.51
051811.018)
0.519 620
0.22
0.2314.0131
0.528:1-013)
05511E.013)
BRR
0.13
0019
0.304
03131
0.11
0199
033165000
0.32
1.354 2011
Table 1: Mean (#: standard deviation) for three random initializations of model weights averaged over 12 domain adaptation tasks for AmazonReviews and five tasks for HHAR and MiniDomainNet; full tables in Subsections
and C.3
ACKNOWLEDGMENTS
Part of this work was done while Werner Zellinger was affiliated with the Johann Radon Institute for Computational and Applied Mathematics of the Austrian Academy of Sciences.
I thank five anonymous reviewers and an anonymous area chair for helpful discussions. I also thank Dinu for helpful discussions on their Pytorch frameworks.
Lukas Gruber and
The ELLIS Unit Linz, the LIT AI Lab, the Institute for Machine Learning, are supported by the Federal State Upper Austria. We thank the projects Medical Cognitive Computing Center (MC3), INCONTROL-RL (FFG-88 (EFG-881064)) 1064), PRIMAL (FFG-873979), S3AI (FFG-872172), DL for Gran- ularFlow (FFG-871302), EPILEPSIA (FFG-89217D), AIRI FG 9-N (FWF-36284, FWF- 36235), AlAGrreenHeatinGGrids (FFG- 899943), INTEGRATE (FFG-892418), ELISE (H2020-ICT-- 2019-3 ID: 951847) SttersWaters COBRIZON-CL--200 -LL-202I-CLIMATEE -LIMATE--1-00)) We thank Audi.JKU Deep Learning Center, TGW LOGISTICS GROUP GMBH, Silicon Austria Labs (SAL), FILL Gesellschaft mbH, Anyline GmbH Google, ZF Friedrichshafen AG, Robert Bosch GmbH, UCB Biopharma SRL. Merck Healthcare KGaA, Verbund AG, Software Competence Center Hagenberg GmbH, Borealis AG, TÚV Austria, Frauscher Sensonic, TRUMPF and the NVIDIA Corporation.
10

---

Published as a conference paper at ICLR 2025
REFERENCES
János Aczél and Johann Pfanzagl. Remarks on the measurement of subjective probability and infor- mation. Metrika, 11(1):91-155, 1967.
Han Bao. Proper losses, moduli of convexity, and surrogate regret bounds. In Annual Conference on Learning Theory, pp. 525-547. PMLR, 2023.
Steffen Bickel, Michael Brückner, and Tobias Scheffer. Discriminative learning for differing training and test distributions. In International Conference on Machine Learning, pp. 81-88, 2007.
Steffen Bickel, Michael Brückner, and Tobias Scheffer. Diseriminative learning under covariate shift. Journal of Machine Learning Research, 10(9), 2009.
John Blitzer, Ryan McDonald, and Fernando Pereira. Domain adaptation with structural corre- spondence learning. In Conference on Empirical Methods in Natural Language Processing, pp. 128, 2006.
Lev M Bregman. The relaxation method of finding the common point of convex sets and its applica- tion to the solution of problems in convex programming. USSR Computational Mathematics and Mathematical Physics, 7(3):200-217, 1967.
Andreas Buja, Werner Stuetzle, and Yi Shen. Loss functions for binary class probability estimation and classification: Structure and applications. Working draft, November, 3:13, 2005.
Chao Chen, Zhihang Fu, Zhihong Chen, Sheng Jin, Zhaowei Cheng, Xinyu Jin, and Xian-Sheng Hua. HoMM: Higher-order moment matching for unsupervised domain adaptation. Association for the Advancement of Artificial Intelligence, 2020.
Kuang Fu Cheng and Chih-Kang Chu. Semiparametric density estimation under two-sample den- sity ratio model. Bernoulli, 10(4):583-604 2004.
Kristy Choi, Madeline Liao, and Stefano Ermon. Featurized density ratio estimation. Conference on Uncertainty in Artificial Intelligence, pp. 172-182, 2021.
Marius- Coritus-onnttntin Dinu, Markus Holzleitner, Maximilian Beck, Hoan Due Nguyen, Andrea Hu- ber, Hamid Fghhal-zadeh, Bernhard A Moser, Sergei Pereverzyev, Sepp Hochreiter, and Werner Zellinger. Addressing parameter choice issues unsupervised domain adaptation by aggregation. International Conference Learning Representations, 2023.
Richard M Dudley. Cambridge studies in advanced mathematics: Real analysis and probability. 74. Cambridge University Press, 2nd edition, 2002.
Yoav Freund and Robert E Schapire. A -esiciin-thhoreticc generalization on-line learning and an application to boosting. In European Conference on Computational Learning Theory, pp. 23-37. Springer, 1995.
Yaroslav Ganin, Evgeniya Ustinova, Hana Ajakan, Pascal Germain, Hugo Larochelle, François Laviolette, Mario Marchand, and Victor Lempitsky. Domain-adversarial training of neural net- works. Journal of Machine Learning Research, 17(1):2096-25) 2030, 2016.
Elke R Gizewski, Lukas Mayer, Bernhard A Moser, Due Hoan Nguyen, Sergiy Pereverzyev Jr, Sergei V Pereverzyev, Natalia Shepeleva, and Werner Zellinger. On regularization of unsuper- vised domain adaptation in RKHS. Applied and Computational Harmonic Anulysis, 57:201-227, 2022.
Tilmann Gneiting and Adrian E Raftery. Strictly proper scoring rules, prediction, and estimation. Journal of the American Statistical Association, 102(477):35-337) 2007.
Aditya Grover, Jiaming Song, Ashish Kapoor, Kenneth Tran, Alekh Agarwal, Eric J Horvitz, and Stefano Ermon. Bias correction of learned generative models using likelihood free importance weighting. Advances in Neural Information Processing Systems, 32, 2019.

---

Published as a conference paper at ICLR 2025
Lukas Gruber, Markus Holzleitner, Johannes Lehner, Sepp Hochreiter, and Werner Zellinger. Over- coming saturation in density ratio estimation by iterated regularization. In International Confer- on Machine Learning. pp. 16502-165222 2024.
Michael U Gutmann and Aapo Hyvärinen. Noise-contrastive estimation of unnormalized statistical models, with applications to natural image statistics. Journal of Machine Learning Research, 13 (2). 2012.
Omar Hagrass, Bharath K Sriperumbudur, and Bing Li. Spectral regularized kernel two-sample tests. arXiv preprint arXiv:2212.09201/ 2022.
David P Helmbold, Jyrki Kivinen, and Manfred K Warmuth. Relative loss bounds for single neurons. IEEE Transactions on Neural Networks, 10(6):1291-1344 1999.
Shohei Hido, Yuta Tsuboi, Hisashi Kashima, Masashi Sugiyama, and Takafumi Kanamori. Statis- tical outlier detection using direct density ratio estimation. Knowledge and Information Systems, 26:309-336, 2011.
Takafumi Kanamori, Shohei Hido, and Masashi Sugiyama. A least-squares approach direct im- portance estimation. Journal of Machine Learning Research, 10:1391-1445, 2009.
Takafumi Kanamori, Taiji Suzuki, and Masashi Sugiyama. homogeneity test under semiparametric density- models. IEEE Transactions on Information Theory, 58(2):708-77) 700- 2011.
A f-divergence estimation and two-sample
Takafumi Kanamori, Taiji Suzuki, and Masashi Sugiyama. Statistical analysis of kernel-based least- squares density-ratio estimation. Machine Learning, 86(3):335-367, 2012.
Masahiro Kato and Takeshi Teshima. Non-negative Bregman divergence minimization for deep direct density ratio estimation. In International Conference on Machine Learning, pp. 5320- 5333, 2021.
Masahiro Kato, Takeshi Teshima, and Junya Honda. Learning from positive and unlabeled data with selection bias. International Conference on Learning Representations, 2019.
Dongjun Kim, Ycongmin Kim, Se Jung Kwon, Wanmo Kang, and I1-Chul Moon. Refining gen- erative process with discriminator guidance in score-based diffusion models. In International Conference on Machine Learning, 2023.
Ycongmin Kim, Byconghu Na, Minsang Park, JoonHo Jang, Dongjun Kim, Wanmo Kang, and II- Chul Moon. Training unbiased diffusion models from biased dataset. International Conference on Learning Representations, 2024.
George S Kimeldorf and Grace Wahba. A correspondence between bayesian estimation stochastic processes and smoothing by splines. The Annals of Mathematical Statistics, 41(2):995-502, 1970.
Ryuichi Kiryo, Gang Niu, Marthinus C du Plessis, and Masashi Sugiyama. Positive-unlabeled learning with non-negative risk estimator. In Advances in Neural Information Processing Systems, 2017.
Jyrki Kivinen and Manfred KK Warmuth. Relative loss bounds for multidimensional regression problems. Advances in Neural Information Processing Systems, 10, 1997.
Qiao Liu and Hui Xue. Adversarial spectral kernel matching for unsupervised time series domain adaptation. International Joint Conference on Artificial Intelligence, 30, 2021.
Mingsheng Long, Zhangjie Cao, Jianmin Wang, and Michael Jordan. Conditional adversarial domain adaptation. Advances in Neural Information Processing Systems (NeurIPS), 31, 2018.
Aditya Menon and Cheng Soon Ong. Linking losses for density ratio and class-iro estima- tion. In International Conference on Machine Learning, pp. 304-313. PMLR, 2016.
Shakir Mohamed and Balaji Lakshminarayanan. Learning in implicit generative models. arXiv preprint 16 03483, 2016.
12

---

Published as a conference paper at ICLR 2025
John Ashworth Nelder and Robert WM Wedderburn. Generalized linear models. Journal of the Royal Statistical Society Series A: Statistics in Society, 135(3)3370-385 1972.
Jerzy Neyman and Egon Sharpe Pearson. Ix. on the problem of the most efficient tests of statistical hypotheses. Philosophical Trunsactions of the Royal Society of London. Series A, Containing 1933.
Papers of a Mathemutical or Physical Character, (694-777)))
Duc Hoan Nguyen, Werner Zellinger, and Sergei Pereverzyev. On regularized Radon-Nikon Radon-Nikodym differentiation. Journal of Machine Learning Research, 25, 2024.
Xuan Long Nguyen, Martin J Wainwright, and Michael Jordan. Estimating divergence functionals and the likelihood ratio by penalized convex risk minimization. Advances in Neural Information Processing Systems, 20, 2007.
Xuan Long Nguyen, Martin Wainwright, and Michael I Jordan. Estimating divergence functionals and the likelihood ratio by convex risk minimization. IEEE Transactions on Information Theory, 56(1 (6(11)5777-8811 2010.
Ramin Hikzad-Langerodi Werner Zellinger, Susanne Saminger-Platz and Bernhard Moser. Domain- invariant regression under Beer-Lambent's ' law. IEEE International Conference on Machine Learning and Applications, pp. 581-586. IEEE, 2019.
Domain-invariant
Jorge Nocedal and Stephen Wright. Numerical optimization. Springer, 1999.
Xingchao Peng, Qinxun Bai, Xide Xia, Zijun Huang, Kate Saenko, and Bo Wang. Moment matching for multi-source domain adaptation. International Conference on Computer Vision, pp. 1406- 1415, 2019.
Jing Qin. Inferences for case-control and semiparametric two-sample density ratio models. Biometrika, 85(3):619-630, 1998.
Mohamed Ragab, Emadeldeen Eldele, Wee Ling Tan, Chuan-Sheng Foo, Zhenghua Chen, Min Wu, Chee-Keong Kwoh, and Xiaoli Li. Adatime: A benchmarking suite for domain adaptation on time series data. ACM Transactions on Knowledge Discovery from Data, 17(8):1-18, 2023.
Mohammad Mahfujur Rahman, Clinton Fookes, Mahsa Baktashmotlagh, and Sridha Sridharan. On minimum discrepancy estimation for deep domain adaptation. Domain Adaptation for Visual Understanding, 2020.
Mark D Reid and Robert C Williamson. Surrogate regret bounds for proper losses. In International Conference on Machine Learning, pp. 897-904, 2009.
Mark D Reid and Robert C Williamson. Composite binary losses. Journal of Machine Learning
Research, 11:2387-2422 2010.
Mark Reid and Robert C. Williamson. Information, divergence and risk for binary experiments.
Journal of Machine Learning Research, 12(22)731-817, 2011.
Benjamin Rhodes, Kai Xu, and Michael U Gutmann. Telescoping density-ratio estimation. Ad-
vances in Neural Information Processing Systems, 33:4905-4916 2020.
Leonard Savage. Elicitation of personal probabilities and expectations. Journal of the American
Statistical Association, 66(336):783-80) 1971.
Ingmar Schuster, Mattes Mollenhauer, Stefan Klus, and Krikamol Muandet. Kernel conditional density operators. In International Conference on Artificial Intelligence and Statistics, pp. 993- 1004. PMLR, 2020.
Hidetoshi Shimodaira. Improving predictive inference under covariate shift by weighting the log- likelihood function. Journal of Statistical Planning and Inference, 90(2):27-2444, 2000.
Rui Shu, Hung H Bui, Hirokazu Narui, and Stefano Ermon. A dirt-t approach to unsupervised domain adaptation. International Conference Learning Representations, 2018.
13

---

Published as a conference paper at ICLR 2025
Emir H Shuford Jr, Arthur Albert, and Edward H Massengill. Admissible probability measurement procedures. Psychometrika, 31(2):125 145, 1966.
Alex Smola, Le Song, and Choon Hui Teo. Relative novelty detection. In Artificial Intelligence and Statistics, pp. 536-543, PMLR, 2009.
Akash Srivastava, Seungwook Han, Kai Xu, Benjamin Rhodes, and Michael U. Gutmann. Esti- mating the density ratio between distributions with high discrepancy using multinomial logistic regression. Transactions on Machine Learning Research, 2023.
Carl-Axel $ Staël von Holstein. Assessment and evaluation of subjective probability distributions. Econdomic Research Institute, Stockholm School of Economics, 1970.
Allan Stisen, Henrik Blunck, Sourav Bhattacharya, Thor Siiger Prentow, Mikkel Baun Kjærgaard, Anind Dey, Tobias Sonne, and Mads Møller Jensen. Smart devices are different: Assessing and mitigatingmobile sensing heterogeneities for activity recognition. In ACM Conference on Embedded Networked Sensor Systems, pp. 127-140, 2015.
Masashi Sugiyama, Matthias Krauledat, and Klaus-Robert Müller. Covariate shift adaptation by importance weighted cross validation. Journal of Machine Learning Research, 8(5). 2007.
Masashi Sugiyama, Shinichi Nakajima, Hisashi Kashima, Paul Buenau, and Motoaki Kawanabe. Direct importance estimation with model selection and its application to covariate shift adaptation. In Advances Neural Information Processing Systems, pp. 1433-1440, 2008a.
Masashi Sugiyama, Taiji Suzuki, Shinichi Nakajima, Hisashi Kashima, Paul von Bünau, and Mo- toaki Kawanabe. Direct importance estimation for covariate shift adaptation. Annals of the Insti- tule of Statistical Mathematics, 60(4):699-746, 2008b.
Masashi Sugiyama, Taiji Suzuki, and Takafumi Kanamori. Density ratio estimation in machine learning. Cambridge University Press, 2012a.
Masashi Sugiyama, Taiji Suzuki, and Takafumi Kanamori. Density-ratio matching under the Breg- man divergence: a unified framework of density-ratio estimation. Annals of the Institute of Sta- tistical Mathematics, 6455):1009-1044 2012b.
Baochen Sun, Jiashi Feng, and Kate Saenko. Conrelation alignment for unsupervised domain adap- tation. Domain Adaptation in Computer Vision Applications, pp. 153-171, 2017. 2017.
Zhuowen Tu. Learning generative models via discriminative approaches. In 2007 IEEE Conference on Computer Vision and Pattern Recognition, pp. 1-8. IEEE, 2007.
Eric Tzeng, Judy Hoffman, Ning Zhang, Kate Saenko, and Trevor Darrell. Deep domain confusion: Maximizing for domain invariance. arXiv preprint arXiv: 1412.3442. 2014.
Garrett Wilson, Janardhan Rao Doppa, and Diane J Cook. Multi-source deep domain adaptation with weak supervision for time-series sensor data. In International Conference Knowledge Discovery & Data Mining, pp. 1768-1778, 2020.
Werner Zellinger, Thomas Grubinger, Edwin Lughofer, Thomas Natschläger, and Susanne Saminger-Platz Central moment discrepancy (emd) for domain-invariant representation learn- ing. International Conference on Learning Representations, 2017.
Werner Zellinger, Bernhard A Moser, Thomas Grubinger, Edwin Lughofer, Thomas Natschläger, and Susanne Saminger-Platz. Robust unsupervised domain adaptation for neural networks via moment alignment. Information Sciences, 483:174-191, 2019.
Werner Zellinger, Natalia Shepeleva,
Dinu, Hamid Eghbal-zadeh, Hoan Duc Nguyen, Bernhard Nessler, Sergei Pereverzyev, and Bernhard Moser. The balancing principle for parameter choice in distance-regularizee domain adaptation. Advances in Neural Information Processing Systems, 34, 2021.
Werner Zellinger, Stefan Kindermann, and Sergei V. Pereverzyev. Adaptive learning of density ratios in RKHS. Journal of Machine Learning Research, 24(395):1-28, 2023.
14

---

Published as a conference paper at ICLR 2025
Yongchun Zhu, Fuzhen Zhuang, Jindong Wang, Guolin Ke, Jingwu Chen, Jiang Bian, Hui Xiong, and Qing He. Deep subdomain adaptation network for image classification. IEEE Transactions on Neural Networks and Learning Systems, 32(4):1713-1722, 2021.
15

---

Published as a conference paper at ICLR 2025
Appendix
Table of Contents
Auxiliaries
16
A.
Weight Representation of Bregman Divergences
16
A.2
Characterization of Proper Losses
17
A.3
Bregman Divergence Representation Excess Risk
18
A.4
Characterization of Convex Losses
20
B
Proofs of Novel Results
21
B.1
Key Lemma on Conditional Bayes Risk
................................
21
B.2 Main Characterization of Losses
................................
24
B.3
Characterization of Convex Losses
25
Details on Empirical Evaluations
26
Amazon Reviews
27
HHAR
29
C3
NeniDomainNel
31
A AUXILIARIES
A.1
WEIGIIT REPRESENTATION OF BREGMAN DIVERGENCES
The second derivative d" of the Bregman generator & of Eq. can be weight of the Bregman divergence as follows.
Lemma 2 (Reid & Williamson (2099). Let Φ : R + R be strictly convex and twice differentiable. Then
Bo( = Eane [//² )((0\5.(((=),BBa)))) de
(15)
with
(r c) <c
(r( = (c-r) if<<esh othenwise
(16)
For the convenience the reader, we provide a proof of Lemma based on Taylor's theorem.
b R and Φ [a. + R. Taylor's theorem grants us the expansion (cf. Reid & Williamson (2099 Corollary 7))
Proof of Lemma For any Sg. s
(is) = ((50) d'(so(ss so) 1" do(s, (o)
(17)
It follows that
Bo( Enn
-((0)) B(x))
((0))B(())
Enn
(()6
[/o (1000)
Exaq
16

---

Published as a conference paper at ICLR 2025
Lemma 2 illustrates prioritization of large values 72 ri over smaller ones for increasing weight function o
ds(r2,72 A) do(ri, A)
(18)
for the distance
dø so) o(s) o( - 2 (s0) su)
that determines (3, = R (3) Eq. 18 follows from Lemma and the substi-
tution ((x) == + 12,
de(r1, A) =
11 -1)0" de
(r1 (c)
p(())
(n1 c))
(12-1) A)
8 (p(c)) - =f(c)) (c) de
-
m
(ct - r2) c) de
12
("(c)(r2 c) de
-
(r 12 - A);
where the inequality holds for increasing "" (c- (r2 r)) S (c).
A.2 CHARACTERIZATTON OF PROPER LOSSES
For convenience of the reader, in the following, we provide self-inclusive proofs of all mathematical statements used to prove our results.
Menon & Ong 2016) discovered a connectior between d" in Lemma 2 and the weight in funda-
mental characterization of proper loss functions due to (Shuford al. 1966 Theorem 1) (cf. Aczél
& Pfanzagl (1967); Stael von Holstein 19700), which re-state here following Buja et al. (2005);
Reid & Williamson (2010).
Theorem 2 (hufford Jr et al. 1966
Theorem 1)). A loss function X : f } [0 + R with differentiable partial losses λ1, A-1 strictly proper and only for all 1 E (0,1)
(1) (_(
w(1)) =w(1)
(19)
n
7
for some weight function w. : (0, + [0, oo) such that S w(c) de < co for all c > O.
Proof of Theorem 2 If A proper, then L(n) 17) = inffee(0.1) L(n. 7) which implics that
θ I.
(--))_((0))+++- + x)
Tn i=n
and consequently also Eq. 19
Eq. 19 gives
0
L(n. = =(1 711 + (1)
=n( 1) ( (1 ( = (ñ n)w 7)
(20)
which provides unique minimum L(n, n) inf (0..) L(n, 7) since w(n) 0.
'The relation (c) 1 w (..)) is proven in Menon & Ong
2016 Lemma 4) with as defined in
Theorem
2
17

---

Published as a conference paper at ICLR 2025
Another fundamental characterization of proper loss functions is due to Savage (1971). which re-state here in the notation of (Reid & Williamson 2010 Theorem 4).
Theorem (Savaee (1971). loss function À { {-1. 1 [0. 1] + R is proper if and only if its pointwise Bayes risk L is concuve and for each [0. and rj E (0,
L(n,,
L 7 + (17 7) (7).
(21)
The next proof of Theorem 3 is given by Reid & Williamson (2009).
Proof of Theorem
To prove the first direction, note that the properness of the loss À means that
L(n. n) L(n) == inf L(n. 7)
]E
for all 17 E [0, 1]. Expanding the derivative of the pointwise Bayes risk gives
0 L' L'(n)
L(n.n)
Ōn
01 ((1 n)) (-)((-))))) + n)),,n)
->(-1, n) + (1 11-n))((1111) A(L, (X(()
=-X(-1, + >(1,7),
where the last equality follows from the fact that
L( Pi=n
Oij
Eq. 21 can now be obtained as follows:
L + (1 7 L' (7) -1- + (1) + (1 x(1
= =11 -1- + m)(
where the last line is just the definition L(17, 7)
To prove the other direction, assume that o [0, - R is concave function and
L( 7 = ( + (17 (7))
Taylor's theorem grants us the expansion (cf. (Reid Williamson, 2009 Corollary 7))
p(n) = x(7) + + (/'' @.(1, (c) de.
for de defined in Lemma
It follows that
L(n. 7) Φ(1) - -f (e(n. x(7)
(22)
since o is concave and d= < 0. The integral in Eq. 22 minimized to zero in the case of
f)
A.3 BREGMAN DIVERGENCE REPRESENTATION OF EXCESS RISK
To prove Lemma I we will need the following Bregman identity.
Lemma 3 Meeon & Ong 2016 Lemma 2)). Let Φ [0, + R be twice differentiable strictly
convex and denote by (y) y) == Φ(v) x(ū) - o' (y()(y 7). Then, for all x, y E [0, xi), it holds that
9
(1 + x) de
doo (x, v),
(23)
1 x 1
where (z) :- (1 2) 1+
18

---

Published as a conference paper at ICLR 2025
Proof of Lemma 3
Menon & Ong 2016 Section 5.1). Eq. 17 gives, for any 80. R:
do(s, sp) / Øe so (0)00 de
with do(s, so) = Is cl- [[s (50 - c) < OJ. where [o] is 1 if the statement o is true and it is 0 otherwise. For so 1+h and = 1t. we obtain
dø
x
y
(=) dz.
+ x 1 1+
1 1 x
Applying the substitution
2
Ita
with da
du (1+ gives
dp
ravy
2
0""
du
1 +
1+ 3/
1
1 1+1
1 1
(1 u)²
yAv
Vvvy
2 ml
(1 + u)² du
+ (1+x)(1 +
+ v
22V
1 ul 8"
u
1
du
+
1 u
(1 2)
do (I.
+
where the last line uses
(1)) (1+z)) 21+
(2) 1+ 1+ z
+ o
and (do)" (z) 0"
+
We are now ready provide proof of Lemma I For convenience of the reader, we don't 't just repeat the arguments Menon & Ong (2016) Proposition 3), but we also include the required basis arguments of Reid & Williamson (2009) and Reid & Williamson 2010).
Proof of Lemmal
7
The propemess of l and Theorem 3 jmply
L(7. 5) LL + -7- - L') (T).
(24)
Denoting by nTx) := ply = 1 (x) and following (Reid & Williamson 2010 Corollary 13), the excess risk can be expressed as expected Bregman divergence as follows:
R(f) R(f*)
(assmo [())) - E (a,))~> ~p
Ex-px [((()),((z)) - Exrox L(n(z))
Exwpx (f(=)) + (π)()) - -((()))
L()()))]
Excep [d_ 10 (f(==)
(25)
Note that the Bregman generator - L is strictly convex, since L is concave due to Theorem and it is strictly concave due to the uniqueness of the minimum
with de as defined in Lemma 3
L(n) = minger L(17, Ø)
Following the proof of Menon & Ong 2016 Proposition 3), in the next step, apply the Radon- Nikodým theorem to obtain:
E Ex<ox dd_ [((()), V (-
ExPP (n(x)) (f(()()))) +
+ Enne [d_ (7(a), v v-(((())))
1
2
Ran (8(x)d (((x))) + 1
En (((())
2
Ban (1++3))))
2
19

---

Published as a conference paper at ICLR 2025
Bayes' theorem gives
B(x) p(a y 1)
p(z, y 1)py(y -1)
p(a.y ())x(2))
p( y = -1)
p 1 -1)py(y =) p(a, y = 11pxx(1
p 1|x)
ply 1|x)
77xx)
p = -1 xx) 1- p = 1/2) = 1 1-n(z)
which implies 772))
[xx) 1+((a)
Bickel et al.
2009). Applying Lemma 3 yields the desired result.
A.4 CHARACTERIZATTION OF CONVEX LOSSES
A characterization of convex proper composite loss functions can be stated follows.
Theorem 4 (Reid & Williamson (2110) Theorem 24)). Consider a strictly proper composite loss function e: 1-1,1 + R with invertible link V [0. + R such that (V-1) (z) > 0 for all
E TR and denote by w(n) = 1 (T)).. Then & and 8-1 are convex if and only if
w'
V"
VVE (0,
(26)
Theorem suggests canonical link (Buja et al. corresponds to the notion of "matching loss" studied by Kivinen & Warmuth 1997 Helmbold et al. 1199)).
2005) for convex losses is v w ( which
The following proof from Reid Williamson (0100) reads in our notation as follows.
Proof of Theorem
Let denote by (()) V-) Then, the definition of and Thm. give
[( 2)
== = (1 - n)( (1--n)_ (1(=) (=))) (=) ((
O2
(1 1) 11) (7)) (()() ())
(g(()) 17) w) )
(27)
necessary and sufficient condition for by for being convex for n E {0, 1} that
82
(1, 2)
20, VE.
2=
From Eq. 27
we obtain
(w(a(())<
(g())
11) w(((0) (7) 2 0, Ay e R,
(28)
where
((l(()) (D)) = w(q
Oy
As it is noted in (Reid & Williamson 2010 Eq. 28 can be found in Buja et al. 2005 Eq. (39)), To proceed further, note that Eq, 28 leads, for
1} to the pair inequalities
w w(a()) (T)) 2 ( (w(a(()g (D)) w (((()) (7) (1 (1- (1- (w( (7))
(29)
which holds for all ② E R. Note that Eq. 29 holds for q(-) = and 1 1 q(-) = because q'(-) and w(.) > 0. Without loss of general lity.ality. lit,, in the following,
restrict attention to the set (((0,)) 1)) = (a E TR [q()) + 0, ((x) + 0}. From Eq. 29 we equivalently have
2
(w(a(()) of (0) w ((((T)
2 (0) (
Vy E (( 1));
(30)
where we have used the fact that -9(2) = - - -w-1(0) is always negative, that (0)) g are non-zero for ÿ E ((0,, 1)) and w(-) > 0 for strictly proper loss function É. Using that
[w(g(-))q ()) = ('((())) (o)()) w(g(-))q"()
20

---

Published as a conference paper at ICLR 2025
obtain, equivalently to Eq. 30
w'((())
(0())
-1
Vø E ( 1)).
(31)
( w(g(T)
(7'
By substituting f) = (V) so that
q (7) = (7). we get
1
w/ (((())
w (7)) z -1
AHE (0, 1).
(32)
1
Note that ((((()))
(q (m) (M and Eq. 32 is equivalent to
d'la (6))
1
w/ (v
2
VNE (0,
(33)
1- i]
w1
Observe that g'() (x)) কল- and therefore
= = ( = (V (V())=
V' (V)-
which implies that
-1 "M
VM
(T)
(v (1)
(M (
VM
Plugging this into Eq. 33 which is equivalent to the convexity of e in its second argument, completes
the proof.
B PROOFS OF NOVEL RESULTS
B.1 KEY LEMMA ON CONDITIONAL BAYES RISK
Before proving our main theorem, we characterize the conditional Bayes risk of strictly proper composite loss function e (-1.1) I - IR that satisfies the questioned Bregman representation R(f) R(f*) [B. f). We start with the following technical lemma.
Lemma 4. Let Φ: : [0, o0) + IR be convex and twice differentiable and denote by
((=) = --:00(1- 1
(1 11-27 2).
If it holds that
Exxpx (d_ Loly 1|x), (())),))))))))) 8)) Ex-ox (d_ clccpp 1|x),, ūi]
for all ā E [0, and all P,Q E M (X) with P < Q then there exist 02 IR such that
L(b) = ((b) + cab + c1. bb E 10.
(34)
With u u= p(y = 1|z) it holds that Proof.
E Exxex [d_ [d_ ū) Exxox [d_(u, ū)]
Ex-px [-L(u) L(ū) L' (u ū)] Exxpx (-( + ((T) + (' 7)
for all P.Q E M (X) with P < Q and for all E [0, 1]. In particular, for = we have
Exxox (((n) - L(u)] =
= Exroxx [s(1) L(1)] + Ex-px L(()))u -1]
--1(((11)-''0))) - Exzupx (5(1) L(1) + Expax [u((((1) II(())] + +Rmpp (-(()))
Exaox (S(1) -L(1) + HE--px [u] (L()) -1(5(11 =c,
21

---

Published as a conference paper at ICLR 2025
where
Ex [4] = Bxmpxpoy [|]]] 2
and
Exxxx (<(1) L(1)+ 1 (c(1) - 1
constant. To summarize the steps above, may state with o = Ç - L that
Ea-pxls(ple = 1x))) =
(35)
for all P.Q E MI(T) with P <.
the following, we will construct probability measures witnessing the affinity s. Let us therefore assume without loss of generality 99 that 0 E & is in the interior of X' such that we can fix some c < 1 satisfying e] C.
We now distinguish between two cases. For the first case, let 1 and define two measures Pr. Q E MM+ by
E (o((x()))....6 ...66(((())
if a0)) E [ otherwise
Px
0
Q(x) 1 60(x(()) 60(x((1) 0
if xf E [0, otherwise
where (10) denotes the i-th element of the vector X and & (x) is the Dirac measure being equal to whenever # Both measures P, and Q are probability measures since
f dP..())= [. S e
dz( d ... (z)()(
and fx dQ(x)) analogously. It holds that
ply 11x)) plaly 1)py(= 1)
px(x))
(x
(x(x) Q(x)
- if x1) [0, +] and 2 =(.)=
otherwise
It follows that
Expapx (p(p(y 1[x))]
/ p(ply- 1|x)) (P,(x) +
1 / s(ply dQ(x)
x
5x
r
dx doo ... (2)()
2
2
30
J
r
+
1
1
+
(x (z)( ... ddo (o)(2(()))
2
Jo
0
r+1
1
(0) doo (d)
2
Jo
r
(0) (1- (1- 2)
6
+
5
+ 2
2
r
r+1
2r
r+1
r
2h
((b)
(0)
2b
2h
We always choose some arbitrary hut fixed inner point E X' and shift all our constructions by choosing instead of
22

---

Published as a conference paper at ICLR 2025
for b = +. E 1 1] with b in the limit. As consequence, get from Eq. 35
f(b) = b(2c 2p((0) + +
(36)
for all b 12.1] 1].
For the second case, let 0 S r E 1 and define the probability measures
do(x:(2)) ... (a((1))
if af) [0,
(x( =
2-r 60(x(2)) ... 60(x((1)
x( E [
0
otherwise
2-r -g(x(()) ... 50(x((1)
if xf [0,
Q(x) =
do(x!(2) ... d0 (o(
(2) E [2
0
otherwise
holds that
a
[0, 5 and 2x =...
= x
ply 1|x)
1
a [ c) and x(2) =... x(d)
0
otherwise
which gives
E Expox [p(p(y
1[x))]
4 p
r
2
+
r
4
40 (1
5)
2
7)
+
2
4
2
1
+ F
2
-
(1- 2
1
(1 2)
2
+
2
(37)
Applying Eq. 36 with 5 =
2 E 2 1] gives
((6) 4 ((1 - 7) (11 ") (2c 2p(0))
which yields, together with Eq. 37
2 2
1
+
1
(2c (C-22((0))+ 70(
2
2
(2c - 20)()
(1
+ +2p(0) 2c - (00)
2
2
r
= (2c 22
(r
{- - + (2c - 20(()) - (0)
2
r
8
2 2 (2c (0)( + (0))
Choosing b: _ E [0, [) proves, as in the first case, that y is an affine function:
((b) b(2c (0) =0))
(38)
The statement of the lemma follows for c₁ (0) and C2 = (2c (0)().)
With our technical result Lemma aabove, are now able to characterize the conditional Bayes risk of loss functions satisfy the questioned Bregman representation R(ʃ) - -R( = BB g o f).
Lemma 5. Let 8 1-1,1] R + E be strictly proper composite with invertible link function
v-1(m) (m)
V
J0, 1] + R and denote by g(ū) = I- (3)
If there is a strictly convex twice differentiable
6: 0, x0) IR such that
R(f) R(f*) Bo( P.QQeM P L
(39)
then
v- V (
L( 0
1- v (1- v
v-
+ (T)
c11
VØ R
(40)
for some 02 R.
23

---

Published as a conference paper at ICLR 2025
The following proof of Lemma 5 has three parts. The first part inverts the constructions developed in (Menon & Ong 2016 Appendix B), the second part is to apply (Reid & Williamson, 2010 Corollary 13), and, the third part is to apply Lemma
Proof. From Eq.
39 and Bayes' theorem, we know that
2(R(1) - R())) = Bo(3,g0 =x (6 V- (((z) )-))))))))))))))))
1 V- 1 (-(((()
Applying Lemma with Φ0 = 2 == - vf
(f((a) and ÿ = v (f(x)) such that
1-2 (f(x))
1+2
Vr -r (x)) and I+v - (f(z) gives
1 - -1-4 v (f 1- v (-(xx) ) -1++ (f(x)))
v (f)
L (f(x))
do
for
((z) = - o 1- (1 2),
where we have substituted = 1 in the definition of (2) in Lemma obtain
(-w) (1 - () 1-u ) -((u),
( (1 u)
which twice differentiable and strictly convex as Φ is twice differentiable and strictly convex. Denoting by u (f" (x)) and ū == v (f(x)) and summarizing the steps above, have
2(R(f) - R()*)) = E [(1 + ((x))d_{(u, ū)]
The transformation used in the proof of (Menon & Ong.
2016 Proposition 3) is the following:
1
Ex-o [11 + ((x)))dc((u.)))
2 Bz-a ((xx)d-c(u, [u, + B [d_c(u, 7)]
2 Bymp [d_ (u, x) + 1
1
Exna (d, ())
2
Exmox [d_c( (1, K]],
which gives
2(R(f) R(F")) Bx-ox cu,(u, ū)]
(41)
At the same time, we know from Reid & Williamson (2010 Corollary 13) that (see Eq. 25 for a proof)
2(R(J) - R((*)) = Expox [d]_ (w, ū)]
(42)
From the strict properness of l we get that p(y 1/x=) and Eq. follows from Lemma 4 L
B.2 MAIN CHARACTERIZATION OF LOSSES
Next, we prove our main characterization result.
Proof of Theorem 4 To prove sufficiency of Eq. g note that the conditional Bayes risk of the loss function in Eq. 8 satisfies
L(n. 9) mT((1, + (1-1))((11,))) = =>(V~ + (17 - ())) ((V)
with strictly concave defined by the strictly convex function Φ. follows from Theorem 3 that é with = 7 - proper composite with link V | + K. Moreover, is strictly proper
composite since L"(T) = - < 0 for all ÿ E R. Applying Lemma I gives
R(U) - R(")
_ B (8
1
B B (B 5 Bo(B,go
1
24

---

Published as a conference paper at ICLR 2025
because (2) x((2) 02 C1, the fact that Bregman divergences unique up to affine terms of the generator and the equality g
(v) (g) which follows from the definition of yr
For proving necessity of Eq. 8 assume & is a strictly proper composite loss function with invertible
link function V/ [0, 1] - R satisfying Eq. 7 From vr ( = I+alū) we obtain g(7) I- (7)
.(())
(x)
That is, all requirements for Lemma 5 are satisfied and it's application gives, for all r E [0, 1].
L(7) --0=--1-000 (7)
(43)
Next, we construct strictly proper loss function in the same way is done in (Reid & Williamson 2010 Theorem 7) and (Bao 2023 Corollary 3). We use Theorem to obtain
L( 7) 7( (v) + (17 - V (7) (T)
for all 7 E [0, 1] and ÿ IR, which be applied since VT 1 (0) E (0, 1) g is strietly monotoni- cally increasing. The form of the loss function in Eq. [ follows now from the definition
L(n 9) = m 7 + ()1-m)e(-11)))
for 1 = and = 1.
B.3 CHARACTERIZATION OF CONVEX LOSSES
Next, prove characterization of convex losses by properties of Φ.
Proof of Corollary 7 Our aim is to apply Theorem In the following denote by 7 = (0)
Applying Theorem with w defined in Eq. 19 gives
L(n 0)
m))(((0)) m (7)
0y
= n ( - w(7) + (1 - 1)) (T)
(m--))-(i))
and, together with Theorem
3 we have
0 0y
w
I 1
11
0
(L(7) (1 11 L(
1 n Dy
1 0
(L'(M) L'( + (7 7 L"())
11 ay
(f)
=-)
where the last equality follows from Lemma
Note that
7
Φ
+
7
1
f)
1
n
1 1)3
("
7
6"
17 1- "1
(M)
1
and
-1
1 1-
(
V
11 77)2 (g 1)
1 1-
(1 7 (o-1)"
2
(g
-1-)²
1
(1- +
(1
7
(1-
25

---

Published as a conference paper at ICLR 2025
which gives
6
w'
1-
10
6"
(1)
1)
-1
1-
7
(7)
2
(7) 11
7
It holds that v/ (2) (x) for R R being strictly monotonically increasing. That is,
( (V- (0))((2) (z) > 0 and Theorem grants convexity of losses if and only if
6"
1-
S 1
Vh
1 6"
1)2
(g 1)=
9
(T)
1)2
1 n-
1-
1-
1-n
Substituting == such that f T4x - = 1+
and
+ x gives
1 1+a σ
d" (z)
(0- (x)
(1 + x²²
x) +
e
(g 1)-)((z)
and consequently
x)+
(x)x)
(1 (1+x)²- (x-1))(x)
(1 + x) 00,
Vo E 10,
(z)
(g ()))
C DETAILS ON EMPIRICAL EVALUATIONS
The computation of the results for the domain adaptation benchmark experiment is based on gradient-based training of overall 9174 models which we built following Grannerrrrbeerrrr et al. (2024) which uses the code-base Dinu et al. (003).. We obtain the number of trained models follows.
AmazonReviews (text data): 11 methods x 14 parameters x 12 domain adaptation tasks x 3 seeds 5544 trained models
MiniDomainNNt (image data): 11 methods x 8 parameters x 5 domain adaptation tasks x 3 seeds 1320 trained models
HHAR (sensory data): 11 methods x 14 parameters x 5 domain adaptation tasks x 3 seeds = 2310 trained models
Following Dinu et al. (023) we used 11 domain adaptation methods from the AdaTime (Ragab) al. (023) benchmark. For each of these methods and domain adaptation modalities (text, image, sensory) we evaluate 4 density ratio estimators (KuLSIF, LR, Boost and EW). For our experiments
we follow the model implementations and experimental setup of Dinu al. (2023) which results in the usage of fully connected networks for Amazon Reviews and pretrained ResNet-18 18 backbone for MiniDomainNet.
For training and selecting the density ratio estimation methods within this pipeline we follow Gruber al. (2024) perform an additional train/val split of 80/20 on the datasets that are used for training the domain adaption methods. The regularization parameter À is selected from (10, 10-1 10-3 3 and the number of iterations of the BFGS algorithm is fixed with 100. We follow Kanamori et al. (2012) in using the Gaussian kernel with kernel width set according to the median heuristic for all compared density ratio estimation methods. Each experiment is run times. To test the performance of the compared methods the classification accuracy on the respective test sets of the target distribution is evaluated. The rcond parameter for inverting the Gram matrix in (Dinu et al. 2023), we used a value of 10-3
26

---

Published as a conference paper at ICLR 2025
C.I AMAZON REVIEWS
Inevstance Wingheed Valdhrson
Impurtnce We Nagphed Ameraation
50
(WVV
(WW)
LREWWN
EW (WW)
Agg)
KaL (Agg)
(Agg)
(W7Ag
TK
B-D 07740122011
0.713
0.722
0.77011111 0771
0.78 0.78211:0055
0.715
005)
0.766:1.0077
0. 0
B--
0
2544 089)
0.78
0083
0077
0.7542.011)
0.7521..00 618
0.258 0123
0. 0.751 0071
0.745:
012)
0.762:1.812)
0. 0.756C 023)
B-K
2692 0211
0.762
0163
0.790+-0001 0111
0.720=-017) 007)
6.771(1.813)
0.068
0.066 085)
0 0.71("0000 0161
0.724
0175
0.761(1.814)
0. 7.47 014)
D-5
0.77 264)
8.78% 9664
0.779)+1009 0091
0.778 (030)
0.782 (65)
0.381 0109
0 0.780 0023
0.78%
002)
0.292/+-00 2011
0. 28 107)
D-1F 0.766 0131
0.7732 0005
0.766(00111
0.764-001 0013
0.772 8644
0.775 005
0.776 0080
0.773-0222
012
0.781 (06)
0. 0.776 185)
D-K 0.777:+0015
0.785 (003
0.
0.000
0.783 (08)
0.78%
0
0768
0 (39)
0. 0.786 0.792(+8811
1009
E-48 0.658::00155
0 021)
0.695220199
0.944_0211
0.70112.009)
0,69::-0200
0.703
0.691=0019)
6.744:2820
0. 2
E-3D 0.714:2:0055
0.717 0043
0.71712.0044
0.725
0.73512.808)
0.715:2-211
0.7262.0077
0.723:2000 013)
4112
0.737
B-
0.859:10004
0.85K = 010)
0.8991.009
0.863 01.)
0.86511.077)
0.861(2-007)
0.8462200131
0 012)
0.873(1.012)
0.865 005)
K K-8B
0713:1.0022
0.710 969)
0.710.2.009) (09)
0.700 (09)
0.70811.014)
0,05511 009)
0.2211..000
0.70%
014)
4.222.1.004)
0.718 NIT)
K-DD 0.741: 010)
015)
8220)
013))
0.748 0211
0.732 014)
0.752 8066
0.74(=
014)
0.751
0.756
K-IE 0,848 005)
0.851 (NE)
0.848:::000
0 (07)
0.8461 005)
0.84% 012)
0 0440
0.847c=001
011)
0.856:=.007)
0.850
mon
(09)
0,96K: 010)
0. 0126001)11)
0.7671 01.3)
6.770(1.410)
0,76K 013)
0 0.773 (09)
0.76% 012)
(.77%:1.010)
(1.))
Table 2: AdvSKM
Ircantune
Valaaron
Imputian: Wighesd Avarcpation
KOLSIE
LICTIWV
OWV)
KOUSIF (Agg)
EW (A02)
0.75K
0231
0-78K 0023
8.13
07764-007)
0.15
6 0179000000
0.785-0005
0.758:
0131
0.770:-0199
0.759
0.75(=0000 0100)
0.763 007)
0:772 014)
0.777=-008)
0.766--017)
-.783
0.775 (08)
8-KK 0.709
019)
0:383 6073
0.783
0.782= 007)
0.783 606
0:777 0-43)
0,77(= (26)
0.78k=-077 (22)
0.293c..013
0.796 01
D-1B
0.786 0100
0:785 0063
0.785 006)
0.785 000
0.7871
(07)
0 0000
07911=001
0.787= 011)
0.791c (005)
D
0
0013
0:77 0153
0.775
0.79C 023)
0.77311 004)
0 (120
079714-008)
0.791c 010)
0.00312.007
0.700/1 (08)
D D-K
0.778::
015)
0133
0.78512.013)
0.789 007)
0.794:1.015 015)
0.803 (120
080112-010
0.79% 013)
0.807110014
0.901
B-B
0.69211: 014)
0:095:1
6.70312.023
0.690
0.63311-013)
0.700 620)
0.7011.023
0.994 022)
0.7191.8011
0.711 (22)
B-D
0.72011 015)
0.723:1 0153
0.7381100121
0.72%
0.718:1.000
0.031 061)
072011.0191
0.205 045)
0.74112.005)
0,7281100 (112)
0.8001
0:857:
0.871
(05)
0 025)
0.881100141
02601 020)
0.988 0.87511.012)
0.706:
0.721
0.718130091
0.2211
0.719:10013
0.09711
022808.009
0.71%
0.76K
018)
01255(00015)
07777
000>
0.630+
01667(+2013
0.714(-012)
(2733
11773
00778(+2009)
07730(20044)
0.775c-0019
0.783(1
Table 3: CDAN
Weighidd
Impootacee Weighed
50
KOLSIFTIWY)
EW (WWV)
B
0.700 D15)
11 (025)
0.7711
0.99CE
0.784:100 1035
0
0.745 012)
0.761:2 0255
0.759L6
0.761C=.025
0.770:1 005)
0.73% 025)
0.759
0.756
0.78122.011
0.780V1
B
0.763:1-015 015)
0.756 048)
0.77361.012)
0.755
0.78711.013 (13)
0,743 085)
0.792120011
0.79X=0012 012)
0.796...012
0
01000
0,78411
0073
0.79311.007)
0.7831 (07)
0.79311.000 0071
0,715 115)
(022)
0.78K 010)
0.8771.0077
0,794/1 006)
0.70811 0031
0,776
016)
0.714L1.015)
0.7831
0.78411.00 0221
0,792L1
016)
0 0.79211.009 009)
0,795 01.)
0.8661.011
0,798/10015
D-K
0.717L1 0121
0,787:1 0113
0.777L1.012)
0.79112002 (27)
0.80211.019
0.757L1 049)
0,29K
016)
0.8411.006
0,811C1.007 (00)
B-8 0.6991
0141
0,00011 010)
0. (866L-099
0.691 01.3
6.713:1.014)
0.685.1014) 014)
6111711.011
069102 01.)
0/712(1.01) 0:2261.0155
B-D
0.723
0081
0.742/:-011 -100)
0. 1220
0.341=0000
0.74211-010)
0.11(11022 021)
0 0755( 0091
0.724=0015) 015)
0.74211.044)
0,738C1 (07)
R-K
0.800+
0131
0.13)
0.85%+ 005)
08871-013 012)
0.850 (06)
0113)
0 0187(+ .00)
08774-010 016)
0.875++001)
0.671C+ 014)
K-18
0.718: 005)
6.725: 011)
(714))-004)
0.734)-866)
0.713 (05)
0.666(+065
0
(199(=042)) (42)
0.734+-001)
0. 0,7400+ (IT)
K-ID 0.748: 003)
6,739
010)
0.73%
0.723 (26)
0.755++.015)
0.75((=006)) 0.69)(20011
11728=00611
0.762(=.03)
0,761 026)
K--E 0.42 005)
0.547:=.015 016)
0.847c2-015)
0844(=.015)
0.855(+.847)
0.54(=-004) 0.812(2-011)
0.867(2.044)
0.858 (05)
06777: 089)
0.775 012)
0.714(1:013/
0.774 019)
0.77911.011)
0.445 046)
0 0.262(2-013)
0.77% 021)
0.791(1.010)
0,787(1-0) 011)
Table 4: CMD
Importance Weighnd Validrson
Importance Weighed Apprepation
30
Boost
LWV
KOLSIFUWW
LICIWVV
EW (WWV
BosstAge
KoLSIF (Agg)
(AAU)
EW (Agg)
TB
B-30 (-33:+0133)
0.79% .144)
(1799=0014)
(7788/72055
(135
0 (179((0011)
11801=100) 1036
0.80IC+
014)
B-E
0.755
6.766 009)
(09)
(.18)
0.764(+-019)
0.763/+-02 024)
0.774(+-000) 220)
0.775=1120)
0.7851=-007)
0.802 00%)
B-X
0.711: 6231
0.817[=1000 (08)
(755)
0.803 (08)
0.798(+-0111
0.793(2-02) 621)
6X11(=-0011
0.507c= 106)
(00)
0.817(+-0077 (07)
D-
0.774 201)
6.777 006)
0.777(8.005) 005)
0.78(c2.01 0113
6.791(1.811)
0.78511-023 621)
0 0080
I.755 014)
(306)3.009)
0.800:2-013
D-46
0
(22921 003
0.508 0223
0. 0.75805015 015)
0.80 002)
681111.818)
0.288.1001
019)
0 0788(5.010 0100
0786
009)
0.877116100
0.817C1-01
D-K
0.782 2211
0.90 041)
08006+10111 (411
0880 (41)
0.823)+.019)
0.208 000)
08000 01.0
na1 016)
0.824- .12)
0. 0.878 (10)
E-48
01687
015)
0.723
045)
6.220
0722- (45)
07181+-035)
0.213 (400)
0.720 (45)
0.221
(08)
0. 0.733 033)
B-4D 01330=
0.721 020)
6.725 4261
01725 026)
0.725:+-026)
0.732
0 0 (07)
0741-=122 027)
0.751(=.200
0. 0.736 012)
E-X
0.89)
0.862/ 014)
(012
01600=001 011)
0.8651--010
0.857++0055
0 0368 0111
0.864 022)
0U87N 008)
0.870 012)
K-B
071122.0023
0.731 022)
6.73
0.708 036)
0.72214.045)
0.735:1
0739
0.72 123)
0.745::
(05)
0. 0.751 016
K
0.73011.022)
0.774 005)
6774)1.005)
0.748 (24)
0.741 0.749L1.0177
015)
0 075415.010
0.7344 121)
0.762/1.008)
0, 0.764 010)
K-E
084511.000)
0.54K 2: 005)
(009)
0.566 (08)
085311.002)
0.855 004)
0 0.001--0004
0.854 - (09)
0.868-1.008 (08)
0,836 & 003)
(118)
0.78 (03)
0.7 017
0.777
0 00000
0.774 (18)
0. 0.10
Table 5: CoDATS
27

---

Published as a conference paper at ICLR 2025
Impertnnce Weighed Volidzson
Importncce Weggted Agprepiiinn
50
Boos
KOL (WV)
LR
EW (WW)
KOLSIF (AgE)
EW LAgg)
16
B-0 (164)
0.78K 0200
7099
(133)
6.7991.000)
0.282(10115)
0 (7750-2000 2044
0.801
B-
0.752/+0011
0.770=0015 018)
011
0.772/-0016) (016)
+.800
0. 0.700=00100
0.747c=122)
0.790)-0044)
0.800 005)
B-K
0.767:+0022)
0.791 020)
0.754
0.7641+.020
0.779)2-0200
0. 0.79812-0131
0.785:=000 0185
0.804 (07)
0.820 104
D-1
0.779:2.001]
0.786 012)
0.754(2:015)
0.787: 907)
0.791:..001
0.785:2-0100
(7571
0.772: 1235
0.806:: 866
0. 6.799K11000 003
D-+5
0.2672 0151
0,802 .099
0.991 (.90
0.78911 006
0,000 022)
0800 (07)
0 0175
0.814
0.81S
D-K
0.7544 0131
0.805 007)
0.000-0007
0 (034)
0.8071 (07)
0.774 011)
0 000000 (07))
0 (18)
0.0231
0.877 (85)
F-48
0.201:+001 019)
6.71%
035)
0714
0714- (15)
0.719 -000
0.71% 018)
7-00
0.11 036)
0.7231--014
6 (.0)
F-D
0.736: 005)
0.727
0.733(+018))
0 (022)
0.736:1:000 (00)
0.718 025)
0 0777(=0014)
0.741= 012)
0.757 6.249=-002))
E-K8
0 016)
0.85% (19)
0.663 0017
0 016)
0.86)
0 012)
0 00
0.367 (18)
(.144)
0 (01))
K-B 0
0.743)=0017 0173
0.721+0017
0.221 (04)
0.
0.72 630)
0.311
0.748 (055
8.7531=.007)
6 (21)
K-2D 0.738:: 006)
018)
0.747120499
0.735 01.3
0.75112:015
0,741 (20)
0.7724400211
0.75k 1844)
0.78612.013)
0.762 (21))
K-B 0.33
0,83 034)
6.8331.0011
0.850 (07)
0.81911 005)
0.549 016)
0 0.5511.0033
0.88%
0.669120007 (007)
0.859
0.780 0135
(1787
0.777 0133
0.772 0.715
0254)
0. 01700
1177X (03)
0.600
Table 6: DANN
Incurtence Wiigheed
Valihrson
Imguetance Waighed Apgraation
30
V
KOLSUP(RV)
LR
LW (WV)
EW (Ag2)
B-
0.781
0.77K 008)
0.79
0.7751=:000) (03)
6.78211.000 205)
0.73011005) 0.77
0.785(20000
0.79011:005
0.736
B-E
0,25211
0071
0.056:3 008)
0.254L1.007)
0.754:_.006)
(06)
0,253.1.013
0,341C1 013)
0. 0.755LL.01))
0,248 015)
0,758 0.7661..11.
012)
B-K
0.78
0.766:1 0151
0.775C 019)
0.77361:0191
0.773
0.794:1.018)
0.96701
036)
0.777th (26)
0.772
013)
0.78211.117
0,750L1 014)
D-48
0.781 0081
0.78% 0003
1-11)
0.781 (07)
0.7791 069)
0.776 023)
0 0.720- 016)
0.778
016)
0.758 0.70)+-011
(09)
D-FF
0.767)+ 0155
0.773/ 006)
0.714(+001)
0.774 (06)
0.776(+.804
0.771(+-014 0144
0 0,767 009)
0.72
(170-0018) 018)
(02)
0.778 010)
D D-K
0.783 01
0.784:-.010)
0.763(+-013)
0 013)
0.783(+-014)
0.771(+-681)
0.78 0.781++008))
013)
0177
0 005)
E-88
0.693
015)
0.65 014)
014)
0665(= 014)
0.691(+.819)
0.05)
0 0.70(=-0077
011
0.702
025)
0,704(+-015
B-D 0.725 086)
0.737 028)
0.727(a.0117
(.74K=.024)
0.7361 026)
0.737(+.024)
0.742(=.07)
0,7366 0.757(2.011)
025)
B B-88
01157
0
11:00:2000 0363
0.860:2.0122
0.26
(880)12.055)
0.541(1.64))
0357220010
0.833(2.009)
0.864/2.
(099
K K-
0.713:1 002)
0.716 904)
0.716:2.001)
0.713(2.0005)
0 023)
0.7161.011)
0.709
6.72211.066)
0,719K
(05)
KD 074111.015)
0.745
025)
6.15411.816)
0 016)
0.737L1.009 019)
0. 0.747110188
0244620044
014)
0.75122.011
0,748(1.0119
K-B
0,51111 001)
0,54K 010)
(004
0.8491
010)
0.84611 005)
0.645 015)
0.8411 005)
0343 014)
0.877L1.006
0.80ML1.11
0,705
0123
1,770 0115
0.70
0,703
0.7721::014)
11 015)
0.79
Table 7: DDC
Ircantance
Valchraon
Impurtan:: Waphasi Appecptiion
KaL (TWV)
LICTSVV
EW (WV)
Baal (Ags)
Kot3IF(Agg)
LIE (Ags)
B-D
0.778:10115)
0.700 011)
..11
0.78%==004)
0.28111 (09)
0.08KC1000 (000)
0.295=0005
0.7911.009)
0: 0.296
8-
0.749 0131
0.758 015)
0.255/+0195
0.255- 015)
6.7611-8811
0.748:+-000 008)
6.754(-001 01
0.79x 071)
6.762+-.814
0.0670+
8-
0.700 011)
0:775 0155)
8.17
0.744-0022) 0175
0.773 021)
0.068
020)
6.783(-.818)
(.77)c-000
0 015
0.77%
D-18
0.733
0099
0:780 0033
0.777(+-007
0.78K=0003
0.780 012)
0.775 025)
0.201=-007)
0.783
0.786 6.78h=-00)
D-16
0.760 013)
0.762 014)
0.771(e-011 011)
0.760= 014)
0.775 608)
0.7671
021)
0.711
0.76k= m15)
0.784:--043)
0.785
D-1K
0.750:
0.780 016)
0.77%is.015
0.7800 016)
0.78911-817
0.761:
054)
0.7214.0277
0.774: (23)
0.797
0.789 017)
B-B
0.600:1 0099
0,095 0083
6.70311.000
0.70211 020)
0.710 017)
6.71112.011
0.700=0033
0.70912.00 0211)
0.708/1
B-D
0.722 0091
0.731: 0083
0.72611.015)
0.734
0.72911.000 006)
0.735/1 0083
6.74512.044)
0.744-0005
0.74012.003)
0,739L110000
B-K
088911
0101
035911000 0100
0.82911.011
0
0.86111 (09)
0.841:1
044)
0.700
0155)
6.87411.008)
0.86211.005)
K-1
0.720L1
0101
0,721:1 0083
6.71111.08)
0,7211
0.71711.000
0.729
006)
0.731210111
0.725
6.75711.000)
0,740L1.005)
K-D 0.733
0111
0,734:1
6.74511.015)
0.05511 005)
075002.0181
0.251220122
0.76h11.010)
0,051C1.000
0.841
0854(1
01766
0,76KK
0.700
0.778
0.77%(-014)
Table 8: DCoral
Importance Weighed Validrson
Importance Weighed Apprepation
30
Boogt
LW
KOLSUFUWN)
LICIWVV
EW DWW
BasttAge
KOLSIF (ARS)
LK (AAK)
EW (Agg)
BBD
0,796
014)
017001201111
681912:011
B B-E
(75611.113
0.77M 022)
0.76612.011)
0.743(2.014)
0.753L1.010
0,770L1 (07)
(777L1.004)
(118)
0.777(1.012)
0.804L1.0055
B-K
0.7701.00 013
0.772
0313
0.79611.020
0.78912.0123
0.7911.820
0,745L1 049)
0.79811.0051
0.78k20017)
2.99)
0.815L1.1055
D-6
0.77 008
0.79K31
0133
081001.017)
0815
081111.01 (16)
0.06711.00 042)
0. 0.796(1.019)
0.78K 0200
1.06
0.611C1.D17)
D-
0.757 020)
0.7622 0325
0. 1191
0.78 012)
6.778(1882)
0.776 035)
0 0'280 .00)
0.764 078)
0.81111.066)
0.812C1 196)
D-K
0.77
0161
6.530
015)
-.15)
0.22 .15)
08144+-100 (01)
0.622 006)
0836 0022
0814 0075
0.828--0000 .10)
0.878 010)
E-48
0.200
015)
0 025)
0 0.160
025)
0.697 (010- 1)
0.601C+-00 014)
0 0.087=0017)
0
0. 0:200 (42)
B-4D
07760
(05)
0.715 0143
0.715(=0014
0.715=002 014)
0.718+--010
(7040+-010)
0731
0 0185
47221
0. 0.730 (01)
E-
0113:
0.871:
0209
0.82
0 020)
0.879L:-019)
0,848
0.66
0 031)
0U89): (05)
0.859 004)
K-B
0.00:2.012
0,699
037)
0.719)2:015
0.719 01.0
0.71311 8.73511.420)
0.3515.0099
0.724
1275
0.7451:.009 (00)
0. 0.72341 (07)
K
0.7611..113
0.750
011)
0.74911.009)
0.74 005)
0.74811.022 (02)
0,704 025)
0 0.4555.0055
0.724 0095
0. 0.736 (00
(07)
K-B
0842/1.009) (09)
0.557
012)
0.15
01853 (1.)
0.86111.0199
0,85 0053
0 0.001150000
0,857
- 0035
0.865X--013 0.753)..005)
013)
0,715
0.30
0.700
0.77 01.5
0,700520101
1.7.4 (166
Table 9: DIRT
28

---

Published as a conference paper at ICLR 2025
Impertnnce Weighed Volidzson
Importince Weggied Agprepiiinn
50
Boos
KOLNIF (WV)
VR
EW (WW
KOLSIF (AgE)
LR LAgA
16
B-0 0-799+-003
0.701 000)
0.794-77051 7081
0:285(40013)
0 (0700 2036
0.794 0006
B- 0.753/+4005)
0.766 022)
0.766x-022)
(02))
0.788
0.03)))
0.760 023)
0.780x=127)
0.797(=-005)
0. 0.796 012)
B-K
0.769
6.77%: 033)
(.76)
0.78K(-033) (31)
0.753 027)
0.795 026)
0.0001-0133
0.799c=1050 036)
0.111==0000
0.816 (85)
D-1
0.782:4:008 (08))
0.784 0053
0.78% 0019)
0.784k-0000 005)
0.79011.806
0.782/ 027)
0.7051800777
0.703: (39)
0.799/: 008)
0.802 2 (07)
D-+5 0.731: 2021
0.78K 018)
0 19111-0141
0,001 (090
0.8011 409)
0.298 0111
1790 009)
0.99K (055
0.808:1000 (0.3)
0.5131 100)
D-* 0.756 0131
0:800: 002)
0.881 0177
0.8081
0.657 277)
6 7700
0.784 013)
0.810
D16)
F-48
0.200:+ 0211
0:710:--010)
22440+0119 0199
070K4 (07))
206 (08)
0.71% 022)
-.734--8881
0.777-1000 (134)
0.719 01.5)
0.711 (10)
F-DD 0.725 2077
0.736:-015)
0
0 (26)
0.731 019)
0.74%/+-015 015)
6.241-.014)
0.74k (144
00744==000 0111
0.732 (19)
E-48
0
0:600 025)
0.87
0157 011)
0.876:5.006
0.675 006)
0874
0.871c= (07))
0.865:=009))
0.876 (00)
K-B 0716::-016)
6.72%
020)
0.713+-014)
0,730 (20))
0.7211::000
(21)
0:71 (00)
0. 07291-0113
0.70K (13))
0.74M)2.0044
6
K-ID 0.739:1 023)
0:738 016)
8.7442.817)
0 017)
0.74311:000
(20
0.741 027)
0.7611.0019
0.744 (025)
6.778(2.013)
0.76214
K-B 0.84014-0100
007)
0.85411.0071
0 011)
0.858L1 (099
0.852 000)
0 0.5812.004)
0
0.85711 (04)
0:775 014)
0.775(+0013)
0.777 0100
0.733
0.009
037)
0.783 0177
0.790/200111
Table 10: DSAN
Ingartance Wiiglase Vallbraon
Ingurianc Waglnnd Agt--gaiio
80
Boost ([W)
KaLSIF (IWV)
(R((WV)
EW (IWV)
(ogg
KoLSIF (Agg)
LR (Agg)
EW (Ag2
TB
B-D 0.798:--010)
6.77k 011)
6.751
0.78x=0013)
0.279(1-0133 (781(1-013)
0.790
0.78% 0133
0. 78 (09)
B-4F
0.245;+ 014)
0.75k 018)
017)
0.240-017) 0117)
0.755'+ 014)
0.347(+-000) 010)
0.700 (04)
0.75x (12)
0.761(+.11)
0. 0.758 (17)
B-K
0.770 018)
63766 017)
0122
0.76%X 022)
0.774(+.814)
0.067 (400
0. 0.770 003)
0.767c-053
032)
0.779 614
0. 0.776 016)
D-1
0.755: (86)
03760 0003
0.778(+006)
(01)
0151
0.855 025)
0.750==0006 006
0.7764 012)
0.78% (671
0. 0.787 107)
D-1E
0.772:+.0113
03367 0044
0.75%++011
0.768K (06)
8.76%:5.805
07681=0010
0.763 (065
0.777
201
0. 0.77% (044
D-K
0.783:2.0155
0:311:2 000)
6, 9122)
0.783 007)
0.78413.0123
038-41 001)
07500+20211
0.775 120)
0.787 048
0. 2066
B-18
0.702:4.0251
0:6872 017)
01777
0,68K 010)
8.69111.8201
0:9771-000
0101
069411.000
0,69 (14)
0.69%
0. 0,700-- (200
B-D
0.733:1.0013
0.723 013)
0.73211.0099 0099
0.725 (09)
0.73511.813)
0739k 0057
073411.010)
0.724 (06)
0.734:..099
17755-1013
B-*
0.8621. 0091
0,85 0083
486211.809)
0.551 905)
0.899110005
0.4311 0241
08061110077
0.86112000 011)
(883311990)
0.863 (94)
K-+0
0,215L1.000 (08]
0.71K 000)
0.100 905)
0.
0.20811003)
0,20711 022)
021811.000
0.10X = 016)
6.72311.000
0.71211 (95)
K-+0 (245L1.011)
0.745
000)
0. 22101:005)
0,2611 005)
0.76611.00 000
0.788
022)
6.748 .091
0.734 2: 005)
2.2611 (02)
0.748LL D1
K-+6
0844 (09)
0.54K 008)
0 011
0.84611 010)
0.899 016
0857( (06)
0.84% 0.9)
0.88811.007
0.649C1 (0T)
0,79
0,705
010)
0,700:20111
0:00
0.76111.011
0,702
0213
0.7711220133
0.765
013)
0.77612.0091
0,77
Table 11: HoMM
Impostace Weighns Valiacaon
Importnce Weighed Aparopation
90
(LVV
KOLSUFOWN)
LRCIWV)
LW (WWV
BoosrLAge!
KOLSIP(AgE
(AgE)
TB
BB
X15)
116
(09))
0
0300
B-E
0.752
065)
0.76M
013)
0.75W)2.005)
0.754(2.014)
0.758:.011)
0.734/1
632)
0.76(220024
(.75))..137)
0.771(2.019)
B K
0.765
009)
012)
6.771:2.0177
0.783(2.010)
0.78411.02)
(211
0.79611001 (11)
0.793220014
0.794.20119
0.79713.013
D -B
0.78311 001)
0.783
0063
6.70611.010)
0.779(2.014)
0.777LL.00 069)
0,789 034)
0.75712.000 (04)
(008)
(4.5)
0,790L1.000
D-1 0.78411
003)
0.76721 017)
O. 0.709LL.020)
0.774(1.89)) 0.774(2.009)
0,771L1 015)
0.78112.010)
0.78M (08)
0. 0,799L1.000 (.04)
(O0)
D-K 0.778
21
0.77722 0053
0, 0.777L3000)
0.77722.005 (05)
6.8111. (07)
0,085L1 022)
0. (12)
0.78122015)
6.81911.006
0.800L1.D13)
E-48 0.201: 015
0.703
0133
O 0099C51005 005)
0.699 (00)
0.699)1009)
0,207L1 (05)
0. 2111 008)
0,692 018)
0. 0.207UE-I13 0.7191.0015)
DI3)
B-4D)
0.758 010)
6.009- 022)
6 1-11
07330- 01:4)
0.729 (05)
0.792 017)
6.74((-889)
0.33c- 072)
0.739 (05)
0. 0.718 (20)
6- 0155:+0115
6.857
011)
0117
01851-001 0)
+.80)
0.85% (00)
0 0 0031
0.864 - (08)
8.8TH--000 (65)
0 06867 (94)
K-B 0.715 (05)
0.710
017)
722))-0921
0.710 017)
8.735
0.726 000)
0 6736 -20
0.717= 005)
(243)+-010)
0. 0.780 (07)
K-ID 0.736:+014)
0.740:=
031)
0 0.740)00011
0.740 (31)
8.247)+.8099
0 023)
0777
0.735 (42)
0.768)--007
0. 7177
K-E 0.421+-0055)
0.511 (00)
0.544+-000)
0.83IC= (08)
0.84314-4211
0.523 (24)
0.83%
0,825 (21)
(U6SM-+-009
0.847 010)
014)
0.12
0.700.00122
0.75
0.7 015)
107
0.78
Table 12: MMDA
C.2 HHAR
Valitation
Aguegation
Boost
KEUSIE
LKGWWV
EW (WT)WW
(Age)
KOLSIF (Age)
(K
DW (WTAAg)
01922120015
0.02011:4411
0.82122.346)
0.80612.0061
0,806C
0.806120066)
0.86611.066)
0.73711.045)
0.740:1.840)
0.725/1.049)
0.588/10141
0.45511.070)
0.89713.0677
0.482/1
0.49713.000 (67)
0.69711.000 067)
0.490L1.055)
0.88(1.099))
0
0.49611.034
0.574/11
0.790L1.11
0.759LL.0055)
0.79911.00 005)
0.79911. 005)
0.768L1 004)
0,771
0,70811.007)
0.77011.000
0.81811.01
(.822L.0055)
0.850L3.025)
0.85015.002
0.50LL (25)
0.771LL.04 (4E))
0.785
0.785L8.025)
0.790
0.595C1.010
0.71862-344)
0.729G8.025 0281
(255
0.72915.025
0.729 025)
0.6821 (42))
(.63)
0.654)1.034)
0.679/3
0.771/
Table 13: AdvSKM
Imporaaee Walgueed Voliaaion
Importaee Weishied Aggecation
S0
Boost (WW
N.V!
LKJWWN)
EWIWWN
BoostAgg)
KOLSIF LAgA)
LK
LW TARS
1966
1-16
0.742
2010
0117
(1942)-0011)
031215-0011
0.141+5.011
0.799
064)
(182
0.807(+-0511
0. 0361
0.946
(00)
2-17
0 (011
8.0001
8411
(.564421064)
0.53313 029)
0.562/3.015 019)
0.490 0144
0.5422
0.507/4.045 044)
0.567
0661
060441
014)
0U77060:0069
(669)
GUROTELL000 0043
0.77IL.0009
069)
0.801E8 0044
0,80113 004)
0.8031 0633
0.815(1.035)
0.84411
(489)
0.987VL 006)
4-1
0.899 0141
0U875LL.0077
00871
0,504
180)
(875)1.087)
0.7511.007 0877
0.800L-0100
0.870X 800
0.88214.000 0221
0.876/4 026
0,992/ A. (002)
2443]
6.799 8351
1800
0051102011
00300
0.719
0.755 (63)
Table 14: CDAN
29

---

Published as a conference paper at ICLR 2025
Imporinee Weighed Validation
Imporance Weighted Aggregation
Booet V
KILSIF V:
LK
EW
BesottAge
KOLSIPTAGET (Age)
[K
16
6
0:74
6.73k=1821
6.731+28211
0.597((-229)
0.558;:20091
0.572147566
0.736
1-
(+:85)
0.868)++00233 023)
+.023)
0.86%)
0.868:
0.703:+-000)
0.844
0.857++-015)
0.861/2.810)
0.02 (94)
2-77
0.573
0.577(+.0111
(.58%)..109)
0.577t=.0111 11
0.577/+-011
0.63))+.084)
0.591(=.155)
0.699:00411
0610
0.686440066
0.799
8.79)
0.790X
0799
0.7991..016
0.114:10055)
0.79422.050
(330
0.790/ 036)
0.532/4.000 0.00
4-
(.84))
(633)
0.799X 1443)
0.799L8.1455
0.541
0631
0.796 (45)
0.79212.043 2.43)
0.809:4.051
0.89911
0.5611 024)
0 01748 (06)
(3 (1990)
0433
07577+ (453
0.768 1 (027)
0.744 078)
1.722% (J441)
0.702
Table 15: CMD
Imporranee Waigaeet Valmaton
Imporance Weisheed Asgropatin
50
Rrsat 15W5
KEL.SIFOWY
LKOWN)
EW VW
Buol (Ag6)
KaESIPFAAn
EW LAR0
T6
1290
0.65912
1-46
0.77312..911
(189612.0699
1.85%
0.89618.0000 (009)
2.96)2.869)
0.839220177)
2.1222
0.835:2.017)
(999
2-+7
0.52714..001
0.47214.0199
0.47215.019)
0.47212.019)
0.501:20100
0.515(2.029)
0.891:2.0099
0558/ 0:901:1000
(63)
1-18
(0.0)
0.785
0.76913.0101
0.78911.010)
0.79811.002) (02))
0.79X..044)
(.80611.008)
05987 (02)
4-15
0.827LL.009
0.85138.094 (94)
0,87MC1.099 (999))
0.8611L.064)
(84911.074)
0.8661 003)
0,85 (.00)
0.83714-025)
0,83211 025)
0577K J. 005)
0L7100 (6:3)
8.741
00739 0741
0.751(2.15))
0.721
Table 16: CODATS
Importanee Weighte Validation
Impomance Weighted Aggregation
50
KILSIF DVV
LK(W))
BecsttAga
KOLSIFTAGET (Age)
LK (KUg)))
LW (Agg)
0
0.78(+2000)
7,0655
(L737(7.827)
0.717(+2077)
1-
00220+-00041
0.833(=.190)
0020)=0004))
+.929;+.004)
0.839:+0013)
0.849 0.89=-0389
(499
2--7
0.5911=:0033
06189+20029 0209
0.586)2.090)
0.626:20088)
0.997 030)
0.606::0066)
0.15(1.060)
0665 025)
3 2-
0.70112.0000 6380
0,9090 2.008)
079618.0311
0.796: 033
0.772 064)
0.0052.0013
0. (7961).0377
0.89711.025)
0.983/1 (RE)
4-15
0.866650000
(022)
6.92214..881
0.91944.1776
0.9211..775
0.92214.000
075)
0.85% 945)
0.800081135)
(.85)
0,859/1 (50)
0.960V & (00)
01757
2011
0.79XC4-01)
0.700 207)
6.7911-400
0,765(+2097)
0.767(1-340)
Table 17: DANN
Imporiance Waggane Valdatoon
Importace Weishted Asgropatio
50
Bnset (WW
CALSIFOIWY
LKOWN)
EWTIWV
Bisot (AgR)
KoLSIF (AYS)
LR1Ag)
EW (AXS)
166
1451
1-16
0U875 4)
0.85611.8671
0.844)1008))
0.85612.007)
(077)
0856:2.007)
0.84211.040)
0.82K2..137)
0.83612.042)
044
10000
2 2-17
0.4671L.016)
1.46
1054)
0.80513.0731
0.438:3.054)
0.5011.:445
0.66L1.029)
0.44311.0811
0.53X (07)
2-14
0.115
0,300
1.09)
0.8773L.055
0.22113.055
025)
0.79011022)
0.800(1.035)
0.80611.025)
0202 1. (292)
4-15
0.292LL..00 600)
0.77PCL-03
033)
0.2901..000
0.7921..000
0.00
0.81411.064)
0.70711 000)
0,288 A (.55)
0L7100 2016
8.71315.8811
UL70IC2 (40)
0.713122423
0.1000.0555
0.700122035
0.7350
Table 18: DDC
Importance Weighted Volidation
Imporance Weighted Aggregation
Boost DWW
MILSIFWWWY
LKWW
EW (IWV)
BoostLAgg (Agg)
KOLSIF (Aga)
LK
LW (A39)
76
0
0.200
0.374(42208)
0.566/41990
0.7000 +0000
1-16
0.8641+-0011
0.864(5.0177
0.864:+.017)
(785)+-032)
2.846))
0.700:+-006)
0.7891+40244
2--7
(799)
(.487(2.0119
0.499)..879
0.409:::000 079))
0.4911::000 081)
0.51814.035)
0.515/1.033
3-
0. 0.79912.016)
0 (195
0.7902110183
0.79913.0133
0.7991..013)
0.74312:0133
0.737(2.00.3)
0.745/1.0000 006)
0.7EM11.005)
0.000
4-1
0.86214.006
0.80413.0055
0051
0,584 1.03)
0.8681500531
0.80811.00 065)
0.7281 145)
0.771C5.074)
6.74414.054)
0.78911.0411
05900-10000 000)
2019
1.74444-1555
01748 (016
6.748:1-866
0677 7544)
0.687:1-811)
Table 19: DCoral
Imporincee Weigoned Valdatoon
Importnce Weighted Aggregatio
S0
Buset W/
KaL.SFF
LKOWW)
EWTIWW
Binol (Ags)
KoLSIP (AMs)
LK(Agg))
EW (Aug)
76
D-
1-66
d (1
0.756(-206)) .68)
1]
0.814+-0066 (066
6.1611.8111
(1879)+.164)
0.814(+-059 (486)
0.814 (66)
0.768:+0075)
D.RR 1:144)
11784-00922
0.610/8.1197
0.785(+0040)
0.717
0000/+1000
(044
2-
0.531(+0057)
0.573(+-065)
(16377---172)
0.573/+0085)
0.579)+0085
0.509 (67)
-.542)
0.504)4-0033
003)
06080 (044
0.807
(000
0.807)+.000 008)
1L866C--1100
0.8077 008)
0.807 001)
0.832(+0081)
0.836(-.151)
0.855(4005)
1001
11/4000000
4..
0.590+00066
0.850+4-006
016)
019821
0.859
08391--016)
0.8611+-034)
0.857(5.849)
0.8544 020)
0.845 000)
000641- (044
Sicus
d ULYER12.000 2200
U.S14 (.00))
0.750..000)
0.74713.1077
0.515 (220)
Table 20: DIRT
Imporiaee Walgueed Valiiaion
Importaee Weisheed Aggecation
S0
BOOsE OWW
KILSIFTIWY
LKUWN)
EWIIWV
BoostLAgs LAge)
KOLSIFLAGE LAgA)
LK
LW TARE
1913)
11357CH.078)
0.57213,105 105)
03572/8.105
0.633:3111)
1044
2-47
1-
0L4800H.00533
0.89413..300
0.466-11000
(1894(2.155)
0.43228.0121
0.908:20655
0.47618 009)
0.001 065)
0.8774-072)
0.497 020)
0814(=.0813
0J89314.0000
0000
00000-10000 0255
0.586610065 0833
0.040-1
3-48
0 0.79762.016)
0.357LL.091
081662.0100
081612.0100
0.888110144
0.823
0080
0.83214.0101 0000
00866110004 (044
0.982/ L. 010)
4-5
1436
0.84413.1221
0,799K 0500
0.8441..122
0.4414.122 1221
0.22 022)
0.332L 0188
0.826 005)
0, 0.822144000 000)
0.992/ A 005)
01404
0023
00711
1.736C (337
0. 0.7157 (094)
0. 0.731
Table 21: DSAN
30

---

Published as a conference paper at ICLR 2025
Importnee Weigned Validation
Imporance Weighted Aggregation
Boost V.
D:
LK
EW
Bisot (Ag8)
KOLSIFTAgeT (Age)
LK
EW 1A39
16
6
0:301C+0000
6.15(+.819)
0.583(4.139)
(5561+1691
1807
1-
0.749)+-00600 0.767(+.110)
+J882)
0.749)=-060)
0.749): (66)
0.771(+0012)
0.75k+-0355)
0.774(+-006)
0.774(+.085)
0014:40000
2-77
0.5286#: 1631
0.857
0.516(2.119)
0.550ts.079)
0.59::2.87)
0454
055)
0.49(220045)
0.691220000 (66)
0.871+-0677
0.546:2.271
3-
0.797
0.89512.010)
0.770X 2.116))
0.8051200001
0.805
0.729 080)
0.763(2.049)
0.755/2.063)
0.792:1.445
0.511/4 (22)
4-
(022)
0.8441220161
0.84412.0161
0.64412.016
0.8161 037)
0.82212.0400
9400
0.822 036)
0.83111
0.940 (400)
0.799+22097
2077
(1799)(2241
0.728 (07]
0.7788 207)
0 (611
0 9.466 14465
0.783/1-8377
0.793
Table 22: HoMM
Weighted Volidation
Weighted Aporesation
Boost
KOLSIP
LKOWW)
EW
Beost
KOLSIF (AgE)
LK0X69
EW LA32
8,09912::8511
00699::0055
0:571:5:7661
(.397(=:0125)
0:57410:0000
0.63612.0377
0.700==099)
0.7321+
+.722/+00099)
0.73215:000)
0.73214:0009
0.75714-0300
0.751==0010
0.7381:
0.768
0:914 (28)
0733
0.54962.0511
(171)
0.54913:0722
0.53014-0711
0.9921:000 (90)
0.515(2.048)
0.89612.083)
0.531:2.035)
0.552/
0.80212.0181
0.902120015
0.80212-0151
0.779:10090 (90)
0.801120011
0.99112.875)
0.796/1_000
0802/ (05)
0.8531
0.84612
L003
0.8601 000)
0.4611 066)
082611.016
0.82412.014
0.82411
0S4TVLL
0.72
6.7585-0044
0.650
0.710
Table 23: MMDA
C.3 MINIDOMAINNET
Impociance Welgieed Nossition
Impustance Weigined Apsropation
5O
Bone (WW))
KoL SIFTSWYS
FWOWW:
AapstAppa)
Kal (IF
(Ap2)
EWTAgo
T6
R- 0.54k==0177
017,
ILS4K=
0:566:=0177
0:56K(0011)
665561+:017
0.503
R-11
0.380x=-00
018)
0.394
0250
0.394:20221
- 0.394/+-21)
0.483:2-0180
0.400=0021)
0.39564.005 005)
0.395/+ (20
R-
0.723
(09%
0.7105.0035
0.711)20015
0.11112.015)
0.71212.017)
0.707
0.0002.0211
0.715620009
0.7064.0221
0.723/1
009)
R-00
0.3223
0223
0.340LS.000
004)
0033
0.34912.004)
0.34011 004)
0.310 0155
0.31712.0113
(315
0.31541.013)
0.335/1 094)
R-+5
0.575K
2 0173
0.58665.001 (011
0.586:10001 001)
0.58611.001
0.58611 0013
0.5921
2 000)
0.592L 4000
0.59
0.58814.00
006)
0,992/4 0099
0.309X
(014)
0.314(50144
0.115 2014)
0.315 0125
0.513
0.311(-0015 013)
0.355/10115
Table 24: AdvSKM
Emportance Weigieed
Importaacee Weighee Agsragaton
Boos (TWN)
KAUSII
ERTIWV)
DooRT/AgE
LRCONEE
0:53TCE: 1000
03344250008
(130
0-38662 0100
0.35765000 0275
0014
0.357LL.027)
0.37411-00 015)
0.32K
(383)
0.383020013
0.38M(1.007)
0.586/1.000
0.000 - 0223
0.08K22.00 01:43
0,708811
0.70811:025
0.71011 923)
0.77(1 (20)
0.71412:013)
0.719
0.7091L.000 (25)
0.71661.055)
0.338
(098)
0.338/11.000
0.33811:000 (08)
0.33811 0083
0.351C1 016)
0.35212.013)
0.35002.023
0.35161.000
0.44D13.114)
0-226K1
9273
(008)
0.59W.1011)
0.59911.011
0.59911 011)
012)
0.60
0.000-0015
0.00811.01
0.06
1C1 (555)
D.S14
0133
0311E5
10.3)
OLSTOTE:001 2011
(133
Table 25: CDAN
Impostanc: Weigited Vaistatin
Impostanc Wagglnd Appigation
S0
Bong (TWV)
Kal.SIP (IWV)
(RIWWW)
EW (IWV)
(ogst(Apg)
Kul SIF(Ag))
(.gg
FW (Agg)
TB
0.555 012
0.557C..033
0355110011
0,56
03255120255
13411=2000
0341118077
009)
0.395
0.388110000
0.38211.009)
0,379(1.00) (07)
021)
0.90811.024)
0.406:.000)
0.397(1.029)
0.389/ 032)
0,104 0209
0.727V 02.3
0.72261002) 023)
0.72711.035
0.727L1 023)
0.74% .144)
0.240(1.018)
0.331.0011
0.23911002)
0.733L. 014)
R
0.332 0155
0.3854 038)
0,358L1 035)
0.35511.009 (05)
0.358 0355
0.316
0.312L1.0400 040)
031512.455
0.3661.046)
0,378L1 605)
0.578 1 0225
0.592 005)
0.00))
0.8921..005
0.5921 405)
0.590
019)
6.599(1.01))
0.59H=.0r)
0594)1-015)
0,960C1 025)
0.505K
0225
0.325 .02)
0,502 023)
0.500/1 021)
0.517C 025)
0.566(+025)
0.519 -029)
0518)+2881
0,542(+ 021)
Table 26: CMD
Impoctance Weipited
Noldiaion
Impastance Wegiindd Aparogation
50
Hone (WWV)
NOWS
LRTWWN)
EWOWWV
Kol SIFSIF
(Rgg)
FW (AAO
TR
0.S5%
(9:45:015)) 1015)
0357:2200)
0357
0.506
0.35T(0044)
0L500
205
(054)
0.377(--015)
0.36)(+.044
0.377
0.36
0.368:20255
1.36)
0.371++.83
1124
(020)
0220
0.22311.200
0.72311.020)
015)
0.1
7.2664 0000
2 0223
0.32%:
0332%:00023 (233
0.32811
023)
0.329 0.32911.023)
023)
0.359
2 022)
0.35715.0155
0.364
020
0.34111.44
948
0447140025
01897C2
0.5881-.012 0023
0.568/
0.58811 0.58811.012
012)
0611 004)
0.01111.000
0.08 006
0U614 008)
0.027 005)
0.500
- 0330
0.314C5 1033)
031 2051
0000000000
0.920
0.3)
03506
0332--0011
03 0.560
Table 27: CoDATS
31

---

Published as a conference paper at ICLR 2025
Irpprtiance Weightdd Validatoon
Imprrtince Weighed Apprepiiion
50
KOLSIF TWW
LKUWY)
BW
BoostAgg
KILSIF(AS)) (AKP)
LK
15
R-C
0.51W-0115
0133
0L382 .017)
0157
0.56X-T003
8.563(7.8051
015574-11000
R-1
0.5764=000
016)
0.379==.076)
02558:+-025
0.399+.886)
0.38)
+233)
0.379::-025
0251
0.38k-006)
(1.2)
0.400 025)
P-P
0.705
0113
0.699550000
(006
0.703:+-01)
0.200=-0011
0.7
0.7122-000 004)
0.777
(.719-0000)
0.7124
006
013
R-
0.3000
8433
(3:5
0.35
0.33312.015)
0.329:2-0133
0
023)
0.315:2.025)
(3316-0022)
(22)
0.392 005)
R
0.50
0149
0.5966..002
4044
0254
0.5911.0004
0.596 624)
0.60
0100
0.607--0113
0.615=0001)
0.6016+-00
01.1
0155
0,490
019
(310
0.31
0.51 2011
0.51511 1053
031302
0.510:1:0111
0.314500100
0.3131..0141
0.35%
Table 28: DANN
Empoctance Waigieed Noncaton
Impoctance Weighed Apsropanion
50
Bonc (TWN)
GOLNIFTIWYS
LKUWWN)
BWOW
KILNIFFAAD)
(A
0.555E
1155
(.531:2.013)
005
D.56XC2
637764
034)
0.355
0148
0.379X..000 (009)
0.57
0.379L32/0091
0.38612.00P)
2.297(2-017)
0.394:2.025)
0.398/20020
0.39462.033
04166:0014)
0.0000 10000:1100 0233
8.70865-0223
0.70761-013)
0.70015.012)
0.706:2-0123
0.7772 (02)
0.71714.0022
0.715620003
(03)
0.72411.005) 805)
0.71711
(23)
2 1003
0:333 (955
0.335K11005)
0.33313.005)
0.333L1-000 0003
0.323 023)
0.32411.022)
0.334 022)
0.32411.105
0.377
(04)
0.500K
2 0223
0.587LE 0099
0.58% 0099
0.586LL.000 (09)
0.596LL (05
0.58K 000)
0.57%L..009 .099
0.577 008)
0.5714.000
005
0,58% (.02))
D.STON
(010
0155
03115/00000 (010)
0.308
Table 29: DDC
Impoctance Welpited Valsiaion
Impustance Wegined Apsropation
30
(INN
KOLSIF TWW)
LKOWN)
EWOWY.
BooRLAgA
KOLSIFIARD (AR))
LK
LAgg
115
0.55% 1845)
0.565C+10055 1985)
0.599
0.54K(T.805
(.553)7.809)
0.00 1880
8
R-+1
(011
0.377(+-015)
8.719:2.011
0.373/+-007)
0.799:2-099)
0.573/+0007)
0.330 003)
-.37%
0.728:00066
0.37)/--810)
0.37%=0011
0.734:2000
004)
0.376c 0055
0.729:+0011
0. 0.774 014)
0. 0.586 107)
R:
0:3366::100
1941
033666 10233
0.963
0.363:2.023)
0.363:2.0233
0.334
026)
0.33314.021)
0.337(2.029)
03321120022
0,388 (02)
S
0.569K
2 1200
0.5555.002
00:43
0.585 1 (.14)
0.58511.014 0.585L1.014)
014)
0.11 014)
0.60811.0100
0.60%
909)
0.611:1.002 9621
0.00/ (33)
0.505
- 11383
0-524 00123
13144
05311 00.3
00502 0155)
0.500-001
0155
0.500(00051
0.521-0013 01.33
(522111.015
01 0:554/
Table 30: DCoral
Impoctance Waighted Valdiaion
Imprctance Waighted Appropation
50
Bone (IWN)
KIWSIFTIWV
LRTWWN)
EW (IW)
(aE3)F(AA))
L. (Agg)
EWTAgg
R-CC
0.530x20013
013)
0.538:2.000 (005)
0.595C1-051)
(.56))
0.541(1-025)
0.574(1.022)
0.568C 0261
0.566=0006
0.569:1.015)
0.58208 087)
R-11
0:373+
0533
0.367+ (198)
02579(+-021)
0.379)+-821)
0.560((-01) (11)
0.392( 046)
0.621((-812)
0410x-00:3
(000))0000))
0.411+ (45)
R-
0.704.
081)
01667+ (190)
(.663(+-830)
(.663)+)430)
0.71 10110+000 025)
0.712(+.828)
a71lc-006)
0.700+-0000 (00)
0.755++0055
R-0Q
03340=0002
020)
0.326 -886)
(99)))+999)
0.3261=-00 (86)
0.256:+-080 (45)
0.323--0120
(3833(00028)
0.315-0022 022)
03316 009)
0. 0.583 027)
R-.S
0-550x=
028)
1.5664 (008)
0.566c+ 098)
0.5661
(06)
0.569 (-44)
0.589 (333)
0.592
(199)=002))
0. 0.548 0.592/+.886
033)
micus
0,49%
2: 1220
0.50011..39 0390
0045 0040
0.310 0510
0.310 (20)
0.319120211
0,311
Table 31: DIRT
Impociance Welgited Nalsiaion
Imgustance Wegiindd Apsr-saaion
50
Bone (WW))
KOLSIFTIWY
FW
Kal SIF
FW (AgO
TR
0.55%
1.555
1942)
0.5555225 1039)
0.568:5.020
0.37AC2 (100
00566
(331(=10m)
130
8:
R-+1
0
1
8-21165-023
033-65.847)
07711-1011
0.341(+-847)
0.11(2.012)
0.711:2.012
0.383:2-021)
0.733(20133
0.35x=20155
0.368:2-0099
0.723:-0011
0.37K-014)
0.726=2000
004)
03365+200777
(1166110000
(000
0:333::
0123
(3836:.000)
0.177
0.33212.0111
0.339K20000 0.392/4.013
016)
0.388)1.004)
0.336620088) 008)
(.999
(13)
S
0.577E-
0264
0.58755.000 (00)
0.587
0 0.587L1.0033
003)
0,607L 015)
0.60015.012)
0660N6.0022
0.0014.000
069
0.0RK 052)
= 1:20
11 0.504 10303
0500 2000
0.336
0.31K=0005
0.50)(00141
0.555-015)
6514/11010
Table 32: DSAN
Impariance Welgitad Valiiation
Imgustanc Weglnsd Apppgation
30
Bome (IWV)
KOLSIF (JWY)
LRUWNN
EW (IWV
BoosttAge)
Kol (Agg)
(Agg)
EWT.Ag
R-CC
0.582x
041)
0.568 .862)
0.56Rct-D67) (67)
(.588(1-06)
0.546(1-04) (41))
0.574(1.014)
0.579'+.017 0171
0.57<=0019
0.522:1.022)
0.0070 022)
R-11
0.97%
0143
0.376
(021)
0.991)1-136)
0.550 0.570[--0155
027)
0.98X-044 044)
0.0000 069)
0 0.398 (04)
0.41311.009) (09)
0.41 30 (4
R-
0.70%
+ m26)
0.714 (005)
6714(+.885)
0.714)+.ME)
0.525
018)
8.724(=.015)
0.724-0014) 014)
0.723+40044 014)
0. 0.721 (03)
R R-
0.3820
= 800%
(L365(=.014)
01345)+-04)
(.35)(e.014)
0.355
0.325
010)
0.331(e-0099
0.324+.0000 0100
0133
R-..
0.570k
022g
0.5764 (063
0.576-+-806)
0.57815.806)
0.57615-066
0.599 (12))
0.908:--115
01 0.596=20022 002)
0.5966
012)
0.6994-0013
sicus
0.500% 2: 0233
0.318C5
033161 2000
0.211
(19)
0322
0317701.014
Table 33: HoMM
Impoctance Weiglted Nolsiaion
Impastance Weggined Aparqgaiion
50
Bone (WN)
KOLSIFTIVVS
LRTWWN)
EW [WW)
(apst(Apa)
Kol SIFSIAAA
(Agg)
FW (Agg
TR
0.S5K
0.50 2011)
0.5820 (135
6393
2011
0.385 7:54
2-+1
100% 400%
1200
0.398
0:380:.044 (440)
(010)
0.700
0.57
0.379)..6300
0.20912.900
0.70913.010)
0.579:5.030)
0.72
0.38%(21099)
104)
0.374/2.013)
0.732:50017
0.380--000
0.732 0085
0189
073311.900
067
0099
0465
(07)
0:33322
02002220000 0153
03353:.025 025)
0.353/10255
0.2531L.025
0.35311 025)
0.343 0133
0.341::.0123
0.3462.009
009
0.4211
020
0.99214.018 (18)
0.3784
2 0224
0.8005 (00)
0 096)
0.58811
012)
0,00K
2 010)
0 8.60011.000
0.601 009)
066044
0099
0,00%
( 005)
0.500
- 1723
0.322
(030)
D11)
03181 008)
03510 0000
0.550 0125
0.35K 106
20.5
03 0.546000177
Table 34: MMDA
32

---

Published as a conference paper at ICLR 2025
33


Reviews:
[{"review_id": "YZ9ICVmgiU", "reviewer": "Reviewer_FZmy", "summary": "The authors characterize the set of loss functions that, when used in density ratio estimation for binary classification, lead to the minimization of a particular Bregman divergence. This approach is motivated by the observation that some commonly used losses (such as exponential or logistic) yield density ratio estimates that minimize a similar Bregman divergence expression. After identifying these losses, they design a new family of losses aimed at accurately estimating large values of the density ratio, in contrast to standard losses that focus on estimating small ratio values. They apply these designed losses to estimate density ratios in Gaussian RKHS and for unsupervised domain adaptation.", "questions": "1. Could you briefly outline the most significant theoretical contribution of your paper and the challenges involved in achieving it?\n\n2. Could you explain the intuition behind the flatness of your method in Figure 2? If the method aims to estimate the ratio accurately for high values, shouldn’t it follow the top of the curve closely? Furthermore, in the lower row of the figure for $\\alpha=0.01$, I am not sure I understand why one would prefer your estimate over KuLSIF.\n\n3. Since you mention that standard methods prioritize estimating smaller values and you focus on higher values, what happens if one applies standard methods to the inverse ratio, i.e., estimating $dQ/dP$?\n\nTypo: In footnote 2, the next-to-last equation contains probabilities that should be conditioned on $x$: it should be $\\rho(y=1∣x)\\rho(x)$ instead of $\\rho(x,y=1)\\rho(x)$.", "limitations": "", "rating": 6, "confidence": 2, "soundness": 3, "presentation": 2, "contribution": 2, "strengths": "1. The paper appears to be self-contained and introduces all the tools and notation it uses.\n\n2. The experimental results are fairly extensive for a theoretical paper.\n\n3. The theoretical results, especially Lemma 4 in the appendix, seem to rely on non-trivial applications of several previously established results.", "weaknesses": "1. The presentation of the paper could be significantly improved. The paper uses heavy notation — for example, understanding $B_{-\\underline{L}^\\circ}(\\beta,g\\circ f)$ requires substantial effort. Another clear example is Remark 1, which is completely incomprehensible unless the reader is already familiar with everything it covers.\n\n2. The novelty of this work is unclear. I believe the authors would agree that the main contribution of this paper is theoretical and primarily represented by Theorem 1. However, given Remark 1, it is not evident how substantial this theorem’s contribution really is.\n\n3. It is not clear why one should focus on minimizing equation (1). According to the beginning of Section 2, there are \"many\" density ratio estimation methods that lead to minimizing (1), with four examples provided. What does \"many\" mean in this context, and why should one limit themself to this specific type of minimizers?\n\n4. Some parts of the experimental results, like Section 6.2, contain too much irrelevant information for readers, making it easy to miss the main points. I would consider moving some of this information to the appendices.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "lRRVap2gZ2", "reviewer": "Reviewer_1wrz", "summary": "The authors present theoretical results characterizing strictly proper binary loss functions that lead to minimizers of Bregman divergences in probability density estimation. According to these theoretical results, they propose a novel loss function that prioritizes accurate estimation of large density ratio values over smaller ones. They also empirically validate the effectiveness of their proposed loss function through numerical experiments, demonstrating that the novel loss function can lead to improvements in parameter selection for domain adaptation tasks.", "questions": "- Considering major weaknesses 1 and 2 discussed above, could you provide more additional discussions to clarify the novel contributions of your study?\n- Considering major weaknesses 3 and 4 discussed above, could you provide further detailed information to elucidate the effectiveness of your approach discussed in Section 5?", "limitations": "", "rating": 6, "confidence": 2, "soundness": 2, "presentation": 2, "contribution": 1, "strengths": "References to existing research are sufficiently provided.", "weaknesses": "#### Major Weaknesses:\n1. There are concerns regarding the novelty of the theoretical results presented in this study. Specifically, results such as the necessity of Equation (8) in Theorem 1 appear to be easily derived from findings  in prior work referenced by this study ([1], [2], and [3]). A detailed examination of this issue is provided below.\n2. Additionally, the canonical form of the density ratio link, given in Equation (10), does not constitute a new result, as it can be derived from results presented in prior studies ([1]). A detailed examination of this issue is provided below.\n3. The proposed loss function, derived from Equation (11) in Section 5, lacks a clear connection to the theoretical results previously established in Section 4.\n4. Moreover, it is unclear how the proposed loss function specifically addresses shortcomings associated with existing loss functions. The authors are encouraged to include mathematical analysis, such as theorems, to clarify the properties of the proposed loss function.\n\n#### Minor Weakness:\n5. Figures 2 and 3:\n    - The axis titles are missing, making it difficult to interpret the graphs.\n    - In particular, Figure 3 lacks explanatory labels for each axis, which are needed to understand these experimental results.\n\n\n---\nHereafter, details of the major weaknesses, specifically Weaknesses 1 and 2, are discussed.\n\n#### About major weakness 1:\nEquation (8) can be derived as follows:\n1. From Theorem 4 in [2], we know that $B_{\\phi} = B_{\\phi'}$ for any $\\phi'$ with $\\phi'(y) = \\phi(y) + c_2 y + c_1$, where $c_2$ and $c_1$ are constants. This fact implies that terms such as $\\hat{\\eta} c_2 $ and $c_1$ in the definition of $\\gamma(\\cdot)$ (line 236) are redundant.\n2. From Theorem 4 in [1], we know that $L(\\eta, \\mu) = \\underline{L}(z) + (\\eta - \\mu) \\underline{L}'(\\mu)$.\n3. Additionally, we have $\\underline{L}(\\eta) = - \\phi(\\eta)$ because $\\underline{L} = L(\\eta, \\eta) = \\eta l_1(\\eta) + (1 - \\eta) l_2(\\eta) = \\gamma(\\eta) = - \\phi(\\eta)$.\n4. Thus, $L(\\eta, \\Psi^{-1}(y)) = - \\phi(\\Psi^{-1}(y)) - (\\eta - \\Psi^{-1}(y)) \\phi' (\\Psi^{-1}(y))$, where Equation (8) represents the cases for $\\eta = 0$ and $\\eta = 1$ in this equation.\n\n#### About major weakness 2:\nFrom Corollary 3 in [1] and the discussion in Section 6.1 of [1], it follows that $(g^{-1}_{can})' (c) = w(c) = - \\underline{L}''(c) = \\phi''(c)$. There appears to be no significant difference between Equation (10) and this equation.\n\n---\n\n[1] Reid, M. D., & Williamson, R. C. (2010). Composite binary losses. The Journal of Machine Learning Research, 11, 2387-2422.\n\n[2] Reid, M. D., & Williamson, R. C. (2011). Information, Divergence and Risk for Binary Experiments. Journal of Machine Learning Research, 12(3).\n\n[3] Menon, A., & Ong, C. S. (2016, June). Linking losses for density ratio and class-probability estimation. In International Conference on Machine Learning (pp. 304-313). PMLR.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "Lv3hhPHWiW", "reviewer": "Reviewer_6SXE", "summary": "The paper addresses the challenge of estimating the ratio of two probability densities from observations. Typically, this is done using binary classifiers, but the efficacy of the estimators is significantly affected by the choice of the binary loss function used. The authors characterize loss functions that result in statistically favorable density ratio estimations, particularly focusing on achieving low errors in large density ratio values—a departure from classical approaches that perform well on small values. They introduce novel loss functions and demonstrate their application in parameter selection for deep domain adaptation tasks. Numerical experiments and real-world applications illustrate the practical benefits of these loss functions.", "questions": "1. Is the existence of the density ratio always guaranteed? For example, if the two densities have no overlapping support, in which case, the definition seems to fail.  Can the proposed method perform a good estimation?\n2. If one distribution has a light tail and the other a heavy tail, how would that impact the estimation?\n3. It seems the author considered only a one-dimensional case in the experiment, how about a multi-dimensional case when $d>0$? This is more common in covariate shift problems.\n4. What are the biggest difficulties and challenges for deriving the the sample complexity of the proposed methods?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. Compared to the related literature, this paper introduces a new framework for constructing novel loss functions, prioritizing an accurate estimation of large density ratio values over smaller ones.\n2. It provides a thorough mathematical foundation, characterizing the types of loss functions that align with specific error measures derived from Bregman divergences. The comparison with the related literature is good.\n3.  The work shows large practical implementation through empirical data and real application in deep domain adaptation. The simulation work is extensive to demonstrate the effectiveness of the proposed method.", "weaknesses": "1. The paper does not delve into the sample complexity of the proposed methods, which could be critical for understanding their efficiency in various scenarios.\n2. While it improves estimation for large density values, the impact on performance for smaller values isn't thoroughly explored.\n3. A more detailed introduction of the experiments should be considered.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "X8BPkzKxhY", "reviewer": "Reviewer_Ps3k", "summary": "This work examines the estimation of the Radon-Nikodym derivative, which is the ratio of two probability densities. In classical algorithms, an incorrect choice of binary loss function can lead to biased estimates. The author first derived the necessary properties for an appropriate loss function. Based on this analysis, novel loss functions were proposed, demonstrating improved parameter selection in both simulated and real data examples.", "questions": "1. Theorem 1 extends previous results. What is the main challenge in extending these, and what is the key novelty in the proof?\n2. Table 1 shows that EW consistently performs best for Amazon Reviews under Importance Weighted Aggregation. Is there any intuition behind this outcome?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "The theoretical analysis appears solid, and several real datasets are used to demonstrate the proposed loss function. The authors provided detailed comparison between new results and previous analysis.", "weaknesses": "Although the author discussed how their results improve upon previous work, they did not elaborate on how their proof differs from prior analyses. Consequently, the challenges of the proof, as well as the novelty and contributions of the theoretical analysis, remain unclear. Additionally, the writing could be improved; including more high-level explanations of the motivation and results would make it easier to follow.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "l6IUwipRuJ", "reviewer": "Reviewer_55K4", "summary": "A standard technique to estimate the ratio between densities of two probability distributions $P$ and $Q$ from samples from each of the distributions is to train a binary classifier that distinguishes the samples. Namely, one labels samples from $P$ with the label $1$, and samples from $Q$ with the label $-1$, and empirically minimizes a loss function on these samples. Depending on the loss function that is chosen, it is known that the as the number of samples tends to infinity, the empirical minimizer effectively minimizes a certain Bregman divergence between the true density ratio and the estimated density ratio.\n\nHowever, the convex potential of this Bregman divergence depends on the chosen loss function. Namely, the Bregman divergence to the true density ratio that our estimator ends up minimizing depends heavily on the loss function that we chose. As it turns out, most commonly used loss functions (like the logistic loss), correspond to Bregman divergences that do not appropriately penalize discrepancies at large density ratio values, instead penalizing density ratio errors at smaller values---this might lead to suboptimal density ratio estimates in many applications.\n\nThis paper takes an inverted approach: given a convex potential $\\phi$ (and a ``probability link function'' $g$), the paper characterizes a unique loss function $l_{\\phi, g}$, such that upon empirically minimizing $l_{\\phi, g}$, as the number of samples grows, the Bregman divergence with potential $\\phi$ is minimized. The paper furthermore proposes a canonical link function $g$, which induces a convex loss function $l$, and is hence computationally amenable to empirical risk minimization.\n\nOne convenient application of the characterization in this work is that it gives a way for a practitioner to design a loss function that would have the properties they desire. For example, as elaborated in Section 5 by the authors, if one cares about penalizing errors in larger density ratios more than errors in smaller density ratios, one can specify a potential $\\phi$ for a Bregman divergence $D_\\phi$ that ensures this, and thereafter using the characterization in the paper, obtain the associated loss function $l_\\phi$. If one now minimizes $l_\\phi$ on the samples, then in the limit, one would be minimizing $D_\\phi$ (which was chosen so as to prioritize accurate estimation of large density ratios). In particular, the authors specifically propose two convex potentials $\\phi$ for this purpose--the Exponential Weight (EW) function and polynomial weight functions, and derive the associated loss functions from their characterization.\n\nFinally, the authors empirically validate minimizing these loss functions as compared to the standard loss functions on a variety of synthetic as well as real-world datasets. They show that minimizing their loss functions leads to better performance on importance weighting tasks on a range of datasets. The experimental evaluation appears quite thorough and extensive.", "questions": "In the conclusion, you mention that the sample complexity of these tasks is not known. Do you mean to say that the empirical risk minimizer of the loss converges to the minimizer of the Bregman divergence only as the number of samples goes to infinity, but we do not know finite-sample error bounds for the empirical minimizer (similar to how in PAC learning theory, this would correspond to an additional \"complexity of F\"/#samples error term)?", "limitations": "", "rating": 8, "confidence": 2, "soundness": 4, "presentation": 3, "contribution": 3, "strengths": "The characterization provided by the authors significantly completes the picture laid out in prior work by Menon and Ong 2016, and work by Reid and Williamson (see Remark 1 for specifics). The motivation for considering the inverted problem is convincing---one can imagine ascribing certain desired properties of an estimator to the minimizer of a Bregman divergence, and thereafter, using the characterization derived in the paper, obtain the correct loss function to minimize on the data that realizes the minimization of this Bregman divergence (and hence has the desried property). The authors specifically consider the property \"small errors on large density ratios\", and obtain strong empirical results for minimizing the loss functions through their characterization. In my view, this is a valuable contribution that enhances our toolbox for estimating density ratios in a principled manner. The writing of the paper is also geneally good, although at some places, it becomes dense with a lot of assumed context.", "weaknesses": "Up until you mention your first contribution, the reader has only looked at the pseudocode of Algorithm 1, which uses a probability link function $\\Psi$. There has been no mention of $g$ as yet. Correct me if I am wrong, but my understanding is that $g(x)$ in Algorithm 1 is simply $\\Psi^{-1}(x)/1-\\Psi^{-1}(x)$---it would be helpful to at least mention this before introducing \"Density ratio link $g$\" in line 78. Because otherwise, the reader, who has just seen $\\Psi$ in Algorithm 1, is a little confused about where $g$ sprang out of nowhere, and how it is relevant.\n\n---\n\nMinor/typos: \\\nLine 78: I believe the loss function for an arbitrary $g$ is not convex, but only strictly roper composite (the loss function for the canonical $g$, as stated in the next sentence, is convex).\n\nLine 273: I believe in the denominator, there is a typo (should be $g_{can}$ instead of $g$)", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)