A Short Survey on Importance Weighting for Machine Learning
Authors: Masanari Kimura
Hideitsu Hino
Venue: Accepted by TMLR ()

Abstract: Importance weighting is a fundamental procedure in statistics and machine learning that weights the objective function or probability distribution based on the importance of the instance in some sense. The simplicity and usefulness of the idea has led to many applications of importance weighting. For example, it is known that supervised learning under an assumption about the difference between the training and test distributions, called distribution shift, can guarantee statistically desirable properties through importance weighting by their density ratio. This survey summarizes the broad applications of importance weighting in machine learning and related research.

A Short Survey on Importance Weighting for Machine Learning
Masanari Kimura
m.kimumm@hnimmelbb.... edu...
School of Mathematics and Statistics The University of Melbourne
Hidcitsu Hino
hino @esm.ac.jp
The Instibute of Statistical Mathematics
Center for Advanced Intelligence Project, RIKEN
Abstract
Importance weighting fundamental procedure in statistics and machine learning that weights the objective function or probability distribution based on the importance of the instance in some sense. The simplicity and usefulness of the idea has led to many applica- tions of importance weighting. For example, it is known that supervised learning under an assumption about the difference between the training and test distributions, called distribu- lion shift, can guarantee statistically desirable properties through importance weighting by their density ratio. This survey summarizes the broad applications of importance weighting machine learning and related research.
1 Introduction
Taking population or empirical average of a given set of instances is an essential procodure in machine learning and statistical data analysis. In this case, it frequent that each instance plays a different role in terms of importance. For example, consider the expectation
(i) [g(x)] = J g(x))(x)dd
(1)
of any function g(x) of random variable æ π(Φ). If we denote the independent samples from (()) by (ar,..., .r.. the expected value to be obtained can be estimated as follows:
f i T Σ g(x.).
(2)
From the law of large numbers, i converges to / when T + co, and such an approximation of the expectation is called Monte Carlo integration. For applications to Bayesian statistics, π(x) corresponds to the posterior distribution. In many cases, the posterior distribution is so complex that is difficult to sample directly from x()). Then, considering another probability distribution q(x),, which allows for easy sampling, we have
1 = / ((x))(())de = / g(xr) q(=:)dx = Eyyz)) g(x) g(x:)
T(x)
π (x)
((x))
Then, by using w(x) = π (x))/{()), we can estimate 1 as
w wxx))g(x1).
(3)
T

---

Distribution Shift Adaptation (Sec. 3)
Active Learning (Sec. 4)
Sample selection for labeling
Importance Weighted ERM
Covariate Shift (Sec.
Model Calibration (Sec. 5)
Target Shift (Sec. 3.2)
Down-weighting well-classified instances
Sample Selection Bias (Sec. 3.3)
Subpopulation Shift 3.4)
Feedback Shift (Sec. 3.5)
PU Learning (Sec. 6)
Domain Adaptation (Sec. 3.6)
Weighting unlabeled data according to positivity
Multi-Source Domain Adaptation (Sec. 3.6.1)
Partial Domain Adaptation (Sec. 3.6.2)
Open-Set Domain Adaptation (Sec. 3.6.3)
Label Noise Correction (Sec. 7)
Universal Domain Adaptation (Sec. 3.6.4)
Weighting as the label noise probability
Distributionally Robust Optimization (Sec. 3.7)
Subgroup Fairness (Sec. 8)
Constructing uncertainty set
Weighting to groups based on sensitive features.
Figure 1: Applications of importance weighting.
This technique is known as importance sampling (Kahn & Harris 1951 2000) and is a useful procodure in computational statistics.
Tokdar & Kass 2010 Owen & Zhou
This simple and effective idea has been adapted in various areas of machine learning in recent years. A similar procedure is called importance weighting in many machine learning studies, and its usefulness in various applied tasks has been reported. Figure [ summarizes the subficlds of machine lcarning for which importance weighting has been reported to be useful. This survey focuses on summariking examples where importance weighting is used in machine learning.
2
Preliminary
First, we formulate the problem of supervised learning. We denote by X' C I2d the d-dimensional input space, where d N. The output space denoted by y C IR (regression) tion). We assume that training examples {( 3f Her are independently and identically distributed (i.i.d)) according to some fixed but unknown distribution Per (x, which can be decomposed into the marginal distribution and the conditional probability distribution i.e., (tr u) = Per (x)ptt We also denote the test examples by ((x)", "" drawn from a test distribution (tc y) Pta Pa)(P))te Here dur E and The E N are training and test sample size, respectively. Let H be a hypothesis class. The goal of supervised I (h E H. with the training examples that minimizes the expected
) C (1...., K} (K-class classifica-
learning is obtain a hypothesis loss over the test distribution.
Definition 1 (Expected error). Given a hypothesis and probability distribution p(æ, y) the expected error R(;;p) is defined
R(h:p) = E(a.y)pte ((h(()))))))))
(4)
where é: R x the loss function that measures the discrepancy between the true output value y and the predicted value == h(x).
In this work, we assume that l bounded from above, i.c., for some constant C > 0, Ely. v\) < C (Vy. e)). The expected error of a hypothesis h not directly accessible since the underlying distribution p(x, y) is unknown. Then, we have to measure the empirical error of hypothesis h on the observable labeled sample.
2

---

Definition (Empirical error). Given hypothesis h H and labeled sample S, (1)),) the empirical error R(h; Sn) is defined
1 Sn; = Σ {(h(a;), vi),
(5)
where R:Rx y - I is the loss function.
The procedure of minimizing Eq. called Empirical Risk Minimization (ERM), known to be useful
learning with i.idd settings (Sugiyama, 2015) James al.) 2013 Hastie al, 2009 Bishop & Nasrabadi] 2006 Vapnik 1999a b). For example, we assume that the labeled sample S is set if.d.. examples according
to distribution p(a. v)- Then, the empirical error R(;; S,)) is an unbiased estimator of the expected error p): follows. Bp(m.)) [R(h:S.) = _ p(a,)) [e(h(x)) v)] = R(h:p) p):
(6)
2.1 Density Ratio Estimation
Density ratio estimation is method in machine learning used to discern the difference between the distri- butions of two data sets. Its objective is to calculate the density ratio, denoted r(x) (tc( (x:); which expresses the ratio of the probability density functions between the test distribution Pte (x) and the training distribution (æ))
Moment matching is one of the most prevalent frameworks in density ratio estimation. This approach focuses on aligning the moments of two distributions, fre(z) = f(x))p. and (x) One common example is matching their means, as shown by the equation:
J f(x))tt = x. Pte (x))x.
It important to acknowledge that matching a finite number of moments does not necessarily lead to the true density ratio, even in asymptotic scenarios. Kernel mean matching (Huang al. 2006: Gretton et al. (200)) offers a solution by efficiently aligning all moments within the Gaussian Reproducing Kernel Hilbert Space (RKHS) S::
f (f(z)) (2)) - / K K(x, .per (x)dx
min PEB
(8)
Beilback-Leibbeee Importance Estimation Procedure (KLIEP) minimizes KT-divergence from (a) to (x:) = F(xx) (x:)
Nguyen et al.
2010 Sugiyama et al.
2007b
Pro (x)
min D (Pt [Ppce = min mm
pro(2x)
da.
((r))pt
Least-Squares Importance Fitting (LSIF) Kanamori et al. 2009 minimizes squared loss:
min / (f(x)) (((()) Per (x))da.
The estimated density ratio obtained in these manners plays central role in the importance of weighting- based methods. When the disparity between the two distributions is substantial, the accuracy of density ratio estimation significantly deteriorates. Rhodes et al. (2020) propose Telescoping Density Ratio Estimation (TRE) following a divide-and-conquer framework. TRE initially generates a chain intermediate datasets between two distributions, gradually transporting samples from the source distribution Po to the target distribution g = Pm:
(o(
(a (x) Pm 2(x)) (2x) Pı (x)
(9)
Pm Pm(
Pm 1 Pm (x)
3

---

Experimental results have demonstrated the effectiveness of density ratio estimation along this chain. Moreover, Yin (0023 revealed several statistical properties of TRE. Featurized Density Ratio Estimation (FDRE) (Choi et al. 2021 considers the use of invertible generative models to map two distributions to a common feature space for density ratio estimation. In this feature space, the two distributions become closer each other in a way that. allows for easy calculation of the density ratio. Furthermore, because the generative model used for mapping invertíble, guarantees that the density ratio calculated in this common space is identical in the input space well. Trimmed Density Ratio Estimation (TDRE) (Liu al,, 2017) is robust method that automatically detects outliers, which can cause density ratio estimation values to become large, and mitigate their influence. Kumagaa et al. (2021) utilize a meta-learning framework for relative density ratio estimation.
As we will see later, much of the discussion about importance weighting in machine learning is strongly related to density ratios. Therefore, one can say that density ratio estimation foundamental technique that supports importance weighting.
3 Distribution Shift Adaptation
Many supervised learning algorithms assume that training and test data are generated from the identical distribution, as p(x,y) y) (tr( y) y). However, this assumption is often violated. This problem setting is called the dataset shift or distribution shift. In this section, we introduce how the importance weighting methods are utilized to tackle the distribution shift assumption.
3.1 Covariate Shift
The covariate shift assumption was reported by Shimodaira (2000) and is formulated as follows.
Definition 3 (Covariate Shift (Shimodaira (2000)) When distributions (er( y) and pte (e( v) satisfy the following condition, is said that the covariate shift assumption holds:
(a) + (x)) Ptr (v(x)) (ullx) = p(ylæ).
Under this assumption, we can see that the unbiasedness of estimator obtained by Empirical Risk Minimiza- tion is not satisfied. This issue is known to be solvable by utilizing the density ratio between the training distribution and the testing distribution, as follows:
Pte (2) ((b(z))))
(re(x) [(((x), v) (x, y)dwdy
E Epcr (mn.9)
Per (x)
xxx Per
Ptr (x)
/ Pte [((((x))))) v) Per (x) drdd
xxx Per (x)
xxx (te(z))(((<z) v) Per (yl dxddy
[ ((h(z),y) (ae y)ddd = (w.)) (e(h(z),))) Jxxy
(10)
Minimization of the weighted loss in Eq.. 10 is called the Importance Weighted Empirical Risk Minimization (IWERM) (Shimodaira] 2000 IWERM is thcoretically valid, but it has been reported to be unstable in nunerical experiments. To stabilize IWERM, Adaptive Importance Weighted Empirical Risk Minimization
(AIWERM) (Shimodaira 2000), which replaces importance weighting with [zt((z) À E [0, 1], has been (pt
proposed, and its ellectiveness has been demonstrated. Relative Importance Weighted Empirical Risk Min- imization (RIWRM) Yamada et al. 2013) is another alternative to IWERM, achieving stability by using (t)(w)
(2)) as importance weights, where À E [0, 1). A comparison of IWERM, AIWERM and RIWERM shown in Table Moreover, it demonstrated that IWERM, AIWERM and RIWERM are generalized in terms of information geometry (Kimura & Hino 2022
4

---

Table 1: Comparison of IWERM, AIWERM, and RIWERM. IWERM is statistically unbiased and behaves well when the sample size is large enough, while it is numerically unstable when the sample size is small. On the other hand, AIWERM and RIWERM gain robustness by allowing for bias in estimators. It is also known that RIWERM is computationally more stable because IWERM and AIWERM include the density ratio in the calculation of the weight function, while RIWERM replaces the denominator with a weighted
average of the two distributions.
Algorithm
Weighting function w(xx
Parameters
Pros (+) & Cons (-)
Unbiased estimators
IWERM Shimodaira 2000
+ No tuning parameters
Numerically unstable
AIWERM Shimodaira
2000
(pte(t=)/ptr(zz)}
A [0, 1]
Robust estimators
Biased estimators
RIWERM
Yamada et al. 2013 Pie (x)) ((1 ()p)r(x)) + (2)c(2))
A [0,1]
Tuning parameter À
Importance weighting is also known to be effective in model selection. vross-validation is an essential and practical strategy for model selection but is subject to bias under covariate shift. We define the error
R(n) = (m.)) [e(hs, (z),
(11)
of a hypothesis hs. trained by a sample Sn of size n. The Leave-One-Out Cross Validation (T.OOCV) Loocv of the error R(n) is
estimate R (n)
[m) LOocY
=
1 Σ E(hsg (a;). W;),
(12)
where Sy
{x;; ) j
is known that, if Ptr = Ptcy LOOCV gives an almost unbiased estimate of the Smola & Schölkopf
risk Luntz 1969
1998) as
Es,
R (n) [Rooov] = R (1-1)) R()).
(13)
However, this property is no longer true under covariate shift. Importance Weighted Cross Validation (IWCV) (Sugiyama al.) 2007a), variant of the cross-validation that applies importance weighting, over- comes this problem. The
Importanco-Weigheee Cross Validation (LOOIWCV) is defined
Pie (te( e( (z))
R (m) R =
(14)
Per (x)))
We see that the validation error in the cross-validation procedure is weighted according to the importance of input vectors. For any j nk), we have
Es,, (pro(2g)) E(hsg ub) = Bs [tr((2,))
Pro (re(())) e(h (mz).
Per (tr))
(te(xe) (( gg (xz)), ys yer (xx, )dx;
Bgi.vr
Per Per( (xx,)
Ess.vs
Pie (te(xx)e((gg) (z) 3y )dz,
=Esi
Ptelar)((h))) (x)) y) Esi.o ((0))(())))) R(n-1) R()).
5

---

Covariate Shift Shift of input vector distributions
Budd
Input x
Hypothesis h
Output Φ
Ground truth y
A:30
Target Shift
Shift of ground truth label distributions
Figure 2: Illustration of two distribution shifts. Covariate shift assumees shift in the distribution of the imput vector a and target shift assumes shift in the distribution of the output
and
ptal(t))
Es, Rioowwev]
Es,
((1 (3))
1
per (2r,)
j=1
1
Es..
Pto (zj))
((h (zz). Uj)
1
R (n-1) R((a1) R()).
Per (xt))
Subsequently, observe that LOOIWCV provides an almost unbiased estimate of the error, even in the presence of covariate shift. This proof can be readily extended to include an aaportance-weightee version of k-fold cross validation.
However, Kouw Loog 2016 reported their experimental findings, indicating various estimates of IWERM consistently underestimated the expected error. Moreover, classical results have highlighted the benefits of importance weighting specifically when the model is parametric. Gooooashvilli et al. (2023) demonstrated the necossity of importance weighting, even in scenarios where the model non-parametric and misspecified.
3.2
Target Shift
In the context of target shift (Zhang et al., 2013), it is assumed that the target marginal distribution undergoes changes between the training and testing phases, while the distribution of covariates conditioned on the target remains unchanged.
Definition 4 (Target Shift (Zhang et al. 2013 When distributions (2. and (tc v) satisfy the following condition, it is said that the target shift assumption holds:
Per (v) + (y); Per (xr()) Pte (xly)) = p(xl)).
Figure shows conceptual illustration of covariate shift and target shift. In cases where the target variable is continuous, additional complexities arise, leading to new research opportunities. Chan & Ng (2005) employ the Expectation- Maximization (EM) method to infer the distribution p(y), which also entails estimating p( y). Nguyen al. (2016) introduce a technique for calculating importance weights Pte (v)/Ptr (y) tackle the shift in continuous target variables within a semi-supervised framework. Black Box Shift Estimation (BBSE) (Lipton et al. 2018) estimates importance weights by solving the equation Cu b through the

---

confusion matrix C of a black box predictor (where denotes the predictor's average output). Lipton al. (2018) show that the error bound for the weighting v estimated by BBSE under appropriate assumptions can be derived as
C X
<Iw||2log 72
(|w
log
klo m m
N wll
(15)
where Amin is the smallest cigenvalue of C, n is the sample size from the training distribution, m is the sample size from the test distribution and & is the number of classes.
3.3 Sample Selection Bias
Although it bears a close resemblance to the covariate shift assumption, many studies also address similar
problem settings under the umbrella of sample selection bias (Tran 2017 Winship & Mare 1992
Vella
1998
Berk 1983
Numerous studies (Zadrozny 2004
Tran & Aussem 2015) introduced a random variable to represent
sample selection bias, formulating the test distribution as:
Ptc(2, y) Σ
(16)
where 8 indicates selection of the example, and its non-selection. Here, the assumption of the sample selection bias is defined as follows.
Definition 5 (Sample Selection Bias). Let (s x) and Pic(s x) be the probability of sample selection given at training and testing phase, respectively. Then, when they satisfy the following condition, it is said that the sample selection bias assumption holds:
(sr x)) + Pie(s x).
(17)
For example, Bayesian classifiers, the posterior probability P④y x:) for discrete y is written as
Ply x) Ply x = 1) P(x y. s 1)P(y 1=
(18)
P( s s= 1)
where represents the probability discrete random variables. Based on these assumptions, they illustrate the efficacy of importance weighting, where
P(s 1)
w(x)
(19)
P(s x)
The following theorem demonstrates the usefulness of importance weighting in machine learning under sample selection bias.
Theorem Zadrozzy 20044) If we assume that P(s = | x.,)) = P(s = 1 x)), then we have
Exr (=()))((((())) I Ete (((h))).))]
(20)
under the sample selection bias.
The key calculation rewriting of the following conditional expectation with importance weighting.
Eyr (w<)))(((()))))))))))))))))))))) v) s= 1] Σ ux) s=
a.,y
= Pio(s Pso(s=1)
Σ
Pia(s = 1
(()(
E.e [((h(x)) v)]

---

This means that importance weighting achieve correction for sample selection bias.
Nevertheless, in cases with pronounced sample selection bias, Liu & Ziebart 2014 highlight that the errors resulting from importance weighting can become excessively large.
3.4 Subpopulation Shift
Subpopulation shift (Santurkar et al, 2020 Yang et al. 2023 in machine learning literature describes a scenario where the statistical characteristios of a specific subset or subpopulation of data undergo changes between the training and testing phases. This implies that the data distribution within a particular subgroup, such as a certain demographic, goographical location, or any other distinct subset, varies between the training and lesting datasets. The formal definition of the subpopulation shift is follows.
Definition 6 (Subpopulation Shift). Assune that the train and test distributions have the following struc- ture:
m
supp(Per(x)) = U S
supp(pic(x:)
Tü i=1
(21)
where
Vi * S = TT -T =0
(22)
Then, when they satisfy the following condition, it said that the subpopulation shift assumption holds: Eie [1,m], (S,) + (T.)..)
(23)
Nurnerous studies suggest. reweighting sample inversely proportional to the subpopulation frequencies CCo et al.) 2019b Cui et al. 2019 Liu & Chawla 2011). As an alternative, UMIX (Han et al. 2022) is a
variant of mixup data augmentation technique, designed to calculate importance weights and enhance model performance in the presence of subpopulation shift. The original mixup training (Zhang et al. 2018a
Kimura, 2021 Zhang & Deng 2021 is data augmentation in which the weighted average of the two input
instances is taken as the new instance:
ر Aay + (1 A)ær
and its objective function to be minimized is given as
((0,)))))) (X((M(a..)) + (1 2))(M(a..))
where æi Fy Mi, Uj are the i-th and j-th input instances and corresponding labels in the training dataset, respectively. UMIX modifies this procedure by using the importance weighting
(( ((( vi) w(ax) ( A)((M((..))
where w(x;)) is obtained by the uncertainty of classifier h.
3.5 Feedback Shift
In the advertising industry, accurately predicting the number of clicks leading to purchases is vital. However, it is widely recognized that time lag exists between clicks and actual purchases in practical scenarios. This issue is commonly referred to as feedback shift. or delayed feedback. Delayed feedback was initially researched by Chapelle (2014) and has since led to the proposal of numerous techniques to address this problem YYoshikawa & Imail 2018 Ktena et al.] 2019 Safari et al.) 2017 Tallis Yaday 2018). Additionally, the concept of delayed feedback has been examined the context of bandit problems (Joulani et al, 2013 Pike-Burke et. al. 2018 Vernade et. al. 2017). The formal definition of the feedback shift. is as follows.
8

---

Definition 7 (Feedback Shift). Let {0, 1 be random variable indicating whether sample is correctly labeled if 8 = in the training data, d E IR be a random variable of the delay which is a gap between a sample acquisition and its feedback, and e E R be a random variable of the elapsed time between a sample acquisition and its labeling. some positive samples are mislabeled (s = 0) when their clapsed time e is shorter than their delay d as
e d + ==
then it is said that the feedback shift. assumption holds.
Owing to feedback shift, there are instances where data that should be labeled positively is incorrectly labeled negatively. For instance, due to the delay in feedback, purchases might not have occurred at the time of collecting training instances. To address this issue, Yasui et al. (2020) utilize importance weighting, termed feedback shift. importance weighting (FSIW). Let C be a {0, -valued random variable indicating whether a conversion occurs if C = 1. In addition, let Y be a {0, 1 p-valued random variable indicating whether a conversion occurs during the training term if Y = 1. Then, applying importance weighting P(C=y|X=2) yiclds w (z;) = P(Y X m
EE (m.a))
P(C P(Y y X = x)
X x) ((h(z))
=E
Σ
P(C X x)
P(( =x,Y= =))
P(Y y X = x)
(E(0.1)
P(C X = =x)
Σ
Σ
= P(X-
X = x))
P(Y X x)
TE (E())
Σ
Σ
P(C = y X (1((())))))(( =xx== =x)
TEA vE(0.1)
Σ
Σ
( c X x) x) E (z,,)) [((h(z)) c)]
ZEX (0.1))
This suggests that FSIW leads the consistent estimator under the feedback shift.
3.6 Domain Adaptation
Domain adaptation (Patel et al. 2015 Ben-David et al. 2006: Wang & Deng 2018 Farahani et al., 2021
Csurka 2017 Wilson & Cook 2020 in machine learning encompasses the adaptation of a model trained on one domain (the source domain) to accurately predict in a different, yet often related, domain (the target domain). Domains may exhibit. variations in aspects such as data distribution, feature representations, underlying data generation procosscs.
We borrow the definition of domain adaptation from the book by Redko et al. 2019
Definition 8 (Domain Adaptation). Let X's x V's be the source input and output spaces associated with source distribution Ps: and Xr x Vr be the target input and output spaces associated with a target distribution T. We denote (z(x) and pr(x) the marginal distributions of X's and Xy. Then, domain adaplation aims to improve the learning of the larget predictive function ly: Xx + Vr using knowledge gained from ps where ps(a,y) + pr (2,
The difference between the above definition and covariate shift or target shift whether or not the conditional identity of the source and target distributions is assumed. In fact, while Definitions 3 and assume that Po-(x (x y) = (x y) or Py x) Pte æ), respectively, the domain adaptation problem setting makes no such assumptions about the conditional distribution.
Significant research focuses on the importance of weighting for domain adaplation. A key concept among these studies is the allocation higher weights to source domain data more similar to the target domain. Loe al. (2018) suggested a technique that approximates importance weighting using distance kernel for domain adaptation. Mashayekhi 22022) developed a method to estimate importance weighting through adversarial learning, in which generator is trained to assign weights to training instances to mislead
9

---

discriminator. Xiao & Zhang 2021 also introduced importance weighting that considers the difference
in sample sizes between the source and the target domains. Aviwadenesheli et al. (0019) established a generalization error bound for domain adaptation via importance weighting.
Jiang & Zhai (0077] employed importance weighting for domain adaptation in NLP tasks. Nevertheless, some studies NL.P have reported unsatisfactory results with domaín adaptation based on importance weighting (Plank et al., 2014). This might be attributed the specific characteristies of NLP data, such word occurrence rates. Acknowledging these limitations, Xia et al. (2018) observed that existing strategies primarily address sample selection bias but fall short in handling sample selection variance. To address this issuc, they proposed several weighting methods that take into account the trade-off between bias and variance.
Furthermore, domain adaptation encompasses several subfields, such as:
Multi-source domain adaptation: involving the use of multiple source domains.
Partial domain adaptation: where the larget domain features fewer classes than the source domain.
Open-set domain adaptation: characterized by unknown classes in both domains.
Universal domain adaptation: which operates without prior knowledge of the label sets.
3.6.1 Multi-Source Domain Adaptation
The objective of multi-source domain adaptation is to utilize the information from the multiple source domains to adapt the model to perform well on the target domain, despite potential differences in data distribution. Sun al. (2011) introduced two-stage multi-source domain adaptation framework designed to calculate importance weights for instances from multiple sources, aiming to minimize both marginal and conditional probability discrepancies between the source and target domains. [Wen et al. (2020) formulated an algorithm capable of autonomously determining the importance weights of data from various source domains. (Shui al.] (2021) also propose to assign different importance weights to each source distribution depending on the similarity between them and the sample size obtained from them.
3.6.2 Partial Domain Adaptation
Traditional domain adaptation methods mitigate domain discrepancies three ways: by developing feature representations invariant across domains, by generating features or samples tailored for target domains, or by transforming samples between domains using generative models. These approaches assume identical label sets across domains. Partial domain adaptation challenges this by assuming that the target domain comprises fewer classes than the source domain (Cao et al. 2019b Liang et al. 2020 Cao et al. 2018b/. In partial domain adaptation, the class space of the target domain is a subset of the source domain class space. Consequently, aligning the entire source domain distribution Ptr with the target domain distribution Pte is problematic. In addition, based on the discussion by Shhi et al. (2021), partial domain adaptation can be regarded as a special case of multi-source domain adaptation. In fact, in multi-source domain adaptation, we can recover the problem sctting of partial domain adaptation by considering the case where the weight coefficients of some source distributions are zero.
Zhang et al. (018b)) suggested that incorporating importance weighting into adversarial nets- based domain adaptation (IWAN) is effective for the partial domain adaptation problem. Domain adaptation methods based on adversarial neural networks (Creswell et al., 2018 Goodfellow et al. 2020) utilize a discriminator
to classify whether input examples originate from real or generative distributions (Long et al. 2018
Tzeng
ct al. 2017). The weighting function of IWAN formulated as:
1
w (x) = 1 D' =
(24)
Por (z)
where D (x=) denotes the optimal discriminator, Per the source distribution, and Pte the target distribu- tion. This weighting function resembles the unnormalized version of relative importance weighting (Yamada)
10

---

Closed Domain Adaptation
Partial Domain Adaptation
Open-Set Domain Adaptation
Universal Domain Adaptation
?
?
Source Label Space
Target Label Space
Figure 3: Illustration of different domain adaptation settings.
et al.
2013
Pte (te(x:)
Pte (x)
1 ('
Per (ar) Pie (x) +1
Pte (x:
Ptr (x: PPee (x)
(ar Pte (
2019a extends IWAN by jointly optimizing domain invariant features and importance weights for relevant source instances. Selective Adversarial Network (SAN) (Cao ct al. 2018ah implements multiple adversarial networks with an importance weighting mechanism to filter outsource examples in outlier classes. Furthermore, an extension of IWAN for regression and general domain adaptation is proposed Ide Mathelin al. 2021).
Example Transfer Network (ETN) Cao et al.
3.6.3 Open-Set Domain Adaptation
Recent studies aim to relax the constraint of having identical label sets across domains by introduiig open-set domain adaptation. Panareda Busto & Gall (0117) incorporated unknown classes in both domains, operating under the assumption that shared classes between the two domains are identified during training. Formallyy the open-set domain adaptation problem is defined follows.
Definition 9 (Open-Set Domain Adaptation). Let V/s and Vr be the label spaces of the source and target distributions, respectively. The open-set domain adaptation is problem that aims obtain good predictor under the following assumptions.
Ys + YT, Y's n Dr + 6.
(25)
Liu el al. 2019) emphasized the significance of considering the openness of the larget domain, defined as the proportion of unknown classes relative to all target classes. Their methodology employs coarse-to-fine weighting mechanism to progressively differentiate between samples of unknown and known classes, while emphasizing their contribution to aligning feature distributions. Kundu al. (2020) introduced a metric termed model inherilahilily to evaluate the effectiveness of model trained the source domain when
11

---

applied to the target domain. They advocate employing this metric in importance weighting for open-set domain adaptation. Garg et al. (2022) further developed method for open-set domain adaptation using
the concept of Positive-Unlabele (PU) learning (Bekker & Davis 2020 Kiryo et al. 2017).
3.6.4 Universal Domain Adaptation
Universal domain adaptation is the most challenging problem sctting in domain adaptation as it requires
no prior knowledge about the label sets in the target domain. Figure shows the illustration of different
domain adaptation settings. You et al. (2019) have proposed the Universal Adaptation Network (UAN).
which performs instance weighting based on domain similarity and prediction uncertainty, Fu et al. (2020)
leverage model calibration techniques to enhance the effectiveness of importance weighting source domain
data based on the uncertainty in universal domain adaptation ion. They utilize multiple indicators of uncertainty (entropy, confidence and consistency) to define the weighting function.
3.7 Distributionally Robust Optimization
Distributionally Robust Optimization (DRO) Rahimian & Mehrotra 2019; Goh Sim 2010 Duchi &
Namkoong, 2018 Bertsimas et al., 2019 Levy et al.) [0200] Delage & Ye 2010 focuses on minimizing the expected loss under the most challenging conditions, considering a probabilistay uncertainty set U(po) inferred
from observed samples and characterized by certain attributes of the actual data generation distribution.
Specifically, the objective of the DRO problem is
minimizenen R(h; = sup (( [e(M(a)).)) qEU(pu)
(26)
where H is the hypothesis class and U(po) is the uncertainty set associated with the training distribution Po- The field encompasses various uncertainly sets (Ben-Tal et al.] 2013 Bertsimas et al. 2018 Blanchet iet al. 2019 Esfahani & Kuhn 2015). with the goal of developing models that maintain robust performance amidst perturbations in the data-generating distribution. DRO can also be interpreted as accounting for a worst-case scenario in the face of unknown distribution shifts.
One way to interpret the worst-case formulation in DRO is as importance weighted loss minimization without specifying target domain. Utilizing set of weighting functions W, the uncertainty set can be constructed as
Uw (po) = (w(x) po(x); e W].
(27)
Duchi & Namkoong (2021) elaborated on the DRO framework by amplifying the tail values of ((h(2)) y)
resulting in worst-case objective that concentrates challenging regions within the input domain X. Awad & Atia 2022) devised an uncertainty set centered around the importance-weighlee source domain distribution, showing its high likelihood of covering the true target domain distribution. As practical optimization method for deep neural network models under data imbalance, a variant of momentum SGD called Attentional- Binsed Stochastic Gradient Descent (ABSGD) (Qi et al., 2022) has been proposed. AB- SGD implements importance weighting for each sample in a mini-batch, benefiting from the theoretical underpinnings of DRO.
Additionally, Agarwal & Zhang 2022 have introduced Minimax Regret Optimization (MRO) as variant of the DRO problem formulation. They have proven that, for both DRO and MRO, the upper bound of their error is determined by the variance of the weighting.
Contrasting with IWERM, which applies weights to the training data using the density ratio Pte (te)2))P (æ), robust algorithms-orige originating from Distributionally Robust Optimization-asiin using (z))
weights the test data Pte (x) (Liu & Ziebart, 20144 2017 Chen et al., 2016).. Martin et al. (2023) introduced Double- Weighting Covariate Shift Adaptation incorporating balance between these two weighting approaches.
12

---

4 Active Learning
Active learning (Settles 2009 Hino 2020 is founded on the principle that a learner, by selecting specific
instances for labeling, can reduce the necessary sample size to achieve adequate performance. Within active
learning, methods vary ranging from those based on uncertainly (Lewis & Catleti 1994 Yang & Loog 2016
Zhu et al. |009]] Li & Sethi 2006), diversity (Brinker 2003] Yang et al.] 2015 Agarwal et al. 2020), repre-
scntativenoss Du ct al., 2015; Huang ct al., 2010 Kimura et al. 2020), ; reducing expected error (Mussmann
et al. 2022 Jun & Nowak 2016), to maximizing expected model changes (Cai et al. 2013 Freytag et al,
2014 Käding et al.]
2016). A particular focus within these is on density ratios and importance weighting.
Importance Weighted Active Learning (IWAL) (Beygekimne et al. 2009) is a pioneering approach that in- corporates importance weighting into active learning. IWAL determines whether to label unlabeled instances æt based on their features and the history of labeled data, with a probability pt. The weight of the instance is then regarded as 1/p, during the learning. The probability pt is calculated using the hypothesis space He, which includes the most contrasting functions f and g in terms of training data accuracy at time t. Specifically,
Pr max max (E(()(((())))))) - e(g(ar)) v)), f.ge4111
(28)
H4+1 (h E Ha; (1)) S D + +
(29)
where (h) and L represents the empirical loss and the minimum achievable loss at time t, respectively, and A, is a small quantity. IWAL has been validated for its consistency and effectiveness through importance weighting. This framework has been reported to be effective in a wide range of applications, including image rocognition (Wu et al.] 2011), point cloud segmentation (Wu et al. 2021), sleep monitoring (Hossain et al., 2015), and battery health management (Chen & Li 2011). Beygelzimer et al. (2011) introduced practical implementations of IWAL. by setting rejection threshold based on a deviation bound, indicating that TWAT. can produce reusable samples. Here, sample reusability [Tomanek] 2010 Tomanek & Morik 2011) examines if labeled sample from one classifier can train another in an active learning setup. Contrarily, Van Tulder 2012) reported the infeasibility of generating reusable samples in IWAL in their subsequent work.
Regarding model misspecification, similar issues have been explored. Sugiyama (0055) found that active learning methods can endure minor model misspecifications and that importance weighting can address larger discrepancies. Bach (2006) investigated the asymptotic properties of active learning in generalized lincar models under model misspeciffiation and proposed efficient instance selection based on importance weighting according to their findings.
Active Learning by Learning (ALBL) (Hsu & Lin 2013) combines multiple instance selection algorithms in active learning using a multi-armed bandit framework, selecting the most suitable for a given task. ALBL enhances IWAL by introducing Importance Weighted Accuracy as a reward function for the embedded multi-armed bandit algorithm.
One common challenge in active learning is the difficulty of performance evaluation. Active learning al- gorithms contimuously select highly informative instances for labeling, leading to labeled dataset whose distribution may diverge from that of the original data. This phenomenon exemplifies sampling bias. Kottke]
ltt al. (2119) observed that reweighted cross-validation, natural extension of IWAL, does not fully rectify these distributional disparities. Zhao et al.] (2021) demonstrate that combining importance weighting with class-balanced sampling can reduce sample complexity for arbitrary label shifts. Mohamad et al. (2018) suggested stream-basee active learning method using importance weighting under the concept drift as- sumption. SS et al. (2020) presented a domain adaptation method merging domain adversarial learning and active learning. In this approach, importance weighting is applied by approximating the density ratio using the output of the discriminator in adversarial learning. Additionally, active testing [Kossen et al. 2021
Farquhar et al. 2021 is a known method for model evaluation using loss-propottional sampling.
Furthermore, in active learning, addressing the variability in annotator quality is paramount. In scenarios like crowdsourcing, where annotators may have varying levels of expertisc, addressing the issuc of annotator quality is indeed erucial. Zhao al. (2012) have proposed a pool-based active learning approach based on the concept. of IWAL, which is robust to annotation noise.
13

---

Calibrated model
Non-calibrated model
CE
CE(p) = log(ps)
(Y= 0.5)
FL()) -(1 p) log(pr)
FL ()=1)
100
outputs
70
oulputs
3
FL (y
80
60
(Y=5)
accuracy
60
2
60
40
well-classifidd examples
40
30
20
20
10
0.6
0.6
0.0
0.8 probability of ground truth class
0.2
0.4 0.6
1.0
confidence
confidence
Figure 4: Left panel: the reliability diagrams of two models. Left one is well calibrated than right one. Right panel: illustrative plot of focal loss. By increasing the focal loss parameter 7, we can see that the well-classified examples are downweighted.
Another challenge in active learning is the issue of computational cost. Agarwal et al. (0013 employed the IWAL framework to parallelize the identification of data needing labeling.
Furthermore, aclive learning and importance weighting can be related from another perspective. When matching the data distribution estimated on unlabeled (pooled) data with the query distribution in the active learning problem sctting such as (hhi ct al. (2020), based Rényi-divergence, the expected value weighted by the density ratio appears and this indicales that importance weighting is still important.
5 Model Calibration
In many machine learning models, output probabilities are often equated with confidence in their predictions. However, these probabilities may not well-calibrated, indicating a discrepancy between them and the actual likelihood of predicted outcomes, For example, a model might assign probability of 0.7 to an event, whereas the real occurrence rate of that event might be approximately 0.9. Model calibration typically involves modifying the output probabilities using various methods, aligning the predicted probabilities with
the observed event. Irequencies in the data Hill 2000 Gupta et al. 2006 Eckhardt et al. 2005).
Vaicenavicius el al. 2019 Beven & Binley 1992:
Focal loss is one the most prominent mothods for model calibration. Originally developed for object detection in computer vision (Lin al. 2017 Li et al., 2020bb Mulyanto el al. 2020 Tran el al. 2019
focal loss has been validated as effective for model calibration in later studies (Mukhoti al. 20200]. Focal loss addresses the calibration problem by down-weighting the casy-to-classify examples (those with high confidence predictions) during training, effectively giving more emphasis to the difficult-t--classfe examples. This is achieved with a modulating factor applied to standard cross-entropy loss. This approach can be seen as applying an importance weighting w(x;) = =(1- p that depends on the predictive probabilities of the model for instance x, and 7 E [0, 1]. Figure shows a graphical example of the model calibration and a plot of the focal loss behavior.
The simplicity of the idea of focal loss has inspired a plethora of variations. Cyclical focal loss (mmthh 2022 is variant of the focal loss inspired by curriculum learning (Bengio et al. 2009 Wang al. 2021 Graves et al. 2017: Hacohen & Weinshall 2019), which applies weighting to each instance based its classification difficulty during each epoch. AdaFocal (Ghosh et al. 2022) utilizes the calibration properties of focal loss and dynamically modulates the parameter for different sample groups based on the model's previous performance and its propensity for overconfidence, as observed in the validation set. Dual focal loss Tao et al., (023) refines focal loss by striking balance between overconfidence and underconfidence. Adversarial Focal loss (AFL) (Liu et al. 2022) employs a distinct adversarial network to generate a difficulty score for each input.
14

---

Theoretical insights into focal loss have also emerged. Charoenphakdee et al. 20211 proved that the optimal classifier trained with focal loss can achieve the Bayes-optimal classifier. Another study (Naganuma & Kimura 2023) showed that focal loss reduces the curvature of the loss surface.
6 Positive-Unlabelled (PU) learning
PU learning, an acronym for Celabile- learning, refers to type of machine learning paradigm where the goal is to build classifier when you have dataset with positive examples and unlabeled exam- ples (Bekker & Davis 2020; Kiryo et al. 2017). The principal challenge in PU learning is the absence of labeled negative examples, which are typically integral to training binary classifiers. In this paradigm, the positive class is well-defined, but the negative class is composed of both true negatives and unlabeled exam- ples. The main objective PU learning is to develop algorithms that can learn a reliable classifier despite the absence of labeled negatives. This particularly useful in situations where obtaining represcntative sample of negatives is difficult or costly.
Elkan & Noto 22008), the pioneers of PU Learning, based their algorithm on the following lemma:
Lemma 1 (Ellkan Noto 2008)). Let æ be an example and let y E {0. 13 be binary label. Let s if the ezample is labeled, and lel if is unlabeled. Then, for the completely unlabeled random ezample P(s=1|=)
æ, we have P 1[2) P(==1ly=1)
We demonstrate the relationship between PU learning and importance weighting as follow. For the unlabeled example,
Pιy 1/2, 0) P(s = 0/2. y = 1) 1/x)
(1 - c) P(s = 1|x)/ 1-c P(s P(s= = 1(x))
(30)
P(s = 0))
- P(s 1
1 1 P(s = 1
where c= P(s 1/y- = =1). Then,
Bey. ()( = =/x P(x) P(s = 1/x)/((x,1)) P(s= 0|) = 1/x, 0)h(ar 1) + P(y = 0|ær = 0)h((r, (o)aa..))) 01))
The plugin estimator of Em,,, [1(2,,)]] is
Σ m,a=1
h(2,1) 1)
Σ w(x)h(x 1) (1-v())))))) (m,am0)
(31)
where w(x) P(y 1 0) 1-c P(s=1a=) I-P(===1 Thus, PU learning can be regarded as importance
weighting according to the probability of labeling unlabeled data.
Owing to its theoretical underpinning, numerous PU Learning studies have embraced this weighting ap- proach (Liu et al. 2003 Du Plessis et al. 2015 Kiryo et al.) 2017]] Chen et al.) 2021 Du Plessis et al., 2014). Gu et al. (2022) propose Entropy Weight Allocation (EWA), which is an instanco-dependent woight- ing method. EWA constructs an optimal transport from unlabeled instances to labeled instances and uses the transport distance to perform weighting. AAcharya et al. (0022) are applying the recently popularized learning framework called contrastive learning (Tian al. setting.
2020 Khosla et al. 2020 in the PU learning
7 Label Noise Correction
Label noise correction (Nicholson et al. 2016 2015 Song et al. 2022 in machine learning involves identifying
and correcting crrors or inaccuracies in labeled training data for supervised lcarning tasks. This approach is particularly beneficial when working with datasets prone to manual labeling errors when utilizing data from various sources that exhibit differing levels of accuracy.
15

---

Among the leading methods for label noise correction, Patrini et al. (2017 proposed technique that uses matrix corresponding to the probability of label noise occurrence to weight the loss function. This method can be viewed as applying importance weighting that downweights instances with high probability of label noise occurrence. Now, assume that each label y in the training set is flipped to ÿ E y with probability P(ūlv), and P(x, Ú) = Eg P(ūly) P(zr). Denote by T E [o, 1] []xx x the noise transition matrix specilying the probability of one label being flipped to another, where K E is the number of classes. Here, we assume that the output of hypothesis h is the vector value as h : X' + R and let & [0, 1j vector- valter loss function. Then, Patrini et al. (2017) utilizes the weighted loss
R be the
L (softmax(h(z))) y) = T ((softtaxx((())))
where softmax(-) is the softmax function. Indeed,
Ezl [((sottmax(h((t)), w)] = Eyle (T((soltmax(((())) = Byle ((softtaaxx(((())))
= Enja (((sfttmxx((()))).)))
and the corrected loss is unbiased.
Since the noisy class posterior can be estimated by using the noisy training data, the key problem is how to effectively estimate the transition matrix T. Given only noisy data, T' is unidentifiable without any
knowledge (Xia et al. 2019). The key assumption of the current stato-of-the-art methods is the transition
matrix is class-dependent. and instancc-indeeeeddent (Han et al. 2018b a Natarajan et al. 2013 Chuang
et al. 2017).
Noise Attention Learning (Lu et al. 2022al) automatically models the probability of label noise using at- tention and employs it for weighting during gradient updates. In a similar vein, Attentive Feature Mixup (AFM) (Peng et al., 2020) combines attention mechanism with mixup, a well-known data augmentation technique, to perform weighting based on the probability of label noise occurrence.
8 Subgroup Fairness
Fairness in machine learning is a research area that deals with the problem of disparity in prediction due to sensitive attributes such as race, gender, and disability (Caton & Haas, 2020
Pessach & Shmueli, 2022;
Barocas et al. 2023). Interest such fair learming growing as various real-world applications of machine learning algorithms progress, such medical image analysis (Mehta et al. 2024
McCradden al. 2020), credit scoring (Kozodoi et al., 2022), face recognition (Menezes et al. 2021) In particular, the problem of aiming for parity in prediction performance among subgroups is called subgroup fairness (or group fair-
ness) Kearns et al. 2019 Martinez et al. 2021 Shui et al. 2022b a). The following is a definition of the
problem setting of subgroup fairness by Laholi et al. (2020)
Definition 10 (Subgroup Fairness). Let = {(x,, y1) Vor be the dataset, and assume that there exist K protected groups where each example my there exists an unobserved random variable 8; over { k= The set of examples with membership in group given by D. = {( v6) 8, = Mar Then, the goal of subgroup fairness to obtain hypothesis h E H that is fair to groups.
K
For the formalization of Sugroup fairness, for example, the following Rawlsian Max-Min fairness is known.
Definition 11 (Rawlsian Max-Min Fairness (Ralss 2001 ). A hypothesis ht E H is said to satisfy Rawlsian Max-Min fairness principle if it satisfies the following.
h = arg min max L., (h),
(32)
8
where (h) (( [((h(x).
To solve this problem, Lahoti et al. 2020 consider the following min max problem.
Jhh X) min max Σ (h).
(33)
16

---

eooch
spoch
spoch 1000
10000
0.9
0.9
0.8
0.6
0.5
0.4
Weighted
0.6
0.2
0.1
50
100
150
200
250
300
350
epoch
Figure Left pancl: decision boundarics over epochs of training. Right pancl: the effect of carly stopping with importance weighting. These figures from Figure and 5 of Bfrd & Lipton (2019), and reprinted with permission from the authors.
where < A S and Σ, 1. They propose an adversarial algorithm framework in which X, is a parametric learnable function and the above optimized to improve the worst case. This be related to the importance weighting for each subgroup.
Also, many studies on fairness have been discussed in relation to distribution shifts such as covariate shift. and many of them explicitly utilize importance weighting (Havaldar et al. 2024 Martinez et al. 2021 Kitsuki & Managi 2023
9 Importance Weighting and Deep Learning
The behavior of Importance Weighted ERM in wvr-paramettrize deep neural networks was unclear. Byrd & Lipton (2019) recently reported that the efficacy of importance weighting diminishes over extended iterative learning. Additionally, their experiments indicated that L. regularization and batch normalization partially restore the effectiveness of importance weighting, but the extent of this effect depends on parameters un- related to importance weighting. Figure 5 shows the convergence of decision boundaries over epochs of training, and the effect of early stopping with importance weighting. Their results suggest that as training progresses, the effects due importance weighting vanish. Specifically, while importance weighting impacts decision boundary of model early phase in training, in the limit, models with several weighting appear to approach similar solutions. They also report that these findings hold for both simple convolutional neural networks and state-of- oftte--f- transformer- based models.
They also provide an intuitive discussion that connects the experimental results obtained with the implicit bias (Soudry et al. 2018 Ji & Telgarsky 2018b Nacson et al.) 2019) of the gradient descent. method. Xu et al. (2021) took this argument further, providing theoretical support for their experimental results. Through their thcoretical analysis, they characterized the impact of importance weighting on the implicit bias of gradient descent and provided the generalization bound that hinges on the importance weights. Their findings suggest that, in finite-step training, the impact of importance weighting the generalization bound is dependent on its effect on the margin and its role in balancing the source and target distributions.
Several studies have aimed at implicitly learning importance weightings using neural networks. Ren et al. (2018) introduced an online meta-lcarning algorithm aimod at adjusting the weights training examples, with the goal of training deep learning models that are more robust.. Their objective is
Real
w" arg min
E(hig (wx), 9))),
(34)
10
17

---

where nval > 0 is the size of validation sample, and
The
Nw arg min
Σ
w; Ut).
(35)
h
Through experiments, they empirically demonstrated that their proposed method outperforms existing ex- plicit weighting techniques. For the End-to-Enn approach, Fang et al. (020)) reported that approximating the weighting function with neural network induces bias. That is, given that the weight estimator w(x)) and weighted classifier (x) are each represented by neural networks, the poor performance of one negatively affects the performance of the other, since w(xr) and (a:) are dependent each other. Therefore, they proposed to address this issue by optimizing the weighting neural network and the classifier alternately follows:
Define the classification model h and the weight estimator w
h = fog.
w π 9.
where g: X' + Rd is a common feature extractor, f:: Ed + y is a classification layer, and m: IRd [0, is a weight estimation layer.
ii) Update the classification model with the weighted loss function w(x))((((x)vv) = g(z))
ii) Update the weight estimator model with fixed parameters for the feature extractor
iv) Iterate steps ii) and tif) alternately.
They show that this framework, called Dynamic Importance Weighting (DIW), resolve the circular dependency between weighted classifiers and weight estimators.
Holtz et al. 2002)p reported that learning through importance weighting improves robustness against adver-
sarial attacks (Chakraborty et al. 2021 Xu et al. 2020 Huang et al., 2017 Guo et al. 2019). They propose a method to acquire this weighting through an adversarial learning framework.
As an alternative to importance weighting, Lu al. (2022b) propose importance tempering to enhance the decision boundary of preaparrmetrized neural networks. The central idea behind importance tempering is to introduce temperature parameter into the softmax, which corresponds importance weighting. They reported its effectiveness in training over-parametrized neural networks.
Other research areas include importance weighting for generative models. Burda et al. 2015 pointed out that the critcrion of original Variational Auto Encoder (VAE) (Kingma & Welling 2013) too strict, and proposed Importance Weighted VAE (IWVAE) as a variant. The evidence lower bound (ELBO) maximized by the VAE
g(<z x)) ]
p(xx, 2)
log p(x) 2 Ez-g(=/a)
log
LvAB(g),
(36)
and IWAE maximizes the following multi-sample ELBO.
]
1 p(z, 2i)
log 2 E R 2
log
LIwAE(g).
(37)
g(z x))
t=1
Given a batch of samples 22...., 2y from q(2 x)) we can see that TWVAE utilizes the following importance weighted distribution.
p(a..) q( (an)
7( xr, Z2.x)
( x)).
(38)
p(a. o(#, (6)
=1
18

---

Note that when k 1, q is equal to q, and as k - 00, E.2...... I, #2:4]] approaces the true posterior p( æ)) pointwise. TWAE has been the subject of much discussion due to its effectiveness Rainforth et al.
(2117) and Nowozin (2018) provide the analysis of IWVAE bounds and other Monte Carlo objectives. Daudel ct al 2023) generalized IWVAE by a-divergence and characterized its statistical properties.
10
Conclusion and perspective
In this review, we explored the application of importance weighting operations within the realm of machine learning. Notably, certain methodologies not traditionally identified as importance weighting can be reinter- preted as such operations (for instance, focal loss in model calibration and label noise correction). Morcover, numerous studies, particularly in recent times, have introduced akin methods without directly referencing Importance Weighted Empirical Risk Minimization (IWERM) (Shimodaira) 2000). Viewing these diverse techniques through the lens of importance weighting offers several benefits.
Utilization of existing results: Many theoretical results are known for IWERM. For example, Shimodaira (2000) gives the information criterion for IWERM, and Cortes et al. (2010) provides a generalization iization error analysis of importance weighted supervised learning. By interpreting one technique as importance weighting, we can take advantage of these existing theoretical results.
ii) Determination of weighting functions: When technique can be considered as importance weighting, the key question is how to determine the weighting function. As we observed in the examples of importance sampling in Section [ and unbiasedness of IWERM under the covariate shift in Section 3.1 the selection of the weighting function is strongly linked to the density ratio. When the optimal weighting function for a given method can be expressed using density ratio, we may be able to take advantage of density ratio estimation techniques as reviewed in Section 2.1
iii Sharing negative results: Several studies have reported that importance weighting does not work well in some cases (c.g., Byrd & Lipton| (2019)) We can take these negative results into account when considering the experimental results of methodologies that can be considered as importance weighted learning.
While there are many advantages to a unified interpretation as a framework of importance weighting as described above, it is possible that many of the methods not covered in this survey could also be considered as IWERM and its variants. For example, the IWAN (Zhang et al. 2018b) in Section 3.6.2 can be shown to reproduce the relative importance weighting (Yamada et al. 2013) by simple calculation, but this fact is not mentioned in the original IWAN paper. However, the relative importance weighting has been shown to be a density ratio approximation based on minimizing Pearson divergence, and several theoretical results are known. This suggests that some techniques can be taken slep further by leveraging existing known results, and unified interpretation through the importance weighting framework makes this possible.
The error bound is an important topic to discuss. For example, DIW (Fang et al. 2020), one of the experi- mentally successful importance weighting methods in deep learning, uses Kernel Mean Matching (KMM) for updating the weight estimator, and its error bound is O(n 21 + 2+ Li et al. (020aa) propose robust which is superior to the
importance weighting method for covariate shift, and achieves On + n,e best-known rate of KMM. However, currently only a limited number of studies provide the error bounds of each method.
Another future work would be to analyze the behavior of importance weighting in more modern machine learning problem settings. In Section g we mentioned studies on the behavior of importance weighting in deep learning. However, the bchavior of importance weighting in large neural networks, such as the Large Language Models (LLM) (Brown et al. 2020: Floridi & Chiriatti, 2020 Chang et al. 2023) that have recently achieved significant results, is not yet understood. Theoretical and experimental verification which cases of importance weighting work well and which fail would be very useful. As we mentioned, while many of the techniques can be considered special variants of importance weighting, they do not seem make full use of the known results. We hope that this survey is useful to address these unresolved issues.
19

---

Acknowledgement
Part of this work is supported by JST CREST (JPMJCR2015), JST Mirai Project (JPMJMI21G2), and JSPS (AKENH((P22H00063)) Finally, we express our special thanks to the editor and anonymous reviewers whose valuable comments helped to improve the manuscript.
References
Anish Acharya, Sujay Sanghavi, Li Jing, Bhargav Bhushanam, Dhruv Choudhary, Michael Rabbat, and Inderjit Dhillon. Positive unlabeled contrastive learning. arXiv preprint arXiv:2206. 01206, 2022.
Alekh Agarwal and Tong Zhang. Minimax regret optimization for robust machine learning under distribution shift. In Conference on Learning Theory, pp. 2704-2729. PMLR, 2022.
Alekh Agarwal, Leon Botton, Miroslav Dudik, and John Langford. Para-active learning. orXiv preprint arXiv:1310.66 8248, 2013.
Sharat Agarwal, Himanshu Arora, Saket Anand, and Chetan Arora. Contextual diversity for active learn- ing. In Computer Vision-ECCV 2020: 16th European Conference, Glasgow, UK, August 28-28, 2020, Proceedings, Part XVT 16, pp. 137-153. Springer, 2020.
Akram Awad and George K Atia. Distributionally robust domain adaptation. arXiv:2210. 16894, 2022.
arXiv preprint
Kanyar Azizzadenesheli, Anqi Liu, Fanny Yang, and Animashree Anandkumar. Regularized learning for domain adaptation under label shifts. arXiv preprint Nix::10300993. 2019.
Francis Bach. Active learning for misspecified generalized linear models. Advances in neural information processing systems, 19, 2006.
Solon Barocas, Moritz Hardt, and Arvind Narayanan. Fairness and machine learning: Limitations and opportunities. MIT Press, 2023.
Jessa Bekker and Jesse Davis. Learning from positive and unlabeled data: A survey. Machine Learning, 109:719-760, 2020.
Shai David John Blitzer, Koby Crammer, and Fernando Pereira. Analysis of representations for domain adaptation. Aduances in neural information processing systems, 19, 2006.
Aharon Ben-Tal, Dick Den Herlog, Anja De Waegenaere, Bertrand Melenberg, and Gijs Rennen. Robust solutions of optimization problems affected by uncertain probabilities. Monagement Science, 59(2):341- 357, 2013.
Yoshua Bengio, Jérôme Louradour, Ronan Collobert, and Jason Weston. Curriculum learning. In Proceedings of the 26th annual international conference machine learning, pp. 41 48, 2009.
Richard A Berk. An introduction to sample selection bias in sociological data. American sociological review, pp. 386 398, 1983.
Dimitris Bertsimas, Vishal Gupta, and Nathan Kallus. Data-driven robust optimization. Mathematical Programming, 167:235 292, 2018.
Dimitris Bertsimas, Melvyn Sim, and Meilin Zhang. Adaptive distributionally robust optimization. Man- agement Science, 65(2):604 618, 2019.
Keith Beven and Andrew Binley. The future of distributed models: model calibration and uncertainty prediction. Hydrological processes, 6(3):279 298, 1992.
Alina Beygelzimer, Sanjoy Dasgupta, and John Langford. Importance weighted active learning. In Proceed- ings of the 26th annual international conference machine learning, pp. 49 49-56, 2009.
20

---

Alina Beygelzimer, Daniel Hsu, Nikos Karampatziakis, John Langford, and Tong Zhang. Efficient active learning. In ICML 2011 Workshop on On-line Trading of Exploration and Faploilation, 2011.
Christopher M Bishop and Nasser M Nasrabadi. Pattern recognition and machine learning, volume 4. Springer, 2006.
Jose Blanchet, Yang Kang, and Karthyck Murthy. Robust wasserstein profile inference and applications to machine learning, Journal of Applied Probability, 56(3):830 857, 2019.
Klaus Brinker. Incorporating diversity in active learning with support vector machines. In Proceedings of the 20th international conference on machine learning (ICML-08), pp. 59-66, 2003.
Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. Language models are few-shot learners. Advances in neural information processing systems, 33:1877-1901, 2020.
Yuri Burda, Roger Grosse, and Ruslan Salakhutdinov. Importance weighted autoencodors. arXiv preprint arXiv:1509.cm 00519, 2015.
Jonathon Byrd and Zachary Lipton. What is the effect of importance weighting in deep learning? In International conference on machine learning, pp. 872 881. PMLR, 2019.
Wenbin Cai, Ya. Zhang, and Jun Zhou. Maximizing expected model change for active learning in regression.
In 2013 IEEE 13th international conference on data mining, pp. 51 60. IEEE, 2013.
Zhangjic Cao, Mingsheng Long, Jianmin Wang, and Michael I. Jordan. Partial transfer learning with selective adversarial networks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 80.
Zhangjie Cao, Lijia Ma, Mingsheng Long, and Jianmin Wang. Partial adversarial domain adaptation. In Proceedings of the European conference on computer vision (ECCV), pp. 135-150, 2018b.
Zhangjie Cao, Kaichao You, Mingsheng Long, Jianmin Wang, and Qiang Yang. Learning transfer examples for partial domain adaptation. In Proceedings of the IEEE/CVF Conference on Compuler Vision and Pattern Recognition (CVPR), June 2019a.
Zhangjie Cao, Kaichao You, Mingsheng Long, Jianmin Wang, and Qiang Yang. Learning to transfer examples for partial domain adaptation. Proceedings of the IEEE/CVP conference on computer vision and pattern recognition, pp. 2985 2994, 2019b.
Simon Caton and Christian Haas. Fairness in machine learning: A survey. ACM Computing Surveys, 2020.
Anirban Chakraborty, Manaar Alam, Vishal Dey, Amupam Chattopadhyay, and Debdcep Mukhopadhyay. A survey on adversarial Lacks and defences. CAAT Transactions on Intelligence Technology, 6(1):25 45, 2021.
Yee Seng Chan and Hwee Tou Ng. Word sense disambiguation with distribution estimation. In IJCAI, volume 5, pp. 1010-5, 2005.
Yupeng Chang, Xu Wang, Jindong Wang, Yuan Wu, Kaijie Zhu, Hao Chen, Linyi Yang, Xiaoyuan Yi, Cunxiang Wang, Yidong Wang, et al. A survey on evaluation of large language models. arXiv preprint arXiv:2307.03100 2023.
Olivier Chapelle. Modeling delayed feedback in display advertising. In Proceedings of the 20th ACM SIGKDD international conference on Knowledge discovery and data mining, pp. 1097 1105, 2014.
Nontawat Charoenphakdee, Jayakorn Vongkulbhisal, Nuttapong Chairatanakul, and Masashi Sugiyama. On focal loss for class-post passs-oottrroo probability estimation: A theoretical perspective. In Proceedings of the IEFE/CVF Conference on Compuler Vision and Pattern Recognation, pp. 5202 12215,1,, 2021.
21

---

Huimin Chen and X Rong Li. Distributed active learning with application to battery health management. In 14th International conference on information fusion, pp. 1-7. IEFE, 2011.
Xiangli Chen, Mathew Monfort, Anqi Liu, and Brian D Ziebart. Robust covariate shift. regression. In Artificial Intelligence and Statistics, pp. 1270-1279. PMLR, 2016,
Xiuhua Chen, Chen Gong, and Jian Yang. Cost-sensitive positive and unlabeled learning. Information Sciences, 558:229 245, 2021.
Kristy Choi, Madeline Liao, and Stefano Ermon. Featurized density ratio estimation. In Uncertainty in Artificial Intelligence, pp. 172-182. PMLR, 2021.
Isaac T, Chuang, Tailin Wu, and Curtis G Northeutt. Learning with confident. examples: Rank pruning for robust classification with noisy labels. 2017.
Corinna Cortes, Yishay Mansour, and Mehryar Mohri. Learning bounds for importance weighting. Advances in neural information processing systems, 23, 2010.
Antonia Creswell, Tom White, Vincent Dumoulin, Kai Arulkumaran, Biswa Sengupta, and Anil A Bharath. Generative adversarial networks: An overview. IEEE signal processing magazine, 35(1):53-65, 2018.
Gabriela Csurka Domain adaptation for visual applications: A comprehensive survey. arXiv preprint arXiv:1702.m 05374, 2017.
Yin Cui, Menglin Jia, Tsung-Y Yi Lin, Yang Song, and Serge Belongie. Class-balanced loss basod on effec- live number of samples. In Proceedings of the IEEE/CVF conference on computer vision and paltern recognition, pp. 9268-9277, 2019.
Kamélia Daudel, Joe Benton, Yuyang Shi, and Arnand Doucet. -iphe-civeeeence variational inference meets importance weighted auto-encoders Methodology and asymptotics. Journal of Machine Learning Research, 24223)): 2023.
Antoine de Mathelin, Guillaume Richard, François Deheeger, Mathilde Mougeot, and Nicolas Vayatis. Ad- versarial weighting for domain adaptation in regression. In 2021 IEEE 33rd International Conference on Tools with Artificial Intelligence (ICTAI), pp. 49-56. IEEE, 2021.
Erick Delage and Yinyu Ye. Distributionally robust optimization under moment uncertainty with application data-driven problems. Operations research, 58(3):595 592-612 2010.
Bo Du, Zengmao Wang, Lefei Zhang, Liangpei Zhang, Wei Liu, Jialic Shen, and Dacheng Tao. Exploring representalivennss and informativeness for active learning. IEEE transactions on cybernelics, 47(1):14 26, 2015.
Marthinus Du Plessis, Gang Niu, and Masashi Sugiyama. Convex formulation for learning from positive and unlabeled data. In International conference on machine learning, pp. 1386 1394. PMLR, 2015.
Marthimus C Du Plessis, Gang Niu, and Masashi Sugiyama. Analysis of learning from positive and unlabeled data. Advances in neural information processing systems, 27, 2014.
John Duchi and Hongseok Namkoong. Tearning models with uniform performance via distributionally robust optimization. arXiv preprint 15f:::81000875:: 2018.
John C Duchi and Hongseok Namkoong. Learning models with uniform performance via. distributionally robust optimization. The Annols of Statistics, 49(3):1378 1406, 2021.
Klaus Eckhardt, Nicola Fohrer, and Hans-Georg Frede. Automatic model calibration. Hydrological Processes: An International Journal, 19(3):651-658, 2005.
Charles Elkan and Keith Noto. Learning classifiers from only positive and unlabeled data. In Proceedings of the 14th ACM SIGKDD international conference on Knowledge discovery and data mining, pp. 213 113-220, 2008.
22

---

Peyman Mohajerin Esfahani and Daniel Kuhn. Data-driven distributionally robust optimization us- ing the wasserstein metric: Performance guarantees and tractable reformulations. arXiv preprint arXiv:1505.05. 2015.
Tongtong Fang, Nan Lu, Gang Niu, and Masashi Sugiyama. Rethinking importance weighting for deep learning under distribution shift. Advances in neural information processing systems, 33:11996 12007, 2020.
Abolfazl Farahani, Sahar Voghoei, Khaled Rasheed, and Hamid R Arabnia. A brief review of domain adaptation. Advances data science and information engineering: proceedings from ICDATA 2020 and IKE 2020, pp. 877-894, 2021.
Sebastian Farquhar, Yarin Gal, and Tom Rainforth. On statistical bias in active learning: How and when to fix it arXiv preprint arXiv:2101.1160 2021.
Luciano Floridi and Massimo Chiriatti. Gpt-3: Its nature, scope, limits, and consequences. Minds and Machines, 30:681-699 694, 2020.
Alexander Freytag, Erik Rodner, and Joachim Denzler. Selecting influential examples: Active learning with expected model output changes. In Computer Vision ECCV 2014: 13th European Conference, Zurich, Switzerlandd, September 6-12, 2014, Proceedings, Part IV 13, pp. 562 577. Springer, 2014.
Bo Fu, Zhangjie Cao, Mingsheng Long, and Jianmin Wang. Learning detect open classes for universal domain adaptation. In Computer Vision ECCV 2020: 16th European Conference, Glasgow, UK, Augusl 23 23-28, 2020, Proceedings, Port XV 16, pp. 583-583. Springer, 2020.
Saurabh Garg, Sivaraman Balakrishnan, and Zachary Lipton. Domain adaptation under open set label shift. Advances in Newral Information Processing Systems, 35:22531 22546, 2022.
Arindam Ghosh, Thomas Schaaf, and Matthew Gormley. Adafocal: Calibration-aware adaptive focal loss. Advances in Newral Information Processing Systems, 35:1583 1595, 2022.
Davit Gogolashvili, Mattco Zecchin, Motonobu Kanagawa, Marios Kountouris, and Maurizio Filippone. When importance weighting correction needed for covariate shift adaptation? arXin preprinl arXiv:2303.0,00 040000, 2023.
Joel Goh and Melvyn Sim. Distributionally robust optimization and its tractable approximations. Operations research, 58(4-part-1):902 917, 2010.
Ian Goodfellow, Jean Pouget-Abadic, Mchdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Generative adversarial nelworks. Commuunicaiinns of the ACM, 63(11): 139 144, 2020.
Alex Graves, Marc G Bellemare, Jacob Menick, Remi Munos, and Koray Kavukcuoglu. Automated curricu- lum learning for neural networks. In international conference on machine learning, pp. 1311 1320. Pmlr, 2017.
Arthur Gretton, Alex Smola, Jiayuan Huang, Marcel Schmittfull, Karsten Borgwardt, and Bernhard Schölkopf. Covariate shift by kernel mean matching. Dalasel shift in machine learning, 3(4):5, 2009. Wen Gu, Teng Zhang, and Hai Jin. Entropy weight allocation: Positive-unlabeldd learning via optimal transport. In Proceedings of the 2022 SIAM International Conference on Data Mining (SDM), pp. 37-45.
SIAM, 2022.
Chuan Guo, Jacob Gardner, Yurong You, Andrew Gordon Wilson, and Kilian Weinberger. Simple black-box adversarial allacks. In International Conference on Machine Learning, pp. 2484 2493. PMLR, 2019.
Hoshin V Gupta, Keith J Beven, and Thorsten Wagener. Model calibration and uncertainty estimation. Encyclopedia of hyddrological sciences, 2006.
23

---

Guy Hacohen and Daphna Weinshall. On the power of curriculum learning in training deep networks. In International conference on machine learning, pp. 1535-2544 PMTR, 2019.
Bo Han, Jiangchao Yao, Gang Niu, Mingyuan Zhou, Ivor Tsang, Ya Zhang, and Masashi Sugiyama. Masking: A new perspective of noisy supervision. Advances in neural information processing systems, 31, 2018a.
Bo Han, Quanming Yao, Xingrui Yu, Gang Niu, Miao Xu, Weihua Hu, Ivor Tsang, and Masashi Sugiyama. Co-teaching: Robust training of deep neural networks with extremely noisy labels. Advances in neural information processing systems, 31, 2018b.
Zongbo Han, Zhipeng Liang, Fan Yang, Liu Liu, Lanqing Ti, Yatao Bian, Peilin Zhao, Bingzhe Wu, Changqing Zhang, and Jianhua Yao. Umix: Improving importance weighting for subpopulation shift via uncertainty-awaat mixup. Aduances in Neural Information Processing Systems, 35:37704-37718, 2022.
Trevor Hastie, Robert Tibshirani, Jerome H Friedman, and Jerome H Friedman. The elements of statistical learning: data mining, inference, and prediction, volume 2. Springer, 2009.
Shreyas Havaldar, Jatin Chauhan, Karthikeyan Shanmugam, Jay Nandy, and Aravindan Raghuveer. Fairness under covariate shift: Improving fairness-accuracy tradeoff with few unlabeled test samples. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pp. 12331 12339, 2024.
Mary C Hill. Methods and guidelines for effective model calibration. In Building partnerships, pp. 10. 2000.
Hideitsu Hino. Active learning: Problem settings and recent developments, arXiv preprint arXiv:2012.04225, 2020.
Chester Holtz, Beui-Wei Weng, and Gal Mishne. Learning sample reweighting for accuracy and adversarial robustness. Xiv preprint or Xiv:22200. 11513, 2022.
HM Sajjad Hossain, Nirmalya Roy, and Md Abdullah Al Hafiz Khan. Sleep well: sound sleep moni- toring framework for community scaling. In 2015 16th IEEE International Conference on Mobile Data Management, volume 1, pp. 44-53. IEFE, 2015.
Wei-Ning Hsu and Hsuan-Tice Lin. Active learning by learning. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 29, 2015.
Jiayuan Huang, Arthur Gretton, Karsten Borgwardt, Bernhard Schölkopf, and Alex Smola. Correcting sample selection bias by unlabeled data. Advances in newral information processing systems, 19, 2006.
Sandy Huang, Nicolas Papernot, Ian Goodfellow, Yan Duan, and Pieter Abbeel. Adversarial attacks on neural network policies. arXiv preprint arXiv:1702.0228. 2017.
Sheng-Jun Huang, Rong Jin, and Zhi-Hua Zhou. Active learning by querying informative and representative examples. Advances in neural information processing systems, 23, 2010.
Gareth James, Daniela Witten, Trevor Hastie, and Robert Tibshirani. An introduction to statistical learning, volume 112. Springer, 2013.
Ziwei Ji and Matus Telgarsky. Gradient descent aligns the layers of deep linear networks. arXio preprinl arXiv:1810.09000 02032, 2018a.
Ziwei Ji and Matus Telgarsky, Risk and parameter convergence of logistic regression. ar preprint arXiv:1803.07300 2018b.
Jing Jiang and ChengXiang Zhai. Instance weighting for domain adaptation in nlp. ACL, 2007.
Pooria Joulani, Andras Gyorgy, and Csaba Szepesvári. Online learning under delayed feedback. In Interna- tional Conference on Machine Learning, pp. 1453 1461. PMLR, 2013.
24

---

Kwang-Sung Jun and Robert Nowak. Graph-based active learning: A new look at expected error minimiza- tion. In 2016 TEEE Global Conference Signal and Information Processing (GlobalSTP), pp. 1325-132) LEEE, 2016.
Christoph Käding, Alexander Freytag, Erik Rodner, Andrea Perino, and Joachim Denzler. Large-scale active learning with approximations of expected model output changes. In Pattern Recognation: 38th German Conference, GCPR 2016, Hannover, Germany, Seplember 12-15, 2016, Proceedings 38, pp. 179 191. Springer, 2016.
Herman Kahn and Theodore E Harris. Estimation of particle transmission by random sampling. National Bureau of Stondards applied mathematics series, 12:27-30, 1951.
Takafumi Kanamori, Shohei Hido, and Masashi Sugiyama. A least-squares approach to direct importance estimation. The Journal of Machine Learning Research, 10:1391-1445, 2009.
Michael Kearns, Seth Neel, Aaron Roth, and Zhiwei Steven Wu. An empirical study of rich subgroup fairness for machine learning. In Proceedings of the conference on fairness, accountability, and transperency, pp. 100-109, 2019.
Prannay Khosla, Piotr Teterwak, Chen Wang, Aaron Sarna, Yonglong Tian, Phillip Isola, Aaron Maschinot, Ce Liu, and Dilip Krishnan. Supervised contrastive learning. Advances in neuraa information processing systems, 33:18661 18673, 2020.
Masanari Kimura. Why mixup improves the model performance. In International Conference on Artificial Neural Networks, pp. 275-286, Springer, 2021.
Masanari Kimura and Hideitsu Hino. Information geometrically generalized covariate shift adaptation. Newral Computation, 34(9):1944-1977, 2022.
Masanari Kimura, Kei Wakabayashi, and Atsuyuki Morishima. Batch prioritization of data labeling tasks for training classifiers. In Proceedings of the AAAI Conference on Human Compulation and Crowdsourcing, volume 8, pp. 163 163-167, 2020.
Diederik P Kingma and Max Welling. Auto-encoding variational bayes arxiv preprint ar Xrxiv:1322.61 2013.
Ryuichi Kiryo, Gang Niu, Marthinus C Du Plessis, and Masashi Sugiyama. Positive-unlabeled learning with non-negative risk estimator. Advances in neural information processing systems, 30, 2017.
Akinori Kitsuki and Shunsuke Managi. Importance woighting in subjective well-being measures: Using marginal utilities as weights for domain satisfaction. Journal of Huppiness Studies, 24(3):1101 1120, 2023.
Jannik Kossen, Sebastian Farquhar, Yarin Gal, and Tom Rainforth. Active testing: Sample-cfficient model evaluation. In International Conference on Machine Learning, pp. 5753-5763. PMLR, 2021.
Daniel Kottke, Jim Schellinger, Denis Huseljic, and Bernhard Sick. Linitations of assessing active learning performance at runtime. arXiv preprint srXiw:1901.10322 2019.
Wouter M Kouw and Marco Loog. On regularization parameter estimation under covariate shift. In 2016 23rd International Conference on Pottern Recognition (ICPR), pp. 426-431. IEEE, 2016.
Nikita Kozodoi, Johannes Jacob, and Stefan Lessmann. Fairness in credit scoring: Assessment, implemen- tation and profit implications. European Journal of Operational Research, 297(3):1083-1094 2022.
Sofia Tra Ktena, Alykhan Tejani, Lucas Theis, Pranay Kumar Myana, Deepak Dilipkumar, Ferene Huszár, Steven Yoo, and Wenzhe Shi. Addressing delayed feedback for continuous training with neural networks in ctr prediction. In Proceedings of the 13th ACM conference on recommender systems,, pp. 187-195, 2019.
Atsutoshi Kumagai, Tomoharu Iwata, and Yasuhiro Fujiwara. Meta-learning for relative density-ratio esti- mation. Advances in Neural Information Processing Systems, 34:30426-30433 2021.
25

---

Jogendra Nath Kundu, Naveen Venkat, Ambareesh Revanur, R. Venkatesh Babu, et al. Towards inheritable models for open-set domain adaptation. In Proceedings of the IEFE/CVF conference on computer vision and pattern recognition, pp. 12376 12385, 2020.
Preethi Lahoti, Alex Beutel, Jilin Chen, Kang Lee, Flavien Prost, Nithum Thain, Xuezhi Wang, and Ed Chi. Fairness without demographics through adversarially reweighted learning. Advances in neural information processing systems, 33:728-740, 2020.
Woojin Loe, Jacwook Lee, and Sacrom Park. Instance weighting domain adaptation using distance kernel. Industrial Engineering & Management Systems, 17(2):334 340, 2018.
Daniel Levy, Yair Carmon, John Duchi, and Aaron Sidford. Large-scale methods for distributionally robust optimization. Advances in Newral Information Processing Systems, 33:8847-888 -8600, 2020.
David Lewis and Jason Catlett. Heterogeneous uncertainty sampling for supervised learning. In Machine learning proceedings 1994, pp. 148-156. Elsevier, 1994.
Fengpei Li, Henry Lam, and Siddharth Prusty. Robust importance weighting for covariate shift. In Inter-
national conference on artificial intelligence and statistics, pp. 352-362. PMI.R, 2020a.
Mingkun Li and Ishwar K Sethi. Confidence-based active learning. IEEE transactions on pallern analysis and machine intelligence, (588)11251116)) 2006.
Xiang Li, Wenhai Wang, Lijun Wu, Shuo Chen, Xiaolin Hu, Jun Li, Jinhui Tang, and Jian Yang. Generalized focal loss: Learning qualified and distributed bounding boxes for dense object detection. Advances in Newral Information Processing Systems, 13:21002-21012 2020b.
Jian Liang, Yunbo Wang, Dapeng Hu, Ran He, and Jiashi Feng. A balanced and uncertainty-awarr approach for partial domain adaptation. In European conference on computer vision, pp. 123-140. Springer, 2020.
Tsung-Yi Yi Lin, Priya Goyal, Ross Girshick, Kaiming He, and Piotr Dollár. Focal loss for dense object detection. In Proceedings of the IEEE international conference computer vision, pp. 2980 2988, 2017.
Zachary Lipton, Yu-Xiang Wang, and Alexander Smola. Detecting and correcting for label shift with black box predictors. In International conference machine learning, pp.
PMLR, 2018
Anqi Liu and Brian Ziebart. Robust classification under sample selection bias. Advances in neural information processing systems, 27, 2014.
Anqi Liu and Brian D Ziebart. Robust covariate shift prodiction with general losses and feature views. arXiv preprint ar Xiv: 1712. 10043, 2017.
Bing Liu, Yang Dai, Xiaoli Li, Wee Sun Lee, and Philip S Yu. Building text classifiers using positive and unlabeled examples. In Third IEFE international conference on data mining, pp. 179-186. TEEE, 2003.
Chen Liu, Xiaomeng Dong, Michael Potter, Hsi-Ming Chang, and Ravi Soni. Adversarial focal loss: Asking your discriminator for hard examples. arXiv preprint orXiv:2207.07733 2022.
Hong Liu, Zhangjie Cao, Mingsheng Long, Jianmin Wang, and Qiang Yang. Separate to adapt: Open set domain adaptation via progressive separation. In Proceedings of the IEEF/CVF conference on computer vision and pattern recognition, pp. 2927-2936, 2019.
Song Liu, Akiko Takeda, Taiji Suzuki, and Kenji Fukumizu. Trimmed density ratio estimation. Advances in newral information processing systems, 30, 2017.
Wei Liu and Sanjay Chawla. Class confidence weighted nn algorithms for imbalanced data sets. In Advances in Knowledge Discovery and Data Mining: 15th Pacific-Asia Conference, PAKDD 2011, Shenzhen, China, May 24-27 2011, Proceedings, Part II 15, pp. 345- 356. Springer, 2011.
Mingshong Long, Zhangjic Cao, Jianmin Wang, and Michael I Jordan. Conditional adversarial domain adaptation. Advances in neural information processing systems, 31, 2018.
26

---

Yangdi Lu, Yang Bo, and Wenbo He. Noise attention learning: Enhancing noise robustness by gradient scaling. Advances in Neural Information Processing Systems, 35:23164-231777 2022a.
Yiping Lu, Wenlong Ji, Zachary Izzo, and Lexing Ying. Importance tempering: Group robustness for
overparameterized models. arXiv preprint 00i1:2200008745, 2022b.
Aleksandr Luntz. On estimation characters obtained in statistical procedure of recognition. Technicheskaya Kibernetica, 1969.
Jose Ignacio Segovia Martin, Santiago Mazuelas, and Anqi Liu. Double-weighting for covariate shift adap- tation. In International Conference on Machine Learning, pp. 30439-30457. PMLR, 2023.
Natalia L Martinez, Martin A Bertran, Afroditi Papadaki, Miguel Rodrigues, and Guillermo Sapiro. Blind pareto fairness and subgroup robustness. In International Conference Machine Learning, pp. 7492-7501. PMLR, 2021.
Nima Mashayekhi. An Adversarial Approuch to Importance Weighting for Domain Adaplation. PhD thesis, 2022.
Melissa D McCradden, Shalmali Joshi, Mjaye Mazwi, and James A Anderson. Ethical limitations of al- gorithmic fairness solutions in health care machine learning. The Lancet Digital Health, 2(1):c221-)223, 2020.
Raghav Mchta, Changjian Shui, and Tal Arbel. Evaluating the fairness of deep learning uncortainty estimates in medical image analysis. In Medical Imaging with Deep Learning, pp.
PMLR, 2024.
Hanna F Menezes, Arthur SC Ferreira, Eanes T Pereira, and Herman M Gomes. Bias and fairness in face detection. In 2021 34th SIBGRAPI Conference on Graphics, Patterns and Images (SIBGRAPI), pp. 247-254. TEEE, 2021.
Saad Mohamad, Moamar Sayed-Mouchaweh, and Abdelhamid Bouchachia. Active learning for classifying data streams with unknown number of classes. Neural Networks, 98:1 15, 2018.
Jishnu Mukhoti, Viveka Kulharia, Amartya Sanyal, Stuart Golodetz, Philip Torr, and Puneet Dokania. Calibrating deep neural networks using focal loss. Advances in Neural Information Processing Systems, 33:15288 12299998-15299 2020.
Mulyanto Mulyanto, Muhamad Faisal, Setya Widyawan Prakosa, and Jenq-Shiou Leu. Effectiveness of focal loss for minority classification in network intrusion detection systems. Symmetry, 13(1):4, 2020.
Stephen Mussmann, Julia Reisler, Daniel Tsai, Ehsan Mousavi, Shayne O'Brien, and Moises Goldszmidt. Active learning with expected error reduction. ar preprint arXiv:2211.098 09283, 2022.
Mor Shpigel Nacson, Suriya Gunasekar, Jason Lee, Nathan Srebro, and Daniel Soudry. Lexicographic and depth-sensitive margins homogencous and non-homogencous deep models. In International Conference on Machine Learning, pp. 4683 4692. PMLR, 2019.
Hiroki Nagamuma and Masanari Kimura. Necossary and sufficient hypothesis of curvature: Understand- ing connection between out-of-distributiin generalization and calibration. ICLR workshop on Domain Generalization, 2023.
Nagarajan Natarajan, Inderjit S Dhillon, Pradeep K Ravikumar, and Ambuj Tewari. Learning with noisy labels. Advances in neural information processing systems, 26, 2013.
Tuan Duong Nguyen, Marthinus Christoffel, and Masashi Sugiyama. Continuous target shift adaptation in supervised learning. In Asian Conference on Machine Learning, pp. 285 300. PMLR, 2016.
XuanLong Nguyen, Martin J Wainwright, and Michael Jordan. Estimating divergence functionals and the likelihood ratio by convex risk minimization. IEEE Transoctions on Information Theory, 56(11): 5847-5861, 2010.
27

---

Bryce Nicholson, Jing Zhang, Victor S Sheng, and Zhiheng Wang. Label noise correction methods. In 2015 IEFE International Conference on Dala Science and Advanced Analylics (DSAA), pp. 1-9. TEFE, 2015.
Bryce Nicholson, Victor S Sheng, and Jing Zhang. Label noise correction and application in crowdsourcing. Expert Systems with Applications, 66:149 162, 2016.
Sebastian Nowozin. Debiasing evidence approximations: On importance-weiglieee autoencoders and jackknife variational inference. In International conference on learning representations, 2018.
Art Owen and Yi Zhou. Safe and effective importance sampling. journal of the Americon Statistical Asso- cialion, pp. 135 135-143, 2000.
Pau Panareda Busto and Juergen Gall. Open set domain adaptation. In Proceedings of the IEFE international conference on computer vision, pp. 754 763, 2017.
Vishal M Patel, Raghuraman Gopalan, Ruonan Li, and Rama Chellappa. Visual domain adaptation: A survey of recent advances. IEEE signal processing magazine, 32(3):53-69) 2015.
Giorgio Patrini, Alessandro Rozza, Aditya Krishna Menon, Richard Nock, and Lizhen Qu. Making deep neural networks robust to label noise: A loss correction approach. In Proceedinggs of the TEFE conference on computer vision and pottern recognition, pp. 1944 1952, 2017.
Xisojiang Peng, Kai Wang, Zhaoyang Zeng, Qing Li, Jianfei Yang, and Yu Qiao. Suppressing mislabeled data via grouping and self-attention. In Compuler Vision ECCV 2020: 16th Europewn Conference, Glasgow, UK, August 23 28, 2020, Proceedings, Part XVI 16, pp. 786-802. Springer, 2020.
Dana Pessach and Erez Shmueli. (CSUR), 55(3):1-44, 2022.
A review on fairness in machine learning. ACM Computing Surveys
Ciara Piko-Burke, Shipra Agrawal, Csaba Szepesvari, and Steffen Grunewalder. Bandits with delayed, ag- gregated anonymous feedback. In International Conference on Machine Learning, pp. 4105 4113. PMLR, 2018.
Barbara Plank, Anders Johannsen, and Anders Søgaard. Importance weighting and unsupervised domain adaptation of pos taggers: a negative result. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), pp. 968 973, 2014.
Qi Qi, Yi Xu, Wotao Yin, Rong Jin, and Tianbao Yang. Attentional-biased stochastic gradient descent. Transactions on Machine Learning Research, 2022.
Hamed Rahimian and Sanjay Mehrotra. Distributionally robust optimization: A review. orXiv preprint arXiv:1908. 05659, 2019.
Tom Rainforth, Robert Cornish, Hongseok Yang, Andrew Warrington, and Frank Wood. On the opportu- nities and pitfalls of nesting monte carlo estimators. arXiv preprint arXiv: 709.06181, pp. 38, 2017.
John Rawls. Justice as fairness: A restatement. Harvard University Press, 2001.
levgen Redko, Emilie Morvant, Amaury Habrard, Mare Sebban, and Younes Bennani. Advances in domain adaplation theory. Elsevier, 2019.
Mengye Ren, Wenyuan Zeng, Bin Yang, and Raquel Urtasun. Learning to reweight examples for robust deep learning. In International conference machine learning, pp. 4334 4343. PMLR, 2018.
Benjamin Rhodes, Kai Xu, and Michael U Gutmann. Telescoping density-ratio estimation. Advances in neural information processing systems, 33:4905-4999 4916, 2020.
Abdollah Safari, Rachel MacKay Altman, and Thomas M Loughin. Display advertising: Estimating conver- sion probability efficiently. arXio preprint arxim: 171000880, 2017.
28

---

Shibani Santurkar, Dimitris Tsipras, and Aleksander Madry. Breeds: Benchmarks for subpopulation shift. arXiv preprint arXii:2008.0458 2020.
Burr Settles. Active learning literature survey. 2009.
Hidetoshi Shimodaira. Improving predictive inference under covariate shift by weighting the log-likelihood function. Journal of statistical planning and inference, 90(2):227 244, 2000.
Changjian Shui, Fan Zhou, Christian Gagné, and Boyu Wang. Deep active learning; Unified and principled method for query and training. In International Conference on Artificial Intelligence and Statistics, pp. 1308 1308-1318. PMLR, 2020.
Changjian Shui, Zijian Li, Jiaqi Li, Christian Gagné, Charles X Ling, and Boyu Wang. Aggregating from multiple target-shifted sources. In International Conference on Machine Learning, pp. 9638 9648. PMLR, 2021.
Changjian Shui, Qi Chen, Jiaqi Li, Boyu Wang, and Christian Gagné. Fair representation learning through implicit path alignment. In International Conference on Machine Learning, pp. 20156-200 20175. PMLR, 2022a.
Changjian Shui, Gezheng Xu, Qi Chen, Jiaqi Li, Charles X Ling, Tal Arbel, Boyu Wang, and Christian Gagné. On learning fairness and accuracy on multiple subgroups. Advances in Neural Information Pro- cessing Systems, 35:34121-344 34135, 2022b.
Leslie N Smith. Cyclical focal loss. arXiv preprint arXiw:2202.0897) 2022.
Alex J Smola and Bernhard Schölkopf. Learning with kernels, volume 4. Citeseer, 1998.
Hwanjun Song, Minseok Kim, Dongmin Park, Yooju Shin, and Jae-Gil Lee. Learning from noisy labels with deep neural networks: A survey. IEEE Transactions Neural Networks and Learning Systems, 2022.
Daniel Soudry, Elad Hoffer, Mor Shpigel Nacson, Suriya Gunasckar, and Nathan Srebro. The implicit bias of gradient descent on separable data. The Journal of Machine Learning Research, 19(1):2822 2878, 2018.
Jong-Chyi Su, Yi-Hsuan Tsai, Kihyuk Sohn, Buyu Liu, Subhransu Maji, and Manmohan Chandraker. Active adversarial domain adaptation. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 739-748, 2020.
Masashi Sugiyama. Active learning for misspecified models. Advances in neural information processing systems, 18, 2005.
Masashi Sugiyama. Introduction to statistical machine learning. Morgan Kaufmann, 2015.
Masashi Sugiyama, Matthias Krauledat, and Klaus-Roberl Müller. Covariate shift adaptation by importance weighted cross validation. Journal of Machine Learning Research, 8(5), 2007a.
Masashi Sugiyama, Shinichi Nakajima, Hisashi Kashima, Paul Buenau, and Motoaki Kawanabe. Direct importance ostimation with model selection and its application to covariate shift adaptation. Advances in neural information processing systems, 20, 2007b.
Qian Sun, Rita Chattopadhyay, Sethuraman Panchanathan, and Jieping Ye. A two-stage weighting frame- work for multi-source domain adaptation. Advances in neural information processing systems, 24, 2011.
Marcelo Tallis and Pranjul Yadav. Reacting to variations in product demand: An application for conversion rate (cr) prediction in sponsored search. In 2018 IEEE International Conference on Big Data (Big Data), pp. 1856-1864. IEEE, 2018
Linwei Tao, Minjing Dong, and Chang Xu. Dual focal loss for calibration. arXiv preprint orXiv:2305.1365 2023.
29

---

Yonglong Tian, Chen Sun, Ben Poole, Dilip Krishnan, Cordelia Schmid, and Phillip Isola. What makes for good views for contrastive learning? Advances in newral information processing systems, 33:6827-668 6839, 2020.
Surya T Tokdar and Robert E Kass. Importance sampling: a review. Wiley Interdisciplinary Reviews:
Computational Statistics, 2(1):54-60, 60, 2010.
Katrin Tomanek. Resource-aware annotation through active learning. 2010.
Katrin Tomanek and Katherina Morik. Inspecting sample reusability for active learning. Tn Active Learning and Experimental Design workshop In conjunction with AISTATS 2010, pp. 169-181. JMLR Workshop and Conference Proceedings, 2011.
Giang Son Tran, Thi Phuong Nghiem, Van Thi Nguyen, Chi Mai Luong, Jean- Dean-Christophe Burie, et al. Improving accuracy of lung nodule classification using deep learning with focal loss. Journal of healthcare engineering, 2019, 2019.
Van-Tinh Tran. Selection bias correction in supervised learning with importance weight. PhD thesis, Univer- sité de Lyon, 2017.
Van-Tinh Tran and Alex Aussem. Correcting class of complete selection bias with external data based on importance weight estimation. International Conference on Neural Information Processing, pp. 111-118. Springer, 2015.
Eric Tzeng, Judy Hoffman, Kate Saenko, and Trevor Darrell. Adversarial discriminative domain adaptation. In Proceedings of the IEFE conference on computer vision and pottern recognition, pp- 7167-7176, 2017.
Juozas Vaicenavicius, David Widmann, Carl Andersson, Fredrik Lindsten, Jacob Roll, and Thomas Schön. Evaluating model calibration in classification. In The 22nd International Conference on Artificial Intelli- gence and Statistics, pp. 1459-3461 PMLR, 2019.
Gijs Van Tulder. Sample reusability in sepertannc-weihhtee active learning. 2012.
Vladimir Vapnik. The nature of statistical learning theory. Springer science & business media, 1999a.
Vladimir Vapnik. An overview of statistical learning theory. IEEE transactions on neural networks, 10(5): 988 999, 1999b.
Francis Vella. Estimating models with sample selection bias: survey. Journal of Human Resources, pp. 127 127-169, 1998.
Claire Vernade, Olivier Cappé, and Vianney Perchet. Stochastic bandit models for delayed conversions. arXiv preprint (rXtc:1706.09180 2017.
Mei Wang and Weihong Deng. Deep visual domain adaptation: A survey. Newrocomputing, 312:135-1533 2018.
Xin Wang, Yudong Chen, and Wenwu Zhu. A survey on curriculum learning. IEEE Transactions on Pattern Analysis and Machine Intellgencee 44(9):4555 4576, 2021.
Junfeng Wen, Russell Greiner, and Dale Schuurmans. Domain aggregation networks for multi-source domain adaptation. In International conference on machine learning, pp. 10214-10224. PMLR, 2020.
Garrett Wilson and Diane J Cook. A survey of unsupervised deep domain adaptation. ACM Transactions on Intelligent Systems and Technology (TIST), 11(5):1-46, 2020.
Christopher Winship and Robert D Mare. Models for sample selection bias. Annual review of sociology, 18 (1):327-350, 350, 1992.
30

---

Tsung- Han Wu, Yuch-Chenn Liu, Yu-Kai Huang, Hsin-Ying Ying Lee, Hung-Ting Su, Ping- Chia Huang, and Winston H Hsu. Redal: Region-based and diversity-aware active learning for point cloud semantic segmen- tation. In Proceedings of the IEEE/CVF international conference on computer vision, pp. 15510 15519, 2021.
Weining Wu, Maozu Guo, Yang Liu, and Runzhang Xu. Active learning with optimal distribution for image classification. In 2011 International Conference on Multimedia Technology, pp. IEEE, 2011.
Rui Xia, Zhenchun Pan, and Feng Xu. Instance weighting for domain adaptation via trading off sample selec- tion bias and variance. In Proceedings of the 27th International Joint Conference on Artificial Intellignnce, Stockhobm, Sweden, pp. 13 19, 2018.
Xiaobo Xia, Tongliang Liu, Namman Wang, Bo Han, Chen Gong, Gang Niu, and Masashi Sugiyama. Are anchor points really indispensable in label-noise learning? Advances in neural information processing systems, 32, 2019.
Ni Xiao and Lei Zhang. Dynamic weighted learning for unsupervised domain adaptation. In Proceedings of the IEEE/CVE conference on compuler vision and pallern recognilion, pp. 15242- 15251, 2021.
Da Xu, Yuting Ye, and Chuanwei Ruan. Understanding the role of importance weighting for deep learning. arXiv preprint ar Xrx:2:2100 103. 15209, 2021.
Han Xu, Yao Ma, Hao-Chen Liu, Debayan Deb, Hui Liu, Ji-Liang Tang, and Anil K. Jain. Adversarial attacks and defenses in images, graphs and text: review. International Journal of Automation and Computing, 17:151-178, 2020.
Makoto Yamada, Taiji Suzuki, Takafumi Kanamori, Hirotaka Hachiya, and Masashi Sugiyama. Relative density-ratio estimation for robust distribution comparison. Neural computation, 2555):1324-1370, 2013.
Yazhou Yang and Marco Loog, Active learning using uncertainty information. In 2016 23rd International Conference Pallern Recognilion (ICPR), pp. 2646 2651. IEEE, 2016.
Yi Yang, Zhigang Ma, Feiping Nie, Xiaojun Chang, and Alexander G Hauplmann. Multi-class active learning by uncertainty sampling with diversity maximization. International Journal of Computer Vision, 113:113- 127, 2015.
Yuzhe Yang, Haoran Zhang, Dina Katabi, and Marzych Ghassemi. Change is hard: A closer look at subpopulation shift. arXiv preprint ar xrx::22002.122 12254, 2023.
Shota Yasui, Gota Morishita, Fujita Komei, and Masashi Shibata. A feedback shift correction in predicting conversion rates under delayed foedback. In Proceedings of The Web Conference 2020, pp. 2740-2746. 2020.
Jiayang Yin. On the Improvement of Density Ratio Estimation vio Probabilistic Caasiffer-Theoretice Study and Its Applications. PhD thesis, The University of British Columbia. (Vancouver, 2023.
Yuya Yoshikawa and Yusaku Imai. A nonparametric delayed feedback model for conversion rate prediction. arXiv preprint ar Xiv: 1802. 00255, 2018.
Kaichao You, Mingsheng Long, Zhangjie Cao, Jianmin Wang, and Michael I. Jordan. Universal domain adaptation. In Proceedings of the IEFE/CVE Conference on Compuler Vision and Pallern Recognation (CVPR), June 2019.
Bianca Zadrozny. Learning and evaluating classifiers under sample selection bias. In Proccedings of the twenty-forst international conference on Machine learning, pp. 114, 2004.
Hongyi Zhang, Moustapha Cisse, Yann N Dauphin, and David Lopez-Paz. mixup: Beyond empirical risk minimization. In International Conference on Learning Representations, 2018a.
31

---

Jing Zhang, Zewei Ding, Wanqing Li, and Philip Ogunbona. Importance weighted adversarial nets for partial domain adaptation. In Proceedings of the TEFE Conference on Compuler Vision and Pattern Recognation (CVPR), June 2018b.
Kun Zhang, Bernhard Schölkopf, Krikamol Muandet, and Zhikun Wang. Domain adaptation under target and conditional shift. In International Conference Machine Learning, pp. 819-827. PMLR, 2013.
Linjum Zhang and Zhun Deng. How does mixup help with robustness and generalization? The Ninth International Conference on Learning Representations, 2021.
Eric Zhao, Anqi Liu, Animashree Anandkumar, and Yisong Yue. Active learning under label shift. In International Conference on Artificial Intelligence and Stalislics, pp. 3412 3420. PMLR, 2021.
Liyue Zhao, Gita Sukthankar, and Rahul Sukthankar. Importance-weighted label prediction for active learn- ing with noisy annotations. In Proceedings of the 21sl. International Conference on Pallern Recognation (ICPR2012), pp. 8476-3479. LEEE, 2012.
Jingbo Zhu, Huizhen Wang, Benjamin K Tsou, and Matthew Ma. Active learning with sampling by uncer- tainty and density for data annotations. IEEE Transactions audio, speech, and language processing, 18(6):1323 1331, 2009.
A Proof of Lemma 1
Proof. From the assumption, have P(s 11y = 1, P(s 1/1 1). We also have
P(s = 1|x) = Ply 1 A 1(x)
= P(y = 11z) 1 = 1, x)
= P(y = 1 1ly = 1).
(39)
By dividing each side by P(s = 1/y 1), we can obtain the proof.
32


Reviews:
[{"review_id": "ggu2zbyBTV", "reviewer": "Reviewer_3jUS", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "As a survey paper, this paper demonstrates a solid formulation and its linkage to various machine learning problems. However, as a survey, I believe there should be more analysis of the different works rather than just listing them in a table. For instance, readers might be interested in understanding which issues are effectively addressed by importance weighting and what challenges still remain for each machine learning problem utilizing importance weighting. Additionally, a survey should provide a comprehensive overview of all the problems and draw insightful conclusions regarding future directions or challenges within the surveyed topic.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper aims to review certain machine learning problems related to sample weighting. By presenting a previous definition and mathematical model of importance weighting, the paper reviews many pieces of machine learning problems using weighting to tackle challenges, thereby transforming these problems into a unified weighting technique.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "I believe there is ample opportunity for improvement in the survey by not merely introducing the topics and concluding.", "broader_impact_concerns": "No.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "i6lVlffHTJ", "reviewer": "Reviewer_7PJm", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "### Strong point\n\nFirst of all, I really like this short survey. This paper is a timely summary of Importance Weighting (IW) in various aspects. I am familiar with some concepts and others are quite new to me. The writing is quite accessible to me. \n\n### Weak point\n\nSince this is a survey paper, there are no technical concerns. However, such a paper generally has a very high standard of organization and illustration. Please check the requested changes as the concerns during my reading. I hope this could be helpful to improve this survey paper. \n**Please note that all the questions in the request changes are related to the clarifications. In fact, I fully understand the concept, it is more on the issue of clarity.**", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This short survey paper discusses the importance weighting in machine learning, a very popular and practical tool in the context of distribution shift. This paper systematically conducted the analysis on importance weighting in different aspects.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "### Page 2\n- In Fig 1. Each term in applications of IW could be associated with the concrete sections (e.g., Target shift (Sec 3.2)).\n- In Sec 2. The definitions 1 and 2 could be simplified. Indeed, this is well-known for most readers in ML, it could be simplified. \n\n### Page 3\n- As for the distribution shift adaptation. I think this title seems a bit confusing because Sec 4 is also called domain adaptation. Is there any difference between the two adaptations?\n- In Sec 3.1 (the proof can be put into appendix). Authors are encouraged to include some figures in the covariate shift. \n- In IWERM, RIERN, AIWERM. It is much better to make a table to compare and discuss. \n\n### Page 4\n- The derivation of Eq(11) can be put in the appendix. \n- The Dro paragraph could be merged with the DRO later. \n- It is better to show some figures to illustrate the target shift. (This could be discussed together with covariate shift).\n\n### Page 5\n- Sec 3.3 The sample selection bias has no formal definition. \n- It is better to illustrate how to use this in machine learning. \n- Sec 3.4. No definitions for subpopulation (or subgroup) shift. \n- There is a notation error on the bottom of page 5. What is x_i,j. It is a typo?\n\n### Page 6\n- The definition of x_i, x_j, y_i and y_j are finally defined. But it should be present at Page 5. \n- Sec 3.5. It is much better to provide a formal definition of feedback shit. Please note that these differences shifts are related. It is important to clarify and discuss their differences. \n- Sec 4. There is no definition on domain adaptation. \n\n### Page 7\n- In sec 4.1, the multi-source domain adaptation could have different views on IW. It can be the weights on the domains or weights on the covariates/targets. For example, paper [1] discussed two weights. \n- In sec 4.2, partial DA. I would think paper [1] also bridged the IW with partial DA. Indeed, partial DA could be viewed as a specific target shift problem. If we have a weight coefficient, it is just some coefficient that has zero elements, due to the partial labeling. \n- Bottom of Page 7, a typo ((Yanma et.al 2013)) -> (Yanma et.al 2013). \n\n### Page 8\n- Sec 4.3, 4.4. It could be better to use a formal definition to discuss different DA settings. \n\n### Page 9\n- In sec 5. The AL can be viewed as IW in an alternative way. For example, if we design a query distribution $q(x)$ (something we want to learn) to match the entire unlabelled data $D(x)$.(e.g, see paper [2]). If we set a distribution divergence metric as Renyi (i.e, min_q Renyi(q(x), D(x))), I think this could be recovered to some IW methods. \n\n### Page 10\n- In sec 10, model calibration. It is strongly recommended to use some figures and tables to illustrate calibration and focal loss, adaptive focal loss.\n\n### Page 11\n- Proof of Lemma 1 can be put in the appendix.  \n- Eq(22) is the demonstration rather than the derivation, right?\n\n\n### Page 12\n- Sec 10. I think this part needs major revisions. I know all the material but I could not get the point of IW. Or I do not know how to use this in machine learning…\n\n### Page 13\n- Sec 11. It can be a bit more detailed about this important issue. \n\n### Page 14\n- The conclusion seems a bit short. I would like to see clear future directions on IW. \n- Besides, some topics related to IW seem missing. For example, (1) IWVAE, using importance weighting in the generative model such as paper [3] . (2) Fairness and subgroup discriminations such as paper [4,5,6].\n\n\nReferences:\n\n[1] Aggregating From Multiple Target-Shifted Sources. ICML 2021\n\n[2] Deep Active Learning: Unified and Principled Method for Query and Training. Aistats 2020.\n\n[3] Importance Weighted Autoencoders. ICLR 2015\n\n[4] Fair Representation Learning through Implicit Path Alignment. ICML 2022\n\n[5] On Learning Fairness and Accuracy on Multiple Subgroups. NeurIPS 2022\n\n[6] Fairness without demographics through adversarially reweighted learning. NeurIPS 2020.", "broader_impact_concerns": "N/A.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "nuGVbJ95uE", "reviewer": "Reviewer_mPi6", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n\n1. This survey covers a broad range of topics related to importance weighting, making it a valuable resource for researchers and practitioners.\n\n2. The authors provide a clear and accessible explanation of complex concepts, making the material understandable to readers with varying levels of expertise. The inclusion of both theoretical and practical aspects of importance weighting offers a good view.\n\nWeaknesses:\n\n1. While the survey is comprehensive, it may benefit from more detailed case studies or examples to illustrate the practical applications of importance weighting in real-world scenarios. This survey covers a broad range of importance weighting methods but none of these has been formally introduced in details.\n\n2. There are several confused points about the paper structure: (a) Why Density Ratio Estimation is missed from the structural diagram in Figure 1? (b) Domain Adaptation, which is inherently a specific type of Distribution Shift Adaptation, is presented as a separate entity in the paper. (c) It seems that Distributionally Robust Optimization is one part of Distribution Shift Adaptation within the diagram (Figure 1), but the Section of Distributionally Robust Optimization is positioned between Active Learning and Model Calibration sections.\n\n3. The paper could potentially include a more extensive discussion on the computational complexity and scalability of the various importance weighting methods, which are crucial considerations for large-scale machine learning tasks.\n\n4. Although the survey mentions the use of importance weighting in deep learning, it may not fully capture the details of its utilization in deep learning. I think this part could be extend considering that deep learning is dominating modern machine learning.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper provides an overview of the concept of importance weighting, its applications, and related research in machine learning. The paper discusses various methods for handling distribution shift, such as covariate shift, target shift, and sample selection bias, highlighting the effectiveness of importance weighting in these scenarios. The authors also present the role of importance weighting in addressing various machine learning tasks such as active learning, model calibration and so on.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "Please refer to Weaknesses.", "broader_impact_concerns": "N/A.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: As the authors emphasized in the paper, importance weighting is an essential technique in machine learning.
This paper successfully summarizes the broad yet relevant topics in importance weighting.
Moreover, as the authors noted through the paper, some popular methods can be interpreted as a special case of importance weighting even though they do not explicitly state in that way.
This paper provides a unified view of these existing techniques through the lens of importance weighting, which provides several benefits listed in conclusion.

In the reviews, all the reviewers were positive towards this paper.
The authors addressed the concenrs raised by the reviewers, which effectively improved the contents of the paper.

Lastly, I would like to ask the authors to proofread the paper for the final submission.
Few corrections will be needed including typos, mostly in the updated parts, as below.

> p.3

`Kernel mean matching Huang et al. (2006); Gretton et al. (2009)` will be something like `Kernel mean matching (Huang et al. 2006; Gretton et al. 2009)`
The same goes for `(KLIEP) Nguyen et al. (2010); Sugiyama et al. (2007b)` and `(LSIF) Kanamori et al. (2009)`

> p.7, Theorem 1

If I understand correctly, this should be 

$$
\mathbb{E}\_{tr}[w(x)\ell(h(x), y) | s = 1] = \mathbb{E}\_{te}[\ell(h(x), y)], 
$$

which states that the IWERM under the selection bias in the left is equivalent to the test error in the right.

> p.8, Eq.(22)

This will be $\cap$ instead of $\cup$.

> p.11

A space is missing after (2018b) in `Zhang et al. (2018b)suggested`

> p.15

There is a space in the beginning of the sentence `where $w(x) = ...$` just after Eq.(31).

> p.18

``where $n_{v}al$`` will be ``where $n_{val}$``

There is a space in the beginning of the sentence `Through experiments, they empirically ...` just after Eq.(35).

> p.20

The term ``computatinal complexity`` here seems to refer something like `error bound`.
Please consider whether the use of this term is appropriate.
The term ``computatinal complexity`` generally refers to time or space complexity needed to execute an algorithm, but not the quality of the output of an algorithm such as the decay of an estimation error with respect to the data size.
Decision: Accept (poster)