Understanding new tasks through the lens of training data via exponential tilting
Authors: Subha Maity
Mikhail Yurochkin
Moulinath Banerjee
Yuekai Sun
Venue: ICLR 2023 poster (2023)
Keywords: Out-of-distribution generalization
model selection
subpopulation shift
concept drift

Abstract: Deploying machine learning models on new tasks is a major challenge due to differences in distributions of the train (source) data and the new (target) data. However, the training data likely captures some of the properties of the new task.  We consider the problem of reweighing the training samples to gain insights into the distribution of the target task. Specifically, we formulate a distribution shift model based on the exponential tilt assumption and learn train data importance weights minimizing the KL divergence between labeled train and unlabeled target datasets. The learned train data weights can then be used for downstream tasks such as target performance evaluation, fine-tuning, and model selection. We demonstrate the efficacy of our method on Waterbirds and Breeds benchmarks.

Published as a conference paper at ICLR 2023
UNDERSTANDING NEW TASKS THROUGH THE LENS OF TRAINING DATA VIA EXPONENTIAL TILTING
Subha Maity
Mikhail Yurochkin
Moulinath Banerjee
Department of Statistics University of Michigan sma i ymyuuiiimoo ch. edu
IBM Research
Department of Statistics University of Michigan
MIT-IBM Watson AI lab mi khaiili 11 ..riill.uuuooo yurochk e bm. com
mou 1 i umi ch. edu
Yuekai Sun
Department of Statistics University of Michigan yueka i @umi ch. edu
ABSTRACT
Deploying machine learning models on new tasks is major challenge due to dif- ferences in distributions of the train (source) data and the new (target) data. How- ever, the training data likely captures some of the properties of the new task. We consider the problem of reweighing the training samples gain insights into the distribution of the target task. Specifically, we formulate distribution shift model based on the exponential tilt assumption and learn train data importance weights minimizing the KL divergence between labeled train and unlabeled target datasets. The learned train data weights can then be used for downstream tasks such as tar- get performance evaluation, fine-tuning, and model selection. We demonstrate the efficacy of our method on WATERBIRDS and BREEDS benchmarks.
INTRODUCTION
Machine learning models are often deployed in target domain that differs from the domain in which they were trained and validated in. This leads to the practical challenges of adapting and evaluating the performance of models on new domain without costly labeling of the dataset of interest. For example, in the Inclusive Images challenge (Shankar et al., 2017), the training data largely consists of images from countries in North America and Western Europe. If model trained this data is presented with images from countries in Africa and Asia, then (i) it is likely to perform poorly, and (ii) its performance in the training (source) domain may not mirror its performance in the target domain. However, due to the presence of a small fraction of images from Africa and Asia in the source data, may be possible to reweigh the source samples mimic the target domain.
In this paper, we consider the problem of learning set of importance weights so that the reweighted source samples closely mimic the distribution of the target domain. We pose an exponential tilt model of the distribution shift between the train and the target data and an accompanying method that leverages unlabeled target data fit the model. Although similar methods are widely used in statistics Rosenbaum & Rubin (198)) and machine learning Sugiyama et al. (2012) to train and evaluate models under covariate shift (where the decision function/boundary does not change), one of the main benefits of our approach is it allows concept drift (where the decision boundary/function are expected differ) (Cai & Wei 2019 Gama al. 2014) between the source and the target domains. We summarize our contributions below:
In Section 3 we develop a model and an accompanying method for learning source importance
weights to mimic the distribution of the target domain without labeled target samples. In Section 4 we establish theoretical guarantees on the quality of the weight estimates and their utility in the downstream tasks of fine-tuning and model selection.
Cooees be found in http://dihiaoooommmmiyyymlllolot

---

Published as a conference paper at ICLR 2023
We demonstrate applications of our method on WATERBIRDS (Sagawa et al. 2019) (Section BREEDS (Santurkar et al. 2020) (Section [ and synthetic (Appendix datasets.
RELATED WORK
Outtof-distribution generalization is essential for safe deployment of ML models. There are two prevalent problem settings: domain generalization and subpopulation shift (Koh et al. 2020). Do- main generalization typically assumes access to several datasets during training that are related to the same task, but differ in their domain environment (Blanchard et al., [201]] MMandet et al.) 2013). The goal is to learn a predictor that can generalize to unseen related datasets via learning invariant representations (Ganin al., 2016 Sun & Saenko (2016), invariant risk minimization (Ar- jovsky et al. 2019 Krueger et 2021 ). or meta-learning (Dou et al.] 2019)). Domain generalization a very challenging problem and recent benchmark studies demonstrate that corresponding meth- ods rarely improve over vanilla empirical risk minimization (ERM) on the source data unless given access to labeled target data for model selection (Gulrajani & Lopez-Paz! 2020; Koh et al. 2020).
Subpopulation shift setting assumes that both train and test data consist of the same groups with different group fractions. This setting typically approached via distributionally robust optimiza- tion (DRO) to maximize worst group performance (Duchi et al. 2016 Sagawa et al. 2019), various reweighing strategies (Shimodairal 2000) Byrd & Lipton] 2019, Sagawa al.) [220]] Idrissi et al.] 2021]. These methods require group annotations which could be expensive to obtain in practice. Several methods were proposed to sidestep this limitation, however they still rely on a validation set with group annotations for model selection to obtain good performance (Hashimoto et al. 2018 Liu ct al. 2021 Zhai et al., 2021; Creager et al.) 202T). Our method is most appropriate for the subpop- ulation shift setting (see Section however it differs in that it does not require group annotations, but requires unlabeled target data.
Model selection on out-of-distribution (OOD) data is an important and challenging problem as noted by several authors (Gulrajani & Lopez- Paz 2020 Koh et al.) [020) /hai et al. [002]] [002|/Cres Creager al. 2021) Xu & Tibssiraani (2022); Chen et (202 propose solutions specific
covariate
shift based on parametric bootstrap and reweighing; Garg et al. (202)); Guillory et al. 1); Yu
Jiann al. (2022) align model confidence and accuracy with threshold: Jiang et al. (2021 Chen et al.
(2021a) tram several models and use their ensembles or disagreement. Our importance weighting approach is computationally simpler than the latter and is more flexible in comparison to the former, as it allows for concept drift and can be used in downstream tasks beyond model selection as we demonstrate both theoretically and empirically.
Domain adaptation is another closely related problem setting. Domain adaptation (DA) methods require access to labeled source and unlabeled target domains during training and aim to improve target performance via combination of distribution matching (Ganin et al. 2016 [Sun& & Saenko 2016 Shen et al. 2018), self- ooff-training (Shu et al., 2018 Kumar et al.) 2020). data augmentation (Cai et al.] 2021| Ruan et al. 2021), and other regularizers. DA methods are typically challenging to train and require retraining for every new target domain. On the other hand, our importance weights easy learn for a new domain allowing for efficient fine-tuning, similar to test-time adaptation methods (Sun et al.] 2020 Wang et al.) 2020/Zhang et al., (020), which adjust the model based on the target unlabeled samples. Our importance weights can also be used to define additional regularizers to enhance existing DA methods.
Importance weighting has often been used in the domain adaptation literature on label shift (Lipton) et al. 2018; Azizzadenesheli et al.) 2019: Maity et al., 2022) and covariate shift (Sugiyama et al. 2007 Hashemi & Karimi 2018) but the application has been lacking in the area of concept drift models (Cai & Wei 2019, Maity et al. 2021), due to the reason that it is generally impossible to estimate the weights without seeing labeled data from the target. In this paper, we introduce an ex- ponential tilt model which accommodates concept drift while allowing us to estimate the importance weights for the distribution shift.
3 THE EXPONENTIAL. TILT MODEL
Notation We consider a K-class classification problem. Let X' E Rd and y & [K] be the space of inputs and set of possible labels, and P and Q be probability distributions on X' x y for the
2

---

Published as a conference paper at ICLR 2023
source and target domains correspondingly. A (probabilistic) classifier is map 8 A
We define pfx,1 Y = k] as the weighted source class conditional density, i.e. p(æ, Y k} = pfz
Y = k} x P(Y k), where pf Y = k} is the density of the source feature distribution in class k and P(Y = k is the class probability in source. We similarly define g(,,, Y = k} for target.
We consider the problem of learning importance weights on samples from source domain that the weighted source samples mimic the target distribution. We assume that the learner has access to labeled samples {(Xps, )== from the source domain and and unlabeled samples { from the target domain. The learner's goal is to estimate a weight function w(x, y) > 0 such that
(X,, (Xp, Yr)] R ((g Yo)] for all (reasonable) g X' x y - R.
(3.1)
Ideally, w = JP is the likelihood ratio between the source and target domains (this leads to equality in (..)), but learning this weight function is generally impossible without labeled samples from the target domain (David et al. 2010). Thus we must impose additional restrictions on the domains.
4Q
The exponential tilt model We assume that there a vector of sufficient statisties T X - R and the parameters {0,- E R2, Ork E R] k= such that
log
d<x,Y=k] p[z,Y=k]
0 T(x) + ag for all & E [K];
(3.2)
Le q(=,Y====>>. 1}} is a member of the exponential family with base measure p(=, k} and sufficient statistics T. We call (3.2) the exponential tilt model. It implies the importance weights between the source and target samples are
w(r v) exp(O, T(x) + aay)
Model motivation The exponential tilt model is motivated by the rich theory of exponential fam- ilies in statistics. In machine learning, it was used for learning with noisy labels and for improving worst-group performance when group annotations are available (Li et al. 2020 202T). It is also closely related to several common models in transfer learning and domain adaptation. In particu- lar, it implies there is a linear concept drift between the source and target domains. It also extends the widely used covariate shift (Sugiyama & Kawanabe 2012) and label shift models (Alexandari let al. 2020 Lipton et al.) 2018 Azizzadenesheli et al. 2019 Maity et al. 2022, Garg et al.] 2020) of distribution shifts. It extends the covariate shift model because the exponential tilt model permits (linear in T(X)) concept drifts between the source and target domains; it extends the label shift model because allows the class conditionals to differ between the source and target domains. does, however, come with limitation: implicit in the model is the assumption that there is some amount of overlap between the source and target domains. In the subpopulation shift setting, this assumption is always satisfied, while in domain generalization it may be violated if the new domain drastically differs from the source data (see Appendix C for a synthetic data example).
Choosing T The goal of T is to identify the common subpopulations across domains, such that
(Xp,Yp) (T(Xp) = -, Y, = k. a (Xq: (T(XQ) = t, Y, = k}. k}.
If T segments the source domain into its subpopulations (i.e. the subpopulations are { y) X' x y' T(z) t, y k} for different values of t's and kts), then it is possible to achieve perfect reweighing of the source domain with the exponential tilt model: the weight of the (T(X) = t, Y k) subpopulation is exp(O) ( + ou.. However, in practice, such T that perfectly segments the subpopulations may not exist (e.g. the subpopulations may overlap) or is very hard to learn (e.g. we don't have prior knowledge of the subpopulations to guide T)).
no prior knowledge of the domains is available, we can use a neural network to parameterize T and learn its weights along with the tilt parameters, or simply use a pre-trained feature extractor as T. which we demonstrate to be sufficiently effective in our empirical studies. We also study the effects of misspecification of T using a synthetic dataset example in Appendix C
Fitting the exponential tilt model We fit the exponential tilt model via distribution matching. This step is based on the observation that under the exponential tilt model (3.2
ex (x) Ekink p(z, Y k))xp(0 T(x) + ox),
(3.3)
3

---

Published as a conference paper at ICLR 2023
where 9x is the (marginal) density of the inputs in the target domain. It is possible to obtain an estimate qx of qx from the unlabeled samples (Xiq]== and estimates piz, k) of the pl, Y k} from the labeled samples {(Xi, (ip This suggests we find 0g 's and og such that
Ekm = k) T(x) + ox) R (x).
Note that the 0g 's and Ork 's are dependent because 7x must integrate to one. We enforce this restric- tion constraint in the distribution matching problem:
arg min min((on, (( aN)) D
(xx (=)] E F1., Y = T(xx + +ax))
((0,,āz) E
subject to EE- k)exp(0 ox)de=1 1,
(3.4)
where a discrepancy between probability distributions on Although there are many possible choices of D, pick the Kullback-Leibler (KL) divergence in the rest of this paper because it leads to some computational benefits. We reformulate the above optimization for KL-divergence to relax the constraint which we state in the following lemma.
Lemma 3.1. If D is the Kullback-Leibler (KL. divergence then optima in ((Ox, Hx))k= where
3.3 is achieved at
{(0,, OW)K arg max (ax.maxX( (10, ²ax [log Ek (X) + +4
log (Ep[exp(0) (x))) + +b)}
T- = (h 1K is probabilistic classifier for P and 0g = a - log E exp T(X) + +}}]]]
One benefit of minimizing the KL divergence is that the learner does not need to estimate the p(z, Y = k)'s, a generative model that is difficult to train. They merely need to train a discrimina- tive model to estimate îir from the (laheled) samples from the source domain.
We plug the fitted Ôg's and âg 's into (35 obtain Exponential Tilt Reweighting Alignment (Ex- TRA) importance weights:
((x v) = exp(0, + &,y) We summarize the ExTRA procedure in Algorithm in Appendix B.2
(3.5)
Next we describe two downstream tasks where ExTRA weights can be used:
ExTRA model evaluation in the target domain. Practitioners may estimate the target perfor- mance of a model in the target domain by reweighing the empirical risk in the source domain:
E ((()())), YQ) L EE ((f(),;)) (p,))(XP,,, Yp,)),
(3.6)
where l is a loss function. This allows to evaluate models in the target domain without target labeled samples even in the presence of concept drift between the training and target domain. 2. ExTRA fine-tuning for target domain performance. Since the reweighted empirical risk (in the source domain) good estimate of the risk in the target domain, practitioners may fine-tune models for the target domain by minimizing the reweighted empirical risk:
fo E arg mig E (X)) ())(X
(3.7)
We note that the correctness of depends the identifiability of the 0 and a 's from (3.)) fee. the uniqueness of the parameters that satisfy (3.3). As long as the tilt parameters are identifiable, then (3.4) provides consistent estimates of them. However, without additional assumptions on the p(a, == ks's i and T, the tilt parameters are generally unidentifiable from (33). Next we elaborate on the identifiability of the exponential tilt model.
THEORETICAL PROPERTIES OF EXPONENTIAL TILTING
4.1 IDENTIFIABILITY OF THE EXPONENTIAL TILT MODEL
To show that the 0g 's and ag 's are identifiable from (3.3), we must show that there is a unique solu- tion (33 Unfortunately, this is not always the case. For example, consider linear discriminant
4

---

Published as a conference paper at ICLR 2023
analysis (LDA) problem in which the class conditionals drift between the source and target domains:
p Y = k} T&0(I HP,,), g(x, Y k} =T&Ō(I - (Q.,),
where & is the standard multivariate normal density, E (0, are the class proportions in both source and target domains, and j...'s 's (resp. the μQ..k 's) are the class conditional means in the source (resp. target) domains. We see that this problem satisfies the exponential tilt model with T(x)
(zz log p(z,==k] = (1Q., - / To x 21lmo. 11 + 21lurr1112
This instance of the exponential tilt model is not identifiable. Any permutation of the class labels [K] + [K] also leads to the same (marginal) distribution of inputs:
Et= Y = keexp (uuq, - (r... Ta + 2 1 =11
= p(z,y Y eee (('Q MP,x) Tx x + 1 11 - [I)
From this example, we see that the non-identifiability of the exponential tilt model is closely related to the label switching problem in clustering. Intuitively, the exponential tilt model in the preceding example is too flexible because it can tilt any p(z, Y k} to g(x, Y I). Thus there is ambiguity in which pf Y = }} tilts to which (x, , Y = 1}. In the rest of this subsection, we present an identification restriction that guarantees the identifiability of the exponential tilt model.
A standard identification restriction in related work on domain adaptation is a clustering assumption. For example, Tachet al. (2020) assume there is partition of X' into disjoint sets Nr such that PPp Y = k)), (Qp Y = k}} C X₂ for all E [K]. This assumption is strong: implies there is a perfect classifier in the source and target domains. Here we consider weaker version of the clustering assumption: there are sets S₁ such that
P(Y = k X Sz) = Q(Y = k Sx)=
We note that the S 's can be much smaller than the X3 s; this permits the supports of P(. Y = k} and Pf- Y 1} to overlap.
Definition 4.1 (anchor set), A set S₁ C X is an anchor set for class k if p(x, Y = k} > and p(a, Y [] 0, + k for all Sk.
Proposition 4.2 (identifiability from anchor sets). If there are anchor sets Sy for all K classes (in the source domain) and TScc) is p-dimensional, then there is most set of 02 's and ag 's that satisfies 3.3)
This identification restriction is also closely related to the linear independence assumption in Gong
ct al. (2016). Inspecting the proof of proposition 4.2 (see Appendix A.3). we see that the anchor set assumption implies linear independence of (px (exp((0) T(x) a ) for any set of 0g 's and ag 's.s. We study the anchor set assumption empirically in a synthetic experiment in Appendix [ Our experiments show that the assumption is mild and violated only under extreme data scenarios.
4.2 CONSISTENCY ESTIMATION THE TILT PARAMETERS
Here, we establish a convergence rate for the estimated tilt parameters (Lemma 3.1 and the Ex-
TRA importance weights (Equation (3.I). ). To simplify the notation, we define S(x) (1,7()) T)
as the extended sufficient statistics for the exponential tilt and denote the corresponding tilt param- eters as & = (ax, f T. We let E = (at, 07 ) be the true values of the tilt parameters & 's and let { ( <k) TT R k be the long vector containing all the tilt parameters. We recall that estimating the parameters from the optimization stated in Lemma 3.1 requires classifier fP on the source data. So. we define our objective for estimating through generic classifier 77 A Denoting (x) as the k-th co-ordinate of 77(x) we define the expected log log-likelioood objective
E) = Eox
(x)(x)xx(( (
and its empirical version as
Ln E) E Dog E The (X) S(X)) (og
To establish the consistency of MLE we first make an assumption that the loss & + -D(np, is strongly convex at the true parameter value.
5

---

Published as a conference paper at ICLR 2023
Assumption 4.3. The loss -((),,)) is strongly convex g*. i.e., there exists constant ja such that for any & it holds:
Snp, <) 2 - -i(p) 0. - E
1² -
We note that the assumption a restriction on the distribution Q rather than the objective itself. For technical convenience we next assume that the feature space is bounded.
Assumption 4,4. is bounded, iee, there exists an M > such that X C ((0,M)
Recall, from Lemma 3.1 that we need fitted source classifier 7iP to estimate the tilt parameter: 5* is estimated by maximizing (()),)) rather than the unknown ((np. {). While analyzing the convergence of we are required to control the difference () 2(np. Q). To ensure the difference small, assume the pilot estimate of the source regression function îip is consistent at some rate Tnp-
Assumption 4.5. Let (x) log 2 1 (og(np,((a))
We assume that there exist for (x)){ such that the following holds: there exists a constant c > 0 and a sequence Tup + 0 such that for almost surely [[Px] it holds
an estimators
P(Ilf(()) fz(2)112 > t) exp(- / /p t>
We use the estimated logits (frxa()) to construct the regression functions as (1) = cxp(fr=(=))/(E}=
(((ff((()))] which we use in the objective stated in Lemma 3.1 analyze the convergence of the tilt parameter estimates and the ExTRA weights. With the above assumptions re now ready to state concentration bounds for Ç -- and w", where the true importance weight w defined as (*, y) exp(Eī ((x)).
Theorem 4.6. Let the assumptions 4.3
and 4.5 hold. For the sample sizes np, nQ define 1)ppmp) 1/2 + 1)Knng) | There exists constants
4.4
anr.no -nr (log(nQ) + {(p +
k1, K2 such that for any 8 > 0 with probability at least 1 (2K 1) the following hold:
l| - -- < ki Vlog(1/5), and ||ا - wl,, < k2aap.ng Vlo((16))
In Theorem 4.6 we notice that as long as Top log(nq) + 0 for np, nQ + oo we have Cap..ng + 0- This implies both the estimated tilt parameters and the ExTRA weights converge to their true values as the sample sizes n.p, nQ - 30.
We next provide theoretical guarantees for the downstream tasks (1) fine-tuning and (2) target per- formance evaluation that we described in Section R
Fine-tuning
We establish a generalization bound for the fitted model (3.7) using Weighted-ERM on source domain. We denote F as the classifier hypothesis class. For f E F and weight function X x y + R define the weighted loss function and its empirical version on source data as:
Lr(f, w) Ex(w(X, (XL((X) Y)), Lr(S.
We also define the loss function on the target data as: Lo(s) = Egg ((((())))) If ((0) Kf) the true value of the tilt parameters (.2pp Le., the following holds:
g(,, Y k} pfx, Y k))xp(aj + (#))TT(=)): k [K]
then defining w² (x. k) = expfat + (0:) as the true weight we notice that Cr(f, Lo(f), which easily observed by setting g(x, = {(f(x), y) in the display (3.).
To establish our generalization bound we require Rademacher complexity (Bartlett & Mendelson 2002) (denoted as Rap (9); see Appendix A.I for details) and the following assumption the loss Tunction.
Assumption 4.7. The loss function is bounded, i.e., for some > 0, (e(f(z), yH < B for any F. x X and y E [K].

---

Published as a conference paper at ICLR 2023
With the above definitions and the assumption establish generalization bound.
Lemma 4.8. For weight function and the source samples {(XP,,, Yp3) of size np let fo arg mez (f w). There exists constant c > O such that the following generalization bound holds with probability at least -6
CQ(fw) minjer LQ(f) S 2R (g) + Bll 10 + oV
log(1/6)
(4.1)
up
where Rap (9) is the Rademacher complexity of g Appendix A.I.
: FE defined in
In Theorem 4.6 we established an upper bound for the estimated weights w, which concludes that fo has the following generalization bound: for any 5 > O, with probability at least 1 (2K + 2)
Co(fw) minser CQ(f) & m (g) + ko waaasmm/lllae (log(1/6) + cc/log(L/0)//p
where 22 is the constant in Theorem 4.6 and is the constant in Lemma 4.8
The generalization bound implies that for large sample sizes pn, n2 - 0o) the target accuracy weighted ERM on source data well approximates the accuracy of ERM on target data.
Target performance evaluation We provide a theoretical guarantee for the target performance evaluation (3.6) using our importance weights. Here only consider the functions X' x - R which are bounded by some B > 0, i.e. (g y)1 S B for all E X' and E y. The simplest and the most frequently used example is the model accuracy which uses 0-1-loss
the loss function: y} is bounded with B = 1. For such functions we
for a model f the loss g(z, y) = [x)(() notice that (X,, Y)] = Eplg( Y jw (X, as observed display 3.1). This implies the following bound on the target performance evaluation eπror
(Eolg(X,))) - () Y) Ep ( (X, Y) Eplo(X, ()X(X,
S BE, | (X,Y) w (X, < B| -
We recall the concentration bound for || - w 1|1. from Theorem 4.6 and conclude that the esti- mated target performance in (366 converges to the true target performance at rate anp.no'
WATERBIRDS CASE STUDY
To demonstrate the efficacy of the ExTRA algorithm for reweighing the source data (i) verify the ability of ExTRA to upweigh samples most relevant to the target task; (ii) evaluate the utility of weights in downstream tasks such fine-tuning and (iii) model selection.
WATERBIRDS dataset combines bird photographs from the Caltech-UCSD Birds-200 200-2011 (CUB)
dataset (Wah al. 2011) and the image backgrounds from the Places dataset (Zhou et al.) 2017)
The birds are labeled as one of y = (waterbird, landbird] and placed on one of A water background, land background). The images are divided into four groups: landbirds
land (0);
landbirds on water (1): waterbirds on land (2); waterbirds on water (3). The source dataset is highly imbalanced, i.c. the smallest group (2) has 56 samples. We embed all images with pre-trained ResNet18 (He et al. 2016). See Appendix B.I for details.
We consider five subpopulation shift target domains: all pairs of domains with different bird types and the original test set (Sagawa et al. 2019) where all 4 groups are present with proportions vastly different from the source. For all domains, we fit ExTRA weights (with ResNet18 features T(x)) from 10 different initializations and report means and standard deviations for the metrics. See Ap- pendix B.2 for the implementation details.
EXTRA weights quality For a given target domain it is most valuable to upweigh the samples in the source data corresponding to the groups comprising that domain. The most challenging the target {1, 2) consisting only of birds appearing on their atypical backgrounds. Groups {1, 2} correspond to 5% of the source data making them most difficult to "find". To quantify the ability of EXTRA report precision (proportion of samples from groups {1, 2} within the top x% of the weights) and recall (proportion of [1,2] 2} samples within the top 2% of the weights)
to upweigh these samples
in Figure I
We notice that samples corresponding to 10% largest ExTRA weights contain slightly

---

Published as a conference paper at ICLR 2023
Table Model selection results on WATERBIRDS
larget accuracy
rank conrelation
larget groups
ExTRA
SreVal
ATC-NE
ExTRA
SrcVal
ATC-NE
(0, 2]
0.819#00.01
0.854
0.871
0.419+0.011
0.807
0.760
(1, 2]
0.741=000047
0.616
0.646
0.747=0.106
-0.519
-0.590
(0, 3}
0.978=000011
0.978
0.976
0.962=000044
0.956
0.906
[1,3]
0.757100.011
0.737
0.747
0.361-10.16
-0,318
-0.411
(0, 1, 31
0.856.10.034
0.803
0.818
0.658
0.263
0.178
average
0.83
0.798
0.812
0.753
0.166
0.110
over 80% of the groups {1, 2 in the source data (recall). This demonstrates the ability of ExTRA to upweigh relevant samples. We present examples of upweighted images and results for other target
domains in Appendix B.3
Model fine-tuning We demonstrate the utility of ExTRA weights in the fine-tuning (3.7p. The basic goal of such importance weighing to improve the performance in the target in comparison to training on uniform source weights T, iee. ERM. Another baseline is the DRO model (Hashimoto et al. 2018) that aims to maximize worst-group performance without access to the group la- bels, and JTT (Liu et al. 2021) that retrains a model after upweighting the misclassified samples by ERM. We con- sider two additional baselines that utilize group annota- tions to improve worst-group performance: re-weighing the source equalize group proportions (RWgr) gr) and group DRO (gDRO) (Sagawa et af., (019). The afore- mentioned baselines do not try adjust the target domain. Finally, we compare to тT T that fine-tunes the model only using the subset of the source samples corresponding to the target domain groups. In all cases we use logistic regression as model class.
precision for groups (1.2)
10
groups {1.2}
0.4
precision
0:2
02
10
x weights Figure 1: ExTRA precision and recall for samples with top 4% weights.
We compare target accuracy across domains in Fig- ure[] Analogous comparison with area under the re- ceiver operator curve can be found in Gig Ap- pendix [ Model trained with EXTRA weights outperforms all "fair" baselines and matches the per- formance of the three baselines that had access to ad- ditional information. In all target domains ExTRA fine-tuning is comparable with the TT -> support- ing its ability upweigh relevant samples. Notably, on (1,2) domain of both minority groups and on (0.3) domain of both majority groups, EXTRA out- performs RW gr and gDRO that utilize group annota- tions. This emphasizes the advantage of adapting to the target domain instead of pursuing more conser- vative goal of vorst-group performance maximization. Finally, we note that ExTRA fine-tuning did not perform as well on the domain [1,3), however neither did TT -> T.
target accuraci
0.6
RW
DRO
JT
0.2
EXTRA
0.0
(0.
(.22}
(1.
2)
(0.
(0.3)
(1. 3)
10. 1. 3)
average
target domain groups
Figure Performance on WATERBIRDS.
Model selection out-of distriibbit is an important task, that is difficult to perform without target data labels and group annotations (Gulrajani & Lopez-Paz, 2020: Zhai et al. 2021). We evaluate the ability of choosing a model for the target domain based on accuracy on the EXTRA reweighted source validation data. We compare to the standard source validation model selection (SreVal) and to the recently proposed ATC-NE (Garg et al., 2022) that uses negative entropy of the predicted probabilities on the target domain score models. We fit a total of 120 logistic regression models with different weighting (uniform, label balancing, and group balancing) and varying regularizers. See Appendix [B2 for details.
In Table we compare the target performance of models selected using each of the model evaluation scores and rank correlation between the corresponding model scores and true target accuracies.
8

---

Published as a conference paper at ICLR 2023
Model selection with ExTRA results in the best target performance and rank correlation on 4 out of 5 domains and on average. Importantly, the rank correlation between the true performance and ExTRA model scores is always positive, unlike the baselines, suggesting its reliability in providing meaningful information about the target domain performance.
6 BREEDS CASE STUDY
BREEDS (Santurkar et al., 2020) is a subpopulation shift benchmark derived from ImageNet (Deng et al. 2009). It uses the class hierarchy to define groups within classes. For example, in the Entity- 30 task considered in this experiment, class fruit is represented by strawberry, pineapple, jackfruit, Granny Smith in the source and buckeye, corn, ear, acorn in the target. This is an extreme case of subpopulation shift where source and target groups have zero overlap. We modify the dataset by adding a small fraction of random samples from the target to the source for two reasons: (i) our exponential tilt model requires some amount of overlap between source and target; (ii) arguably, in practice, it is more likely that the source dataset has at least small representation of all groups.
Our goal is to show that EXTRA can identify the target samples mixed into the source for efficient fine-tuning. We obtain feature representations from a pre-trained self- supervised SwAV (Caron et al., 2020). To obtain the Ex- TRA weights we use SWAV features as sufficient statistic. We then train logistic regression models on (i) the source dataset re- weighted with ExTRA, (ii) uniformly weighted source (S -> T), (iii) target samples mixed into the source (xT -> T), (iv) all target samples (oracle). See Appendix B. B.2 for details. We report performance for varying mixing proportion ı in Figure 3 First, we note that even when π = 0, L.c. source and target have completely dis- joint groups (similar to domain generalization). ExTRA improves over the vanilla S -> T. Next, see that $ -> T improves very slowly in comparison to EXTRA as we increase the mixing propor- tion; T -> T improves faster as we increase the number of target samples it has access to, but never suppresses ExTRA and matches its improvement slope for the larger π values. We conclude that ExTRA can effectively identify target samples mixed into source that are crucial for the success of fine-tuning and find source samples most relevant to the target task allowing it to outperform xT T. We report analogous precision and recall for the WATERBIRDS experiment in Appendix B33
argeet accuracy
0.7
target
0.6
T-T
EXTRA
0.5
oracle
0.0 0.0025 0.005 0.01
0.02
0.01 0.08 0.16
mixing proportion
Figure 3: Performance BREEDS.
CONCLUSION
In this paper, we developed an importance weighing method for approximating expectations of inter- est new domains leveraging unlabeled samples (in addition to labeled dataset from the source domain). We demonstrated the applicability of our method on downstream tasks such as model evaluation/selection and fine-tuning both theoretically and empirically. Unlike other importance weighing methods that only allow covariate shift between the source and target domains, we permit concept drift between the source and target. Though we demonstrate the efficacy of our method in synthetic setup of concept drift (Appendix[C) in a future research it would be interesting to in- vestigate the performance in more realistic setups (e.g. CIFARI0.2 to CIFARIO.2 (Lu et al. 2020). Imagenet to Imagenetv2 (Recht et al. 2019))
Despite its benefits, the exponential tilt model does suffer from a few limitations. Implicit in the exponential tilt assumption is that the supports of the target class conditionals have some overlap with the corresponding source class conditionals. Although this assumption likely satisfied in many instances of domain generalization problems (and is always satisfied in the subpopulation shift setting), an interesting avenue for future studies is to accommodate support alignment in the distribution shift model, í.e. to align the supports for class conditioned feature distributions in source and target domains. One way to approach this is to utilize distribution matching techniques from domain adaptation literature (Ganin et al. 2016; Sun & Saenko, 2016 Shen et al. 2018). similarly to (Ca et al. (202I). We hope aligning supports via distribution matching will allow our method to succeed on domain generalization problems where the support overlap assumption is violated.

---

Published as a conference paper at ICLR 2023
8 ETHICS STATEMENT
We recommend considering the representation of the minority groups when applying ExTRA in the context of faimmes--ennsitive applications. The goal of ExTRA is to approximate the distribution of the target domain, thus, in order to use ExTRA weights for fine-tuning or model selection to obtain a fair model, the target domain should be well representative of both privileged and unprivileged groups. If the target domain has miss/under-representee groups, model obtained using ExTRA weights may be biased.
ACKNOWLEDGMENTS
This paper is based upon work supported by the National Science Foundation (NSF) under grants 2027737 and 21 113333.
REFERENCES
Amr Alexandari, Anshul Kundaje, and Avanti Shrikumar. EM with Bias-Corrected Calibration is Hard-To-Beat at Label Shift Adaptation. arXiv:1901.06832 [cs, stat], January 2020.
Martin Arjovsky, Léon Bottou, Ishaan Gulrajani, and David Lopez-Paz. Invariant Risk Minimiza- tion. arXiv: [cs, stat), September 2019.
Kamyar Azizzadenesheli, Anqi Liu, Fanny Yang, and Animashree Anandkumar. Regularized Learn- ing for Domain Adaptation under Label Shifts.
[cs, stat], March 2019.
Peter L Bartlett and Shahar Mendelson. Rademacher and gaussian complexities: Risk bounds and structural results. Journal of Machine Learning Research, 3(Nov):463 482, 2002.
Gilles Blanchard, Gyemin Lee, and Clayton Scott. Generalizing from several related classification tasks to a new unlabeled sample. In Proceedings of the 24th International Conference on Neural Information Processing Systems, NIPS' 11, pp. 2178 2186, Red Hook, NY, USA, December 2011
Curran Associates Inc. ISBN
Jonathon Byrd and Zachary Lipton. What is the Effect of Importance Weighting in Deep Learning? In International Conference on Machine Learning, pp. 872-881. PMLR, May 2019.
T. Tony Cai and Hongji Wei. Transfer Learning for Nonparametric Classification: Minimax Rate and Adaptive Classifier. MxXiv:990.00290 [cs, math, stat], June 2019.
Tianle Cai, Ruiqi Gao, Jason D. Lee, and Qi Lei. A Theory of Label Propagation for Subpopulation Shift, arXiv:2102. 11203 [cs, stat), February 2021.
Mathilde Caron, Ishan Misra, Julien Mairal, Priya Goyal, Piotr Bojanowski, and Armand Joulin. Unsupervised learning of visual features by contrasting cluster assignments. Advances in Neural Information Processing Systems, 13:9912-99924 2020.
Jiefeng Chen, Frederick Liu, Besim Avci, Xi Wu, Yingyu Liang, and Somesh Jha. Detecting errors and estimating accuracy unlabeled data with self-training ensembles. Advances in Neural Information Processing Systems, 34, 2021a.
Mayee Chen, Karan Goel, Nimit S Sohoni, Fait Poms, Kayvon Fatahalian, and Christopher Ré. Mandoline: Model evaluation under distribution shift. In International Conference on Machine Learning, pp. 1617-1629. PMLR, 2021b.
Elliot Creager, -eem-- Jacobsen, and Richard Zemel. Environment Inference for Invariant Learning. In Proceedings of the 38th International Conference Machine Learning, pp. 2189- 2200. PMLR, July 2021.
Shai Ben David, Tyler Lu, Teresa Luu, and David Pal. Impossibility Theorems for Domain Adap- tation. In Proceedings of the Thirteenth International Conference on Artificial Intelligence and Statisties, pp. 129-136. JMLR Workshop and Conference Proceedings, March 2010.
10

---

Published as a conference paper at ICLR 2023
Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scate scale hi- erarchical image database. In 2009 JEEE conference on computer vision and pattern recognition, pp. 248-255. Icee, 2009.
Qi Dou, Daniel C. Castro, Konstantinos Kamnitsas, and Ben Glocker. Domain Generalization via Model-Agnostic Learning of Semantic Features. arXiv:1910. 13580 [cs], October 2019.
John Duchi, Peter Glynn, and Hongseok Namkoong. Statistics of Robust Optimization: A General-
ized Empirical Likelihood Approach. arXiv:/610.0342) [star], October 2016.
João Gama, Indré Žliobaité, Albert Bifet, Mykola Pechenizkiy, and Abdelhamid Bouchachia. A survey on concept drift adaptation. ACM Computing Surveys, (6(4):44:1-44:333 March 2014. ISSN 0360-0300. doi: 10.1 10.1145/253383.
Yaroslav Ganin, Evgeniya Ustinova, Hana Ajakan, Pascal Germain, Hugo Larochelle, François Laviolette, Mario Marchand, and Victor Lempitsky. works. The Journal of Machine Learning Research, 17(1):2096 January 2016. ISSN 1532-4435.
training of neural net-
Saurabh Garg, Yifan Wu, Sivaraman Balakrishnan, and Zachary C. Lipton. A Unified View of Label Shift Estimation. arXiv:2003.07554 [es, stat), March 2020.
Saurabh Garg, Sivaraman Balakrishnan, Zachary C Lipton, Behnam Neyshabur, and Hanie Sedghi. Leveraging unlabeled data to predict out-of-distribution performance. arXiv preprint arX1v:2201.04234, 2022.
Mingming Gong, Kun Zhang, Tongliang Liu, Dacheng Tao, Clark Glymour, and Bernhard Scholkopf. Domain Adaptation with Conditional Transferable Components. Proceedings of Machine Learning Research, volume 48, pp. 17, New York, New York, USA, June 2016.
Devin Guillory, Vaishaal Shankar, Sayna Ebrahimi. Trevor Darrell, and Ludwig Schmidt. Predicting with confidence on unseen distributions. Proceedings of the IEEE/CVF International Confer- ence on Computer Vision, pp. 1134-1 114,, 2021.
Ishaan Gulrajani and David Lopez-Paz. In Search of Lost Domain Generalization. In International Conference on Learning Representations, September 2020.
Mahdi Hashemi and Hassan Karimi. Weighted machine learning. Statistics, Optimization and Information Computing, 6(4):497-525, 2018.
Tatsunori B. Hashimoto, Megha Srivastava. Hongseok Namkoong, and Percy Liang, Fairness With- out Demographics in Repeated Loss Minimization. crXiv:1806.08011 [cs, stat), June 2018.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep Residual Learning for Image Recognition. 2016 IEEE Conference on Computer Vision and Puttern Recognition (CVPR), pp. 770-778, Las Vegas, NV, USA, June 2016. IEEE. ISBN 978-1-4673-8851- doi: 10.1 CVPR.2016.90.
Badr Youbi Idrissi, Martin Arjovsky, Mohammad Pezeshki, and David Lopez-Paz. Simple data balancing achieves competitive worst-group-accuracy arXiv preprint 10.iv:2/10.14500) 2021.
Yiding Jiang, Vaishnavh Nagarajan, Christina Baek, and Zico Kolter. Assessing generalization of sgd via disagreement. arXiv preprint arXiv:2106.133 13799, 2021.
Diederik Kingma and Jimmy Ba. Adam: A Method for Stochastic Optimization. arXiv:1412.6980 [cs], January 2017.
Pang Wei Koh, Shiori Sagawa, Henrik Marklund, Sang Michael Xie, Marvin Zhang, Akshay Bal- subramani, Weihua Hu, Michihiro Yasunaga, Richard Lanas Phillips, Sara Beery, Jure Leskovec, Anshul Kundaje, Emma Pierson, Sergey Levine, Chelsea Finn, and Percy Liang. WILDS: A Benchmark of in-the-Wild Distribution Shifts. arXiv:2012.072 07442 [cs], December 2020.
David Krueger, Ethan Caballero, Joem-Henrik Jacobsen, Amy Zhang, Jonathan Binas, Dinghuai Zhang, Remi Le Priol, and Aaron Courville. disto--distriuution generalization via risk extrapo- lation (rex). In International Conference Machine Learning, pp. 5815-5826. PMLR, 2021.

---

Published as a conference paper at ICLR 2023
Ananya Kumar, Tengyu Ma, and Percy Liang. Understanding Self-Training for Gradual Domain Adaptation. arXiv:2002.1136/ [cs, stat]. February 2020.
Ananya Kumar, Aditi Raghunathan, Robbie Jones, Tengyu Ma, and Percy Liang. Fine-Tuuing can Distort Pretrained Features and Underperform nut-of-Distribution. February 2022. doi:
00.48550/arxi.2222..15 10054.
Tian Li, Ahmad Beirami, Maziar Sanjabi, and Virginia Smith. Tilted Empirical Risk Minimization. In International Conference on Learning Representations, September 2020.
Tian Li, Ahmad Beirami, Maziar Sanjabi, and Virginia Smith. On tilted losses in machine learning: Theory and applications. arXiv preprint orXiv:2109.0614, 2021.
Zachary C. Lipton, Yu-Xiang Wang, and Alex Smola. Detecting and Correcting for Label Shift with Black Box Predictors. arXiv:1802.03916 [cs, stat], July 2018.
Evan Z. Liu, Behzad Haghgoo. Annie S. Chen, Aditi Raghunathan, Pang Wei Koh, Shiori Sagawa, Percy Liang, and Chelsea Finn. Just Train Twice: Improving Group Robustness without Training Group Information. In Proceedings of the 38th International Conference on Machine Learning, PMLR, July 2021.
pp.
Shangyun Lu, Bradley Nott, Aaron Olson, Alberto Todeschini, Hossein Vahabi, Yair Carmon, and Ludwig Schmidt. Harder or different? closer look distribution shift in dataset reproduction. In ICML Workshop on Uncertainty and Robustness in Deep Learning, 2020.
Subha Maity, Diptavo Dutta, Jonathan Terhorst, Yuekai Sun, and Moulinath Banerjec. A linear adjustment based approach to posterior drift in transfer learning. arXiv:2111.10841 [stat], De- cember 2021.
Subha Maity, Yuekai Sun, and Moulinath Banerjee. Minimax optimal approaches the label shift problem in non-parametric settings. Journal of Machine Learning Research, 23(346):1-45, 2022.
Krikamol Muandet, David Balduzzi, and Bernhard Schölkopf. Domain Generalization via Invariant Feature Representation. January 2013.
Benjamin Recht, Rebecca Roclofs, Ludwig Schmidt, and Vaishaal Shankar. Do imagenet classifiers generalize to imagenet? In International Conference on Machine Learning, pp. 5389-5400.
PMLR, 2019.
Paul R Rosenhaum and Donald Rubin. The central role of the propensity score in observational studies for causal effects. Biometrika, 70(1):41-55, 1983.
Elan Rosenfeld, Pradeep Ravikumar, and Andrej Risteski. Domain-adjuster regression or: Erm may already learn features sufficient for out-of-distribution generalization. arXiv preprint arXiv:2202.0668 06856, 2022.
Yangjun Ruan, Yann Dubois, and Chris Maddison. Optimal representations for covariate shift. arXiv preprint arXiv:2201.00057 2021.
Shiori Sagawa, Pang Wei Koh, Tatsunori B. Hashimoto, and Percy Liang. Distributionally Robust Neural Networks for Group Shifts: On the Importance of Regularization for Worst-Case Gener- alization. arXiv:1911.087. [cs, stat], November 2019.
Shiori Sagawa, Aditi Raghunathan, Pang Wei Koh, and Percy Liang. An Investigation of Why Overparameeerrzzatoon Exacerbates Spurious Correlations. arXiv:2005.04:: 04345 [cs, stat), August 2020.
Shibani Santurkar, Dimitris Tsipras, and Aleksander Madry. BREEDS: Benchmarks for Subpopu- lation Shift. August 2020. doi: 00.48550/axxx.2200804443
Shreya Shankar, Yoni Halpem, Eric Breck, James Atwood, Jimbo Wilson, and D. Sculley. No Classification without Representation: Assessing Geodiversity Issues in Open Data Sets for the Developing World. arXiv: [stat], November 2017.
12

---

Published as a conference paper at ICLR 2023
Jian Shen, Yanru Qu, Weinan Zhang. and Yong Yu. Wasserstein distance guided representation learning for domain adaptation. In Thirty-second AAAI conference on artificial intelligence, 2018.
Hidetoshi Shimodaira. Improving predictive inference under covariate shift by weighting the log- likelihood function. Journal of Statistical Planning and Inference, 90(2):227-244, October 2000. 15-4.
ISSN 0378-3758. doi: 10.
Avanti Shrikumar and Anshul Kundaje. Calibration with bias-corrected temperature scaling im- proves domain adaptation under label shift in modern neural networks. Preprint at https://axxiv. 0rg/abs/1901.06835 vl, 2019.
Rui Shu, Hung Bui, Hirokazu Narui, and Stefano Ermon. A DIRT-T Approach to Unsupervised Domain Adaptation. International Conference on Learning Representations, February 2018.
Masashi Sugiyama and Motoaki Kawanabe. Machine Learning in Non-Stationary Environments: Introduction Covariate Shift Adaptation. Adaptive Computation and Machine Learning Series. MIT Press, Cambridge, MA, USA, March 2012. ISBN 978-0-262-01709-11
Masashi Sugiyama, Matthias Krauledat, and Glaus-Robert Müller. Covariate Shift Adaptation by Importance Weighted Cross Validation. The Journal of Machine Learning Research, 8:985-1005, December 2007. ISSN 1532-4435.
Masashi Sugiyama, Taiji Suzuki, and Takafumi Kanamori. Density Ratio Extimation in Machine Learning, Cambridge University Press, Cambridge, 2012. ISBN 978-0-521-19017-6. doi: 10.
1017/C88097811993356.3
Baochen Sun and Kate Saenko. Deep CORAL: Correlation Alignment for Deep Domain Adaptation. In Gang Hua and Hervé Jégou (eds.), Computer Vision ECCV 2016 Workshops, Lecture Notes in Computer Science, pp. 443-450, Cham, 2016. Springer International Publishing. ISBN 978-3-
319-49409-8, doi: 10. (0007978-3-319-49444
Yu Sun, Xiaolong Wang, Zhuang Liu, John Miller, Alexei A. Efros, and Moritz Hardt. Test- Time Training with Supervupervii for Generalization under Distribution Shifts. arXiv:1909.1233 [cs, stat], July 2020.
Remi Tachet, Han Zhao, Yu-Xiang Wang, and Geoff Gordon. Domain Adaptation with Conditional Distribution Matching and Generalized Label Shift. March 2020.
A. W. van der Vaart and Jon A Wellner. Weak Convergence and Empirical Processes: With Appli- cations to Statistics. Springer, New York, 2000. ISBN 978-0-387--46400-5 977
Catherine Wah, Steve Branson, Peter Welinder, Pietro Perona, and Serge Belongie. The caltech-uesd birds-200-2011 dataset. 2011.
Dequan Wang, Evan Shelhamer, Shaoteng Liu, Bruno Olshausen, and Trevor Darrell. Tent: Fully Test-Time Adaptation by Entropy Minimization. International Conference on Learning Rep- resentations, September 2020.
Jon Wellner et al. Weak convergence and empirical processes: with applications statisties. Springer Science & Business Media, 2013.
Hui Xu and Robert Tibshirani. Estimation of prediction error with known covariate shift. arXiv preprint 2005002205.0189, 2022.
Yaodong Yu. Zitong Yang, Alexander Wei, Yi Ma, and Jacob Steinhardt. Predicting out-of- distribution error with the projection norm. arXiv preprint arXiv:2202.05834, 2022.
Runtian Zhai, Chen Dan, Zico Kolter, and Pradeep Ravikumar. DORO: Distributional and Outlier Robust Optimization. Proceedings of the 38th International Conference on Machine Learning. pp. 12345-12355. PMLR, July 2021.
Marvin Zhang, Henrik Marklund, Nikita Dhawan, Abhishek Gupta, Sergey Levine, and Chelsea Finn. Adaptive Risk Minimization: A Meta-Learning Approach for Tackling Group Shift.
02229) [es, stat], October 2020.
13

---

Published as a conference paper at ICLR 2023
Bolei Zhou, Agata Lapedriza, Aditya Khosla, Aude Oliva, and Antonio Torralba. Places: A 10 million image database for scene recognition. IEEE transactions on pattern analysis and machine intelligence, 4066)1455--144, 2017.
14


Reviews:
[{"review_id": "tYDk8v2LB-y", "reviewer": "Reviewer_UqFX", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper considers the problem of reweighting training samples to improve model performance on out-of-distribution test samples. The approach formulates real distribution shifts (covariate and concept related) using the exponential tilt assumption. With this assumption, the problem of improving performance on OOD samples simplifies to learning data importance weights. The paper has some theoretical analysis on the properties of exponential tilting. The paper also applies this method to improve performance on Waterbirds and BREEDS-Entity30. ", "summary_of_the_review": "See strengths and weaknesses above", "strength_and_weaknesses": "Strengths \n\n- The paper is well-written. The exponential tilt model is clearly explained. \n- Analyzing learned importance weights. \n\n\nWeaknesses\n- Exponential tilt assumption unjustified. It is not clear why complex distribution shits in practice should be parameterized with the exponential tilt model. The paper states that this problem can be applied to concept drift settings but has no experiments / analysis on concept drift.\n\n- When does this re-weighting approach fail? A more quantitative approach to this question would be insightful. The paper loosely talks about this  (i.e. distribution needs some overlap etc).\n\n- Limited empirical evaluation. Waterbirds and Entity30 are datasets where you know what the \"ground-truth\" importance weights should look like (https://proceedings.mlr.press/v177/idrissi22a.html). Showing that the proposed methods works well in these settings is a good first paper, i think the paper will be significantly stronger if the paper improves performance on more general / realistic distribution shifts (e.g. CIFAR 10.2, ImagenetV2) and then analyzes the learned importance weights. The empirical section should have additional baselines (e.g. https://proceedings.mlr.press/v162/zhou22d/zhou22d.pdf, https://proceedings.mlr.press/v177/idrissi22a.html) to clearly contrast this approach from previous methods.\n\n- Theoretical analysis somewhat tangential and not insightful vis-a-vis the paper's main focus. I would rather first read whether this method works well on realistic distribution shifts and then discuss properties like consistency and identifiability of the parameters. \n\nOverall the paper is well-written and focuses on a principled approach (reweighting) to improve OOD performance. However, the paper has two major issues: (a) exponential tilt assumption is not clearly justified, (b) empirical evaluation is quite limited.\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "hVLe8FDRo0", "reviewer": "Reviewer_3m7n", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "In this paper, the authors propose a method for domain adaptation when you have labelled data from source P and unlabelled data from target Q. Their method learns a reweighting of the source data sample through a learned exponential tilting, where a neural network learns the sufficient statistics of the tilting. They show that this tilting is identifiable when \"anchor sets\" exist (a subset of points which only have probability mass for their class label) in the source distribution, and show that when identifiable, the parameter estimate converges. Experimentally, they show that this method is competitive on domain generalization benchmarks.", "summary_of_the_review": "I think the technical work here provides a clean, effective direction to building approaches for domain adaptation/generalization without group labels. My main question/confusion with the work is re: its connection to concept drift, and I'm not convinced by the paper that this approach is relevant. But even without that, I think the work is good, so I'm recommending acceptance.", "strength_and_weaknesses": "This seems like technically strong work, and the proofs look sound and interesting. The experimental work supports the theoretical components and shows that this method is an interesting possibility for domain adaptation/generalization without group labels.\n\nSome feedback:\n- the authors argue that their method moves beyond covariate shift to concept drift - however, the experimental datasets (Waterbirds + Breeds), do not provide any instances of concept drift, only testing covariate shift. Experiments that test the method in a concept drift setting would make the paper's argument stronger\n- Prop 4.2: I don't understand what the \"T(S_k) is p-dimensional\" portion of this assumption means - I think if I understood this piece, then the identifiability argument + concept drift application might be clearer to me. Also, in the appendix, it seems to say d-dimensional instead\n-Assumption 4.5 - I don't totally understand this - I've seen constructions like it before but I don't recall how the sequence r_np is used in this statement and it's not clear from the body of the text: would be good to have this be explicit somewhere.\n- Experiments: I don't understand why the method \"\\pi T -> T\" doesn't perform close to perfectly on some of these target domain groups. In Waterbirds, some pairs of domains have all the same label (I think for instance environment 0 and 1 both are all Y=0). Then, shouldn't finetuning on data from those domains yield a really easy fine-tuning problem - learning a constant output?\n\n- page 3, choosing T: the intuition provided her around how T should identify \"subpopulations\" isn't quite clear to me, would appreciate a bit more of a walkthrough of this\n- Thm 4.6 - where is M used here? I don't see it but might be missing it\n- Lemma 4.8: remind us what G is here, I don't remember\n\n-middle of p7: guaranty -> guarantee\n\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "4: The contributions are significant, and do not exist in prior works.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "M9KxoGdZZOL", "reviewer": "Reviewer_AxBN", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper introduces a framework for computing importance weights for labeled source data with respect to a target domain where only unlabeled target data is available. Unlike previous methods, these weights allow for concept drift in addition to covariate shift. The paper proves the statistical consistency of the estimators and generalization bounds and demonstrates the efficacy of the methods over two data sets.", "summary_of_the_review": "Overall, the paper seems strong albeit with limited experiments vs baselines. The theoretical results are nice and the insights are well-supported.", "strength_and_weaknesses": "Strengths\n- The exponential tilt model is well described and the procedure for obtaining the importance weights is clear\n- The weights demonstrate value in both domain adaptation and evaluation\n- I did not thoroughly check the theoretical proofs but the general steps seem correct\n\nWeaknesses\n- Since the method is supposed to work also on domain adaptation, it would be nice to have presented results on some more standard domain adaptation tasks and baselines (e.g., Office).", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}]

Author rebuttal:

Decision: Accept: poster