SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Tongtong Fang $^{1}$  Nan Lu $^{2,3}$  Gang Niu $^{3*}$  Masashi Sugiyama $^{3,1}$
<sup>1</sup>The University of Tokyo, Japan
$^{2}$ University of Tübingen, Germany  $^{3}$ RIKEN, Japan
Distribution shift (DS) may have two levels: the distribution itself changes, and the support (i.e., the set where the probability density is non-zero) also changes. When considering the support change between the training and test distributions, there can be four cases: (i) they exactly match; (ii) the training support is wider (and thus covers the test support); (iii) the test support is wider; (iv) they partially overlap. Existing methods are good at cases (i) and (ii), while cases (iii) and (iv) are more common nowadays but still under-explored. In this paper, we generalize importance weighting (IW), a golden solver for cases (i) and (ii), to a universal solver for all cases. Specifically, we first investigate why IW might fail in cases (iii) and (iv); based on the findings, we propose generalized IW (GIW) that could handle cases (iii) and (iv) and would reduce to IW in cases (i) and (ii). In GIW, the test support is split into an in-training (IT) part and an out-of-training (OOT) part, and the expected risk is decomposed into a weighted classification term over the IT part and a standard classification term over the OOT part, which guarantees the risk consistency of GIW. Then, the implementation of GIW consists of three components: (a) the split of validation data is carried out by the one-class support vector machine, (b) the first term of the empirical risk can be handled by any IW algorithm given training data and IT validation data, and (c) the second term just involves OOT validation data. Experiments demonstrate that GIW is a universal solver for DS problems, outperforming IW methods in cases (iii) and (iv).
Deep supervised classification has been successful where the training and test data should come from the same distribution (Goodfellow et al., 2016). When this assumption does not hold in practice, we suffer from distribution shift (DS) problems and the learned classifier may often generalize poorly (Quionero-Candela et al., 2009; Pan and Yang, 2009; Sugiyama and Kawanabe, 2012). Let  $\mathbf{x}$  and  $y$  be the instance (i.e., input) and class-label (i.e., output) random variables. Then, DS means that the underlying joint density of the training data  $p_{\mathrm{tr}}(\mathbf{x},y)$  differs from that of the test data  $p_{\mathrm{te}}(\mathbf{x},y)$ .
There are two levels in the DS research. At the first level, only the change of the data distribution is considered. With additional assumptions, DS can be reduced into covariate shift  $p_{\mathrm{tr}}(\pmb{x}) \neq p_{\mathrm{te}}(\pmb{x})$ , class-prior shift  $p_{\mathrm{tr}}(y) \neq p_{\mathrm{te}}(y)$ , class-posterior shift  $p_{\mathrm{tr}}(y \mid \pmb{x}) \neq p_{\mathrm{te}}(y \mid \pmb{x})$ , and class-conditional shift  $p_{\mathrm{tr}}(\pmb{x} \mid y) \neq p_{\mathrm{te}}(\pmb{x} \mid y)$  (Quionero-Candela et al., 2009). We focus on joint shift  $p_{\mathrm{tr}}(\pmb{x}, y) \neq p_{\mathrm{te}}(\pmb{x}, y)$ , as it is the most general and difficult case of DS. At the second level, the change of the support of the data distribution is also considered, where given any joint density  $p(\pmb{x}, y)$ , its support is defined as the set  $\{(x, y) : p(x, y) > 0\}$ . More specifically, denote by  $S_{\mathrm{tr}}$  and  $S_{\mathrm{te}}$  the support of  $p_{\mathrm{tr}}(\pmb{x}, y)$  and  $p_{\mathrm{te}}(\pmb{x}, y)$ , respectively. When considering the relationship between  $S_{\mathrm{tr}}$  and  $S_{\mathrm{te}}$ , there can be four cases:
(i)  $S_{\mathrm{tr}}$  and  $S_{\mathrm{te}}$  exactly match, i.e.,  $S_{\mathrm{tr}} = S_{\mathrm{te}}$
*Correspondence to: GN <gang.niu.ml@gmail.com>, TF <fang@ms.k.u-tokyo.ac.jp>
Figure 1: An illustration of the relationship between the training support and the test support.
(ii)  $S_{\mathrm{tr}}$  is wider and covers  $S_{\mathrm{te}}$ , i.e.,  $S_{\mathrm{tr}} \supset S_{\mathrm{te}}$  and  $S_{\mathrm{tr}} \setminus S_{\mathrm{te}} \neq \emptyset$ ;
(iii)  $S_{\mathrm{te}}$  is wider and covers  $S_{\mathrm{tr}}$ , i.e.,  $S_{\mathrm{tr}} \subset S_{\mathrm{te}}$  and  $S_{\mathrm{te}} \setminus S_{\mathrm{tr}} \neq \emptyset$ ;
(iv)  $S_{\mathrm{tr}}$  and  $S_{\mathrm{te}}$  partially overlap, i.e.,  $S_{\mathrm{tr}} \cap S_{\mathrm{te}} \neq \emptyset$ ,  $S_{\mathrm{tr}} \setminus S_{\mathrm{te}} \neq \emptyset$ , and  $S_{\mathrm{te}} \setminus S_{\mathrm{tr}} \neq \emptyset$ .<sup>1</sup>
The four cases are illustrated in Figure 1. We focus on cases (iii) and (iv), as they are more general and more difficult than cases (i) and (ii).
Problem setting Denote by  $\mathcal{X}$  and  $\mathcal{Y}$  the input and output domains, where  $\mathcal{Y} = \{1,\dots ,C\}$  for  $C$  -class classification problems. Let  $\pmb {f}:\mathcal{X}\rightarrow \mathbb{R}^{C}$  be a classifier (to be trained) and  $\ell :\mathbb{R}^C\times \mathcal{Y}\rightarrow$ $(0, + \infty)$  be a loss function (for training  $\pmb {f}$ ). Then, the risk is defined as follows (Vapnik, 1998):
$$
R (\boldsymbol {f}) = \mathbb {E} _ {p _ {\mathrm {t e}} (\boldsymbol {x}, y)} [ \ell (\boldsymbol {f} (\boldsymbol {x}), y) ], \tag {1}
$$
where  $\mathbb{E}[\cdot ]$  denotes the expectation. In the joint-shift problems, we are given a training set  $\mathcal{D}_{\mathrm{tr}} = \{(x_i^{\mathrm{tr}},y_i^{\mathrm{tr}})\}_{i = 1}^{n_{\mathrm{tr}}}$  i.i.d.  $\tilde{p}_{\mathrm{tr}}(\boldsymbol {x},y)$  and a validation set  $\mathcal{D}_{\mathrm{v}} = \{(x_{i}^{\mathrm{v}},y_{i}^{\mathrm{v}})\}_{i = 1}^{n_{\mathrm{v}}}$  i.i.d.  $p_{\mathrm{te}}(\boldsymbol {x},\boldsymbol {y})$  , where  $\mathcal{D}_{\mathrm{tr}}$  is much bigger than  $\mathcal{D}_{\mathrm{v}}$  , i.e.,  $n_{\mathrm{tr}}\gg n_{\mathrm{v}}$  . The goal is to reliably estimate the risk from  $\mathcal{D}_{\mathrm{tr}}$  and  $\mathcal{D}_{\mathrm{v}}$  and train  $\pmb{f}$  by minimizing the empirical risk, which should outperform training  $\pmb{f}$  from only  $\mathcal{D}_{\mathrm{v}}$
Motivation Importance weighting (IW) has been a golden solver for DS problems (Sugiyama and Kawanabe, 2012), and there are many great off-the-shelf IW methods (Huang et al., 2007; Sugiyama et al., 2007a,b; Kanamori et al., 2009). Recently, dynamic IW (DIW) was proposed to make IW compatible with stochastic optimizers and thus it can be used for deep learning (Fang et al., 2020). However, all IW methods including DIW have assumed cases (i) and (ii)—in cases (iii) and (iv), IW methods become problematic. Specifically, as the importance weights are only used on  $S_{\mathrm{tr}}$ , even though they become ill-defined on  $S_{\mathrm{te}} \setminus S_{\mathrm{tr}}$ , IW itself is still well-defined. Nevertheless, in such a situation, the IW identity will become an inequality (i.e., Theorem 2), which means that what we minimize for training is no longer an approximation of the original risk  $R(\pmb{f})$  and thus IW may lead to poor trained classifiers (i.e., Proposition 3). Moreover, some IW-like methods based on bilevel optimization share a similar issue with IW (Jiang et al., 2018; Ren et al., 2018; Shu et al., 2019), since  $\pmb{f}$  is only trained from  $D_{\mathrm{tr}}$  where  $D_{\mathrm{v}}$  is used to determine the importance weights on  $D_{\mathrm{tr}}$ . In fact, cases (iii) and (iv) are more common nowadays due to data-collection biases, but they are still under-explored. For example, a class has several subclasses, but not all subclasses are presented in  $D_{\mathrm{tr}}$  (see Figure 2). Therefore, we want to generalize IW to a universal solver for all the four cases.
Contributions Our contributions can be summarized as follows.
- Firstly, we theoretically and empirically analyze when and why IW methods can succeed/may fail. We reveal that the objective of IW is good in cases (i) and (ii) and bad in cases (iii) and (iv).
- Secondly, we propose generalized IW (GIW). In GIW,  $S_{\mathrm{te}}$  is split into an in-training (IT) part  $S_{\mathrm{te}} \cap S_{\mathrm{tr}}$  and an out-of-training (OOT) part  $S_{\mathrm{te}} \setminus S_{\mathrm{tr}}$ , and its objective consists of a weighted classification term over the IT part and a standard classification term over the OOT part. GIW is justified as its objective is good in all the four cases and reduces to IW in cases (i) and (ii). Thus, GIW is a strict generalization of IW from the objective point of view, and GIW is safer to be used when we are not sure whether the problem to be solved is a good case or a bad case for IW.3
- Thirdly, we provide a practical implementation of GIW: (a) following the split of  $S_{\mathrm{te}}$ ,  $D_{\mathrm{v}}$  is split into an IT set and an OOT set using the one-class support vector machine (Schölkopf et al., 1999);
<sup>1</sup>When  $S_{\mathrm{tr}}$  and  $S_{\mathrm{te}}$  differ, they differ by a non-zero probability measure; otherwise, we regard it as case (i). For example,  $S_{\mathrm{te}} \setminus S_{\mathrm{tr}} \neq \emptyset$  in cases (iii) and (iv) means that  $\sum_y \int_{\{\pmb{x}:(\pmb{x},y) \in S_{\mathrm{te}} \setminus S_{\mathrm{tr}}\}} p_{\mathrm{te}}(\pmb{x},y) \mathrm{d}\pmb{x} > 0$ .
2The positivity of  $\ell$ , i.e.,  $\ell(\pmb{f}(\pmb{x}), y) > 0$  rather than  $\ell(\pmb{f}(\pmb{x}), y) \geq 0$ , is needed to prove Theorem 2. This assumption holds for the popular cross-entropy loss and its robust variants, since  $\pmb{f}(\pmb{x})$  must stay finite.
<sup>3</sup>Even though we have divided all DS problems into four cases according to support shift (SS), we cannot empirically detect tiny or huge SS (under joint shift). Given that there is already DS, existing two-sample test methods for detecting DS are not good at detecting whether there is SS or not—their results must be positive.
Figure 2: Two concrete examples of the success and failure of IW in case (iii).
(b) the IT set, instead of the whole  $\mathcal{D}_{\mathrm{v}}$ , is used for IW; and (c) the OOT set directly joins training together with  $\mathcal{D}_{\mathrm{tr}}$  since no data in  $\mathcal{D}_{\mathrm{tr}}$  comes from the OOT part.
- Finally, we design and conduct extensive experiments that demonstrate the effectiveness of GIW in cases (iii) and (iv). The experiment design is also a major contribution since no experimental setup is available for reference to simulate case (iii) or (iv) on benchmark datasets.
Organization The analyses of IW are in Section 2, the proposal of GIW is in Section 3, and the experiments are in Section 4. Related work and additional experiments are in the appendices.
2 A deeper understanding of IW
First, we review the traditional importance weighting (IW) and its modern implementation dynamic importance weighting (DIW). Then, we analyze when and why IW methods can succeed/may fail.
A review of IW Let  $w^{*}(\pmb{x}, y) = p_{\mathrm{te}}(\pmb{x}, y) / p_{\mathrm{tr}}(\pmb{x}, y)$ , which is the ratio of the test density  $p_{\mathrm{te}}(\pmb{x}, y)$  over the training density  $p_{\mathrm{tr}}(\pmb{x}, y)$ , known as the importance function. Then, the expected objective of IW can be expressed as
$$
J (\boldsymbol {f}) = \mathbb {E} _ {p _ {\mathrm {t r}} (\boldsymbol {x}, y)} \left[ w ^ {*} (\boldsymbol {x}, y) \ell (\boldsymbol {f} (\boldsymbol {x}), y) \right]. \tag {2}
$$
In order to empirically approximate  $J(\pmb{f})$  in (2), we need to have an empirical version  $\widehat{w}(\pmb{x}, y)$  of  $w^{*}(\pmb{x}, y)$ , so that the empirical objective of IW is<sup>4</sup>
$$
\widehat {J} (\boldsymbol {f}) = \frac {1}{n _ {\mathrm {t r}}} \sum_ {i = 1} ^ {n _ {\mathrm {t r}}} \widehat {w} \left(\boldsymbol {x} _ {i} ^ {\mathrm {t r}}, y _ {i} ^ {\mathrm {t r}}\right) \ell \left(\boldsymbol {f} \left(\boldsymbol {x} _ {i} ^ {\mathrm {t r}}\right), y _ {i} ^ {\mathrm {t r}}\right). \tag {3}
$$
The original IW method is implemented in two steps: (I) weight estimation (WE) where  $\widehat{w}(\boldsymbol{x}, y)$  is obtained and (II) weighted classification (WC) where  $\widehat{J}(\boldsymbol{f})$  is minimized. The first step relies on the training data  $\mathcal{D}_{\mathrm{tr}}$  and the validation data  $\mathcal{D}_{\mathrm{v}}$ , and it can be either estimating the two density functions separately and taking their ratio or directly estimating the density ratio (Sugiyama et al., 2012).
As an empirical risk estimator,  $\widehat{J}(\pmb{f})$  in Eq. (3) is not unbiased to  $J(\pmb{f})$  in Eq. (2), since  $\widehat{w}(\pmb{x}, y)$  is not unbiased to  $w^{*}(\pmb{x}, y)$ . As far as we know, in IW, there is no unbiased importance-weight estimator and thus no unbiased risk estimator. That being said,  $\widehat{J}(\pmb{f})$  can still be (statistically) consistent with  $J(\pmb{f})$  under mild conditions if  $\widehat{w}(\pmb{x}, y)$  is consistent with  $w^{*}(\pmb{x}, y)$ .
A review of DIW The aforementioned two-step approach is very nice when the classifier  $\pmb{f}$  is a simple model, but it has a serious issue when  $\pmb{f}$  is a deep model (Fang et al., 2020). Since WE is not equipped with representation learning, in order to boost its expressive power, we need an external feature extractor such as an internal representation learned by WC. As a result, we are trapped by a circular dependency: originally we need  $w^{*}$  to train  $\pmb{f}$ ; now we need a trained  $\pmb{f}$  to estimate  $w^{*}$ .
DIW (Fang et al., 2020) has been proposed to resolve the critical circular dependency and to make IW usable for deep learning. Specifically, DIW uses a non-linear transformation  $\pi$  created from the current  $\pmb{f}$  (being trained) and replaces  $w^{*}(\pmb{x},y)$  with  $w^{*}(z) = p_{\mathrm{te}}(z) / p_{\mathrm{tr}}(z)$ , where  $z = \pi (\pmb{x},y)$  is the current loss-value or hidden-layer-output representation of  $(x,y)$ . DIW iterates between WE for estimating  $w^{*}(z)$  and WC for training  $\pmb{f}$  and thus updating  $\pi$  in a seamless mini-batch-wise manner. Given that WE enjoys representation learning inside WC, the importance-weight estimation quality of WE and the classifier training quality of WC can improve each other gradually but significantly.
Risk consistency/inconsistency of IW Now, consider how to qualify good or bad expected objectives under different conditions. To this end, we adopt the concepts of risk consistency and classifier consistency from the label-noise learning literature (Xia et al., 2019, 2020; Yao et al., 2020).
Definition 1. Given an (expected) objective  $J(\pmb{f})$ , we say it is risk-consistent if  $J(\pmb{f}) = R(\pmb{f})$  for any  $\pmb{f}$ , i.e., the objective is equal to the original risk for any classifier. On the other hand, we say  $J(\pmb{f})$  is classifier-consistent if  $\arg \min_{\pmb{f}} J(\pmb{f}) = \arg \min_{\pmb{f}} R(\pmb{f})$  where the minimization is taken over all measurable functions, i.e., the objective shares the optimal classifier with the original risk.
In the definition above, risk consistency is conceptually stronger than classifier consistency. If an objective is risk-consistent, it must also be classifier-consistent; if it is classifier-consistent, it may sometimes be risk-inconsistent. Note that a risk-inconsistent objective is not necessarily very bad, as it can still be classifier-consistent.<sup>5</sup> Hence, when considering expected objectives, risk consistency is a sufficient condition and classifier consistency is a necessary condition for good objectives.
In what follows, we analyze when and why the objective of IW, namely  $J(\pmb{f})$  in (2), can be a good objective or may be a bad objective.
Theorem 1. In cases (i) and (ii),  $IW$  is risk-consistent. $^6$
Proof. Recall that  $S_{\mathrm{tr}} = \{(x,y) : p_{\mathrm{tr}}(x,y) > 0\}$  and  $S_{\mathrm{te}} = \{(x,y) : p_{\mathrm{te}}(x,y) > 0\}$ . Under case (i) or (ii), let us rewrite  $R(\pmb{f})$  and  $J(\pmb{f})$  with summations and integrals:
$$
R (\boldsymbol {f}) = \sum_ {y = 1} ^ {C} \int_ {\{\boldsymbol {x}: (\boldsymbol {x}, y) \in S _ {\mathrm {t e}} \}} \ell (\boldsymbol {f} (\boldsymbol {x}), y) p _ {\mathrm {t e}} (\boldsymbol {x}, y) \mathrm {d} \boldsymbol {x},
$$
$$
\begin{array}{l} J (\boldsymbol {f}) = \sum_ {y = 1} ^ {C} \int_ {\{\boldsymbol {x}: (\boldsymbol {x}, y) \in S _ {\mathrm {t r}} \}} \ell (\boldsymbol {f} (\boldsymbol {x}), y) w ^ {*} (\boldsymbol {x}, y) p _ {\mathrm {t r}} (\boldsymbol {x}, y) \mathrm {d} \boldsymbol {x} \\ = \sum_ {y = 1} ^ {C} \int_ {\left\{\boldsymbol {x}: (\boldsymbol {x}, y) \in \mathcal {S} _ {\mathrm {t r}} \right\}} \ell (\boldsymbol {f} (\boldsymbol {x}), y) p _ {\mathrm {t e}} (\boldsymbol {x}, y) \mathrm {d} \boldsymbol {x}, \\ \end{array}
$$
where  $w^{*}(\pmb{x},y) = p_{\mathrm{te}}(\pmb{x},y) / p_{\mathrm{tr}}(\pmb{x},y)$  is always well-defined over  $S_{\mathrm{tr}}$  and we safely plugged this definition into the rewritten  $J(\pmb{f})$ . Subsequently, in case (i),  $S_{\mathrm{tr}} = S_{\mathrm{te}}$  and thus  $J(\pmb{f}) = R(\pmb{f})$ . In case (ii),  $S_{\mathrm{tr}} \supset S_{\mathrm{te}}$  and then we further have
$$
J (\boldsymbol {f}) = \sum_ {y = 1} ^ {C} \int_ {\{\boldsymbol {x}: (\boldsymbol {x}, y) \in \mathcal {S} _ {\mathrm {t e}} \}} \ell (\boldsymbol {f} (\boldsymbol {x}), y) p _ {\mathrm {t e}} (\boldsymbol {x}, y) \mathrm {d} \boldsymbol {x} + \int_ {\{\boldsymbol {x}: (\boldsymbol {x}, y) \in \mathcal {S} _ {\mathrm {t r}} \setminus \mathcal {S} _ {\mathrm {t e}} \}} \ell (\boldsymbol {f} (\boldsymbol {x}), y) p _ {\mathrm {t e}} (\boldsymbol {x}, y) \mathrm {d} \boldsymbol {x}.
$$
By definition,  $p_{\mathrm{te}}(\pmb{x},y) = 0$  outside  $S_{\mathrm{te}}$  including  $S_{\mathrm{tr}} \setminus S_{\mathrm{te}}$ , and thus  $J(\pmb{f}) = R(\pmb{f})$ .
Theorem 2. In cases (iii) and (iv),  $IW$  is risk-inconsistent, and it holds that  $J(\pmb{f}) < R(\pmb{f})$  for any  $\pmb{f}$ . Proof. Since  $w^{*}(\pmb{x},\pmb{y})$  is well-defined over  $S_{\mathrm{tr}}$  but it becomes ill-defined over  $S_{\mathrm{te}} \setminus S_{\mathrm{tr}}$ , we cannot naively replace the integral domain in  $J(\pmb{f})$  as in the proof of Theorem 1. In case (iii),  $S_{\mathrm{tr}} \subset S_{\mathrm{te}}$ , and consequently
$$
\begin{array}{l} R (\boldsymbol {f}) = \sum_ {y = 1} ^ {C} \int_ {\{\boldsymbol {x}: (\boldsymbol {x}, y) \in S _ {\mathrm {t r}} \}} \ell (\boldsymbol {f} (\boldsymbol {x}), y) p _ {\mathrm {t e}} (\boldsymbol {x}, y) \mathrm {d} \boldsymbol {x} \\ + \sum_ {y = 1} ^ {C} \int_ {\left\{\boldsymbol {x}: (\boldsymbol {x}, y) \in S _ {\mathrm {t e}} \backslash S _ {\mathrm {t r}} \right\}} \ell (\boldsymbol {f} (\boldsymbol {x}), y) p _ {\mathrm {t e}} (\boldsymbol {x}, y) \mathrm {d} \boldsymbol {x}. \\ \end{array}
$$
For example, if  $J(\pmb{f}) = R(\pmb{f}) / 2$  or if  $J(\pmb{f}) = R(\pmb{f})^2$ , minimizing  $J(\pmb{f})$  will give exactly the same optimal classifier as minimizing  $R(\pmb{f})$ .
In fact, if there is only the instance random variable  $x$  but no class-label random variable  $y$ , this result is simply the importance-sampling identity that can be found in many statistics textbooks. We prove it here to make our theoretical analyses self-contained and make later theoretical results easier to present/understand.
According to Theorem 1 for case (i), the first term in the rewritten  $R(\pmb{f})$  equals  $J(\pmb{f})$ . Moreover, the second term is positive, since  $\ell(\pmb{f}(\pmb{x}), y) > 0$  due to the positivity of  $\ell$ , and  $p_{\mathrm{te}}(\pmb{x}, y) > 0$  over  $S_{\mathrm{te}}$  including  $S_{\mathrm{te}} \setminus S_{\mathrm{tr}}$ . As a result, in case (iii),  $R(\pmb{f}) > J(\pmb{f})$ .
Similarly, in case (iv), we can split  $S_{\mathrm{te}}$  into  $S_{\mathrm{te}} \cap S_{\mathrm{tr}}$  and  $S_{\mathrm{te}} \setminus S_{\mathrm{tr}}$  and decompose  $R(\pmb{f})$  as
$$
\begin{array}{l} R (\boldsymbol {f}) = \sum_ {y = 1} ^ {C} \int_ {\left\{\boldsymbol {x}: (\boldsymbol {x}, y) \in S _ {\mathrm {t e}} \cap S _ {\mathrm {t r}} \right\}} \ell (\boldsymbol {f} (\boldsymbol {x}), y) p _ {\mathrm {t e}} (\boldsymbol {x}, y) \mathrm {d} \boldsymbol {x} \\ + \sum_ {y = 1} ^ {C} \int_ {\left\{\boldsymbol {x}: (\boldsymbol {x}, y) \in \mathcal {S} _ {\mathrm {t e}} \backslash \mathcal {S} _ {\mathrm {t r}} \right\}} \ell (\boldsymbol {f} (\boldsymbol {x}), y) p _ {\mathrm {t e}} (\boldsymbol {x}, y) \mathrm {d} \boldsymbol {x}. \\ \end{array}
$$
Note that  $S_{\mathrm{te}} \cap S_{\mathrm{tr}} \subset S_{\mathrm{tr}}$ , so that according to Theorem 1 for case (ii), the first term equals  $J(\pmb{f})$ . Following case (iii), the second term is positive. Therefore, in case (iv),  $R(\pmb{f}) > J(\pmb{f})$ .
Theorem 1 implies that the objective of IW can be a good objective in cases (i) and (ii). Theorem 2 implies that the objective of IW may be a bad objective in cases (iii) and (iv). As a consequence, the theorems collectively address when and why IW methods can succeed/may fail.7
When the IW objective may be bad and IW methods may fail, whether an IW method fails or not depends on many factors, such as the underlying data distributions, the sampled data sets, the loss, the model, and the optimizer. To illustrate this phenomenon, here we give two concrete examples belonging to case (iii), where IW has no problem at all in one example and is as poor as random guessing in the other example.
Two concrete examples We have seen the examples in Figure 2. In both examples, there are two classes marked with red and blue colors and distributed in four squares. Each square has a unit area and is the support of a uniform distribution of  $\pmb{x}$ , i.e.,  $p(\pmb{x},1) = 1$  and  $p(\pmb{x},0) = 0$  if its color is red, and  $p(\pmb{x},0) = 1$  and  $p(\pmb{x},1) = 0$  if its color is blue. There is a margin of 0.1 between two adjacent squares. The training distribution consists of the two squares on the left, and the test distribution consists of all the four squares. In the first example, on  $S_{\mathrm{te}} \setminus S_{\mathrm{tr}}$ , the label is red on the top and blue on the bottom, as same as the label on  $S_{\mathrm{tr}}$ . In the second example, on  $S_{\mathrm{te}} \setminus S_{\mathrm{tr}}$ , the label is blue on the top and red on the bottom, as opposite as the label on  $S_{\mathrm{tr}}$ .
We experimentally validated whether DIW works or not. The number of training data was 200. The number of validation data was only 4: we sampled one random point from each training square and added the center point of each test-only square. We can see that DIW performs very well in the first example, better than training from only the validation data; unfortunately, DIW performs very poorly in the second example, even worse than training from only the validation data.
The observed phenomenon should not be limited to DIW but be common to all IW methods. Here, we analyze why this phenomenon does not depend on the loss, the model, or the optimizer.
Proposition 3. In the first example, IW is classifier-consistent, while in the second example, IW is classifier-inconsistent. $^{8}$
Proof. Without loss of generality, assume that  $\ell$  is classification-calibrated (Bartlett et al., 2006).<sup>9</sup> Let  $(c^{(1)}, c^{(2)})$  be the center of  $S_{\mathrm{te}}$ , and then the four squares are located on the top-left, bottom-left, top-right, and bottom-right of  $(c^{(1)}, c^{(2)})$ . For convenience, we abbreviate  $f(x^{(1)}, x^{(2)})$  for  $x^{(1)} > c^{(1)}, x^{(2)} > c^{(2)}$  as  $f(+, +)$ ,  $f(x^{(1)}, x^{(2)})$  for  $x^{(1)} > c^{(1)}, x^{(2)} < c^{(2)}$  as  $f(+, -)$ , and so on.