SlowGuess/NeurIPS_2023_OCR huggingface dataset:
P. L. Bartlett, M. I. Jordan, and J. D. McAuliffe. Convexity, classification, and risk bounds. Journal of the American Statistical Association, 101(473):138-156, 2006.
S. Ben-David, J. Blitzer, K. Crammer, and F. Pereira. Analysis of representations for domain adaptation. In NeurIPS, 2006.
J. Duchi, P. Glynn, and H. Namkoong. Statistics of robust optimization: A generalized empirical likelihood approach. arXiv preprint arXiv:1610.03425, 2016.
T. Fang, N. Lu, G. Niu, and M. Sugiyama. Rethinking importance weighting for deep learning under distribution shift. In NeurIPS, 2020.
Y. Ganin, E. Ustinova, H. Ajakan, P. Germain, H. Larochelle, F. Laviolette, M. March, and V. Lempitsky. Domain-adversarial training of neural networks. Journal of Machine Learning Research, 17(59):1-35, 2016.
S. Goldwasser, A. T. Kalai, Y. Kalai, and O. Montasser. Beyond perturbations: Learning guarantees with arbitrary adversarial test examples. In NeurIPS, 2020.
M. Gong, K. Zhang, T. Liu, D. Tao, C. Glymour, and B. Schölkopf. Domain adaptation with conditional transferable components. In ICML, 2016.
I. Goodfellow, Y. Bengio, and A. Courville. Deep learning. The MIT Press, 2016.
K. He, X. Zhang, S. Ren, and J. Sun. Deep residual learning for image recognition. In CVPR, 2016.
S. Hido, Y. Tsuboi, H. Kashima, M. Sugiyama, and T. Kanamori. Statistical outlier detection using direct density ratio estimation. Knowledge and information systems, 26:309-336, 2011.
W. Hu, M. Wang, Q. Qin, J. Ma, and B. Liu. Hrn: A holistic approach to one class learning. In NeurIPS, 2020.
J. Huang, A. Gretton, K. Borgwardt, B. Scholkopf, and A. Smola. Correcting sample selection bias by unlabeled data. In NeurIPS, 2007.
S. Ioffe and C. Szegedy. Batch normalization: Accelerating deep network training by reducing internal covariate shift. In ICML, 2015.
L. Jiang, Z. Zhou, T. Leung, L.-J. Li, and L. Fei-Fei. Mentornet: Learning data-driven curriculum for very deep neural networks on corrupted labels. In ICML, 2018.
T. Kanamori, S. Hido, and M. Sugiyama. A least-squares approach to direct importance estimation. Journal of Machine Learning Research, 10(7):1391-1445, 2009.
D. P. Kingma and J. L. Ba. Adam: A method for stochastic optimization. In ICLR, 2015.
A. Krizhevsky and G. Hinton. Learning multiple layers of features from tiny images. Technical report, CiteSeer, 2009.
Y. LeCun, L. Bottou, Y. Bengio, and P. Haffner. Gradient-based learning applied to document recognition. Proceedings of the IEEE, 86(11):2278-2324, 1998.
S. Motiian, M. Piccirilli, D. A. Adjeroh, and G. Doretto. Unified deep supervised domain adaptation and generalization. In ICCV, 2017.
S. Pan and Q. Yang. A survey on transfer learning. IEEE Transactions on Knowledge and Data Engineering, 22 (10):1345-1359, 2009.
P. Perera, P. Oza, and V. M. Patel. One-class classification: A survey. arXiv preprint arXiv:2101.03064, 2021.
J. Quionero-Candela, M. Sugiyama, A. Schwaighofer, and N. Lawrence. Dataset shift in machine learning. The MIT Press, 2009.
M. Ren, W. Zeng, B. Yang, and R. Urtasun. Learning to reweight examples for robust deep learning. In ICML, 2018.
K. Saito, Y. Ushiku, and T. Harada. Asymmetric tri-training for unsupervised domain adaptation. In ICML, 2017.
B. Schölkopf, R. C. Williamson, A. Smola, J. Shawe-Taylor, and J. Platt. Support vector method for novelty detection. In NeurIPS, 1999.
J. Shu, Q. Xie, L. Yi, Q. Zhao, S. Zhou, Z. Xu, and D. Meng. Meta-weight-net: Learning an explicit mapping for sample weighting. In NeurIPS, 2019.
M. Sugiyama and M. Kawanabe. Machine learning in non-stationary environments: Introduction to covariate shift adaptation. The MIT Press, 2012.
M. Sugiyama, M. Krauledat, and K. Müller. Covariate shift adaptation by importance weighted cross validation. Journal of Machine Learning Research, 8(5):985-1005, 2007a.
M. Sugiyama, S. Nakajima, H. Kashima, P. Buenau, and M. Kawanabe. Direct importance estimation with model selection and its application to covariate shift adaptation. In NeurIPS, 2007b.
M. Sugiyama, T. Suzuki, and T. Kanamori. Density ratio estimation in machine learning. Cambridge University Press, 2012.
M. Sugiyama, H. Bao, T. Ishida, N. Lu, T. Sakai, and G. Niu. Machine learning from weak supervision: An empirical risk minimization approach. The MIT Press, 2022.
V. N. Vapnik. Statistical learning theory. John Wiley & Sons, 1998.
X. Xia, T. Liu, N. Wang, B. Han, C. Gong, G. Niu, and M. Sugiyama. Are anchor points really indispensable in label-noise learning? In NeurIPS, 2019.
X. Xia, T. Liu, B. Han, N. Wang, M. Gong, H. Liu, G. Niu, D. Tao, and M. Sugiyama. Part-dependent label noise: Towards instance-dependent label noise. In NeurIPS, 2020.
M. Yamada, T. Suzuki, T. Kanamori, H. Hachiya, and M. Sugiyama. Relative density-ratio estimation for robust distribution comparison. In NeurIPS, 2011.
Y. Yao, T. Liu, B. Han, M. Gong, J. Deng, G. Niu, and M. Sugiyama. Dual T: Reducing estimation error for transition matrix in label-noise learning. In NeurIPS, 2020.
M. Z. Zaheer, J.-h. Lee, M. Astrid, and S.-I. Lee. Old is gold: Redefining the adversarially learned one-class classifier training paradigm. In CVPR, 2020.
In this section, we discuss relevant prior studies for addressing distribution shift problems, including importance weighting (IW), IW-like methods, and domain adaptation (DA).
Importance weighting (IW) IW has been a powerful tool for mitigating the influence of distribution shifts. The general idea is first to estimate the importance, which is the test over training density ratio, and then train a classifier by weighting the training losses according to the importance. Numerous IW methods have been developed in this manner, utilizing different techniques for importance estimation.
The kernel mean matching (KMM) approach (Huang et al., 2007) learns the importance function by matching the distributions of training and test data in terms of the maximum mean discrepancy in a reproducing kernel Hilbert space, while the Kullback-Leibler importance estimation procedure (KLIEP) (Sugiyama et al., 2007b) employs the KL divergence for density fitting and the least-squares importance fitting (LSIF) (Kanamori et al., 2009) employs squared loss for importance fitting. The unconstrained LSIF (uLSIF) (Kanamori et al., 2009) is an approximation version of LSIF that removes the non-negativity constraint in optimization, allowing for more efficient computation. To boost the performance of such traditional IW methods, dynamic importance weighting (DIW) (Fang et al., 2020) is recently proposed to make them compatible with stochastic optimizers, thereby facilitating their effective integration with deep learning frameworks.
However, in order to establish a well-defined notion of importance, all IW methods including DIW assume cases (i) and (ii), while they become problematic in cases (iii) and (iv).
IW-like methods A relevant IW invariant is the relative unconstrained least-squares importance fitting (RuLSIF) (Yamada et al., 2011), which considers a smoothed and bounded extension of the importance. Instead of estimating the importance  $w^{*}(\boldsymbol{x},y) = p_{\mathrm{te}}(\boldsymbol{x},y) / p_{\mathrm{tr}}(\boldsymbol{x},y)$ , they estimate the  $\eta$ -relative importance  $w_{\eta}^{*}(\boldsymbol{x},y) = p_{\mathrm{te}}(\boldsymbol{x},y) / (\eta p_{\mathrm{te}}(\boldsymbol{x},y) + (1 - \eta)p_{\mathrm{tr}}(\boldsymbol{x},y))$  where  $0 \leq \eta \leq 1$ . While the relative importance is well-defined in cases (iii) and (iv), experiments have demonstrated that it is inferior to GIW, since its training does not incorporate any out-of-training (OOT) data.
Moreover, some reweighting approaches based on bilevel optimization look like DIW in the sense of iterative training between weighted classification on the training data for learning the classifier and weight estimation with the help of a small set of validation data for learning the weights (Jiang et al., 2018; Ren et al., 2018; Shu et al., 2019). However, they encounter a similar issue as IW and RuLSIF, where validation data is solely used for learning the weights, while the training data (without any OOT data) is used for training the classifier. This makes them hard to handle the cases (iii) and (iv).
Domain adaptation (DA) DA relates to DS problems where the  $p_{\mathrm{te}}(\pmb{x}, \pmb{y})$  and the  $p_{\mathrm{tr}}(\pmb{x}, \pmb{y})$  are usually named as target and source domain distributions (Ben-David et al., 2006), or in-domain and out-of-domain distributions (Duchi et al., 2016). It can be categorized into supervised DA (SDA) or unsupervised DA (UDA): the former has labeled test data while the latter has unlabeled test data. The setting of SDA is similar as that of GIW. One representative SDA work is classification and contrastive semantic alignment (CCSA) (Motiian et al., 2017) method. In CCSA, a contrastive semantic alignment loss is added to the classification loss, for minimizing the distances between the samples that come from the same class and maximizing the distances between samples from different classes. In DA research, UDA is more popular than SDA. Based on different assumptions, UDA involves learning domain-invariant (Ganin et al., 2016) or conditional domain-invariant features (Gong et al., 2016), or giving pseudo labels to the target domain data (Saito et al., 2017).
Note that DA can refer to either certain problem settings or the corresponding learning methods (or both). When regarding it as problem settings, SDA is exactly the same as joint shift and UDA is fairly similar to covariate shift, which assumes  $p(y|x)$  doesn't change too much between the training and test domain. When regarding it as learning methods, the philosophy of both SDA and UDA is to find good representations to link the source and target domains and transfer knowledge from the source domain to the target domain, which is totally different from the philosophy of IW.
Figure 8: A plot of the training data and validation data in Color-MNIST dataset.
B Supplementary information on experimental setup
In this section, we present supplementary information on the experimental setup. All experiments were implemented using PyTorch 1.13.1 $^{14}$  and carried out on NVIDIA Tesla V100 GPUs $^{15}$ .
B.1 Datasets and base models
MNIST MNIST (LeCun et al., 1998) is a  $28^{*}28$  grayscale image dataset for 10 hand-written digits (0-9). The original dataset includes 60,000 training data and 10,000 test data. See http://yann.lecun.com/exdb/mnist/ for more details.
In the experiments, we converted it for binary classification to classify even/odd digits as follows:
Class 0: digits '0', '2', '4', '6', and '8';
Class 1: digits '1', '3', '5', '7', and '9'.
In our setup, the training data only included 4 digits (0-4). The test data could access all digits (0-9) in case (iii) and 8 digits (2-9) in case (iv). Since the number of training data was reduced, we added two data augmentations to the training and validation data: random rotation with degree 10 and random affine transformation with degree 10, translate of (0.1, 0.1) and scale of (0.9, 1.1). Note that the data augmentations were only added during procedure 2 MODELTRAIN in Algorithm 1.
Accordingly, we modified LeNet-5 (LeCun et al., 1998) as the base model for MNIST:
0th (input) layer:  $(32^{*}32)$  1st layer: C(5\*5,6)-S(2\*2)- 2nd layer: C(5\*5,16)-S(2\*2)- 3rd layer: FC(120)- 4th to 5th layer: FC(84)-2,
where  $\mathrm{C}(5^{*}5,6)$  represents a  $5^{*}5$  convolutional layer with 6 output channels followed by ReLU,  $\mathrm{S}(2^{*}2)$  represents a max-pooling layer with a filter of size  $2^{*}2$ , and  $\mathrm{FC}(120)$  represents a fully connected layer with 120 outputs followed by ReLU, etc. The hidden-layer-output representation of data used in the implementation was the normalized output extracted from the 3rd layer.
Color-MNIST Color-MNIST was modified from MNIST for 10 hand-written digit classification, where the digits in training data were colored in red and the digits in test/validation data were colored in either red, green or blue evenly. See Figure 8 for a plot of the training data and validation data. We did not add any data augmentation for experiments on Color-MNIST.
$^{14}$ https://pytorch.org
15https://www.nvidia.com/en-us/data-center/v100/
To process RGB input data, we modified LeNet-5 as the base model for Color-MNIST:
0th (input) layer:  $(32^{*}32^{*}3)$
1st layer: C(5*5,20)-S(2*2)-
2nd layer: C(5*5,50)-S(2*2)-
3rd layer: FC(120)-
4th to 5th layer: FC(84)-10,
where the abbreviations and the way of extracting the hidden-layer-output representation of data were the same as that in MNIST.
CIFAR-20 CIFAR-100 (Krizhevsky and Hinton, 2009) is a  $32*32$  colored image dataset in 100 classes, grouped in 20 superclasses. It contains 50,000 training data and 10,000 test data. We call this dataset CIFAR-20 since we use it for 20-superclass classification—the predefined superclasses and classes as shown below, where each superclass includes five distinct classes. See https://www.cs.toronto.edu/~kriz/cifar.html for more details.
aquatic mammals
food containers
fruit and vegetables
household electrical devices
household furniture
large carnivores
large man-made outdoor things
large natural outdoor scenes
large omnivores and herbivores
medium-sized mammals
non-insect invertebrates
vehicles 1
vehicles 2
(beaver, dolphin), otter, seal, whale
(aquarium fish, flatfish), ray, shark, trout
(orchids, poppies), roses, sunflowers, tulips
(bottles, bowls), cans, cups, plates
(apples, mushrooms), oranges, pears, sweet peppers
(clock, computer keyboard), lamp, telephone, television
(bed, chair), couch, table, wardrobe
(bee, beetle), butterfly, caterpillar, cockroach
(bear, leopard), lion, tiger, wolf
(bridge, castle), house, road, skyscraper
(cloud, forest), mountain, plain, sea
(camel, cattle), chimpanzee, elephant, kangaroo
(fox, porcupine), possum, raccoon, skunk
crab, lobster), snail, spider, worm
(baby, boy), girl, man, woman
(crocodile, dinosaur), lizard, snake, turtle
(hamster, mouse), rabbit, shrew, squirrel
maple, oak), palm, pine, willow
(bicycle, bus), motorcycle, pickup truck, train
(lawn-mower, rocket), streetcar, tank, tractor
In our setup, the training data only included the data in 2 out of the 5 classes per superclass, i.e., the classes in () were seen by the training data as shown above. The test data included the data in all classes. Since the number of training data was reduced, we added several data augmentations to the training and validation data: random horizontal flip, random vertical flip, random rotation of degree 10 and random crop of size 32 with padding 4. Same as that in MNIST experiments, the data augmentations were only added during procedure 2 MODELTRAIN in Algorithm 1.
As for the base model for CIFAR-20, we adopted ResNet-18 (He et al., 2016) as follows:
1st to 5th layers:  $\mathrm{C}(3^{*}3,64)$  -[C(3\*3,64),C(3\*3,64)]\*2-
6th to 9th layers:  $\left[\mathrm{C}(3^{*}3,128),\mathrm{C}(3^{*}3,128)\right]^{*}2-$
10th to 13th layers:  $\left[\mathrm{C}(3^{*}3,256),\mathrm{C}(3^{*}3,256)\right]^{*}2-$
14th to 17th layers:  $\left[\mathrm{C}(3^{*}3,512),\mathrm{C}(3^{*}3,512)\right]*2-$
18th layer: Global Average Pooling-20,
where  $[\cdot, \cdot]$  denotes a building block (He et al., 2016) and  $[\cdot]*2$  means 2 such blocks, etc. Batch normalization (Ioffe and Szegedy, 2015) was applied after convolutional layers. The hidden-layer-output representation of data was the normalized output after pooling operation in the 18th layer.
B.2 Experiments under support shift
For all compared methods except Val-only, we pre-trained the model for 10 epochs as the initialization. For the one-class support vector machine (O-SVM) (Schölkopf et al., 1999), we adopted
the implementation from scikit-learn $^{16}$ , where the radial basis function (RBF) kernel was used:  $k(\pmb{x}_i, \pmb{x}_j) = e^{-\gamma \| \pmb{x}_i - \pmb{x}_j \|^2}$  with  $\gamma = 10000$ . All other hyperparameters about O-SVM were set as the default. For the distribution matching by dynamic importance weighting (DIW) (Fang et al., 2020), we again used the RBF kernel where  $\gamma$  was the median distance between the training data. And we used  $\pmb{K} + \omega \pmb{I}$  as the kernel matrix  $\pmb{K}$ , where  $\pmb{I}$  was an identity matrix and  $\omega$  was set to be 1e-05. The upper bound of weights was set as 50.
In all experiments under support shift, Adam (Kingma and Ba, 2015) was used as the optimizer, the learning rate was 0.0005, decaying every 100 epochs by multiplying a factor of 0.1, and the batch size was set as 256. For MNIST, Color-MNIST, and CIFAR-20 experiments, the weight decay was set as 0.005, 0.002, and 0.0001, respectively.
B.3 Experiments under support-distribution shift
For experiments under support-distribution shift, all the setups and hyperparameters about the initialization, O-SVM, and distribution matching by DIW were the same as that in Section B.2. Moreover, the same that Adam was used as the optimizer, the learning rate was 0.0005, decaying every 100 epochs by multiplying a factor of 0.1, and the batch size was set as 256. Next we show the setups and hyperparameters specific to the support-distribution shift.
Label-noise experiments On top of the support shift in Section B.2, we added a symmetric label noise to the training data, where a label may flip to all other classes with an equal probability (this probability was defined as the noise rate, set as  $\{0.2, 0.4\}$ ). The type of label noise and the noise rate were unknown to the model. For MNIST, Color-MNIST, and CIFAR-20 experiments, the weight decay was set as 0.005, 0.002, and 0.008, respectively.
Class-prior shift experiments We induced class-prior shift in the training data by randomly sampling half of the classes as minority classes (other classes were the majority classes) and reducing the number of samples in minority classes. The sample size ratio per class between the majority and minority classes was  $\rho$ , chosen from  $\{10, 100\}$ . The randomly selected minority classes in class-prior shift experiments were shown as follows:
- MNIST: class of odd digits;
Color-MNIST: digits '1', '2', '6', '7', and '8';
- CIFAR-20: superclasses of 'fish', 'fruit and vegetables', 'household electrical device', 'household furniture', 'large carnivores', 'large omnivores and herbivores', 'medium-sized mammals', 'people', 'small mammals', and 'vehicles 1'.
For MNIST, Color-MNIST, and CIFAR-20 experiments, the weight decay was set as 0.005, 1e-05, and 1e-07, respectively. Since we did not split the validation data in class-prior shift experiments, we set the  $\alpha$  in (5) as 0.5 for class-prior shift experiments on all datasets.
In this section, we present supplementary experimental results, including the histogram plots of the learned O-SVM score, more ablation study results on the validation data split error, visualizations of convolution kernels for all methods under label noise, additional experimental results for case (iv), and the summary of classification accuracy.
C.1 On the learned O-SVM score
Figure 9 shows the histogram plots of the learned O-SVM score on MNIST, Color-MNIST, and CIFAR-20 under support shift. From the histogram plots, we observe that the score distribution consists of two peaks without overlapping; therefore, any value between the two peaks (e.g., 0.4) could be made as a threshold to split the validation data into two parts. If the score of validation data is higher than the threshold, then the data is identified as an (in-training) IT validation data; otherwise, it is an (out-of-training) OOT validation data.
$^{16}$ https://scikit-learn.org/stable/modules/generated/sklearn.svm.OneClassSVM.html
(b) Color-MNIST
(c) CIFAR-20
Figure 9: Histogram plots of the learned O-SVM score under support shift.
Table 2: Mean accuracy (standard deviation) in percentage on MNIST over the last ten epochs under support shift with label noise (5 trials). OOT/IT is short for out-of-training/in-training data. OOT  $\rightarrow$  IT means OOT data flips to IT data and vice versa. LN is short for label noise. Flip rate is the percentage of OOT/IT data that randomly flipped to IT/OOT data.
Flip rate	OOT → IT	IT → OOT
LN 0.2	LN 0.4	LN 0.2	LN 0.4
0	94.60 (0.22)	93.74 (0.53)	94.60 (0.22)	93.74 (0.53)
0.1	94.54 (0.31)	94.04 (0.54)	94.76 (0.33)	93.66 (0.44)
0.3	92.09 (0.99)	92.38 (1.14)	94.62 (0.26)	92.78 (0.90)
0.5	88.98 (1.73)	88.83 (2.26)	94.03 (0.64)	92.25 (1.36)
0.7	86.76 (1.28)	85.94 (2.14)	93.55 (0.82)	90.53 (1.22)
After splitting the validation data,  $\alpha$  in (5) is estimated as the ratio of the sample size between the IT validation data and the whole validation data, i.e.,  $\widehat{\alpha} = n_{\mathrm{v1}} / n_{\mathrm{v}}$ . For example, in Figure 9(a),  $\widehat{\alpha} = \frac{8}{8 + 12} = 0.4$ , which is equal to the true value in MNIST experiments. Similarly, it can be verified that the  $\alpha$  in Figure 9(b) and 9(c) are also accurately estimated.
C.2 Additional results on the validation data split error
As mentioned in the main paper in Figure 7(a), generally GIW is quite robust to split errors and OOT  $\rightarrow$  IT is more problematic than the other direction. Here we present additional results on MNIST under 0.4 label noise in Table 2. We can further observe that as the label noise increases from 0.2 to 0.4, IT  $\rightarrow$  OOT is more susceptible to such negative impacts than the other direction.
C.3 Visualizations of convolution kernels under label noise
Then, we visualized the learned convolution kernels for all methods on Color-MNIST under 0.2 label noise. From Figure 10, we can see that the results aligned with the discussions about Figure 4. Previous IW or IW-like methods (i.e., DIW, R-DIW, Reweight and MW-Net), DA methods (i.e., CCSA and DANN), and Pretrain-val learned most weights on the red channel, which may cause the failure on the test data with green/blue color. Although Val-only had weights on all color channels, it may fail to learn useful data representation due to its limited training data. Only GIW could successfully recover the weights on all color channels while capturing the data representation.
C.4 Additional experimental results for case (iv)
Here we present the results on MNIST in case (iv) under distribution-support shift, comparing with IW-like and domain adaptation (DA) baselines. From Figure 11, we can see that GIW outperforms other methods by a large margin in case (iv) under both label-noise and class-prior-shift settings.
C.5 Summary of classification accuracy
Table 3 and 4 present the mean accuracy (standard deviation) in percentage on MNIST, Color-MNIST, and CIFAR-20 over the last ten epochs under support shift, comparing with IW-like methods and domain adaptation (DA) methods respectively, corresponding to Figure 3. Table 5 and 6 present such results under support-distribution shift, corresponding to Figure 5 and 6. Table 7 and 8 shows the summary of results in case (iv) on MNIST, corresponding to Figure 11.
MW-Net
R-DIW
DIW
Val-only
Pretrain-val
CCSA
DANN
GIW
Figure 10: Visualizations of the learned convolution kernels on Color-MNIST under 0.2 label noise.
Figure 11: Results on MNIST in case (iv) under distribution-support shift (5 trails).
Table 3: Mean accuracy (standard deviation) in percentage on MNIST, Color-MNIST, and CIFAR-20 over the last ten epochs under support shift in case (iii) with IW-like baselines (5 trials). Best and comparable methods (paired  $t$ -test at significance level  $5\%$ ) are highlighted in bold. This result corresponds to the top row in Figure 3.
Data	Case	Val-only	Pretrain-val	Reweight	MW-Net	R-DIW	DIW	GIW
MNIST	(iii)	78.94 (0.99)	88.31 (1.02)	82.03 (0.13)	82.17 (0.32)	81.73 (0.42)	80.53 (1.23)	95.16 (0.20)
MNIST	(iv)	79.86 (0.26)	86.25 (0.95)	77.78 (0.91)	76.73 (0.63)	77.11 (0.63)	75.11 (1.43)	90.94 (0.91)
Color-MNIST	(iii)	31.19 (0.65)	80.40 (3.62)	38.39 (1.03)	39.26 (0.01)	39.28 (0.03)	39.50 (0.18)	93.87 (0.19)
CIFAR-20	(iii)	28.61 (0.66)	42.79 (0.41)	56.17 (0.29)	57.90 (0.18)	58.02 (0.35)	55.78 (0.34)	59.73 (0.47)