SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Zhouxing Shi*
zshi@cs.ucla.edu
Nicholas Carlini
ncarlini@google.com
Ananth Balashankar
ananthbshankar@google.com
Ludwig Schmidt
schmidt@cs.washington.edu
Google, UCLA
Alex Beutel*
alexb@openai.com
Yao Qin
UCSB, Google Research
yaoqin@ucsb.edu
"Effective robustness" measures the extra out-of-distribution (OOD) robustness beyond what can be predicted from the in-distribution (ID) performance. Existing effective robustness evaluations typically use a single test set such as ImageNet to evaluate the ID accuracy. This becomes problematic when evaluating models trained on different data distributions, e.g., comparing models trained on ImageNet vs. zero-shot language-image pre-trained models trained on LAION. In this paper, we propose a new evaluation metric to evaluate and compare the effective robustness of models trained on different data. To do this, we control for the accuracy on multiple ID test sets that cover the training distributions for all the evaluated models. Our new evaluation metric provides a better estimate of effective robustness when there are models with different training data. It may also explain the surprising effective robustness gains of zero-shot CLIP-like models exhibited in prior works that used ImageNet as the only ID test set, while the gains diminish under our new evaluation. Additional artifacts including interactive visualizations are provided at https://shizhouxing.github.io/effective-robustness.
Robustness against distribution shifts is important for machine learning models to work reliably across various environments. For natural distribution shifts on image classification datasets, Taori et al. (2020) proposed the notion of effective robustness to control for in-distribution (ID) accuracy when evaluating out-of-distribution (OOD) accuracy. Following a long line of work that has found a strong correlation between ID and OOD accuracy on many test sets (Recht et al., 2019; Yadav & Bottou, 2019), effective robustness allows researchers to assess whether an apparently improved OOD accuracy is a result of effectively improved robustness or is simply an expected outcome of enhanced ID accuracy.
Unfortunately, the current definition of effective robustness has a subtle limitation: it requires a fixed ID test set, which is typically ImageNet (Deng et al., 2009) when using ImageNet-like OOD test sets
in Taori et al. (2020) or CIFAR-10 (Krizhevsky et al., 2009) when using CIFAR-like OOD test sets in Miller et al. (2021). It is acceptable when models are trained predominately on only one dataset. However, the emergence of many large-scale models trained on significantly different datasets makes it necessary to evaluate and compare models trained on different data distributions, under which it becomes unclear which ID test set should be used.
In particular, models from Contrastive Language-Image Pre-training, such as CLIP (Radford et al., 2021) and ALIGN (Jia et al., 2021) have recently exhibited unprecedented effective robustness gains during zero-shot inference (Radford et al., 2021; Fang et al., 2022; Nguyen et al., 2022). However these previous works simply take ImageNet as the single ID test set, even though the models are not trained on ImageNet. We demonstrate that the results of evaluating effective robustness using a single ID test set can vary drastically depending on the selection of the ID test set. Therefore, this imprecise treatment on the ID test set in existing works could end up exaggerating the effective robustness of zero-shot CLIP models compared to models that are exactly trained on ImageNet.
In this paper, we propose to more precisely evaluate and compare the effective robustness of models trained on different datasets. Instead of controlling for a single ID accuracy that may bias towards models from a particular training distribution, we propose to use multiple ID test sets that cover the training distributions of all the models. In particular, previous works performed single-dimensional linear regression on a set of baseline models to predict OOD accuracy from a single ID accuracy (Taori et al., 2020). And they then evaluate the actual OOD accuracy of the models beyond the expected value that can be predicted from the fitting line, as the effective robustness. We expand on this definition by allowing for multiple ID test sets, and perform multi-dimensional linear regression to fit a plane to predict OOD accuracy from the accuracy on multiple ID test sets.
- We reveal a limitation in the existing effective robustness evaluation when used to compare models trained on different data distributions.
- We then propose a new effective robustness evaluation which uses multiple ID test sets to more precisely compare the effective robustness of models trained on different data.
- We show that the OOD accuracy of various models including zero-shot CLIP models can usually be better predicted from accuracies on multiple ID test sets compared to using only one ID test set.
- Our results provide new understandings on the effective robustness gains of CLIP-like models observed in prior works only using ImageNet as the ID test set, while the gains diminish under our new evaluation.
2 Background of Effective Robustness
Under natural distribution shifts, the OOD accuracy of a model is often correlated with the ID accuracy. After applying a logit transformation on the accuracy, a linear trend between the transformed ID accuracy and OOD accuracy holds across many datasets (e.g., a distribution shift from ImageNet (Deng et al., 2009) to ImageNetV2 (Recht et al., 2019), or from CIFAR-10 (Krizhevsky et al., 2009) to CIFAR-10.2 (Hendrycks & Dietterich, 2018)) and models with various architectures and training methods (Taori et al., 2020; Miller et al., 2021). This phenomenon implies that most models showing higher OOD accuracies naturally resulted from better ID performance.
To eliminate the confounding effect of ID accuracy on OOD performance, Taori et al. (2020) proposed effective robustness that measures the OOD performance beyond the expected OOD accuracy given the ID accuracy, where the expected OOD accuracy is predicted according to the fitted linear trend of baseline models. Since they only use a single ID test set, we refer to this version of effective robustness as single-ID effective robustness.
Suppose there are  $n$  baseline models  $f_{1}, f_{2}, \dots, f_{n}$ . A baseline function  $\tilde{\beta}(x)$  is constructed to predict the OOD accuracy of each baseline model,  $\mathrm{acc}_{\mathrm{odd}}(f_{i})$  ( $1 \leq i \leq n$ ), given the single ID accuracy of the model  $x = \mathrm{acc}_{\mathrm{id}}(f_{i})$ . The baseline function is instantiated as:
$$
\tilde {\beta} (x) = \operatorname {e x p i t} (w \log \mathrm {i t} (x) + b), \tag {1}
$$
where  $w$  and  $b$  are parameters,  $\operatorname{logit}(x) = \ln \left( \frac{x}{1 - x} \right)$  is the logit transformation, and  $\operatorname{expit}(x)$  is the inverse of  $\operatorname{logit}(x)$ . Since  $\operatorname{logit}(\tilde{\beta}(x)) = w \log \operatorname{lit}(x) + b$ , the baseline function is essentially a linear
(a) ImageNet-V2 accuracy against ImageNet accuracy.
(b) ImageNet-V2 accuracy against YFCC accuracy.
Figure 1: Class-subsampled ("css." for short) ImageNet-V2 accuracy against ImageNet accuracy and YFCC accuracy, respectively, for 36 ImageNet models and 13 YFCC models that are also used in Table 3a. A linear fit is generated for ImageNet models and YFCC-15M models, respectively. Accuracies and linear fits are under the logit scale. Class-subsampling is used to only include classes that appear in all the involved test sets (see Section 5.1).
function after applying a logit transformation on the accuracies, and it can be determined by solving a linear regression. Then the single-ID effective robustness of a model  $f$  is evaluated as
$$
\tilde {\rho} (f) = \operatorname {a c c} _ {\mathrm {o o d}} (f) - \tilde {\beta} (\operatorname {a c c} _ {\mathrm {i d}} (f)), \tag {2}
$$
which subtracts the predicted OOD accuracy based on the ID accuracy  $\mathrm{acc}_{\mathrm{id}}(f)$ , from the actual OOD accuracy  $\mathrm{acc}_{\mathrm{odd}}(f)$ .
3 Limitation of the Single ID Test Set
The existing effective robustness evaluation in Section 2 fixes a single ID test set for all the models, which is reflective of the ID performance only if all the models are trained on the same dataset that matches the ID test set. However, as large-scale pre-trained models emerge, it becomes necessary to compare models trained on different datasets, in order to know if the latest pre-training techniques can yield effective robustness gains. In this section, we use the comparison between zero-shot CLIP models and standard ImageNet models as an example to show the limitation of using a single ID test set: when only one ID test set is used, using different ID test sets leads to contradictory conclusions.
Following Fang et al. (2022), we compare models trained on ImageNet (Deng et al., 2009) and YFCC-15M (Thomee et al., 2016), respectively. On ImageNet, we include standard classifiers, and we also train CLIP models using templates filled with an ImageNet class name as the caption in a format of "A photo of a {class name}" We also train CLIP models on YFCC-15M, a dataset with image-text pairs. And we use ImageNet-V2 (Recht et al., 2019) as the OOD test set. We consider two different ID test sets. One ID test set is simply ImageNet. The other ID test set is constructed from YFCC-15M, since we have CLIP models trained on YFCC. We refer to this test set as "YFCC test set", and we refer to the accuracy on this test set as "YFCC accuracy". We discuss its details in Section 5.1 and Appendix B.2. Both ID test sets we consider here match the training distribution of some of the models (ImageNet models and YFCC models respectively) but not all the models.
We then plot the ImageNet-V2 accuracy of the models against their ImageNet accuracy and YFCC accuracy, respectively. There is a strong linear trend between the scaled ID accuracy and OOD accuracy for ImageNet models and YFCC models, respectively, and we plot fitting lines for these two sets of models, respectively. When the ID test set is ImageNet, Fig. 1a shows that the fitting line for YFCC models is generally above the fitting line for ImageNet models (except for the regime with extremely low accuracies), which appears to suggest that YFCC models have effective robustness gains over ImageNet models, as also suggested in Fang et al. (2022). However, in Fig. 1b which uses YFCC as the ID test set, the fitting line of ImageNet models are now mostly above YFCC models, which instead appears to suggest that ImageNet models have greater effective robustness than YFCC models. We observe that when there is a mismatch in the training data and the ID test data, the
models appear to have greater effective robustness (YFCC models in Figure 1a and ImageNet models in Figure 1b), as their performance on the ID test data and the OOD performance predicted from the single ID accuracy tend to be lower. This makes it difficult to compare models trained on different data and leads to imprecise conclusions on effective robustness if only one ID test set is used.
4 Multi-ID Effective Robustness
Considering the limitations of using a single ID test set, we propose a new way for effective robustness evaluation using multiple ID test sets that cover the training data distributions of all the involved models. We name it multi-ID effective robustness. Specifically, for each training distribution, we propose to prepare an ID test set that matches the training distribution, respectively. In particular, we focus on comparing models trained on two different datasets at a time in this paper, and we thereby use two ID test sets, where each of them corresponds to one of the training datasets.
While we refer to them as ID test sets, each of them is only the exact ID test set for some of the considered models that are trained on the distribution matching the test set, and it is not exactly an ID test set for all the considered models. However, we assume that the training distributions of all the models are still relatively close compared to the OOD test distributions (e.g., images normally collected from social medias in ImageNet (Deng et al., 2009), YFCC (Thomee et al., 2016), and LAION (Schuhmann et al., 2021) are relatively close compared to the OOD images in ImageNet-Sketch (Wang et al., 2019) that consists of sketch images specifically). In this way, both ID test sets are relatively ID for all the models compared to the OOD test sets, and it can be meaningful to control for the performance on these ID test sets when comparing the OOD performance.
We still use  $\mathrm{acc}_{\mathrm{odd}}(\cdot)$  to denote the OOD accuracy, and we use  $\mathrm{acc}_1(\cdot)$  and  $\mathrm{acc}_2(\cdot)$  to denote the accuracy on the two ID test sets, respectively. In contrast to the previous baseline function  $\tilde{\beta}(x)$  in Eq. (1), we propose a new baseline function  $\beta(x,y)$  that predicts the OOD accuracy based on the accuracies  $x$  and  $y$  on the two ID test sets, respectively.
All the models in Figure 1 are trained on either ImageNet or YFCC. Thus, to compare their effective robustness under our new evaluation, we use two ID test sets for ImageNet and YFCC at the same time, in contrast to Figure 1a and 1b which use one ID test set separately at each time and results on the two different ID test sets lead to contradictory conclusions. As shown in Figure 2, we plot the OOD accuracy against the two ID accuracies on both two ID test sets in a 3D space. We observe that the data points approximately lie on a plane when plotted on the logit scale. This motivates us to instantiate  $\beta (x,y)$  as:
$$
\beta (x, y) = \operatorname {e x p i t} \left(w _ {x} \operatorname {l o g i t} (x) + w _ {y} \operatorname {l o g i t} (y) + b\right), \tag {3}
$$
where  $w_{x}, w_{y}, b$  are parameters.  $\beta(x, y)$ , which is the plane in Figure 2, is also a linear function w.r.t.  $x$  and  $y$  under the logit scale, and thus it is a reasonable extension from  $\hat{\beta}(x)$  by using a multi-dimensional linear function on the logit scale. We determine the parameters by solving an ordinary least squares regression to fit the accuracies. Metrics for linear regression such as the coefficient of determination, a.k.a.  $R^2$ ,
can be used to evaluate the fitting quality of the baseline function. A high  $R^2$  value indicates that the OOD accuracy is accurately predicted by the baseline function from the ID accuracies, and thus the evaluated models have similar effective robustness. And our multi-ID effective robustness for a model  $f$  is defined as
$$
\rho (f) = \operatorname {a c c} _ {\mathrm {o o d}} (f) - \beta (\operatorname {a c c} _ {1} (f), \operatorname {a c c} _ {2} (f)).
$$
Compared to the existing definition for effective robustness in Eq. (2), the major difference is the inclusion of two ID accuracies  $\mathrm{acc}_1(f)$  and  $\mathrm{acc}_2(f)$  in the baseline function, compared to using a single ID accuracy  $\mathrm{acc}_{\mathrm{id}}(f)$ .
Figure 2: Class-subsampled ("css." for short) ImageNet-V2 accuracy against both ImageNet accuracy and YFCC accuracy for ImageNet models and YFCC models used in Figure 1 which shows the projections when only one of ImageNet accuracy and YFCC accuracy is used.
Generalizing to more than two training datasets. Although we focus on handling two training datasets at a time, our method may be generalized to more than two datasets in principle, by defining a baseline function based on multiple ID accuracies  $\mathrm{acc}_1(\cdot),\dots ,\mathrm{acc}_k(\cdot)$ . However, it could be costly as it would require training more models to fit a high-quality baseline function. We leave it for future works to reduce the cost when dealing with a larger number of datasets.
Models. In order to fit a baseline function, we need a large amount of models with diverse accuracies. To this end, we follow Taori et al. (2020) to train models with various proportions of data by subsampling from the entire training set (namely dataset subsampling), which effectively produces models with diverse accuracies. Moreover, we also combine examples from two datasets with different sampling ratios and train models on these combined datasets. This produces models with training distributions varying between the two training datasets and it is supposed to yield different combinations of the two ID accuracies. We use models trained on each single dataset as well as the combined datasets in the same fitting process, so that the baseline functions do not bias towards models trained on certain data. Our experiments include the following models:
- Standard classifiers on CIFAR-10 and ImageNet. We train standard classifiers on CIFAR-10 and ImageNet (Deng et al., 2009). We use ResNet-18, ResNet-50, and ResNet-101 (He et al., 2016). Additionally, we train models by combining CIFAR-10 and ImageNet at various ratios, where we upsample CIFAR-10 images from  $32 \times 32$  to  $224 \times 224$ . Furthermore, we include ViT-S/16, ViT-B/16, ViT-L/16 models (Dosovitskiy et al., 2021) pre-trained on the whole ImageNet.
- CLIP models. On YFCC-15M (Thomee et al., 2016) and LAION-15M (Schuhmann et al., 2021) which consist of image-text pairs, we train CLIP models using ResNet-50 and ResNet-101. We also train models by combining ImageNet and YFCC-15M and LAION-15M, respectively. We discard models with ImageNet accuracy below  $5\%$ . Additionally, in Section 5.4, we also have downloaded ViT-based models from Mu et al. (2022); Ilharco et al. (2021) and CLIP models fine-tuned on ImageNet, which are only used for evaluation but not fitting the baseline functions. We provide additional details in Appendix B.1.
We use “{Name_of_dataset} models” to denote models trained only on the dataset, e.g., “CIFAR-10 models”. And we use “{Name_of_dataset_A} + {Name_of_dataset_B} models” to represent models trained on a combination of two datasets, e.g., “CIFAR-10+ImageNet models”.
ID test sets. We focus on image classification. Labeled image classification datasets such as ImageNet can be directly used for evaluating ID accuracy. For datasets that consist of image-text pairs for language-image pre-training without original labels, including YFCC and LAION, we automatically generate classification labels by matching captions with ImageNet classes, which has been similarly performed in Fang et al. (2022) for training classifiers using caption data, and we then hold out a balanced test set from the original dataset. More details are reported in Appendix B.2. Although it is possible to obtain a higher-quality test set by human labelling, we will show that using the automatically labelled test sets can already produce reasonable results.
OOD test sets. To compare the effectiveness robustness of models trained on CIFAR-10 and ImageNet, we use 3 CIFAR-like OOD test sets with natural distribution shifts, including CIFAR-10.1 (Recht et al., 2019), CIFAR-10.2 (Lu et al., 2020), and CINIC-10 (Darlow et al., 2018). We use 4 ImageNet-like OOD test sets to compare models trained on ImageNet with models trained on YFCC and LAION: ImageNet-V2 (Recht et al., 2019), ImageNet-R (Hendrycks et al., 2021a), ImageNet-Sketch (Wang et al., 2019), and ObjectNet (Barbu et al., 2019). We do not use ImageNet-A (Hendrycks et al., 2021b) which involves adversarial filtering and has a different behavior in effective robustness evaluation (Taori et al., 2020; Fang et al., 2022).
Class subsampling and mapping. Considering that different test sets may not have the same classes, we follow prior works (Taori et al., 2020; Fang et al., 2022) to use class subsampling<sup>2</sup> to
2We reuse the term "class subsampling" from prior works (Taori et al., 2020; Fang et al., 2022), although it is not a random sampling.
(a) Using CIFAR-10.2 as the OOD test set. The ImageNet accuracy is mapped to CIFAR-10 classes (see Section 5.1).
(b) Using ImageNet-R as the OOD test set.
Figure 3: Visualization of the multi-ID effective robustness. The colored plane stands for the baseline function. Figure 4 and Figure 5 (in Appendix A.1) show various projected 2D views. See our website (https://shizhouxing.github.io/effective-robustness) for an interactive visualization.
retain classes appearing in all the test sets. We also follow Miller et al. (2021) to map a subset of ImageNet classes to CIFAR-10 classes when comparing CIFAR-10 models and ImageNet models..
5.2 Evaluation on CIFAR-like OOD Test Sets
(a) CIFAR-10.2 accuracy against CIFAR-10 accuracy. ImageNet models have higher CIFAR-10.2 accuracy compared to CIFAR-10 models when controlling for CIFAR-10 accuracy only.
Figure 4: Projected views of Figure 3a. Figure 4a and Figure 4b correspond to single-ID evaluations using different ID test sets and yield contradictory conclusions on the effective robustness. Our multi-ID evaluation provides a more holistic view where all these models are approximately on a same plane and thus have similar effective robustness.
(b) CIFAR-10.2 accuracy against ImageNet accuracy. ImageNet models have lower CIFAR-10.2 accuracy compared to CIFAR-10 models when controlling for ImageNet accuracy only.
We first experiment with models trained using CIFAR-10 and ImageNet on CIFAR-like OOD test sets. We show the fitting quality in Table 1a and the effective robustness of various models in Table 1b. Compared to the single-ID evaluation, our multi-ID evaluation achieves a better fitting quality and predicts the OOD accuracy from the ID accuracies more precisely (higher  $R^2$  and lower MAE), and thus provides a more precise understanding on the effective robustness. Specifically, while both single-ID effective robustness and multi-ID effective robustness have relatively high fitting quality on CIFAR-like test sets, using multi-ID effective robustness further improves the fitting quality. In terms of the effective robustness, under the single-ID evaluation, ImageNet models achieve  $3.91 \pm 2.20$  (\%) and  $2.77 \pm 1.25$  (\%) effective robustness on CIFAR-10.2 and CINIC-10, respectively. The positive
Table 1: Results on CIFAR-like OOD test sets. 148 models are included, including CIFAR-10 models, ImageNet models, and CIFAR-10+ImageNet models (CIFAR+IN for short). The multi-ID evaluation achieves better fitting quality where the effective robustness values of CIFAR-10 models and ImageNet models also become closer to 0.
(a) Fitting quality evaluated by  $R^2$  and mean absolute error (MAE).
Test set	R2(↑)	MAE (%,↓)
Single-ID	Multi-ID	Single-ID	Multi-ID
CIFAR-10.1	0.996	0.997	1.07	0.93
CIFAR-10.2	0.981	0.996	2.22	0.95
CINIC-10	0.978	0.990	2.41	1.49
(b) Effective robustness values (\%). We report the mean and standard deviation for three groups of models with different training data, respectively.
Test set	Evaluation	CIFAR-10 21 models	ImageNet 89 models	CIFAR+IN 38 models
CIFAR-10.1	Single-ID	-1.68±0.92	1.05±1.27	0.02±1.10
Multi-ID	-1.43±0.92	0.10±1.12	0.19±1.01
CIFAR-10.2	Single-ID	-1.65±0.70	3.91±2.20	-0.64±1.79
Multi-ID	-0.76±0.77	0.56±1.27	0.03±1.29
CINIC-10	Single-ID	-0.96±1.43	2.77±1.25	-0.10±2.81
Multi-ID	-0.08±1.52	-0.52±0.98	0.63±2.10
Average	Single-ID	-1.43±0.53	2.58±1.32	-0.24±1.58
Multi-ID	-0.76±0.63	0.04±0.67	0.28±1.04
effective robustness values seems to suggest an advantage of ImageNet models compared to CIFAR-10 models, which is consistent with the findings in Miller et al. (2021). However, under the multi-ID evaluation, the advantage of ImageNet models diminishes, and the effective robustness values of both CIFAR-10 models and ImageNet models are much closer to 0. Therefore, the apparent advantage reported by prior works can be explained as the effect of training data on the single-ID evaluation, and our multi-ID evaluation resolves this confounder to provide a more precise understanding.