SlowGuess/NeurIPS_2023_OCR huggingface dataset:
In this section, we analyze why some models generalize better than others by testing the Factorization and Feature Weighting hypotheses (see Sec. 1). We do this through a regression analysis that shows that metrics for model factorization and feature weighting help predict OOD generalization, while controlling for in-distribution accuracy and model objectives. We also show that the relationship between the metrics and OOD generalization is causal by directly manipulating feature factorization and feature weighting. Later in Sec. 7, we leverage this analysis to improve model generalization via new objectives. We use the same datasets and population of models as the previous section (Sec. 5).
6.1 Regression Analysis
Using our population of models, we a fit simple linear regression that predicts a model's average accuracy between the two OOD settings based on several metrics measured on ID test data. Each metric can be measured for a given hidden layer of a model; we measure each metric across the 3 blocks of Wide-ResNet and include each measurement as a variable in the regression. As a result, we have 9 factorization metrics and 4 feature weighting metrics (see Appendix B for the full list). In total, the regression variables we consider include ID accuracy, model objectives, and our proposed factorization and feature weighting metrics. For each dataset, we run a bootstrap with  $10\mathrm{k}$  samples,
using  $80\%$  of the 170 models for training and  $20\%$  for testing, to obtain an average test  $R^2$  regression value that we report along with its standard deviation.
Factorization Metrics. We measure model factorization using three metrics described below. These metrics are computed on representations from a given hidden layer of the model.
1. Linear Probing. To measure factorization with a linear probe, we first create foreground-only and background-only inputs using segmentation masks available in our datasets. We then extract model representations and predict the foreground or background class with a linear model. That is, if there are 10 foreground and 10 background classes, we train a linear classifier to do 20-way classifications. The probe classification accuracy measures factorization because if the model represents foregrounds and backgrounds in overlapping subspaces, the linear classifier will mix up the foreground and background classes, leading to low decoding accuracies [28].
2. Representation Similarity Analysis (RSA). RSA is a method for measuring the separation of inter-class representations and closeness of intra-class representations, which yields more fine-grained information than linear probing [20, 34]. Again, we extract model representations using foreground-only and background-only inputs. Then we compute pairwise similarities of the extracted features with Euclidean distance, giving a Representation Dissimilarity Matrix (RDM). We then calculate the correlation between the model RDM and a predefined reference RDM, which represents the ideal factorized representation structure where each foreground/background class has small intra-class distances and large inter-class distances. For details and figure of the reference RDM, see Appendix B.
3. Geometric Analysis. While linear probing and RSA are influenced by class separation within each foreground/background subspace, a geometric method may more directly measure factorization independently of this class separation. We adopt the method from Lindsey and Issa [28]. For complete details, see Appendix B, but in short: we compute the factorization of foregrounds against backgrounds as factorization  $f_{g} = 1 - \frac{\mathrm{var}_{fg|bg}}{\mathrm{var}_{fg}}$ , where  $\mathrm{var}_{fg}$  denotes the variance in the representation space induced by foreground perturbations and  $\mathrm{var}_{fg|bg}$  denotes the variance induced by foreground perturbations within the background subspace. If representations are factorized, changing foregrounds should not affect the background representations, so  $\mathrm{var}_{fg|bg}$  should be near zero, and factorization  $f_{g}$  will be near 1.
Feature Weighting Metric. We propose a new method, called the Foreground-Background Perturbation Score (FBPS), to measure the relative weight that a model assigns to foreground features vs. background features. First, drawing on ID test data, we create a set of inputs with random foreground and backgrounds. Then, we randomly switch the foreground of each input to get  $X_{fg\_ flipped}$  and measure the distance in model representations (or outputs),  $\delta_{fg} = \mathrm{dist}(f(X), f(X_{fg\_ flipped}))$ , with dist as the L2 distance (or KL divergence for model outputs). Similarly, we randomly switch the background contexts to create  $X_{bg\_ flipped}$  and measure the change in model representations  $\delta_{bg} = \mathrm{dist}(f(X), f(X_{bg\_ flipped}))$ . Finally, we compute the difference in these terms  $\mathrm{FBPS} = \delta_{fg} - \delta_{bg}$ . This approach yields four measurements per ResNet, one per block of the model and one for the model output. This metric is computed and averaged over 5000 samples. See Appendix B for more details.
Results. In Table 1, we report  $R^2$  values for regressions predicting the average of the BACKGROUND-INVARIANCE and OBJECT-DISAMBIGUATION accuracies, based on a population of models trained on each dataset. The ID accuracy alone obtains an  $R^2$  of 0.191 and 0.691 on the two datasets, respectively. The model objectives serve as a control here, showing significant improvement over ID accuracy alone. Meanwhile, adding either feature factorization or feature weighting metrics can significantly improve  $R^2$  in predict
Table 1: Regression analysis between metrics and the average OOD accuracy. Confidence intervals are one standard deviation over 10k bootstrapping.
Metric	R2 predicting average OOD Acc
COLOROBJECT	SCENEOBJECT
ID Acc	0.191±0.150	0.691±0.091
+ Obj.	0.505±0.106	0.696±0.084
+ Obj. & Factor.	0.925±0.023	0.838±0.044
+ Obj. & Ft. Wgt.	0.961±0.013	0.857±0.043
+ All	0.963±0.010	0.868±0.041
ing average OOD accuracies. But adding feature factorization and feature weighting metrics together achieves extremely high  $R^2$  values of 0.963 and 0.868 for predicting OOD accuracy on
Figure 4: Feature Weighting vs OOD Accuracies. The three plots show the relationship between feature weighting metrics and BACKGROUND-INVARIANCE, OBJECT-DISAMBIGUATION, and the average OOD accuracies in COLOROBJECT. The green lines show the mean and one standard deviation of each bin of data. As models weigh foreground features more, the performance increases in BACKGROUND-INVARIANCE and decreases in OBJECT-DISAMBIGUATION, resulting in an initially increasing and subsequently decreasing average OOD performance. We observe the same trend in ScENEOBJECT as well (see Appendix C).
the two datasets. We note that in past studies, various model properties do not predict OOD generalization better than ID acc does [43, 47]. See Appendix C for additional regression visualizations and results in individual OOD settings. Overall, our results suggest that both feature factorization and feature weighting play important roles in models' adaptive use of context.
In Fig. 4, we show the relationship between feature weighting metrics (as measured by FBPS at the final hidden layer of the Wide-ResNet models) and OOD generalization performances in COLOROBJECT. We observe that as the feature weighting metric increases, the performance increases in BACKGROUND-INVARIANCE and decreases in OBJECT-DISAMBIGUATION, resulting in an initially increasing and subsequently decreasing average OOD performance. The trend is the same in SCENEOBJECT too (see Appendix C). This shows that generalization is maximized at a specific feature weighting level, suggesting the need to find an appropriate weight between foreground and background features. We show that this relationship is also causal in the next subsection.
6.2 Causal Manipulation Analysis
Apart from the regression analysis, we also manipulate factorization and feature weighting in the representational space directly to show their causal effects on performances. We conduct these experiments with 10 ERM models, for both datasets.
Factorization Manipulation. Following Lindsey and Issa [28], we directly manipulate the factorization of the model representations. We first identify foreground and background subspaces and then rotate the model representations so that the foreground subspace overlaps more with the background subspace. This transformation directly decreases the feature factorization while holding constant other representation properties that may affect model accuracy [28] (full details in Appendix B). As this intervention applies to the final hidden states of the model, we compute model predictions by applying the linear classifier layer to the transformed representations.
Feature Weighting Manipulation. We manipulate the feature weighting by manipulating the variance in a certain subspace, which would bias a linear classifier with L2 regularization to rely on that subspace more (higher variance) or less (lower variance) [14, 41]. We do so by computing the importance, with respect to a subspace of interest (foreground or background), of each feature in the original space and manipulating them by a boost factor  $g$ . Note this step is followed by retraining the final classifier layer using the adjusted hidden representations. See Appendix B for full details.
Results. After we rotate the data to overlap the foreground and background subspaces, feature factorization (measured by the geometric method) drops significantly in ScENEOBJECT, as expected. As a result, the generalization performance in BACKGROUND-INVARIANCE decreases while the
(a) Factorization manipulation
(b) Feature weighting manipulation
Figure 5: Causal intervention results using 10 ERM models in SCENEOBJECT. (a) Rotating the foreground subspace into the background subspace has the effect of reducing model factorization and changing the OOD accuracies in opposite directions. Lighter-colored bars show metrics after the intervention. (b) Decreasing the foreground (or background) subspace by a factor of 64 has the effect of hurting (or improving) BACKGROUND-INVARIANCE accuracy and improving (or hurting) OBJECT-DISAMBIGUATION accuracy. We observe the same trend in COLOROBJECT (see Appendix Fig. 11).
performance in OBJECT-DISAMBIGUATION increases. This is also expected since the less factorized model conflates foreground and background information, and background information is irrelevant for BACKGROUND-INVARIANCE but perfectly correlated with object identity in OBJECT-DISAMBIGUATION. Shown in Fig. 5(a), the factorization metric drop by  $85.63\%$  (paired t-test:  $p < 1\mathrm{e} - 8$ ), and the BACKGROUND-INVARIANCE performance drop by  $18.24\%$  ( $p < 1\mathrm{e} - 4$ ), while the OBJECT-DISAMBIGUATION performance increases by  $5.97\%$  ( $p < 1\mathrm{e} - 5$ ). Therefore, we conclude that the change in performance is a causal result of decreased factorization. For results on COLOROBJECT, see Appendix C.
As shown in Fig. 5(b), when we decrease model weight on foreground features, the BACKGROUND-INVARIANCE performance decreases by 0.63 points (paired t-test:  $p < 1\mathrm{e} - 5$ ) while the OBJECT-DISAMBIGUATION performance increases by 0.68 points ( $p < 1\mathrm{e} - 5$ ) in SCENEOBJECT. This shows that increasing the feature weighting of foreground over background causally helps in BACKGROUND-INVARIANCE but hurts in OBJECT-DISAMBIGUATION. In the other direction, when we decrease reliance on background features, the BACKGROUND-INVARIANCE performance increases by 0.35 points ( $p = 1\mathrm{e} - 3$ ) while the OBJECT-DISAMBIGUATION performance decreases by 0.92 points ( $p < 1\mathrm{e} - 5$ ), showing that decreasing the feature weighting causally hurts BACKGROUND-INVARIANCE generalization but helps in OBJECT-DISAMBIGUATION. These results highlight the dichotomy between the two OOD settings we test, since our interventions improve performance in one setting while hurt in the other. This trend is expected since the background is irrelevant in BACKGROUND-INVARIANCE (where decreasing the background importance helps), while the background is helpful in OBJECT-DISAMBIGUATION (where decreasing background importance hurts). Overall, our results show the causal effect of feature weighting on model generalization, further strengthening our hypothesis.
7 Proposed Methods: Objectives for Adaptive Contextual Perception
Finally, in this section, we propose an augmentation method for improving the generalization ability to both OOD settings, based on our analysis in Sec. 6. Like prior methods [18, 11], our approach leverages additional annotations, including background identity and segmentation masks of foreground objects, which are available in our COLOROBJECT and SCENEOBJECT datasets. We also experiment with using Segment-Anything (SAM) to automatically generate segmentation masks to alleviate the need for additional annotation and achieve similar performances (see Appendix C.4 [19]).
Table 2: Accuracy of our proposed method (plus/minus one standard deviation over 10 seeds). OOD1 denotes BACKGROUND-INVARIANCE, and OOD2 denotes OBJECT-DISAMBIGUATION
Method	COLOROBJECT
ID	OOD1	OOD2	Avg OOD	Factorization	FBPS
ERM	87.15±0.40	29.11±3.39	95.24±0.83	62.17±1.40	71.27±1.14	0.012±0.013
Best Baseline	88.48±0.85	41.13±7.27	92.65±2.37	66.89±2.73	72.87±1.58	0.044±0.027
Ours	87.45±0.48	42.97±3.02	92.02±1.06	67.50±1.80	76.93±1.33	0.047±0.018
SCENEOBJECT
Method	ID	OOD1	OOD2	Avg OOD	Factorization	FBPS
ERM	72.28±0.93	35.37±1.97	71.93±1.27	53.65±0.70	64.07±0.98	0.000±0.006
Best Baseline	73.47±0.79	36.91±1.49	71.30±0.74	54.11±0.61	64.95±0.85	0.005±0.005
Ours	74.13±0.48	36.83±1.34	73.86±1.11	55.34±0.51	68.22±1.01	0.011±0.008
Augmentation Objectives. Our augmentation method consists of two parts: random-background and background-only augmentation. First, we extract the foreground object from an image and combine it with a randomly chosen background segmented from another image in the batch. This process encourages the model to recognize the object regardless of the background.
The cross-entropy loss for these images is denoted as  $\mathcal{L}_{fg}$ . Second, we create background-only images by replacing the foreground objects with black pixels and then ask the model to predict background labels. This prevents the model from ignoring all backgrounds so that it can still rely on the background when the foreground object is hard to recognize. We denote cross-entropy loss for background-only images as  $\mathcal{L}_{bg}$ . The final loss is:  $\mathcal{L} = \alpha_0\mathcal{L}_{task} + \alpha_1\mathcal{L}_{fg} + \alpha_2\mathcal{L}_{bg}$ , where  $\mathcal{L}_{task}$  is the task loss and  $\alpha_0, \alpha_1$  and  $\alpha_2$  are scalar weights for different losses. By adjusting the value of  $\alpha_0, \alpha_1$ , and  $\alpha_2$ , we can find the optimal balance among the original task loss and two augmentation types, enabling the model to learn more effectively from both background and foreground features and adaptively generalize to the two OOD settings. See Appendix A for method and tuning details.
Results. Our proposed augmentation method demonstrates significant improvements in both ID and OOD accuracies compared to ERM and the best baseline, Fish [42]. On COLOROBJECT, we match the best baseline on both OOD settings. Here, ERM still does better on OBJECT-DISAMBIGUATION, but on average OOD accuracy, our method improves over ERM by over 5.6 points, matching the best baselines. On SCENEOBJECT, we match the best baseline on BACKGROUND-INVARIANCE while significantly improving over it by 2.5 points on OBJECT-DISAMBIGUATION. We also improve upon ERM by 1.9 points on OBJECT-DISAMBIGUATION, where no other baseline methods can beat ERM. We also improved the average OOD performance by 1.2 points over ERM and other baselines. We get similar improvements over baselines for our method with pseudo-masks generated by SAM (see Appendix Table 6). And the improvements are even more significant with smaller training sizes (also see Appendix Table 6). In both datasets, our method improves over ERM or the best baseline on feature factorization and feature weighting, suggesting a causal relationship between these model properties and model generalization. For results of varying combination weights for the two augmentations, see Appendix C.4.
In summary, we find that: (1) Vision models face a tradeoff in performing well on BACKGROUND-INVARIANCE and OBJECT-DISAMBIGUATION tests. By evaluating a population of 170 models, we find a strong negative correlation coefficient of about  $r = 0.9$  between the accuracies for each OOD test. (2) OOD generalization in models is explained by the Factorization Hypothesis and the Feature Weighting Hypothesis. In our correlational analysis, metrics for factorization and feature weighting are highly predictive of OOD generalization ( $R^2$  up to .96, controlling for in-distribution accuracy). In our causal analysis, we find that directly manipulating factorization and feature weighting in models allows us to control how well they generalize. (3) Lastly, we propose data augmentation methods that achieve Pareto improvements over strong baselines, as they are able to improve either BACKGROUND-INVARIANCE or OBJECT-DISAMBIGUATION without hurting the other setting.
We thank Jack Lindsey and Elias Issa for valuable discussion regarding the causal analysis. This work was supported by ARO Award W911NF2110220, DARPA Machine-Commonsense (MCS) Grant N66001-19-2-4031, ONR Grant N00014-23-1-2356, DARPA ECOLE Program No. HR00112390060, and a Google PhD Fellowship. The views contained in this article are those of the authors and not of the funding agency.
[1] Faruk Ahmed, Yoshua Bengio, Harm Van Seijen, and Aaron Courville. Systematic generalisation with group invariant predictions. In International Conference on Learning Representations, 2021.
[2] Martin Arjovsky, Léon Bottou, Ishaan Gulrajani, and David Lopez-Paz. Invariant risk minimization, 2020.
[3] Moshe Bar. Visual objects in context. Nature Reviews Neuroscience, 5(8):617-629, 2004.
[4] Moshe Bar and Elissa Aminoff. Cortical analysis of visual context. Neuron, 38(2):347-358, 2003.
[5] Yoshua Bengio, Aaron Courville, and Pascal Vincent. Representation learning: A review and new perspectives. IEEE transactions on pattern analysis and machine intelligence, 35(8): 1798-1828, 2013.
[6] Irving Biederman. Recognition-by-components: a theory of human image understanding. Psychological review, 94(2):115, 1987.
[7] Stefania Bracci, Jakob Mraz, Astrid Zeman, Gaëlle Leys, Hans Op de Beeck, and Stefania Bracci. Object-scene conceptual regularities reveal fundamental differences between biological and artificial object vision. Center for Mind/Brain Sciences-CIMeC, 7:8, 2021.
[8] Marc-Andre Carbonneau, Julian Zaidi, Jonathan Boilard, and Ghyslain Gagnon. Measuring disentanglement: A review of metrics. IEEE Transactions on Neural Networks and Learning Systems, 2022.
[9] Ekin D Cubuk, Barret Zoph, Dandelion Mane, Vijay Vasudevan, and Quoc V Le. Autoaugment: Learning augmentation policies from data. arXiv preprint arXiv:1805.09501, 2018.
[10] Weijian Deng, Stephen Gould, and Liang Zheng. On the strong correlation between model invariance and generalization. In Advances in Neural Information Processing Systems, 2022.
[11] Hao-Shu Fang, Guansong Lu, Xiaolin Fang, Jianwen Xie, Yu-Wing Tai, and Cewu Lu. Weakly and semi supervised human body part parsing via pose-guided knowledge transfer. arXiv preprint arXiv:1805.04310, 2018.
[12] Yaroslav Ganin, Evgeniya Ustinova, Hana Ajakan, Pascal Germain, Hugo Larochelle, François Laviolette, Mario Marchand, and Victor Lempitsky. Domain-adversarial training of neural networks. The journal of machine learning research, 17(1):2096-2030, 2016.
[13] Ian J Goodfellow, Jonathon Shlens, and Christian Szegedy. Explaining and harnessing adversarial examples. arXiv preprint arXiv:1412.6572, 2014.
[14] Trevor Hastie, Robert Tibshirani, Jerome H Friedman, and Jerome H Friedman. The elements of statistical learning: data mining, inference, and prediction, volume 2. Springer, 2009.
[15] Dan Hendrycks and Thomas Dietterich. Benchmarking neural network robustness to common corruptions and perturbations. arXiv preprint arXiv:1903.12261, 2019.
[16] Dan Hendrycks, Kevin Zhao, Steven Basart, Jacob Steinhardt, and Dawn Song. Natural adversarial examples. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 15262-15271, 2021.
[50] Chiyuan Zhang, Samy Bengio, Moritz Hardt, Benjamin Recht, and Oriol Vinyals. Understanding deep learning (still) requires rethinking generalization. Communications of the ACM, 64(3): 107-115, 2021.
[51] Dinghuai Zhang, Kartik Ahuja, Yilun Xu, Yisen Wang, and Aaron Courville. Can subnetwork structure be the key to out-of-distribution generalization? In International Conference on Machine Learning, pages 12356-12367. PMLR, 2021.
[52] Hongyi Zhang, Moustapha Cisse, Yann N Dauphin, and David Lopez-Paz. Mixup: Beyond empirical risk minimization. In International Conference on Learning Representations, 2017.
[53] Mengmi Zhang, Claire Tseng, and Gabriel Kreiman. Putting visual object recognition in context. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 12985-12994, 2020.
[54] Bolei Zhou, Agata Lapedriza, Aditya Khosla, Aude Oliva, and Antonio Torralba. Places: A 10 million image database for scene recognition. IEEE transactions on pattern analysis and machine intelligence, 40(6):1452-1464, 2017.
A Experiment Design Details
A.1 Dataset Details
Dataset License. Our datasets are created using MSCOCO and Places [26, 54], both under the CC BY 4.0 license.
Synthetic Dataset. We design a synthetic dataset for a clear demonstration of our hypothesis and results. Following the data generation process formulated in Sec. 3, we create synthetic data with distribution shifts in co-occurrence probability and noise in foreground features. The ground truth label  $Y$  and latent variables  $Z_{c}, Z_{s}$  are all in  $\{0,1\}$ .  $Y$  and  $Z_{c}$  is sampled uniformly:  $Y = Z_{c} \sim U(\{0,1\})$ . The causal features  $\mathbf{x}_c$  and spurious features  $\mathbf{x}_s$  are generated from Gaussian distributions based on  $Z_{c}$  and  $Z_{s}$  respectively:  $x_{c} \sim \mathcal{N}(\mu_{c}^{i}, \sigma_{c}^{2})$  for  $z_{c} = i$ , and  $x_{s} \sim \mathcal{N}(\mu_{s}^{i}, \sigma_{s}^{2})$  for  $z_{s} = i$ . We choose continuous features and Gaussian distributions instead of binary features and Bernoulli distributions like prior works [51, 27] to model noise and the OBJECT-DISAMBIGUATION setting more easily. We set  $\mu_{r0} = \mu_{nr0} = 0.2$ ,  $\mu_{r1} = \mu_{nr1} = 0.8$ , and  $\sigma_r = \sigma_{nr} = 0.2$ . The dimensions of inputs are  $80 + 10 + 10 = 100$  for useless, robust, and non-robust features respectively. We then add noise to causal features:  $\epsilon \sim \mathcal{N}(0, \sigma_{\epsilon}^{2})$ . Finally, we have irrelevant features  $\mathbf{x}_i$  that do not correlate with the label and are sampled from another Gaussian distribution:  $x_{n} \sim \mathcal{N}(\mu_{n}^{i}, \sigma_{n}^{2})$ . Finally, we combine the noisy causal features, spurious features, and irrelevant features to create the complete inputs. In training sets, the co-occurrence probability  $p_{co}$  varies between [0.5, 0.98] and the noise level  $\sigma$  between [0.6, 1.6] (see Sec. 3 for definitions). Each train set has 1000 samples. In BACKGROUND-INVARIANCE setting,  $p_{co} = 0$  and  $\epsilon = 0$ , while in OBJECT-DISAMBIGUATION setting,  $p_{co} = 1$  and  $\epsilon = 1.5$ .