SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Figure 10: Feature Weighting vs OOD Accuracies in SCENEOBJECT. The three plots show the relationship between feature weighting metrics and BACKGROUND-INVARIANCE, OBJECT-DISAMBIGUATION, and the average OOD accuracies.
(a) Factorization manipulation
Figure 11: Causal intervention results using 10 ERM models in COLOROBJECT.
(b) Feature weighting manipulation
et al. [22], providing ground for future work to explore. As shown in Table 6, with the generated pseudo-masks, we achieve similar performances as the original method using the ground-truth masks.
Training size ablation. We evaluate the effectiveness of our augmentation methods on various training sizes (4k, 8k, 16k) in ScENEOBJECT. The effect size of our data augmentation method is more significant when the data size is smaller. The improvements go up to  $4.5\%$  on BACKGROUND-INVARIANCE and  $1.1\%$  on OBJECT-DISAMBIGUATION compared to the baseline ERM with 4k training data, while the best baseline (Fish) does not achieve better OOD performances than the baseline.
Effect of Mixing Weight on OOD Performances. We show the effect of mixing weight on OOD performances in this section. The two loss items in our proposed augmentation method are combined using a mixing weight (from 0 to 1) so that we can keep the learning rate fixed. We show the impact of mixing weight on BACKGROUND-INVARIANCE, OBJECT-DISAMBIGUATION, and the average accuracies in both datasets in Fig. 13. In both COLOROBJECT and SCENEOBJECT, as the mixing weight increased, BACKGROUND-INVARIANCE accuracy increases while OBJECT-DISAMBIGUATION accuracy decreases gradually. This suggests that the models become more invariant to the backgrounds and rely more on the foreground features. However, the average accuracy exhibits a different pattern. Initially, as the mixing weight increased, the average accuracy showed an upward trend. The accuracy reached a peak and then started to decrease gradually as the mixing weight continued to increase. This finding indicates that there exists an optimal mixing weight that maximizes the overall accuracy of the model on different OOD settings, suggesting the need to find an appropriate weight between foreground and background features.
Figure 12: Effect of Boost Factor on OOD Performances.
Table 6: Accuracy of our proposed method with pseudo-masks generated by Segment-Anything (SAM) or with various training sizes (4k, 8k, 16k) on ScENEOBJECT [19]. Our method with pseudomasks achieves similar generalization performances as the one with ground-truth masks. The effect size of our data augmentation method is more significant when the data size is smaller.
Method	SCENEOBJECT
ID	OOD1	OOD2	Avg OOD
ERM	72.28±0.93	35.37±1.97	71.93±1.27	53.65±0.70
Best Baseline	73.47±0.79	36.91±1.49	71.30±0.74	54.11±0.61
Ours (pseudo-masks)	75.00±1.08	37.96±2.04	73.34±2.09	55.65±2.07
Ours (gt-masks)	74.13±0.48	36.83±1.34	73.86±1.11	55.34±0.51
ERM (8k data)	66.97±1.05	31.29±1.37	68.57±0.60	49.93±0.99
Best Baseline (8k data)	67.13±0.74	32.19±0.64	67.08±0.46	49.64±0.55
Ours (8k data)	71.43±0.89	34.52±0.97	72.16±1.24	53.34±1.11
ERM (4k data)	60.74±0.89	26.27±1.08	62.79±1.04	44.53±1.06
Best Baseline (4k data)	61.35±1.60	26.77±1.45	63.34±0.90	45.06±1.18
Ours (4k data)	63.94±0.51	30.72±0.60	63.86±0.54	47.29±0.57
Figure 13: Effect of Mixing Weight on OOD Performances. As the mixing weight increases, the BACKGROUND-INVARIANCE performance increases while the OBJECT-DISAMBIGUATION performance decreases. The average OOD performances either flattens out (as in COLOROBJECT) or peaks and then starts decreasing (as in SCENEOBJECT)