SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Real-world shortcut We have also evaluated our method in the Waterbird dataset (Sagawa et al. [2019]). This is a real-world natural image dataset where the distribution shift occurs as the natural habitat depicted in the background changes between an aquatic and land setting. From the two kinds of birds in the dataset (water and land birds), water birds were assigned to training data and land birds were set as an anomaly. To make this a more challenging setup, we have defined a highly unbalanced distribution of the environments, with 184 images in training data with a water background and 3498 with a land background. The evaluation of the methods was performed in images with only a water background for out-of-distribution and only land background for in-distribution.
Synthetic covariate shifts To evaluate the robustness against covariate shifts, we use the DiaViB-6 dataset Eulig et al. [2021] for our experiments. This dataset comprises modified and upsampled images from MNIST Deng [2012], where the generative factors of the image can be altered, leading to changes in texture, hue, lightness, position, and scale.
The training data is comprised of two distinct environments, generated by manipulating original instantiations of each factor, and ensuring all factors differed between the two environments. For the test data, we defined five distinct environments denoted as  $e_0, e_1, \ldots, e_4$ . The environment  $e_0$  corresponds to images from MNIST for a specific digit under an original instantiation of each factor. All images of that digit are labeled as normal, while images of any other digit are considered anomalies. For  $0 < i \leq 4$ , each environment  $e_i$  corresponds to images where  $i$  factors have been modified. In  $e_i$ , all factors modified in  $e_{i-1}$ , plus an additional unique factor, are altered. The task is to classify an image of a handwritten digit as either normal or an anomaly.
We also adapted this setup to include the Fashion-MNIST dataset, a more challenging synthetic benchmark. The same cumulative covariate shifts were induced from an original in-distribution environment,  $e_0$ , to subsequent environments  $e_{1-4}$ . Two specific classes were chosen to generate normal and abnormal samples.
Anomaly detection methods tested We undertook a comprehensive evaluation of our proposed regularization term, PCIR, and integrated it into six diverse methodologies for deep AD. These include: STFM Wang et al. [2021], Reverse Distillation Deng and Li [2022], CFA Lee et al. [2022], MeanShifted Reiss and Hoshen [2021], CSI Tack et al. [2020], and Red PANDA Cohen et al. [2023]. For a detailed exposition of these methods, refer to the supplementary material.
Real-world anomaly detection under domain shift Notably, even with the complexity and realworld variability introduced by the domain shift, the incorporation of partial conditional distribution invariance still resulted in notable improvements for both training setups, in particular with MeanShift and Red PANDA retrieving almost in-distribution performance (see Fig. 3). This suggests that our regularization term is robust to more substantial changes associated with domain shifts.
(a) Camelyon17: 2 environments
Figure 3: Results on realistic domain shift Camelyon17 and realist shortcut learning in Waterbirds. (background transparent bar-plots: in-distribution evaluation; foreground opaque bar-plots: out-of-distribution evaluation). (a) Camelyon17: Setup with two in-distribution training environments and three out-of-distribution test environments. (b) Camelyon17: Setup with three in-distribution training environments and two out-of-distribution test environments (c) Waterbirds: Shortcut learning setup.
(b) Camelyon17: 3 environments
(c) Waterbirds
Focusing on the out-of-distribution setting, our results highlight a consistent pattern of performance enhancement when applying the partial conditional invariance regularization term, ranging from  $2\%$  to  $8\%$  increase in the AUROC when comparing regularized and un-regularized methods. The consistency of this performance boost across different models and with both two and three in-distribution training environemnts underscores the potential of PCIR as a beneficial regularization technique in out-of-distribution settings.
Realist shortcut learning We observe that in all original unregularized methods evaluated, there is a noticeable drop in performance when comparing in-distribution (background transparent plots) to out-of-distribution (foreground opaque plots) that is attributed to the models effectively capturing the exposed shortcut. Models with PCIR nearly recover the in-distribution performance, showing the effectiveness of this approach to ignore uninformative environment features even when exposed through a shortcut. Furthermore, in all models tested, there's a consistent observation that the model performance not only stabilizes but also increases when adding the regularization term even for in-distribution scenarios. The increase in model performance, attributed to the addition of PCIR to each method, exhibits an improvement range from  $5\%$  to  $15\%$  AUROC.
MMNIST and Fashion-MNIST under covariate shift Models that absorb shortcut features have been observed to be especially susceptible to covariate shifts (Eulig et al. [2021], Geirhos et al. [2020]). Therefore, if a model abuses a shortcut, then inducing a covariate shift in the shortcut feature often significantly deteriorates performance. This effect is discernible in Fig. 4, where performance drops progressively increase for all models as more covariate shifts are induced in the images, and thus more shortcut features deviate from their original form in the training data.
However, note that by adopting regularization based on partial conditional invariance, we have been able to construct anomaly detectors that consistently exhibit enhanced robustness against induced covariate shifts. In some cases, even in-distribution performance increases through partial conditional invariance. These findings corroborate our hypothesis that partial conditional distribution invariance serves as a sufficient prior for robustness to covariate shifts in AD. For a more comprehensive analysis of performance improvements achieved refer to the supplementary material.
Ablation study In our ablation study, we held all method parameters constant to exclusively examine the impact of the weight assigned to the partial conditional invariance regularization term.
Across all tested methods, the relationship between regularization weight and performance exhibited a concave shape. This pattern suggests a simple linear search could be sufficient to identify an optimal weight for the regularization term. This observed behaviour aligns intuitively with the nature of invariance regularization. Over-emphasizing the regularization term could potentially cause the model's encoder to generate non-discriminative and non-informative representations. In an extreme case, the model could collapse all inputs into a single value. Interestingly, the behaviour of the regularization term remained consistent across both datasets used in our study. This reinforces the robustness of our approach across diverse datasets.
(a)  $e_1$ : MNIST s.t. single covariate shift
(b)  $e_2$ : MNIST s.t. double covariate shift
(c)  $e_3$ : MNIST s.t triple covariate shift
(d)  $e_4$ : MNIST s.t. 4-fold covariate shift
(e)  $e_1$ : Fashion-MNIST s.t. single covariate shift
(f)  $e_2$ : Fashion-MNIST s.t. double covariate shift
(g)  $e_3$ : Fashion-MNIST s.t. triple (h)  $e_4$ : Fashion-MNIST s.t. 4-fold covariate shift covariate shift
Figure 4: Experimental results on synthetic covariate shift (background transparent bar-plots: in-distribution evaluation; foreground opaque bar-plots: out-of-distribution evaluation) (a-d) Results in MNIST. (e-h) Results in Fashion-MNIST.
(a) MNIST
Figure 5: Ablation study over the weight of the regularization term for MNIST and Fashion-MNIST under distribution shift. Regularization weight determines the weight of PCIR in the training loss.
(b) Fashion-MNIST
Invariance regularization also improves in-distribution AD A surprising result from this study was the consistent enhancement in in-distribution performance across most methods when partial conditional invariance was incorporated. This observation can be explained by recent advances suggesting that invariance also bolsters in-distribution robustness against noise and sampling variability (Lopes et al. [2019], Mitrovic et al. [2020]) - factors to which anomaly detection is inherently vulnerable (Ramotsoela et al. [2018]). By design, invariance regularizers discourage representations encoding style features. Hence, representations only carry over information that is inherently related to the object rather than the environment. This leads to models identifying meaningful patterns instead of noisy variations in the data, even in-distribution.
Unlabeled environments A prominent limitation of invariance regularization lies in its dependence on environment labels (Veitch et al. [2021], Jiang and Veitch [2022], Arjovsky et al. [2019]). Additionally, as evidenced in our experimental comparisons between covariate and domain shifts, interventions directly applied to the covariates during environment generation yield enhanced robustness under the constraints of partial conditional invariance. This underscores that conditional invariance regularization's effectiveness diminishes when the environments are not distinctly segregated. However, in situations where datasets lack explicit environment partitions, alternative strategies can be employed. As demonstrated by Lin et al. [2022] it remains feasible to jointly estimate environment partitions and invariant representations, provided there is access to sufficient auxiliary information. This finding opens the door to expanding the applicability of partial conditional invariance in AD, even in scenarios with limited information on environmental conditions or corrupted separation between environments.
Invariance beyond partial conditioning Theorem 1 shows that in the presence of confounding, learning invariant representations require that representations are independent from the environment, when conditioned on  $W$ . Note that our partial conditional invariant regularization conditions only on  $W = 0$  and not on  $W = 1$  (see Equation 4). This is an inherent limitation due to the fact that we only have normal objects in the sample. However, our regularization is still powerful enough to provide improvements over the state of the art. We argue that the quality of this improvement depends on how disentangled content features are from style features (i.e.,  $X_{a}$  and  $X_{e}$  in Figure 2). Consider, for example, an MNIST setting where normal objects are images of particular digit and images of any other digit are anomalies. Furthermore, assume that images from one environment have one background color and images from another environment have another background color. There is a clear disentanglement between style (i.e., the digit) and content (i.e., the background color). In such settings, attaining invariance to the background color just with  $W = 0$  leads also to invariance to the background color with  $W = 1$ . We conjecture that partial conditional invariant regularization is sufficient when  $W$  does not influence  $X_{e}$ ; that is, when there is no arrow between these variables in Figure 2.
Multi-shift environments Certain datasets, such as the one described by Christie et al. [2018], take into account data shifts across two different kinds of domain shifts (e.g. time and location), equivalent to dual interventions in a bivariate  $E$  in our formulation. While the extension of our work to address multi-attribute settings might be plausible in a dual intervention scenario, it presents a non-trivial challenge as the number of different possible interventions increases, given that it requires the induction of pairwise invariance across environments. This issue continues to be an area of active research.
Fairness One potential extension of this work concerns the setting where either a covariate shift is induced over a protected attribute (e.g., gender or race), or a domain shift that permeates to such attributes. Such nuances have been brought in the context of invariant representation learning by Veitch et al. [2021] and could serve as a potential application of this work in AD. In particular, we could see our regularization term be implemented similarly to Louizos et al. [2015], but instead with the intent of effectively discarding incidental correlations that might reflect societal or systemic biases present in the data.
Our study sheds light on the significant challenges anomaly detectors face in the context of distribution shifts. We proposed a novel, robust solution centered around invariant representations, which mitigates the impact of shortcut learning by enforcing statistical independence between the representations and the environment. Empirical validation of our theoretical proposals confirmed the effectiveness of our approach, with a regularization term inducing partial conditional distribution invariance, significantly improving model performance under covariate and domain shifts. We believe these findings pave the way for a deeper understanding of AD methods' robustness and how to mitigate their vulnerability to distribution shifts.
D. Tellez, G. Litjens, P. Bandy, W. Bulten, J.-M. Bokhorst, F. Ciompi, and J. Van Der Laak. Quantifying the effects of data augmentation and stain color normalization in convolutional neural networks for computational pathology. Medical Image Analysis, 58:101544, 2019.
N. Tishby and N. Zaslavsky. Deep learning and the information bottleneck principle. In 2015 IEEE Information Theory Workshop (ITW), pages 1-5. IEEE, 2015.
V. Veitch, A. D'Amour, S. Yadlowsky, and J. Eisenstein. Counterfactual invariance to spurious correlations: Why and how to pass stress tests. arXiv preprint arXiv:2106.00545, 2021.
G. Wang, S. Han, E. Ding, and D. Huang. Student-teacher feature pyramid matching for anomaly detection. In The British Machine Vision Conference (BMVC), 2021.
R. Wang, M. Yi, Z. Chen, and S. Zhu. Out-of-distribution generalization with causal invariant transformations. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 375-385, 2022.
Z. Wang and V. Veitch. A unified causal view of domain invariant representation learning. arXiv preprint arXiv:2208.06987, 2022.
M. Wu, C. Zhuang, M. Mosse, D. Yamins, and N. Goodman. On mutual information in contrastive learning for visual representations. arXiv preprint arXiv:2005.13149, 2020.
H. Xiao, K. Rasul, and R. Vollgraf. Fashion-MNIST: a novel image dataset for benchmarking machine learning algorithms. arXiv preprint arXiv:1708.07747, 2017.
X. Yan, H. Zhang, X. Xu, X. Hu, and P.-A. Heng. Learning semantic context from normal samples for unsupervised anomaly detection. Proceedings of the AAAI Conference on Artificial Intelligence, 35(4):3110-3118, May 2021. doi: 10.1609/aaai.v35i4.16420. URL https://ojs(aaai.org/index.php/AAAI/article/view/16420.
H. Yao, Y. Wang, S. Li, L. Zhang, W. Liang, J. Zou, and C. Finn. Improving out-of-distribution robustness via selective augmentation. In Proceedings of the Thirty-ninth International Conference on Machine Learning, 2022.
A Theoretical Details
A.1 Measurability
Let  $\mathcal{B}(\mathbb{R}^n)$  be the Borel  $\sigma$ -algebra over  $\mathbb{R}^n$ , for some  $n \in \mathbb{N}$ .
Definition 4. The  $\sigma$ -algebra generated by a random variable  $Y$  is  $\sigma(Y) = \{Y^{-1}(B) : B \in \mathcal{B}(\mathbb{R}^n)\}$ .
Definition 5. A random variable  $X$  is  $Y$ -measurable if  $\sigma(X) \subseteq \sigma(Y)$ .
Intuitively, the  $\sigma$ -algebra generated by  $Y$  describes all events that  $Y$  "can express" and can be measured in probability. If  $X$  is  $Y$ -measurable, that means that  $Y$  can express all what  $X$  can express.
A.2 D-separation and Confounding
We provide here a brief overview on d-separation and confounding and refer the reader to Bishop and Nasrabadi [2006] for details.
Definition 6. Two random variables in a Bayesian network are confounded if they share a latent parent.
Definition 7. A path is a sequence of random variables  $(V_{1},\ldots ,V_{n})$  in a Bayesian network, where  $V_{i}$  is a parent or child of  $V_{i + 1}$ , for  $i < n$ . For  $1 < i < n$ , the variable  $V_{i}$  is a collision if  $V_{i}$  is a child of both  $V_{i - 1}$  and  $V_{i + 1}$ .
Definition 8. A path is blocked if  $V_{1}$  and  $V_{n}$  are not observed and at least one of following holds:
- For some collision  $V$  in the path, neither  $V$  nor any of its descendants is observed.
- For some variable  $V$  in the path that is not a collision,  $V$  is observed.
We now recall the d-separation principle.
Lemma 1. Let  $W$  be a set of observed variables and  $V_{1}$  and  $V_{2}$  two variables not observed. If any path from  $V_{1}$  to  $V_{2}$  is blocked, then  $V_{1} \bot V_{2} \mid W$ .
A.3 Detailed Proof of Theorem 1
Figure 6: Causal graphs for anomaly detection. The left figure shows the case of no confounding. The right figure shows the case of confounding. An intervention at the  $E$  variable induces a domain shift (gray hammer), whereas an intervention at the  $X_{e}$  variable induces a covariate shift (black hammer).
We repeat in Figure 6 the causal graph for anomaly detection here for convenience. To prove Theorem 1, we use the following lemma.
Lemma 2. If  $W$  and  $E$  are confounded, then  $X_{a}\perp E\mid W$ . Otherwise,  $X_{a}\perp E$ .
Proof. We prove this via d-separation. We start with the case of  $W$  and  $E$  not being confounded. Note that there are two paths from  $X_{a}$  to  $E$ . One via  $W$  and the other via  $Z$ . The path via  $W$  is blocked, because  $X_{e}$  is a collision and neither  $X_{e}$  nor its descendant  $Z$  are observed. The path via  $Z$
is blocked, because  $Z$  is a collision with no descendants and is not observed. Hence,  $X_{a} \bot E$ , when  $W$  and  $E$  are not confounded. In the case of  $W$  and  $E$  being confounded, we can show analogously that  $X_{a} \bot E \mid W$ . Note that there are three paths from  $X_{a}$  to  $E$ . The first one is via  $U$ , but this is blocked, because  $W$  is in that path, it is not a collision, and it is observed. The second one is via  $W$  and  $X_{e}$ , but  $W$  is in that path and it is not a collision there, so that path is also blocked. The third one is via  $Z$ , but  $Z$  is a collision in that path with no descendants and it is not observed, so it is also blocked. Since all paths are blocked, when  $W$  is observed, we conclude that  $X_{a} \bot E \mid W$ , when  $W$  and  $E$  are confounded.
Theorem 2. Suppose that  $f$  learns invariant representations. If  $W$  and  $E$  are confounded, then  $Z \perp E \mid W$ . Otherwise,  $Z \perp E$ .
Proof. Recall that if  $f$  learns invariant representations, then we assume it to be  $X_{a}$ -measurable. This assumption is justified in Veitch et al. [2021]. As a result, since  $Z = f(X_{a},X_{b})$ , we have that  $Z$  is  $X_{a}$ -measurable.
We assume that  $W$  and  $E$  are not confounded and show that  $Z \perp E$  by proving that  $p(Z \in A, E \in B) = p(Z \in A)p(E \in B)$ , for any  $A, B \in \mathcal{B}(\mathbb{R}^n)$ . Note that  $p(Z \in A, E \in B) = p\left(Z^{-1}(A) \cap E^{-1}(B)\right) = p\left(X_a^{-1}(C_A) \cap E^{-1}(B)\right)$ , for some Borel set  $C_A$ . The last equality follows from  $Z$  being  $X_a$ -measurable, which implies that  $Z^{-1}(A) = X_a^{-1}(C_A)$ , for some Borel set  $C_A$  by Definition 5. By Lemma 2, we have that  $X_a \perp E$ . This implies that  $p\left(X_a^{-1}(C_A) \cap E^{-1}(B)\right) = p\left(X_a^{-1}(C_A)\right)p\left(E^{-1}(B)\right) = p(Z \in A)p(E \in B)$ , which is what we wanted to show.
We now assume that  $W$  and  $E$  are confounded and show that  $Z \bot E \mid W$  by proving that
$$
p (Z \in A, E \in B \mid W \in C) = p (Z \in A \mid W \in C) p (E \in B \mid W \in C),
$$
for any  $A, B, C \in \mathcal{B}(\mathbb{R}^n)$ . By an analogous argument, we can show that  $p(Z \in A, E \in B \mid W \in C) = p(X_a^{-1}(C_A) \cap E^{-1}(B) \mid W^{-1}(C))$ . By Lemma 2, we have that  $p(X_a^{-1}(C_A) \mid W^{-1}(C)) \cdot p(E^{-1}(B) \mid W^{-1}(C))$ . With arguments similar to those above, we can show that the last expression is equal to  $p(Z \in A \mid W \in C) p(E \in B \mid W \in C)$ , which is what we wanted to show.
B.1 Camelyon17
Our realistic anomaly detection dataset was derived from the Camelyon17 dataset (Koh et al. [2021], Bandi et al. [2018]), and contains  $3 \times 96 \times 96$  patches of whole-slide images of lymph node sections sourced from patients who may have metastatic breast cancer. This dataset encompasses tissue patches obtained from five different hospitals. The objective here is to accurately predict the presence of tumor tissue within the patches drawn from hospitals that were not part of the training data. Prior work has shown that differences in staining between hospitals are the primary source of variation in this dataset, however, other divergent factors in the sampling distribution include different acquisition protocols and patient populations (Tellez et al. [2019]).
The in-distribution data was comprised of 151, 280 images evenly distributed across three hospitals, or 100, 810 images evenly distributed across two hospitals depending on the training setting. The other out-of-distribution data covered two additional datasets, the first with 34, 904 patches, and the second with 85, 054 patches. Note that to adapt this dataset to the anomaly detection setting, only normal images were included in the in-distribution training data.
B.2 Synthetic datasets
The synthetic datasets employed in this study were derived from the DiagViB-6 benchmark (Eulig et al. [2021]). This benchmark uniquely allows for the manipulation of five independent generative factors from colored images: overlaid texture, object size, object position, lightness, and saturation, in addition to the semantic features that correspond to the label. Our synthetic experiments utilized two datasets: MNIST (Deng [2012]) and Fashion-MNIST (Xiao et al. [2017]). All images in both datasets were upsampled to dimensions of  $3 \times 256 \times 256$ .
Figure 7: Illustration of our experimental setup for the synthetic covariate shift experiment. The image demonstrates representative examples of training data from two distinct environments, alongside instances of normal and abnormal test data subject to progressively accumulated covariate shifts. This configuration embodies the nuanced challenge of identifying subtle, yet potentially consequential, changes in the data distribution.
Initially, we generate two unique and distinct environments specifically designed for the training data. Our primary goal during this stage was to guarantee that all these factors exhibited noticeable differences when compared across the two generated environments.
Following the generation of these training environments, we proceeded to develop another pair of environments. These new environments were crafted for the validation data. To ensure consistency, these validation environments were fashioned in such a way that they closely mirrored or replicated the factor configuration that was present in the initial training environments, thus retaining an in-distribution setting.
In the final step, six additional environments, denoted as  $e_0, e_1, \ldots, e_5$ , were generated. Each environment  $e_i$  consists of images in which  $i$  factors have been altered with respect to  $e_0$ . For a depiction of the samples for these different environments, please refer to Fig. 7.
In devising our evaluation setup, we opted for inducing covariate shifts that are minor deviations from the original in-distribution environments. This decision was motivated by our goal to simulate subtle yet potentially detrimental covariate shifts, particularly in comparison to the challenge of differentiating normal from abnormal.
A description of the accumulated covariate shifts in the test environments,  $e_0, e_1, e_2, e_3, e_4$ , is provided in Table 1.