SlowGuess/NeurIPS_2023_OCR huggingface dataset:
In this work, we use a total of 9 continuous and 3 categorical datasets, with their dimensions and data types shown in Table 2. The Gas, Covid, and Energy datasets are the same as those used in [9], and we apply identical preprocessing procedures. The Musk2 dataset provides information on musk and non-musk molecules. The Scene dataset describes image characteristics. The MNIST dataset consists of images depicting handwritten digits. The Dilbert dataset is an image recognition dataset of pictures of objects rotated from various orientations. We additionally include 3 internal datasets of biomedical data: Phenotypes, Canine, and Founders datasets. The Phenotypes dataset contains a subset of categorical phenotypes from the UK Biobank, following the same pre-processing as in [103]. The Canine and Founders datasets comprise binary-coded sequences of DNA including Single Nucleotide Polymorphisms (SNPs), representing data for multiple dog breeds and human populations, respectively.
The Polynomial and Cosine are two internally generated datasets consisting of values obtained from deterministic simulations. The Polynomial dataset includes samples where each feature value is obtained by evaluating a second-degree polynomial function  $f(x) = ax^2 + bx + c$ . The parameters  $a, b$ , and  $c$  are fixed for each sample. The feature values are derived by evaluating the polynomial function for the different  $x$  values within a sample. The values for  $x, a, b$ , and  $c$  are uniformly sampled from the range  $[-10, 10]$ . The Cosine dataset consists of samples with features following a cosine function  $f(x) = a \cdot \cos(bx + c)$ . Similar to the Polynomial dataset,  $a, b$ , and  $c$  are fixed for each sample, and each feature value is obtained by evaluating the cosine function for the different  $x$  values within a sample. Here, the values of  $b$  and  $c$  are uniformly sampled within the range  $[- \pi, \pi]$ .
Table 2: Datasets used to evaluate DataFix.
Dataset	No. of attributes	No. of samples	Data
Gas	8	12,815	Cont.
Covid	10	9,889	Cont.
Energy	26	19,735	Cont.
Musk2	166	6,598	Cont.
Scene	294	2,407	Cont.
MNIST	784	70,000	Cont.
Phenotypes	1,227	31,424	Cat.
Dilbert	2,000	10,000	Cont.
Founders	10,000	4,144	Cat.
Canine	198,473	1,444	Cat.
Cosine	1,000	10,000	Cont.
Polynomial	1,000	10,000	Cont.
B.2 Simulated Probabilistic Datasets
We generate 15 simulated datasets containing 1000 features and 5000 samples by sampling from predefined probabilistic distributions, including multivariate Gaussians, with and without transformations, multivariate Bernoulli distributions, Gaussian mixture models, and Bernoulli mixture models. A total of 200 features are shifted in each dataset such that  $|C| = 200$  and  $|\overline{C}| = 800$ . Table 3 describes the distributions used in each dataset. For every dataset, two distributions,  $p$  and  $q$ , are defined, so that  $D(p,q) > 0$  but  $D(p_{\overline{C}},q_{\overline{C}}) = 0$ . In fact, for all datasets except for dataset 8, we also have that  $D(p_{C},q_{C}) > 0$ . A further discussion of the effect of shifts with  $D(p_{C},q_{C}) > 0$  and with  $D(p_{C},q_{C}) = 0$ , and its relation to the equivalence of feature shift localization and feature selection, is provided in the following sections. Because we have access to  $p$  and  $q$ , we can compute the real divergence between the distributions. In practice, we make use of a Monte Carlo estimate, as shown in Figures 5 and 25, because the divergences might not have a closed-form solution or can be computationally intractable.
Datasets 1-3 are based on a multivariate Gaussian, with diagonal covariance used in datasets 1 and 2, and a covariance  $\Sigma$  used in dataset 3. The covariance matrix  $\Sigma$  is defined by a Gaussian kernel such that the  $ij$  component is  $\Sigma_{ij} = \exp \frac{-||i - j||^2}{s}$ , where  $s$  acts as a scale parameter, and  $0 \leq \Sigma_{ij} \leq 1$  with  $\Sigma_{ii} = 1$ . In practice, when constructing the covariance matrix, we perform a shuffle of the feature order to better depict tabular data, where, in many cases, the correlation between features does not follow any specific ordering (opposed to images or audio). We use  $s = 0.05$  to define  $\Sigma_{ij}$  in dataset 3. Datasets 4 and 5 follow a lognormal distribution, defined as  $X = \exp(V)$  with  $V \sim \mathcal{N}(\mu, \Sigma)$  and  $X \sim \mathrm{Lognormal}(\mu, \Sigma)$ . We use  $s = 0.05$  and  $s = 0.002$  to define  $\Sigma_{ij}$  in datasets 4 and 5, respectively. Datasets 6-8 follow a logit-normal distribution defined as  $X = \sigma(V)$  with  $V \sim \mathcal{N}(\mu, \Sigma)$  and  $X \sim P(\mathcal{N}(\mu, \Sigma))$ , where  $\sigma$  is the sigmoid transformation. We use  $s = 0.05$ ,  $s = 0.002$ , and  $s = 0.002$  to define  $\Sigma_{ij}$  in datasets 6, 7, and 8, respectively. Datasets 9-12 follow a multivariate Bernoulli with independent features. Each feature  $i$  has a frequency of  $f_i$ , where  $f \sim P(\mathcal{N}(0, 2I)), \epsilon \sim \mathcal{N}(0, I)$ , and  $(\cdot)_{0,1} = \mathrm{clamp}_{0,1}(\cdot) = \min(\max(\cdot, 0), 1)$  is the clamping function to ensure that the frequencies are between 0 and 1. Dataset 13 follows a Gaussian Mixture Model distribution, with 3 mixtures of equal weights,  $\mu_i \sim \mathcal{N}(0, 0.01I)$ , and  $\Sigma_i$  defined with  $s = 0.3$ . Datasets 14 and 15 follow a Bernoulli Mixture Model distribution with 3 mixtures and  $f_i \sim \mathrm{Uniform}^d(0, 1)$ .
Datasets 1, 3, 4, 6, 9, 10, 11, 12, 13, and 15 apply a shift to the marginal means, such that for all  $i \in C$ ,  $\mathbb{E}[p_{C_i}] \neq \mathbb{E}[q_{C_i}]$ . Such datasets include marginal shifts of a similar nature as the shifts generated by manipulation types 1, 2, and 6 applied to real datasets. Dataset 2 performs a shift of the marginal standard deviation while maintaining their mean such that for all  $i \in C$ ,  $\mathbb{E}[p_{C_i}] = \mathbb{E}[q_{C_i}]$  but  $p_{C_i} \neq q_{C_i}$  and  $\mathrm{Var}[p_{C_i}] \neq \mathrm{Var}[q_{C_i}]$ , leading to a marginal shift similar to the one applied by manipulation type 4 used in the real datasets. Datasets 13 and 15 apply a shift to the mean of just one mixture of the mixture model, leading to only  $1/3$  of the samples being shifted, while still ensuring that  $\mathbb{E}[p_{C_i}] \neq \mathbb{E}[q_{C_i}]$ . Datasets 5, 7, and 14 apply a distribution shift consisting of removing the correlation between features, equivalently to manipulation type 3 applied in real datasets, such that  $q_C = \prod_{i \in C} q_i$  and  $p_C \neq q_C$ . Dataset 8 also applies a shift originating from modifying the correlation between features, but in this case,  $\Sigma'$  is defined as:
$$
\Sigma_ {i j} ^ {\prime} = \left\{ \begin{array}{l l} \Sigma_ {i j} & \text {i f} i, j \in C, \text {o r} i, j \in \overline {{C}} \\ 0 & \text {i f} i \in C \text {w i t h} j \in \overline {{C}}, \text {o r} j \in C \text {w i t h} i \in \overline {{C}} \end{array} \right. \tag {9}
$$
where the correlation of the features within  $C$  and within  $\overline{C}$  are maintained, but the cross-correlations between the  $C$  and  $\overline{C}$  are lost, which leads to a shift equivalent to manipulation type 8 applied in real datasets, such that  $p_{C} = q_{C}, p_{\overline{C}} = q_{\overline{C}}$ , but  $p \neq q$ .
Table 3: Probabilistic datasets used to evaluate DataFix.
ID	pc	qc	Description
1	N(0, I)	N(0.5I, I)	Multivariate Gaussians with diagonal covariance and a shifted mean.
2	N(0, I)	N(0, 1.5I)	Multivariate Gaussians with diagonal covariance and a shifted scale.
3	N(0, Σ)	N(0.5I, Σ)	Multivariate Gaussians with non-diagonal covariance and a shifted mean.
4	Lognormal(0, Σ)	Lognormal(0.5I, Σ)	Multivariate lognormal with non-diagonal covariance and a shifted mean.
5	Lognormal(0, Σ)	Lognormal(0, I)	Multivariate lognormal with non-diagonal (pC) and diagonal (qC) covariance.
6	P(N(0, Σ))	P(N(0.5I, Σ))	Multivariate logit-normal with non-diagonal covariance and a shifted mean.
7	P(N(0, Σ))	P(N(0, I))	Multivariate logit-normal with non-diagonal (pC) and diagonal (qC) covariance.
8	P(N(0, Σ))	P(N(0, Σ'))	Multivariate logit-normal with different non-diagonal covariances.
9	Bernoulli(f)	Bernoulli((f + 0.05ε)0,1)	Multivariate independent Bernoulli with a shifted mean.
10	Bernoulli(f)	Bernoulli((f + 0.1ε)0,1)	Multivariate independent Bernoulli with a shifted mean.
11	Bernoulli(f)	Bernoulli((f + 0.5ε)0,1)	Multivariate independent Bernoulli with a shifted mean.
12	Bernoulli(f)	Bernoulli((f + 1.0ε)0,1)	Multivariate independent Bernoulli with a shifted mean.
13	1/3 ∑i=13 N(μi, Σi)	1/3 ∑i=13 N(μi', Σi)	Gaussian Mixture Model with one mixture shifted such that μ1' = μ1 + 10, μ2' = μ2, and μ3' = μ3.
14	BMM([f1, f2, f3])	BMM([f', f', f'])	Bernoulli Mixture Model with different means f' = f1+f2+f3/3.
15	BMM([f1, f2, f3])	BMM([f1 + 0.2)0,1, f2, f3])	Bernoulli Mixture Model with one mixture shifted.
C Feature Selection and Feature Shift Localization Equivalence
DF-Locate combines feature importance and feature selection techniques to localize the feature subset originating the distribution shift. Feature selection techniques often solve, either implicitly or explicitly, the following problem:
$$
C = \underset {| C | \leq k} {\operatorname {a r g m a x}} I \left(z _ {C}; t\right) \tag {10}
$$
where  $C$  is the feature subset of size up to  $k$ , which maximizes the mutual information  $I$  between the input sample  $z$ , restricted to the features in  $C$ , and the label  $t$ . In practice, the distributions of  $z$  or  $t$  are unknown, making Eq. 2 intractable. Instead, most methods predict a score that approximates the amount of information relative to the label present at each feature:  $\beta = F(Z,T)$ , where  $\beta \in \mathbb{R}^d$  and  $F(\cdot)$  is a function mapping samples  $Z$  and labels  $T$  to an importance score for each feature. Informally, these scores provide an ordering such that if features  $i$  and  $j$  have  $|\beta_i| > |\beta_j|$ , then  $I(z_{i};t) \lesssim I(z_{j};t)$ .
Section 4 shows that by defining  $z$  as the samples coming from  $X$  and  $Y$ , and  $t$  as the binary label of origin, such that  $z|_{t=0} \sim p$  and  $z|_{t=1} \sim q$ , then Fano's inequality [71] and LeCam's method [72] can be used to relate the mutual information between  $z$  and  $t$  and the total variation distance between  $p$  and  $q$ . Namely, when the number of manipulated features is known  $|C| = k$ , and the divergence of the corrupted features is larger than 0, such that  $D_{TV}(p,q) = D_{TV}(p_C,q_C) = 1$ , and  $D_{TV}(p_{\overline{C}},q_{\overline{C}}) = 0$ , then feature selection and feature shift localization are equivalent:
$$
C = \underset {| C | \leq k} {\operatorname {a r g m a x}} I \left(z _ {C}; t\right) = \underset {D _ {T V} \left(p _ {\overline {{C}}}, q _ {\overline {{C}}}\right) = 0} {\operatorname {a r g m i n}} | C | \tag {11}
$$
Even if  $|C|$  is unknown, if only marginal shifts are present, one can perform feature shift detection by iteratively solving Eq.2 with  $k = 1$ , and removing the detected features at each iteration from  $z$  as long as  $I(z_{C};t) > 0$ , making the iterative feature selection and feature shift localization equivalent problems:
$$
C _ {i} = \underset {| C | = 1} {\operatorname {a r g m a x}} I \left(z _ {C}; t\right) \tag {12}
$$
where  $C = \bigcup_{i=1}^{l} C_i$ . In fact, the approach presented in section 4, DF-Locate, can be seen as an approximation of this iterative process, where one or more features are selected at each step by the feature removal policy function.
However, the equivalence of both tasks breaks down for distribution shifts such as the one applied in the manipulation type 8 for real datasets, and in dataset 8 of probabilistic simulations, where  $p_C = q_C$  and  $p_{\overline{C}} = q_{\overline{C}}$ , but  $p \neq q$ . That is, when considering only the corrupted features  $C$  or the non-corrupted features  $\overline{C}$  in isolation, the shift is impossible to detect unless all features are considered jointly. Therefore, any feature selection technique that approximates either implicitly or explicitly equation 2, will need a subset of features  $G$  containing features from both  $C$  and  $\overline{C}$  in order to obtain  $I(z_G; t) > 0$  because  $I(z_C; t) = 0$  and  $I(z_{\overline{C}}; t) = 0$ . Furthermore, if  $|C| = |\overline{C}|$  the problem of feature shift detection becomes unsolvable. This is because, even if a technique was able to properly identify a subset of features  $G = C$ , it would not be possible to know if the detected subset  $G$  contains corrupted or non-corrupted features, that is if  $G = C$  or  $G = \overline{C}$ , making the assumption of  $|C| < |\overline{C}|$  a necessary condition.
Figure 3 (right) and Figure 17 show the F-1 score for each manipulation type, indicating that DataFix is able to correctly localize manipulated features with manipulation type 8 on real datasets, despite breaking the equivalence between feature selection and feature shift localization approaches. In contrast, Figure 18, which illustrates the average F-1 on the probabilistic datasets, shows that dataset 8 is the one providing the lowest F-1 scores. While the low F-1 score in dataset 8 can be partly caused by the mismatch between feature selection and feature shift localization problems, it can also originate from the difficulty of detecting shifts caused by mismatching correlations, as it provides similar performance as in datasets 5 and 7, where correlation shifts (with  $p_C \neq q_C$ ) are applied.
D Feature Shift from Imputation Methods
Imputation and supervised methods trained to reduce the expected mean square error (MSE) between the predicted  $\hat{x}_C$  and real  $x_C$  features of a given sample  $x$  can lead to distribution shifts. Note that the optimal function minimizing  $\mathbb{E}_{x\sim P}[||x_C - f(x_{\overline{C}})||^2]$  is the expected value of  $x_C$  conditioned in  $x_{\overline{C}}$ , that is  $f^{*}(x_{\overline{C}}) = \mathbb{E}[x_{C}|x_{\overline{C}}]$ . Therefore, the method that predicts the corrupted (or missing) features  $C$  given the non-corrupted (non-missing) features  $\overline{C}$ , which provides the lowest MSE, will generate predicted samples  $\hat{x}_C = f^* (x_{\overline{C}})$ , with a distribution where  $\mathbb{P}(\hat{x}_C) = 1$  for  $\hat{x}_C = \mathbb{E}[x_C|x_{\overline{C}}]$ , and  $\mathbb{P}(\hat{x}_C) = 0$  everywhere else. If  $\mathrm{Var}[x_C|x_{\overline{C}}] > 0$ , then  $D(\mathbb{P}(\hat{x}_C),\mathbb{P}(x_C)) > 0$  because  $\mathrm{Var}[\hat{x}_C|x_{\overline{C}}] = 0$ .
For example, given a dataset of samples  $x_{1}, x_{2}, \ldots, x_{N}$ , with  $x_{i\overline{C}} = x_{j\overline{C}}$  and  $x_{iC} \neq x_{jC}$  for all  $i$  and  $j$ , in other words,  $\operatorname{Var}[x_C|x_{\overline{C}}] > 0$ , the optimal regression model, in terms of MSE, will predict  $\hat{x}_{iC} = \mathbb{E}[x_{iC}|x_{i\overline{C}}]$  for all  $i$ , such that  $\operatorname{Var}[\hat{x}_C|x_{\overline{C}}] = 0$ . Similarly, consider a dataset where the features  $C$  and  $\overline{C}$  are independent, such that  $p = p_{C}p_{\overline{C}}$ , and  $p_{C} = \mathcal{N}(\mu, I)$ , then  $\hat{x}_C = \mathbb{E}[x_C|x_{\overline{C}}] = \mathbb{E}[x_C] = \mu$ , where  $\mu \in \mathbb{R}^{|C|}$  is a constant vector. The simulated probabilistic dataset 1 follows this form, with  $p_{C} = \mathcal{N}(0, I)$  and  $q_{C} = \mathcal{N}(0.5I, I)$ , where the method providing the lowest MSE will be the one predicting  $\hat{x}_C = \mathbb{E}[x_C|x_{\overline{C}}] = \mu = 0$  for all samples. Figure 6 shows the histogram of the first feature values before and after performing feature correction with multiple methods. Most techniques, especially imputation-based techniques, output the mean or values highly close to it, which while producing minimum MSE, does not reflect the real distribution, removing or reducing its variance and leading to a distribution shift.
Figure 6: Histograms of first feature values before and after shift correction with various techniques for probabilistic dataset 1.
E DF-Locate
Figure 7: DF-Locate diagram.
DF-Locate (Section 4, Figures 1 and 7, and Algorithm 1) is the proposed method within DataFix that localizes the features originating the distribution shift by performing feature selection in an iterative way. First, starting with  $i = 0$ , and a reference  $X^{(0)} = X$  and query  $Y^{(0)} = Y$  datasets, a set of discriminators are trained  $\theta = \operatorname{argmax}_{\theta} \hat{D}_{\theta}(X^{(i)},Y^{(i)})$ . The discriminators are used to predict the empirical total variation distance (TVD) between distributions  $\hat{D} = \hat{D}_{\theta}(X^{(i)},Y^{(i)})$ , and a feature importance score for each feature  $\beta = F_{\theta}(X^{(i)},Y^{(i)})$ . The divergence and feature importances are used to select potentially corrupted features with the feature removal policy function (see section 4)  $C_i = \psi_\tau(\beta, \hat{D})$ . Then, the detected features are removed from  $X$  and  $Y$ , such that  $X^{(i+1)} = X\frac{(i)}{C_i}$  and  $Y^{(i+1)} = Y\frac{(i)}{C_i}$ . The process is repeated as long as the estimated divergence is smaller than a threshold  $\hat{D}_{\theta}(X^{(i)},Y^{(i)}) > \epsilon$ , less than half of the features of the dataset are removed  $k^{(i)} < \frac{|Y|}{2}$ , or at least one feature is selected by the feature removal policy function at each step  $k^{(i)} - k^{(i-1)} > 0$ . In this work we use  $\epsilon = 0.02$  as the stopping threshold. After the iterative process is stopped, a refinement step, described below, is applied to remove features that might have been incorrectly selected as corrupted.
E.1 Shift Detection
We detect whether a shift is present by estimating an  $f$ -divergence between the distributions  $p$  and  $q$ . Specifically, we make use of the variational form [67, 68, 69]:
$$
D _ {f} (p, q) \geq \mathbb {E} _ {x \sim p} \left[ f ^ {\prime} \left(r _ {\theta} (x)\right) \right] - \mathbb {E} _ {y \sim q} \left[ f ^ {*} \left(f ^ {\prime} \left(r _ {\theta} (y)\right)\right) \right] \tag {13}
$$
where  $r_{\theta}(x)$  is a function approximating the likelihood ratio between  $p$  and  $q$ . The inequality becomes an equality when  $r_{\theta}(x) = r^{*}(x) = \frac{p(x)}{q(x)}$  is the true likelihood ratio function, defining the Bayes decision rule that optimally separates  $p$  and  $q$ .  $f$  is the generator function of the  $f$ -divergence,  $f'$  is its first order derivative and  $f^{*}$  is its Fenchel conjugate [70].
In practice, estimating  $D_{f}(p,q)$  is intractable, as we do not have direct access to  $p$  and  $q$ . Hence, we use an empirical estimator of the divergence by training a binary classifier  $\mathcal{D}_{\theta}(x)$  to discriminate the samples coming from  $X = \{x_{1},\dots,x_{N}\}$  and  $Y = \{y_{1},\dots,y_{N}\}$ , with  $x_{i}\sim p$  and  $y_{j}\sim q$ . The likelihood ratio can be easily estimated as  $r_{\theta}(x) = \mathcal{D}_{\theta}(x) / 1 - \mathcal{D}_{\theta}(x) = \exp \sigma^{-1}(\mathcal{D}_{\theta}(x))$ , where  $\sigma^{-1}$  is the inverse sigmoid. We use  $k$ -fold train-evaluation split, where the dataset is divided into  $k = 5$  subsets. At each iteration,  $80\%$  of the samples are used to train a random forest classifier  $\mathcal{D}_{\theta}(x)$ , and the  $20\%$  left is used to estimate the empirical divergence:
$$
\hat {D} _ {\theta} (X, Y) = \frac {1}{N _ {x}} \sum_ {i = 1} ^ {N _ {x}} f ^ {\prime} \left(r _ {\theta} \left(x _ {i}\right)\right) - \frac {1}{N _ {y}} \sum_ {j = 1} ^ {N _ {y}} f ^ {*} \left(f ^ {\prime} \left(r _ {\theta} \left(y _ {j}\right)\right)\right) \tag {14}
$$
where  $N_{x}$  and  $N_{y}$  are the number of testing samples of the reference and query datasets respectively, and  $D_{f}(p,q)\geq \mathbb{E}_{x\sim p,y\sim q}[\hat{D}_{\theta}(X,Y)]$ . By training the random forest, we are approximately finding the discriminator that provides a maximal divergence  $\theta^{*} = \operatorname{argmax}_{\theta}\hat{D}_{\theta}(X,Y)$  within the random forest hypothesis space.
E.2 Feature Removal Policy Function
The feature removal policy function  $\psi$ , introduced in section 4, makes use of the predicted feature importances  $\beta = F_{\theta}(Z,T)$  and the estimated total variation distance  $\hat{D} = \hat{D}_{TV}(X,Y)$  to select the corrupted features  $C = \psi(\beta, \hat{D})$ . This function utilizes the sorted normalized absolute value of the importance scores  $\beta_{\pi}' = \frac{|\beta|_{\pi}}{\sum_{i=0}^{d-1} |\beta_i|}$  with  $\beta_{\pi(0)}' \geq \ldots \geq \beta_{\pi(d-1)}'$ . At each iteration, the first  $J$  features are selected as corrupted, where  $J$  is dynamically selected at each iteration (equation 4) as the smallest index  $k$  such that  $\gamma(k) = \sum_{j=0}^{k} \beta_{\pi(j)}' \geq \tau \hat{D}$  (Eq. 4), with  $\tau$  selected by cross-validation. The features selected as corrupted  $C$  include the features from 0 to  $J$  with scores higher than  $\frac{1}{d}$  (Eq. 5).
The product  $\tau \hat{D}$  specifies a threshold that defines how many features are selected as corrupted at each iteration. When  $\tau \hat{D}$  is small,  $\psi$  returns the feature with the highest feature importance, while a larger  $\tau \hat{D}$  leads to more features flagged as corrupted. The value of  $\tau$  is selected through hyperparameter optimization using the simulated datasets and maintained fixed afterward, and  $\hat{D}$  is computed at each iteration with values that are, in expectation, non-increasing. When the empirical shift is low, the product  $\tau \hat{D}$  is small, leading to a smaller number of features being flagged as corrupted. Figure 8 presents a comparative analysis of the sorted normalized feature importance scores  $\beta_{\pi}^{\prime}$  in the first, middle, and last iterations of the DF-Locate iterative process at the first, second, and third figure column respectively. Different thresholds  $\tau$  (0.05, 0.1, 0.2, and 0.3) are used to localize and filter the corrupted features at each iteration, with each row representing a specific threshold value. The features flagged as corrupted (as part of  $C$ ) by the feature removal policy function  $\psi$  are indicated in red color. As the number of iterations increases,  $\hat{D}$  decreases and fewer features are flagged as corrupted, with the last iteration containing only a small number of remaining corrupted features and most features exhibiting similar low importance scores. Larger thresholds  $\tau$  lead to more features selected at each iteration. However, for thresholds higher than 0.1, some of the selected features can be falsely classified as corrupted, indicating the need for lower threshold values.
Figure 9 shows the number of removed features at each iteration with different threshold values  $\tau$ . Across all threshold values, the number of removed features decreases as the number of iterations increases, and in turn, the empirical divergence decreases. Note that higher threshold values lead to more features removed at each iteration, resulting in a smaller total number of iterations and faster computational times. However, higher thresholds also lead to an increased number of false positives, where many non-corrupted features are incorrectly selected as corrupted, leading to low F-1 scores. Therefore, the  $\tau$  threshold provides a trade-off mechanism between feature shift localization speed and accuracy.
E.3 Refinement Stage