Feature Shift Localization Network
Authors: Míriam Barrabés
Daniel Mas Montserrat
Kapal Dev
Alexander G. Ioannidis
Venue: ICML 2025 poster (2025)
Keywords: feature shift localization
shift detection
shift
feature shift
distribution shift
Primary area: deep_learning->everything_else
TL;DR: We introduce the Feature Shift Localization Network, a model that can localize feature shifts in a single forward pass, without requiring dataset-specific training.

Abstract: Feature shifts between data sources are present in many applications involving healthcare, biomedical, socioeconomic, financial, survey, and multi-sensor data, among others, where unharmonized heterogeneous data sources, noisy data measurements, or inconsistent processing and standardization pipelines can lead to erroneous features. Localizing shifted features is important to address the underlying cause of the shift and correct or filter the data to avoid degrading downstream analysis. While many techniques can detect distribution shifts, localizing the features originating them is still challenging, with current solutions being either inaccurate or not scalable to large and high-dimensional datasets. In this work, we introduce the Feature Shift Localization Network (FSL-Net), a neural network that can localize feature shifts in large and high-dimensional datasets in a fast and accurate manner. The network, trained with a large number of datasets, learns to extract the statistical properties of the datasets and can localize feature shifts from previously unseen datasets and shifts without the need for re-training. The code and ready-to-use trained model are available at \url{https://github.com/AI-sandbox/FSL-Net}.

Feature Shift Localization Network
Míriam Barrabés" Daniel Mas Montserrat
Kapal Dev Alexander G. loannidis
Abstract
biomedical, and life sciences datasets, where different sam- ples are generated at different organizations (e.g., hospitals, labs), with differing lab technologies, hardware, and data processing producing unique structural biases. In clinical ge- nomics, feature shifts can arise from heterogeneous data ac- quisition protocols, which may involve differences in geno- typing arrays or phenotype curation procedures (Moreno- Grau et al., 2024). Similar shifts are found in social sciences, streaming, and e-business applications, where combining tabular and structured data from multiple sources, regions, and times without proper homogenization steps can lead to mismatching and biased features due to incorrect data collection procedures, human entry errors, faulty standard- ization, or erroneous data processing (Barchard Pace, 2011; Dai et al., 2015). Similarly, multi-sensor applica- tions in the manufacturing industry, medicinal monitoring, finance analysis, and defense can suffer feature shifts due to faulty sensors and measuring devices (Qian al., 2022; Barrabés et al., 2023). While numerous techniques enable pre-processing data sources to reduce feature shifts, proper data homogenization can be a challenging task requiring data-dependent and domain-speciff techniques (Lim et al., 2018). When data homogenization fails, unattended feature shifts can negatively impact downstream applications, lead- ing to erroncous scientific results or biased machine learning models, which makes feature shift localization critical in many data-driven domains.
Feature shifts between data sources are present in many applications involving healthcare, biomedi- cal, socioeconomic, financial, survey, and multi- sensor data, among others, where unharmonized heterogeneous data sources, noisy data measure- ments, or inconsistent processing and standard- ization pipelines can lead to erroneous features. Localizing shifted features is important to ad- dress the underlying cause of the shift and cor- rect or filter the data to avoid degrading down- stream analysis. While many techniques detect distribution shifts, localizing the features originating them is still challenging, with cur- rent solutions being either inaccurate or not scal- able to large and high-dimensional datasets. In this work, we introduce the Feature Shift Lo- calization Network (FSL-Net), neural network that can localize feature shifts in large and high- dimensional datasets in a fast and accurate man- ner. The network, trained with large number of datasets, learns to extract the statistical proper- ties of the datasets and can localize feature shifts from previously unseen datasets and shifts with- out the need for re-training. The code and ready- to-use trained model are available https: / //github. aaaa//IIIaaano//s-eeeeeee.
Feature shift localization is the task of enumerating which features of mulei-dimeenional datasets are originating the distribution shift between two or more data sources. The localization step is necessary identify and correct the er- source, whether by data removal or correction in tabular data-based applications or through physical intervention in multi-sensory scenario applications. Extensive literature on anomaly detection and distribution shift detection (Yu et al., 2018; Pan et al., 2020) has led to numerous tools for automated shift detection that are common in data-centric AI (DCAT) and machine learning systems (MLSyss) tech- nologies, providing functionalities for data quality control, homogenization, and monitoring (Ginart et al., 2022; Piano et al., 2022; Zha et al., 2023; Subasri et al., 2023). While many methods focus on asserting whether two datasets fol- low the same distribution, most do not localize the exact features causing the shift, and recent promising shift lo- calization techniques still fail to scale to large and high-
1. Introduction
Feature distribution shifts between data sources are common in many real-world applications using multi-dimensional data composed of set of "corrupted" features (i.e., di- mensions) with mismatching statistical qualities between sources. These feature shifts are prevalent in healthcare,
Equual contribution 'Department Biomedical Data Science, Stanford University, Stanford, CA 94305 USA 'Department of Computer Science. Munster Technological University, Cork T12 P928, Ireland 'Department of Biomolecular Engineering, Univer- sity of California Santa Cruz, Santa Cruz, CA 95060 USA. Corre- spondence to: Alexander G. loannidis <ioannidis edu.
Proceedings of the ndnd International Conference Machine Learning. Vancouver, Canada. PMLR 267, 2025. Copyright 2025 by the author(s).

---

Feature Shift Localization Network
dimensional datasets common in many areas (Kulinski et al., 2020; Barrabés et al., 2023).
between distributions, feature shift localization methods aim to predict which features are causing the shift. A no- table contribution in this area is the work by (Kulinski et al., 2020), which introduces a conditional test capable of ac- curately identifying shifted features with model-free and model-based approaches: K-Nearest Neighbors with KS statistic (KNN-KS), multivariate Gaussian with KS (MB- KS), multivariate Gaussian and Fisher-divergence test statis- ties (MB-SM), and deep density neural models with Fisher- divergence test (Deep-SM). DataFix (Barrabés et al., 2023) is a more recent method that improves localization accu- racy by iteratively training a random forest to distinguish between reference and query distributions, removing the fea- tures with the highest impurity -based importance scores un- til divergence is minimized, and applying a knee-detection algorithm to determine the optimal stopping point. Although DataFix performs well in many cases, it struggles with de- tecting challenging feature shifts and scales poorly with high-dimensional and large datasets. Its repeated use of random forest training results in significant computational overhead, limiting its applicability to real-world applica- tions involving massive datasets. In this paper, we adopt the same evaluation benchmark as DataFix and introduce FSL-Net to overcome these limitations.
In this work, we introduce a novel neural network, the Feature Shift Localization Network (FSL-Net). which can localize shifts with high accuracy while scaling to high- dimensional and large datasets. The network extracts sta- tistical descriptors from two datasets and then processes them localize the features originating the distribution shift. Namely, FSL-Net has two subnetworks, Statistical Descriptor Network that compresses datasets into statisti- cal functionals that summarize their underlying distribution, and a Prediction Network that combines the statistical de- scriptors between datasets to predict the probability of being corrupted for each feature. FSI.-Net makes use of convo- lutional and pooling layers to achieve invariance to sample order and approximate equivariance to feature order. The network is trained end-to-end using multiple datasets with different types of simulated feature shifts and is evaluated on previously unseen datasets and shift types, showing that it generalizes well out-of-the-boo to large variety of data and shifts without the need for re-training. FSL--Net surpasses previous feature shift localization methods in localization accuracy and speed.
Our contributions include: (1) we propose a novel neural network architecture that provides invariance to sample order and equivariance to feature order while scaling to large and high-dimensional datasets; (2) we design a training approach leading to a network that generalizes to unseen data and shifts without the need for re-training; (3) we provide in-depth experimental evaluation with multiple manipulation types, datasets, and network configurations.
Feature Selection. Feature selection methods localize the most relevant features for classification or regression, pro- viding interpretability and removing redundancy. Wrapper (Maldonado & Weber, 2009; Mustaqeem et al., 2017). fil- tering (Nasir et al., 2020; Hopf & Reifenrath, 2021), and embedded methods (Tran al., 2016; Huang et al., 2018) are among the most common techniques. Wrapper meth- ods select features of interest by training ML models and adding or removing features through a search process. Fil- tering methods include Mutual Information (MI) (Battiti, 1994), ANOVA-F test (Elssied et al., 2014), Chi-square test (Bahassine et al., 2020), Minimum Redundancy Max- imum Relevance (MRMR) (Ding & Peng, 2005; Li et al., 2018), and Fast-Conditions Mutual Information Maximiza- tion (FAST-CMIM) (Fleuret, 2004), among others. Such methods extract statistical information from the data to rank the importance of each feature. Embedded methods rank fea- tures using built-in scores from ML models, such as logistic regression weights (Cheng et al., 2006) or the mean decrease in impurity (Gini index) in random forests (Sylvester et al., 2018), selecting those with the highest scores.
2. Related Work
Distribution Shift Detection. The detection of distribu- tion shifts consists of predicting if p + q. where p and q are the reference and query distributions, respectively. Nu- merous techniques exist for detecting distribution shifts in univariate distributions (Gama et al., 2014; Lu et al., 2018; Pan et al., 2020), and there is a growing focus on multivari- ate data (Rabanser et al., 2019), which can exhibit various types of shifts such as marginal, concept, covariate, or la- bel shifts (Lu et al., 2016; Losing et al., 2016; Liu et al., 2020). Recent shift detection techniques include (Yu et al., 2018), which makes use of hypothesis testing for concept drift detection, and (Rabanser et al., 2019), which applies two-sample multivariate hypothesis testing via Maximum- Mean Discrepancy (MMD) (Gretton et al., 2012), univariate hypothesis tests with marginal Ballmogroov-mmmmo (KS) tests, and dimensionality reduction techniques.
Optimal Transport. Optimal transport (OT) theory com- pares probability distributions by computing the minimal cost required to transform one into another, inducing a mean- ingful distance that reflects both global structure and the geometry of the underlying space. Its formulation as a lin- ear programming problem (Kantorovitch, 1958) connected OT to the broader field of optimization (Quanrud, 2018). Its relevance has since expanded across fields, including
Feature Shift Localization. While distribution shift de- tection techniques focus on detecting whether shift exists
2

---

Feature Shift Localization Network
computer vision (Izquierdo & Civera, 2024), economies (Galichon, 2018), logistics (Nadal-Roig & Pla-Aragonés, 2015), and statistical inference (Goldfeld et al., 2024). Re- cent advances in scalable numerical solvers, such as entropic regularization and Sinkhorn iterations, have enabled OT to scale to high-dimensional settings and find applications in data science (Peyré et al., 2019; Montesuma et al., 2024), generative modeling (Sanjabi et al., 2018), and domain adap- tation (Courty al., 2016). However, while OT techniques can characterize divergences, they do not provide a direct methodology to localize divergent features, which is the main focus of this paper, and would require modifications in order to be applied for the feature shift localization task.
3. Feature Shift Localization Network
Problem Formulation. We follow a similar problem for- mulation as described in (Kulinski et al., 2020; Barrabés et al., 2023), with minor adaptations:
Definition 1. [Empirical Feature Shift Localization Task] We are given two sets of d-dimensional samples x= {I1, 22,.,, and Y = {311 V2...., yM) from dis- tributions p and 9 respectively, with Iz, Ys E R |X = N, and |Y] = M. The feature shift localization task con- sists of predicting the subset of shifted features C from the input data X and Y using a mapping F, such that C= FYX,Y), that satisfies D(pT. = -0, D(p. q) > O. and C = arg min D(ve-9)==0 |C), where D is valid statis- tical distance.
Data-centric AI. Data-centric AI (DCAT) is the paradigm that encapsulates tools and techniques aimed at improving data quality and quantity build robust, accurate, and effi- cient AI systems. Unlike model-centric approaches, which prioritize refining models while working with fixed dataset, DCAI emphasizes improving datasets through systematic and iterative processes. Key aspects include expanding datasets through data collection (Ghosh Kaviraj, 2023),
A feature shift between d- dimeennsional distributions p and q is present if after removing the "corrupted" dimensions C and keeping only the "non-corruptdd dimensions 5 C, with d |S| + [C], the divergence between the restricted distributions is D(ps, qs) = O. The number of corrupted features |C] is assumed to be unknown. We refer to X and Y as the "reference" and "query" datasets, and to and as the "reference" and "query" distributions, respectively. In practice, p and are unknown and only accessible through the samples X and Y. requiring the task to be approximated by method F that maps the input data the predicted set of corrupted features: Ö = F(X, Y). Such method F can have the form of machine learning- based hypothesis testing (Kulinski et al., 2020), iterative heuristic algorithms as in DataFix (Barrabés et al., 2023), or the end-to-end trained parametric neural network introduced in this paper. Typ- ically, F is designed to predict set Õ' that is as close as possible to the true set of corrupted features C. This set can be represented either a collection of positional in- dices or as d dd-iimensional Boolean vector C' E {0. 1)4, where each entry indicates whether a corresponding feature is corrupted or not. Note that the vector representation is equivariant to the feature ordering. In this paper, we use the set and vector notations interchangeably unless unclear from the context. We focus on scenarios where either no features or only subset exhibit a shift. If all features expe- rienced shift, it would be impossible to determine whether the differences arise naturally between the reference and the query. Thus, we base our approach on the assumption that the true, unmodified query originates from the same distribution as the reference.
annotation (Boecking et al., 2020; Caporali et al., 2023),
augmentation (Montserrat et al., 2017; Geleta al., 2023),
and integration (Xiaojuan & Yu, 2023), as well as refining data through cleaning (Krishnan & Wu, 2019; Costanzo, 2023; Barrabés et al., 2024) and feature engineering (Sinaci et al., 2023; Buckley et al., 2023). The increasing com- plexity and scale of datasets have made automated methods indispensable for data refinement. growing focus within DCAI is the localization of feature shifts (Zha et al., 2023; Barrabés et al., 2023).
Deep Sets, Equivariant, and Invariant Networks. In- variance and equivariance properties are important in many applications and have proved successful in modeling physics and chemical systems, with numerous neural networks de- veloped to have such properties (Benton et al.. 2020; Batzner et al., 2022; Ruhe et al., 2024). Graph neural networks and attention-based networks have been shown to provide sim- ilar properties (Lim & Nelson, 2022). Deep Sets (Zaheer et al., 2017) introduced an architecture for modeling set- structured data, ensuring invariance on the order of samples. Our proposed FSL-Net adopts similar design choices as Deep Sets to obtain sample-order invariance and approxi- mate feature-order equivariance.
Section A provides a more detailed description of the bench- marking methods evaluated in this paper, including DataFix, MB-SM, MB-KS, KNN-KS, and Deep-SM, as well as MI, SelectKBest, MRMR, and F...
As described in (Barrabés et al., 2023), the presented defi- nition of feature shifts covers a wide range of distribution shifts: marginal shifts with D(p,, q:) > O, where p; and qi represent the marginal distribution of the ith dimension; cor- relation shifts with D(p, g) > 0 and D(p,, q:) = 0 for all i, where marginal distributions match but the multi-variate distributions do not; and similarly, correlation shifts with
3

---

Feature Shift Localization Network
Moment Extraction Net
Statistical Measures
Conv
Noan
90
BN
MAD
pordar Morert
Activation
Reference X
Query
Heetgrom
Emprica
N
Normalization
pgg ds(py)
Mcan-pooling
Residual Block
Statistical Descriptor Net OB
Neural Embedding Net
tz
.xL
Statistical Measures
Sample-wise
Conv match
Mr (PNN
Moment Extraction
Concat
Nouna Embedding Net
Prediction Net
Conv
Conv
Combination
1s
V
BN
t1 MS
Activation
d
Residual Block
Conv
BN
Prediction Net
BN
710
M
Residual Block
d
Yo Pby
M
4
Conv
Binarization
Mcan-pooling
Mcan-pooling
Sigmoid
= (t: P(EE))
d
(p(P
(u(gu)
Figure 1. Diagram illustrating the FSL-Net architecture.
D(ps qs) = 0 and D(pc, (c) = 0 but D(p, 9) > 0, where correlations are maintained locally, but shift is present when considering C' and S simultaneously. Note that this framework can also model label shifts in regression or clas- sification tasks by simply considering the label as addi- tional dimension of p and q.
erates finiteddimensional vector summarizing the input dis- tribution AP (e(p) (and equivalently for q), and the Pre- diction Network 77o which takes both vectors and predicts the corruption probability for each feature: p = "Te (pp(p. Mg)). The network is designed generalize well across datasets of different feature dimensionalities and sample sizes, and by using convolutions, it can scale to high nigh-ddmensional and large datasets without requiring re-training.
Feature Shift Localization Network Overview. The pro- posed Feature Shift Localization Network (FSL-Net) is a model trained end-to-end to predict the set of corrupted features: Ć = (X, Y). FSL-Net infers the probabil- ity each feature being part of C; that is, the network takes as input the reference and query datasets and predicts d di-imensional vector of probabilities, P who (X, Y), such that the ith dimension indicates the probability of the ith feature being corrupted P(i E C)= P: tho Y):. The complete predicted set of corrupted features C can then be obtained by selecting all the features with a probability higher than 0.5: Ć {i P(i E C*) > 0.5).
Statistical Functionals and Statistical Functional Maps. A statistical functional is a mapping T that takes as in- put a cumulative density function (CDF) P (or similarly pdf) of a distribution p and outputs scalar or vector μ = T(P) T(p). Some examples of statistical function- als include the mean, variance, mode, and histograms of the distribution. Because and g are unknown and only accessible through X and Y, we extract the statistical func- tionals from the empirical distributions p.v and gas. which are constructed by assigning equal probability mass to each of the N and M samples of X and Y, respectively. Sta- tistical functionals of interest include linear functionals of
The Feature Shift Localization Network has main sub- networks: the Statistical Descriptor Network dg. which gen-

---

Feature Shift Localization Network
the form μ A(p) J (x)) which can be ex- pressed as simple average for empirical distributions: JN A(p) = - - g(z;). Note that g(-) does not need to be a linear mapping and can be any (potentially non- linear) function, even a neural network. Empirical mean and histogram estimates are examples of linear functionals.
and query datasets. Namely, these measures include the mean (indicating the average value), standard deviation (indicating the spread), median (indicating the midpoint), mean absolute deviation (indicating the average distance from the mean), p p-order moments (capturing univariate higher-order characteristics), marginal histograms (approxi- mating marginal pdfs), and empirical marginal CDFs, Table presents the formulas for each measure. Note that except for histograms, empirical CDFs, and p p-order moments, all measures can be represented with arrays of dimensionality d x 1, while the histograms and empirical CDFs are repre- sented in tensors of size d X th. where fa. is the number of bins or powers p used. By concatenating all the measures, we obtain a statistical functional map μag E Rdxt,
In this paper. we extend the concept of statistical functionals to statistical functional maps mappings T that project a d- dimensooional multivariate CDF P (or equivalently a pdf p) into a d x l tensor μ E Rdxt, where the ith and kth component jiik = (P) is obtained by applying the mapping T to the multivariate distribution P while using the positional information (i.e., dimension index). An ex- ample of a statistical functional map is tensor representing a histogram of bins for each of the d marginal distribu- tions, which can be computed as H4, N Σ}= 1 (z),s), with 1 S h < and S i S d. where zJ, is the ith di- mension of the jth sample, and 1s, (1r) equals 1 if x is in the interval by defining the kthh bin, and 0 otherwise. Other examples include tensors capturing the first t moments of the d marginal distributions or the d x covariance matrix " = Cov,(i, k) of distribution p, with d. Statistical functional maps, indexed by dimension provide finite- dimensional summaries of multivariate distributions, where each ith component captures the statistical properties of the ith dimension of the distribution and its interactions with other dimensions.
Table I. Statistical measures the Statistical Descriptor Network. N is the number of observations in X, 2i denotes the ith dimen- sion of the jth sample (subscript i omitted for brevity), e a small positive constant for numerical stability, be and c2 are his- togram intervals and CDF thresholds, respectively, and x and a are the empirical mean and standard deviation of the ith dimension.
Siutistical Measure
Linear
Equation P5,k
Mcan
Yes
N
Standard Deviation
No
R Ey
(ay 2)²
Median
No
Mean Absolute Deviation
Yos
12g -x]
p. p-order Moments
Yes
x
Histogram
Yos
1(25 E (b.)
Statistical Descriptor Network. The Statistical Descriptor Network is the first component of Fet-Net. This network ex- tracts multiple statistical functional maps from the reference and query datasets, which are then fed into the Prediction Network to localize potential shifts. The Statistical Descrip- tor Network extracts three statistical functional maps: (1) a non- non-parametric map μ = (pxx)) with μuş E IRdx;, con- sisting of simple statistical measures such as marginal means and histograms; (2) a map predicted by a parametric shallow network named Moment Extraction Network μ4, e- (PN),, ). with μ4, E dx designed to extract second and higher moments of the data; and (3) final map extracted by a parametric deep residual network named Neural Embedding Network t.e = (Px), ), with Aw E Raxts, designed to extract a richer representation of the dataset. The maps are concatenated, generating a unique map H, E gxxt describ- ing distribution p (and similarly for q). with Note that d varies with each input dataset, but t remains fixed across datasets and only depends on the network hy- perparameters. We use Ue = (pN) = [us; pri Hw] to denote the complete mapping of the three statistical func- tional maps for distribution pN-
Empirical CDF
Yes
Σm 1(2j S cx)
Moment Extraction Network, The previously described sta- tistical measures work well for capturing marginal distri- butions, but they fail capture correlations and higher- order relations between dimensions. While the covariance matrix could address this, it becomes impractical for high- dimensional datasets due to its quadratic growth with the number of features. In order to capture higher-order rela- tions between features, we make use of the Moment Extrac- tion Network f4, = o, = + M (Wx;+b). where (W, b) represent an affine mapping. and (.)+ the ReLU activation function. In order to adapt to changing input di- mensionalities d, the affine mapping is parametrized using a convolutional layer and a batch normalization layer. By applying padding, the output of the network has dimension- ality d x 12, where 12 is the number of output channels of the convolution. The convolutional and ReLU layers are applied to each sample zy independently, and a sample-wise mean pooling operation is applied to obtain a dataset-level vector. Note that this network can be seen as approximating gener- alized moments of the data (Gretton et al., 2012; Li et al., 2015; Perera et al., 2022). Both the Moment Extraction Network and the Neural Embedding Network are trained jointly with the Prediction Network by using a cross-entropy loss and auxiliary loss (see the loss functions Section).
f + + la-
Statistical Measures. The first statistical functional map M4s (pN) contains various measures that capture key statistical properties of individual features in the reference
5

---

Feature Shift Localization Network
Neural Embedding Network. To complement the statistical measures and the Moment Extraction Network, we include convolutional deep residual network to predict linear sta- tistical functional map: A = (PN) = N Σソ= wa (Iz))
P= phooppNNqni = vo((X, () = 70(oo(pn)) (00(o())))o0((m)))) (o((M))
(1)
The Neural Embedding Network, as well as the Prediction Network detailed below, is built upon residual blocks. Each residual block consists of alternating ID-convolutional lay- ers across features, batch normalization (BN), and a hyper- bolic tangent activation function (Tanh). Skip connections are incorporated to facilitate efficient information flow and mitigate vanishing gradients. We conducted experiments with attention based Multi-Layer Perceptrons (MLPs) using unit- neit-kenel convolutions, but a fully convolutional design worked best (see Section D).
Loss Functions. FSL-Net is trained end-to-end using the binary cross-entropy loss between the predicted prob- abilities P and the ground truth corrupted feature set C: Pc - Σ C2 log(Ps) + (1 O+) log(1 Pa). We also add auxiliary loss function to the pre- dicted statistical functional maps to encourage the learn- ing of useful discriminative features and enforce local- ity: faux( (p: = I -love-Buelll" The loss is com- puted and averaged across multiple labeled datasets with simulated shifts, denoted as D = {(C(*), ,x(+), y(+))
The ith output of the statistical functional map predicted by the Neural Embedding Network has the form Mug
S 2 S (ND). resulting in the total loss func- tion: L(vo, D) = Exe lce (Ct+)) ()(()))) (())) + mux Mu (C(2), (x)(x),,, ()(())))) In practice, L(v9. D) is approximated by using mini-batches and gradient accumu- lation and optimized with the Adam optimizer.
(Way,+ + H wn(r))). where Waya is an affine transformation of the ith input feature, and wn (zz) de-
N -
notes the output of each of the H residual blocks, capturing non-linear relationships between features.
Sample- wise Invariance, Feature-wise Equivariance, and Locality. Ensuring feature equivariance and sample invari- ance can help neural networks generalize across datasets with varying numbers of features and samples; (a) Sample- wise invariance: The mean pooling operation used in the lin- ear functionals, Neural Embedding, and Moment Extraction Networks provides maps with shapes that are independent of dataset size, allowing FSL-Net to generalize across datasets of different dimensions. Additionally, the non-linear statisti- cal functionals are computed with the samples sorted from smallest to largest, making them invariant to sample order; (b) Feature-wise invariance: The statistical measures are applied to the marginal distributions, making them equiv- ariant to feature ordering. Convolutions, however, are not typically feature-wisd equivariant. Instead, FSL-Net ap- proximates feature invariance by shuffling features in each training mini-batch, enforcing learned representations to be invariant to feature order; (c) Locality: We enforce that the ith dimension of the statistical functional maps primarily captures the statistical properties of the ith input feature by (1) using marginal distributionnbased statistical measures, (2) incorporating a residual connection in the Neural Em- bedding Network, and (3) applying the auxiliary loss to the statistical functional maps.
The complete Statistical Descriptor Network is applied to X and Y, yielding Py Φ do (X) and My Φ0(gw)) = (Y). Note that (o(-) is shared between the reference and the query. These descriptors are then fed into the Prediction Network.
Prediction Network. The Prediction Network no com- bines the statistical functional maps M4 = (pxx and Me = de (qM) to predict a vector of probabilities P = To(ppp indicating the likelihood of each feature be- longing to the corrupted set C. These maps are combined through an operation (-)), producing a joint statistical map Pp.4- = a(jp; pa)). After evaluating various merging opera- tions (see Section E.3), we selected the normalized squared difference: a<ppp 14)) [ام_ where is a small posi- T|,,|+e
tive constant for numerical stability. This approach enables the network to compare statistical maps in manner that accounts for their relative magnitudes. The resulting joint representation has dimensions Hp.9 E Rdxt, The Predic- tion Network then applies multiple residual blocks to the joint map Hp.g and produces the final probability estimates through a sigmoid activation layer. Note that, by employing the squared difference, the Prediction Network observes a difference between statistical maps resembling the Max- imum Mean Discrepancy (MMD) metric (Gretton et al., 2012; Li et al., 2015) and acts as a mapping from a distance between distributions into shift probability estimates.
Training and Validation Datasets. We source a total of 1,032 diverse tabular datasets from OpenML (Van Rijn et al., 2013), with 10-28k features and 500-3.6M samples, cov- ering continuous, categorical, and mixed data types. Addi- tionally, we generate 368 simulated datasets: 184 based on probabilistic distributions (Gaussian, Bernoulli, and Beta mixtures) and 184 from algebraic functions (Polynomial, Sine, and Logarithmic). each with 5,000 samples and 1,000 features. In total, 1,350 datasets are used for training, with
The complete structure of the feature shift localization net- work has the following form:
6

---

Feature Shift Localization Network
Table Manipulation types applied to continuous and/or categorical features during training and validation.
Type
Mapping
Description
Shift
T1
8,2,
Each value is multiplied by a random number
Pi # 4-
Cont.
Uniform(.
between and I.
T2
(
2) + 9,)a
Each value is replaced by a lincar combination of x and negation.
Pi 1f
Cont.
Uniform(0,) 1)
T3
The ath feature of Y is replaced by sampling from
Ps
of qo-
Both
ref. empirical marginal distribution PN,
42 = Thee 9:
T4
clampo,1 (ar
Add Gaussian noise with Uni (-0.2, 0.2) and a ~ Uni (0.001,
P: + 4. Rlgil Rs Efpa] +
Cont.
Normal(a, o) CNN(2)
T5
Forward through CNN with min-max normal- ization or binarization.
Pi #
Both
T6
FC PNC
Similar manipulation (c), bui all the features within are sampled simu Innenusyyy from PNN et Predict feature with KNN (Regressor).
Pr = PG gc.
Both
T7
KNN(#)
Cont.
K= (1-1,7-9)) (NN(F)
T8
Predict feature with KNN (Classiier).
Cat.
k= (1-1,79)
50 reserved for validation. Section describes in detail the dataset selection, preprocessing. and simulation procedures.
Table Datasets used during benchmarking.
Dataset Type
Datasct
# Features
Samples
Training and Validation Manipulations Simulations. During training, datasets are shuffled in sample and fea- ture order, and random subsets of samples (from 1,000 to 10,000) and features (from 8 to 256) are selected, with each feature normalized to a range of to Each subset is then split equally into reference and query samples. In the query set, a random subset of features (up to 25%) is ma- nipulated based on feature type (continuous categorical). Validation batches follow the same process but are limited to 2,048 features. Manipulations, outlined in Table 2 (TI to T8), are selected with probabilities inversely proportional to the validation F-1 score performance observed during training for each given type. Multiple manipulations are used to simulate wide range of feature shifts. Note that the manipulations applied during training and validation differ from those applied to the test set.
Gas
8
12,815
Covid
9,889
Energy
26
19,735
Musk2
166
6.59%
Continuous
Scene
294
2,407
MNIST
784
70,000
Polynomial
1,000
10,000
Cosine
I 1,000
10,000
Dilbert
2,000
10,000
Phenotypes
1,227
31,424
Categorical
Founders
10,000
4,144
Canine
198,473
1.444
those in (Kulinski et al., 2020). The benchmark datasets vary in data type, feature dimensionality (from to 198,473), and sample size (from 1,444 70,000). Table provides an overview of each dataset, detailing its data type (continu- ous categorical), sample size, and feature dimensionality. The continuous datasets are sourced from the UCI repository (Gas (Huerta et al., 2016), Energy (Candanedo et al., 2017), and Musk2 (Blake, 1998)) and OpenML (Scene (Boutell et al., 2004), MNIST (Deng, 2012), and Dilbert (Vanschoren et al., 2014)). Additionally, Covid-19 dataset (Force, 2022) and two simulated datasets generated from algebraic functions that differ from those used in our training and validation simulations (Cosine and Polynomial) (Barrabés et al., 2023) are included. The categorical datasets consist of high- -igh-dimensional biomedical data, including the Phe- notypes dataset (Qian et al., 2020), a subset of categorical traits from the UK Biobank, the Founders dataset containing binary-codee human DNA sequences (Perera et al., 2022), and the Canine dataset comprising binary-coded dog DNA sequences (Barrabés et al., 2023). Each dataset is normal- ized on per-feature basis to range of O to 1, and the samples are evenly divided into two subsets, forming the reference and query sets. As in (Barrabés et al., 2023). a random fraction of query features (5%, 10%, or 25%) un- dergoes one of 10 manipulation types for continuous data or
4. Experimental Results
Evaluation Setup. Our evaluation setup is consistent with (Barrabés et al., 2023), using the same reference and query sets and optimized benchmarking methods. Hyperparam- eter tuning for FSL-Net is detailed in Section E. We com- pare FSL-Net against five feature shift localization methods (DataFix, MB-SM, MB-KS, KNN-KS, and Deep-SM) and four feature selection methods (MI, SelectKBest, MRMR, and Fast-CMIM). The SelectKBest method employs the Chi- square test for categorical datasets and the ANOVA-F test for continuous datasets. For MB-SM, MB-KS, KNN-KS, and Deep-SM, the number of manipulated features |C] is provided, while the other methods, including FSL.-Net, do not require it, reflecting more realistic setting.
Evaluation Data. We use the same evaluation manipula- tions and evaluation datasets as in (Barrabés et al., 2023), with the Gas, Covid, and Energy datasets also aligning with

---

Feature Shift Localization Network
0.9
FSL Net
0.9
DataFix
0.9
PSNNe:
DataFix
DataR
0.7
0.7
0.7
KNN-KS
MI
KNN-KS
F1 ScScree
Deep C00--6
SM
0.6
Deep-SM
SelectKbast
selee
0.4
0.4
FASTICHIN
MB-KS
MB KS
MB- HB-KS
MRMR
0.2
0.2
FAST-CMIM
0.2
FAST-CMIM
MRM
0.0
2.5
5.0
7.5
10.0
12.5
15.0
10
15
20
25
Mean Runtime (hours)
Runtime (hours)
E4.2
E4.1
Musk2
10
Scene
Energy
E4.3
E3
MNIST
Covid
E5
E2
Runtime
10
E6.1
0.0
0.2
0.4
0.6
0,8
1.0
E1Paly
0.0
0.2
0.4
0.6
1.0
Gas
E6.2
E10
Cosine
Canine
10
10
energh 4.
maxz L.0
Ae
E7
Dilbert
Phenotypes Dataset
Founders
scons
alberr
phanetycck
Manipulation Type
codden
FSL-Net
MRMR FAST.CMIM
MB-SM
KNN-KS
Deep
Figure Performance and runtime comparison across feature shift localization methods: a) mean F-1 scores across manipulation types, fractions of manipulated features, and datasets; b) mean F-1 scores vs. mean runtime; c) mean F-1 scores vs. maximum runtime; d) mean runtime vs, sample-feature size product per dataset; e) mean F-1 scores by manipulation type; f) mean scores by dataset.
8 for categorical data (referred to as manipulations EI-E10). This process generates 30 query sets for continuous data and 24 categorical data, each with unique manipulation applied to given feature subset. Section C details the eval- uation manipulation types. Note that both testing datasets and manipulations differ from those used during FSL-Net training and validation (see Table 2).
Feature Shift Localization Performance and Runtime Comparison. Figure presents the performance and runtime comparison across different methods. Figure 2a presents the average F-1 scores of feature shift localization computed across manipulation types, fractions of manip- ulated features, and datasets. Figure 2b depicts the mean F- scores against the mean runtime in hours, while Figure 2c illustrates the F-1 scores against the maximum runtime, highlighting the worst-case computational demands. Higher positions in these plots indicate better performance, whereas leftward positions indicate lower runtimes, emphasizing the balance between effectiveness and efficiency. SelectKbest and MI, despite their efficiency, exhibit poor performance in feature shift localization. Methods such MRMR and FAST-CMIM struggle with scalability and display limited localization capabilities. Notably, FSL-Net achieves the highest average F-1 score, with feature shift localization per- formance comparable that of DataFix but at significantly
Evaluation Protocol and Hardware Specifications. Per- formance is evaluated using the F-1 score for feature shift localization accuracy and wall-clock runtime for computa- tional efficiency. Each experiment is for up 30 hours, which prevents some methods from completing evaluations on large datasets. To handle incomplete evaluations, pute missing results with the lowest F-1 score from the same experiment among competing methods and assign them a 30-hour limit. All evaluations were conducted on an Intel Xeon Gold with CPU cores.
im-
8

---

Feature Shift Localization Network
faster speeds. Specifically, FSL-Net is approximately 36x faster on average than DataFix, and up to 136x on the high- dimensional Phenotypes dataset. Additionally, FSL-Net significantly outperforms MB-SM, MB-KS, KNN-KS, and Deep-SM, even though these methods have the advantage of accessing the ground truth |C). while also demonstrating substantially faster computational speeds. While Datalix relies on a computationally intensive iterative optimization heuristic, and MB-SM, MB-KS, KNN-KS, and Deep-SM require sataset- training. FSL-Net only requires per- forming a forward pass through the network for each dataset, providing highly fast inference. Figure 2d shows the mean runtime as a function of the product between the sample and feature sizes for each dataset, confirming that FSL-Net con- sistently outperforms DataFix in speed across all datasets, ranking as the second fastest method after selectKBest, com- parable MI, but with much higher accuracy. Namely, DataFix requires up to 13 hours on the high-dimensional Phenotypes dataset from the UK Biobank and 9 hours on the Canine dataset, whereas FSL-Net completes these tasks in just 2 minutes and 12 minutes, respectively. These results highlight the practical advantage of FSL--Net: is well- suited for processing high-dimennionall large databases, such as those commonly found in biomedicine and e-commerce, making it a valuable alternative to DataFix due to its efficient scalability and excellent localization accuracy.
only baseline, where predictions are obtained by directly thresholding the statistical measures. Table 4 presents the mean F-1 score achieved by each configuration. Training lasted three days for all variants, except for the full model, which was trained for an extended period of seven days. Us- ing the Prediction Network in the SM-only baseline greatly improves feature shift localization performance compared to simply thresholding the statistical measures (from an F-1 score of 0.307 0.710), emphasizing the crucial importance of the Prediction Network. Each component contributes to performance improvements, suggesting that each compo- nent has unique value, with extended training resulting in additional gains.
Table 4. Mean F-1 scores for various configurations of FSL-Net.
Training Duration
Model Configuration
F- Score
SM
0.307
SM+ PN
0,710
ME+PN
0.742
3-Day
NE+ PN
0.783
ME+ NE
0.770
SM MT+ME + PN
0,855
SM NE + PN
0,878
SM MF NE PN
0.589
7-Day
SM ME PN
0.894
Performance across Manipulation Types and Datasets. Figure 2e and Figure 2f show the mean F-1 scores cate- gorized by type of manipulation and dataset, respectively. FSI.-Net consistently matches or exceeds the performance of competing methods across all manipulation types, ex- cept for E9 and E4, where DataFix exhibits superior perfor- mance. These manipulations introduce only minimal pertur- bations that FSL-Neet fails to localize, though lower prob- ability thresholds may improve detection. DataFix fails to accurately detect E10, shortcoming effectively addressed by FSL-Net. Methods relying on univariate tests, such as MRMR and Fast-CMIM, perform well for manipulations causing marginal distribution shifts but fail entirely with manipulations affecting feature correlations (manipulations E3 and E8). In contrast, techniques based on conditional testing (MB-SM, MB-KS, KNN-KS, and Deep-SM), along with DataFix and FSL.-Net, successfully identify these more complex manipulations. FSL-Net shows clear advantage in high-dimensiooal datasets (Phenotypes, Founders, and Canine), outperforming all competing methods and high- lighting its effectiveness in handling large feature sets.
Additional experimental results are presented in Section F, including: (1) median-based evaluation of feature shift localization performance and runtime across different meth- ods; (2) a detailed performance and efficiency comparison between FSL-Net and DataFix; (3) extended evaluations of FSL-Net and DataFix on nigh-dimensional image datasets (CIFARIO and COIL-100); (4) an analysis of FSL-Netts runtime improvement over DataFix; (5) an evaluation of the threshold- based variant of the SM-only baseline; and (6) qualitative evaluation of FSL-Net on the MNIST dataset.
5. Conclusions
Current feature shift localization methods involve a trade- off between speed and accuracy when dealing with large data volumes. In this work, we introduced FSL-Net, a novel equivariant neural network that matches or surpasses ex- isting taite-of-th--att methods while offering a significant reduction in processing times. FSL-Net leverages neural- learned statistical descriptors, augmented with traditional statistical measures, to effectively capture the input distribu- tions. By contrasting these descriptors across different data sources, the network accurately detects both univariate and multivariate shifts. Most importantly, FSL-Net is designed to manage datasets of varying sizes and to generalize across new distributions without the need for model re- re-training with each new estimation, providing significant speed and scalability advantages over competing methods.
Ablation Analysis. We assess the impact of each compo- nent of FSL-Net's Statistical Descriptor Network by training models with different combinations of its three components: Statistical Measures (SM), Moment Extraction Network (ME), and Neural Embedding Network (NE). All variants include the Prediction Network (PN), except for the SM-

---

Feature Shift Localization Network
Impact Statement
Boecking, B., Neiswanger, W., Xing, E., and Dubrawski, A. Interactive weak supervision: Learning useful heuristics for data labeling, arXiv preprint arXiv: 2020.
This paper presents work whose goal is to advance the field of Machine Learning. There are many potential societal consequences of our work, none which we feel must be specifically highlighted here.
Boutell, M. R., Luo, J., Shen, X., and Brown, C. M. Learn- ing multi-label scene classification. Pattern recognition, 177))115771171, 2004.
References
Buckley, T., Ghosh, B., and Pakrashi, V. A feature extraction & selection benchmark for structural health monitoring. Structural Health Monitoring, 22(3):2082-2127) 2023.
Bahassine, S., Madani, A., Al-Sarem, M., and Kissi, M. Fea- ture selection using an improved chi-square for arabic text classification. Journal of King Saud University- and Information Sciences, 32(2):225-231, 2020.
Candanedo, L. M., Feldheim, V., and Deramaix, D. Data driven prediction models of energy use of appliances in a low-energy house. Energy and buildings, 140:81-97, 2017.
Barchard, K. A, and Pace, L. A. Preventing human error: The impact of data entry methods on data accuracy and statistical results. Computers in Human Behavior, 27(5):
1834-1839, 2011.
Caporali, A., Pantano, Janisch, L.., Regulin, D., Palli, G., and Lec, D. A weakly supervised semi-automatic image labeling approach for deformable linear objects. IEEE Robotics and Automation Letters, 8(2):1013-1020 2023,
Barrabés, M., Bonet, D., Moriano, V. N., Giró-i Nieto, X., Montserrat, D. M., and Ioannidis, A. G. Genomic databases homogenization with machine learning. In 2023 IEEE International Conference on Bioinformatics and Biomedicine (BIBM), pp. 2959. IEEE, 2023.
Cheng, Q.. Varshney, P. K., and Arora, M. K. Logistic regression for feature selection and soft classification of remote sensing data. IEEE Geoscience and Remote Sensing Letters, 34444491-494, 2006.
Barrabés, M., Mas Montserrat, D., Geleta, M., Giró-i Nieto, X., and Ioannidis, A. Adversarial learning for feature shift detection and correction. In Oh, A., Naumann, T., Globerson, A., Saenko, K., Hardt, M., and Levine, S. (eds.). Advances in Neural Information Processing Sys- tems, volume 36, pp. 57597-57638. Curran Associates, Inc., 2023.
Costanzo, L. Data cleaning during the research data man- agement process. Research Data Management in the Canadian Context, 2023.
Courty, N., Flamary, R., Tuia, D., and Rakotomamoniy, A.
Optimal transport for domain adaptation. IEEE transac- tions on pattern analysis and machine intelligence, 39(9): 1865, 2016.
Barrabés, M., Perera, M., Moriano, V. N., Giró-I-Nieto, X., Montserrat, D. M., and loannidis, A. G. Advances in biomedical missing data imputation: A survey. IEEE Access, pp. 1-1, 2024. 3516506.
Dai, T., Hu, H., Wan, Y., Chen, Q., and Wang, Y. A data quality management and control framework and model for health decision support. In 2015 12th International Conference on Fuzzy Systems and Knowledge Discovery (FSKD). pp. 1792-1796, 2015, doi: 10.1109/FSK..2015 7382218.
doi: 10.110//
Battiti, R. Using mutual information for selecting features in supervised neural net learning. IEEE Transactions on neural networks, 5(4):537-550, 1994.
Batzner, S., Musaclian, A., Sun, L., Geiger, M., Mailoa, J. P., Kornbluth, M. Molinari, N., Smidt, T. E., and Kozinsky, B. E (3)-equivariant graph neural networks for data-efficient and accurate interatomic potentials. Nature communications, 13(1):2453, 2022.
Deng, L. The mnist database of handwritten digit images for machine learning research. JEEE Signal Processing Magazine, 29(6): 141-142, 2012.
Ding, C. and Peng. H. Minimum redundancy feature se-
lection from microarray gene expression data. Journal of bioinformatics and computational biology, 3(02):185- 205, 2005.
Benton, G., Finzi, M., Izmailov, P., and Wilson, A. G. Learn- ing invariances in neural networks from training data. Advances in neural information processing systems, 33: 17605-17616, 2020.
Elssied, N. O. F., Ibrahim, O., and Osman, A. H. A novel feature selection based on one-way anova f-test for e- mail spam classification. Research Journal of Applied Sciences, Engineering and Technology, 7(3):25 -38, 2014.
Blake, C. Uci repository of machine learning databases. http:://ww. ics. uci. edu/" mlearn/MLRepository. hmml, 1998.
10

---

Feature Shift Localization Network
Fleuret, F. Fast binary feature selection with conditional mu- tual information. Journal of Machine learning research, 5(9), 2004.
Kantorovitch, L. On the translocation of masses. Manage- ment science, 5(1):1-4, 1958.
Krishman, S. and Wu, E. A. Automatic generation of data cleaning pipelines. arXiv preprint 19X1:1900..1187,, 2019.
Force, C. C. T. United states covid-19 19 cases and deaths by state over time. 2022.
Galichon, A. Optimal transport methods in economics. Princeton University Press, 2018.
Kulinski, S., Bagchi, S., and Inouye, D. Feature shift detection: Localizing which features have shifted via conditional distribution tests. Advances in Neural Infor- mation Processing Systems, 11111922-195333 2020.
Gama, J., Žliobaité, I., Bifet, A., Pechenizkiy, M., and Bouchachia, A. A survey on concept drift adaptation. ACM computing surveys (CSUR), 46(4):1-37, 2014.
Li, J., Cheng, K., Wang. S., Morstatter, F., Trevino, R. P., Tang, J., and Liu, H. Feature selection: data perspective. ACM Computing Surveys (CSUR), 50(6):94, 2018.
Geleta, M., Mas Montserrat, D., Giro-i Nieto, X., and loan- nidis, A. G. Deep variational autoencoders for population genetics. biorxiv, pp. 2023-09, 2023.
Li, Y., Swersky, K., and Zemel, R. Generative moment matching networks. International conference on ma- chine learning, pp. 1718-1727. PMLR, 2015.
Ghosh, E. and Kaviraj, T. Data collection: Best practices and techniques of footprint collection for newborn identi- fication. International Journal of Creative Research and Thoughts (IJCRT), 11(9), 2023.
Lim, L.-H. and Nelson, B. What an equivariant neural network. arXiv preprint arXiv:2205.07362. pp. 5987- 6001, 2022.
Ginart, T., Zhang, M. J., and Zou, J. Midemon: Deployment monitoring for machine learning systems. In Interna- tional Conference on Artificial Intelligence and Statistics, pp. 3962999999999 3997. PMLR, 2022.
Lim, S. B., Tan, S.J., Lim, W.-., and Lim, C. T. A merged lung cancer transcriptome dataset for clinical predictive modeling. Scientific data, 5(1):1-8, 2018.
Liu, A., Lu, J., and Zhang, G. Diverse instance-weighting ensemble based on region drift disagreement for concept drift adaptation. IEEE transactions on neural networks and learning systems, 32(1):293--30 2020.
Goldfeld, 7.,, Kato, K., Rioux, G., and Sadhu, R. Statistical inference with regularized optimal transport. Informa- tion and Inference: A Journal of the IMA, 13())iaad0056, 2024.
Liu, L., Liu, J., and Han, J. Multi-head or single-head? an empirical comparison for transformer training. arXiv preprint arXiv:2 106. /06.09650, 2021.
Gretton, A., Borgwardt, K. M., Rasch, M. J., Schölkopf, B., and Smola, A. A kernel two-sample test. The Journal of Machine Learning Research, 13(1):723-773, 2012.
Losing, V., Hammer, B., and Wersing, H. Knn classifier with self adjusting memory for heterogeneous concept drift. 2016 IEEE 16th international conference on data mining (ICDM), pp. 291-300 IEEE, 2016.
Hopf. K. and Reifenrath, S. Filter methods for feature selection in supervised machine learning applications- review and benchmark. arXiv preprint 1rX1:2211.112140, 2021.
Lu, J., Liu, A., Dong, F., Gu, F,, Gama, J., and Zhang, G. Learning under concept drift: A review. IEEE transac- tions on knowledge and data engineering, 31(12):2346- 2363, 2018
Huang, X., Zhang, L., Wang, B., Li, F., and Zhang, Z. Fea- ture clustering based support vector machine recursive feature elimination for gene selection. Applied Intelli- gence, 48(3):594 607, 2018
Lu, N., Lu, J., Zhang, G., and De Mantaras, R. L. A con- cept drift-tolerant case-base editing technique. Artificial Intelligence, 230:108-133: 133, 2016.
Huerta, R., Mosqueiro, T., Fonollosa, J., Rulkov, N. F., and Rodriguez-Lujan, Online decorrelation humidity and temperature in chemical sensors for continuous monitor- ing. Chemometrics and Intelligent Laboratory Systems, 157:169-176, 2016.
Maldonado, S. and Weber, R. A wrapper method for feature selection using support vector machines. Information Sciences, 179(13):2008-2211 2009.
Izquierdo. S. and Civera, J. Optimal transport aggregation for visual place recognition. In Proceedings of the ieee/cvf conference on computer vision and pattern recognition, pp. 17658-17666 2024.
Montesuma, E. F., Mboula, F. M. N., and Souloumiac, A. Recent advances in optimal transport for machine learn- ing. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2024.

---

Feature Shift Localization Network
Montserrat, D. M., Lin, Q. Allebach, J., and Delp, E. J. Training object detection and recognition enn models using data augmentation. Electronic Imaging, 2017(10); 27-36, 2017.
detection and diagnosis in industrial processes. Chemo- metrics and Intelligent Laboratory Systems, pp. 104711, 2022.
Quanrud, K. Approximating optimal transport with linear programs. arXiv preprint
Moreno-Gre Grau, S., Vernekar, M., Lopez-Pinedte A., Mas- Montserrat, D., Barrabés, M., Quinto-Cortés, C. D., Moatamed, B., Lee, M. T. M.. Yu, Z., Numakura, K., et al. Polygenic risk score portability for common dis- cases across genetically diverse populations. Human Genomics, 18(1):33, 2024.
2018.
Rabanser, S., Günnemann, S., and Lipton, Z. Failing loudly: An empirical study of methods for detecting dataset shift. Advances in Neural Information Processing Systems, 32, 2019.
Ruhe, D., Brandstetter, J., and Forré, Clifford group equiv- ariant neural networks. Advances in Neural Information Processing Systems, 36, 2024.
Mustageem, A., Anwar, S. M., Majid, M., and Khan, A. R. Wrapper method for feature selection to classify cardiac arrhythmia. In 2017 39th Annual International Confer- ence of the IEEE Engineering in Medicine and Biology Society (EMBC), pp. 3656-3659, IEEE, 2017.
Sanjabi, M., Ba, J., Razaviyayn, M., and Lee, D. On the convergence and robustness training gans with regular- ized optimal transport. Advances in Neural Information Processing Systems, 31, 2018.
Roia E. and Pla-Aragonés, L. M. Optimal trans- port planning for the supply to a fruit logistic centre. Handbook of operations research in agriculture and the agri-food industry, pp. 163-177, 2015.
Sinaci, A. A., Gencturk, M., Teoman, H. A., Laleci Er- turkmen, G. B., Alvarez-Romero, C., Martinez-Garvia, A., Poblador-Plou, B., Carmona-Pirez, J., Löbe, M., and Parra-Calderon, C. I.. A data transformation method- ology to create findable, accessible, interoperable, and reusable health data: software design, development, and evaluation study. Journal of medical Internet research, 25:e42822, 2023.
Nasir, I. M., Khan, M. A., Yasmin, M., Shah, J. H., Gabryel, M., Scherer, R., and Damaševičius, R. Pearson correlation-basse feature selection for document classi- fication using balanced training. Sensors, 20(23):6793, 2020.
Pan, J., Pham, V., Dorairaj, M., Chen, H., and Lee, J.-Y. Ad- versarial validation approach to concept drift problem in user targeting automation systems at uber. arXiv preprint arXiv:2004.03045, 2020.
Subasri, V., Krishnan, A., Dhalla, A., Pandya, D., Malkin, D., Razak, F., Verma, A., Goldenberg, A., and Dolatabadi, E. Diagnosing and remediating harmful data shifts for the responsible deployment of clinical ai models. medRxiv. pp. 2023 03, 2023.
Perera, M., Montserrat, D. M., Barrabés, M., Geleta, M., Giró-i Nieto, X., and Ioannidis, A. G. Generative mo- ment matching networks for genotype simulation. In 2022 44th Annual International Conference of the IEEE Engineering in Medicine & Biology Society (EMBC), pp- 1379-1383, IEEE, 2022.
Sylvester, E. V., Bentzen, P., Bradbury, I. R., Clément, M., Pearce, J., Horne, J., and Beiko, R. Applications of random forest feature selection for fine-scale genetic population assignment. Evolutionary applications, 11(2):
165-165,40 2018
Peyré, G. Cuturi, M., et al. Computational optimal trans- port: With applications to data science. Foundations and Trends@ in Machine Learning, 11(5-6):355-607, 2019.
Tran, B., Zhang, M., and Xue, B. A pso based hybrid feature selection algorithm for digeeniinnii classification. In 2016 IEEE congress evolutionary computation (CEC), pp. 3801-3808. IEEE, 2016.
Piano, I.,, Garcea, F., Gatteschi, V., Lamberti, F., and Morra, L. Detecting drift in deep learning: A methodology primer. IT Professional, 24(5):53-60, 2022.
Van Rijn, N., Bischl, B., Torgo, L.., Gao, B., Umaashankar, V., Fischer, S., Winter, P., Wiswedel, B., Berthold, M. R., and Vanschoren, J. Openml: A collaborative science platform. In Machine Learning and Knowledge Discov- ery in Databases: European Conference, ECML PKDD 2013, Prague, Czech Republic, September 23-27, 2013, Proceedings, Part III 13, pp. 645-649, Springer, 2013.
Qian, J. Tanigawa, Y., Du, W., Aguirre, M., Chang, C., Tibshirani, R., Rivas, M. A., and Hastie, T. A fast and scalable framework for large-scale and ultrahigh- dimensional sparse regression with application to the uk biobank. PLoS genetics, 16(10):e100911) 2020.
Vanschoren, J., Van Rijn, J. N., Bischl, B., and Torgo, L. Openml: networked science in machine learning. ACM SIGKDD Explorations Newsletter, 15(2):49-66) -60 2014.
Qian, J., Song, Z., Yao. Y., Zhu, Z., and Zhang, X. A review on autoencoder based representation learning for fault
12

---

Feature Shift Localization Network
Xiaojuan, L. and Yu, Z. data integration tool for the integrated modeling and analysis for east. Fusion Engi- neering and Design, 195: 13933, 2023.
Yu, S., Wang, X., and Príncipe, C. Sequest-and-reverifyy Hierarchical hypothesis testing for concept drift detection with expensive labels. arXiv preprint arXiv:1806. 10131, 2018.
Zaheer, M., Kottur, S., Ravanbakhsh, S., Poczos, B., Salakhutdinov, R. R., and Smola, A. J. Deep sets. Ad- vances in neural information processing systems, 30, 2017.
Zha, D., Bhat, Z. P., Lai, K.-H., Yang, F., Jiang, Z., Zhong, S., and Hu, X. Data-centric artificial intelligence: A survey. arXiv preprint arXiv:2303. 10158, 2023.
Zheng. L., Yuan, J., Wang, C., and Kong, L. Effi- cient attention via control variates. arXiv preprint 244::23302.04442,2222 2023.
13

---

Feature Shift Localization Network
A. Benchmarking Methods
Datalix detects and localizes feature shifts using an iterative adversarial approach called DF-Locate. At each iteration, a random forest classifier is trained distinguish between samples from a reference and query distribution. The predicted class probabilities from this discriminator are used compute the total variation distance (TVD), and its feature importance scores, based on the mean decrease of impurity, are used to locate the features originating the shift. These features are removed in successive rounds until the estimated divergence falls below a threshold or until half the features have been eliminated. A final refinement step uses knce-detection algorithm to choose the optimal stopping point in the removal process. The selection of features is controlled by dynamic threshold defined as the product of the TVD and a hyperparameter T.
Classical univariate statistical filters include Mutual Information statistics (MJ) and selectKbest. These methods rank features based on their statistical association with the target. MI measures entropy shared between each feature and the target, while SelectK Best applies ANOVA-F for continuous features and Chi-squared for categorical ones. ANOVA-F tests variance for mean differences in continuous features across target classes, while the Chi-square test evaluates categorical feature-target dependency by comparing observed frequencies across target classes.
More advanced iterative selection methods include Minimum Redundancy Maximum Relevance (MRMR) and Fast Conditional Mutual Information Maximization (FAST-CMIM). These methods attempt to balance feature relevance with redundancy by considering mutual information conditioned on already selected features. MRMR iteratively selects relevant features while minimizing redundancy with the selected features far. FAST-CMIM iteratively selects features that maximize MI, conditional to previously selected features. These methods, while more expressive than univariate tests, suffer from poor scalability and cannot process large datasets within practical time constraints.
Other benchmarked methods include model-based and statistical testing techniques specifically designed for feature shift localization. These include MB-SM (Multivariate Gaussian with diserr-divergence test), MB-KS (Multivariate Gaussian with KS test), KNN-KS (K-Nearest Neighbors with KS statistic), and Deep-SM (deep density neural models with Fisher- divergence test). These approaches operate under stronger assumptions about data structure and require prior knowledge of the number of corrupted features, which is rarely available in real-world applications.
B. Training and Validation Datasets
B.1. OpenML Datasets
We source a diverse collection of datasets from OpenMI. (Van Rijn et al., 2013), selecting only those in a tabular format with at least 10 features and minimum of 500 samples. To ensure data quality, we preprocess the datasets by removing constant features and addressing missing values using two strategies. In the first strategy, we remove features with more than 40% missing values, followed by the elimination of samples with missing data. In the second strategy, we remove features with more than 70% missing values before discarding samples with missing data. If both strategies preserve the required dimensions, we randomly select one for application. We also carefully remove any OpenML dataset that overlaps with the test datasets. For efficiency, large datasets exceeding 40M cell values are partitioned into ten equal-sized sets to accelerate processing. As a result, we obtain a total of 1,032 cleaned and partitioned datasets from OpenML..
B.2. Algebraic Simulated Datasets
We generate 184 simulated datasets using Polynomial, Sine, and Logarithmic functions, each containing 5,000 samples
and 1,000 features. Feature values from Polynomial functions are derived from f(x = az + hr3 + cx.2 + dr + e, with parameters a, b, c, d, and e uniformly sampled from [-50, 50], and the degree randomly set to either or 4. Sine functions follow f(x) = a cos(bæ + c), where parameter a is sampled from [-50, 50), and b and e are sampled from [- T). Logarithmic functions are defined as log, (z + 6), with base a sampled from [2, 10). The input values, æ, are drawn from the range [-5, 5, 5). Function parameters remain fixed across all samples.
B.3. Probabilistic Simulated Datasets
We generate 184 simulated datasets based on probabilistic distributions, including Gaussian, Bernoulli, and Beta mixture models. Each dataset consists of 5,000 samples and 1,000 features. To construct these datasets, we begin by selecting base distribution D from (Gaussian, Bernoulli, Beta} and initializing parameters for mixture model with a randomly determined
14

---

Feature Shift Localization Network
number of components K, where 1 S K 100. For the Gaussian distribution, the mean vector Mig (1k1, 142. ................................ (dd) drawn from standard normal distribution (O 1), while the variance vector of (o 0k2...... , is sampled from Uniform distribution U(0.1, 1.1) for each feature independently. For the Bernoulli distribution, the probability vector Ps: = (Pk1. Pk22, .kad)) which represents the probability of success for each feature, is sampled from uniform distribution (0, 1). For the Beta distribution, the parameters 0ş = (0&1: O122,..., and A = (Bk1, BA2...., .kd) are drawn from a uniform distribution (1 2). Samples are generated using the weighted sum these mixture components, defined as Σk=1 k=1 n& D(Ok), where represents the mixing coefficients, satisfying Σ Th = 1.
Each dataset has a 25% chance of undergoing multiple transformations, with the number transformation steps / randomly chosen between and 5. These transformations include normalization (standard or min-max scaling), followed by linear transformation in which the dataset multiplied by a randomly sampled matrix W RRdxd The entries of W are drawn from one of four distributions: (0, 1), (O, 1), Beta(1, 1), or Bernoull Additionally, non-linear transformation applied, randomly chosen from the ReLU, GELU, sigmoid, hyperbolic tangent, logarithmic transformation. To preserve structural information, residual connections are sometimes applied, where the transformed data is combined with its previous or initial state, either directly or after undergoing normalization.
15

---

Feature Shift Localization Network
C. Testing Manipulation
Table 5 outlines the manipulation types used to induce shifts during evaluation. Some distort marginal distributions (E1, E2, E4, E5, E6, and E7), with manipulation E4 leaving the mean approximately unchanged. Others shuffle feature values across samples, altering feature correlations but marginal distributions (E3 and E8). Manipulations E9 and E10 use KNN predictions to replace corrupted continuous and categorical features, respectively. For more detailed explanation, (Barrabés et al., 2023).
Table Manipulation types applied to continuous and/or categorical features during benchmarking (Table from (Barrabés et al., 2023)).
Type
Mapping
Description
Shift
Data
EI
Uniforn(O, 1)
Each value is substituted by random number between and
Cont.
Each value negated.
4:. Elg.] 1 B/p.]
Both
P2
P. is random permutation matrix applied to feature
Pi Pi 1 ac:
Both
9c IIee 9,
F..1.4.3
clampu. (x + oa)
Add constant noise with random sign.
E
0: E(p.) sz E(ga]
Cont.
Rademacher(O.55
(0.00 10,02, 0.05, 0.13 4.1-4.3 respectively. Values are binarized.
round(
P. * 9: Elgil
Cont.
B61-6.3
6(1 2) + (1 b)æ
Values are negated with probability (0.2, 0.4, 0.6) for 6.166.3 respectively.
E
9t.
Car.
~u Bernoulli(p) ML.((r)
+ (1 2pp
Forward through MI.P with min-max normal- ication binarization. random permutation features simaltancously.
Both
PX;
applied to
P: P. 90.
KNN(=)
Prodict festure with (Regressor).
Cont.
B10
KNN(=)
Predict festure with KNN (Classilier).
CaL.
16

---

Feature Shift Localization Network
D. Exploration of Attention Mechanisms
We investigated the integration of attention mechanisms across features within the residual blocks of both the Neural Embedding Network and the Prediction Network to effectively capture diverse data patterns and feature correlations. Upon incorporation, the attention mechanisms were coupled with MLPs implemented using convolutional layers with unit-sized kernels.
D.1. Efficient EVA Attention Layers
Using full attention across features was infeasible for Sigh-dimensional datasets due to its quadratic computational re- quirements. Consequently, we investigated the use of EVA attention layers (Zheng al., 2023). Attention layers without positional embeddings are inherently equivariant; therefore, we did not include positional embeddings. However, efficient (approximate) attention layers, such as EVA, can fail to preserve equivariance under certain conditions.
D.2. Sequence Handling and Window Size Adjustment
Key considerations involve ensuring that the input size to the EVA layer is not only smaller than the sequence length but also divisible by it. Managing variable-length sequences requires meticulous data processing, achieved through padding and the application of key padding mask for efficient sequence handling. The window size of the EVA attention was dynamically adjusted based on the sequence length to optimize processing efficiency and effectiveness, specifically determined as w = min (max (8, 1
D.3. Limitations
Despite EVA attention layers being more efficient than traditional attention mechanisms (Liu et al., 2021), their integration into both the Neural Embedding Network and the Prediction Network resulted in CUDA out-of-memory issues for certain configurations. Consequently, we attempted incorporate attention solely within the Prediction Network. While this approach alleviated memory issues, the performance remained below that of our baseline configuration without any attention mechanisms.
17

---

Feature Shift Localization Network
E. Hlppepparameter Tuning
E.1. Training Setup and Hardware Specifications
We conducted a random search across various hyperparameters of the FSL-Net architecture and its training strategy. Training for each network was terminated if the validation loss did improve for 50 consecutive evaluation intervals, with the maximum training duration limited three days. To expedite the training process, single NVIDIA-SMI GPU with 32GB of memory was used. The networks were validated on 50 validation datasets every 2,500 training iterations. The model checkpoint achieving the highest validation accuracy was selected as the optimal network and subsequently trained for up to seven days. This best-performing model was employed for inference and evaluation in the benchmarking experiments.
E.2. Alternative Statistical Mcasures
In addition to the statistical measures delineated in the main text, we investigated the following metries: skewness (indicating asymmetry), kurtosis (reflecting tail heaviness), index of dispersion (representing the variance-to- to--to--oot memn ratio), and trimmed mean deviation (a robust estimator of central tendency that excludes outliers). The formulas for each measure are presented in Table These measures were ultimately excluded from the final analysis, as they did not confer any discernible advantage.
Table Formulas for additional statistical measures of the Statistical Descriptor Network. the number of observations X. 2 represents the ith dimension of the jth sample (we skip the subscript i for hrevity), z(y is the ith dimension of the Ith sample after sorting the samples from smallest to largest, r is the count of observations trimmed from each trimmed mean deviation, and J and a are the empirical mean and standard deviation of the ith dimension.
Statistical Measure
Linear
Equation Jag,A
Skewness
Yes
# EE=
Kurtosis
Yes
E
Index of Dispersion
Yes
Trimmed Mean Deviation
No
N 21jer41 9(()
Σ
E.3. Alternative Merging Operations
In addition to the normalized squared difference discussed in the main text, we examined several alternative merging operations within the Prediction Network. The following methods were evaluated; however, none yielded performance improvements over the existing approach: concatenation ar (4)) = [Mpi Py]: element-wise difference a' M2)) He - Myi and squared difference a(pp, (2) = (Hp ,² Note that, for the concatenation method, the merged feature vector has dimensions Hp.q E Rdx2t, whereas for the other merging operations, has dimensions Up.. E Rdxe
E.4. Hyperparameter Search
E.4.1. NETWORK TUNING PARAMETERS
Table 7 provides an overview of the search spaces and the optimal values determined for each network-related hyperparameter FSL-Net. These parameters encompass configurations for the statistical measures, Moment Extraction Network, Neural Embedding Network, and Prediction Network. For parameters not explicitly specified, default values were applied.
E.4.2. ATTENTION TUNING PARAMETERS
Table 8 presents the hyperparameters of the attention mechanism explored during the optimization process. These parameters regulate the EVA attention layers within FSL-Net, including the number of heads, the number of landmarks, the window factor, and dropout rates.
E.4.3. OPTIMIZATION TUNING PARAMETERS
Table 9 outlines the search space and optimal values for optimization hyperparameters in FSL-Net's training strategy. This includes the loss function and Adam optimizer settings.
18

---

Feature Shift Localization Network
Table 7, Search space and optimal values for tuned network hyperparameters in FSL-Nct.
Component
Hyperparameterr
Possible Values
Optimal Value
Statistical Measures
Mcan
(Troc, Falsc]
True
Standard
[True; Faise]
Median
TTuc, False]
True
Mean Absolute Deviation
False
True
p-onder Moments
[True, False]
True
p
(2). (3). (2.3)
2,
Histogram
True. False)
True
Empirical CDF
TTue. False]
True
Bins
1000: 100.
Skewnoss
Truc. False]
False
Kurtosis
True, False]
False
Index of Dispersion
Thue. Falee]
Fulse
Trimmed Mean Deviation
True, Falke]
False
Trimmed Percentage
[0.1)
Moment Extraction
[32, 128) 125,
Kernel Size
75
Dilation
(1}
Activation
(ReLU,Tanh)
ReLU
Neural Emhedding
Residual Layers
(3.5,7] ?)
# Kemnll
(32, 64)
64
Kernel Size
(5.7)
Dilation Activation
(1)
(Tanh, GELL!
Tanh
Prediction Network
# Residual Layers
(3
[32, 64)
Kernel Size
(6.7)
Dilation
(1)
Activation
Tanb, GELU)
Tanh
Combination
Concatenation. Element-wise difference,
Normalized squared difference
Squared difference, Normalized squared dif- ference)
Table 8. Search space and optimal values for tuned attention hyperparameters.
Component
Hyperrparameter
Possible Values
Optimal Value
Attention
# Heads
(1
Laadmarks
Window factor
[4)
Window size
(7.9)
Overlapping windows
(Tree, False)
False
Atlention drupout
(0. 0.3, 0.5)
Projection dippout
(0)
Table 9, Search space and optimal values for optimization hyperparameters in the training strategy.
Component
Hyperpanmetter
Possible Values
Optimal Value
Loss Function
(0.0001, 0.001)
0.001
Adam Optimizer
Learning
(0.0001, 0.001. 0.01) (0.9, 0.999, 0.9995]
0.001
Learning Rate Gamma
0,9995
19

---

Feature Shift Localization Network
F. Extended Experimental Results
F.I Median-based Feature Shift Localization Performance and Runtime Comparison
Figure 3 presents a median-based analysis of feature shift localization performance and runtime, complementing the mean-based evaluation in the main text. For each method, F-1 scores and runtimes are first averaged across manipulation types and fractions of manipulated features, then aggregated using the median across datasets. This approach eliminates the need for imputation in cases where slower methods fail to complete for some datasets, enabling fairer comparisons across broader set of methods. However, the mean-based results remain more representative for consistently successful approaches like FSL-Net and DataFix, which completed all evaluations. Figure 3a summarizes the resulting median F- scores, while Figure 3b and Figure 3c visualize the trade-off between performance and computational cost, plotting median F-1 scores against median and maximum runtime, respectively. In these plots, higher positions indicate better localization accuracy, while positions further to the left denote lower runtime, These comparisons again confirm that FSL-Net matches DataFix in localization accuracy while offering a substantial advantage runtime efficiency. Despite being only slightly slower than SelectK FSL-Net far exceeds this method in accuracy. Meanwhile, methods such as MRMR and FAST-CMIM continue to struggle with scalability and performance under median-based evaluation. MB-SM, MB-KS, KNN-KS, and Deep-SM show higher F-1 scores when evaluated by the median rather than the mean, with KNN-KS performing the best among them. However, all of these methods still lag behind FSL-Net, despite benefiting from access to the ground truth |C| a condition rarely met in practice and suffer from extremely poor scalability on large datasets. Figure 3d plots the mean runtime against the product of sample and feature sizes for each dataset. FSL-Net maintains significantly lower mean runtime than DataFix across datasets of varying dimensionality. Figure 3e presents median F-1 scores by manipulation type, while Figure 3f shows mean F-1 scores by dataset. FSL-Net matches surpasses the performance of competing methods across nearly all manipulation types, with two exceptions: E9, where DataFix achieves higher performance, consistent with mean-hased results, and E4, where both DataFix and KNN-KS exhibit superior accuracy. Notably, FSL-Net demonstrates clear advantage on high-dimensional datasets such as Phenotypes, Founders, and Canine, outperforming all baselines and highlighting its effectiveness scenarios involving large feature sets.
20

---

Feature Shift Localization Network
1.0
1.0
1.0
FSL-Net
b)
FSL-Net
Datafix
0.9
FSL-et
0.9
DataFii
KNN-KS
0.9
Dataf
0.8
DecP-SM
Decp-SM
MB-5M
MB-SM
MB-SM
Deop SM
MI
MI
Score
0.6
selectKbest
MB-K5
SelectKbest
MB-K5
0.5
MB KS
0.5
0.5
0.4
0.4
0.4
0.3
0.3
0.3
FAST-CMIM
FAST-CMIM
MRAAR
0.2
MRMR
MRMR
10
Median Runtime
Max Runtime (hours)
Musk2
104
Scene
Energy
E4.3
E3
109
MNIST
Covid
10
0.2 0.4
0.6
0.8
1.0
0.4
0.6
0.8
1.0
86.1
E1Poly.
Gas
1
10
Cosine
Canine
10
E6.3
E9
Toundoak S.de"
Dilbert
Founders
10
E8
Phenotypes Dataset
Covld
qas
enargn
cosne pelmumn
allert
conine
Manipulation Type
FSL-Net
DataFik
SelectKbest
FAST-CMIM
MB-KS
Deep-SM
Figure 3. Performance and runtime comparison across feature shift localization methods: a) median F-1 scores across manipulation types, fractions of manipulated features, and datasets; b) median F-1 scores vs. median runtime; c) median F-1 scores vs. maximum runtime; d) runtime saatle-feature size product per dataset; e) median F-1 scores manipulation type; f) mean F-1 scores by dataset.
21

---

Feature Shift Localization Network
F.2. Performance and Computational Efficiency of FSL--Net and DataFix
Figure 4 displays the mean F-1 scores (top), mean runtimes (middle), and maximum runtimes (bottom) for both FSL-Net and DataFix, computed across various manipulation types and fractions of manipulated features. FSL-Net exhibits comparable feature shift localization performance to DataFix the majority of datasets, with notable advantage large datasets such as Phenotypes, Founders, and Canine, while also demonstrating significantly greater computational efficiency. The pronounced disparity between the mean and maximum runtimes of DataFix on certain datasets suggests that its processing time is highly sensitive to the complexity the shifts, potentially necessitating multiple iterations to achieve convergence. In contrast, FSL-Net executes only a single forward pass through the network, ensuring scalability to both high-dimensional and large datasets.
1.0
0.8
F1 Score
0.6
0.4
0.2
0.0
Mean Runtime (hours)
3
Max Runtime (hours)
12
Gas
Covid
Energy
Musk2
Scene
Cosine
Polynomial
Dilbert
MNIST
Phenotypes
Founders
Canine
FSL-Net
DataFix
Figure Mcan F-1 scores (top), mean runtimes (middle), and maximum runtimes (bottom) of FSL-Net and DataFix by dataset.
22

---

Feature Shift Localization Network
F.3. Extended Evaluation of FSL-Nee and DataFix CIFARIO and COIL-1100
Table 10 presents an extended evaluation of FSL-Net and DataFix on two image datasets: CIFARIO (10k samples) and COIL-100. We report the mean F-1 score in feature shift localization and the mean and maximum runtime (in hours), computed across manipulation types and fractions of manipulated features. FSL-Net achieves higher F-1 scores average and consistently outperforms DataFix in terms of computational efficiency, exhibiting significantly lower runtimes in both average and worst-case scenarios.
Table 10, Comparison FSL-Net and DataFix averaged across manipulation types and fractions of manipulated features on two datasets:
CIFARIO (10k) and COIL-100.
Dataset
F-1 Score
Mean Runtime (hours)
Max Runtime (bours)
FSL-Net
DataFix
FSL-Net
DataFix
FSL-Net
DataFix
0.0336
COIL-100
0.9720
0.9805
0,4906
2.9228
0.8406
9.1567
23

---

Feature Shift Localization Network
F.4. Runtime Improvement of FSL.-Net DitaFix
Figure presents the mean runtime improvement FSL-Net over DataFix across datasets, sorted by increasing dataset size (measured as the product of the number of samples and the number of features). The speedup factor defined the ratio of DataFix's runtime to FSL-Net's measures the performance gain, with higher values indicating greater efficiency. The results are averaged across manipulation types and fractions of manipulated features. On average, FSL-Net achieves substantial speedup of 35.8x (indicated by the dashed line), consistently outperforming DataFix across datasets. Among all datasets, the sigh--imensional Phenotypes dataset exhibits the greatest speedup, with FSL-Net outperforming DataFix by a remarkable 136.3x.
X150
X135
X120
Runtime Improvement
X105
X90
X75
X60
X45
X35.8
X30
X15
9.9e
Energy -5.1e3
Scene 7.1e5
Musk2 1.1e
Canine-2 2.9e
Polynomial le?
Dilbert 2e
Phenotypes, 4e
SMNIST-5.40
X1
Covid, Gas
Cosine,
Founders
FSL-Net Runtime Improvement over DataFix
Mean
Figure 5. Runtime improvement of FSL-Net over DataFix across datasets, sorted by dataset size (number of samples x number of features). The speedup factor indicates how many times faster FSL-Net compared to DataFix. The dashed line marks the average speedup of 35.8x.
24

---

Feature Shift Localization Network
F.5. Performance of the Sareshold-Baseer Variant of the SM-only Baseline
We evaluate simplified (though constrained) configuration of FSL-Net that eliminates the Prediction Network and instead relies solely thresholding the combined statistical measures, computed the normalized squared differences between the reference and query. In this threshold-based approach, the statistical measures reduced the mean or the maximum of these differences. Figure 6 shows the mean F-1 scores for the SM-only variant of FSL-Net, where feature shift localization is performed entirely through thresholding. The F-1 scores are averaged across datasets, manipulation types, and fractions of manipulated features. The best performance achieved using the mean- aeaanbasedd method with threshold of 0.002, yielding an optimal F-1 score of 0.307.
scalar score by taking either
F1 Score: 0.307
Threshold: 0.002
Mean (SM)
0.30
Max (SM)
0.25
F1 Score
0.20
0.15
0.10
10
10-3
10
10-1
10
10a
Threshold
Figure 6. Mcan F-1 scores of the SM-only variant of FSL-Net, where the Prediction Network is omitted and feature shift localization relies solely thresholding the statistical measures. The F-1 scores averaged across datasets, manipulation types, and fractions of manipulated features. Statistical measures are reduced to scalar values taking either the mean or maximum of the normalized squared differences between the reference and the query.
25

---

Feature Shift Localization Network
F66. Qualitative Evaluation of FSL-Nee the MNIST Dataset
Figure provides a qualitative assessment of the performance of FSL-Net on the MNIST dataset under various manipulation types. The first subfigure (Figure 7a) illustrates the ability of FSL-Net to identify manipulated features in five sample images. Each row represents distinct manipulation type applied to modify 5% of the image features (i.e., pixels), with the top row corresponding to E5, the middle row to E8, and the bottom row to E1. Correctly detected manipulations are highlighted in green, while undetected modifications (false negatives) and misclassified pixels (false positives) are colored red and orange, respectively. The results indicate that FSL-Net effectively detects manipulation E1, while its performance is less consistent for manipulations ES and E8. The difficulty in detecting E5 comes from its binary rounding of pixel values to or 1, often blending seamlessly into the digit structure. Similarly, E8, which randomly permutes pixel values, disrupts spatial coherence without necessarily creating visually distinct artifacts, making detection more challenging.
The second subfigure (Figure 7b) presents the Mean Squared Error (MSE) between the reference and query statistical func- tional maps derived from statistical measures (including the mean and standard deviation), the Moment Extraction Network, and the Neural Embedding Network. Each har represents a distinet feature, with color coding indicating classification outcomes: true positive (green), false negative (red), false positive (orange), and true negative (blue). Manipulation El (bottom row) is easily detectable, exhibiting high MSE values, particularly in the statistical functional maps derived from the mean, the Moment Extraction Network, and the Neural Embedding Network. In contrast, more subtle manipulations, such E5 (top row) and E8 (middle row), display smaller MSE values between reference and query for certain features, making their detection more challenging. Still, several features are correctly classified for these challenging manipulations, with the Neural Embedding Network and the Moment Extraction Network producing the most distinguishable statistical functional maps. The mean and standard deviation measures are ineffective in detecting manipulation E8, as it only affects correlation, with low MSE values for these measures.
26

---

Feature Shift Localization Network
Manipulation Type:
3
Manipulation Type:
3
Manipulation Type:
2
True Positives
False Negatives
False Positives
(a) Comparison of FSL-Net output for four examples from the MNIST dataset, with manipulation type ES (top row), ES (middle row), and EI (bottom row). In each case. 5% of the image features were manipulated. Green rectangles indicate correctly predicted corrupted features, red rectangles denote false negatives, and orange rectangles represent false positives.
Manipulation Type: ES
2499
0.300
1299
0ABL1S
03300
COCI
GEBUXC
283
02003
Manipulation Type:
Ststetical Meanuree
Moment Extruction
Strd
Neuum Envoeeding
2.9
12.0
E-3155
0.00
C301C
0.00
E.0083
205
906
204
200
400
800
409
406
505
9.0
302
400
430
600
Manipulation Type: E1
2453
15.9
3.0
200
Positives
Negatives
Negatives
(h) Mean Squared Error (MSE) between reference and query statistical functional maps derived from statistical measures, the Moment Extraction Network, and the Neural Emhedding Network. Each bar represents feature, with colors indicating classification outcomes. hottom row corresponds to manipulation type E1, while the middle and top rows correspond to the more challenging manipulation types ES and ES, respectively.
Figure Qualitative evaluation of FSL-Net on the MNIST dataset.
27


Reviews:
[{"review_id": "5WX1ov4DDg", "reviewer": "Reviewer_7bqs", "summary": "This work presents the FSL-Net, a neural network designed to quickly and accurately identify feature shifts in large, high-dimensional datasets, overcoming challenges faced by existing methods. Trained on diverse datasets, FSL-Net can localize shifts in unseen data without requiring retraining. The method looks reasonable to me. However, since I am not familiar with the literature, I am not able to access the novelty of this work.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "*[1] This work presents the FSL-Net, a neural network designed to quickly and accurately identify feature shifts in large, high-dimensional datasets, overcoming challenges faced by existing methods. Trained on diverse datasets, FSL-Net can localize shifts in unseen data without requiring retraining. The method looks reasonable to me. However, since I am not familiar with the literature, I am not able to access the novelty of this work.*\n\n**To the best of our knowledge, FSL-Net is the first network that provides strong pre-training for feature shift localization, allowing it to generalize to both unseen datasets and shift types. This makes it possible to localize shifts with state-of-the-art accuracy by performing a single forward pass, without the need for re-training the network or expensive computations as required in previous works. Note that feature shift localization is an important task in several areas, including healthcare, e-commerce, and industrial applications.** \n\n\n*[2] Since I am not familiar with the literature, so I am not sure how important does the running time means to the task. Specifically, the average running time of DataFix is very close to the proposed method. The only signifcant difference is the max running time. I am not sure how signficant does the max running time mean to this specific task.*\n\n**FSL-Net provides state-of-the-art performance while being orders of magnitude faster than DataFix (up to 135x) across all data sizes. Figure 2d shows the computational time of FSL-Net vs DataFix as the dataset size (num features x num samples) grows, and an additional Figure (https://tinyurl.com/3bh642v5) showing relative speed improvements will be included in the Appendix. This has significant consequences: it allows its application for processing high-dimensional large databases present in biomedicine and e-commerce. For example, the “Phenotypes” dataset from the “UK Biobank”, a major healthcare dataset, requires over 12h of compute time with DataFix, whereas FSL-Net completes the same task in just a few minutes, while also providing improved F-1 score (see Figure 3 in Appendix E). While DataFix provides good localization accuracy, its high computational requirement makes it impractical for real-world large datasets (e.g. UK Biobank with 300k samples and 1M dimensions), making FSL-Net a very valuable alternative by offering excellent localization accuracy alongside efficient scalability. We are updating the main text to clearly highlight these significant speed improvements and its impact in downstream applications.**\n\n***\n\n**Thank you for your constructive review. We have updated the main text to better describe the novelty and impact of our work, and to emphasize the benefits of our significant speed improvements over the previous state of the art. Furthermore, we have enriched the manuscript with: (a) two new benchmark datasets, CIFAR10 and COIL-100 (https://tinyurl.com/3sx9ppwh); (b) a new baseline in the ablation study (using only statistical measures without neural networks) (https://tinyurl.com/4zkm9psn); (c) a more detailed speedup analysis (https://tinyurl.com/3bh642v5);  (d) an improved related work section; and (e) an additional section in the Appendix with a detailed description of the benchmarking methods. We believe that with these additions, the manuscript has improved significantly, and given that our proposed technique is novel and obtains state-of-the-art performance while being orders of magnitude faster than previous works, we kindly suggest increasing the review score.**", "claims_and_evidence": "The deriviation, algorithm implementation as well as the experimental results looks good.", "methods_and_evaluation_criteria": "Yes", "theoretical_claims": "There is no proofs", "experimental_designs_or_analyses": "The experiments looks good to me. The proposed method achieves better tradeoff between running time and the F1 score. Though the F1 score is only slightly better than DataFix. \n\nSince I am not familiar with the literature, so I am not sure how important does the running time means to the task. Specifically,  the average running time of DataFix is very close to the proposed method. The only signifcant difference is the max running time. I am not sure how signficant does the max running time mean to this specific task.", "relation_to_broader_scientific_literature": "I am unfamiliar with the literature.", "essential_references_not_discussed": "NA", "other_strengths_and_weaknesses": "See the above discussion.", "other_comments_or_suggestions": "NA", "questions_for_authors": "NA", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "XpAh1NTTLU", "reviewer": "Reviewer_PC28", "summary": "The authors propose a feature shift detector: the goal is to identify a maximum-size set of features with zero distance for the corresponding marginal distributions. The overall architecture uses three parts for different types of inputs: basic aggregated statistics, MMD-like features generated in linear (moment extraction) and non-linear (neural embedding) ways. The experiments follow the framework introduced in a previous relevant work [Barrabes et al., 2023] by taking both datasets and competitors from that work and showing marginal improvement over the work by Barrabes while the proposed method is more efficient.\n\n## update after rebuttal\n\nI raised my score to weak accept.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "*1. Why would one solve this problem instead of feature selection? Can you expand the experiments to include more traditional problem statements?*\n\n**Feature shift localization appears in many industries. In healthcare, integrating multiple data sources leads to “batch effects” caused by differing data standardization, collection, and processing. E-commerce and industrial applications need to localize when features have a change in their distribution with respect to a reference. While feature selection aims to find a subset good enough for a predictor, shift localization predicts for each feature if it is inducing a distribution shift. While techniques for feature shift localization can be applied to feature selection problems, and vice versa, each has differing goals.**\n\n\n*2. Simple aggregates (mean,std…) have F1=0.71 surpassing competitors. Could you comment on why this performs so strongly and a trade-off between simplicity and performance?*\n\n**In Table 3, all configurations include the “Prediction Network” (PN). Thus, the “SM” configuration incorporates statistical measures (mean, std, etc) which are fed into the Prediction network, obtaining an F-1 score of 0.7. To avoid confusion, we have added “+PN” to all entries. Moreover, we have introduced an SM-only configuration, where no prediction network is used and the statistical measures are thresholded. This SM-only approach sees a drop in performance to an F-1 score of 0.3, showcasing the importance of the PN. See updated Table in (https://tinyurl.com/4zkm9psn).**\n\n\n*3. Have you compared your feature shift detector to Neural Optimal Transport [Korotin et al., 2021] or Sinkhorn divergences [Feydy et al., 2019] to see if they capture distribution characteristics more effectively?* \n\n**While optimal transport techniques can characterize divergences, they do not provide a direct methodology to localize the divergent features, which is the main focus of this paper, and would require modifications in order to be applied for the localization task. However, we agree that such literature should be mentioned in the related work, therefore, we are updating the manuscript to cite optimal transport techniques.**\n\n*4. Would you consider testing with the higher-dimensional protocol [Rabanser, 2019]?*\n\n**Note that our study already includes two very high-dimensional datasets – Founders with 10k features and Canine with 198k features (see Appendix B.1). We have added an extra evaluation using CIFAR10 and COIL-100 from [Rabanser, 2019], comparing FSL-Net and DataFix, where FSL-Net performs better on average while providing significant speed improvements (https://tinyurl.com/3sx9ppwh).**\n\n\n*5. Why did you opt for identifying a set of features with exactly zero marginal distribution distance rather than allowing a small epsilon [Barrabes et al., 2023]?*\n\n**While [Barrabes et al., 2023] introduces an epsilon, it is not applied in their experiments or evaluation at any point. We opted to remove it for simplicity. As it is expected to be small, FSL-Net should remain applicable without issues.**\n\n\n*6. In the DataFix paper, the F1>0.9, yet it does not outperform FSL-Net in your comparisons. Could you clarify these discrepancies and explain whether the experimental setups or evaluation metrics differ?*\n\n**The DataFix paper reports the median across datasets and manipulations, which we believe can be misleading and not truly reflect the performance of the methods. In FSL-Net, we instead report the mean across datasets and manipulations, which explains the small inconsistencies between numbers. We are adding median-based results in the Appendix, showing that FSL-Net provides the best performance in most datasets and manipulations.**\n\n\n\n*7. You use convolutional neural networks. Is there an assumption that the features have an inherent “ordering”? Can you explain why attention-based architectures (common for tabular data) were not explored?*\n\n**We use convolutions to scale across high-dimensional datasets. In Sec. 3: “Sample-wise Invariance, Feature-wise Equivariance, and Locality”, we describe that we obtain (approximate) feature ordering equivariance by shuffling the features at each batch during training, so that the network does not rely on feature ordering. We also explored attention without positional encoding, which has built-in feature ordering equivariance, but as shown in section C (Appendix), it underperformed.**\n\n***\n\n**Thank you for the constructive review. We have included: (a) CIFAR10 and COIL-100 datasets; (b) a new baseline using statistical measures without neural networks; (c) a new speedup analysis (https://tinyurl.com/3bh642v5); (d) an improved related work; and (e) an additional description of the benchmark methods. We believe that with these additions, the manuscript has improved significantly, and given that our proposed technique is novel, accurate, and orders of magnitude faster than previous works, we kindly suggest increasing the review score.**", "claims_and_evidence": "The overall problem statement seems a bit strange to me. Why would one solve this specific problem instead of a general feature selection? While some existing papers have been published in this area, can you expand the experiments to include more traditional problem statements?\n\nHowever, let’s take a further look at a proposed solution. The author combines three types of features, ranging from simple aggregates to learnable nonlinear transformations. The combination of all these features does provide empirical improvement over the existing approaches. Thus, the overall strategy is justified. \n\nHowever, given the ablation in Table 3, the main take-off from this study is that we can achieve an F1 score of 0.71 that surpasses all of the rival methods by just mean, std, …, and histogram—and then apply the squared differences between these features for two datasets! Another minor comment here is that DataFix’s F1 in the original paper exceeds 0.9, thus suggesting that it works better than FSL-Net (I would conclude that both these approaches show similar empirical performance).", "methods_and_evaluation_criteria": "As has been already said, the authors took their protocol as well as used datasets from [Barrabes et al., 2023]. It is OK, but I would be happy to see additional experiments based on the protocol from [Rabanser, 2019] with higher dimensional settings. Another concern is that a similar solution can be provided with modern approaches to measure the distance between distributions like Neural Wasserstein [1] or regularisation provided by Sinkhorn divergences [2]. They should capture the main characteristics of the distribution as well.\n\n[1] Korotin, Alexander, et al. \"Do neural optimal transport solvers work? a continuous wasserstein-2 benchmark.\" Advances in neural information processing systems 34 (2021): 14593-14605.\n[2] Feydy, Jean, et al. \"Interpolating between optimal transport and mmd using sinkhorn divergences.\" The 22nd international conference on artificial intelligence and statistics. PMLR, 2019.", "theoretical_claims": "This paper does not make any theoretical claims. The only thing I don’t understand is why the authors decided to use a strict definition for the mapping F, requiring exactly zero divergence instead of an upper bound of epsilon from [Barrabes et al., 2023]. Another thing that requires clarification is why you use convolutional neural networks instead of attention-based networks for tabular data—do you assume ordering of features?", "experimental_designs_or_analyses": "Apart from the protocol focused on tabular datasets, I’m fine with the experimental design.", "relation_to_broader_scientific_literature": "My main worry about this paper is that it has too narrow focus, while potentially, a bigger impact should be possible for the proposed architecture.", "essential_references_not_discussed": "It seems that all necessary related works were mentioned.", "other_strengths_and_weaknesses": "This paper is promising, as the idea seems interesting, and the method seems to work.", "other_comments_or_suggestions": "-", "questions_for_authors": "Crucial:\n1. The overall problem seems a bit strange to me. I believe a broader adoption of the proposed network would justify its introduction.\n\nMajor:\n2. Your results indicate that simple aggregates (mean, standard deviation, histogram-based features) alone can yield an F1 of 0.71, surpassing specific competitor methods. Could you comment on why this more straightforward configuration performs so strongly and whether you see a trade-off between simplicity and performance?\n3. Have you compared your feature shift detector to modern distribution distance techniques (e.g., Neural Optimal Transport [Korotin et al., 2021] or Sinkhorn divergences [Feydy et al., 2019]) to see if they capture distribution characteristics more effectively?\n4. Would you consider testing with the higher-dimensional protocol from [Rabanser, 2019] to provide a broader picture of your method’s performance?\n\nMinor:\n5. Motivation for Zero-Divergence Criterion: Why did you opt for identifying a set of features with exactly zero marginal distribution distance rather than allowing a small ϵ\\epsilonϵ-threshold (as in [Barrabes et al., 2023]) or a more general feature-selection-based approach?\n6. In the original DataFix paper, the reported F1 surpasses 0.9, yet it does not outperform FSL-Net in your comparisons. Could you clarify these discrepancies and explain whether the experimental setups or evaluation metrics differ?\n7. You use convolutional neural networks (CNNs) for tabular data. Is there an assumption that the features have an inherent “ordering”? Can you explain why attention-based architectures (common for tabular data) were not explored?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "gM0E9PBq25", "reviewer": "Reviewer_tUPM", "summary": "This paper presents FSL-Net, a neural network designed to effectively localize feature shifts through its architectural design. More specifically, FSL-Net is built in two stages, a statistical descriptor network which is proposed to extract underlying distributional information from the inputs, and a prediction network which leverages these signals to localize feature shifts. This work attempts to build upon prior work through significant improvements in scalability. Additionally, the authors provide extensive experimental evaluation across multiple benchmarks.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "*1. It is not clear to the reviewer how applicable feature shift localization would be in the real-world, given that there could be cases where all or none of the feature presents any discernable feature shift.*\n\n**We consider cases where either no features or only a subset exhibit a shift. If every feature experienced a shift, it would be impossible to determine whether the differences arise naturally between the reference and the query. Thus, we base our approach on the assumption that the true, unmodified query comes from the same distribution as the reference. We have revised the text to make this clearer.**\n\n\n*2. However, the reviewer would like to see additional non-tabular datasets included in the experimental analysis.*\n\n**We acknowledge the reviewer’s request for additional non-tabular datasets. Our study includes two genomics datasets (Founders and Canine), and we have added two image datasets (CIFAR-10 and COIL-100). Performance results for these new datasets can be found in  (https://tinyurl.com/3sx9ppwh). On average, FSL-Net outperforms DataFix while offering significant speed improvements.**\n\n***\n\n**Thank you for your constructive review. We have updated the main text to better describe the novelty and impact of our work, and to emphasize the benefits of our significant speed improvements over the previous state of the art. Furthermore, we have enriched the manuscript with: (a) two new benchmark datasets, CIFAR10 and COIL-100 (https://tinyurl.com/3sx9ppwh); (b) a new baseline in the ablation study (using only statistical measures without neural networks) (https://tinyurl.com/4zkm9psn); (c) a more detailed speedup analysis (https://tinyurl.com/3bh642v5); (d) an improved related work section; and (e) an additional section in the Appendix with a detailed description of the benchmarking methods. We believe that with these additions, the manuscript has improved significantly, and given that our proposed technique is novel and obtains state-of-the-art performance while being orders of magnitude faster than previous works, we kindly suggest increasing the review score.**", "claims_and_evidence": "The authors provide compelling claims of FSL-Net's superiority in these areas:\n- It is clear that FSL-Net outperforms prior methods in scalability and speed.\n- Similarly it is also clear to the reviewer how FSL-Net can handle High-Dimensional Features without retraining.\n\nThe areas that I would like to see some additional clarifications on:\n- It is not clear to the reviewer how applicable feature shift localization would be in the real-world, given that there could be cases where all or none of the feature presents any discernable feature shift.", "methods_and_evaluation_criteria": "The reviewers found the proposed evaluations extensive and found that all of the benchmarks proposed match with prior works cited by the author.", "theoretical_claims": "The reviewer found no direct rigorous mathematical proofs for their claims.", "experimental_designs_or_analyses": "The reviewer found the experimental design of the paper to be sound, with benchmarks chosen inline with prior works. However, the reviewer would like to see additional non-tabular datasets included in the experimental analysis.", "relation_to_broader_scientific_literature": "To the reviewer, the primary contribution FSL-Net provides is the scalability and ease of use in real-world applications. With relation to prior works, it does appear that this is an important overlooked aspect in feature shift localization.", "essential_references_not_discussed": "The reviewer is unaware of any nessacary references not included.", "other_strengths_and_weaknesses": "Please see claims & evidence and experimental designs sections.", "other_comments_or_suggestions": "N/A", "questions_for_authors": "N/A", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "qxKbfIPUto", "reviewer": "Reviewer_jv6Y", "summary": "This paper proposes a feature shift localization network that can localize feature shifts between newly given two datasets by learning how to localize feature shifts using multiple datasets with various synthetic feature shifts. The experiments show that the proposed method can accurately and efficiently detect feature shifts compared to existing methods.\n\n## Update after rebuttal\nThanks to your response. \nMy concerns have been addressed. \nAs I no longer find any significant weaknesses, I have decided to increase my score by one.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "*1. Including a more detailed explanation of the comparison methods would make the paper easier to understand.*\n\n**We agree that the paper will benefit from a more detailed description of the competing methods. Therefore, we are adding a new section titled “Benchmarking Methods” in the Appendix that provides a detailed description of  the methods for the camera-ready version of the paper.**\n\n*2. The method has been experimented on multiple datasets, which gives it a certain level of soundness/validity. One point of interest is, how robust is the proposed method to differences in data distributions during training and testing?*\n\n**The datasets and manipulation types used during the network training are different from the ones used during evaluation, showing that the network generalizes well to previously unseen data and manipulation types, making the network applicable out-of-the-box without the need of performing any re-training or fine-tuning.**\n\n*3. The proposed method has the advantage of allowing data comparison without retraining once it has been learned. However, there is little discussion regarding the network's training time. It is expected that the training time will be significant (maybe more than existing methods), but to what extent?*\n\n**Table 3 includes the training time of different configurations and section “D.1. Training Setup and Hardware Specifications” in the Appendix provides additional information regarding the training time and hardware used. Furthermore, as shown in the paper, the network performs excellently in unseen data and manipulations, so we expect that most of the users will make use of the pre-trained network without the need for any fine-tuning or training.**\n\n***\n\n**Thank you for your constructive review. We have updated the main text to better describe the novelty and impact of our work, and to emphasize the benefits of our significant speed improvements over the previous state of the art. Furthermore, we have enriched the manuscript with: (a) two new benchmark datasets, CIFAR10 and COIL-100 (https://tinyurl.com/3sx9ppwh); (b) a new baseline in the ablation study (using only statistical measures without neural networks) (https://tinyurl.com/4zkm9psn); (c) a more detailed speedup analysis (https://tinyurl.com/3bh642v5); (d) an improved related work section; and (e) an additional section in the Appendix with a detailed description of the benchmarking methods. We believe that with these additions, the manuscript has improved significantly, and given that our proposed technique is novel and obtains state-of-the-art performance while being orders of magnitude faster than previous works, we kindly suggest increasing the review score.**", "claims_and_evidence": "As claimed, the effectiveness and efficiency of the proposed method were validated in the experiments.", "methods_and_evaluation_criteria": "The proposed framework is interesting and reasonable to learn how to localize feature shifts.", "theoretical_claims": "N/A", "experimental_designs_or_analyses": "The method has been experimented on multiple datasets, which gives it a certain level of soundness/validity. One point of interest is, how robust is the proposed method to differences in data distributions during training and testing?", "relation_to_broader_scientific_literature": "Yes, feature shifts are important in practice.", "essential_references_not_discussed": "No.", "other_strengths_and_weaknesses": "Please see comments on other questions.", "other_comments_or_suggestions": "- Including a more detailed explanation of the comparison methods would make the paper easier to understand.", "questions_for_authors": "1.  How robustly does the proposed method perform to differences in data distributions during training and testing?\n2. The proposed method has the advantage of allowing data comparison without retraining once it has been learned. However, there is little discussion regarding the network's training time. It is expected that the training time will be significant (maybe more than existing methods), but to what extent?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Meta-review: The paper proposes the Feature Shift Localization Network (FSL-Net) that can localize feature shifts, while scaling to large and high-dimensional datasets.

The reviewers agreed upon positive ratings. Overall I recommend accept and hope the authors incorporate the discussion with the reviewers into the camera-ready version.
Decision: Accept (poster)