Evaluating Latent Space Robustness and Uncertainty of EEG-ML Models under Realistic Distribution Shifts
Authors: Neeraj Wagh
Jionghao Wei
Samarth Rawal
Brent M. Berry
Yogatheesan Varatharajah
Venue: NeurIPS 2022 Accept (2022)
Keywords: dataset shifts
scalp EEG
representation learning
healthcare machine learning
model robustness
latent space
uncertainty quantification
distribution shift

Abstract: The recent availability of large datasets in bio-medicine has inspired the development of representation learning methods for multiple healthcare applications. Despite advances in predictive performance, the clinical utility of such methods is limited when exposed to real-world data. This study develops model diagnostic measures to detect potential pitfalls before deployment without assuming access to external data. Specifically, we focus on modeling realistic data shifts in electrophysiological signals (EEGs) via data transforms and extend the conventional task-based evaluations with analyses of a) the model's latent space and b) predictive uncertainty under these transforms. We conduct experiments on multiple EEG feature encoders and two clinically relevant downstream tasks using publicly available large-scale clinical EEGs. Within this experimental setting, our results suggest that measures of latent space integrity and model uncertainty under the proposed data shifts may help anticipate performance degradation during deployment.

Evaluating Latent Space Robustness and Uncertainty of EEG-ML Models under Realistic Distribution Shifts
Neeraj Wagh', .Jionghao Wei', Samarth Rawal', Brent Berry? Yogatheesan Varatharajah 1.2 UUniersity of Illinois at Urbana-Champaign Mayo Clinic
Abstract
The recent availability of large datasets in bio-medicine has inspired the devel- opment of representation learning methods for multiple healthcare applications. Despite advances in predictive performance, the clinical utility of such methods is limited when exposed to real-world data. This study develops model diagnostic measures to detect potential pitfalls before deployment without assuming access to external data. Specifically, we focus on modeling realistic data shifts in elec- trophysiological signals (EEGs) via data transforms and extend the conventional task-based evaluations with analyses of a) the model's s latent space and b) predictive uncertainty under these transforms. We conduct experiments multiple EEG fea- ture encoders and two clinically relevant downstream tasks using publicly available large-scale clinical EEGs. Within this experimental setting, results suggest that measures of latent space integrity and model uncertainty under the proposed data shifts may help anticipate performance degradation during deployment.
1 Introduction
The availability of large datasets in bio-medicine has inspired the development of deep representa- tion learning methods for diagnostic applications in multiple health domains, such as cardiology, neurology, and radiology. However, there are increasing examples of failures such systems dur- ing real-world deployment, partly due to dataset distribution shifts in the deployment setting 122 3. 4. 5).) Currently, the most common approach undertaken to evaluate healthcare machine learning (ML) models is analyzing task performance in held-out test dataset. The recent advances in covariate-shift detection and adaptation approaches 77, g 9
10: 11 have not been widely adopted in challenges. As such, the practical value of many healthcare ML models remains unclear at present without additional assessments of model resilience in deployment settings (12). This gap reflects major hurdle in translating healthcare ML models that are required to perform consistently under varying noise characteristics, diverse acquisition protocols, and multiple sites and populations. Hence, there a critical need to evaluate the robustness of healthcare MI models to such shifts prior to deployment.
the healthcare ML community because they do not reflect the unique
In this paper, focus particular of healthcare ML pertaining to learning representations of physiological signals such as cicctroenceehalogrrme (EEGs). EEGs are commonly used to diagnose various neurological, psychiatric, and sleep disorders, well as in applications involving brain- machine interfaces. Deep representation learning raw EEG signals has recently gained popularity because of the availability of large-scale EEG datasets 1 and has shown promise in improving the labor-intensive and error- prooe manual process undertaken in clinical EEG reviews
(T4). Various
approaches have been proposed, including fully-supervised deep feature encoders (15 16: 17), self-
supervised encoders (18 19 20), and traditional feature- -aaed encoders using power spectral features
(21). However, many of those approaches struggle generalize previously unseen real-world data,
i.c., data acquired from different populations, institutions, or devices (22 23).
"Corresponding authors: (nwagh2, varatha2) 8illinois....
36th Conference on Neural Information Processing Systems (NeurIPS 2022).

---

We believe that a pronned approach is necessary to holistically evaluate the robustness of healthcare ML models prior deployment (Figure [T]. First, models of realistic data shifts are needed to assess robustness during model development itself. For instance, realistic EEG data shifts may include differences in behavioral states during EEG acquisition (awake or asleep), the number of EEG sensors used, the EEG reference used, noise during acquisition, analog-to-digital conversion, hardware filter settings, and impedance resulting from electrodes. Second, we argue that robustness evaluations may also benefit from analyzing the properties of the latent embedding space 77. Specifically, quantifying the topological changes in the latent space under perturbations can 1) provide early clues into test set performance when ground truth labels are not available; and 2) indicate failure modes of model interpretation techniques that rely on analyzing similarity properties in the latent space. Third, we highlight the need for estimating the uncertainty in model predictions at the output layer (24). Uncertainty modeling is critical in ensuring fail-safety in healthcare MI. as accurate communication of uncertainty allows human experts ignore the model selectively.
In this study, we evaluated the robustness of several previously proposed EEG feature encoders using the above multi-pronged approach. Specifically, we introduced four EEG signal transformations that model the effects of instrumentation--rlaadd real-world EEG variability, namely: hardware band-pass filters, quantization precision, narrow-band impedance noise, and broad-band Gaussian noise. Next, we developed a topological measure of latent space integrity based on Delaunay neighborhood graphs (adapted from (25)) and Monte Carlo dropout-base based approach to estimate predictive uncertainty. In this setting, we then evaluated suply-super,ised, self-supervised, and traditional feature- based EEG encoders in both classification and regression tasks. Our major contributions are the following:
1. We developed four domain-guided EEG data shifts (EEG-DS) reflecting instrumentation- related variability observable in real-world deployment.
2. Using EEG-DS, we devised multi muuli--ponged approach to evaluate the robustness of multiple ML models, including assessments of latent space integrity and predictive uncertainty.
3. Using two large-scale public EEG datasets (TUH (33) and NMT (26)). we show that the proposed approach can help anticipate in-the-wild performance during model development and that popular EEG encoders exhibit variable robustness profiles.
2 Related Work
Generalization to unseen data: The importance of generalization in healtheare ML is widely acknowledged ( Prior work in general ML has proposed, a) methods to detect covariate- shift out-of-distiibution samples 7 28 29); and b) novel learning approaches that generalize well 8
or adapt to new domains (30, 31 32,
[ (0). However, the majority of the adaptive approaches require examples from target distributions and therefore have limited utility. Some methods that
detect covariate-shift focus on specific properties, such as non-stationarity (28; 29). are simplistic and do not reflect the complexity of healthcare data and shifts. In this paper, we focus on detecting failure modes prior to deployment using topological analysis of the latent space and evaluation of model uncertainty under realistic data shifts. In this context, the closest to our work is (7, where authors used measures such as Wasserstein's 's distance for detecting data shifts in the latent space for vision and language tasks. Here we develop more comprehensive topological evaluation of the latent space using Delaunay neighborhood graphs and use it to analyze the robustness of scalp EEG representations. In addition, some recent studies have highlighted the benefits of uncertainty quantification in out out-of-distribution settings, in knowing when not to trust model's predictions (33). Some studies have also indicated that certain types of representation learning approaches exhibit better robustness in terms of predictive uncertainty than others (34). Building on these insights, we utilized a Monte Carlo dropout (MCD)-based uncertainty quantification to analyze the predictive uncertainty of different EEG feature encoders under specific data shifts.
Data shifts in EEG data: Several previous studies have analyzed the performance of EEG-ML models using datasets from multiple institutions (22) 23: (3).. Apart from reporting differences in model performance, these studies did not investigate approaches to identify failure modes prior to deployment, model data shifts, or generate insights for future studies. Some studies have developed data augmentations for EEG signals improve the diversity of training data and have demonstrated some success in achieving better task performance (19; 36 37). While augmentations such as amplitude scaling, time shift, DC shift, and band-stop filtering are useful to increase training data

---

diversity (19), they do not meet the complexity of real-world EEG signal variability observed in deployment settings. Specifically, real-world EEG variability may arise due to diverse physiological states during acquisition, recording conditions and protocols, and internals of the EEG hardware used. Hence, in this paper, we focus on developing domain-guinn saaiid data shifts observable in deployment settings opposed to simpler training time data augmentations.
EEG feature encoders: A variety of EEG feature encoders have been proposed including, fully- supervised deep encoders (13) 16 (7), self-supervised encoders (8) 19 20), and traditional domain- derived feature encoders using power spectral features (2I 38). While the generalization capabilities of the majority of these encoders are unclear, an encoder derived based on self-supervised learning has shown promising trends in generalization when specific
self self-supervision tasks were utilized (18). However, comprehensive robustness evaluation of the different types of EEG feature encoders under realistic EEG data shifts currently lacking.
Background & Methods
In this section, describe the formulations of multi-channnn EEG data, feature encoders, and the proposed multi-pronged robustness evaluation approach including realistic EEG data shifts (EEG-DS), latent space analysis, and uncertainty quantification. The proposed approach (Figure TT is applicable during model development as it does not require additional data or labels from deployment settings.
Data finput)
Encoder
Predictions (output)
2
E T 1 0
T(x)
fo
g
Model Effects of Real-world Shifts Data Transforms
Inspect Encoder Space Integrity
Inspect Uncertainty
Model Predictions
Figure I: Multi-pronged robustness evaluation. Our approach consists of a) implementation of realistic EBG
data shifts (Section 3.2). b) latent space analysis (Section 3.3 and e) uncertainty quantification (Section 3.4
3.1 Learning EEG data representations
We use H; E R M to denote the EEG data of single epoch recorded using M sensors with N samples (i.e.,
time duration x sampling frequency). Typically, full EEG recording consists of multiple such non-overlapping epochs. Consider feature encoder fa MR - I that maps multi-channel EEG epoch x; into d-dimensional latent representation z; E Rdd where z; = fo(2,) In traditional supervised learning, fo is trained jointly with classification or regression head (.) to predict the desired target yi with the following learning objective, where C is loss function.
argmin Σ [(yrgg((fozz))) 0,6
However, in representation learning (e.g., self self-supervised learning), fo is first pretrained using different objective to obtain a feature extractor fe., which produces a latent embedding for a given input. Classification regression heads (.).) can then be trained using the latent embeddings keeping fa- fixed, where the learning objective takes the following form.
argmin Σ
C(yr. go (zx)))
3.2 Realistic data shifts EEG data
In this study, we focus on modeling data shifts in EEG datasets caused by real-world variability. Consider the training (i.e., source) data domain (ar) and the real-world data domain (i.c., target) (T)), where is input data. Covariate shifts are defined when the concept mapping Is fo +
3

---

remains unchanged, resulting in the same conditional distribution of labels i.e., (a(2))) (e))) However, the marginal distributions of data are assumed change i.e., Ps(x) * Pt(z), In this study, we assume no access to external labels or additional data from the real-worid. Instead, we synthetically construct the target domain Pt (æ) based on an expert understanding of the EEG signal acquisition process via set of EEG data transformations T. Therefore, within the scope of the current study, pr(x)) = Ps( (T(()) and the proposed robustness assessment approach can be viewed as quantifying the differences between ps (2) and pa (x)) through set of metrics M.
The effects of real-world variability on EEG segments can be modeled using data transformation function t: R Mx N + R RMN,, where i t(x) represents shifted EEG sample. Real-world EEG populations, behavioral states (e.g.. awake, asleep), the
data shifts may occur due to differences number of EEG sensors used, the EEG reference used, noise during acquisition, amalog-to-dighne conversion, hardware filter settings, and impedance resulting from electrodes. Here, we focus on data shifts induced by acquisition conditions and assume that other variables are controlled. In that context, we develop four EEG transformations: 1) band- pass filter (pp, 2) quantization precision fop, 3) narrowband impedance noise 'IN and 4) broadband noise faN (summarized in Table
Table 1: Realistic EEG data shifts represented transformations.
EEG Shift
Definition
Parameters
Description
Band-pass filter
før == (x, fL, fu)
fu E {0.5, 1 fu E {25, 30)
V: band-pass filter function
fL. fa: low and high cut-off filter frequencies
Quantization pre-
fge := k( D)
k: RR + R {6, 8, 12]
k: restricts decimal digits
cision
Impedance noise
FIN := x7 +
a E {0.001, 0.01, 2. ~ ()
a: noise strength
2,: broadband noise
Wza, 0,
0-1 Hz noise
Broadband noise
IBN := x 6
a C (0.001. 0.13
a: noise strength broadband noise
((0
EEG device manufacturers commonly implement hardware-level band-pass filters in order to restrict the spectral content of the acquired signal. However, those filter settings may vary between different manufacturers and devices. Hence, we define a data shift (pp to reflect the effects of different band-pass filter settings such as, 0.5-30 Hz, 1-30 Hz, and 1-25 Hz.
Band-pass filter:
Quantization precision: In commercial EEG devices, resource constraints limit the precision the recorded EEG data and the analog signal is digitized with a variable number of bits. We implement this effect as a truncation of decimal digits to either 8, or 12 points and denote it using top.
Signal noise:
The acquisition of EEG signals is highly susceptible to multiple sources of noise and artifacts: subject movement-related sources in the form of eye blinks, head movement, jaw clenching, etc., environmental sources such as AC power line interference, and instrumentation- related sources such as electrical impedance due to poor sensor contact with the scalp. We use IIN to model electrical impedance noise due to poor sensor contact or choice of dry/wet electrodes that manifests as narrow-band low-frequency (0-1 Hz) random noise in the EEG signal (39) with different strengths (a {0.001, 0.01, 0.1 Next, we use teN to capture random signal noise that may manifest as broadband (0-45 Hz) noise with different strengths (a e {0.001, 0.01, 0.13).
3.3 Topological analysis of the latent space
Here we aim to quantify the distortion in latent space caused by input data transforms, i.e., differences between = = fo( and = = (xx)). Some previous studies have utilized distance metries such as Wasserstein's ' distance to quantify the differences (7). Here we employ a more comprehensive approach adapted from 400 and (55). where we estimate a manifold that contains point clouds comprising the latent representations } and } and conduct a gcometric/topological analysis on this estimated manifold. We estimate the manifold based on Delaunay neighborhood graphs, which connect spatial neighbors in a way that captures local point density as well as global outliers (25).

---

Suppose we use Z E Rd and ğ E R to denote the sets including all embeddings of raw and transformed input data, respectively, and define a set of vertices V = {z U Z}. Then, the Voronoi cell of v E V is the set of points in E for which u is the closest among V. i.e. Cell(v) = V}. Then, we define edges & by connecting points i and fy whose Voronoi cells intersect, i.e., = {(v,, 0j) E V x V Cell (wt) n Cell (vg) * 0. o vy). This set of vertices V' and edges & form the neighborhoods in the Delaunay graph 9 (V.
{i E Rd1 - S || - will Vu, E
Intuitively, the points in V that share a Voronoi cell boundary are considered natural spatial neighbors irrespective of the distance between them. This property of Delaunay neighborhoods enables robust manifold estimation even when point density varies locally and outlier points exist. Notably, these spatial neighbors or edges can emerge in a way that connects points in Z to points in Z (denoted by R and E' respectively in (25)). Such edges can be considered 'heterogeneous" as opposed to "homogenoous" edges which connect two points belonging to the same set. The degree of edge heterogeneity of G can be measured by simple quality measure (40) ((G) defined in Equation |
In Equation 1 |
l and | denote the number of homogeneous edges within 9 belonging to g
Z and 2. respectively, and |g|e denotes the total edges in G. Therefore, if G exclusively contains homogeneous edges, q (G) = 0, and all edges are heterogeneous, then q (9) = 1. In this work, we use the degree of edge heterogeneity, (G), measure of latent space integrity between un- mhishifted and shifted samples under perturbations T. A visual interpretation of q()) is provided in Figure
(1021 E + g (8)
9 (9) 1
(1)
No shift
IRN: a=0.00
tnn; -=0.0)
fBN: a=0 )
0.53
s(
0.53
9( 0.48
((
0.0
Nodericaau
Hetergreemn edges
Figure 2: Visualization of changes in latent space between unmodified (2; orange) and broadband noise shifted data (ž; blue) as measured by the proportion of heterogeneous edges (grey) in the Delaunay neighborhood graph (Section (.33). Note that as noise strength increases from left to right, there is a decrease in the proportion of heterogeneous edges. This decrease in proportion is interpreted as decreasing latent space integrity and corresponds to lower (()) scores. Figure best viewed in color.
3.4 Quantifying model uncertainty using Monte Carlo dropout
In this work, we use a Monte Carlo dropout (MCD) strategy to examine per-sample variability in predictions to measure model 'uncertainty" for new sample (24.. Specifically, we directly compare the changes in model uncertainty (measured using the standard deviation) under the aforementioned EEG-DS. Here we briefly summarize the theoretical background and our implementation.
In the Bayesian setting, the predictive distribution of a model f with parameters 0 for an input x and output is expressed as p (ů xr D) = I (6 xr 0) p(0) D)de, where 7) is training data. Because exact computation of this expression is intractable, we perform approximate inference using a varia- tional distribution. A previous study showed that training neural networks with Bernoulli dropout variables is equivalent to approximate inference of the posterior of model parameters (24). With T sets of Bernoulli realizations of 0 (0.) the Monte Carlo estimate of the mean prediction and its vari- ance are computed as [[]][=] x ΣT f (x,0), and Var(ğlz)
+E (f 0,) =[ In our experiments, we used a dropout probability of e = 0.5 and repeated forward passes T = 20 times to estimate the mean and variance of predictions ÿ. We then compare the standard deviation and the agreement index of the predictions to make conclusions about the predictive uncertainty of
5

---

regression and classification tasks, respectively. The agreement index between multiple Monte Carlo dropouts for threshold is defined as, @ = + D 1 (z, 0t) 2 Te).
4 Experimental Setup
Our overall experiment setup is illustrated in Figure 3 Using raw EEG signals and their shifted versions under transforms EEG-DS introduced in Section 3.2 evaluate the robustness of EEG encoders in both in-sample and out-of-sample data. Through our experiments, we hope to answer the following questions: a) do latent space analysis and predictive uncertainty help anticipate model behavior in deployment settings?, b) what types of real-world EEG data shifts impact model robustness the most?, and c) do some encoders exhibit better robustness than others? In the following. we briefly describe the components of our evaluation setup: the EEG datasets, preprocessing steps, EEG feature encoders, learning tasks, data splits, and evaluation metries.
fe
G(V.E)
y
7
T(x)
fo
20
96
Apply Realistic Dataset Shifts to Raw EEG Signuls
Analyze Latent Space Integrity using Edge Helerogeneity
Quantify Model Uncertainty using
In the-wild Clinical Task Performance using Labels
Monte Carlo Dropout
The overall experiment setup. First, trained several EEG feature encoders (fe) using unmodified training data in fully supervised, self-supervised. and traditional PSD feature- based settings. Second, we generated latent representations for the unmodified test data and the modified test data (using data shifts T), and compared the topological similarities in the latent space (z and E). Next, we fine-tuned the feature encoders on classification and regression tasks (ge)) and generated predictions y and ÿ for both a and ž. We then quantified the uncertainty in predictions using an MCD strategy. Finally, we evaluated encoder performances in the in-sample test data (for modified and unmodified versions) and in the aut-of-samplc data.
Figure 3:
EEG dataset & preprocessing: We utilize EEG data from two large-scale databases, the TUH-EEG database from Temple University Hospital 133 and the NMT database from Pak-Emirates Military Hospital (26). For our experiments, we utilize 2,993 EEGs from TUH-EEG (a subset known as TUAB) and 1,369 EEGs from the NMT database, both of which contain normal and abnormal labels. All EEGs were recorded according the standard 10-20 montage 41. We extracted age labels from text reports accompanying the TUAB recordings using a previously published tool (42). Next, we preprocessed the EEGs as follows: a) we ordered the EEG channels 'Fpl', 'Fp2', 'F3', 'F4', 'C3', 'C4', 'P'', 'P4', 'O1', '02'. 'F7', 'F8', 'T7', 'T8', 'P7', 'P8', 'Fz'. 'Cz', and 'Pz', b) resampled the recordings 128Hz, e) applied bandpass filter between 0.5Hz and 45.0Hz. d) divided the recordings into contiguous con-overlapping epochs of 10-seconds each, e) identified and removed bad epochs when the total power in their 'Cz' channel exceeded standard deviations as calculated from statistics of each recording, f) clipped amplitude values to not exceed + 800 and finally g) we normalized the recording per-channe using the identified valid epochs. Note that whenever transforms T were utilized, they were applied directly on the raw data before preprocessing.
We train three diverse types of raw EEG encoders: fully-supervised, self-supervised, and traditional feature-based. The fully supervised encoders (FSE) were based on the widely used 'ShallowNet' architecture (15) with additional classification and regression heads described below. The self-supervised encoder (SSE) also utilized the 'ShallowNer' architecture with pre-training performed using a proxy measure of the patient's behavioral state (18). Finally, we utilized power- ppectral-density-bee based encoder (PSDE) with EEG spectral power calculated within 7 frequency bands: (2-4Hz), (4-8Hz), low a (8- (0-)H high a (10-1 (3)), low /3 (13-16Hz), high 3 (16-25Hz), and (25-40Hz) for each of the 19 channels. Both FSE and SSE took raw EEG traces as inputs and produced 128-dimensional embeddings, while PSDE generated 19x7=133 features.
EEG encoders:
We utilized classification and regression heads to develop task-specific predictors using the above encoders. Note that we trained FSE entirely using the task labels, whereas we fine-tuned SSE by training the classification/regrssss
layer while keeping the encoder network frozen. In addition, we employed dropout with p = 0.5 in all heads and used a binary cross-entropy loss and smooth variant

---

of the mean absolute error loss for training the classification and regression heads, respectively. We trained the models up to a maximum of 500 epochs and monitored convergence with 373 validation EEGSs. We used either Adam or SGD optimizers with a cyclic learning rate scheduler (43). The full set of hyper-parameter
provided the supplement and documented in the publicly released code.
Classification and regression tasks: We utilized meaningful classification and regression tasks to evaluate the robustness of the above encoders. EEG grade classification as 'normal' or 'abnormal' is a common task performed by clinical experts via visual review. Performing such classification using ML approaches in an automated fashion is of great interest since it can potentially reduce the workload of clinical experts. Next, the potential to predict an individual's 's age using EEGs, a regression task commonly known as brain age prediction, useful because it can indicate deviations from healthy brain aging and therefore help with early diagnosis of neurodegenerative diseases. Hence, we utilized two tasks: a binary classification task classifying EEG grades as 'normal' or 'abnormal' and a regression task predicting brain age from EEGs to conduct our experiments.
Training, validation, and testing: We split TUAB into fixed training (1,490 EEGs; 258,631 epochs), validation (373 EEGs; 48,493 epochs), and held-out test (466 EEGs; 82,331 epochs) sets. Furthermore, we performed all model training on TUAB at the epoch level without any shifts. Since both EEG grade and age are recording-level measures, during evaluation, we aggregate epoch-level predictions
by averaging the predictions across all epochs recording [8] 21.
In-sample and out-of-sample testing: We evaluated the robustness of EEG encoders in the in- sample setting by comparing performances between the modified and unmodified TUAB test set and out-of-sample (i.e., in-the-wild) performance using the external NMT dataset. We restricted NMT evaluations to data from adults (18+) to avoid any physiological variability unseen during training.
We performed all evaluations on the held-out TUAB test set and the external NMT dataset. The key metrics utilized to perform the proposed encoder assessments are described in Section 3 Based on the graph construction outlined in Section 3.3 we interpreted the measure of edge heterogeneity obtained using Equation I as a score of the integrity of the latent space under perturbation. For each encoder, we compared the representations of raw EEG samples against their shifted versions in the latent space. Next, we used predictive uncertainty to analyze the per-sample variability of encoders under dataset shifts. Specifically, we analyzed the median agreement index (Ø) and standard deviation in model predictions obtained via MCD as measures of uncertainty. Finally, we quantified task performance using the ground truth labels. To analyze task performance, we used the area under the receiver operating characteristic curve (AUC) and the mean absolute error (MAE) metrics for EEG grade and brain age tasks, respectively. Note that we performed the analyses of the latent space using embeddings of all the epochs in the test and analyzed task performance and uncertainty using aggregated EEG-recording level metrics.
Evaluation metrics:
Software & hardware:
The preprocessing, baseline features, and model training were implemented
using a combination of the following Python librarics: 1) MNE-python (44). 3) PyTorch (45), and 3) NumPy (6)). Delaunay graph construction and analysis were performed using code released by (25). The experiments were performed using 2 Nvidia RTX 3090 GPUs
5 Results
Our results are summarized in Figures 4 and and Table 2 We performed latent space analysis and uncertainty quantification without using the ground truth labels and evaluated whether the changes in those measures resulting from test-time data shifts indicated performance degradation in the in-sample and out-of-sample test sets. In the following, we use the following acronyms to refer to the data shifts and encoders: BP- bandpass filter, QP quantization precision, IN impedance noise, BN broadband noise, SSE self-supervisse encoder, FSE- FSE-grade fully-supervised encoder fine-tuned on grade classification, FSE-age suprevsue encoder fine-tuned on age prediction, PSDE traditional power-spectral density-based encoder.
Impact of data-shifts on predictions: affected when the proposed transformations are applied to the input data. In the top row, we compare the predicted probability of EEG grade by the FSE encoder on held-out data under additive broadband noise tan of varying strength a. The probabilities are grouped along the x-axis by expert labels. We observe that the predicted probability distributions for the two ground truth classes overlap more
Figure shows an example of how model predictions are
7

---

tev: a 0.001
tovi d = 0.01
tev: - 0.1
1.0
1.0
1.0
0.8
0.8
0.8
Probabilty
Probabiltt
Probability
0.6
0.2
0.2
0.2
a(x) g(v(x))
0.0
0.0
0.0
normal
abnormal
normal
abnormal
normal
abnormal
faN:
0.001
Cavi m=
Cani o= 0.1
age
60
Prod cted ago
60
Proedi cbed age
60-
40-
60
20
g(x) ((M(x)
60
80
20
Target age
Target age
Target age
Figure 4: The impact of broadband noise on the predictions of FSE encoder in EEG grade and brain age tasks.
under stronger shifts. In the bottom row, show scatter plots of brain age predicted by FSE against the ground truth under similar comparison settings as above. With stronger shifts, we observe collapse in the range of FSE predictions.
Evaluation the latent space: Figure TigureBssoore shows the latent space integrity scores (f) in the in m--maplle (lines) and out-of out-of-sample ("x 's) settings for multiple EEG encoders. The same results are provided tabular form in the supplement including standard deviations. First, we established an empirical baseline for ideal integrity scores by comparing un-modified data with itself for each encoder (labeled as 'No shift' in Figure Then we compared the in-sample integrity scores between the encoders by increasing the strength of each data shift (left - right). We find that all proposed data shifts affect the integrity scores, albeit to varying degrees, with BN and QP being the most and least effective, respectively. BN with d=0.1 is aggressive enough to yield only homogeneous edges, and the measure goes to 0. the 'No shift (baseline)) scenario, the number of homogeneous and heterogeneous edges roughly equal, leading to integrity score of approximately 0.5. Interestingly, that PSDE and SSE encoders produce better overall integrity in the latent space compared to others. We then computed these scores between unmodified in in-sample and out-of out-of-sample test sets for 100 randomly chosen EEGs. We find that there are notable differences between the two test sets, and that the representations generated by SSE preserve integrity the most.
also find
IN
BN
0.6
0.4
0.2
rst grace
FSE ige
PSDE
0.0
No shift
30))3330) 30)
f= 20)
fw[1. 25
No shift
12
No Shift
e 0.001
a 0001
t=01
No stuft
a 0.000
01
Figure 5: Integrity measures (g()) in the in-sample (lines) and oft-ol-sample ('x 8) settings for different encoders. Measures
computed different strengths of shifts in the in-sample setting, and between unmodified TUH and NMT tests the out-of-sample setting.
Task performance & uncertainty: Table lists the task performance measures and the uncertainty measures for EEG grade classification and brain age regression tasks evaluated on the in-sample and oft-lles test sets under different data shifts. For additional context, the same metrics computed on the train and validation sets are provided as supplement. The task performance metrics reported are the means of the recording-level AUCs and MAEs computed based the 20 MCD repeats (standard deviations are provided in the supplement). Regardless of shift, we find that FSE outperforms both SSE and PSDE in both tasks significantly. However, SSE and PSDE exhibit consistent performance

---

under perturbations in the age prediction task although their absolute performance is worse than FSE. Data shifts IN and BN affect all encoders in EEG grade classification, and BP, IN, and BN affect FSE encoder in age prediction. As measures of uncertainty, we report the average agreement index in EEG grade prediction and the standard deviation of the predicted age values across 20 MCD repeats. We find that both SSE and PSDE exhibit overconfidence in model predictions despite achieving worse task performance than FSE in both tasks. Next, all EEG-DS cause only marginal changes uncertainty measures and only the strongest BN (a = 0.1) causes a noticeable change. The out-of-sample evaluation shows a universal degradation in both task performance and uncertainty measures across all encoders compared to the baseline.
Table 2:
Task performance and uncertainty measures for elassification (EEG Grade) and regression (Age) tasks in in-sample and out-of -ft-oo--ample data under different shifts. Abbreviations: AUC area under curve (f). MAE- mean absolute error (L), Ø agreement index (1), and SD standard deviation in predicted values (L). Up/down arrows indicate the direction of desirable trends. Bold highlights indicate the scenarios when the proposed EEG shifts significantly degraded the performance.
EEG Shifts (T)
EEG Grade (AUC)
Age (MAE)
EEG Grade (+)
Age (SD)
SSE
FSE
PSDE
SSE
FSE
PSDE
SSE
FSE
PSDE
SSE
FSE
PSDE
No shift (baseline)
0.77
0.92
0.77
13.74
9.47
15.56
0.99
0.99
0.96
0.10
0.50
0.32
No shift NMT
0.72
0,72
0.64
17.46
12.41
17.74
0.97
0.97
0.85
0.14
0.73
0.52
fge (f [0.5. 30)
0.77
0.77
13.74
11.24
15.52
0.99
1.00
0.96
0.10
0.47
for
[1]3
0.78
0.92
0.78
13.70
11.14
15.44
0.99
1.00
0.95
0.10
0.46
0.33
fge (f = [1,25])
0.78
0.92
0.77
13,70
11.10
15.47
0.99
0.99
0.95
0.10
0.46
0.33
for (12 12)
0.77
0.92
0.77
13.74
9.47
13.56
0.99
0.99
0.96
0.10
0.50
0.32
for (D 8)
0.77
0.92
0.77
13.74
9.47
15.56
0.99
0.99
0.96
0.10
0.50
0.32
fop (D 6)
0.77
0.77
13.75
9.51
15.56
0.99
0.99
0.96
0.10
0.49
0.32
fox (ar 0.001)
0.77
0.92
0.77
13.74
9.47
15.56
0.99
0.99
0.96
0.10
0.50
0.32
fux (ar 0.01)
0.77
0,92
0.76
13.75
9.49
15.58
0.99
0.99
0.96
0.10
0.49
0.32
fax (ar 0.1)
0.91
0.67
13.77
11.17
0.99
0.99
0.98
0.10
0.44
FaN (a 0.001)
0.77
0.92
0.77
13.74
9.50
15.56
0.99
0.99
0.96
0.10
0.49
0.32
faN (a 0.01)
0.78
0.91
0.78
13.84
11.06
15.41
0.99
0.99
0.95
0.08
0.47
0.31
fuN (a = 0.1)
0.72
0.86
0.71
14.08
14.05
14.66
0.83
0.94
0.83
0.11
0.76
0.35
6 Discussion
The aim of this study was to develop model diagnostic measures for BEG feature encoders that help anticipate deployment failures during model development itself. We modeled real-world variability in EEG signals by designing four data shifts, i.e.. EEG-DS [T) that capture the effects of band-pass pass filter settings, mnalog-to-digite quantization precision, and manifestations of noise. Then, we trained diverse types of EEG encoders (self-supervised, fully-supervised. and conventional) on un-shifted data for two clinically important tasks (Section 4 and assessed their robustness under these shifts during model development. Notably, we analyzed the encoders' latent space (Figure (5) and predictive uncertainty (Table 2 in a label-agnostic fashion, in addition to analyzing task performance (Table 2}. Finally, an evaluation using an external dataset indicated that the performance drop observed in the out-of-sample setting is proportional to what we observed the above experiments. The significance of our study is that we propose evaluation metrics beyond traditional task performance that can be computed during model development itself to evaluate model robustness and anticipate future failures in practical deployment settings. Furthermore, emphasize that the benefit of our approach is that it does not require access to additional data or labels from the deployment setting.
The value of latent space analysis: If we consider task performance as a gold-standard metric in deployment settings, the latent space analysis allows us to see 'early' signs that predict degraded task performance (Figure and Table 2)). The results of aggressive shifts (e.g., a=0.1) l) in the in -ampe experiments strongly correlated with the out-of -f-mmple performance, highlighting the potential use of the latent space analysis. Intuitively, a strong distortion in the latent space will likely push samples from one side of the decision boundary to the other, leading to worse performance.
The value of uncertainty quantification: Our uncertainty studies identify a major pitfall that can occur during deployment. We found that both SSE and PSDE encoders were highly confident in their predictions, although their task performances were much worse compared to FSE. This might indicate that the SSE and PSDE encoders were not sufficiently trained to achieve desirable task performances (i.e., classifier bias) and hence showed less variability in predictions. On the other

---

hand, the uncertainty measures of the FSE encoder provided meaningful interpretations. although the changes in uncertainty under the proposed data shifts were marginal. We believe that analyzing measures of uncertainty in conjunction with task performance can help assess real-world robustness.
Data shifts: Trends in Figure[5jidiicaa that a (1-25Hz) band-pass filter and strong additive broadband or narrowband noise (a 0.1) impact latent space the most. Table 2 shows the same trend in task performance, although the effects are more pronounced in age prediction compared to EEG grade classification. From an EEG perspective, sensitivity to band-pass filtering and noise is expected since they affect the spectral content of the signal, which important for the tasks of EEG grade classification and brain age regression. We found that the data shifts did not impact uncertainty measures all that much except for the strongest broadband noise (ar = 0.1). While we do not consider all sources of EEG shift, the fact that the proposed instrumentation-relaa
shifts can anticipate out-of out-of-sample performance suggests that they represent a good fraction of real-world diversity and are indeed valuable in anticipating deployment failures.
EEG feature encoders: A general trend seen in Table[aacooss all shifts is that FSE provides better absolute performance and that SSE and PSDE encoder performances are more stable. This apparent trade-off between absolute performance and robustness can be explained by considering the effective model sizes (i.e., the number of learnable parameters) in each setting. Recall from Section 4 that only the last linear layer (.) is trainable for SSE and PSDE while the entire network ( is trained in case of FSE. Due to disproportionately higher number of parameters, FSE can be expected to perform better than SSE and FSE. However, this also implies that dropout in FSE would inactivate a disproportionately higher number of units, leading to less stable performance.
Limitations and future directions: Besides instrumentation--elae relatte variability, data shifts originating from physiological changes in brain activity (e.g., sleep) may also play a significant role in real-world robustness. In addition, our design of the quantizer precision shift could be made more rigorous by re-mapping the discrete signal amplitude to a restricted set of quantized numerical levels instead of manipulating decimal precision. In future work, we will implement additional data shifts representing the physiological variability in brain activity and investigate whether adversarial training and training with data shifts could improve real-world robustness.
Data and code availability: The TUAB and NMT datasets used in this study are already publicly available. Model checkpoints and code needed reproduce results and scripts to apply the pro- posed robustness evaluation approach on other datasets are provided at htttps://githhn com/ neerajwagh/ www.wa/a/awaaaagg...aaaaaaaaa.a. reggreepessntttionss
7 Conclusion
In this paper, we developed an approach to assess the real-world robustness of EEG-ML models during model development itself. Our approach included the implementation of realistic data shifts and the evaluation of model robustmess using a latent space topological analysis and quantifica- tion of predictive uncertainty. We developed four data shifts for EEG signals reflecting potential instrumentation-relaee
variability observable in the real world. We then performed experiments to evaluate differences in latent space integrity, predictive uncertainty, and task performance under the above data shifts and correlated the results with out-of-sample (i.e., in-the-wild) performance. Within our experimental setting, including three EEG feature encoders, two downstream tasks, and two large-scale EEG datasets, our results demonstrate that the proposed approach can help anticipate real-world performance during model development. In future work, we will consider other plausible data shifts, adversarial training, and training models explicitly realistic data shifts.
Acknowledgments and Disclosure of Funding
This research was partially supported by the National Science Foundation (Award No. IIS-2105233), Mayo Clinie Neurology Artificial Intelligence Program, and Mayo Clinic & Illinois Alliance Fellowship for Technology-Ban -Based Healthcare Research. The authors thank Petra Poklukar for assisting with latent space analysis.
10

---

References
[1] E. A. AlBadawy, A. Saha, and M. A. Mazurowski, "Deep learning for segmentation of brain tumors: Impact of cross-institutional training and testing." " Medical physics, vol. 45, no. 3, pp. 1 1150-1158, 2018.
[2] S. K. Lynch, A. Shah, J. C. Folk, X. Wu, and M. D. Ahramoff, "Catastrophic failure in image-based convolutional neural network algorithmss for detecting diabetic retinopathy." Investigative Ophthalmology & Visual Science, vol. 58, no. 8. pp. 3776-3776, 2017.
[3] E. Strickland, "Ibm watson, heal thyself: How ibm overpromised and underdelivered on ai health care," IEEE Spectrum, vol. 56, no. 4, pp. 24-31, 2019.
[4] D. Wang. L. Wang, Z. Zhang, D. Wang, H. Zhu, Y. Gao, X. Fan, and F. Tian, "'brilliant ai doctor" in rural clinics: Challenges in ai-powered clinical decision support system deployment," in Proceedings of the 2021 CHI Conference on Human Factors Computing Systems, 2021, pp. 1-18.
[5] J. R. Zech, M. A. Badgeley, M. Liu, A. B. Costa, J. Titano, and E. K. Oermann, "Variahle generalization performance of a deep learnngg model to detect pneumonia in chest radiographs: cross-sectional study," PLoS medicine, vol. 15, no. 11, p. c1002683, 2018.
[6] G. Mårtensson, D. Ferreira, T. Granberg, L. Cavallin, K. Oppedal, A. Padovani, I. Rektorova. L. Bonanni, Pardini, M. Kramberger et al., "The reliability of deep learning model in climical out-of-distribution mri data: a multicohort study," Medical Image Analysis, vol. 66, p. 101714, 2020.
[7] L. Betthauser, U. Chajewska, M. Diesendruck, and R. Pesala, "Discovering distribution shifts using latent space representations." " arXiv preprint arXiv:2202.023339 2022.
[8] H. Shimodaira, "Improving predictive inference under covariate shift by weighting the log-likelihood function," " Journal of statistical planning and inference, vol. 90, no. 2. pp. 227-244, 2000.
[9] J. Huang, A. Gretton, K. Borgwardt, B. Schölkopf, and A. Smola, "Correcting sample selection bias by unlabeled data," Advances in neural information processing systems, vol. 19, 2006.
[10] S. Schneider, E. Rusak, L. Eck, O. Bringmann, W. Brendel, and M. Bethge, "Improving robustness against common corruptions by covariate shift adaptation." Advances in Neural Information Processing Systems, vol. 33. pp. 11 539 551, 2020.
[11] O. Wiles, S. Gowal, F. Stimberg, S. Rebis--Reufff, I. Ktena, T. Cemgil et al., "A fine-grainet analysis on distribution shift," arXiv preprint arXiv:2110.11328, 2021.
[12] S. Saria and A. Subbaswany, "Tutorial: safe and reliable machine learning." arXiv preprint arXiv: 10Xiv:1904.0 07204, 2019.
[13] I. Obeid and J. Picone, "The temple university hospital eeg data corpus," Frontiers in neuroscience, vol. 10, p. 196, 2016.
[14] U. R. Acharya, S. V. Sree, G. Swapna, R. J. Martis, and J. S. Suri, "Automated eeg analysis of epilepsy: a review," Knowledge-Bace Based Systems, vol. 45, pp. 147-165, 2013.
[15] R. T. Schirrmiitter, J. Springenberg, L. D. Ficderer, M. Glasstctter, K. Eggensperger, M. Tangermann, F. Hutter, W. Burgard, and T. Ball, "Deep learning with convolutional neural networks for eeg decoding and visualization." Human brain mapping, vol. 38, no. 11, pp. 5391-5420, 2017.
[16] S. Roy. I. Kiral-Kornek, and S. Harrer, "Deep learning enabled automatic abuormal eeg identification," in 2018 40th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC). IEEE, 2018, pp. 2756-2755 2759..
[17] M. Alhussein, G. Muhammad, and M. S. Hossain, "Eeg pathology detection based on deep learning," " IEEE Access, vol. 7, pp. 27 781-27 788, 2019.
[18] N. Wagh. Wei, S. Rawal, B. Berry, L. Barnard, B. Brinkmann, G. Worrell, D. Jones, and Y. Varathara- jah, "Domain- "Domain-guided self-supervision of cog data improves downstream classification performance and generalizabilityy" in Machine Learning for Health. PMLR, 2021, pp. 130-142.
[19] M. N. Mohsenvand, M. R. Izadi, and P. Maes, "Contrastive representation learning for electroenceehalo- gram c in Machine Learning for Health. PMLR, 2020, pp. 238-253.
[20] H. Banville, O. Chehab, A. Hyvärinen, D.-A. Engemann, and A. Gramfort, "Uncovering the structure of clinical eeg signals with self- superviee learning"" Journal of Neural Engineering, vol. 18, no. 4, 046020, 2021.
11

---

[21] N. Wagh and Y. Varatharajah, "Eeg-gean: Augmenting electrenccehallggaa-- baase neurological disease diagnosis using domain-guided graph convolutional neural network," in Machine Learning for Health. PMLR, 2020. pp. 367-378.
[22] Y. Varatharajah, B. Joseph, B. Brinkmann. M. Morita-Sherman, Z. Fitzgerald. D. Vegh, D. Nair. R. Burgess, F. Cendes. Jehi et al., "Quantitative analysis of visually reviewed normal scalp eeg predicts seizure freedom following anterior temporal lobectomy." Epilepsia.
[23] P. Thangavel, Thomas, W. Y. Peh, J. Jing, R. Yuvaraj, S. Cash, R. Chaudhari, S. Karia, Rathakr- ishnan, V. Saini et al., "Time frequency decomposition of scalp electroencephalograms improves deep learning-base based epilepsy diagnosis." International journal of neural systems, vol. 31, no. 08, p. 2150032, 2021.
[24] Y. Gal and Z. Ghahramani, "Dropout as a bayesian approximation: Representing model uncertainty in deep learning"" in international conference machine learning. PMLR, 2016, pp. 1050-1059,
[25] P. Poklukar, V. Polianskii, A. Varava, F. Pokorny. and D. Kragic, "Delaunay component analysis for evaluation of data representations," arXiv preprint arXiv:2202.06866 2022.
[26] H. A. Khan, R. U. Ain, A. M. Kamboh, H. T. Butt, S. Shafait, W. Alamgir, D. Stricker, and F. Shafait, "The nmt scalp eeg dataset: An open-souree annotated dataset of healthy and pathological eeg recordings for predictive modeling," Frontiers in Neuroscience, vol. 15, 2021.
[27] H. Zhang, N. Dullerud, L. Kalantrr,,,,,ari,, Q Morris, S. Joshi, and M. Ghassemi, "An empirical framework for domain generalization in clinical settings," in Proceedings of the Conference on Health, Inference, and Learning, 2021, pp. 279-290.
[28] H. Raza, Prasad, and Y. Li. "Adaptive learning with covariate shift-detection for non-stationary en- vironments," " in 2014 14th UK Workshop on Computational Intelligence (UKCI). IERE, 2014, pp. 1-8.
[29] "Ewma model based shift-detectio methods for detecting covariate shifts in non non-stationary environ- ments," Pattern Recognition, vol. 48, no. 3, pp. 659-669, 2015.
[30] Gulrajani and D. Loopzz-Paz, "In search of lost domain generalization," arXiv preprint 27X00:2007.01434 2020.
[31] D. Li, Y. Yang. Y.Z Song. and T. M. Hospedales, "Deeper, broader and artier domain generalization." in Proceedings of the IEEE international conference on computer vision, 2017, pp. 5542-5550.
[32] V. Piratla, P. Netrapalli, and S. Sarawagi, "Efficient domain generalization via common-spectiioo low-rank decomposition," in International Conference on Machine Learning. PMLR, 2020, pp. 7728-7738 7738..73..
[33] Y. Ovadia, E. Fertig, J. Ren, Z. Nado, D. Sculley, S. Nowozin, J. Dillon, B. Lakshminarayanan, and J. Snock, "Can you trust your model's ' uncertainty? evaluating predictive uncertainty under dataset shift," Advances in neural information processing systems, vol. 32, 2019.
[34] D. Hendrycks, K. Lee, and M. Mazeika, "Using pe--trannng can improve model robustness and uncertainty." in International Conference Machine Learning. PMLR, 2019, pp.
[35] P. Nejedly, V. Kremen, V. Sladky, M. Nasseri, H. Guragain, P. Klimes, J. Cimbalnik, Y. Varatharajah, B. H. Brinkmann, and G. A. Worrell, "Deep-learning for scizure forecasting in canines with epilepsy." Journal of neural engineering, vol. 16, no. 3. p. 036031, 2019.
[36] E. Lashgari, D. Liang, and U. Maoz, "Data augmentation for deep-learning based electroencephalograahhy"" Journal of Neuroscience Methods, vol. 346, p. 108885, 2020.
[37] F. Wang, S.-h. Zhong, J. Peng, J. Jiang, and Y. Liu, "Data augmentation for ecg- ecg- based emotion recognition with deep convolutional neural networks," " International conference on multimedia modeling. Springer, 2018, pp. 82-93.
[38] Y. Varatharajah, M. Chong, K. Saboo, B. Berry, Brinkmann, G. Worrell, and R. Iyer, factor-graph-basee model for capturing spatial, temporal, and observational relationships in electroen- cophalograms," " Advances in neural information processing systems, vol. 30, 2017.
[39] F. Kappenman and S. Luck, "The effects of electrode impedance on data quality and statistical significance in erp recordings," Psychophysiology, vol. 47, no. 5, pp. 888-904, 2010.
12

---

[40] P. Poklukar, A. Varava, and D. Kragic, "Geomca: Geometric evaluation of data representations"" in International Conference Muchine Learning. PMLR, 2021, pp. 8388-8598.
[41] G. F. Chatrian, E. Lettich, and P. L. Nelson, "Ten percent electrode system for topographic studies of spontaneous and evoked eeg activities," American Journal of EEG technology. vol. 25, no. pp. 83-92,
1985.
[42] S. Rawal and Y. Varatharajah, "Score-it: machine learning-based tool for automatic standardization of eog reports." Machine Learning for Healthcare (Clinical Abstract), 2021.
[43] L. N. Smith, "Cyclical learning rates for training neural networks," " in 2017 IEEE winter conference applications of computer vision (WACV). IEEE, 2017, pp. 464-772
[44] A. Gramfort, M. Luessi, E. Larson, D. A. Engemann, D. Strohmeier, C. Brodbeck, R. Goj. M. Jas, Brooks, L. Parkkonen et al., "Meg and eeg data analysis mne-python."" Frontiers in neuroscience, vol. 7. p. 267, 2013.
[45] A. Paszke, S. Gross, Massa, Lerer, J. Bradbury, G. Chanan, Killeen, Lin, N. Gimelshein, L. Antiga et al., "Pytorch: An imperative style, high-performance deep learning library," Advances in neural information processing systems, vol. 32, pp. 8026 8037, 2019.
[46] C. R. Harris, K. J. Millman, S. J. van der Walt, R. Gommers, P. Virtanen, D. Cournapeau, E. Wieser, J. Taylor. S. Berg, N. J. Smith et al., "Array programming with numpy." Nature, vol. 585, no. 7825, pp. 357-362, 2020.
13

---

Checklist
1. For all authors...
(a) Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope? [Yes] The proposed data shifts for EEG signals are defined in Table I and results of latent space analysis and predictive uncertainty are summarized in Figure | and Table 2 respectively. We highlight that this study does not propose an approach to improve model robustness. See Section 6 for summary of the significance and scope of study.
(b) Did you describe the limitations of your work? [Yes] See Section 6 for discussion on study limitations and caveats in results.
(c) Did you discuss any potential negative societal impacts of your work? [No] This study introduces realistic data shifts and approaches that anticipate model failures and issues with model robustness before the model is deployed. Therefore, the proposed approaches are strongly aimed making positive societal impact by attempting to detect costly real-world failures before they happen.
(d) Have you read the ethics review guidelines and ensured that your paper conforms to them? [Yes] This study uses publicly available EEG data and corresponding text reports (13) that have been de-identified before release. The dataset contains patients with various neurological disorders and diverse age and gender groups. The training data used in our models reflect this diversity. While the focus of our study is evaluations of model robustness to EEG instrumentation related shifts before deployment, we acknowledge that additional assessments are needed for clinical and protected sub- groups interest.
2. If you are including theoretical results...
(a) Did you state the full set of assumptions of all theoretical results? [N/A]
(b) Did you include complete proofs of all theoretical results? [N/A]
3. If you experiments...
(a) Did you include the code, data, and instructions needed to reproduce the main ex- perimental results (either in the supplemental material or as a URL)? [Yes] We have released sanitized version of all code and instructions needed to reproduce results in
Figure Figure and Table 2 as a URL in Section 66]
(b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)? [Yes] Training details including data splits (train, validation, test) and major hyperparameters are summarized in Section g The complete set of hyperparameters is enumerated in the supplement and documented in the released code.
(c) Did you report error bars (e.g., with respect the random seed after running experi- ments multiple times)? [Yes] Variability in model performance is central theme of this study. As such, the variability model performance (due to Monte Carlo dropout) on the two EEG tasks is summarized in Table 2 The supplement contains additional variability results that were omitted from the main text for the sake of brevity.
(d) Did you include the total amount of compute and the type of resources used (e.g., type of GPUs, internal cluster, cloud provider)? [Yes] See Section 4 for acknowledgment of software and internal cluster GPU hardware used for training the models.
4. If you are using existing assets (e.g., code, data, models) or curating/releasing new assets...
(a) If your work uses existing assets, did you cite the creators? [Yes] See Section 4 for acknowledgment of authors whose code we built upon.
(b) Did you mention the license of the assets? [No] The code we have used is either publicly released Python library or GitHub repository with their respective open-
source licenses.
(c) Did you include any new assets either in the supplemental material as URL? [Yes] We include links to code and instructions for replicating results as a URL in Section 6
(d) Did you discuss whether and how consent was obtained from people whose data you're using/curating? [N/A]
14

---

(c) Did you discuss whether the data you are using/curating contains personally identifiable information or offensive content? [N/A] The public datasets used in this study were de-identified before release.
5. If you used crowdsourcing or conducted research with human subjects...
(a) Did you include the full text of instructions given to participants and screenshots, applicable? [N/A]
(b) Did you describe any potential participant risks, with links Institutional Review Board (IRB) approvals, if applicable? [N/A]
(c) Did you include the estimated hourly wage paid to participants and the total amount spent on participant compensation? [N/A]
15


Reviews:
[{"review_id": "aJrByuJSur-", "reviewer": "Reviewer_mwZ8", "summary": "The authors motivate the need for better evaluation methods for medical deep learning models, especially methods that do a better job of predicting in-the-wild problems of a model. Given the potential side-effects that an erroneous prediction can have in medicine, it is vital that better ways to predict how a model under distribution shifts of various intensities would perform, or at the very least, the cases in which the behaviour of the model would differ from the one in training and development validation/testing. \n\nThe authors firstly motivate, using domain expertise, four scalp EEG data shifts that would do a good job of capturing some real world variation of such datasets. Then they propose using latent space integrity methods and predictive uncertainty as two ways of evaluating the behaviour of a model under varying types and intensities of the four proposed dataset shifts. \n\nThe authors then proceed to apply their proposed evaluation measures on a number of large-scale pretrained medical encoders and showcase that their methods have some predictive utility over potential pitfalls of such models. ", "questions": "My questions have already been stated in the previous section, but I will restate here for ease of access:\n\n- Clearer correlation figures: Figures directly showing any meaningful correlations between in-the-wild test performance and the proposed methods attempting to predict issues in such performance. The authors state that lower is better in prediction uncertainty -- that seems misleading. Could you explain that statement a bit more?\n- Clarity of training: Are any of the models trained *with* the proposed data shifts? What happens when they are? Does it improve performance on any of the heuristics and/or actual in-the-wild performance? I would rephrase the training section to ensure the reader knows what the models are trained on, in terms of data shifts. \n- Prediction uncertainty discussion: I think it is important for the authors to talk about what prediction uncertainty could be showing here. Yes, if a model is more confident but performs worse, then that's an issue, but wouldn't a better measure be how the training/validation uncertainty changes? Ideally, to reduce any unpredictable model behaviour one would like the model to behave similarly in both data distributions. Furthermore, while the authors compare test performance across different data shifts, they do not show how the training performance of the model looks like. I believe that the training performance of the model should be compared a lot more with the testing performance, such that whatever heuristics are computed will take into account behaviour shifts from a 'known' to an 'unknown' domain in multiple degrees (i.e. from train to val/test, and from val/test to data shifted val/test and other pertubations).", "limitations": "The authors have made a fair effort of addressing the limitations of their work, but I think that some of the conclusions drawn were a bit on the overclaiming end, for example:\n\n- we developed four domain-guided data-shifts for EEG signals that reflect the real-world variability observable during test time\nYes, perhaps it covers some, but not all. That should be made more explicit. \n\n- Through evaluation of multiple EEG feature encoders using large-scale EEG data, we empirically showed that the proposed approach can help in anticipating failure scenarios during deployment\nThis was done on a small set of datasets and tasks and using limited models. This should be integrated in this conclusion such that it does not mislead the reader. ", "rating": 7, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "\nOriginality:\n\nThe work proposed in this paper is one of high utility, and depending on ones definition of originality, could be assigned variable levels of originality. Given that this paper addresses medical deep learning, and proposes some very reasonable and relatively useful means of evaluating in-the-wild performance of models, I would consider it as original work, but not extremely so \n\nI think this work might also enable a string of works targeting this direction which is something the medical deep learning community is definitely in need of. \n\n(Score: 7/10)\n\nQuality:\n\nThe quality of the work is very high, both in terms of execution and presentation. \n\n(Score: 8/10)\n\nClarity:\n\nThe paper is quite clear, but could do with some additional figures showing correlations between in-the-wild performance and the proposed measures for a very direct way of showing to the reader what is going on. \n\n(Score: 7/10)\n\nSignificance:\n\nThe ideas in this paper are of high significance, as they start a research direction towards good heuristics for in-the-wild performance for medical deep learning models. They also provide a set of what are effectively four data augmentation methods particularly suited for scalp EEG data, which could benefit the EEG community in general, and provide inspiration to others. \n\n(Score: 7/10)\n\nStrengths: Tackles an important problem with novel solutions and excellent execution with high degree of attention to detail. Provides interesting figures and tables to draw conclusions from.\n\nWeaknesses: \n\n- More datasets/tasks: It would make the claims made over the usefulness of the performance predictive methods more robust if more datasets/tasks were carried out and evaluated. \n- Clearer correlation figures: Figures directly showing any meaningful correlations between in-the-wild test performance and the proposed methods attempting to predict issues in such performance. \n- Clarity of training: Are any of the models trained *with* the proposed data shifts? What happens when they are? Does it improve performance on any of the heuristics and/or actual in-the-wild performance? I would rephrase the training section to ensure the reader knows what the models are trained on, in terms of data shifts. \n- Prediction uncertainty discussion: I think it is important for the authors to talk about what prediction uncertainty could be showing here. Yes, if a model is more confident but performs worse, then that's an issue, but wouldn't a better measure be how the training/validation uncertainty changes? Ideally, to reduce any unpredictable model behaviour one would like the model to behave similarly in both data distributions. Furthermore, while the authors compare test performance across different data shifts, they do not show how the training performance of the model looks like. I believe that the training performance of the model should be compared a lot more with the testing performance, such that whatever heuristics are computed will take into account behaviour shifts from a 'known' to an 'unknown' domain in multiple degrees (i.e. from train to val/test, and from val/test to data shifted val/test and other pertubations).", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ["I don’t know"]}, "extra_text": {}}, {"review_id": "MqFJNswcTCZ", "reviewer": "Reviewer_PL6S", "summary": "This work aims to provide a holistic analysis of the robustness of EEG data representations after domain-guided data shifts. The authors provides two evaluation metrics to probe the robustness of EEG, namely latent space integrity and predictive uncertainty, on representations produced by different encoder methods. ", "questions": "- I would like some clarifications about Figure 3 under scenario \"no shift\" - according to section 3.3, when all edges are homogeneous (i.e., connecting points belong to the same set), the latent space integrity measure value would be 0, but this is not as suggested in Fig 3. ", "limitations": "Yes.", "rating": 3, "confidence": 3, "soundness": 2, "presentation": 3, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strength:\n- The overall idea is well-motivated and addresses a unique and important challenge in ML for health.\n- The authors performed very thorough analysis in general, testing various data shift and encoder methods - remarkable effort!\n\nWeakness:\n- The problem of evaluating representation/prediction learning accuracy under health data shift is a wide topic, and in many cases it means retaining performance across different sites/demographic groups/out-of-distribution cases. While the authors focus on four EEG device-wise data shift scenarios, the title and the introduction part should be careful not to over-claim the scope of this study.\n- The highlights in Table 2 is misleading - when several numbers are equal, they should all be highlighted instead of just highlighting the numbers favoring an argument. \n- The results are not strong enough for a conclusive analysis or novel insights (numbers reported in Table 2 do not show great difference).", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "5eyHcHO8fm", "reviewer": "Reviewer_VEzQ", "summary": "The paper provides a description how four types of feasible and realistic dataset shifts can be used to estimate their effect on robustness of the classification and regression tasks in EEG analysis. \n\nThe paper uses a latent space of features extracted from the EEG signals and compares how the latent space data points of the original and the transformed data  create a Delaunay graph. If all edges of the in the Delaunay (i.e nearest neighbors)  in the data set would contain only edges from the original latent space points  to the corresponding transformed  latent space points, one would consider the perturbations to be robust.\n\nThe distributions of the outputs for the original and the transformed data are created with Monte Carlo dropout using the model trained by the original model. This way the data shift  effect to the output  distributions can be seen and quantified for robustness estimation. Experiments are performed, for example, with different sizes of added noise and it is seen that noisy data will lead to collapse of the regression task.\n\nThe authors suggest that following the described analysis of the robustness can be used as indication of problems in generalization capability of the model under realistic data shifts in medical domain.", "questions": "To have a more holistic view of the robustness of the trained solution with the use of a given, extensive, data set, one should also check how adversarial training would improve the generalization under the four feasible transformations. This would check and answer if the expected realistic data shifts would be covered automatically by the adversarial robustness. Now, the trained version used is not the optimal model  to estimate the robustness that is achievable by the available data set.\n\nAlso, the result estimation of the robustness reached by augmenting the data with the four known transformations should be performed as a comparison, as it is interesting to see to what extent the knowledge of the possible data shifts that is used in the training would alleviate the problem.\n\nThis paper  a very good and thorough work, but more aspects should be covered. Questions to be answered for better rating:\n1) What happens for the generalization capability if adversarially robust training is used?\n2) What happens for the generalization capability if domain knowledge augmentation (with the four types) is used?\n\nI would appreciate a discussion on these points with the related additional testing.", "limitations": "The paper is a excellent step forward to address the data shifts in the medical domain", "rating": 7, "confidence": 5, "soundness": 4, "presentation": 4, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "The paper is well written and understandable. It provides insights on a long standing problem of training systems with data that originate from restricted domains that are then deployed in other domains. Of course, the ideal case would be to alleviate the problem with big enough data sets that would already contain all the domains. However, even this would not solve the problem for data shifts that happen in time. The problem is highly relevant.\n\nThe paper uses four different realistic transformations, but is not using these to augment the data in the training phase. In many cases augmenting the training data significantly improves the results. Keeping the training data set intact by principle does not bring extra value, because the final verdict really comes from a properly gathered test data that contains the variability of data from different sources and domains. These should cover the data shifts. Using a held out data for testing the generalization is not, in my opinion, a good way to test an AI model in the medical domain.\n\nAlso, using only four transformations is quite small set of variation to the signal. One could argue that it will not provide a comprehensive set of variations that the system will encounter. Using adversarial robustness in training would automatically find the problematic dimensions and improve robustness for a larger, more comprehensive set of  perturbations.\n\nNow, the value of the paper now lies in the analysis of what is happening if the realistic transformations would not be known - and indicate the failures in robustness when this would happen. Choosing these to be feasible actually minimizes the problem, as one may expect that these changes would already occur also inside the original training data, for example in variations of the electrode impedance. Hence, these would be covered by the natural variation. I have some suggestion below to increase the impact of the paper by further analysis.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "5LQD4ZYSR3F", "reviewer": "Reviewer_x119", "summary": "This paper introduces four EEG signal transformations to model the real-world variability observable during deployment. Then, the paper proposes a multi-pronged approach to evaluate the robustness of healthcare ML models. It is well-organized and easy to follow. The extensive experiments demonstrate the paper's claims.", "questions": "1. the paper's title is so broad. it should be focused on a EEGs.\n2. the paper should list baseline models' performance on the related tasks.", "limitations": "1. the novelty of the paper is weak, as it is adapted from an existing model and a Monte Carlo dropout-based method.\n2. The title is to talk about health data representations, but the abstract and introduction talk about specific data - EEGs.\n3. There should be baseline models in the experiments to be compared to demonstrate the proposed model efficiency.", "rating": 4, "confidence": 3, "soundness": 2, "presentation": 3, "contribution": 1, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "1. This paper introduces four EEG signal transformations to model the real-world variability observable during deployment. \n2. The paper proposes a multi-pronged approach to evaluate the robustness of healthcare ML models.\n3. Realistic data shifts in EEG data are explored in the paper.\n4. The extensive experiments are conducted on real-world datasets and demonstrate the paper's claims.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ["Privacy and Security (e.g., consent)"]}, "extra_text": {}}]

Author rebuttal:

Decision: Accept