Self-Supervised Graph Neural Networks for Improved Electroencephalographic Seizure Analysis
Authors: Siyi Tang
Jared Dunnmon
Khaled Kamal Saab
Xuan Zhang
Qianying Huang
Florian Dubost
Daniel Rubin
Christopher Lee-Messer
Venue: ICLR 2022 Poster (2022)
Keywords: Graph neural network
Self-supervision
Interpretability
Visualization
Neuroscience
Electroencephalography
Seizure
Epilepsy
Time Series

Abstract: Automated seizure detection and classification from electroencephalography (EEG) can greatly improve seizure diagnosis and treatment. However, several modeling challenges remain unaddressed in prior automated seizure detection and classification studies: (1) representing non-Euclidean data structure in EEGs, (2) accurately classifying rare seizure types, and (3) lacking a quantitative interpretability approach to measure model ability to localize seizures. In this study, we address these challenges by (1) representing the spatiotemporal dependencies in EEGs using a graph neural network (GNN) and proposing two EEG graph structures that capture the electrode geometry or dynamic brain connectivity, (2) proposing a self-supervised pre-training method that predicts preprocessed signals for the next time period to further improve model performance, particularly on rare seizure types, and (3) proposing a quantitative model interpretability approach to assess a model’s ability to localize seizures within EEGs. When evaluating our approach on seizure detection and classification on a large public dataset (5,499 EEGs), we find that our GNN with self-supervised pre-training achieves 0.875 Area Under the Receiver Operating Characteristic Curve on seizure detection and 0.749 weighted F1-score on seizure classification, outperforming previous methods for both seizure detection and classification. Moreover, our self-supervised pre-training strategy significantly improves classification of rare seizure types (e.g. 47 points increase in combined tonic seizure accuracy over baselines). Furthermore, quantitative interpretability analysis shows that our GNN with self-supervised pre-training precisely localizes 25.4% focal seizures, a 21.9 point improvement over existing CNNs. Finally, by superimposing the identified seizure locations on both raw EEG signals and EEG graphs, our approach could provide clinicians with an intuitive visualization of localized seizure regions.

Published as a conference paper at ICLR 2022
SELF-SUPERVISED GRAPH NEURAL NETWORKS FOR IMPROVED ELECTROENCEPHALOGGRAPICC SEIZURE ANALYSIS
Siyi Tang, Jared Dunnmon, Khaled Saab, Xuan Zhang', Qianying Huang', Florian Dubost, Daniel Rubin', Christopher Lee-Messer
Stanford University, CA, USA
fsiyitang, jdunnmon, ksaab, kayleez, qyhuang, fdubost, rubin, cleemess) istanford.ed)
i Equal contributions. $ Equal contributions.
ABSTRACT
Automated seizure detection and classification from electroencephalograpyy (EEG) can greatly improve seizure diagnosis and treatment. However, several modeling challenges remain unaddressed prior automated seizure detection and classification studies: (1) representing non non-Euulidean data structure in EEGs, (2) accurately classifying rare seizure types, and (3) lacking a quantitative inter- pretability approach measure model ability to localize seizures. In this study, we address these challenges by (1) representing the spatiotemporal dependencies in EEGs using graph neural network (GNN) and proposing two EEG graph structures that capture the electrode geometry or dynamic brain connectivity, (2) proposing self-supervised pre-training method that predicts preprocessed sig- nals for the next time period to further improve model performance, particu- larly on rare seizure types, and (3) proposing a quantitative model interpretabil- ity approach to assess a model's ability to localize seizures within EEGs. When evaluating our approach on seizure detection and classification on large pub- lic dataset (5,499 EEGs), find that our GNN with self-supervised pre-training achieves 0.875 Area Under the Receiver Operating Characteristic Curve on seizure detection and 0.749 weighted Fl-score seizure classification, outperforming previous methods for both seizure detection and classification. Moreover, our self-supervised pre-training strategy significantly improves classification of rare seizure types (e.g. 47 points increase in combined tonic seizure accuracy over baselines). Furthermore, quantitative interpretability analysis shows that our GNN with self-supervised pre-training precisely localizes 25.4% focal scizures, 21.9 point improvement over existing CNNs. Finally, by superimposing the identified seizure locations on both raw EEG signals and EEG graphs, our approach could provide clinicians with an intuitive visualization of localized seizure regions.
INTRODUCTION
Seizures are among the most common neurological emergencies in the world (Strein et al. 2019 Seizures can be chronic as in the case of epilepsy, neurological disease affecting 50 million people worldwide (WHO] 2019). Clinically, definitive detection of seizure is only the first step in seizure diagnosis. An important subsequent step to classify seizures into finer-grained types-such as focal versus generalized seizures- identifying epilepsy syndromes, targeted therapics, and cligi- bility for epilepsy surgery (Fisher et al. (017).
Scalp electreencephaloograhyy (or "EEG") plays critical role in seizure detection and classification. Clinically, EEG-based seizure detection and classification are performed by a trained EEG reader who visually examines patient's 's EEG signals over time periods ranging from hours days. How- ever, this manual analysis is extremely resource- and tnne-intensive. and thus automated algorithms could greatly accelerate seizure diagnosis and improve outcomes for severely ill patients.

---

Published as a conference paper at ICLR 2022
Figure 1: (a) EEG elec- trode placement in the standard 10-20 system. (b) Distance-based EEG graph. (c) An example correlation-based EEG graph. (d) Overview of our methods. The inputs to the models are the EEG graphs where each node feature corresponds to the preprocessed EEG signals in the respective channel. Self-edges are not shown for better visualization.
(b)
Lett
Right
3D View
Wiss Cornectiity
Baong Connactily by
(d)
Gird View
Decoder
Predictorn
Convolational Gatted Recurrent Livis
Encoder:
DOSRU
DCORU
Signals
Se Superviedd Pre- Training
EEG Signal
Prediction Layer
Preproontinng
Feb No Stlcur
EES Crach
Selzure Dotection
Layer
Vike
Selzure Classilication
Although large number of studies have attempted automated seizure detection (Rasheed et al., 2020 Siddiqui et al., 20200 Shocibi et al.
2021 O'Shea et al.] 2020; /aab et al., (2020) or seizure classification (Raghu et al. 2020] Asif et al. 2020 lesmantas & Alzbutas 2020] Ahmedt-Aristizabal et al., 2020; Roy et al., 2019), several challenges remain largely unaddressed. First, most recent studies use convolutional neural networks (CNNs) that assume Euclidean structures in EEG signals or spectrograms (Rasheed et al. 20200 Shhciib et al.) 2021 Raghu et al. 2020 Asif et al.
2020
lesmantas & Alzbutas AAmeed-
et al. 2020; Roy et al. 2019 O Shea et al. 2020
Saab et al.J (020).. However, assumption of Euclidean structure ignores the natural geometry of EEG electrodes and the connectivity in brain networks. EEGs are measured by electrodes placed on
manifold (i.e. patient's scalp) (Figure [T]), and thus have inherent Ruc-Fuclidean structures. Graphs are a data structure that can represent complex, non-Euclidean data (Chami et al., (020;; |roostein al.] (017). and graph theory has been extensively used in modeling brain networks (Bullmore & Sporns] 2009).. We therefore hypothesize that graph-basel modeling approaches can better represent the inherent non mon-Euclidenn structures in EEGs in a manner that improves both the performance and the clinical utility of seizure detection and classification models. Although traditional graph theory has been used (Supriya et al. 2021), only a few deep learning studies have modeled EEGs as graphs for seizure detection. However, these graph-based studies were limited to nonpublic (Covert et al., 2019) or small datasets (Craley et al. 2019 Li et al., 202T), did not leverage modern self-supervised approaches or cxamine seizure classification (Cisotto et al. 2020 Zhao et al. 2021 Li ct al. 202
Second, certain seizure types (e.g. clonic seizures) are rare by nature. Training machine learn- ing models that perform well on these rarer seizure classes using traditional supervised learning approaches is challenging, which could explain the performance difference between major and mi-
nority seizure types in prior studies (Raghu et al.] 22200 lesmantas & Alzbutas 2020; [Ahmedt-
Aristizabal et al. 2020). Several studies have investigated an alternative, self-supervised training strategy (Banville et al., 2020 Mohsennand et al.) 2020; Kostas al. [02T Martini et al.) 2021
Xu et al. 2020). but they did not model EEGs as graphs or address automaatdd seizure classification.
Prior works have shown that self-supervised pre-training significantly improves model performance on data with imbalanced labels in the field of computer vision (Yang & Xu 2200;) Liu et al. (02T). Hence, we hypothesize that self-supervised pre-training can help improve our graph model per- formance on rare seizure types. Moreover, a large portion of EEG signals generally do not have seizures. Self-supervised pre-training strategy would allow the model to leverage the abundant non- seizure EEGs that are readily available in the dataset.
Lastly, for seizure detection and classification models, the ability to not only provide a single pre- diction across all EEG channels, but also to provide the interpretability and the ability to localize seizures would be clinically useful for informing treatment strategy. While prior studies (Saab et al.) 2020 Covert et al.] 2019) have shown qualitative visualization for model interpretability, none have quantitatively assessed the model's ability to localize seizures.
In this work, aim to address these limitations in prior automated seizure detection and classifica- tion studies. First, we propose a graph- based modeling approach for EEG-based seizure detection and classification. Specifically, propose two EEG graph structures that capture BEG sensor
2

---

Published as a conference paper at ICLR 2022
dynamic brain connectivity (Figure I and extend Diffusion Con- volutional Recurrent Neural Network (DCRNN) (Li et al. 2018 an RNN with graph diffusion convolutions, to model the spatiotemporal dependencies in EEGs (Figure TTt.. Second, we improve DCRNN performance using a self-supervised pre-training strategy of predicting the preprocessed EEG signals for the next time period without requiring additional data or labels. Finally, we propose quantitative metrics to assess our model's s ability to localize seizures. In summary:
geometry (Figure 1p)
We propose two EEG graph structures that capture (1) the natural geometry of EEG sensors or (2) dynamic connectivity in the brain, and show that building recurrent graph neural network (GNN) based on these representations yields models for seizure detection and classification that outperform previous approaches on large public dataset (5,499 EEGs).
We propose a superved pre-training strategy to further improve our recurrent GNN model performance, particularly on rare seizure types. To our knowledge, our study is the first to date that combines graph-based modeling and self-supervised pre-training for EEGs. By leveraging graph structure and self-supervision, our method achieves 0.875 Area Under the Receiver Operating Characteristic Curve (AUROC) seizure detection and 0.749 weighted Fi-score on seizure classification, outperforming previous approaches both seizure detection and classification this large public dataset. Moreover, our self- supervised pre- pre-training method substantially improves classification of rare seizure types (e.g. 47 points increase in combined tonic seizure accuracy over baselines).
We propose a quantitative model interpretability analysis that can be used to assess a model's ability to localize seizures, which is critical to determining the course of treat- ment for seizures. We show that by leveraging graph structure and self-supervision, our method precisely localizes 25.4% focal seizures, providing 21.9 points improvement over a prior state-of-the-art CNN. Finally, by displaying the identified seizure regions on raw EEG signals and EEG graphs, our approach could provide valuable insights that support more effective seizure diagnosis in real-world clinical settings.
METHODS
2.1 SEIZURE DETECTION AND CLASSIFICATION PROBLEM FORMULATION
The goal of seizure detection is to predict whether a seizure exists within an EEG clip, and the goal of seizure classification is to predict the seizure type given a seizure EEG clip. Following prior study (Saab et al., 20200) we examine our model's capability for fast and slow detection and classification over 12-s and 60-s EEG clips, respectively.
2.2
GRAPH-BASER MODELING FOR EEGS
2.2.1
REPRESENTING EEGS AS GRAPHIS
We represent an EEG clip as a graph G = EV, WJ. where V denotes the set of nodes (i.e. EEG electrodes/channels), E denotes the set of edges, and W is the adjacency matrix. We propose the following two methods of constructing the EEG graph.
Distance graph. To represent the natural geometry of EEG electrodes, compute edge weight Wi by applying a thresholded Gaussian kernel (Shuman et al. 2013) to the pairwise Euclidean
dist( (v; 1z)² o²
distance between U; and ui, i.e., Wi = =exp
dist(w;, Uy) < K, else 0. Here,
dist(r;, vz) the Fuclidean distance between electrodes v and vy according to the standard 10-20 EEG electrode placement (Jasper 1958). a is the standard deviation of the distances, and is the threshold for sparsity. This results in a universal undirected, weighted graph for all EEG clips. Based on preliminary experiments and BEG domain knowledge, we chose k = 0.9 because it results in reasonable graph that also resembles the EEG montage (Iongitudinal bipolar and transverse bipolar) widely used clinically (Acharya al. 2016). Figure [p shows the distance graph with 0.9. In Appendix K we explore different values of x as well as constructing the distance graph using a Gaussian kernel with pre-specified bandwidth.
3

---

Published as a conference paper at ICLR 2022
Correlation graph. To capture dynamic brain connectivity, we define the edge weight Wiy the absolute value the normalized cross-corvelation between the preprocessed signals in v and v/-- To introduce sparsity to the graph, only the edges whose weights are among the top-T neighbors of each node are kept (plus self-edges), i.e., Wij X x.jsl I if vj E [W()), else O. Here, .t; and :j:: are preprocessed signals in u, and vij represents the normalized cross-correlation, and (w,) represents the top-r neighbors of v, This method results in unique directed, weighted graph for each input EEG clip. Figure shows an example correlation graph with 7 =
2.2.2 GRAPH NEURAL NETWORK
We adapt DCRNN (Li al. 2018), recurrent neural network with graph diffusion convolutions, to model the spatiotemperal dependencies EEG signals. DCRNN was initially developed for traffic forecasting, where the dynamics of traffic flow modeled a diffusion process. Similarly, can also model the spatial dependency in EEG signals as a diffusion process, because an electrode can be influenced more by electrodes in its anatomical proximity (measured by distance) (Acharya et al. 2016) or functional proximity (measured by correlation) (Sakkalis) 2011). Specifically, the diffusion process is characterized by bidirectional random walk on directed graph G, which results in the following diffusion convolution (Li et al. 2018):
K-
X X fo
(0m,(DD w + (D. X.m for m E (1, (1,...,M M}
(1)
A=
where X E IR NxM is the preprocessed EEG clip at time step t E {1...., T} with nodes and M features, fo is the convolution filter with parameters 0 R Do and Dj are the out-degree and in-degree diagonal matrices of the graph, respectively, Do' W and D are the state transi- tion matrices of the outward and inward diffusion processes, respectively, and K is the number of maximum diffusion steps.
For undirected graphs. the diffusion convolution is similar to ChebNet spectral graph convolution (Defferrard et al., 2016) up to constant scaling factor, and thus can be computed using stable Chebyshev polynomial bases as follows eLi al.
2018):
K
K
X mm = 12
Σ k=0
O&A
T
Σ
0g X mm = Σ
(L)x( for m E (l,..., M k=0
k=
where To(x) = (x) = x, and Tx(x) = 2xTh (x) - T2-2(x) for k 2 2 are the bases of the Chebyshev polynomial, L = D DD WDD- T is the normalized graph Laplacian, and L Amez I is the scaled graph Laplacian mapping eigenvalues from [0, Amar] [-1, 1]. We use Equation 1 for directed correlation graphs, and Equation 2 for undirected distance graph.
Next, to model the temporal dependency in EEGs, we employ Gated Recurrent Units (GRUs) (Cho al. 2014), variant of RNN with a gating mechanism. Specifically, the matrix multiplications in GRUs are replaced with diffusion convolutions (or ChebNet spectral graph convolutions for undi- rected distance-based graph) (Li al. 2018), allowing spatiotemporal modeling of EEG signals (referred to as "DCGRU"):
H ((-1)) b..) ( = -tanh( (4) (1 o HH (( H(() (t) O H (1-1) (1 (1- o
(t) - (+) (1-)) + b.)
" = G
(3)
(4)
Here, (t), (t) denote the input and output of DCGRU at time step respectively, denotes Sigmoid function, @ represents the Hadamard product, (())) (1) (itt) denote reset gate, update gate and candidate at time step respectively. *G denotes the diffusion convolution (or ChebNet spectral graph convolution), O,, b,,, o.. b.. Og and be are the weights and biases for the corresponding convolutional filters. Finally, for seizure detection and classification, the models consist of several stacked DCGRUs followed by a fully-connected layer.
2.3 SELF-SUPERVISED PRE-TRAINING
To further improve DCRNN performance, we propose a self-supervised pre-training strategy for EEGs. Specifically, we pre-train the model for predicting the next T' second preprocessed EEG
4

---

Published as a conference paper at ICLR 2022
clips given a preprocessed 12-s (60-s) EEG clip. We hypothesize that by learning to predict the EEG signals for the next time period, the model would learn task-agnostic representations and improve downstream seizure detection and classification tasks. The model for self-supervised pre-training has a sequence-t--sequ
architecture with an encoder and a decoder (Sutskever et al.] 2014), each of which has several stacked DCGRUs (Figure [d]. We use true preprocessed EEG clips and the predicted clips as the loss function. Preliminary experiments suggest that predicting future T" 12 second preprocessed EEG clips results in low regression loss on the validation set, and thus we use T 12 in all self superviee pre- wre-training experiments.
absolute error between the
2.4 MODEL INTERPRETABILIT AND ABILITY TO LOCALIZE SEIZURES
We perform model interpretability analyses using an occlusion-based approach (Zeieer Fergus, 2013). First, for seizure detection, we zero-fill one second EEG signals in one channel at time and compute the relative change the model's output logit with respect to the original non-occluded output (original occluded). Furthermore, we scale the values in each clip to [0, 1] using the minimum and maximum values in that clip. This results in occlusion map M E R Nx where N is the number of EEG channels, T is the clip length, and Me, indicates the relative change in the model output when the j-th second EEG clip in the i-th channel is occluded. A larger Mi indicates that the occluded region is more important for predicting seizure, and vice versa. We visualize M by superimposing it over EEG signals and graph structures.
In addition to visually analyzing the aforementioned occlusion maps, we propose quantitative met- rics evaluate the model's capability of localizing seizures based the occlusion maps. Specif- ically, we define a coverage metric that quantifies how many true seizure regions are detected, as well as localization metric that quantifies how many detected seizure regions are true seizure re- gions. Note that coverage and localization scores are analogous to recall and precision, respectively, in binary classification problems. Mathematically, let - E gR NN x be detailed annotations of seizure duration and location, where M if there is seizure the j-th second in the i-th channel, otherwise Mi = O. Let M E RRT be the occlusion map described above. Then
coverage = Σi, M., M.5 Σ.,, Mo
and localization = Ees 1 M., 0.5 M.
Σ., M,; > 0.5
Finally, we also perform occlusion based interpretability analysis for seizure classification. Since the goal of seizure classification is to predict the seizure type given seizure EEG clip, we hypothesize that the difference in signals between EEG channels are more important for predicting seizure types. Therefore, we completely drop one EEG channel at a time, and compute the relative change in the model output with respect to the original output. This results in an occlusion map M' R where M indicates the relative change in the model output when the i-th channel is dropped.
3 EXPERIMENTS
3.1 EXPERIMENTAL SETUP
Dataset. We use the public Temple University Hospital EEG Seizure Corpus (TUSZ) v1.5.2 (Shah et al. 2018 Obeid & Picone, (0166). the largest public EEG seizure database to date with 5,612 EEGs, 3,050 annotated seizures from clinical recordings, and eight seizure types"] We include 19 EEG channels in the standard 10-20 system (Figure Te)) To evaluate model generalizability to unseen patients, we exclude five patients from the official TUSZ test set who exist both the official TUSZ train and test sets. Moreover, we use the detailed annotations of seizure duration and location available in TUSZ for our interpretability analyses. Table summarizes the TUSZ data.
Data preprocessing. Because seizures are associated with brain electrical activity in certain fre- quency bands (Tzallas et al., 2009). we perform data preprocessing to transform raw EEG signals to the fireueecyy domain. Similar to prior studies (Asif et al.. 2020: Ahmedt- Aristizaaat et al. 2020 Covert et al.) 2019). we obtain the tog-ampittudes of the fast Fourier transtorm of raw EEG sig- nals. For seizure detection and self-supervised pre-training,
both seizure and non-seizure
'The eight seizure types are: focal, generalized non-specific, simple partial, complex partial, absence, tonic, tonic-clonie, and myoclonic seizures.
5

---

Published as a conference paper at ICLR 2022
EEGs, and obtain the 12-s (60-s) EEG clips using non-overlapping 12-s (60-s) sliding windows. For seizure classification, we use only seizure EEGs, and obtain one 12-s (60-s) EEG clip from each seizure event, such that each EEG clip has exactly one seizure type. Appendix A presents details of data preprocessing, and Appendix [ compares results of frequency-domain vs time-domain inputs.
Refined seizure classification scheme. Because simple partial (SP) seizures and complex partial (CP) seizures are focal seizures characterized by a clinical behavior (consciousness during a seizure) and are not distinguishable by EEG signals alone (Fisher et al. (017)) we combine focal non-specific (FN), SP, and CP seizures to form a combined focal (CF) seizure class. We provide extensive experiments Appendix [ showing that these focal seizure types cannot be distinguished by a variety of models. We also exclude myoclonic seizures because only three myoclonic seizures are available in TUSZ. In addition, since tonic and tonic-clonic seizures are rare in the dataset (only 18 tonic seizures and 30 tonic-clonic seizures in TUSZ. train set), and tonic-clonic seizures always start with a tonic phase (Fisher et al., 2017). we combine tonic-clonic seizures with tonic seizures to form combined tonic (CT) seizure class. In summary, there are four seizure classes in total: CF, generalized non-specific (GN), absence (AB), and CT seizures (Table T.. In Appendix L seizure classification results on the original eight seizure types in TUSZ.
we show
Data splits. We randomly split the official TUSZ train set by patients into train and validation sets by 90/10 for model training and hyperparameter tuning, respectively, and we hold-out the official TUSZ test set for model evaluation (excluding five patients who exist in both the official TUSZ train and test sets). The train, validation, and test sets consist of distinct patients. See Appendix D for the number of preprocessed EEG clips and patients in each split.
Table I: Summary of data in train and test sets of TUSZ v1.5.2. used in our study.
EEG Files (% Seizure)
Putients (%6 Seizure)
Total Duration (% Seizure)
CF Seizures (Patients)
GN Seizures
AB Seizures (Patients)
CT Seizures (Patients)
(Patients)
Train Set
4.599 (18.9%)
(34.1%))
77.772 min (6.3%)
1,868 (1 (148)
(68)
(11)
Tost Set
900 (25.6%)
45 (77.8%)
9.031.58 min (9.8%)
297 (24)
114 (11)
49 (5)
61 (4)
Baselines. To compare our DCRNN to traditional CNNs/RNNs, we include three primary base- lines: (a) CNNN (Saab et al. 2020), previous state-of-the-aa CNN for seizure detection, (b) LSTM (Hochreiter & Schmidhuber 1997), , and (c) CNN-LSTM (implemented following [Ahmedt- Aristizabal et al. (2020p). The biselines are trained and evaluated on the same preprocessed data. Additionally. we compare our method to the reported results of two CNNs for seizure classification that use TUSZ and test on unseen patients (Asif et al. 2020: Jesmantas & Alzbutas, 2020).
Model training. Training for all models was accomplished using the Adam optimizer (Kingmal Ba 2014) in PyTorch on a single NVIDIA Titan RTX GPU. Model parameters were randomly initialized for models without self-supervised pre-training, and were initialized with the pre- pre-trained weights of the encoder for models with self-supervised pre-training. All models were run for five runs with different random seeds. Detailed hyperparameter settings are shown in Appendix E Dur- ing training, we perform data augmentation as described in Appendix
3.2 EXPERIMENTAL RESULTS
Graph neural network performance. Following recent studies (Asif et al. 2020 Ahmedt-
AUROC and weighted Fl -ccoee as the main evaluation metrics for seizure detection and classification, re- spectively. Table 2 (3rd-7th rows) shows the performance of our DCRNN (without self-supervised pre-training) and The baselines. Distance graph-based DCRNN (or "Dist-DCRNN" and correlation graph-based DCRNN (or "Cor-DDRRNN") without self superviee pre-training perform on par with or better than the baselines. See Appendix G for additional evaluation scores.
Aristizabal al. 2020 O' Shea et al. 2020 Saab et al. 2020 Covert et al. 2019).
Moreover, DCRNN outperforms the reported results of two existing CNNs (Asif et al., 2020 Arinttt-rrittzaaaall et al. 2020) on seizure classification (Table B.. For fair comparison to Asif et al. 2020), we conduct -llass seizure classification on the same 3-fold patient-wise split.
"hhe 7 classes the original TUSZ. seizure types excluding myoclinic seizure.

---

Published as a conference paper at ICLR 2022
In addition, we conduct an ablation experiment to examine the effectiveness of Fourier transform in our preprocessing step for DCRNNs. We find that frequency-domate inputs substantially outperform time-domain inputs on both seizure detection and classification (Appendix B
Table 2: Seizure detection and seizure classification results. Mean and standard deviations are from five random runs. Best non- non-pretrained and pre-traine pre-trained mean results are highlighted in bold.
Seizure Detection AUROC
Seizure Classification Weighted F1-Score
Model
12-8
60-s
12-s
60-s
Dense-CNN
0.812 + 0.014
0.796 + 0.014
0.576 + 0.101
0.626 + 0.073
LSTM
0.786 1 0.014
0,715 1 0.016
0.652 1 0.019
0.686 + 0.020
CNN-LSTM
0.749 +: 0.006
0.682 1 0.003
0.633 0.025
0.641 + 0.019
Con-DCRNN w/o Pre-training
0.812 = 0.012
0,804 + 0.015
0,710
0.023
0.701 = 0.030
Dist-DCRNN w/o Pre-training
0.824 0.020
0.793 + 0.022
0.703 0.025
0.690 0.035
Con-DCRNN Pre-training
0.861
0.005
0.850 + 0.014
0.723 0.017
0.749 0.017
Dist-DCRNN w/ Pre-training
0,866 0.016
0.875 1 0.016
0.746 0.024
0.749 0.028
Table 3: Comparison between DCRNNs (w/o pre-training) and existing CNNs on seizure classifica- tion. Mean and standard deviations are from five random runs. Best mean results are in bold.
7-Class Classification Weighted F1-Score
CF Seizure AUROC
GN Seizure AUROC
AB Seizure AUROC
CT Seizure AUROC
Model
Asif et al. 2020
0.62
Tesmunins
AlPhutas
2020
0.72
Con--DRNN 12-s
0.619 0.006
0,907 0.008
0,815
0.027
0.972 = 0.013
0.908 + 0.005
Dist DCRNN, 12:s
0.585 0.006
0.896 0.011
0.814
0.027
0,983
0.008
0.890 0.014
Con-DDCNNN 60-s
0.650 0.008
0,914 0.007
0.795
0.031
0.971 0.020
0.939 = 0.008
Dist Dist-DCRNE DOBNDD 60.s
0.606 0.009
0.920 0.004
0.811 1 0.032
0,973 0.009
0.926 1 0.020
Self-supervised pre-training improves graph neural network performance. To assess the ef- fectiveness of self-supervised pre-training in improving DCRNN's performance, we compare the results of DCRNN without and with self-supervised pre-training. As shown in Table 2 (last two rows), DCRNN with self-supervised pre-training outperforms its non-pretrained counterpart on both seizure detection and classification. Notably, Dist-DCRNN with self-supervised pre-training achieves an AUROC of 0.875 for 60-s seizure detection, matching the performance of CNN (AU- ROC=0.88) that was pre-trained using supervised learning on labeled dataset that was five times larger than TUSZ (Sab et al. 2020). Importantly, the pre-trained model weights are used for both seizure detection and classification, which indicates that our sulf-supervised pre-training method provides good model initialization that generalizes across tasks.
Figure P shows the ROC curves of median DCRNNs and baselines for seizure detection. At a low false positive rate (FPR), such as 25% FPR, pre-trained Dist-DCRNN achieves 84.3% true positive rate (TPR) and pre-trained Corr-DCRNN achieves 81.7% TPR on 12-s clips. Conversely, Dense- CNN, LSTM, and CNN-LSTM only achieve 72.8%, 67.0%, and 62.5% TPRs, respectively. Figure 2b shows the confusion matrices for Dist-DCRNNs and baselines for 12-s seizure classification. without self-supervised pre-training achieves 93% accuracy on the rare AB seizures, providing points increase over the best baseline. Furthermore, DCRNN with self-supervised pre-training achieves 74% accuracy on the rare CT seizures, providing 47 points increase in accuracy over the best baseline (Dense-CNN) and 48 points increase over non- pon-preteai Dist-DCRNN.
Surprisingly, despite being a majority class, many GN seizures are misclassified as CF seizures (Figure 2)). A board-certified neurologist manually analyzed the EEGs of 32 misclassified test GN seizures. We find that 27 seizures are in fact focal seizures but are mislabeled as GN seizures. In contrast, only 5 seizures are indeed generalized seizures but are misclassified by our models.
Comparison between self-supervised pre-training and transfer learning. To compare self- supervised pre-training strategy to traditional transfer learning approaches, we pre-train DCRNNs for seizure detection and self-supervised prediction, respectively, FEGs, Table 7 and finetune the models for seizure detection and classification on TUSZ. We find that self self-super pre-training consistently outperforms transfer learning (Appendix[).
large in-house dataset (40,316
Comparison between self-supervised pre-training and auxiliary learning. We also investigate whether using the self-supervised task auxiliary task can outperform self-supervised pre-
7

---

Published as a conference paper at ICLR 2022
(a)
60-s ROC Curves
1.0
0.0
0.0
0.5
(a) ROC curves of median models for seizure de- tection. (b) Confusion matri- ces (averaged across five ran- dom runs) for the baselines and Dist-DCRNN without and with superviee pre-training for 12-s seizure classification. Each row of the confusion matrices is normalized by dividing by the number of examples in the cor- responding class, such that each row sums up one.
Figure 2:
0.4
Can sclee su pRandc-c BN
0.4
Oar coeiss Porose 24 sscuaats
0G0 DuLV meee ong MAROO (900)
0.2
0.2
Lat LMBDEH6 71)
0.0
0.6 Fabal Posdio
80
Faloo Pradlira Ratu
(b)
Dist-DCRNN wio Pre- Traininnn
Dist-DCRNN Pre- Training
Dense-CNN
LSTM
CNN-LLTTM
832
0.10
2:55
237
542
027
0.06
0.10
0.03
0.0
0.01
0.41
0.00
0.55
852
382
1.0
0.05
000
6054
0.00
CT-
0:87
3.30
0.27
0UM
0:31
0022
0.22
8.66
0.37
1.0
(8)
(b)
Figure 3: Occlusion maps for seizure detection obtained from Cor--DCRN model for (a)-b) focal, and (c)-(d) generalized seizures. In each subfigure, left panel shows the occlusion map for 12-s when a seizure occurs and right panel shows the occlu- sion map values averaged along time and overlaid on the corre- sponding correlation graph.
(c)
(d)
Low Swisncy
High Salency
training. We find that auxiliary learning performs comparable to celf-supervised pre-training on 12-s seizure detection, whereas self-supervised pre-training significantly outperforms auxiliary learning on 60-s seizure detection. See Appendix M and Table 10 for details.
Improved model interpretability and ability to localize seizures. We leverage occlusion- based techniques to localize seizures using our model predictions. Figure]]shows example occlusion maps of correctly predicted 60-s test EEG clips from Corr-DCRNN with self-supervised pre-training. We observe that high saliency for focal seizures (Figures 3D localized in the more abnormal EEG channels, whereas high saliency for generalized seizures (Figures 3 3d) is more diffuse across channels. These patterns reflect the underlying brain activity of focal seizures (i.e. localized in one brain area) and generalized seizures (i.e. oceur in all areas of the brain) (Fisher et al., 2017). Moreover, one can also interpret the seizure locations from the occlusion map overlaid on the graphs. In Figure 3. Fp2, F4 and F8 are highlighted on the graph occlusion map, which correspond the right frontal, central frontal, and anterior frontal brain regions that show the most abnormality in the EEG. In Figure 3b, while the seizure starts in P3, Fpl, F3, and F7, the graph occlusion map mainly highlights the central parietal region (P3), which is likely due to the ongoing artifact in channels Fp1-F7, FPI-F3, and F3-C3. Figure 3c-3d show two generalized seizures that occur in all regions of the brain, and the highlighted nodes on the graphs are indeed more spread out across channels. In contrast, high saliency from Dense-CNN does not localize any seizure regions (Appendix H).
Additionally, we leverage detailed annotations of seizure duration and location available in TUSZ dataset, and use coverage and localization scores to quantify the models' ability to accurately local- ize seizures. Figure 4 shows coverage and localization distributions for Dense-CNN and DCRNNs (w/o and with pre-training on correctly predicted 60-s test EEG clips. Both Dist-DCRNN and Con-DCRNN have many more occlusion maps with high coverage and high localization scores than Dense-CNN This suggests that DCRNN more accurately localizes seizures than Dense-CNN, which is intuitive given that DCRNN captures the connectivity of ERG electrodes.
8

---

Published as a conference paper at ICLR 2022
Clinically, localizing focal seizure onset regions is key to epilepsy surgery for patients with focal seizures. Thus, the model's ability to identify focal seizure regions as precisely as possible would be highly desirable. Notably, pre-trained pre-trained Dist-DCRNN precisely localizes 21 21.8% focal seizures (localization score > 0.8, Figure Ap). Conversely, both non-pretrained Dist-DCRNN and Corr-DCRNI precisely localize 6.3% focal seizures, and Dense-CNN only localizes 3.5% focal seizures precisely (localization score > 0.8). Figures 4 show example occlusion maps of test focal seizures whose localization scores > 0.9 from pre-trained Corr- DCRNN, where high saliency overlaps well with annotated seizure regions.
precisely localizes 25.4% focal seizures and
Lastly, interpretability experiments on seizure classification show that high saliency EEG channels correspond to where the seizures are localized in CF seizures (Appendix(I... This again suggests that our method could inform seizure locations for treatment strategy if used clinically.
(9)
Con eo pletraa
Con CCRV pn-ee
Craages Dastorion (Canwrs and Seirume
Locaktatton Dadioulin (Gamard Dac Satoure)
DA--019 or SEEDE
Figure 4: Distributions of (a)-(b) coverage and (c)(d) localization scores for Dense-CNN and DCRNNs correctly predicted 60-s EEG clips for focal and generalized seizures. (c)-()) Example occlusion maps with focal seizure whose localization >0.9 from pre-trained regions with normalized occlusion value >05 are colored. Red boxes annotated seizure regions.
Only
Comparison between graph structures. While Dist-DCRNN and Com-DDRNN perform compa- we observe that Cor-DDCRN
rable to each other on seizure detection and classification (Table 2). better localizes focal seizures than Dist-DCRNN particularly when combined with self-supervised pre-training (e.g.. coverage and localization scores > 0.9 in Figure f.. + This suggests that the correlation- basedd based graph structure could provide better interpretabilityy and representation of focal seizure EEGs. Moreover, the correlation graph structure has two particular advantages: (a) it can be used even when the physical locations of electrodes are unknown and (b) captures dynamic brain connectivity rather than replying purely on spatial sensor information, which is particularly desirable for automated seizure detection and classification models.
4 CONCLUSION
In conclusion, we present novel method combining graph-based modeling and self-supervised pre- training for EEG-based seizure detection and classification, as well as an interpretability method to quantify model ability to localize seizures. Our method sets new state-of-the-art on both seizure detection and classification on a large public dataset, significantly improves classification of rare seizure classes, and more accurately localizes seizures. We also find that the correlation-based graph more accurately localizes focal seizures than distance-based graph. The improved ability to localize seizures and the novel graph visualizations could provide clinicians with valuable insights about localized seizure regions in real-world clinical settings. Looking to the future, because our methods are not confined to EEG alone, our study opens exciting opportunities to build graph-based representations for a wide variety of medical time series.

---

Published as a conference paper at ICLR 2022
ACKNOWLEDGMENTS
This work was supported by Wu Tsai Neurosciences Institute Neuroscience Translate Grant. The authors would like thank Bibek Paudel, Liangqiong Qu, Jean Benoit Delbrouck, and Nandita Bhaskhar for their feedback on the paper.
ETHICS STATEMENT
The Temple University Hospital EEG Seizure Corpus used in our study is anonymized and publicly available with full IRB approval (Shah et al. 2018 Obeid & Picone 2016). No conflict of interest reported from the authors. No harmful insights are provided by the seizure detection and classifi- cation models described in this study. Although we show that our methods could provide improved performance and clinical utility for seizure detection and classification, additional model valida- tions are needed before they can be used in real-world clinical settings, including (a) validation datasets from multiple institutions, (b) validation on different populations and age groups, and (c) validation EEG waveform- based seizure classification scheme that is approved consensus of board-certified neurologists.
REPRODUCIBILITY STATEMENT
The Temple University Hospital EEG Seizure Corpus used in our study is publicly available. De- tailed data preprocessing steps are provided in Appendix A Source code publicly available at
REFERENCES
Jayant N Acharya, Abeer J Hani, Parthasarathy Thirumala, and Tammy N Tsuchida. American clinical neurophysiology society guideline 3: proposal for standard montages to be used in clinical eeg. The Neurodiagnostic Journal, 56(4):253-26)) 2016.
David
Tharindu Fernando, Simon Denman, Lars Petersson, Matthew J Aburn, and Clinton Fookes. Neural memory networks for seizure type classification. In 2020 42nd Annual International Conference of the IEEE Engineering in Medicine & Biology Society (EMBC), pp. 569-575. IEEE, 2020.
Umar Asif, Subhrajit Roy, Jianbin Tang, and Stefan Harrer. Seizurenet: Multi-spectral deep feature learning for seizure type classification. In Seyed Mostafa Kia, Hassan Mohy-ud Din, Ahmed Abdulkadir, Cher Bass, Mohamad Habes, Jane Maryam Rondina, Chantal Tax, Hongzhi Wang. Thomas Wolfers, Saima Rathore, and Madhura Ingalhalikar (eds.), Machine Learning in Clinical Neuroimaging and Radiogenomics in oncoloonc pp. 77-87, Cham, 2020. Springer Interna- tional Publishing.
Hubert Banville, Omar Chehab, Aapo Hyvarinen, Denis Engemann, and Alexandre Gramfort. Un- covering the structure of clinical eeg signals with self-supervised learning. Journal of Neural Engineering, 2020.
Michael M Bronstein, Joan Bruna, Yann LeCun, Arthur Szlam, and Pierre Vandergheynst, Geomet- ric deep learning: going beyond euclidean data. IEEE Signal Processing Magazine, 34(4):18 2017.
Ed Bullmore and Olaf Sporns. Complex brain networks: graph thcoretical analysis of structural and functional systems. Nature reviews neuroscience, 16-3):1861198 2009.
Ines Chami, Sami Abu-El-Haija, Bryan Perozzi, Christopher Ré, and Kevin Murphy. Machine learning on graphs: model and comprehensive taxonomy. arXiv preprint arXiv:2005.03675, 2020.
https: wwwp///wwwwwtt/m///o/////// isip http:////wwaaaaaaoooo/////////ta
10

---

Published as a conference paper at ICLR 2022
Kyunghyun Cho, Bart van Merriënboer, Dzmitry Bahdanau, and Yoshua Bengio. On the properties of neural machine translation: decoderr approaches. In Proceedings of SSST-8, Eighth Workshop Syntax, Semantics and Structure in Statistical Translation, pp. 103-111, Doha, Qatar, October 2014. Association for Computational Linguistics. doi: 10.31 URL htt.t:tppps/.. /www. ac www. www..//ee///.oooo.o.oooooooo/
4-4012
Giulia Cisotto, Alessio Zanga, Joanna Chlebus, Italo Zoppis, Sara Manzoni, and Urszula Markowska-Kaczmaa. Comparison of attention-basen deep learning models for eeg classifica- tion. arXiv preprint arXiv:20/2.01074. 2020.
Ian C. Covert, Balu Krishnan, Imad Najm, Jiening Zhan, Matthew Shore, John Hixson, and Ming Jack Po. Temporal graph convolutional networks for automatic seizure detection. In Pro- ceedings of the 4th Machine Learning for Healthcare Conference, volume 106 of Proceedings of Machine Learning Research, pp. 160-180, Ann Arbor, Michigan, 09-10 Aug 2019. PMLR.
Jeff Craley, Emily Johnson, and Archana Venkataraman. Integrating convolutional neural networks and probabílistic graphical modeling for epileptic seizure detection in multichannel eeg. In In- ternational Conference on Information Processing in Medical Imaging, pp. 291-303. Springer, 2019.
Michaël Defferrard, Xavier Bresson, and Pierre Vandergheynst. Convolutional neural networks graphs with fast localized spectral filtering. Advances in neural information processing systems, pp. 3844 3844-3852, 2016.
Robert S. Fisher, J. Helen Cross, Jacqueline A. French, Norimichi Higurashi. Edouard Hirsch, Floor E. Jansen, Lieven Lagae, Solomon L. Moshé, Jukka Peltola, Eliane Roulet Perez, Ingrid E. Scheffer, and Sameer M. Zuberi. Operational classification of seizure types by the international league against epilepsy: Position paper of the ilae commission for classification and terminology. Epilepsia, 58(4):522-533, 2017. doi: https://doi.org/10.. 1ttp://....iipp.p111// 13670.
Sepp Hochreiter and Jürgen Schmidhuber. Long short-term memory. Neural computation, 9(8):
1735-17888 1997.
Tomas lesmantas and Robertas Alzbutas. Convolutional neural network for detection and classifi- cation of seizures in clinical data. Medical & Biological Engineering & Computing, 58, 06 2020. doi: 15110000
Herbert H Jasper. The ten-twenty electrode system of the international federation. Electroen- cephalogr. Clin. Neurophysiol., 10:370-375, 1958.
Diederík Kingma and Jimmy Ba. Adam: A method for stochastic optimization. International Conference Learning Representations, 12 2014.
Demetres Kostas, Stephane Aroca-Ouellette, and Frank Rudzicz. Bendr: contrastive self-supervised learning task learn from massive amounts of eeg data, 2021.
using transformers and
Yaguang Li, Rose Yu, Cyrus Shahabi, and Yan Liu. Diffusion convolutional recurrent neural net- work: Data-driven traffic forecasting. In 6th International Conference on Learning Representa- tions, ICLR 2018, Vancouver, BC, Canada, April 30- May 3, 2018, Conference Track Proceed- ings, 2018.
Yang Li, Yu Liu, Yu-Zhu Guo, Xiao-Feng Liao, Bin Hu, and Tao Yu. Spatio-temporal-speerral hierarchical graph convolutional network with semisupervised active learning for patient-specific seizure prediction. IEEE Transactions on Cybernetics, 2021.
Hong Liu, Jeff Z HaoChen, Adrien Gaidon, and Tengyu Ma. Self-supervised learning is more robust to dataset imbalance. arXiv preprint 10X0v:21 2021.
Ilya Loshchilov and Frank Hutter. SGDR: stochastic gradient descent with warm restarts. In 5th International Conference on Learning Representations, ICLR 2017, Toulon, France, April 24-26, 2017, Conference Track Proceedings, 2017.

---

Published as a conference paper at ICLR 2022
Michael L Martini, Aly A Valliani, Claire Sun, Anthony B Costa, Shan Zhao, Fedor Panov, Saadi Ghatan, Kanaka Rajan, and Eric Karl Oermann. Deep anomaly detection seizures with paired sterooelettoennephallggaapp and video recordings. Scientific Reports, 11(1):1-11, 2021.
Mostafa Neo Mohsenvand, Mohammad Rasool Izadi, and Pattie Maes. Contrastive representation learning for electroencephalogran classification. In Machine Learning for Health, pp. 238-253. PMLR, 2020.
Iyad Obeid and Joseph Picone. The temple university hospital eeg data corpus. Frontiers in neuro- science, 10: 10,, 2016.
Alison O' O'Shaa Gordon Lightbody, Geraldine Boylan, and Andriy Temko. Neonatal seizure de- tection from raw multi-channel EEG using fully convolutional architecture. Neural Net- works, 123:12-25, 2020. ISSN 08936080. doi: 10. n0.10166..eumee22219.1..1
URL https:
//linkinghub. www.weeeeeeee..eee http://wwww.......o.oo.o.oo..aaa
Shivarudhrappa Raghu, Natarajan Sriraam, Yasin Temel, Shyam Vasudeva Rao, and Pieter Kubben. Eeg based multi-class seizure type classification using convolutional neural network and transfer learning. Neural Networks, 124:202-212, 2020.
Khansa Rasheed, Adnan Qayyum, Junaid Qudir, Shobi Sivathamboo, Patrick Kwan, Levin Kuhlmann, Terence O' Brien,, and Adeel Razi. Machine learning for predicting epileptic seizures using eeg signals: A review. IEEE Reviews in Biomedical Engineering, 14: 14:13--155, 2020.
Subhrajit Roy, Umar Asif, Jianbin Tang, and Stefan Harrer. Machine learning for seizure type classification: setting the benchmark. arXiv preprint arXiv:1902.01012, 2019.
Khaled Saab, Jared Dunnmon, Christopher Ré, Daniel Rubin, and Christopher Lee-Messer. Weak supervision as an efficient approach for automated seizure detection in electreencephalography. upj Digital Medicine, 3(1):1-12, 2020.
Vangelis Sakkalis. Review of advanced techniques for the estimation of brain connectivity measured with eeg/meg. Computers in biology and medicine, 41(12):11 2011.
Vinit Shah, Eva von Weltin, Silvia Lopez, James Riley McHugh, Lillian Veloso, Meysam Golmo- hammadi, Iyad Obeid, and Joseph Picone. The temple university hospital seizure detection corpus. Frontiers in Neuroinformatics, 12:83, 2018. ISSN 1662-5196. doi: 10.3389/finf.2218.000
Afshin Shoeibi, Marjane Khodatars, Navid Ghassemi, Mahboobeh Jafari, Parisa Moridian, Roohal- lah Alizadehsani, Maryam Panahiazar, Fahime Khozeimeh, Assef Zare, Hossein Hosseini-Nejad, et al. Epileptic seizures detection using deep learning techniques: A review. International Journal of Environmental Research and Public Health, 18(1) 1):5700, 2021.
David Shuman, Sunil K Narang, Pascal Frossard, Antonio Ortega, and Pierre Vandergheynst. The emerging field of signal processing on graphs: Extending high-dimensional data analysis net- works and other irregular domains. IEEE signal processing magazine, 30(3):83-98, 2013.
Mohammad Khubeb Siddiqui, Ruben Morales-Menendez, Xiaodi Huang, and Nasir Hussain. A review of epileptic seizure detection using machine learning classifiers. Brain informaties, 7: 1-18, 2020.
Micheal Strein, John P Holton-Burt Burke, LaTangela R Smith, and Gretchen M Brophy. Prevention, treatment, and monitoring of seizures in the intensive care unit. Journal of Clinical Medicine, (8):1177, 2019.
Supriya Supriya, Siuly Siuly, Hua Wang, and Yanchun Zhang. Epilepsy detection from eeg using complex network techniques: A review. IEEE Reviews in Biomedical Engineering, 2021.
Ilya Sutskever, Oriol Vinyals, and Quoc V Le. Sequence to sequence learning with neural networks. In Advances in neural information processing systems, pp. 11143112,210 2014.
Alexandros T Tzallas, Markos G Tsipouras, and Dimitrios I Fotiadis. Epileptic seizure detection in cegs using time--rreueee frequency analysis. IEEE transactions on information technology in biomedicine, 73(5) 2009.
12

---

Published as a conference paper at ICLR 2022
Pauli Virtanen, Ralf Gommers, Travis E. Oliphant, Matt Haberland, Tyler Reddy, David Courna- peau, Evgeni Burovski, Pearu Peterson, Warren Weckesser, Jonathan Bright, Stéfan J. van der Walt, Matthew Brett, Joshua Wilson, K. Jarrod Millman, Nikolay Mayorov, Andrew R. J. Nel- son, Eric Jones, Robert Kern, Eric Larson, C J Carey, Ilhan Polat, Yu Feng, Eric W. Moore, Jake VanderPlas, Denis Laxalde, Josef Perktold, Robert Cimrman, lan Henriksen, E. A. Quintero, Charles R. Harris, Anne M. Archibald, António H. Ribeiro, Fabian Pedregosa, Paul van Mul- bregt, and SciPy 1.0 Contributors. SciPy 1.0: Fundamental Algorithms for Scientific Computing in Python. Nature Methods, 17:261-222 2722, 2020a. doi:
Pauli Virtanen, Ralf Gommers, Travis E. Oliphant, Matt Haberland, Tyler Reddy, David Couraaeau,, Evgeni Burovski, Pearu Peterson, Warren Weckesser, Jonathan Bright, et al. Scipy 1.0: funda- mental algorithms for scientific computing in python. Nature methods, 17(3):261-272 2020b.
WHO. Epilepsy. epi Peplypsyy june 2019.
Junjie Xu, Yaojia Zheng, Yifan Mao, Ruixuan Wang, and Wei-Shi Zheng. Anomaly detection on clectreencephalogrppyy with supe-super learning. In 2020 IEEE International Conference on Bioinformatics and Biomedicine (BIBM), pp. 363-368. IEFE, 2020.
Yuzhe Yang and Zhi Xu. Rethinking the value of labels for improving hasslimbalanced learning. In Conference on Neural Information Processing Systems (NeurIPS), 2020.
Matthew D Zeiler and Rob Fergus. Visualizing and understanding convolutional networks, 2013.
Yanna Zhao, Gaobo Zhang, Changxu Dong, Qi Yuan, Fangzhou Xu, and Yuanjic Zheng. Graph attention network with focal loss for seizure detection on electroencephalograpyy signals. Inter- national Journal of Neural Systems, pp. 2150027, 2021.
13

---

Published as a conference paper at ICLR 2022
APPENDIX
A DATA PREPROCESSING
Because the EEG signals are sampled at different frequencies in the Temple University EEG Seizure Corpus (TUSZ), we resample them to the same frequency of 200 Hz using the "resample" function in SciPy python package (Virtanen et al. 2020a). We perform the following preprocessing steps to obtain EEG clips in the frequency domain and their corresponding labels.
First, for seizure detection, we use both seizure and non-seizure EEGs. We obtain the EEG clips by sliding a 12-s (or 60-s) window over the EEG signals without overlaps, and ignore the last window if it is shorter than the clip length. The label for each clip is 1 if at least one seizure event occurs within this clip, otherwise y O.
Second, for seizure classification, we use only seizure EEGs. We obtain one 12-s (or 60-s) EEG clip for each seizure event starting at 2-s before the annotated seizure onset time, where a 2-s offset accounts for tolerance in the annotations. If a seizure event is shorter than 12-s (or 60-s), the EEG clip is truncated at the end of the seizure to prevent a clip from having multiple seizure types. The label for each clip is the index of the corresponding seizure class, L... y {0, 2,3, which corresponds to combined focal (CF) seizures, generalized non-specific (GN) seizures, absence (AB) seizures, and combined tonic (CT) seizures.
Third, for self-supervised pre-training, we use the same EEG clips as seizure detection.
sttfffaatttiossfffuuurrriiiiiiie pre- training tasks, we perform the following preprocessing steps to transform the signals in time domain to frequency domain: (a) slide f second window the EEG clip without overlap, where is the time step size for networks involving recurrent layers; (b) apply fast Fourier transform (FFT) each second window using the "fft" function in Seipy python package (Virtanen al.) 2020b), and retain the log amplitudes of the non non-negative frequency components similar to prior studies (Asif et al. 2020 Ahmedt-Aristizabal
Fourth, for each EEG clip in each of the seizure detection/seizure
al., 2020 Covert et al. 2019); (c) z-normalize the EEG clip with respect to the mean and standard deviation of the training data. Because EEG clips for seizure classification may have variable lengths due to short seizures, we pad the clips with 0's to facilitate model training in batches. We use / second as natural choice of the time step size.
After preprocessing, each EEG clip can be denoted as X E RTxNXM where T = 12 (or T = 60) represents the clip length, N = represents the number of EEG channels/electroodes, and M 100 represents the feature dimension after the aforementioned Fourier transform.
B EFFECTIVENESS OF FOURIER TRANSFORM
To evaluate the effectiveness of Fourier transform (Appendix we compare the performance of our DCRNN (without self-supervised pre-training) on inputs without and with Fourier transform on both seizure detection and seizure classification. As shown in Table frequency-domat inputs (with Fourier transform) result significantly better performance than time-domain inputs (without Fourier transform). This is likely because seizures are associated with electrical activity in certain frequency bands (Tzallas et al. 2009)., and thus mooot-tiim--nterve frequency-done -doemin inputs could be more informative than time-domain inputs.
C DISTINGUISHABILITY OF FOCAL NON-SPECIFIC, SIMPLE PARTIAL, AND COMPLEX PARTIAL SEIZURES
Because simple partial and complex partial seizures are focal seizures characterized by the clinical behavior, consciousness during seizure (Fisher al.) 2017). they are not distinguishable from other focal seizures from EEG signals alone. In this study, we merge focal non-specific (FN), simple partial (SP), and complex partial (CP) seizures into combined focal seizure class. To justify our decision, we perform classification of these focal seizure types using our DCRNNs and baselines. As shown in Figure 5, SP and CP seizures are largely misclassified FN seizures, suggesting that
14

---

Published as a conference paper at ICLR 2022
Table 4: Seizure detection and classification results from DCRNNs (without self-supervised pre- training) on time-domain inputs and frequency-domain inputs. Mean and standard deviations are obtained from five random runs.
Seizure Detection
Seizure Classification Weighted Fl- Fl-Score
Model
Input Domain
AUROC
12-s
60-s
12-s
60-s
Com-DCRNN
Time
0.717 + 0.003
0.704
0.023
0.597
0.023
0.618
0.018
Without Pre- vre-Training
Frequeney
0.812 + 0.012
0.804 + 0.015
0.710
0.023
0.701 + 0.030
Dist-DCRNN
Time
0.733 + 0.012
0.698
0.003
0.592
0.022
0.608
0.015
Without Pre- Training
Frequency
0.825 0.019
0.793
0.022
0.703 0.025
0.690 0.035
machine learning models may not be able to distinguish these focal seizure types from EEG signals alone. This supports our decision of combining FN, SP, and CP seizures into one class.
D TRAIN, VALIDATION, AND TEST SETS
Table 5 shows the number of EEG clips and number of patients in our train, validation, and sets for self self-superyie supervised pre-traie pre-training, seizure detection, and seizure classification tasks. Train, validation, and test sets consist of distinct patients.
Table 5: Number of EEG clips and patients in the train, validation, and test sets in our study. Train, validation, and test sets consist of distinct patients.
EEG Clip
Train Sct
Validation Set
Test Set
Length
EEG Clips
Patients (% Seizure)
EEG Clips
Patients (% Seizure)
EEG Clips (% Seizure)
Patients (% Seizure)
(Secs)
(% Seizure)
(% Seizure)
38,613
530
5,503
61 (36.1%)
8,848
45
Pre-training & Seizure
60-s
(9.3%)
(34.0%)
(11.4%)
(14.7%)
(77.8%)
196.646
531
28,057
61
44,959
45
Detection
12-s
(6.9%)
(33.9%)
(8.7%)
(36.15)
(10.9%)
(77.8%)
Seizure
60-s & 12-s
1,925
179
450
22
521
34 (100.0%)
Classification
(100.0%)
(100.0%)
(100.0%)
(100.0%)
(100.0%)
E DETAILS OF MODEL TRAINING PROCEDURES AND HYPERPARAMETERS
We performed the following hyperparameter search on the validation set: (a) initial learning rate within range [5e-5, 1e-3]; (b) {2, 3, the number of neighbors to keep for each node in the correlation graphs: (c) the number of Diffusion Convolutional Gated Recurrent Units (DCGRU) lay- within range (2, 4,5) and hidden units within range (32, 64, 128): : (d) the maximum diffusion step K E {2,3, 3, 4): (e) dropout probability in the last fully connected layer. We used a batch size of 40 EEG clips, the maximum possible across all models and baselines on single Titan RTX GPU. The hyperparameters were selected based on the best performance on the validation set, and are detailed in the next sections. We used the cosine annealing learning rate scheduler (Loshchilov & Hutter 2017) in PyTorch for all model training. We ran five runs with different random seeds for all models. In all experiments, model training was early stopped when the validation loss did not decrease for five consecutive epochs.
Model training for seizure detection. For seizure detection, the plentiful negative examples in the train set were undersampled such that the train set had 50% positive examples, which resulted in 27,292 training examples for 12-s clips and 7,188 training examples for 60-s clips. We used binary cross-entropy the loss function to train the seizure detection models. The models were trained for 100 epochs with an initial learning rate of le-4. For the correlation graphs, the top-3 neighbors' edges were kept for each node. The maximum number of diffusion step 2, and the dropout probability was (i.c. no dropout). The model consists of two stacked DCGRU layers with 64 hidden units, resulting in 168,641 trainable parameters for the distance graph and 280,769 trainable
15

---

Published as a conference paper at ICLR 2022
(a) 12-s
Dense-CNN
Corr-DCRNN w/o Pre-Training
Dist-DCRNN w/o Pre-Training
FN
0.81
0.02
0.17
0.89
0.01
0.10
FN
0.01
0.11
True
Sp
True
0.60
0.00
0.40
Trus
SP
CP
0.77
0.01
0.23
0.87
0.01
0.11
CP
0.89
0.01
0.09
Predicted label
Prodicted label
Predicted label
LSTM
Corr-DCRNN w/ Pre-Training
Dist-DCRNN w/ Pre-Training
FN
0.95
0.01
0.04
FN
FN
0.77
0.02
0.21
True ishel
Tru lobel
SP
0.93
0.00
0.07
SP
0. 0.60
0.00
0.40
True
CP
0.00
CP
0.71
0.01
0.28
8
Predicted label
Predicted label
(b)
Dense-CNN
Corr-DCRNN w/o Pre-Training Dist-DCRNN w/o Pre-Training
0.01
True loadl
True label
Sp
0.00
0.20
Sp
0.57
0.00
0.33
Sp
0.67
0.00
0.33
0.0
0.91
0.00
0.09
CP
0.88
0.01
0.11
0.79
0.01
0.20
8
6
Predicted label
Predicted lebel
Predicted label
LSTM
Corr-DCRNN w/ Pre-Training
Dist-DCRNN w/ Pre-Training
FN
0.00
FN
0.88
0.01
0.12
FN
0.73
0.06
0.21
True label
True label
True label
Sp
0.80
0.00
0.20
0.00
0.33
CP
0.22
CP
0.72
0.04
0.24
PH
8
PN
8
8
Predicted label
Predicted label
Figure 5: Confusion matrices (averaged across five random runs) of focal seizure classification on (a) 60-s EEG clips, and (b) 12-s EEG clips. All models fail to distinguish well among these focal seizure types, supporting our decision of merging these seizure types into one combined focal seizure class. Note that each row of the confusion matrices is normalized by dividing by the number of examples in the corresponding class, such that each row sums up to one.
parameters for the correlation graphs. Model training for seizure detection took about 20-min for 12-s EEG clips, and about 30-min for 60-s EEG clips.
To obtain the final seizure/non-seinn prediction, we performed decision threshold search on the validation More specifically, to balance between precision and recall scores, selected the
16

---

Published as a conference paper at ICLR 2022
decision threshold that results in the highest Fl-score the validation set. When evaluating the models on the test set, EEG clips with probabilities above this decision threshold are predicted as seizures, while clips with probabilities below this decision threshold are predicted as non-scizures.
Model training for seizure classification. For seizure classification, we used multi-class cross- entropy the loss function during training. The models were trained for 60 epochs with initial learning rate of 3e-4. For the correlation graphs, the top-3 neighbors' edges were kept for each node. The maximum number of diffusion step was 2, and the dropout probability was 0.5. The model consists of two stacked DCGRU layers with 64 hidden units, resulting in 168,836 trainable parameters for the distance graph and 280,964 trainable parameters for the correlation graphs. Model training for seizure classification took about 3-min for 12-s EEG clips, and about 7-min for 60-s EEG clips.
Model training for self-supervised task. Preliminary experiments suggested that predicting future T" 12 second preprocessed EEG clips results low regression loss on the validation set given previous 12-s (60-s) preprocessed clips, and thus we used 7 12 in all self-supervised pre-training experiments. For self-supervised pre-training. we used mean absolute error (MAE) as the loss func- tion. The models were trained for 350 epochs with an initial learning rate of 5e-4. For the correlation graphs, the top-3 neighbors' edges were kept for each node. The maximum number of diffusion step was 2. The model consists of three stacked DCGRU layers with 64 hidden units in both the encoder and decoder, resulting in 417,572 trainable parameters for the distance graph and 690,980 trainable parameters for the correlation graphs. Model training for self-supervised prediction took about 10-h for 12-s EEG clips, and about 24-h for 60-s EEG clips.
Model training for baselines. For baseline Dense- CNN, employ the same model architecture as that described in Saab et al. (2020). For baseline LSTM (Hochreiter & Schmidhuber 1997), we have the number of LSTM layers and hidden units the same as the number of DCGRU layers and hidden units in our DCRNN model. For baseline CNN-LSTM, we use the same model architecture described in -Aistizaai al. (2020), i.c. two stacked convolutional layers (32 3 ker- nels), one max- max-pooling layer (2 x 2), one fully-connected layer (output neuron = 512), two stacked L.STM layers (hidden size 128), and one fully connected layer.
F DATA AUGMENTATION
During training, we performed the following data augmentations based on EEG domain knowledge: (a) randomly scaling the amplitude of the raw EEG signals by a scale within [0.8, 1.2] and (b) randomly reflecting the signals along the scalp midline.
G ADDITIONAL EVALUATION RESULTS ON SEIZURE DETECTION
Table Fi-score, Area Under the Precision-Recall Curve (AUPR), sensitivity, and specificity of seizure detection models.
H SEIZURE DETECTION OCCLUSION MAPS FROM BASELINE DENSE-CNN
Figure shows example seizure detection occlusion maps obtained from baseline Dense-CNN correctly predicted 60-s EEG clips the test set. Unlike our model (Figure 3), high saliency from Dense-CNN does not localize in any seizure regions.
SEIZURE CLASSIFICATION OCCLUSION MAPS FROM DCRNN
Figure shows example seizure classification occlusion maps from Cor-DDCRN with self- supervised pre-training on correctly predicted 60-s EEG clips in the test set. The occlusion maps are obtained by completely dropping one EEG channel at a time and calculating the relative change in the model output. For CF seizures (a-b), high saliency regions correspond to brain areas where the focal seizures are localized. For the other generalized seizure types (c-e), less salient regions correspond to less abnormal areas.
17

---

Published as a conference paper at ICLR 2022
Table Additional evaluation scores for seizure detection on (a) 12-s EEG clips and (b) 60-s EEG clips. Mean and standard deviations are obtained from five random runs. Best non-pretrained and pre-trained mean results are highlighted in bold.
(a) 12-s EEG clips
FI-Score
AUPR
Sensitivity
Specificity
Model
(mean std)
(mean std)
(mean
std)
(mean + std)
Dense-CNN
0.326 0.019
0.328 0.043
0.293 0.021
0.938 0.014
LSTM
0.376 + 0.021
0.354 + 0.023
0.357 0.045
0.934 + 0.015
CNN-LSTM
0.337 = 0.009
0.309 = 0.015
0.333
0.028
0.920 + 0.021
Corr-DCRNN
0.392 0.027
0.370 0.027
0.373
0.035
0.935 0.012
Without Pre-training
Dist-DCRNN
0.437 + 0.029
0.411 + 0.041
0.411 0.038
0.943 + 0.006
Without Pre-training
Corr-DCRNN
0.484 + 0.011
0.454 + 0.020
0.524 = 0.012
0.922 + 0.004
With Pre-training
Dist-DCRNN
0.487 + 0.042
0.463 + 0.048
0.592 = 0.052
0.897 + 0.012
With Pre-training
(b) 60-s EEG clips
F1-Score
AUPR
Sensitivity
Specificity (mean + std)
Model
(mean + std)
(mean + std)
(mean std)
Dense-CNN
0.404 + 0.022
0.399 + 0.017
0.451 0.134
0.869 + 0.071
LSTM
0.365 + 0.009
0.287 + 0.026
0.463 0.060
0.814 + 0.053
CNN-LSTM
0.330 + 0.016
0.276 + 0.009
0.363 = 0.044
0,857 + 0.023
Cor--DCRNN
0.448 + 0.029
0.440 + 0.021
0.457 = 0.058
0.900 + 0.028
Without Pre-training
Dist-DCRNN
0.341 + 0.170
0.418 + 0.046
0.326 = 0. 0.183
0.932 + 0.058
Without Pre-training
Cor-DDCRNN
0.514 + 0.028
0.539 + 0.024
0.502 - 0.047
0.923 + 0.008
With Pre-training
Dist-DCRNN
0.571 0.029
0.593 0.031
0.570 0.047
0.927 0.012
With Pre-training
J COMPARISON BETWEEN SELF-SUPERVISED PRE-TRANE PRE-TRAINING AND TRANSFER LEARNING
To compare our self-supervised pre-training strategy traditional transfer learning, we perform transfer learning by pre-training DCRNNs for seizure detection on an in-house dataset (40.316 EEGs, Table and finetuning on TUSZ data for seizure detection and classification. Due the lack of fine-grained seizure type labels, we do not pre-train DCRNNs for seizure classification on the in- house dataset. Moreover, we pre-train DCRNNs on the in-house dataset using our self-supervised pre-training strategy and finetuned them for seizure detection and classification TUSZ.
Table 8 shows DCRNN results with self-supervised pre-training and transfer learning from the in- house dataset. Self-supervised pre-training from the in-house dataset (3rd-4th rows) consistently outperforms trasnfer learning from the in-house dataset (5th-6th rows). We speculate that trans- fer learning does not perform comparably distribution shift in the data (i.e., the in-house dataset comes from a different population and uses a slightly different EEG acquisition protocol). In contrast, by learning predict the EEGs for the next
self-supervised pre-training because it suffers from
18

---

Published as a conference paper at ICLR 2022
(a)
(b)
Fp1-F7
Fp1-F7
F7:13
F7-73
13-15
13-15
T5-01
782
Fp2-F8
F8-14
T4-T6
16-02
F3
C3
F3
C3-P3
C3-P3
P3-01
F3-01
FD2-F4
Tp2-4
F4-C4
C4-P8
C2-P2
900
910
920
930
940
950
960
60
70
80
90
100
110
120
time
Eime
(c)
(d)
Fp
Fg1-F7
F7-T8
T3-75
03:01
FB
14-16
T4.76
T6402
T2-02
Fp1-F3
F3
F3-03
C3-P3
C3-93
P3-01
P3-01
Fp2-F4
F22 F4
F4
F4-C4
C4
C4-24
P4.02
F2-C2
Cz
1200
1210
1220
1230
1240
1250
1260
1440
1450
1460
1470
1440
1490
1500
(G)
time (5)
Low Saliency
High Saliency
Figure 6: Example occlusion maps for seizure detection obtained from baseline Dense-CNo on cor- rectly predicted 60-s clips for (a)-(b) focal seizures, and (c)-(d) generalized seizures. Red boxes indicate the duration of the seizures. Note that the values within an occlusion map are normalized, and thus should not be compared across different occlusion maps.
time period, self-supervised pre-training encourages the model to learn task-agnostic representations and thus mitigates the problem of distribution shift.
Table 7: Summary of in- in-house dataset. Only annotations for the start of seizure are available. For EEG files with seizures, EEG clips obtained by taking 12-s (or 60-s) signals starting from the annotated seizure start time. For EEG files without seizures, EEG clips are obtained by taking 12- (or 60-s) signals randomly from the entire signal. We apply the same data preprocessing steps described in Appendix A
EEG Files (% Seizure)
Patients (% Seizure)
EEG Clips (% Seizure)
Total Duration
In-House Train Set
40,316 (24.1% 1%) 853,141.87 min
5,355 (25.7%)
46,613 (34.4%)
K CHOICE OF DISTANCE-BASED GRAPH STRUCTURE
For the distance graph, we used a threholded Gaussian kernel (Shuman et al.) 2013) to compute the edge weight between two electrodes (Section (.2.I). We experimented with K, the threshold for graph sparsity, within range 10.1, 2]. Based preliminary experiments and EEG domain knowl- edge, we chose K = 0.9 because it results reasonable graph (e.g. no long-range connection) that resembles the EEG montage (longitudinal bipolar and transverse bipolar) widely used clinically (Acharya et al., 2016). Figure X shows distance graphs resulting from different thresholds k. In that smaller threshold (e.g., 0.7 or 0.8) results in missing edges hetween
Figure 8
19

---

Published as a conference paper at ICLR 2022
Table 8: Comparison of results between self-supervised pre-training and transfer learning pre- trained on an in-house dataset. 3rd-4th rows: DCRNN results with self-supervised (SS) pre-training on the in-house dataset. 5th-6th rows: DCRNN results with transfer learning pre-trained for seizure detection on the in-house dataset. Mean and standard deviations are from five random runs. Best mean results for each column are highlighted bold.
Seizure Detection
Seizure Classification
Model
AUROC
Weighted F1 F1-Score
12-s
60-s
12-8
60-s
Corr-DCRNN w/ SS
0.863 0.005
0.856 = 0.013
0.736 + 0.007
0.723 + 0.010
Pre-Training
Dist-DCRNN w/ SS
0.879 0.006
0.886 + 0.006
0.767 + 0.038
0.718 + 0.018
Pre- Fre-Training
Com--DCRN w/ Transfer
0.848 + 0.018
0.850 + 0.002
0.733 + 0.027
0.711 + 0.025
Learning
Dist-DCRNN w/ Transfer
0.866 + 0.001
0.847 + 0.004
0.720 + 0.046
0.691 + 0.038
Learning
nearby electrodes. For example, there is no edge between FP] and FZ for threshold 0.8 and no edge between T3 and C3 for threshold 0.7. In contrast, a larger threshold (c.g., 1.0, 1.1, or 1.2) results in edges connecting electrodes that are spatially far apart. For example, there an edge connecting C3 and FZ, as well as an edge between F7 and T5 for threshold 1.2. Using EEG domain knowl- edge provided by board certified neurologist, we believe that threshold 0.9 results in a more reasonable distance-based EEG graph compared to other thresholding values.
In addition, we explore using Gaussian kernel with a specified bandwidth for building the distance
graph, i.e., Wij 22mh22 exp dist(c;, 02)) where Wij is the edge weight between electrodes
1
2h.
oy and uj. dist(u; u., 0t) is the Euclidean distance between 2 and By, and f is the Gaussian kernel band- width. Figure 9 shows the distance graph structures resulting from different values of bandwidth. With bandwidth- 0.06, the distance graph structure resembles the original graph structure using thresholded Gaussian kernel with a threshold of 0.9 (Figure
L 8-CLASS SEIZURE CLASSIFICATION
We also perform seizure classification on the original eight seizure ty available in TUSZ (see Table (). Note that only one patient's two myoclonic seizures are available in the official TUSZ train set and only one patient's one myoclonic seizure is available in the official TUSZ test set. Hence, patient-wise train/validation splits for myoclonic seizure is not possible, and we randomly assign one myoclonic seizure in the official TUSZ. train set to our train split and the other to our validation split. As shown in Table 9 DCRNNS consistently outperform the baselines on 8-class seizure classification.
M USING SELF-SUPERVISED PREDICTION AS AN AUXILIARY TASK
Here, instead of pre-training DCRNNs for the self-supervised prediction task (i.e., predicting pre- processed EEG clips for the next time period), we conduct experiments using the self-supervised prediction task able lengths, we only perform this experiment for seizure detection to facilitate training in batches. Specifically, the loss function is L Lxeizure detection + XLss pediction where is hyperparameter balancing the seizure detection loss and the self-supervised loss and is tuned on the validation set. For 12-s (or 60-s) seizure detection, the auxiliary task is to predict the next 6-s preprocessed EEG clips given the first 6-s (or 30-s) EEG clips.
auxiliary task. Because EEG clips for the seizure classification task have vari-
"The eight seizure types are: focal seizure, generalized non-specific seizure, simple partial seizure, complex partial seizure, absence seizure, tonic seizure, tonic-clonic seizure, and mynclonic seizure.
20

---

Published as a conference paper at ICLR 2022
Table Results (weighted Fl-scores) of seizure classification on original 8 seizure types in TUSZ. Mean and standard deviations are from five random runs. Best mean results for both non-pretrained and pretrained models are highlighted in bold.
Model
12-s
60-s
Dense-CNN
0,431 + 0.037
0,427 + 0.047
LSTM
0.515 0.025
0.525 + 0.017
CNN-LSTM
0.489 + 0.036
0.509 + 0.021
Corr-DCRNN
0.553 + 0.025
0.577 + 0.028
Without Pre-training
Dist-DCRNN
0.570 + 0.027
0.600 + 0.022
Without Pre-training
Comr-DCRNN
0,582 + 0.014
0.591 + 0.008
With Pre-training
Dist- Dist-DCRNN
0.583 + 0.009
0.607 + 0.017
With Pre-training
Table TO shows the seizure detection results for DCRNNs when the self-supervised (SS) prediction task is used as an auxiliary task during training. Compared to self-supervised pre-training (Table 2 last two rows), auxiliary learning only marginally improves Dist-DCRNN performance on 12- s seizure detection (no statistical significance), whereas self-supervised pre-training significantly outperforms auxiliary learning for 60-s seizure detection.
Table 10: Seizure detection results for DCRNNs with self-super pre-training (same as Table 2 and auxiliary learning. Mean and standard deviations are from five random runs. Best mean results for Dist-DCCRNNCCo-DDCCN are bold.
Seizure Detection AUROC
Model
12-s
60-s
Con--DCRNY DC-NN w/ SS Pre-Training (Table
0.861 + 0.005
0.850 + 0.013
Con-DDCRN w/ SS Auxiliary Task
0.851 + 0.008
0.811 + 0.005
w/ SS Pre-Training (Table
0.866 0.016
0.875 + 0.016
Dist-DCRNN w/ SS Auxiliary Task
0.875 + 0.005
0.840 + 0.013
21

---

Published as a conference paper at ICLR 2022
(a)
CF Seizure
(b)
CF Seizure
F7
F0
F7-73
13:5
15-01
1541
F8
18.76
14-76
18:16
16-02
0602
13
F3
C3-1
F4
Fa2-F4
FAC1
04-24
P4-02
F2-22
F2-C2
C2
1080
1000
1100
1110
1120
1130
130
150
170
180
150
200
(c)
GN Seizure
(d)
AB Seizure
F02.67
F7:33
F713
13-15
13-15
T5:01
F02-88
FB-74
16-02
16-02
f3
Fo1-93
73-3
03-11
P1-11
F42-F4
F4-C4
C4-M4
C4-P4
14-02
P4-02
C2
C2
200
210
220
2350
240
250
180
190
200
210
220
230
Eim 04)
time (5)
(e)
CT Seizure
F7-13
13:15
T5-01
FB
15-02
F3
F3
P3:01
F42-44
C4
14-02
C2
Low Saliency
High Saliency
Figure 7: Example occlusion maps for seizure classification obtained from pre-trained Cor--DCRNN model for (a)-(b) combined focal (CF) seizures, (c) generalized non-specific (GN) seizure, (d) absence (AB) seizure, and (e) combined tonic (CT) seizure. In each subfigure, the bottom panel shows the occlusion values for each channel replicated along the time dimension and overlaid on 60-s EEG signals, the red boxes indicate the duration of seizures in the EEG clips, and the top panel shows the occlusion values overlaid on the correlation graph structure. To visualize occlusion map values in this "double banana" montage, we subtract the occlusion values between the corresponding channels in the montage. which results in different values between occlusion maps shown on the EEG signals (bottom panel in each subfigure) and that shown on the graph structures (top panel in each subfigure). Note that the values within an occlusion map are normalized, and thus should not be compared across different occlusion maps.
22

---

Published as a conference paper at ICLR 2022
threshold (original)
threshold = 0.7
threshold 0.8
P2
threshold 1.0
threshold 1.1
threshold 1.2
Weak Connectivity
Strong Connectivity
Figure 8: Distance graphs with different threshold к in the thresholded Gaussian kerne.. Small thresholds (e.g., 0.7 and 0.8) result in missing edges between nearby electrodes, whereas large
thresholds (e.g., 1.0, 1.1, 1.2) result in edges connecting electrodes that are spatially far away.
original
bandwidth = 0.04
bandwidth = 0.05
bandwidth = 0.06
bandwidth = 0.07
bandwidth 0.08
Weak Connectivity
Strong Connectivity
Figure 9: Distance-baseed graphs constructed using Gaussian kernels with different bandwidths. When bandwidth 0.06, the distance graph structure resembles the original graph structure using thresholded Gaussian kerel with a threshold of 0.9 (Figure Tb).
23


Reviews:
[{"review_id": "yPczb4mNIX1", "reviewer": "Reviewer_6FoL", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Pros.\n-The paper combines well-founded strategies for representing EEG data on non-euclidean spaces preserving spatial and temporal patterns.\n-The algorithm can predict and localize seizures.\n-The mathematical background is limited but enough to clarify the main paper insights.\n-The experiments are explained in detail and include ablation procedures.\n-Code is available.\n\nCons. and comments\n-Though the autoencoder-based strategy seems helpful, It could be used as a regularization strategy within an end-to-end training. However, using the autoencoder just for initializing the network weights sounds limited; maybe other initialization and transfer learning strategies could obtain better results.\n-It is not clear to me how a simple correlation can enhance a nonlinear Gaussian kernel. I think the authors need to discuss better how they fix the kernel hyperparameters. For example, you are working on a log-spectral space, and you set the Gaussian bandwidth as the standard deviation of the Euclidean distances. Still, you then impose a manifold-based affinity by thresholding.\n-Authors claim that the non-Euclidean representation (affinity matrix thresholding) favors the EEG analysis. Nonetheless, if you use a Gaussian kernel with a proper bandwidth, you can obtain similar results compared to the thresholding approach without the need to fix such a hyperparameter. So then, is the \"non-Euclidean\" representation needed? Maybe, a good bandwidth with a good network will be enough.\n-Notations in section 2.2.2, Graph neural network, use the EEG trial X as input; however, according to fig 1, the DCRNNs are applied to the affinity matrix. Can you clarify, please?", "summary_of_the_paper": "The authors propose a graph-based representation from thresholded  Gaussian and linear (correlation) kernels (undirected connectivity) coupled with a diffusion convolutional recurrent network. Besides, a Fourier-based preprocessing is carried out with self-supervised (autoencoders) to initialize the network weights.  Experiments are performed on an EEG-based seizure detection and localization task. The seizure localization is conducted using occlusion and dropping approaches on EEG channels. Obtained results elucidate an interesting strategy for seizure analysis on a well-known public database.", "summary_of_the_review": "Overall, the paper covers a research field of interest from theoretical (Spatio-temporal coding of time series using graph and DCRNN) and practical (EEG-based seizure detection and localization) perspectives. The experiments are exhaustive and convincing.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "ScqlsPaQghd", "reviewer": "Reviewer_axL1", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Strengths:\n- The paper is very well written with the 3 goals clearly stated and how they were addressed.\n- The paper proposes two graph structures to model natural geometry of EEG sensors and dynamic brain connectivity. It compares the two results and shows that correlation based structure (dynamic brain connectivity) performed better than the one based on distance (natural geometry) for a rare seizure type\n- The value of pre-training was shown is very clear from experimental results.\n- The accuracy of occlusion based seizure localization was quantified and visualized in a very comprehensible way. It shows the value of pre-training. It was great to see two goals of the paper (interpretability and effect of pre-training) demonstrated in one set of results.\n\nWeaknesses:\n- The paper does not include a lot of comparison with other methods but since it builds on multiple approaches (pre-training and two graph structures) there are 4 models resulting from these.\n- The individual methods are built upon exising work", "summary_of_the_paper": "The paper proposes a method to combine modeling based on graphs mapping electrode geometry and self-supervised pre-training for EEG-based seizure detection and classification. It also proposes an interpretation method using occlusion maps to the demonstrate the model's ability to localize seizure.", "summary_of_the_review": "The paper proposes and  two graph structures to model EEG sensors and brain connectivity and a pre-training method to improve model performance. Both of these are well demonstrated and the results demonstrate how one structure is superior to another for a a particular type of rare seizure. It also includes an occlusion map based seizure localization method. Overall, the paper clearly states its goal and does a good job of reaching those goals using well demonstrated experimental results.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "7fFZJiK170", "reviewer": "Reviewer_a7bX", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "The details of the self supervised training must be added in more detail for clarity and reproducibility. \n\nSelf supervised methods are mostly suited for applications where labels are scarce, the authors should add more justification as to why they have targeted such a method when sufficient labels are available \n\nAn occlusion based method is used for analysis of the interpretability, the authors should comment if the graph structure can be used to interpret the significance of various brain regions (coming from the electrodes) in terms of the seizure. \n\nAlthough the some classes are combined, it would be interesting to see the performance when all eight class labels present in the data are used for classification. ", "summary_of_the_paper": "The paper presents a method for seizure detection and classification. In particular, the method is self supervised, based on graph neural network and use EEG signals. The authors report significant performance in detection and classification, as well as provide methods for qualitative evaluation of model interpretability. ", "summary_of_the_review": "The authors present a graph neural network method for seizure detection and classification. The results are shown to improve current state-of-the-art. However some of the design choices and interpretability analysis needs more background and motivation. In particular, the details of the self supervised implementation needs more elaboration. ", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}]

Author rebuttal:

Meta-review: This work tackles an important clinical application. It is experimentally solid and investigates
novel deep learning methodologies in a convincing way.

For these reasons, this work is endorsed for publication at ICLR 2022.
Decision: Accept (Poster)