Self-Supervised Graph Neural Networks for Improved Electroencephalographic Seizure Analysis

1 INTRODUCTION . Seizures are among the most common neurological emergencies in the world ( Strein et al. , 2019 ) . Seizures can be chronic as in the case of epilepsy , a neurological disease affecting 50 million people worldwide ( WHO , 2019 ) . Clinically , definitive detection of a seizure is only the first step in seizure diagnosis . An important subsequent step is to classify seizures into finer-grained types––such as focal versus generalized seizures––for identifying epilepsy syndromes , targeted therapies , and eligibility for epilepsy surgery ( Fisher et al. , 2017 ) . Scalp electroencephalography ( or “ EEG ” ) plays a critical role in seizure detection and classification . Clinically , EEG-based seizure detection and classification are performed by a trained EEG reader who visually examines a patient ’ s EEG signals over time periods ranging from hours to days . However , this manual analysis is extremely resource- and time-intensive , and thus automated algorithms could greatly accelerate seizure diagnosis and improve outcomes for severely ill patients . Figure 1 : ( a ) EEG electrode placement in the standard 10-20 system . ( b ) Distance-based EEG graph . ( c ) An example correlation-based EEG graph . ( d ) Overview of our methods . The inputs to the models are the EEG graphs , where each node feature corresponds to the preprocessed EEG signals in the respective channel . Self-edges are not shown for better visualization . Although a large number of studies have attempted automated seizure detection ( Rasheed et al. , 2020 ; Siddiqui et al. , 2020 ; Shoeibi et al. , 2021 ; O ’ Shea et al. , 2020 ; Saab et al. , 2020 ) or seizure classification ( Raghu et al. , 2020 ; Asif et al. , 2020 ; Iesmantas & Alzbutas , 2020 ; Ahmedt-Aristizabal et al. , 2020 ; Roy et al. , 2019 ) , several challenges remain largely unaddressed . First , most recent studies use convolutional neural networks ( CNNs ) that assume Euclidean structures in EEG signals or spectrograms ( Rasheed et al. , 2020 ; Shoeibi et al. , 2021 ; Raghu et al. , 2020 ; Asif et al. , 2020 ; Iesmantas & Alzbutas , 2020 ; Ahmedt-Aristizabal et al. , 2020 ; Roy et al. , 2019 ; O ’ Shea et al. , 2020 ; Saab et al. , 2020 ) . However , assumption of Euclidean structure ignores the natural geometry of EEG electrodes and the connectivity in brain networks . EEGs are measured by electrodes placed on a manifold ( i.e . patient ’ s scalp ) ( Figure 1a ) , and thus have inherent non-Euclidean structures . Graphs are a data structure that can represent complex , non-Euclidean data ( Chami et al. , 2020 ; Bronstein et al. , 2017 ) , and graph theory has been extensively used in modeling brain networks ( Bullmore & Sporns , 2009 ) . We therefore hypothesize that graph-based modeling approaches can better represent the inherent non-Euclidean structures in EEGs in a manner that improves both the performance and the clinical utility of seizure detection and classification models . Although traditional graph theory has been used ( Supriya et al. , 2021 ) , only a few deep learning studies have modeled EEGs as graphs for seizure detection . However , these graph-based studies were limited to nonpublic ( Covert et al. , 2019 ) or small datasets ( Craley et al. , 2019 ; Li et al. , 2021 ) , did not leverage modern self-supervised approaches or examine seizure classification ( Cisotto et al. , 2020 ; Zhao et al. , 2021 ; Li et al. , 2021 ) . Second , certain seizure types ( e.g . clonic seizures ) are rare by nature . Training machine learning models that perform well on these rarer seizure classes using traditional supervised learning approaches is challenging , which could explain the performance difference between major and minority seizure types in prior studies ( Raghu et al. , 2020 ; Iesmantas & Alzbutas , 2020 ; AhmedtAristizabal et al. , 2020 ) . Several studies have investigated an alternative , self-supervised training strategy ( Banville et al. , 2020 ; Mohsenvand et al. , 2020 ; Kostas et al. , 2021 ; Martini et al. , 2021 ; Xu et al. , 2020 ) , but they did not model EEGs as graphs or address automated seizure classification . Prior works have shown that self-supervised pre-training significantly improves model performance on data with imbalanced labels in the field of computer vision ( Yang & Xu , 2020 ; Liu et al. , 2021 ) . Hence , we hypothesize that self-supervised pre-training can help improve our graph model performance on rare seizure types . Moreover , a large portion of EEG signals generally do not have seizures . Self-supervised pre-training strategy would allow the model to leverage the abundant nonseizure EEGs that are readily available in the dataset . Lastly , for seizure detection and classification models , the ability to not only provide a single prediction across all EEG channels , but also to provide the interpretability and the ability to localize seizures would be clinically useful for informing treatment strategy . While prior studies ( Saab et al. , 2020 ; Covert et al. , 2019 ) have shown qualitative visualization for model interpretability , none have quantitatively assessed the model ’ s ability to localize seizures . In this work , we aim to address these limitations in prior automated seizure detection and classification studies . First , we propose a graph-based modeling approach for EEG-based seizure detection and classification . Specifically , we propose two EEG graph structures that capture EEG sensor geometry ( Figure 1b ) or dynamic brain connectivity ( Figure 1c ) , and we extend Diffusion Convolutional Recurrent Neural Network ( DCRNN ) ( Li et al. , 2018 ) , an RNN with graph diffusion convolutions , to model the spatiotemporal dependencies in EEGs ( Figure 1d ) . Second , we improve DCRNN performance using a self-supervised pre-training strategy of predicting the preprocessed EEG signals for the next time period without requiring additional data or labels . Finally , we propose quantitative metrics to assess our model ’ s ability to localize seizures . In summary : • We propose two EEG graph structures that capture ( 1 ) the natural geometry of EEG sensors or ( 2 ) dynamic connectivity in the brain , and show that building a recurrent graph neural network ( GNN ) based on these representations yields models for seizure detection and classification that outperform previous approaches on a large public dataset ( 5,499 EEGs ) . • We propose a self-supervised pre-training strategy to further improve our recurrent GNN model performance , particularly on rare seizure types . To our knowledge , our study is the first to date that combines graph-based modeling and self-supervised pre-training for EEGs . By leveraging graph structure and self-supervision , our method achieves 0.875 Area Under the Receiver Operating Characteristic Curve ( AUROC ) on seizure detection and 0.749 weighted F1-score on seizure classification , outperforming previous approaches on both seizure detection and classification on this large public dataset . Moreover , our selfsupervised pre-training method substantially improves classification of rare seizure types ( e.g . 47 points increase in combined tonic seizure accuracy over baselines ) . • We propose a quantitative model interpretability analysis that can be used to assess a model ’ s ability to localize seizures , which is critical to determining the course of treatment for seizures . We show that by leveraging graph structure and self-supervision , our method precisely localizes 25.4 % focal seizures , providing 21.9 points improvement over a prior state-of-the-art CNN . Finally , by displaying the identified seizure regions on raw EEG signals and EEG graphs , our approach could provide valuable insights that support more effective seizure diagnosis in real-world clinical settings . 2 METHODS . 2.1 SEIZURE DETECTION AND CLASSIFICATION PROBLEM FORMULATION . The goal of seizure detection is to predict whether a seizure exists within an EEG clip , and the goal of seizure classification is to predict the seizure type given a seizure EEG clip . Following a prior study ( Saab et al. , 2020 ) , we examine our model ’ s capability for fast and slow detection and classification over 12-s and 60-s EEG clips , respectively . 2.2 GRAPH-BASED MODELING FOR EEGS . 2.2.1 REPRESENTING EEGS AS GRAPHS . We represent an EEG clip as a graph G = { V , E , W } , where V denotes the set of nodes ( i.e . EEG electrodes/channels ) , E denotes the set of edges , and W is the adjacency matrix . We propose the following two methods of constructing the EEG graph . Distance graph . To represent the natural geometry of EEG electrodes , we compute edge weight Wij by applying a thresholded Gaussian kernel ( Shuman et al. , 2013 ) to the pairwise Euclidean distance between vi and vj , i.e. , Wij = exp ( − dist ( vi , vj ) 2 σ2 ) if dist ( vi , vj ) ≤ κ , else 0 . Here , dist ( vi , vj ) is the Euclidean distance between electrodes vi and vj according to the standard 10-20 EEG electrode placement ( Jasper , 1958 ) , σ is the standard deviation of the distances , and κ is the threshold for sparsity . This results in a universal undirected , weighted graph for all EEG clips . Based on preliminary experiments and EEG domain knowledge , we chose κ = 0.9 because it results in a reasonable graph that also resembles the EEG montage ( longitudinal bipolar and transverse bipolar ) widely used clinically ( Acharya et al. , 2016 ) . Figure 1b shows the distance graph with κ = 0.9 . In Appendix K , we explore different values of κ as well as constructing the distance graph using a Gaussian kernel with a pre-specified bandwidth . Correlation graph . To capture dynamic brain connectivity , we define the edge weight Wij as the absolute value of the normalized cross-correlation between the preprocessed signals in vi and vj . To introduce sparsity to the graph , only the edges whose weights are among the top-τ neighbors of each node are kept ( plus self-edges ) , i.e. , Wij = |X : ,i , : ∗X : ,j , :| if vj ∈ N ( vi ) , else 0 . Here , X : ,i , : and X : ,j , : are preprocessed signals in vi and vj , ∗ represents the normalized cross-correlation , and N ( vi ) represents the top-τ neighbors of vi . This method results in a unique directed , weighted graph for each input EEG clip . Figure 1c shows an example correlation graph with τ = 3 . 2.2.2 GRAPH NEURAL NETWORK . We adapt DCRNN ( Li et al. , 2018 ) , a recurrent neural network with graph diffusion convolutions , to model the spatiotemporal dependencies in EEG signals . DCRNN was initially developed for traffic forecasting , where the dynamics of traffic flow are modeled as a diffusion process . Similarly , we can also model the spatial dependency in EEG signals as a diffusion process , because an electrode can be influenced more by electrodes in its anatomical proximity ( measured by distance ) ( Acharya et al. , 2016 ) or functional proximity ( measured by correlation ) ( Sakkalis , 2011 ) . Specifically , the diffusion process is characterized by a bidirectional random walk on a directed graph G , which results in the following diffusion convolution ( Li et al. , 2018 ) : X : ,m ? Gfθ = K−1∑ k=0 ( θk,1 ( D −1 O W ) k + θk,2 ( D −1 I W ᵀ ) k ) X : ,m for m ∈ { 1 , ... , M } ( 1 ) where X ∈ RN×M is the preprocessed EEG clip at time step t ∈ { 1 , ... , T } with N nodes and M features , fθ is the convolution filter with parameters θ ∈ RK×2 , DO and DI are the out-degree and in-degree diagonal matrices of the graph , respectively , D−1O W and D −1 I W ᵀ are the state transition matrices of the outward and inward diffusion processes , respectively , and K is the number of maximum diffusion steps . For undirected graphs , the diffusion convolution is similar to ChebNet spectral graph convolution ( Defferrard et al. , 2016 ) up to a constant scaling factor , and thus can be computed using stable Chebyshev polynomial bases as follows ( Li et al. , 2018 ) : X : ,m ? Gfθ = Φ ( K−1∑ k=0 θkΛ k ) ΦᵀX : ,m = K−1∑ k=0 θkL kX : ,m = K−1∑ k=0 θ̃kTk ( L̃ ) X : ,m form ∈ { 1 , ... , M } ( 2 ) where T0 ( x ) = 1 , T1 ( x ) = x , and Tk ( x ) = 2xTk−1 ( x ) − Tk−2 ( x ) for k ≥ 2 are the bases of the Chebyshev polynomial , L = D− 1 2 ( D −W ) D− 12 = ΦΛΦᵀ is the normalized graph Laplacian , and L̃ = 2λmaxL− I is the scaled graph Laplacian mapping eigenvalues from [ 0 , λmax ] to [ −1 , 1 ] . We use Equation 1 for directed correlation graphs , and Equation 2 for undirected distance graph . Next , to model the temporal dependency in EEGs , we employ Gated Recurrent Units ( GRUs ) ( Cho et al. , 2014 ) , a variant of RNN with a gating mechanism . Specifically , the matrix multiplications in GRUs are replaced with diffusion convolutions ( or ChebNet spectral graph convolutions for undirected distance-based graph ) ( Li et al. , 2018 ) , allowing spatiotemporal modeling of EEG signals ( referred to as “ DCGRU ” ) : r ( t ) = σ ( Θr ? G [ X ( t ) , H ( t−1 ) ] + br ) u ( t ) = σ ( Θu ? G [ X ( t ) , H ( t−1 ) ] + bu ) ( 3 ) C ( t ) = tanh ( ΘC ? G [ X ( t ) , ( r ( t ) H ( t−1 ) ) ] +bC ) H ( t ) = u ( t ) H ( t−1 ) + ( 1−u ( t ) ) C ( t ) ( 4 ) Here , X ( t ) , H ( t ) denote the input and output of DCGRU at time step t respectively , σ denotes Sigmoid function , represents the Hadamard product , r ( t ) , u ( t ) , C ( t ) denote reset gate , update gate and candidate at time step t respectively , ? G denotes the diffusion convolution ( or ChebNet spectral graph convolution ) , Θr , br , Θu , bu , ΘC and bC are the weights and biases for the corresponding convolutional filters . Finally , for seizure detection and classification , the models consist of several stacked DCGRUs followed by a fully-connected layer .

Summary: The authors propose a graph-based representation from thresholded  Gaussian and linear (correlation) kernels (undirected connectivity) coupled with a diffusion convolutional recurrent network. Besides, a Fourier-based preprocessing is carried out with self-supervised (autoencoders) to initialize the network weights.  Experiments are performed on an EEG-based seizure detection and localization task. The seizure localization is conducted using occlusion and dropping approaches on EEG channels. Obtained results elucidate an interesting strategy for seizure analysis on a well-known public database.