Self-Supervised Graph Neural Networks for Improved Electroencephalographic Seizure Analysis

1 INTRODUCTION . Seizures are among the most common neurological emergencies in the world ( Strein et al. , 2019 ) . Seizures can be chronic as in the case of epilepsy , a neurological disease affecting 50 million people worldwide ( WHO , 2019 ) . Clinically , definitive detection of a seizure is only the first step in seizure diagnosis . An important subsequent step is to classify seizures into finer-grained types––such as focal versus generalized seizures––for identifying epilepsy syndromes , targeted therapies , and eligibility for epilepsy surgery ( Fisher et al. , 2017 ) . Scalp electroencephalography ( or “ EEG ” ) plays a critical role in seizure detection and classification . Clinically , EEG-based seizure detection and classification are performed by a trained EEG reader who visually examines a patient ’ s EEG signals over time periods ranging from hours to days . However , this manual analysis is extremely resource- and time-intensive , and thus automated algorithms could greatly accelerate seizure diagnosis and improve outcomes for severely ill patients . Figure 1 : ( a ) EEG electrode placement in the standard 10-20 system . ( b ) Distance-based EEG graph . ( c ) An example correlation-based EEG graph . ( d ) Overview of our methods . The inputs to the models are the EEG graphs , where each node feature corresponds to the preprocessed EEG signals in the respective channel . Self-edges are not shown for better visualization . Although a large number of studies have attempted automated seizure detection ( Rasheed et al. , 2020 ; Siddiqui et al. , 2020 ; Shoeibi et al. , 2021 ; O ’ Shea et al. , 2020 ; Saab et al. , 2020 ) or seizure classification ( Raghu et al. , 2020 ; Asif et al. , 2020 ; Iesmantas & Alzbutas , 2020 ; Ahmedt-Aristizabal et al. , 2020 ; Roy et al. , 2019 ) , several challenges remain largely unaddressed . First , most recent studies use convolutional neural networks ( CNNs ) that assume Euclidean structures in EEG signals or spectrograms ( Rasheed et al. , 2020 ; Shoeibi et al. , 2021 ; Raghu et al. , 2020 ; Asif et al. , 2020 ; Iesmantas & Alzbutas , 2020 ; Ahmedt-Aristizabal et al. , 2020 ; Roy et al. , 2019 ; O ’ Shea et al. , 2020 ; Saab et al. , 2020 ) . However , assumption of Euclidean structure ignores the natural geometry of EEG electrodes and the connectivity in brain networks . EEGs are measured by electrodes placed on a manifold ( i.e . patient ’ s scalp ) ( Figure 1a ) , and thus have inherent non-Euclidean structures . Graphs are a data structure that can represent complex , non-Euclidean data ( Chami et al. , 2020 ; Bronstein et al. , 2017 ) , and graph theory has been extensively used in modeling brain networks ( Bullmore & Sporns , 2009 ) . We therefore hypothesize that graph-based modeling approaches can better represent the inherent non-Euclidean structures in EEGs in a manner that improves both the performance and the clinical utility of seizure detection and classification models . Although traditional graph theory has been used ( Supriya et al. , 2021 ) , only a few deep learning studies have modeled EEGs as graphs for seizure detection . However , these graph-based studies were limited to nonpublic ( Covert et al. , 2019 ) or small datasets ( Craley et al. , 2019 ; Li et al. , 2021 ) , did not leverage modern self-supervised approaches or examine seizure classification ( Cisotto et al. , 2020 ; Zhao et al. , 2021 ; Li et al. , 2021 ) . Second , certain seizure types ( e.g . clonic seizures ) are rare by nature . Training machine learning models that perform well on these rarer seizure classes using traditional supervised learning approaches is challenging , which could explain the performance difference between major and minority seizure types in prior studies ( Raghu et al. , 2020 ; Iesmantas & Alzbutas , 2020 ; AhmedtAristizabal et al. , 2020 ) . Several studies have investigated an alternative , self-supervised training strategy ( Banville et al. , 2020 ; Mohsenvand et al. , 2020 ; Kostas et al. , 2021 ; Martini et al. , 2021 ; Xu et al. , 2020 ) , but they did not model EEGs as graphs or address automated seizure classification . Prior works have shown that self-supervised pre-training significantly improves model performance on data with imbalanced labels in the field of computer vision ( Yang & Xu , 2020 ; Liu et al. , 2021 ) . Hence , we hypothesize that self-supervised pre-training can help improve our graph model performance on rare seizure types . Moreover , a large portion of EEG signals generally do not have seizures . Self-supervised pre-training strategy would allow the model to leverage the abundant nonseizure EEGs that are readily available in the dataset . Lastly , for seizure detection and classification models , the ability to not only provide a single prediction across all EEG channels , but also to provide the interpretability and the ability to localize seizures would be clinically useful for informing treatment strategy . While prior studies ( Saab et al. , 2020 ; Covert et al. , 2019 ) have shown qualitative visualization for model interpretability , none have quantitatively assessed the model ’ s ability to localize seizures . In this work , we aim to address these limitations in prior automated seizure detection and classification studies . First , we propose a graph-based modeling approach for EEG-based seizure detection and classification . Specifically , we propose two EEG graph structures that capture EEG sensor geometry ( Figure 1b ) or dynamic brain connectivity ( Figure 1c ) , and we extend Diffusion Convolutional Recurrent Neural Network ( DCRNN ) ( Li et al. , 2018 ) , an RNN with graph diffusion convolutions , to model the spatiotemporal dependencies in EEGs ( Figure 1d ) . Second , we improve DCRNN performance using a self-supervised pre-training strategy of predicting the preprocessed EEG signals for the next time period without requiring additional data or labels . Finally , we propose quantitative metrics to assess our model ’ s ability to localize seizures . In summary : • We propose two EEG graph structures that capture ( 1 ) the natural geometry of EEG sensors or ( 2 ) dynamic connectivity in the brain , and show that building a recurrent graph neural network ( GNN ) based on these representations yields models for seizure detection and classification that outperform previous approaches on a large public dataset ( 5,499 EEGs ) . • We propose a self-supervised pre-training strategy to further improve our recurrent GNN model performance , particularly on rare seizure types . To our knowledge , our study is the first to date that combines graph-based modeling and self-supervised pre-training for EEGs . By leveraging graph structure and self-supervision , our method achieves 0.875 Area Under the Receiver Operating Characteristic Curve ( AUROC ) on seizure detection and 0.749 weighted F1-score on seizure classification , outperforming previous approaches on both seizure detection and classification on this large public dataset . Moreover , our selfsupervised pre-training method substantially improves classification of rare seizure types ( e.g . 47 points increase in combined tonic seizure accuracy over baselines ) . • We propose a quantitative model interpretability analysis that can be used to assess a model ’ s ability to localize seizures , which is critical to determining the course of treatment for seizures . We show that by leveraging graph structure and self-supervision , our method precisely localizes 25.4 % focal seizures , providing 21.9 points improvement over a prior state-of-the-art CNN . Finally , by displaying the identified seizure regions on raw EEG signals and EEG graphs , our approach could provide valuable insights that support more effective seizure diagnosis in real-world clinical settings . 2 METHODS . 2.1 SEIZURE DETECTION AND CLASSIFICATION PROBLEM FORMULATION . The goal of seizure detection is to predict whether a seizure exists within an EEG clip , and the goal of seizure classification is to predict the seizure type given a seizure EEG clip . Following a prior study ( Saab et al. , 2020 ) , we examine our model ’ s capability for fast and slow detection and classification over 12-s and 60-s EEG clips , respectively . 2.2 GRAPH-BASED MODELING FOR EEGS . 2.2.1 REPRESENTING EEGS AS GRAPHS . We represent an EEG clip as a graph G = { V , E , W } , where V denotes the set of nodes ( i.e . EEG electrodes/channels ) , E denotes the set of edges , and W is the adjacency matrix . We propose the following two methods of constructing the EEG graph . Distance graph . To represent the natural geometry of EEG electrodes , we compute edge weight Wij by applying a thresholded Gaussian kernel ( Shuman et al. , 2013 ) to the pairwise Euclidean distance between vi and vj , i.e. , Wij = exp ( − dist ( vi , vj ) 2 σ2 ) if dist ( vi , vj ) ≤ κ , else 0 . Here , dist ( vi , vj ) is the Euclidean distance between electrodes vi and vj according to the standard 10-20 EEG electrode placement ( Jasper , 1958 ) , σ is the standard deviation of the distances , and κ is the threshold for sparsity . This results in a universal undirected , weighted graph for all EEG clips . Based on preliminary experiments and EEG domain knowledge , we chose κ = 0.9 because it results in a reasonable graph that also resembles the EEG montage ( longitudinal bipolar and transverse bipolar ) widely used clinically ( Acharya et al. , 2016 ) . Figure 1b shows the distance graph with κ = 0.9 . In Appendix K , we explore different values of κ as well as constructing the distance graph using a Gaussian kernel with a pre-specified bandwidth . Correlation graph . To capture dynamic brain connectivity , we define the edge weight Wij as the absolute value of the normalized cross-correlation between the preprocessed signals in vi and vj . To introduce sparsity to the graph , only the edges whose weights are among the top-τ neighbors of each node are kept ( plus self-edges ) , i.e. , Wij = |X : ,i , : ∗X : ,j , :| if vj ∈ N ( vi ) , else 0 . Here , X : ,i , : and X : ,j , : are preprocessed signals in vi and vj , ∗ represents the normalized cross-correlation , and N ( vi ) represents the top-τ neighbors of vi . This method results in a unique directed , weighted graph for each input EEG clip . Figure 1c shows an example correlation graph with τ = 3 . 2.2.2 GRAPH NEURAL NETWORK . We adapt DCRNN ( Li et al. , 2018 ) , a recurrent neural network with graph diffusion convolutions , to model the spatiotemporal dependencies in EEG signals . DCRNN was initially developed for traffic forecasting , where the dynamics of traffic flow are modeled as a diffusion process . Similarly , we can also model the spatial dependency in EEG signals as a diffusion process , because an electrode can be influenced more by electrodes in its anatomical proximity ( measured by distance ) ( Acharya et al. , 2016 ) or functional proximity ( measured by correlation ) ( Sakkalis , 2011 ) . Specifically , the diffusion process is characterized by a bidirectional random walk on a directed graph G , which results in the following diffusion convolution ( Li et al. , 2018 ) : X : ,m ? Gfθ = K−1∑ k=0 ( θk,1 ( D −1 O W ) k + θk,2 ( D −1 I W ᵀ ) k ) X : ,m for m ∈ { 1 , ... , M } ( 1 ) where X ∈ RN×M is the preprocessed EEG clip at time step t ∈ { 1 , ... , T } with N nodes and M features , fθ is the convolution filter with parameters θ ∈ RK×2 , DO and DI are the out-degree and in-degree diagonal matrices of the graph , respectively , D−1O W and D −1 I W ᵀ are the state transition matrices of the outward and inward diffusion processes , respectively , and K is the number of maximum diffusion steps . For undirected graphs , the diffusion convolution is similar to ChebNet spectral graph convolution ( Defferrard et al. , 2016 ) up to a constant scaling factor , and thus can be computed using stable Chebyshev polynomial bases as follows ( Li et al. , 2018 ) : X : ,m ? Gfθ = Φ ( K−1∑ k=0 θkΛ k ) ΦᵀX : ,m = K−1∑ k=0 θkL kX : ,m = K−1∑ k=0 θ̃kTk ( L̃ ) X : ,m form ∈ { 1 , ... , M } ( 2 ) where T0 ( x ) = 1 , T1 ( x ) = x , and Tk ( x ) = 2xTk−1 ( x ) − Tk−2 ( x ) for k ≥ 2 are the bases of the Chebyshev polynomial , L = D− 1 2 ( D −W ) D− 12 = ΦΛΦᵀ is the normalized graph Laplacian , and L̃ = 2λmaxL− I is the scaled graph Laplacian mapping eigenvalues from [ 0 , λmax ] to [ −1 , 1 ] . We use Equation 1 for directed correlation graphs , and Equation 2 for undirected distance graph . Next , to model the temporal dependency in EEGs , we employ Gated Recurrent Units ( GRUs ) ( Cho et al. , 2014 ) , a variant of RNN with a gating mechanism . Specifically , the matrix multiplications in GRUs are replaced with diffusion convolutions ( or ChebNet spectral graph convolutions for undirected distance-based graph ) ( Li et al. , 2018 ) , allowing spatiotemporal modeling of EEG signals ( referred to as “ DCGRU ” ) : r ( t ) = σ ( Θr ? G [ X ( t ) , H ( t−1 ) ] + br ) u ( t ) = σ ( Θu ? G [ X ( t ) , H ( t−1 ) ] + bu ) ( 3 ) C ( t ) = tanh ( ΘC ? G [ X ( t ) , ( r ( t ) H ( t−1 ) ) ] +bC ) H ( t ) = u ( t ) H ( t−1 ) + ( 1−u ( t ) ) C ( t ) ( 4 ) Here , X ( t ) , H ( t ) denote the input and output of DCGRU at time step t respectively , σ denotes Sigmoid function , represents the Hadamard product , r ( t ) , u ( t ) , C ( t ) denote reset gate , update gate and candidate at time step t respectively , ? G denotes the diffusion convolution ( or ChebNet spectral graph convolution ) , Θr , br , Θu , bu , ΘC and bC are the weights and biases for the corresponding convolutional filters . Finally , for seizure detection and classification , the models consist of several stacked DCGRUs followed by a fully-connected layer .

Summary: The paper presents a method for seizure detection and classification. In particular, the method is self supervised, based on graph neural network and use EEG signals. The authors report significant performance in detection and classification, as well as provide methods for qualitative evaluation of model interpretability.