SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Automated seizure detection is of great importance to epilepsy diagnosis and treatment. An emerging method used in seizure detection, stereoelectroencephalography (SEEG), can provide detailed and stereoscopic brainwave information. However, modeling SEEG in clinical scenarios will face challenges like huge domain shift between different patients and dramatic pattern evolution among different brain areas. In this study, we propose a Pretraining-based model for Patient-independent seizure detection (PPI) to address these challenges. Firstly, we design two novel self-supervised tasks which can extract rich information from abundant SEEG data while preserving the unique characteristics between brain signals recorded from different brain areas. Then two techniques, channel background subtraction and brain region enhancement, are proposed to effectively tackle the domain shift problem. Extensive experiments show that PPI outperforms the SOTA baselines on two public datasets and a real-world clinical dataset collected by us, which demonstrates the effectiveness and practicability of PPI. Finally, visualization analysis illustrates the rationality of the two domain generalization techniques.
Epilepsy is a chronic disease of brain that affects more than 50 million people worldwide, and a large proportion of patients have drug resistant epilepsy (DRE) which cannot be controlled by medication. Actually  $70\%$  of them can live seizure-free only if the seizure onset zone (SOZ) can be located and surgically removed [1].
To diagnose epilepsy, the most direct quantitative data to reflect brain function is the electrical activity of the patient's brain. One of the methods to monitor the brain activity is EEG, in which small sensors are attached to the scalp to pick up the electrical signals produced by the brain. Although EEG is widely adopted in brain activity recordings due to its simplicity and relatively low cost, as a non-invasive method, EEG fails to pinpoint the exact seizure focus when the SOZ is located in the deep structure of the brain. In view of the importance of SOZ localization in the treatment of patients with DRE [2], an emerging method called SEEG [3] is applied to locate foci by implanting electrodes in the deep brain. The contacts (also called channels) on the electrodes distribute around the suspected lesion across several brain areas, which provide stereoscopic recordings of the brain from both cortical and subcortical structures simultaneously [4].
Figure 1: Overview of the automated seizure detection pipeline. Firstly, the model is trained utilizing the SEEG data from existing patients. Then for real-world application, the doctors can upload the SEEG data of new patients and obtain the results of seizure detection predicted by the model, to easily enjoy the diagnostic assistance which is helpful for further treatment.
After collecting the SEEG recordings of patients, the process of epilepsy detection and diagnosis is traditionally treated as a manual task that highly depends on a few experienced neuroscientists, requiring considerable time and human resources [5]. Thus, automating this process could greatly improve the efficiency of clinical seizure detection. As shown in Fig. 1, firstly the model is trained using the labeled data from existing patients, and then the trained model will automatically detect seizures of the SEEG recordings of any new patient. Doctors can view SEEG waveforms and model predictions at the same time (shown on the right side of Fig. 1), and refer to the prediction results for more efficient diagnosis and further treatment. However, existing works for SEEG-based seizure detection mainly focus on the patient-specific setting [6, 7], which can only be trained and directly applied on the same patient due to the substantial differences of SEEG data between patients, resulting in hours even days of training for each new patient. Although a few studies follow the patient-independent setting, these works require manual sampling and denoising by a small number of experienced neurosurgeons, leading to a substantial data bias from real clinical data, which is also time consuming and inapplicable in clinical scenarios.
In fact, designing a model for patient-independent seizure detection on SEEG data under clinical requirements is quite an arduous task due to several unique challenges. Owing to the structural and functional differences of brain neural activities and the variation of invasive electrode numbers and locations (see Fig. 2) caused by individual differences in epileptogenic foci, the seizure patterns of SEEG data are quite different among individuals [8, 9]. Therefore, it is very difficult to perform patient-independent seizure detection on SEEG data. In addition, for epilepsy diagnosis, we have to detect seizures for each monitored brain areas to assist localization of SOZ. Meanwhile, for the fact that the cerebral is composed of multiple brain regions that exert a wide variety of functions [10], the
seizure patterns may greatly change across brain areas. Thus, to detect seizures for patients with different lesions, we should preserve the unique characteristics of brain areas. However, this is usually overlooked by existing works and is quite a challenging task under patient-independent setting.
To address the aforementioned challenges above, we propose a novel patient-independent seizure detection model called PPI, which can be successfully applied to clinical SEEG data. PPI adopts a self-supervised learning approach considering significant discrimination of brain areas and contextual coherence of SEEG signals to preserve the patterns of different channels and pretrain on large amount SEEG data. To handle the huge domain shift between patients, we propose channel background subtraction to align the distribution of the same brain region across different patients and brain region enhancement to mitigate the distribution shift brought by different brain regions. In the experimental phase, unlike existing works, we test our model on the real-world clinical data to verify the application value of our method. In summary, our key contributions comprise:
- To the best of our knowledge, we are the first to conduct patient-independent seizure detection on a large-scale real-world clinical SEEG dataset under clinical requirements.
Figure 2: Different SEEG electrode numbers and locations of two patients.
- We propose two novel self-supervised pretraining tasks to preserve the unique patterns of different channels, and two techniques including channel background subtraction and brain region enhancement to handle the domain shift between different patients.
- Extensive experiments show that PPi greatly outperforms the SOTA seizure detection methods (especially on the real-world clinical dataset), demonstrating the application value of our work.
SEEG-based seizure detection. SEEG is an emerging method applied in seizure detection, which can localize the SOZ more precisely than those noninvasive recording methods. However, due to the low-quality, large-amount, high-dimensionality characteristics of SEEG data, it is still challenging to develop an automatic approach in SEEG-based seizure detection. Ganti et al. [11] improve seizure detection by temporal Generative Adversarial Networks (TGAN). Chen et al. [6] adopt a graph structure to detect epileptic wave. Xiao et al. [12] propose an SOZ localization method via analyzing the long-term SEEG monitoring for preoperative planning of epilepsy surgery. Although researchers have explored some possible approaches for SEEG-based seizure detection, almost all of these works focus on a patient-specific setting, none of which can be applied in actual clinical scenarios.
Domain generalization on brain signals. Our goal is to predict epileptic seizures of SEEG from unseen patients, which can be abstracted as a domain generalization (DG) problem on brain signals. Yang et al. [13] develop a new DG method named ManyDG, that can scale to such many-domain problems for seizure detection task on EEG. Ayodele et al. [14] use transfer component analysis and LSTM to detect epilepsy on EEG data. Jeon et al. [15] propose a mutual information-driven method to conduct subject-invariant and class-relevant deep representation learning of EEG. For these current DG works on brain signals, most of them are conducted on EEG data rather than more informative SEEG. Although Wang et al. [16] study SEEG-based seizure detection on the patient-independent setting, they conduct experiments on datasets which are not only much smaller in size than practical records. The datasets are also manually denoised and sampled to a balanced positive-negative sample ratio which brings about a huge data bias from the real clinical data, indicating that their work is still far from clinical requirements.
Self-supervised learning on brain signals. Self-supervised learning is an effective approach when the labeled data is limited. In the field of neural signal (e.g. SEEG, EEG), the label is often hard to obtain. Thus, researchers have developed some SSL methods for this field. Banville et al. [17] utilize relative positioning, temporal shuffling and contrastive predictive coding as the pretext tasks for EEG. Mohsenvand et al. [18] and Kostas et al. [19] model EEG signal using contrastive learning. Cai et al. [20] propose a self-supervised learning framework for brain signals that can be applied to pretrain either SEEG or EEG data. However, these works do not explicitly align the distribution gaps between different domains, which is crucial under the DG setting, especially for data with large domain differences such as SEEG.
Seizure detection on SEEG data can be viewed as a time series classification (TSC) task. The SEEG recording of a patient is a multivariate time series  $\mathbf{T} \in \mathbb{R}^{N \times C}$ , where  $N$  is the length of series, and  $C$  is the number of channels. According to the existing works on TSC task [21-23], given an SEEG recording  $\pmb{x}_c = (x_1, x_2, \ldots, x_N)$  from channel  $c$  of a patient, we divide the contiguous data into small segments to construct data set  $\mathbb{S}_c = \{s_{c,0}, s_{c,1}, \ldots, s_{c,K-1}\}$  and the corresponding label set  $\mathbb{Y}_c = \{y_{c,0}, y_{c,1}, \ldots, y_{c,K-1}\}$ , where  $s_{c,k} = \{x_{l \times k+1}, \ldots, x_{l \times (k+1)}\}$  is the  $k$ -th segment data on channel  $c$  from  $\mathbf{T}$  ( $l$  is the length of each segment,  $K = \lfloor N / l \rfloor$  is the total number of segments on channel  $c$ ), and  $y_{c,k} \in \{0, 1\}$  is the label of  $s_{c,k}$ , which indicates whether the segment contains a seizure event ( $y_{c,k} = 1$ ) or not ( $y_{c,k} = 0$ ).
The problem is a DG study in epileptic diagnosis scenario, in which each patient is regarded as a domain. Conceptually, DG deals with a challenging setting where one or several different but related domain(s) are given, and the goal is to learn a model that can generalize to an unseen test domain.
Figure 3: Self-supervised pretraining of PPi. The pretraining is mainly performed by the two self-supervised tasks shown in the figure. In each task, the pairs sampled by contextual pairwise sampling will be properly processed according to the task and a discriminator is employed to discriminate the positive and negative samples (Sec. 4.1.1). The pretrained encoder will be used for the seizure detection (shown in Fig. 4).
Our goal is to utilize the data of labeled patients (source domains) to train a model which can be directly adopted to the data unseen patients (target domains).
In this section, we introduce the technical details of PPI. As shown in Fig. 3, under the patient-independent setting, we first pretrain an encoder by performing two self-supervised tasks (Sec. 4.1.1) to capture rich information from time domain while preserving unique patterns for each channel, which is consistent with the physiological mechanism of seizures. The pretrained encoder is then applied to the seizure detection task (shown in Fig. 4). To further extract information of SEEG data from a more comprehensive perspective, we also introduce the features from frequency domain by computing the spectral powers in different frequency bands [24] of each segment (Sec. 4.1.2). Finally, based on the learned representations, from time and frequency domains, we adopt channel background subtraction and brain region enhancement techniques to handle the challenge of domain shift (Sec. 4.2).
4.1 SEEG Representation Learning
4.1.1 Self-supervised Learning Framework
Self-supervised learning is effective in extracting features for time series [6, 25, 26] and robust to data imbalance [6, 27]. Considering that the SOZ localization is critical for epilepsy diagnosis and treatment, the seizures should be detected on each channel located in different brain areas. However, different brain regions exert a great variety of brain functions [10] and often exhibit different seizure patterns [4]. Therefore, learning representations that preserve the unique patterns of each channel is more consistent with the physiological mechanism of seizures. Thus we design channel discrimination and context swapping tasks (shown in Fig. 3) considering significant discrimination of channels and contextual coherence of SEEG data, respectively.
Contextual pairwise sampling. It is unreasonable to predict whether a short segment is onset or not without contextual data because seizure patterns may vary greatly between different patients and even between different channels of the same patient. To overcome the limitation, we are inspired by the empirical criteria of neurosurgeons in judging seizures that the waveform characteristics of the seizure segment are significantly different from those of nearby normal signals, which is also quantified by Smith [28] as more spikes and sharp waves. Therefore, we introduce the nearby contextual information of a segment in our self-supervised tasks to enhance the discrimination ability of segment representations for detecting seizures. Specifically, the input covers not only the target segment  $s_{c,k}$  which will be predicted (defined in Sec. 3), but also the  $h$  nearby segments on the left and right, which are called the context above and context below of  $s_{c,k}$  respectively (See Fig. 3). The whole sequence we sample for the target segments  $s_{c,k}$  is denoted as  $\mathbf{u}_{c,k}$ . Contextual pairwise sampling is then defined as the operation of randomly sampling two sequences  $\mathbf{u}_{c_1,k_1}^m, \mathbf{u}_{c_2,k_2}^m$  from channel  $c_1$  and  $c_2$  respectively, where  $m$  is the index for the sample set.
Channel discrimination. In order to preserve the unique characteristics of each channel, the model should identify how channels are different from each other. Motivated by this, we design a task guiding the model to differentiate whether the given two sequences are from the same channel or not. Specifically, we first sample a sequence pair  $\boldsymbol{u}_{c_1,k_1}^{m_1}, \boldsymbol{u}_{c_2,k_2}^{m_1}$  by contextual pairwise sampling, where the two sequences are sampled from the same or different channels with equal probability. Then  $\boldsymbol{u}_{c_1,k_1}^{m_1}, \boldsymbol{u}_{c_2,k_2}^{m_1}$  will be encoded to feature vectors  $\boldsymbol{h}_{c_1,k_1}^{m_1}, \boldsymbol{h}_{c_2,k_2}^{m_1}$ . We next obtain the difference vectors by computing an element-wise absolute difference:
$$
\boldsymbol {h} _ {c d} ^ {m _ {1}} = \operatorname {a b s} \left(\boldsymbol {h} _ {c _ {1}, k _ {1}} ^ {m _ {1}} - \boldsymbol {h} _ {c _ {2}, k _ {2}} ^ {m _ {1}}\right). \tag {1}
$$
After that, we utilize a discriminator to predict whether the sampled sequence pair comes from the same channel or not. We apply the binary cross-entropy, denoted as  $\mathcal{L}_{cd}$ , as the loss function of the channel discrimination task. Meanwhile, to avoid representation collapse and exploit the information from time domain, we apply a decoder to reconstruct the original sequences, denoted as  $\hat{\pmb{u}}_{c_1,k_1}^{m_1},\hat{\pmb{u}}_{c_2,k_2}^{m_1}$ . Based on  $M_{1}$  sequence pairs sampled in the channel discrimination task, the objective function for the reconstruction task is defined as:
$$
\mathcal {L} _ {\text {r e c}} = \sum_ {m _ {1} = 1} ^ {M _ {1}} \left(\left\| \boldsymbol {u} _ {c _ {1}, k _ {1}} ^ {m _ {1}} - \hat {\boldsymbol {u}} _ {c _ {1}, k _ {1}} ^ {m _ {1}} \right\| ^ {2} + \left\| \boldsymbol {u} _ {c _ {2}, k _ {2}} ^ {m _ {1}} - \hat {\boldsymbol {u}} _ {c _ {2}, k _ {2}} ^ {m _ {1}} \right\| ^ {2}\right). \tag {2}
$$
Context swapping. Given that the model leverages contextual information to detect seizures, we also need to enhance the coherence semantic uniqueness of SEEG data. We then propose a task leading the model to identify whether the context has been replaced by that of other channels. First, we also adopt contextual pairwise sampling to sample two sequences from different channels. For the sampled sequence pair, we perform the swapping operation according to the following rules: (1) swap their context above with a probability of 0.25; (2) swap their context below with a probability of 0.25; (3) do not swap, otherwise. We do not perform reconstruction task here by reason of possibly corrupted sequences. After the processed data is fed into the encoder, we concatenate the encoded representations of the sequence pair to obtain the joint vectors. Finally, an MLP-based discriminator is utilized to discriminate whether the joint vectors are from swapped sequence pair or not. The binary cross-entropy, denoted as  $\mathcal{L}_{cs}$ , is also employed as the objective function of the context swapping.
Putting the objective functions all together, the overall self-supervised model will be jointly trained according to the objective function given by  $\mathcal{L}_{ssl} = \mathcal{L}_{rec} + \mathcal{L}_{cd} + \mathcal{L}_{cs}$ .
4.1.2 Frequency Domain Features
The proposed self-supervised learning tasks have a good capability of extracting features from time domain. To exploit SEEG data from a more comprehensive view, we adopt PSD (power spectral density), which has the ability to track the transient changes before and during seizure [29], to extract features from frequency domain. The PSD of a signal describes the distribution of the signal's total average power over frequency. Specifically, we first split the frequency domain into several bands according to the standard description for rhythmic activity [24]: (1)  $\theta$  (4-8Hz), (2)  $\alpha$  (8-13Hz), (3)  $\beta$  (13-30Hz), (4)  $\gamma 1$  (30-50Hz), (5)  $\gamma 2$  (50-70Hz), (6)  $\gamma 3$  (70-90Hz), (7)  $\gamma 4$  (90-110Hz), (8)  $\gamma 5$  (110-128Hz). The absolute spectral power of a signal in a frequency band is then computed as the
Figure 4: Overall architecture of PPi. The representations encoded from the pretrained encoder and spectral powers (Sec. 4.1.2) are aggregated by a self-attention strategy to obtain the aggregated representation  $\mathbf{z}_{c,k}$ .  $\mathbf{z}_{c,k}$  is then be processed by channel background subtraction and brain region enhancement to obtain the difference representation  $\mathbf{z}_{c,k}^{\prime}$  for seizure detection (Sec. 4.2).
logarithm of the sum of PSD coefficients within that frequency band. Mathematically, for segment  $s_{c,k}$ , the absolute spectral power in the  $i$ -th frequency band is computed as:
$$
P _ {c, k} ^ {i} = \log \sum_ {\omega \in \operatorname {b a n d} (i)} P S D _ {\mathbf {s} _ {c, k}} (\omega), \quad i \in \{1, 2, \dots , 8 \}. \tag {3}
$$
In order to capture underlying correlations between different frequency bands and local patterns of contextual segments, the spectral powers will be encoded by a CNN-based encoder to obtain the representations from frequency domain of SEEG signals.
After obtaining the time domain representations from self-supervised learning tasks and frequency domain representations from spectral powers, a self-attention strategy is used to adaptively aggregate them to obtain the segment representations  $\mathbf{z}_{c,k}$ .
4.2 Patient-independent Seizure Detection
In order to solve the domain shift problem, we propose an assumption from a higher-order perspective that the differential distribution of normal and seizure segments of channels located in the same brain region would be similar in different patients. The assumption is reasonable because of the similarity in structure and function of the same brain region among patients, and the case study in Sec. 5.6 further supports it. Therefore, based on the assumption, we design two techniques called channel background subtraction and brain region enhancement, which fully consider the characteristics of SEEG, to align the data distribution between patients.
Channel background subtraction. Channel background subtraction enables the model to exclusively focus on the differences between seizure and normal segments of each channel, which aligns the distribution of channels within the same brain region across patients. In the field of neuroscience, Staba and Worrell [30] prove that the background brain activity, which refers to the brain activity when individuals are at rest or during sleep, contains important information about brain function and dysfunction in epilepsy. In our scenario, given the patient's limited mobility while monitored by SEEG, background is regarded as the SEEG signal in the absence of epileptic seizures. Therefore, we calculate the background representation  $e_{c,k}$  for the segment representation  $z_{c,k}$  as the average of all the seg
Algorithm 1 Background Representation Calculation
for  $c = 0,1,\ldots ,C - 1$  do  $\begin{array}{rl} & {\pmb{e}_{c,0}\leftarrow \pmb{z}_{c,0}}\\ & {n_c\leftarrow 1}\\ & {\hat{y}_{c,0}\leftarrow 0^1} \end{array}$    
for  $k = 1,2,\dots ,K - 1$  do  $\begin{array}{r}\pmb {e}_{c,k}\gets \pmb{e}_{c,k - 1}\\ \text{if}\hat{y}_{c,k - 1} <   0.5\text{then}\\ \pmb{e}_{c,k}\leftarrow \frac{\pmb{e}_{c,k}\times n_c + \pmb{z}_{c,k - 1}}{n_c + 1}\\ n_c\leftarrow n_c + 1 \end{array}$    
end if   
 $\pmb{z}_{c,k}^{\prime}\gets \pmb{z}_{c,k} - \pmb{e}_{c,k}$    
Predict the seizure probability  $\hat{y}_{c,k}$  using  $\pmb{z}_{c,k}^{\prime}$    
end for   
end for
ment representations currently (before index  $k$ ) predicted to be normal in channel  $c$ . We show the details of the iterative updating method in Algo. 1. In channel background subtraction,  $\mathbf{z}_{c,k}$  will subtract  $\mathbf{e}_{c,k}$  to obtain the difference representation  $\mathbf{z}_{c,k}^{\prime}$ . The operation aligns the distribution of channels located in the same brain region between patients, which is supported by the case study in Sec. 5.6.
<sup>1</sup>Since the patient must be in normal state at the start of the SEEG recording, it is reasonable to set  $\hat{y}_{c,0}$  to 0.
Brain region enhancement. Channel background subtraction has aligned channel representations with similar latent differential distributions. However, with prior information of brain regions, explicit supervised learning can further constrain the differential distribution of channels in the same brain region between different patients, thus enhancing the generalization of PPI. Therefore, we extend the classification task from binary to multi-class, so that the classifier not only predicts whether the seizure occurs, but also determines which brain region the seizure locates in $^2$ . Specifically, taking  $z_{c,k}^{\prime}$  as input, the classifier outputs a tuple  $(i_{c,k}, y_{c,k})$ , where  $i_{c,k}$  is the index of the brain region that  $s_{c,k}$  locates in and  $y_{c,k} \in \{0,1\}$  is the prediction of whether  $s_{c,k}$  is seizure or not. For the implementation, we construct integer multi-class labels by converting the tuple  $(i_{c,k}, y_{c,k})$  to an integer  $y_{c,k}^{\prime}: y_{c,k}^{\prime} = i_{c,k} * 2 + y_{c,k}$ . Therefore, the parity of  $y_{c,k}^{\prime}$  reflects whether the prediction result corresponds to a normal state or a seizure. We use a cross-entropy loss as the objective function of brain region enhancement.