SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Public datasets. The public datasets used in our paper, MAYO and FNUSA [32], are collected from two institutions: the Mayo Clinic (Rochester, Minnesota, United States of America) and St. Anne's University Hospital (Brno, Czech Republic), respectively. The MAYO dataset has 18 patients in total, including 56730 normal samples and 15227 seizure samples, respectively. The FNUSA dataset has 13 patients in total, including 94560 normal samples and 52470 seizure samples, respectively. For each dataset, we first remove the power line noise and down sample the dataset to  $500\mathrm{Hz}$ . Then we divide the patients into 6 groups without overlapping (details in App. B). We randomly choose 5 groups as the source domains (4 of which are used for training and 1 for validation) and the remaining group serves as the target domain. The experiments are repeated on all groups to test the average performance.
Clinical dataset. The clinical SEEG dataset we collect is from a first-class hospital. For the patients, 4 to 10 invasive electrodes with 52 to 126 channels are implanted in the brain to obtain  $1000\mathrm{Hz}$  SEEG signals. Since the clinical data are recorded with a high frequency on multiple channels, the dataset has more than 738 hours of recording and contains 123 patient files with an average size of 7.1 GB each. As for the annotation, professional neurosurgeons participate in seizure labeling. The positive sample ratio of a single patient in the dataset is around 0.004 on average, which is extremely imbalanced. We remove the power line noise and down sample the dataset to  $250\mathrm{Hz}$ . For the 7 patients in the clinical dataset, we split the patients into training, validation and test set with 5, 1 and 1 patients, respectively. We also repeat the experiments on all patients to obtain an overall results.
To evaluate the performance of the models, we conduct adequate experiments on two public datasets (MAYO and FNUSA) and the real-world clinical dataset. All experiments run on a Linux system with 2 CPUs (AMD EPYC 7H12 64-Core Processor) and 4 GPUs (NVIDIA GeForce RTX 3090). Our code is available at https://github.com/yzz673/PPi_public.
Evaluation Metrics. To comprehensively evaluate the experimental results, we use precision, recall, F1 and F2 as evaluation metrics. Here the F-score can be calculated by  $F_{\beta} = \frac{(1 + \beta^{2})\times\text{precision}\times\text{recall}}{\beta^{2}\times\text{precision} + \text{recall}}$ . Usually, F2 is adopted in critical applications that value information retrieval more than accuracy (i.e., accepting a relatively large number of false positives but virtually guaranteeing that all the true positives are found). In our medical scenario, F2 is more valued than F1, since ignoring any seizure is costly in diagnosis.
Baselines. We compare our model with some DG methods for brain signals (SICR [15], SEEG-Net [16]). Also, we compare with other DG algorithms designed for more general fields
In our study, brain regions are divided according to a medical standard template called automated anatomical labeling (AAL) [31], which is a digital atlas of the human brain. AAL defines 116 different regions in total, of which 90 are in the cerebrum and 26 are in the cerebellum.
(CDANN [33], CORAL [34], GroupDRO [35], MLDG [36], MMD [37], MTL [38], SANDMask [39], SD [40], SelfReg [41], TRM [42], VREx [43], IB-ERM [44], IB-IRM [44]) and adopt two different feature extractors (TCN [45], MiniRocket [46]) to evaluate their performance based on different kinds of features. Furthermore, we select some self-supervised learning approaches for brain signals (BENDR [19]) and time series (Franceschi et al. [47]). More details of the baselines are shown in App. A.
Table 1: Average performance of patient-independent seizure detection tasks on two public datasets and our clinical dataset. The v indicates the first in a column,  $\underline{\mathbf{v}}$  indicates the second, and  $*\mathrm{v}$  indicates the third. The performance with standard deviation is given in App. C.
Dataset Model	MAYO	FNUSA	Clinical
Pre.	Rec.	F1	F2	Pre.	Rec.	F1	F2	Pre.	Rec.	F1	F2
TCN	CDANN	16.13	*68.14	24.74	37.84	32.71	69.62	43.09	54.88	1.42	44.20	2.72
CORAL	17.01	64.08	25.73	38.30	33.87	65.46	43.63	53.84	1.59	47.24	3.03
GroupDRO	17.26	64.25	26.02	38.68	33.24	62.14	41.90	51.27	1.53	47.13	2.91
MLDG	18.88	61.38	27.96	*40.39	32.81	56.81	40.41	48.24	28.85	6.68	2.55
MMD	15.79	67.41	24.45	37.68	32.87	*69.38	43.51	55.30	1.90	46.84	3.26
MTL	16.13	51.87	22.56	31.74	32.70	53.10	38.48	44.78	1.43	9.19	2.30
SANDMask	16.47	68.66	25.00	38.15	32.16	59.46	39.62	48.59	1.34	*47.22	2.60
SD	15.29	60.70	23.10	34.67	34.30	65.54	43.77	53.88	1.52	46.46	2.91
SelfReg	9.42	58.65	15.50	26.29	25.35	66.75	35.87	48.79	0.92	44.48	1.76
TRM	16.94	59.60	24.47	35.67	34.32	62.26	43.39	51.93	1.53	47.34	2.91
VREx	16.74	60.41	25.06	36.87	33.17	61.75	41.84	51.11	10.98	42.17	2.78
IB-ERM	17.27	64.29	26.04	38.71	34.34	62.94	42.93	52.22	1.53	47.34	2.91
IB-IRM	16.75	60.45	25.08	36.90	34.26	63.06	43.10	52.44	1.52	45.65	2.89
MiniRockets	CDANN	16.22	15.44	15.14	15.20	48.73	36.97	38.26	37.02	2.09	45.27	3.98
CORAL	47.61	25.30	27.57	25.46	68.46	46.55	52.50	48.36	1.68	42.54	3.20
GroupDRO	41.99	34.02	35.98	34.49	69.79	48.74	55.31	50.83	1.37	46.95	2.64
MLDG	10.32	50.04	15.35	24.70	34.67	63.56	37.88	46.75	0.52	15.58	1.01
MMD	22.34	21.82	19.58	20.33	69.33	46.63	50.04	47.06	5.10	39.97	3.95
MTL	21.67	46.11	27.72	35.03	56.85	59.71	*56.93	58.28	12.57	45.79	4.02
SANDMask	4.32	33.33	7.40	13.14	12.56	33.33	18.14	24.88	1.49	44.87	2.59
SD	37.57	30.90	32.46	31.10	68.74	50.59	53.26	50.92	6.40	39.73	9.43
SelfReg	31.09	14.46	18.02	15.65	61.18	33.26	39.29	34.99	16.33	42.06	*14.51
TRM	35.97	40.34	35.51	37.15	66.32	53.63	53.50	52.68	5.00	33.31	7.81
VREx	38.63	33.92	33.37	32.88	65.23	55.20	54.34	53.91	7.47	44.50	11.37
IB-ERM	35.02	43.92	36.79	39.62	66.34	53.41	53.17	52.40	4.94	43.01	8.12
IB-IRM	36.27	43.22	*37.36	39.57	64.67	54.02	53.69	53.00	8.56	44.63	11.18
BENDR	23.26	45.02	25.90	30.23	40.45	37.22	34.42	34.01	2.48	28.99	3.58	5.79
Franceschi et al.	34.21	40.98	33.94	35.15	43.28	50.56	44.03	48.97	2.62	44.74	4.26	9.65
SICR	10.18	4.56	6.10	5.06	23.51	7.16	9.79	8.01	*25.34	29.22	9.19	9.80
SEEG-Net	*45.41	45.62	43.54	44.22	*69.39	53.75	60.02	*55.99	20.06	32.81	20.82	22.92
PPi	49.85	69.67	54.35	61.07	71.73	70.81	70.61	70.55	29.76	47.59	30.92	35.51
5.3 Experimental Result
Tab. 1 summarizes the main results of our model and baselines on the two public datasets and the clinical dataset. Overall, our model outperforms all baselines on every metric on the three datasets, which demonstrates the excellent performance of PPi in the patient-independent seizure detection on SEEG data.
On the public datasets, PPi improves the performance $^3$  by  $38.10\%$  and  $24.98\%$  over the best-performing baseline model in terms of F2-score respectively, showing that PPi has stronger generalization ability in seizure detection than other baselines. On our clinical dataset, PPi improves the performance by  $54.93\%$  on F2-score. Compared with other baselines that treat all channels equally, our designed self-supervised tasks can preserve more unique characteristics of each channel. These tasks help our model to learn more informative representations from SEEG data, leading to a much better performance on channel-level seizure detection. For SEEG-Net [16] which also focuses on patient-independent seizure detection on SEEG, it can maintain a more balanced precision and recall than other baselines, and achieves almost the second/third highest F1-score in all datasets. The possible reason for its good performance is that SEEG-Net also considers the contextual information of SEEG data like our model. However, PPi still outperforms SEEG-Net by a large margin, as PPi
Here we calculate the relative improvement.
not only preserve unique characteristics of each channel, but also introduces the extra information of brain regions to enhance the generalization ability. In particular, we achieve better improvement on clinical dataset (54.93% on F2-score) compared with the public datasets (38.10% and 24.98% on F2-score) mainly because the self-supervised pretraining on unlabeled data reduces reliance on labels, which makes PPI more adaptive to extremely imbalanced dataset than other supervised methods.
To evaluate the effectiveness of each component in our model, we first conduct ablation experiments on four model variants to verify the effectiveness of the proposed self-supervised framework, including: (1) PPI- $SSL_{1}$ : PPi without the channel discrimination task; (2) PPI- $SSL_{2}$ : PPi without the context swapping task; (3) PPI- $SSL$ : PPi without all self-supervised tasks; (4) PPI-reconstruction: PPi without reconstruction loss. Then we use PPi-power: PPi without features of spectral powers, to demonstrate the benefits of introducing PSD based features. For the aggregation strategy, we replace the self-attention with mean pooling (denoted as PPi-self attention). Another two experiments are conducted to verify the significant generalization ability of our two techniques respectively, which are denoted as PPi-background and PPi-brain region. Due to the requirement of brain region labeling in brain region enhancement, the lack of such labels makes brain region enhancement inapplicable to the two public datasets.
Table 2: Results of ablation experiments on two public datasets and the clinical dataset.
Dataset Model	MAYO	FNUSA	Clinical
Pre.	Rec.	F1	F2	Pre.	Rec.	F1	F2	Pre.	Rec.	F1	F2
PPi-SSL1	43.69	48.01	31.22	35.37	58.46	59.39	56.05	57.27	20.68	44.23	19.13	24.63
PPi-SSL2	40.69	38.32	34.72	35.74	47.69	50.53	46.79	48.42	20.75	42.82	18.72	23.73
PPi-SSL	35.65	22.89	25.92	23.77	69.00	44.29	53.04	47.29	7.53	30.75	10.94	16.08
PPi-reconstruction	44.01	50.29	34.15	36.89	60.78	62.31	59.48	60.02	22.04	44.84	21.23	25.01
PPi-power	48.60	31.43	29.01	28.31	62.89	54.48	56.77	55.13	28.13	30.48	19.75	20.87
PPi-self attention	48.82	60.20	51.15	55.41	65.14	66.91	62.17	63.98	28.67	46.98	29.56	32.59
PPi-background	46.80	31.31	31.08	30.05	57.83	50.20	51.68	50.28	20.42	35.40	21.83	26.74
PPi-brain region	-	-	-	-	-	-	-	-	18.01	46.86	21.53	28.33
PPi	49.85	69.67	54.35	61.07	71.73	70.81	70.61	70.55	29.76	47.59	30.92	35.51
The comparison results of the ablation experiments on all the three datasets are presented in Tab. 2. It shows that PPi beats other model variants on all metrics, proving the contribution of each component in our model. The performances of the variants that remove the self-supervised tasks drop greatly, which shows that our designed self-supervised tasks are capable of extracting informative representations from large amount imbalanced SEEG data. PPi-background and PPi-brain region cannot achieve performances that are competitive with the full model, validating the strong generalization ability of PPi to handle the huge domain shift across different patients. The improvement is empowered by the channel background subtraction and brain region enhancement techniques, which will be explored further in Sec. 5.6.
5.5 Hyperparameter Analysis
The contexts half length  $h$  (i.e. the length of context above or context below) is an important hyperparameter in PPI. Thus we evaluate the performance on the clinical dataset under different contexts half length (shown in Fig. 5). The results exhibit an increase of performance as  $h$  becomes larger, which illustrate that introducing the contexts of the target segment to increase the receptive field allows the model to compare the target segment with its nearby waveforms, resulting in better performance. However, the increase of contexts half length is also accompanied by a higher computational overhead, necessitating consideration of trade-offs when selecting the value of  $h$ .
Figure 5: Performance on clinical dataset under different contexts half length  $h$ .
5.6 Case Study
Visual analysis of channel background subtraction. To handle domain shift, PPi adopts channel background subtraction which has the ability to align the distribution of the same brain region from different patients. In the visual analysis, we visualize such ability by comparing the aggregated representation  $z_{c,k}$  (before subtraction) and the difference representation  $z_{c,k}'$  (after subtraction) using t-SNE [48] plots. Specifically, we select two channels which locate in the same brain region from two different patients (denoted as  $p_i$ ,  $p_j$ ) (results for more patients are shown in App. D.1). The t-SNE plots of their aggregated representations and difference representations are shown in Fig 6. The aggregated representations of  $p_i$  and  $p_j$  obviously exhibit
different distributions. After channel background subtraction, the distributions of the samples from  $p_i$  and  $p_j$  are very close, which illustrates that channel background subtraction successfully aligns the distribution space of the channels located in the same brain region between patients.
normal from  $p_i$  seizure from  $p_i$  normal from  $p_j$  seizure from  $p_j$
aggregated representations
Figure 6: t-SNE plots of aggregated representations (left) and difference representations (right) of two channels from the same brain region of patients  $p_i$  and  $p_j$ .
difference representations
Confusion matrix of brain region enhancement. In order to demonstrate the effectiveness of brain region enhancement, we calculate the confusion matrix of the multi-classification. Fig. 7 shows the confusion matrix from one of the patients (the confusion matrix of all the patients are shown in App. D.2), in which the vertical axis represents the multi-class label and the horizontal axis represents the multi-class prediction results. In the confusion matrix, most samples are distributed on the main diagonal, which reflects the good performance of the multi-classification task, illustrating the effectiveness of brain region enhancement.
Figure 7: Confusion matrix of brain region enhancement.
In this paper, we propose PPi to conduct patient-independent seizure detection on SEEG in the clinical scenario. To detect seizures more accurately, PPi adopts a self-supervised pretraining strategy to extract information from SEEG signals while preserving the unique characteristics of each channel. Furthermore, we propose channel background subtraction and brain region enhancement to improve the generalization ability of PPi. Extensive experiments demonstrate the superior performance of PPi in the patient-independent seizure detection on two public and a clinical SEEG datasets (Sec. 5.3). The case study (Sec. 5.6) further illustrate the effectiveness of our proposed two techniques to reduce the huge domain shift between different patients.
Limitations and future works. Compared with EEG, SEEG is an emerging technique and the related research is limited. Although our work outperforms other methods by a large margin on the clinical dataset, in applications, the predicted results of PPI are mainly serve as a reference to assist doctors to achieve more efficient clinical diagnosis and treatment, rather than completely replace doctors in seizure detection. For the application, we have reached a cooperation with a first-class hospital and a related institution that is responsible for the development of an application software. In the future, our model will be integrated into the software to assist doctors in seizure diagnosis.
[33] Ya Li, Xinmei Tian, Mingming Gong, Yajing Liu, Tongliang Liu, Kun Zhang, and Dacheng Tao. Deep domain generalization via conditional invariant adversarial networks. In ECCV, pages 624-639, 2018.
[34] Baochen Sun and Kate Saenko. Deep coral: Correlation alignment for deep domain adaptation. In ECCV, pages 443-450. Springer, 2016.
[35] Shiori Sagawa, Pang Wei Koh, Tatsunori B Hashimoto, and Percy Liang. Distributionally robust neural networks for group shifts: On the importance of regularization for worst-case generalization. arXiv preprint arXiv:1911.08731, 2019.
[36] Da Li, Yongxin Yang, Yi-Zhe Song, and Timothy Hospedales. Learning to generalize: Meta-learning for domain generalization. In AAAI, volume 32, 2018.
[37] Haoliang Li, Sinno Jialin Pan, Shiqi Wang, and Alex C Kot. Domain generalization with adversarial feature learning. In CVPR, pages 5400-5409, 2018.
[38] Gilles Blanchard, Aniket Anand Deshmukh, Ürun Dogan, Gyemin Lee, and Clayton Scott. Domain generalization by marginal transfer learning. The Journal of Machine Learning Research, 22(1):46-100, 2021.
[39] Soroosh Shahtalebi, Jean-Christophe Gagnon-Audet, Touraj Laleh, Mojtaba Faramarzi, Kartik Ahuja, and Irina Rish. Sand-mask: An enhanced gradient masking strategy for the discovery of invariances in domain generalization. arXiv preprint arXiv:2106.02266, 2021.
[40] Mohammad Pezeshki, Oumar Kaba, Yoshua Bengio, Aaron C Courville, Doina Precup, and Guillaume Lajoie. Gradient starvation: A learning proclivity in neural networks. Advances in Neural Information Processing Systems, 34:1256-1272, 2021.
[41] Daehee Kim, Youngjun Yoo, Seunghyun Park, Jinkyu Kim, and Jaekoo Lee. Selfsupervised contrastive regularization for domain generalization. In ICCV, pages 9619-9628, 2021.
[42] Yilun Xu and Tommi Jaakkola. Learning representations that support robust transfer of predictors. arXiv preprint arXiv:2110.09940, 2021.
[43] David Krueger, Ethan Caballero, Joern-Henrik Jacobsen, Amy Zhang, Jonathan Binas, Dinghuai Zhang, Remi Le Priol, and Aaron Courville. Out-of-distribution generalization via risk extrapolation (rex). In ICML, pages 5815-5826. PMLR, 2021.
[44] Kartik Ahuja, Ethan Caballero, Dinghuai Zhang, Jean-Christophe Gagnon-Audet, Yoshua Bengio, Ioannis Mitliagkas, and Irina Rish. Invariance principle meets information bottleneck for out-of-distribution generalization. Advances in Neural Information Processing Systems, 34: 3438-3450, 2021.
[45] Colin Lea, Rene Vidal, Austin Reiter, and Gregory D Hager. Temporal convolutional networks: A unified approach to action segmentation. In ECCV, pages 47-54. Springer, 2016.
[46] Angus Dempster, Daniel F Schmidt, and Geoffrey I Webb. Minirocket: A very fast (almost) deterministic transform for time series classification. In SIGKDD, pages 248-257, 2021.
[47] Jean-Yves Franceschi, Aymeric Dieuleveut, and Martin Jaggi. Unsupervised scalable representation learning for multivariate time series, 2020.
[48] Laurens Van der Maaten and Geoffrey Hinton. Visualizing data using t-sne. Journal of machine learning research, 9(11), 2008.
A Details of Baselines
Firstly, we compare our model to some cross-subject methods on brain signals:
- SICR [15]: a framework that learns class-relevant and subject-invariant feature representations, which shows a promising performance in non-invasive brain-computer interface.
- SEEG-Net [16]: a model that can address the problems of sample imbalance, cross-subject domain shift, and poor interpretability and realizes high-sensitivity SEEG pathological activity detection. The source code of SEEG-Net is not released, so we implement it by ourselves to conduct the experiments.
Secondly, to further compare our model to some DG algorithms in more general areas, experiments were conducted with methods as follows:
- CDANN [33]: an end-to-end conditional invariant deep DG approach by leveraging deep neural networks for domain-invariant representation learning.
- CORAL [34]: an unsupervised domain adaptation method that aligns the second-order statistics of the source and target distributions with a linear transformation.
- GroupDRO [35]: a model coupling group DRO models with increased regularization, where DRO allows to learn models that instead minimize the worst-case training loss over a set of groups.
- MLDG [36]: a model agnostic training procedure for DG, which simulates train/test domain shift during training by synthesizing virtual testing domains within each mini-batch.
- MMD [37]: an adversarial autoencoder framework to learn a generalized latent feature representation across domains.
- MTL [38]: a representative framework for DG, which augments the original feature space with the marginal distribution of feature vectors.
- SANDMask [39]: a masking strategy, which determines a continuous weight based on the agreement of gradients, in order to control the amount of update in each step of optimization under the notion of Out-of-Distribution (OOD) Generalization.
- SD [40]: a regularization method aimed at decoupling feature learning dynamics, improving accuracy and robustness in cases hindered by gradient starvation.
- SelfReg [41]: a regularization method for DG based on contrastive learning, self-supervised contrastive regularization.
- TRM [42]: a robust estimation criterion that is specifically geared towards optimizing transfer to new environments.
- VREx [43]: a penalty on the variance of training risks as a simpler variant based on a form of robust optimization over a perturbation set of extrapolated domains.
- IB-ERM [44]: a DG method that improve generalization via minimizes the empirical risk over multiple domains.
- IB-IRM [44]: a DG method that improve generalization via minimizes the invariant risk over multiple domains.
Furthermore, we choose some self-supervised learning approaches on brain signals or general time series as our baselines:
- BENDR [19]: A self-supervised training model that learn compressed representations to model completely novel raw EEG sequences recorded with differing hardware, and different subjects performing different tasks.
- Franceschi et al [47]. This work combines an encoder based on causal dilated convolutions with a novel triplet loss employing time-based negative sampling, obtaining general-purpose representations for variable length and multivariate time series.
B.1 Public Datasets
The group dividing strategy and detailed statistics of the two public datasets are shown in Tab. 3. The datasets are publicly available to use under CC0 license and might be downloaded from https://springernature.figshare.com/collections/Multicenter_intracranial_EEG_dataset_for_classification_of graphoelements_and Artifactual Signals/4681208.
Table 3: Group dividing strategy and detailed statistics of two public datasets.
Dataset Group id	MAYO	Dataset Group id	FNUSA
Patient id	Artifacts	Seizure	Normal	Patient id	Artifacts	Seizure	Normal
Group1	0	2318	0	330	Group1	1	0	1912	0
18	1700	0	3126	5	5059	1527	5452
21	58	3432	0
Total	4076	3432	3456	Total	5059	3439	5452
Group2	1	0	883	8653	Group2	2	2892	1657	7809
9	740	0	0	9	0	6750	0
19	5613	0	0
Total	6353	883	8653	Total	2892	8407	7809
Group3	2	466	1923	399	Group3	3	12	8076	0
5	1002	0	6583	4	8463	0	0
16	3699	0	177	12	1343	7710	38217
Total	5167	1923	7159	Total	9818	15786	38217
Group4	3	4636	0	2057	Group4	6	0	1554	962
4	2063	0	790	7	5416	7738	2689
23	761	2747	644
Total	7460	2747	3491	Total	5416	9292	3651
Group5	6	12873	0	0	Group5	8	18	1896	20860
7	0	0	25951	10	5786	4260	1545
8	0	2816	0
Total	12873	2816	25951	Total	5804	6156	22405
Group6	14	0	3426	498	Group6	11	3339	4072	2890
17	4096	0	6098	13	181	5318	14136
20	1278	0	1424
Total	5374	3426	8020	Total	3520	9390	17026
B.2 Clinical Dataset
The detailed information of the clinical dataset is shown in Tab. 4 and the sample rate of all the patients is  $1000\mathrm{Hz}$ .
Table 4: Details information of the clinical dataset.
Patient id	Time (hours)	#Electrodes	#Channels	Positive sample ratio	#Timestamps
0	121.4	10	126	0.0028	180632184
1	34.7	4	52	0.0020	21642853
2	167.7	10	126	0.0011	241488147
3	73.7	8	116	0.0077	102649651
4	161.3	8	112	0.0037	195457035
5	54.3	7	93	0.0016	43684998
6	125.2	5	59	0.0167	70605073
C Experimental Results with Standard Deviation
The average performance with standard deviation of MAYO, FNUSA and clinical dataset are shown in Tab. 5, Tab. 6 and Tab. 7, respectively.
D Full Results of Case Study
D.1 Visual Analysis of Channel Background Subtraction