SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Table 7: Details of the pre-training dataset.
Subject id	Recording hours	#Electrodes	#Channels	#Sample points
1	235.34	10	124	105,055,776,000
2	82.39	4	52	15,423,408,000
3	393.09	10	120	169,814,880,000
4	137.45	10	133	65,811,060,000
5	214.22	8	116	89,458,272,000
6	386.70	8	101	140,604,120,000
7	111.55	7	67	26,905,860,000
8	207.42	5	47	35,095,464,000
9	759.80	11	134	366,527,520,000
Table 8: Details of the downstream dataset.
Subject id	Recording hours	#Seizure timestamps	#Normal timestamps
A	3.46	3,298,000	383,154,000
B	4.63	539,000	216,288,000
C	5.23	2,129,000	563,091,000
D	6.00	3,427,000	622,977,000
E	6.00	1,311,000	544,090,000
F	6.00	496,000	361,282,000
G	5.92	7,278,000	243,255,000
H	5.98	3,330,000	717,831,000
F Details of Brant with Different Scales
To explore the effect of model scales on performance, we additionally pre-train three variants of Brant with smaller size: Brant-tiny, Brant-small and Brant-medium on the same dataset. Detailed configurations of Brant and its variants are shown in Tab. 9.
Table 9: Configurations of different scales of Brant.
Config
Model	Temporal/Spatial Encoder Layer	Model Dimension	Inner Dimension	Parameter Number
Brant-tiny	8/4	768	2048	68.49M
Brant-small	8/4	1024	2048	103.90M
Brant-medium	12/5	1280	3072	249.22M
Brant	12/5	2048	3072	505.68M
G Details of the Generalization Ability Analysis
As a supplement to the experiments on downstream dataset in the main text, to further verify the generalization ability of Brant on more subjects with more heterogeneity, we evaluated the model on data of 31 unseen subjects from two public datasets named MAYO and FNUSA [42]. The MAYO dataset includes intracranial neural signals of 18 subjects collected from Mayo Clinic (Rochester, Minnesota, United States of America). The FNUSA dataset comprises intracranial data of 13 subjects recorded from St. Anne's University Hospital (Brno, Czech Republic).
Table 10: Performance on the short- and long-term forecasting tasks on two public datasets.
Model\Task & Dataset	Short-term signal forecasting	Long-term signal forecasting
MAYO	FNUSA	MAYO	FNUSA
MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE
RP	1.3881	3.0642	1.3786	3.0317	1.4025	3.1277	1.3839	3.0562
TS	0.9464	1.4572	0.9505	1.4846	0.8318	1.1138	0.8353	1.1213
CPC	0.8009	1.0311	0.7922	1.0306	0.7979	1.0254	0.7898	1.0249
BENDR	0.9673	1.4950	0.9507	1.4585	1.0943	2.4782	1.0921	2.4325
MVTS	1.1027	1.7043	1.0838	1.6627	1.2475	2.8251	1.2450	2.7731
BrainBERT	*0.7971	1.0110	0.7829	0.9971	*0.7943	*1.0183	*0.7872	*1.0202
PatchTST	0.7969	*1.0227	*0.7879	*1.0237	0.7479	0.9815	0.7498	1.0183
TSTCC	1.4434	2.7430	1.4532	3.0083	1.7217	2.7587	1.7246	2.8697
TFC	2.1544	4.0941	2.1689	4.4900	2.5697	4.1174	2.5741	4.2831
CoST	0.8812	1.2783	0.8935	1.2895	0.7996	1.0299	0.8094	1.0582
Brant	0.6860	0.8064	0.6915	0.8249	0.7329	0.8856	0.7374	0.9118
Table 11: Performance on the frequency-phase forecasting task and the imputation task on two public datasets.
Model\Task & Dataset	Frequency-phase forecasting	Imputation
MAYO	FNUSA	MAYO	FNUSA
ph. PLV	freq. MAE	ph. PLV	freq. MAE	MAE	MSE	MAE	MSE
RP	0.2194	1.0222	0.2232	1.0225	0.7025	0.8340	0.7079	0.8556
TS	0.5064	0.0501	0.5076	0.0517	0.7381	0.8935	0.7427	0.9215
CPC	*0.5601	0.1183	*0.5616	0.1119	0.7725	0.9623	0.7737	0.9796
BENDR	0.5459	0.0638	0.5486	0.0639	0.8906	1.0880	0.9001	1.1510
MVTS	0.5389	0.0672	0.5397	0.0674	0.8097	0.9891	0.8187	1.0463
BrainBERT	0.5075	0.0355	0.5091	0.0356	0.7908	1.0045	0.7812	1.0039
PatchTST	0.5627	*0.0455	0.5640	*0.0470	0.4934	0.4753	0.5073	0.4896
TSTCC	0.1285	0.2756	0.1288	0.2752	0.7411	0.9201	0.7451	0.9488
TFC	0.0714	0.5367	0.0712	0.5366	0.7640	0.9486	0.7681	0.9781
CoST	0.5279	0.0482	0.5292	0.0484	*0.4964	*0.4783	*0.5088	*0.4914
Brant	0.5824	0.0304	0.5814	0.0305	0.4887	0.4622	0.5020	0.4835
Table 12: Performance on the seizure detection task on two public datasets.
Task &
Dataset
Model	Seizure detection
MAYO	FNUSA
Acc.	Prec.	Rec.	F1	F2	Acc.	Prec.	Rec.	F1	F2
Spec Power	58.79	38.77	44.27	41.34	43.05	58.08	38.48	46.42	42.08	44.58
Rhythm Spec	58.06	31.23	*45.06	36.89	41.39	56.27	30.53	47.64	37.21	42.84
aEEG	58.36	31.19	43.98	36.50	40.65	57.83	36.52	38.70	37.58	38.25
SEEG-Net	57.92	38.38	46.72	*42.14	44.77	58.21	39.12	*49.29	*43.62	*46.85
RP	86.51	46.19	20.29	28.20	22.86	70.19	59.29	19.88	29.78	22.93
TS	72.55	48.72	17.72	25.98	20.30	69.00	52.88	22.64	31.70	25.56
CPC	*87.29	52.90	24.01	33.03	26.96	68.34	50.40	24.98	33.40	27.78
BENDR	80.95	23.95	21.11	22.44	21.62	62.28	5.70	34.04	9.76	17.06
MVTS	69.19	53.30	24.99	34.03	27.96	67.11	49.66	19.68	28.19	22.38
BrainBERT	85.39	44.13	44.92	44.52	*44.76	68.31	44.96	49.50	47.12	48.52
PatchTST	87.39	53.02	29.42	37.84	32.29	*69.94	*55.81	26.08	35.55	29.19
TSTCC	77.26	10.47	9.16	9.77	9.40	63.83	24.40	11.36	15.50	12.71
TFC	82.09	11.41	4.91	6.87	5.54	68.71	29.36	5.05	8.62	6.05
CoST	87.39	*53.02	29.42	37.84	32.29	65.92	36.34	33.65	34.95	34.16
Brant	89.40	58.78	70.80	64.23	68.02	83.51	83.60	54.18	65.75	58.28
The results of generalization study are given in Tab. 10, Tab. 11 and Tab. 12. Compared to other baseline models, Brant holds the consistent SOTA performance on these downstream tasks on the unseen 31 heterogeneous subject data collected from different hospitals in different countries, highlighting the generalization ability of our model.