The Case for Cleaner Biosignals: High-fidelity Neural Compressor Enables Transfer from Cleaner iEEG to Noisier EEG

Authors: Francesco S. Carzaniga, Gary Tom Hoppeler, Michael Hersche, Kaspar Schindler, Abbas Rahimi
Primary area: learning on time series and dynamical systems
Keywords: eeg, ieeg, intracranial eeg, electroencephalography, compression, transfer, seizure, seizure detection, motor imagery

Abstract:
All data modalities are not created equal, even when the signal they measure comes from the same source. In the case of the brain, two of the most important data modalities are the scalp electroencephalogram (EEG), and the intracranial electroencephalogram (iEEG). iEEG benefits from a higher signal-to-noise ratio (SNR), as it measures the electrical activity directly in the brain, while EEG is noisier and has lower spatial and temporal resolutions. Nonetheless, both EEG and iEEG are important sources of data for human neurology, from healthcare to brain–machine interfaces. They are used by human experts, supported by deep learning (DL) models, to accomplish a variety of tasks, such as seizure detection and motor imagery classification. Although the differences between EEG and iEEG are well understood by human experts, the performance of DL models across these two modalities remains under-explored. To help characterize the importance of clean data on the performance of DL models, we propose BrainCodec, a high-fidelity EEG and iEEG neural compressor. We find that training BrainCodec on iEEG and then transferring to EEG yields higher reconstruction quality than training on EEG directly. In addition, we also find that training BrainCodec on both EEG and iEEG improves fidelity when reconstructing EEG. Our work indicates that data sources with higher SNR, such as iEEG, provide better performance across the board also in the medical time-series domain. This finding is consistent with reports coming from natural language processing, where clean data sources appear to have an outsized effect on the performance of the DL model overall. BrainCodec also achieves up to a 64x compression on iEEG and EEG without a notable decrease in quality. BrainCodec markedly surpasses current state-of-the-art compression models both in final compression ratio and in reconstruction fidelity. We also evaluate the fidelity of the compressed signals objectively on a seizure detection and a motor imagery task performed by standard DL models. Here, we find that BrainCodec achieves a reconstruction fidelity high enough to ensure no performance degradation on the downstream tasks. Finally, we collect the subjective assessment of an expert neurologist, that confirms the high reconstruction quality of BrainCodec in a realistic scenario. The code is available at https://github.com/IBM/eeg-ieeg-brain-compressor.

Decision: accept (poster)
Reviewer scores: 3, 8, 8, 8, 6

Reviews and discussion:
<thread>
creation date:
1730125468962

thread_type:
official_review

<thread_object>
creator: reviewer_cgqc

<content>
summary:
This paper introduces BrainCodec, a novel neural network-based compressor for EEG and iEEG signals. BrainCodec demonstrates promising compression ratios of up to 64x while achieving relatively high reconstruction fidelity, particularly when transferring learning from iEEG to EEG.

strengths:
The core contribution of training a compressor on high-SNR iEEG data and applying it to low-SNR EEG data is novel and directly addresses a critical challenge in neuroelectrophysiological signal processing. The demonstrated improvement in reconstruction fidelity through this transfer learning is a significant finding.

weaknesses:
1) The reliance on PRD as the primary metric for evaluating reconstruction fidelity is inadequate. PRD, being a global measure, can mask localized distortions that may significantly affect downstream task performance. Subtle yet clinically relevant distortions in the reconstructed signals could easily go undetected by PRD, potentially impairing the accuracy of subsequent analyses.

2) The ablation study is incomplete. While the authors mention exploring different architectural choices, they do not present the results of such explorations. A systematic investigation varying key architectural parameters (e.g., quantizer types, encoder/decoder depths and kernel sizes, activation functions) is necessary to understand their impact on compression and reconstruction quality, providing a stronger justification for the chosen architecture.

questions:
1) What plans do the authors have to conduct a more in-depth analysis comparing the baseline BrainCodec with its GAN-augmented version, particularly in terms of frequency-specific characteristics of the reconstructed signals and their impact on downstream tasks?

2) How does the authors’ approach to evaluation change if they incorporate comparisons with established decoders from the literature, such as those cited in Zhang et al. (2023)? Would this increase the robustness and generalizability of the proposed compression technique?

3) In light of the recommendation to replace PRD with a more objective measure of signal quality derived from downstream motor imagery classification performance, how do the authors intend to implement this evaluation? 

4) Given the suggestion to shift the emphasis from PRD to more direct assessments of classification accuracy, how might this impact the interpretation of BrainCodec's effectiveness in real-world applications? What additional data or analyses might be needed to support this shift in evaluation focus?

Reference:
Zhang, Y., Qiu, S., & He, H. (2023). Multimodal motor imagery decoding method based on temporal spatial feature alignment and fusion. Journal of Neural Engineering, 20(2), 026009.
</content>
</thread_object>

</thread>
<thread>
creation date:
1730483530035

thread_type:
official_review

<thread_object>
creator: reviewer_hbil

<content>
summary:
The authors introduce RVQ-based neural signal compressor, BrainCodec, which achieves a $64\times$ compression on (i)EEG signals without a notable decrease in quality. The authors also indicate that clean data sources appear to have an outsized effect on the performance of DL model -- training BrainCodec on iEEG and then transferring to EEG yields higher reconstruction quality than training on EEG directly.

**Dataset**: Two iEEG datasets (i.e., SWEC iEEG, MC iEEG), and three EEG datasets (i.e., CHB-MIT, BONN, BCI Competition IV-2a) are used for this study. The downstream classification tasks include 2-way seizure detection (on iEEG datasets) and 4-way motor imagery (on BCI Competition IV-2a).

**Model**: The authors propose BrainCodec, which treats different channels independently. BrainCodec utilizes a stack of convolution layers to compress the raw (i)EEG signals into embeddings, and further quantizes them through Residual Vector Quantization.

**Experiement**: The authors utilize the reconstructed (i)EEG signals for downstream classification tasks.

strengths:
**Significance**: This study highlights the importance of clean data for pre-training. The authors find that training BrainCodec on iEEG and then transferring to EEG yields higher reconstruction quality than training on EEG directly. This finding may help the development of some VQ-based EEG foundation models (e.g., LaBraM[1]), as iEEG signals can further improve the representation quality compared to EEG signals.

**Clarity:** The text has a good structure and is well-written. The figures also help in understanding the method.

**Reference**

[1] Jiang W B, Zhao L M, Lu B L. Large brain model for learning generic representations with tremendous EEG data in BCI[J]. arXiv preprint arXiv:2405.18765, 2024.

weaknesses:
**Major**

1. It seems the compression ratio only depends on the depth of the Encoder (Line 201). How about the effect of different configurations of RVQ? Could the authors provide some ablation studies over the configurations of RVQ (i.e., the codex size, the dimensions of each code, etc.)?

2. As AE+RVQ has a certain ability for representation learning, how about the classification performance from the quantized embeddings (after quantizer)? 

**Minor**

1. I’m not sure whether the amount of iEEG datasets is enough, as the SWEC iEEG dataset only contains 14 hours of recording (across 15 subjects). Additional publicly available iEEG datasets the authors should be aware of:
   - Brain TreeBank (https://neurips.cc/virtual/2024/poster/97751): Their dataset (https://braintreebank.dev/) contains 43 hours of iEEG recording (not preprocessed).
   - Du-IN (https://arxiv.org/abs/2405.11459): Their dataset contains 36 hours of iEEG recording (after bipolar reference).

    To further prove the advantage of pre-training on clean data source (i.e., iEEG) -- **training BrainCodec on iEEG and then transferring to EEG yields higher reconstruction quality than training on EEG directly**, could the authors provide results of the model pre-trained on both SWEC iEEG dataset and Brain TreeBank? Besides, could the authors provide the effect of different reference methods (e.g., laplacian/bipolar reference instead of median reference, Line 274)?

2. More downstream classification can be included, e.g., 4 classification tasks in BrainBERT[1] based on the Brain TreeBank dataset.

**Reference**:

[1] Wang C, Subramaniam V, Yaari A U, et al. BrainBERT: Self-supervised representation learning for intracranial recordings[J]. arXiv preprint arXiv:2302.14367, 2023.

questions:
None
</content>
</thread_object>

</thread>
<thread>
creation date:
1730498450808

thread_type:
official_review

<thread_object>
creator: reviewer_wj41

<content>
summary:
The authors propose a method to map brain signals --- invasive (iEEG) and non-invasive (EEG) --- to representations that live in a lower-dimensional, discrete space. They do adapting a model designed for audio (Zeghidour et al., 2022 and Defossez et al., 2023): that model is a convolutional auto-encoder where the representation is quantized. The authors test the "quality" of the representations using two metrics: first, how well they can reconstruct the original signal and second, and how well the reconstructed signals predict downstream information. In both cases, compared to other methods, the authors are able to obtain higher metrics using lower-dimensional representations, which means effectively better compression.

strengths:
The paper is very clearly written and is impactful. The authors convincingly make the point the representations learnt using the cleaner iEEG signals are useful to reconstruct the noisier EEG signals, in fact, more so than if the representations were learnt using EEG signals in the first place. This is an instance of favorable generalization from one data modality (iEEG) to another (EEG). Their comparison of different cost functions for training, adversarial and non-adversarial, is interesting, especially as it does not conclude that adversarial training is always beneficial.

weaknesses:
- The authors "train the model on one single subject, and test the reconstruction on the remaining subjects". I am surprised that the authors obtain near-zero relative error in Figures 3 and 4. In EEG, the differences between subjects can be quite high, due to different physiologies, electrode positions, and reponse delays. So one would expect training on one subject and testing on another to lead to low-quality reconstruction. Can the authors comment on this surprising point?

- It would be relevant to include [1] in related works, as it uses a very similar model with the same name but applied to fMRI data. 

[1] BrainCodec: Neural fMRI codec for the decoding of cognitive brain states. Yuto Nishimura, Masataka Sawayama, Ayumu Yamashita, Hideki Nakayama, Kaoru Amano. 2024.

questions:
Could the authors address the points in the "weaknesses" section?
</content>
</thread_object>

</thread>
<thread>
creation date:
1730705417594

thread_type:
official_review

<thread_object>
creator: reviewer_p69f

<content>
summary:
BrainCodec leverages the higher signal-to-noise ratio (SNR) of iEEG data by training on iEEG and then transferring this model to compress noisier EEG data. The model achieves up to 64x compression with minimal loss of reconstruction fidelity, outperforming existing methods. Experimental results demonstrate that training on high-SNR iEEG data consistently provides better performance on EEG compression tasks than models trained directly on EEG. BrainCodec’s performance remains strong across various EEG and iEEG datasets, showing promise for reducing storage and transmission costs in clinical and research settings without compromising downstream task performance, such as seizure detection and motor imagery classification.

strengths:
The methodology is rigorously implemented, leveraging recent advancements in neural audio compression (e.g., quantized autoencoders) and adapted for EEG and iEEG data.BrainCodec’s performance is thoroughly validated across multiple datasets, including both iEEG (SWEC, MC) and EEG (CHB-MIT, BONN, BCI IV-2a), with comprehensive baseline comparisons. The cross-modality improvement, where a model trained on iEEG performs better on EEG than a model trained solely on EEG, adds to the literature, reinforcing the importance of high-SNR data for model training. The high compression ratios achieved (up to 64x without notable degradation) have immediate applications in reducing storage and transmission costs, relevant for clinical environments.

weaknesses:
Figures could use refinement for clearer interpretation, especially when comparing compression ratios and PRD values. While the model’s effectiveness across modalities is demonstrated, the paper doesn’t explore the impact of training on a combined EEG and iEEG dataset, which could potentially yield a more robust generalizable model. In the future more baseline comparisons would be useful — perhaps across the Mother of all BCI Benchmarks (MOABB), or braindecode datasets would further validate this approach. Also, further clinical validation would be valuable to ascertain the practical application of this work in the clinical setting. The cross-modality improvement, where a model trained on iEEG performs better on EEG than a model trained solely on EEG, adds to the literature, reinforcing the importance of high-SNR data for model training.

questions:
Could you clarify the rationale behind not training BrainCodec on a combined dataset of both EEG and iEEG? Do you anticipate any limitations or challenges with such an approach?

Did you consider other performance metrics beyond PRD and classification accuracy for assessing reconstruction fidelity?
</content>
</thread_object>

</thread>
<thread>
creation date:
1731139405858

thread_type:
official_review

<thread_object>
creator: reviewer_5spf

<content>
summary:
This paper focuses on the neural signal compression. The authors propose the BrainCodec model for both EEG and iEEG data compression. The proposed method has been tested on several data sets, and  achieves superior reconstruction fidelity compared with other competing methods. The experiments further validate its effectiveness for downstream tasks.

strengths:
1.	It is a universal compression model for both iEEG and EEG.
2.	It achieves high-fidelity compression of EEG signals up to a compression ratio of 64.
3.	It has been validated both experimentally and being rated by the neurologists.

weaknesses:
1.	BraiCodec is developed based on the quantized autoencoder design. The methodology novelty is not clear. 
2.	The experimental validation is insufficient. The effectiveness of the loss functions and different modules are not examined. 
3.	The evaluation metrics of the compression performance are limited, and the rational of choice of some parameters for evaluation are not stated.

questions:
1.	Neural signal compression has less been investigated. Real EEG/iEEG monitoring often requires real-time analysis. What are the application scenarios for their compression?
2.	Please pay attention to the table format.
</content>
</thread_object>

</thread>


Metareview:
metareview:
This work proposes a neural compressor for EEG and iEEG data that is inspired by the Encodec and Soundstream approaches proposed for audio signals in 2023 and 2022. Three reviewers acknowledge the originality of the approach and are enthusiastic about the contribution that is judged convincing and well presented.

The paper is considered relevant and timely for the ICLR 2024 community, in particular the researchers interested in deep learning on biosignals.

As the AC, I encourage the authors to improve their supplementary material that contains the code. Please add a readme and the tooling necessary to more easily replicate the model training. You may also consider adding a notebook and a checkpoint showing how to compress a public data with your model.