Kati8/ICLR_2023_2025 huggingface dataset:
authors: Teng Fei
authors: Abhinav Uppal
authors: Ian L. Jackson
authors: David Wang
authors: Virginia R. de Sa
reviews: 
Comments: The paper presents an approach for reconstructing viewed images from EEG recordings. The authors employ a linear decoder that maps EEG signals to image latents, leveraging CLIP embeddings and latent diffusion models to achieve state-of-the-art quantitative reconstruction performance. The study explores the use of PCA and ICA components to capture luminance and hue-related information from EEG. The linear model provides interpretable EEG features for differentiating semantic categories of images and creates spatiotemporal semantic maps reflecting the temporal evolution of class-relevant semantic information.
Strengths: The paper introduces an application of EEG in the domain of image reconstruction, which is traditionally dominated by fMRI and MEG. The use of CLIP embeddings and latent diffusion models for EEG-based image reconstruction is innovative.

The methodology is rigorous, with a clear pipeline from EEG signal processing to image reconstruction. The use of multiple metrics for performance evaluation ensures a comprehensive assessment of the reconstruction quality.

The paper is well-organized, with a clear presentation of the reconstruction pipeline and detailed explanations of the methods and results. The figures and tables effectively support the textual content.
Weaknesses: 1. The authors do not clearly state the main contribution of this paper. Since reconstructing visual images from EEG is not a novel concept and has been extensively studied in the literature, it is unclear what new contributions this paper makes to the field. Is it merely the use of more powerful generative models? If the reconstructed image quality is higher, is it because the generative models used are superior, or because more information has been decoded from the brain signals?

2. It appears that the authors only report metrics for image reconstruction and do not calculate accuracy metrics for feature decoding from EEG.

3. The authors validate the effectiveness of their method using only one EEG dataset. Given the extensive literature on similar work based on fMRI or MEG, the authors should demonstrate the superiority of their proposed method across more datasets (e.g., NSD).

4. The authors should provide a deeper analysis of why visual stimulus information (such as color, shape, texture, etc.) can be reconstructed from EEG. Does this mean that EEG already carries enough information from the primary visual cortex, or is this information merely an illusion created by the generative models?
Recommendation: 5
Reviewer confidence: 5
Soundness/correctness: 3
Clarity: 3
reviews: 
Comments: The paper adapts an fMRI brain-to-image decoding pipeline based on linear Ridge regression and frozen image latents to EEG brain data. Linear decoders are trained per-subject and token of each latent representation (CLIP-Vision, CLIP-Text and VDVAE) on the averaged trials of each 16,540 train images (repeated 4 times each). Predicted latents on a test set of 200 images (averaged over 80 trials per image) are then fed to a Versatile Diffusion model, which generates images whose similarity to the ground truth is evaluated with different image metrics. Ablations suggest using the full EEG windows (800 ms) and all three latents yield the best performance. Additional analyses show (1) reconstructions based on PCA and ICA image features, (2) the effect of changing the layout of EEG electrodes, (3) the impact of swapping out subwindows between EEG examples on the reconstruction, and (4) spatiotemporal activation maps for three different categories of images.
Strengths: * Quality: Relevant analyses and ablations are proposed (e.g. impact of averaging trials, use of the different latents, neuroscience-inspired analyses). The time-swapping experiment yields interesting results.
* Clarity: The paper is overall clear and the different analyses are explained appropriately.
* Significance: The paper shows that a simple pipeline based on linear models can achieve good performance on an image decoding task.
Weaknesses: 1. The novelty of the work is limited. There are no new methodological contributions as the proposed approach appears to be a direct adaptation of Ozcelik & VanRullen (2023), but on EEG rather than fMRI. The proposed analyses (reconstructions based on PCA/ICA, effect of electrode layouts, etc.) appear to be new and may be interesting from a brain decoding perspective, but their significance is limited for the wider ML/AI community.
2. The comparison to existing work is limited. The results of Li et al. (2024) and Benchetrit et al. (2024) were for cross-subject models, while results presented here are for subject-specific models only. I believe the proposed approach may not be straightforward to apply to a multi-subject setting given the use of linear models, but this would be a better point of comparison. Moreover, the direct comparison to the results on the THINGS-MEG dataset is not appropriate as the brain data itself is different (EEG vs. MEG) and the number of image repetitions is not the same (1 vs. 4 presentations of training images; 12 vs. 80 presentations of test images), see Q4.
Recommendation: 5
Reviewer confidence: 4
Soundness/correctness: 2
Clarity: 2
reviews: 
Comments: The paper presents Perceptogram, a novel approach for reconstructing viewed images from EEG recordings. The method utilizes a linear decoder to map EEG signals to image latents, leveraging latent diffusion guided by CLIP embeddings for image reconstruction. The study also explores reconstructions using PCA and ICA components capturing luminance and hue-related information. The linear model provides interpretable EEG features for differentiating semantic categories, and the results demonstrate state-of-the-art quantitative reconstruction performance.
Strengths: The paper uses a liner regressor to correlate the EEG space and VAE/CLIP space and achieves good performance in EEG-based image recognition. Meaningful analysis has been conducted to provide insights, including ICA and PCA features, scalp-asymmetry effect, and spatial-temporal semantic maps.
Weaknesses: The linear model approach is a conventional method for brain visual reconstruction, offering high interpretability. However, further clarification is needed to enhance the reliability of this work:
1. How effectively does the regressor perform in mapping EEG embeddings to CLIP embeddings? Could specific visual components represented by different tokens show higher correlations with EEG features?
2. The study lacks key conclusions derived from performance metrics and analysis on the nature of visual information learned from EEG signals.
Recommendation: 5
Reviewer confidence: 4
Soundness/correctness: 3
Clarity: 2