Title: Self-Calibrating BCIs: Ranking and Recovery of Mental Targets Without Labels
Venue: NeurIPS 2025
Accepted: yes
Keywords: EEG

brain-computer interfacing

calibration-free
Abstract: We consider the problem of recovering a mental target (e.g., an image of a face) that a participant has in mind from paired EEG (i.e., brain responses) and image (i.e., perceived faces) data collected during interactive sessions without access to labeled information. The problem has been previously explored with labeled data but not via self-calibration, where labeled data is unavailable. Here, we present the first framework and an algorithm, CURSOR, that learns to recover unknown mental targets without access to labeled data or pre-trained decoders. Our experiments on naturalistic images of faces demonstrate that CURSOR can (1) predict image similarity scores that correlate with human perceptual judgments without any label information, (2) use these scores to rank stimuli against an unknown mental target, and (3) generate new stimuli indistinguishable from the unknown mental target (validated via a user study, N=53). We release the brain response data set (N=29), associated face images used as stimuli data, and a codebase to initiate further research on this novel task.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper presents a self-calibrating brain-computer interface approach to identify and reconstruct a mental image target (a face that the user is thinking of) using only unlabeled EEG signals and stimuli embeddings. Unlike prior approaches that rely on labeled data or pre-trained decoders, CURSOR estimates target similarity scores in a continuous latent space derived from a generative model, without requiring any supervision. The method defines a score function based on how well a regressor trained on EEG-to-distance pairs generalises compared to a shuffled baseline. Using this score, CURSOR ranks and optimises hypothesised mental targets in latent space. The authors also describe a large EEG dataset collected under controlled conditions and they validate their approach through correlation with human perceptual judgments, control experiments, ablations and benchmarks.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n\n- Proposes the first self-calibrating BCI that operates in continuous latent spaces (e.g., 512D GAN embeddings) instead of discrete labels. Enables mental target recovery without calibration, labels, or pre-trained decoders.\n\n- Introduces a relative RMSE-based score comparing aligned and shuffled EEG-stimulus datasets. This breaks the circular dependency between unknown distances and regressors. An interesting approach.\n\n- Demonstrates high alignment with human perception, accurate ranking and successful optimisation to sub-perceptual distances.\n\n- An EEG dataset with continuous latent-space stimuli and task-specific engagement. \n\nWeaknesses:\n\n- The introduction is not really clear and fails to explain the core problem and contributions of the paper. While it gestures at mental target recovery and self-calibration, it does not clearly explain how CURSOR works or what makes the problem novel. Terms like \"continuous domains\" and \"mental targets\" are used without grounding the reader in how EEG signals and image embeddings are being connected. Key elements, like the role of the latent space, the unsupervised regression formulation or the scoring mechanism, are absent or vague. As a result, the actual technical framing only becomes understandable much later in the method section, which weakens accessibility for readers who are not experts.\n\n- The abbreviation CURSOR (Consistency-based Unsupervised Regression for Similarity-based Optimization and Ranking) feels contrived and only loosely tied to the actual method. While the method does involve unsupervised regression and ranking based on similarity scores, the notion of \"consistency\" is not a clearly defined or central mechanism in the algorithm. There is no consistency regularisation or self-distillation and there is only a relative RMSE comparison between aligned and shuffled EEG–stimulus pairs. Additionally, optimisation and ranking refer to downstream uses of the score, not core components of the algorithm itself.\n\n- Line 159 claims that \"This logarithmic sampling aligns with neuroscientific methodologies for studying perceptual thresholds and neural adaptation\". This needs to be cited.\n\n- Line 161 mentions “mitigations for these biases” but neither defines the biases nor explains the mitigations later. There is no clear follow-up. This statement is unsubstantiated and should be clarified or removed.\n\n- The justification for the two-stage evaluation, ranking in full space and optimisation in reduced-dimensional space, feels weak and underexplained. The authors cite computational challenges as the reason, but provide no concrete analysis of resource constraints or runtimes. More importantly, reducing the dimensionality (from 512D and 203D to 10D and 20D) changes the problem structure and may artificially boost optimisation performance. This shift is not examined and the paper treats it as a benign workaround rather than a methodological compromise.\n\n- The paper is difficult to read and hard to follow.\n\nQUESTIONS:\nCan you justify the need for optimisation in reduced-dimensional spaces beyond computational cost? Does optimisation in 10D/20D reliably reflect performance in the original 512D/203D latent spaces? \n\nLine 159 (logarithmic sampling) and line 161 (“bias mitigation”) lack citations or clarification. Can you point to specific justifications and follow-ups?\n\nHow stable is the CURSOR score across different random splits for shuffled baselines? Since the score is computed via a relative RMSE ratio, do you observe high variance depending on the shuffling?\n\nLIMITATIONS:\nCURSOR enables mental target reconstruction using EEG, which raises important ethical concerns around cognitive privacy and unintended inference. The possibility of inferring internal mental content without explicit user intention should be acknowledged. A brief discussion of these risks and possible safeguards (e.g., user consent, secure model deployment, opt-in constraints) would strengthen the paper's ethical framing.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 2, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-06-06"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis work introduces a self-calibrating framework that recovers mental targets (e.g., imagined faces) from EEG and image data without using labeled data or pre-trained models. In particular, the framework generates a hypothetical mental target as an embedding of a face the user might be imagining and constructs a dataset that relates observed faces to their distances from this target. It then trains an estimator to predict these distances from EEG data and compares its performance to that of an estimator trained on a mismatched dataset. The resulting score reflects how well the hypothesis matches the true mental target, with the score expected to peak when the hypothesis aligns with the actual target the user had in mind. Through this mechanism, one can predict perceptual similarity, rank images relative to an unknown target, and generate target-like images. The authors also release a new EEG dataset, face stimuli, and codebase to support future research.\n\nSTRENGTHS AND WEAKNESSES:\nA major strength of this work is its introduction of a label-free method for mapping EEG to image data through self-calibration, overcoming the limitations of prior approaches that depend on labeled EEG-image pairs. Another key contribution is the release of a paired EEG and image dataset, which can support and encourage further research in this area. \n\nAlthough I’m not deeply familiar with standard practices in BCI, I found it particularly clever that the authors operate in the latent space of images—this enables manipulation of ground-truth distances prior to the generation of observable data. \n\nAs acknowledged by the authors, a primary limitation of the approach lies in the search procedure: the framework restricts the hypothesis space to a predefined set that includes the true mental target. This makes the framework effective as a discriminator, but as a classifier, it requires manual curation and prior knowledge to ensure the target is within the search set. I also raise some concerns about the evaluation, which I elaborate on in the Questions section below.\n\nQUESTIONS:\n1. Did you experiment with non-linear estimators? I have my doubts a linear estimator is the best choice for this problem.\n2. For the dummy condition, in L239 the authors say “average of all distances in Gamma set”. Is this average computed in the training split or all the samples?\n3. I miss results with at least one supervised method from prior work. Can you include some? I understand the premise here is to avoid labeled data but it would be good to show the gap between the presented approach and the current label-based methods in terms of accuracy. \n4. In table 1, I would like to see top-k accuracy as well to have a better intuition of the performance. The distance metric is somewhat hard to use for comparison. Accuracy would also be needed if item 3 is included.\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 2, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-06-30"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes a self-calibrating BCI to retrieve face images from raw EEG responses projected to a lower-dimensional space. Instead of training the system for classification using labelled data, which is customary, it exploits an estimator to predict distances to potential targets in the EEG domain, with the estimator trained using distances between face images in an image latent space. An error function is used as a measure of predictability to locate a target image, an image that a human participant has first looked at and then been asked to remember, while a batch of images is quickly shown to the participant, and EEG responses are recorded.\n\nSTRENGTHS AND WEAKNESSES:\nThis paper introduces the first self-calibrating BCI for continuous domains, which is shown in experiments to work well for mental target retrieval using face images as targets. A dataset of as many as 9234 image-EEG pairs collected from 29 subjects is also presented. A user study is also being conducted, from which a strong correlation between the proposed scores and perceived similarities could be observed. \n\nIt is mentioned seven times on the first two pages that the proposed framework is the first self-calibrating BCI for continuous domains. The question is whether this is indeed the case. The problem formulation and assumptions in this paper seem to be very similar to those in [1], which contains a system for iteratively searching for a target face image by gradually locating a subregion in the image latent space using sequences of EEG responses. This paper does not assume labeled training examples either, which is true for other earlier examples referred to in the same paper.\n\nThe proposed error function, with its associated score function, measures how well an estimator can be trained to predict distances to a given target from EEG, using a set of paired images and EEG responses. The assumption is that if the error is small, the target image is likely to be correct, but if the target is incorrect, the error ought to be larger. From the experimental results, this seems to be a valid assumption, at least for an estimator based on linear regression, but not at all for one based on a multi-layer perceptron. The problem with this argument is that it will never work with more competent estimators. \n\nModern neural networks can be trained to predict any function, even if the distances in the training data are completely random, as long as the amount of data is limited, which is true in this case. For such a network, you are thus likely to get almost no errors at all. That is probably why the linear regressor seems to be preferred in this case. It could be that the distance function becomes more linear when the target is correct, not that the error becomes smaller and the predictability higher.\n\nIt is claimed that the proposed method includes an iterative search for a target, but it is not described how this search is done. Section 4.3 is vague indeed and includes few details. There is nothing that excludes the possibility that the actual search is indeed brute force through either all examples or a reduced set of examples. \n\nUnfortunately, there seem to be no experiments on any real face images. The retrieved images are almost exactly the same as the target one, if everything works well, but this is highly unrealistic in a real context, when you use anything beyond a limited set of generated images.\n\n[1] Rajabi et al., “Mental Face Image Retrieval Based on a Closed-Loop Brain-Computer Interface”, HCII, 2023.\n\nQUESTIONS:\nWhat does it mean that the proposed algorithm is self-calibrating? In what sense are other algorithms calibrated? Calibration typically means that you determine a set of parameters using a set of known examples, and then keep these parameters fixed in further experiments. What would those parameters be here?\n \nWhy would it be a novelty to work in continuous domains, and what does this mean in this context? The only relevant domains are the image domain, the latent image domain, and the EEG domain. Which of these are typically not continuous?\n\nHow is the iterative search being done? Iterative over what?\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 3, "REVIEWER_CONFIDENCE": 5, "CLARITY": 3, "IMPACT": 1, "ORIGINALITY": 3, "DATE": "2025-07-01"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper presents an approach to recover the image a participant holds in their mind given their EEG brain data in response to multiple images that differ more or less from the target image. This approach does not require labelled data and relies on previous research showing that distance between visual concepts can be predicted from the EEG. The authors collected a new EEG dataset on 31 participants, in which images of faces generated with a GAN (previously trained for this purpose) are presented in an RSVP oddball paradigm. The proposed algorithm, CURSOR, is then used to predict the distance (in latent space) between each shown image and a series of hypothesis images, given EEG responses. Results suggest that the version of the algorithm based on linear regressors enables retrieving or reconstructing the target image, or a perceptually indistinguishable variant, as opposed to shuffled/dummy baselines.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths\n* Quality: The submission appears technically sound, with claims supported by experimental results. For instance, the alignment between the CURSOR score and human judgment of distances is first validated, as well as the distance threshold for perceptually similar images (Figure 2). The comparison between the proposed linear regression-based model and controls (a version with a dummy regressor or a version trained on a shuffled dataset) supports the feasibility and effectiveness of the approach. Finally, the examples shown in Figures 4 and 10 overall support the quantitative results.\n* Significance: I am not familiar enough with the specific problem of mental visual target recovery to evaluate the impact this approach may have on the community. However, given the new unsupervised paradigm, and unique accompanying dataset, this submission is likely to help design further studies on this topic.\n* Originality: the submission’s approach to the recovery of mental visual targets appears to be novel, as is the data that has been collected as part of the study. Relevant previous work is cited, and the difference with existing paradigms is described (see Q3 for a suggestion to further clarify this).\n\nWeaknesses\n\n* Clarity: I found the manuscript overall difficult to navigate, though this may be due to me being less familiar with the topic. A key point that I think should be clarified is how CURSOR can be evaluated on all subjects and trials at once, i.e. what a N=9234 actually means (see Q1). Also, it was not obvious from the main text and appendix that the total number of examples (9234) was obtained by only keeping the 28.5% of oddball images from the 1120 images shown in a trial.\n\nQUESTIONS:\n1. I don’t understand how CURSOR can be evaluated on all subjects and trials at once. From what I understand, the score depends on how well the distance between a hypothesis image and a presented image can be modelled from EEG. If this distance is well modelled (i.e., high RMSE ratio), then the hypothesis image is a good candidate. However, since the target image differs from trial to trial, this process must be run for each trial individually. In this case, what does it mean to scale the dataset size beyond the number of related images available in a trial (e.g. in Figure 3)? I believe I must be missing something obvious here.\n2. What is the trajectory the related images are sampled from (line 157), i.e., is another point in latent space randomly sampled and used to provide the direction?\n3. This is purely a suggestion to help clarify how CURSOR differs from a supervised approach (conceptually, for the reader, but also in terms of performance) where labels are available. Would it be possible to integrate one such approach as a baseline? For instance, by training a predictor in leave-one-subject-out (or leave-one-trial-out) fashion that learns on ($e_i$, $z^*$) pairs, and comparing its ranking/predictions to the model.\n\nSmall comments\n\n* I assume that each channel is averaged across time within each time window, however it is not specified in the text, e.g. line 167.\n* Appendix B says there are 16 target images, but Section 5 (l156) says there are 17.\n\nLIMITATIONS:\nYes.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 2, "CLARITY": 2, "IMPACT": 2, "ORIGINALITY": 3, "DATE": "2025-07-04"}