Reducing Semantic Mismatch in Brain-to-Text Decoding Through Personalized Multimodal Masking
Authors: Jiaxuan Chen
Yu Qi
Yueming Wang
Gang Pan
Venue: ICLR 2026 Poster (2026)
Keywords: Brain-to-text reconstruction
Neural decoding
Semantic decoding
Multimodal learning
Primary area: applications to neuroscience & cognitive science

Abstract: The rapid progress of large vision-language models (VLMs), such as CLIP, has spurred the development of a wide range of neural decoding frameworks. Nevertheless, most existing approaches still suffer from semantic mismatches during representational alignment. This challenge may stem from the fact that the human brain does not distribute attention uniformly across a visual scene, but rather selectively encodes salient or relevant regions. Moreover, such selectivity is closely related to individual interests and varies from person to person. To address this challenge, we propose Yo'Mind, a novel optimal transport (OT)-driven personalized multimodal semantic masking framework designed to bridge the semantic gap between brain and machines in interpreting visual scenes. Technically, Yo'Mind introduces a dynamic semantic pruning and allocation mechanism that adaptively masks redundant visual semantic components in stimulus images based on individual neural responses—without requiring extra human supervision or hyperparameter tuning. This strategy can be used to enhance semantic consensus between brain and machine representations during decoding. Furthermore, the inherent flexibility of OT theory enables Yo'Mind to perform brain-visual-linguistic alignment and cross-subject decoding within a unified end-to-end architecture. Extensive experiments demonstrate that our Yo'Mind offers several advantages, including state-of-the-art brain-to-text reconstruction performance and improved interpretability of the decoding process.

Published as a conference paper at ICLR 2026
REDUCING SEMANTIC MISMATCH IN TE-XXT DECODING THROUGH PERSONALIZED MULTIMODAL MASKING
Jiaxuan Chent.2, Yu Qi Yueming Wang's & Gang Pan
CCollege of Computer Science and Technology, Zhejjang University State Key Lab of Brain-Machine Intelligence, Zhejiang University Affiliated Mental Health Center and Hangzhou Seventh People's Hospital, MOE Frontier Science Center for Brain Science and { jjiaxuan_hhae qiyu, ymingwang, gpan e ju. edu. cn
Integration, Zhejiang University
ABSTRACT
The rapid progress of large vision-language models (VLMs), such CLIP, has spurred the development of a wide range of neural decoding frameworks. Nev- ertheless, most existing approaches still suffer from semantic mismatches during representational alignment. This challenge may stem from the fact that the human brain does not distribute attention uniformly across visual scene, but rather se- lectively encodes salient or relevant regions. Moreover, such selectivity is closely related to individual interests and varies from person person. To address this challenge, we propose Yo Mind, a novel optimal transport (OT)-driven person- alized multimodal semantic masking framework designed bridge the seman- tic gap between brain and machines in interpreting visual scenes. Technically, Yo" Mo'Mind introduces a dynamic semantic pruning and allocation mechanism that adaptively masks redundant visual semantic components in stimulus images based on individual neural responses- without requiring extra human supervision or hy- perparameter tuning. This strategy be used to enhance semantic consensus between brain and machine representations during decoding. Furthermore, the in- herent flexibility of OT theory enables Yo" Mond to perform brain-visual linguiiig alignment and cross-subject decoding within a unified end-to-end architecture. Extensive experiments demonstrate that our Yo' Mind offers several advantages, including state-of-the-art brain-to-text reconstruction performance and improved interpretability of the decoding process.
INTRODUCTION
Artificial intelligence (AI) researchers have long sought to develop systems that understand the
physical world human-like fashion (Duéñez-Guzmáán et al., 2023 Chen et al. 2025b). Recent years have seen incredible strides in artificial neural networks (ANNs) through diverse large-scale datasets and increased model sizes (Radford et al. 2021 Brown et al.
2020). Despite findings from
computational neuroscience suggesting that representations of certain large models (LMs) serve as better proxies for interpreting neural responses (Wang et al. 2023), substantial semantic biases still remain between the representations of biological and artificial systems (Chen et al.) 2025b] 2024a) ANNs often latch onto statistical patterns present in the static training data (Duéñez- Guzmán et al., 2023), thereby allowing their embeddings to exhaustively encode nearly every visible element in complex scene. In contrast, humans only selectively attend to those -isual-semantic elements deemed important for understanding or interacting with the scene, exhibiting significant individual variability (Chen et al. 2025c 2023a).
For example, consider scene in which young boy flying kite along a scenie lakeshore, accompanied by nearby dog (see Fig. [. While some individuals may focus primarily on the boy and his behaviour, others might be drawn the tranquil beauty of the surrounding landscape.
"Corresponding author.

---

Published as a conference paper at ICLR 2026
odapthe masking
Removed 5 pitches
brate
brote
atmuuue
watch
watch
Iscechors
percelted suvartice
X
dynank:
eanariic CONAERLL
(4) Sumantic betweee the brain and modids
[b) Removing the food number patches (eg, Mind-SA)
(c) The propased approxch
(a) Illustration of semantic biases between brain and model embeddings, and the limi- tations of global alignment in neural decoding. (b) Semantic consensus preserving by removing a fixed number of image patches. (c) Our proposed Yo" fo'Mind with an OT-driven personalized seman- tic masking mechanism, designed to support subject-adaptive, fine-grained alignment. Darker (or lighter) colors represent stronger (or weaker) semantic responses.
Figure 1:
Therefore, directly aligning image representations with the neural patterns recorded while subjects were viewing the image can be problematic (Chen al. 2025c) (Fig. a Such an issue becomes even more pronounced when attempting cross-subject decoding within single model, as different subjects may interpret the same image in distinct ways. A more promising paradigm is to dynam- ically detect the visual-semantic elements that the brain actually focuses on and leverage them as supervision to guide semantic reconstruction from brain activity. Typically, during fMRI acquisition, subjects instructed to maintain central fixation, but fixation single point does not necessarily imply uniform semantic attention due to the possibility of covert attentional shifts (Posner) 1980).
Recently, Chen et al. (2025c) have highlighted the semantic mismatch issue in neural decoding, and have introduced a new reconstruction model Mind-SA. The core idea is to preserve semantic con- sensus between brain activity and image embeddings by removing fixed number of image patches, as demonstrated in Fig. (b). Although Mind-SA can be trained in an end-to-end manner with the help of differentiable techniques (Jang al. 2016), it still suffers from several limitations. First, us- ing a fixed number of patches limits generalization, as scenes of varying complexity may require the removal of different amounts of redundant semantic content. Second, hard selection strategies lack fine-grained control. In many cases, semantic component may be weakly represented in neural signals rather than entirely absent, making inappropriate to discard it completely.
To address the limitations of existing methods (Takagi & Nishimoto, 2023b
Ferrante et al. 2023
Mai & Zhang, 2023) Xia et al., 2024) Shen al. [024] Chen et al.]2025c a). we propose Yo Mind, novel alternative decoding paradigm, revisit the semantic mismatch issue in neural decoding.
Our insight is that we can leverage the optimal transport (OT) framework (Peyré et al., 2019 Cuturi 2013) to establish assignment structure between visual stimulus and its evoked neural responses by minimizing the matching cost, where each stimulus image is viewed as being composed of a set of semantic components. The key here lies in how to deal with semantic components that not encoded in the neural patterns, without the need for any additional annotations or hyperparameter tuning (e.g., setting a threshold suppress semantic components).
In Yo" Mind, we augment each set with a virtual dustbin so that unrelated semantic components can be explicitly assigned it, as shown in Fig. (c). This formulation avoids the of hard masking and enables a dynamic soft selection of semantic elements, making it highly suitable for fine-grained alignment and cross-subject decoding. Notably, with the help of multimodal large language mod- els (MLLMs), Yo" Mind also extends traditional ttrin-visual alignment into siiti-visll---nnguee alignment by incorporating rich linguistic semantic cues related to the target stimuli. In brief, our contributions can be summarized as follows: 1) We introduce an OT-driven, personalized semantic masking mechanism, designed to seek semantic consensus between brain signals and machine repre- sentations, without requiring any annotations or hyperparameter tuning; 2) We propose novel brain semantic decoder, Yo" Mind, which leverages the personalized semantic masking to support subject- adaptive, fine- grained alignment across neural, visual, and linguistic modalities: 3) We demonstrate that our Yo' Mind outperforms existing methods. Extensive results confirm that it improves the in- terpretability of neural decoding and achieves state-of-the-art cross-subject decoding performance.
2

---

Published as a conference paper at ICLR 2026
2
RELATED WORK
Neural decoding, solid foundation for brain-computer interfaces, offers a promising pathway for bridging the gap between humans and machines. In recent years, fueled by the success of large vision-language models (VLMs) such as Radford et al. (2021) and generative models (e.g., diffusion models (Rombach et al. 2022), neural decoding has attracted increasing attention and has been extensively studied. Here, we briefly introduce representative studies most relevant to our Yo" Mind.
Brain-to-Image Decoding, To reconstruct (or identify) presented visual stimuli from fMRI, early attempts (Miyawaki et al., 2008 [Schoenmakers et al.) 2013) Kay et al. 2008: Horikawa et al. 2013) often combined linear regression with handcrafted descriptors. Even though these approaches have achieved satisfactory performance in decoding simple stimuli, they are constrained by oversimplified
mappings, resulting in degraded performance when decoding complex natural scenes (Rakhimber- dina et al., 202T). Building upon powerful generative models such stable diffusion (SD) (Rom- bach et al. 2022). recent works have achieved unparalleled performance in brain-to-image recon- struction. For example, [Takagi & Nishimoto (2023a) map fMRI voxels to the latent space of SI> using a ridge regression model, and then feed the predicted IMRI embeddings into the frozen de- coder of SD reconstruct images. A concurrent work (Chen et al. 2023b) also used the diffusion model but adopted self- self-supervised modeling to learn fMRI representations. MindEye (Scotti et al., 2024) incorporates contrastive learning and a diffusion prior for brain-to to-iatt retrieval and recon- struction. Beyond the aforementioned methods, several other representative studies (Chen et al.) 2024b,[Luet al. 2023 Chen et al. 2025a) have also explored diffusion models conditioned on brain signals, which have grown to be dominant paradigm for brain-to-image decoding. The success of this paradigm depends on the aligmment between fMRI signals and the target representations, and CLIP has been demonstrated to be good proxy model (Chen et al. (2024)). More recently, sev- eral studies (Scotti et al.) Wang et al. 2024) have also investigated cross-subject decoding, with the central idea being learn shared representational space.
Brain-to-Text Decoding. Unlike pixel-level reconstruction of stimulus images, brain-to-text de- coding focuses on translating brain activity into semantically meaningful word sequences, which is considered a more efficient form "mind reading" (Chen et al.) 2022a)). This is because the human brain does not function like camera that records every pixel (Chen al. (023a). Technically, the key to brain-to-text decoding remains the alignment between IMRI signals and the target representa- tional space. The difference that visual generative models are replaced by large language models (LLMs) such as GPT-2 (Radford et al.] 2019). For brain-to-texx decoding, early works (Ferrante al. 2023] Mai & Zhang [2023] Takagi Nishimoto/ (023b) also relied on linear regression to bridge the gap between fMRI and model embeddings such as GIT (Wang et al., (022) and BLIP (Li et al. 2022). [Chnn et al. (2025a) proposed an end-to-end brain semantic decoder, MindGPT, trained via self-supervised learning. MindGPT achieves performance improvements, but remains limited by the per-subject-per-me pedel paradigm. To enable cross-subject decoding, UMBRAE (Xia et al. 2024) proposed multi-subject training strategy to map subject-specific features to shared latent space. On the other hand, [Shen et al. (2024) designed a Vision Transformer 3D for learn- ing cross-subject embeddings. MindLLM (Qiu al. 2025) learned brain representations using subject-agnostic IMRI encoder with neuroscience-inforeee
attention mechanism. A recent study, Mind-SA (Chen et al.] 2025c), was the first highlight the semantic mismatch problem during decoding. and addressed it by removing a fixed number of image patches. Our Yo' Mind goes one step further, and introduces a dynamic semantic pruning and allocation mechanism, thus enabling fine-grained multimodal alignment across multiple subjects.
METHOD
Unlike existing global alignment approaches (Chen et al.) 2025a 2024b) Shen et al., 2024, Fer- rante et al. 2023 Mai & Zhang 2023 Takagi & Nishimoto, 2023b a). Yo Mind is unified cross- subject decoding architecture that performs fine-grained aligmment across neural, visual and linguis- tic modalities (see Figs. and B. During multimodal alignment, Yo' Mind adaptively filters out redundant semantics that are not perceived by the subjects, thereby avoiding the semantic mismatch problem. We introduce the details below.
3

---

Published as a conference paper at ICLR 2026
Stimulus image
OT-Driver Personalized Multimodal Semantic Masking
mutii Enooder (IT))
Vilcon
Yo Minel
Subject
Subject's
Subject
(Hammon(Qwen2..5) !
/wott embedding
Diperiieery rignd
What forturettor dosingu shinn thi mage?
discarded
pasch umbediding
sooor ded
MindGPt (Neural Decader)
seen image shows-
Seaiale landioupe with soltare bidd rodis
landscape
bird
Duattal senvery showsosing the
beach, datamtt
soscape oversad dide
Nils.
Coastal scenery showssssing the cosun beach, and distant hils.
Soaside kandboope with solftary bird rocks
Loddioonll dapyrinos
perched lerge near shore.
COCO
Figure 2: Overview of the proposed personalized multimodal semantic masking mechanism. We first construct a multimodal semantic set for each stimulus image, where the visual semantics are derived from image patches, while the textual semantics are generated by Harmon (Qwen2.5) (Wu al., 2025). Both the visual and textual data are fed into CLIP to extract representations. Mcan- while, we split fMRI signals into fixed-size patches, and feed the resulting sequence of vectors to standard VIT. Then, compute the least costly transport plan under the OT framework, during which irrelevant semantic elements are automatically masked.
3.1 CONSTRUCTION OF FINE-GRAINED SEMANTIC SETS
People live in an ever-changing environment full of objects, which shapes our perception be both directed and selective (i.e., attention). As a result, different people may interpret the same visual scene in different ways (Chen et al. 2025c). To incorporate this mechanism into neural decoding models, the first necessary step is to construct fine-grat line-grained semantic set for each stimulus image.
Formally, given an image x, we first reshape x into sequence of hon-overlapping patches, and use frozen CLIP vision encoder to generate N patch embeddings { Mcanwhile, we use Harmon (Wu et al. 2025), an advanced multimodal large language model (MLLM), to produce detailed semantic description of using the fixed prompt: "What are useful features for distinguishing this image?". Generally, MLLMs can produce rich set of discriminative semantic attributes that capture fine-grained object details, contextual dependencies, and even nuanced scene atmosphere, shown in Fig. 2 Left. Similarly, we use the CLIP text encoder to produce a set of word embeddings {4, [ from the semantic description.
J.N
3.2 OT OR-DRIVEN PERSONALIZED SEMANTIC MASKING
Once semantic sets for stimuli are constructed, our goal is to determine which semantic elements truly perceived by the brain. To this end, we propose an OT-driven personalized semantic masking
module (see Fig. 2) which produces a partial assignment matrix [0, K K Here, K
denotes the number of fMRI patch embeddings. Following Chen et al. 2025a cD. we use a standard VIT (Dosovitskiy et al., 202T) as fMRI encoder.
The OT problem involves finding transport plan I that transforms one probability distribution into another at minimal cost (Peyré al. 2019). use the pairwise cosine distances between the fMRI embeddings and the semantic embeddings to construct the cost matrix C:
C₁, = 1- Ti, >, E (61, ( ,ttt
where is the fMRI patch embedding, and < ., denotes the inner product. Next, can use the Sinkhorn algorithm (Cuturi 2013) to solve the entropy- reguuay-egullaiized optimal transport problem mingen (T, C) = HI),, which strictly convex optimization. Here, minren represents finding a transport plan I that minimizes the cost, and II denotes the set of all joint distributions whose marginals match the given source and target distributions. H(T) Eig PiylogT:; ij is entropy

---

Published as a conference paper at ICLR 2026
tnaado teathrrpe rodiyn
Dorsal pathway
Seaside landscape with
Jentral pathwdayy.
Text Encoder
Visual Encoder
Language Decoder (GPT-2)
early
OT-Driven Personalizad Muttimodal Semantic Masking
The seen image shows
Encoder-Decode Cross Attention
Personalized somantic masking results
alignment (MSE)
optimization procadure
fMRI Encoder (ViT)
IMRI patch embeddings
Split operation
Training only
Figure 3: Our end-to-end Yo Mind architecture. During training, the resulting pruned semantic set serves supervision signal for brain-to-tex, reconstruction.
constraint term and E > 0 trade-off hyperparameter. However, in OT framework, transport plan restricted
PlN4M = 1x and 1 = 1N+M,
where 1 x represent an K-dimensional vector with all entries equal to 1. Inspired by graph matching technique (Sarlin al. 2020; DeTone al. (018)), we add each with virtual dustbin explic- itly mask redundant elements in the semantic set. Specifically, we augment the cost matrix C to C* by appending extra row and column, filled with learnable parameter. Under the cireumstances, semantic elements will be assigned to fMRI patches or the dustbin. After Sinkhorn iterations, we can recover partial assignment matrix from the transport plan T, i.e., P (1NK, by dropping the dustbins. The P now has the constraints:
PlN+M S 1x and P=1x S 1N+M.
3.3 THE Yo' MIND ARCHITECTURE
Alignment. Different from traditional representation-level alignment, we use only the semantic elements that may be of interest human brains as supervision signals (see Fig. Given that CLIP has aligned visual and textual modalities into shared embedding space, our optimization goal is to minimize weighted mean squared error loss, which can formulated as
Brain- Viswa--Liiiiuueee
N+M
Calign Σ In
Σ 1=1
Pr.ss112.
where denotes the partial assignment matrix between r {r1.... ,Tk and 8 {ur,... UN, t..... }. Note that redundant semantic elements in have been masked during the alignment process, as the dustbins are dropped.
Brain-to-Tee Text Reconstruction. With the help of Sinkhorn algorithm, both operations in Yo Mind are differentiable. Therefore, our method can be seamlessly integrated into off-the-shelf brain-to- text reconstruction frameworks in end-to-end fashion. To ensure fair comparison, Chen et al. (2025c) and adopt MindGPT as the brain-to-text module of our method. MindGPT is an autoregressive language model conditioned brain signals, built upon the GPT-2 architecture [Radford et al. 2019). MindGPT connects an fMRI encoder and GPT-2 through cross cross-attention mechanism (Vaswani et al. 2017). Therefore, integrating our Yo Mind with MindGPT simply in- volves replacing its original encoder with our OT-guided fMRI encoder F(-), as shown in Fig. 3 By doing so, the brain-to-texx reconstruction can be optimized with the following loss
follow
P(W) = logP(w:[[usleei F)y):
(5)
5

---

Published as a conference paper at ICLR 2026
Language Similarity Metrics 1
Model
Dataset
Sukject
B@1
B@2
B-8
B@4
METEOR
ROUGE
CIDEi
SPICE
UniBmann Mai & Zhang 2023
NSD
16.90
22.20
Brii Coptioning (Ferranic
2023
NSD
2.2,5.7
54,82
35.14
21.96
14.04
16.24
40.39
38.86
8.84
Takagi et al. (Takagt & Nisammtoo 20256
NSD
1,2,5.7
35.22
16.37
7.22
3.30
9.77
24.61
13.52
4,88
MindGPT Clen et al, 20250
NSD
1.2.7.7
56.91
36.95
23.41
15.87
20:04
38.41
43.56
10.97
Neuro2l angllage (Shen
NSI
1.2.3.7
57.19
37.17
23.78
15.85
18.00
36.67
49:51
12.39
UMBRAE" Xia all 6024
NSD
1.2.3.7
57.38
37.98
25.18
17.11
18.27
41.99
51.68
11.82
UMBRAE
Xia ct al. 2014
NSD
1,2,5.7
39.09
40.13
22.05
18.40
19:24
43.64
57.76
12.42
MindAst
NSD
1.2.57
25.17
21.93
42.86
12:27
Mind-SA Chen
X0250
NSD
1,2.5.7
59,76
39.81
27.05
18.86
38.08
42.72
54.03
12.25
Yo Mind (Ouns)
NSD
1,2,5.7
61.81
42.75
29.76
20.88
38.40
43.63
56.07
12.42
Neuro2Language (Shen ct 2024]
NSD
1,2.5.7
64.26
51.44
47.70
32.17
20.41
52.61
83.94
18.27
MindAr' Chen et al.
2024bb
NSO
1.2.3.7
64.56
51.57
46.35
31.90
37.54
50.01
78.35
18.21
Mind- SA CCen 2025c
NSD
2.2.
64.78
53.54
46
32,46
36.25
54,26
79.48
18.22
Yo Mind! (Ours)
NSD
1,2,5.7
65.19
53.53
48.46
33.36
39.25
55.19
16
19.25
Table 1: Quantitative results of brain-to-text reconstruction on the NSD. Here, UMBRAE" is trained on data from single subject, whereas UMBRAE is trained using data from all four subjects. and I indicate results obtained with BLIP2- and Harmon (Qwen2.5) gegerrtteed captions as ground truth, respectively. Some decoding results are derived from Xia al. 2024 : [Cen et al. (2025c). The best and second- best results are highlighted using Bold and Underline, respectively.
where y denotes the fMRI signals, 0 are the learnable parameters of cross-attention layer in MindGPT, and W [w:] indicates the reconstructed word sequence conditioned the fMRT patch embeddings r [71 ,rk)
EXPERIMENTS
4.1 IMPLEMENTATION DETAILS
The Yo'Mind framework comprises three encoders: a vision encoder, a text encoder, and an fMRI encoder. Specifically, the vision and text encoders are initialized from a pre-trained CLIP model (ViT-B/32), and their parameters remain frozen throughout training. The IMRI encoder is imple- mented as a standard ViT with 16 layers and 16-head self-attention. The input IMRI signals flat- tened into 1D sequence and uniformly divided into K 8 non-overlapping patches, which are then fed into the transformer encoder. Note that fMRI voxels are selected from the ventral, lateral, and parietal streams, as well as the early visual cortex (27, 2,,638 voxels). We adopt the frozen GPT-2 for brain-to-text reconstruction. To connect the fMRI encoder and the GPT-2 decoder, each of the 12 layers of the GPT-2 decoder incorporates 12-head cross-attention layer, where the dimensionality of the projection matrices is set to 4. Pre-trained CLIP and GPT-2 are publicly available on Hugging- Face (Woff et al., 2020). During the training phase, we perform 100 Sinkhorn iterations to obtain assignment matrix (c 1), and only the parameters of the fMRI encoder and the cross-attention lay- are learnable. We use the Adam optimizer (Kingma & Ba, 2014) (3, 0.9, 82 0.999) with a learning rate of le-4 and a weight decay of le-4, maintained until the model converges. Yo Mind trained on data from four subjects (see details below) and implemented in PyTorch using four NVIDIA GeForce RTX 3090 GPUs
4.2 DATASETS AND EVALUATION METRICS
Datasets. In this paper, the Natural Scenes Dataset (NSD) (Allen et al.) 2022) is used to benchmark the proposed Yo" Mind framework. NSD is high-quality dataset that provides high-resolution fMRI recordings collected from eight subjects over 30-40 sessions using 7-Tesla scanner. Each subject viewed 10,000 natural images, each repeated three times, selected from the Microsoft COCO dataset (Lin et al.) 2014) and cropped to 425 x 425 (if needed). Following the prevailing practice (Takagi) & Nishimoto 2023a Shen et al. 2024: Chen et al., 2025c), our experiments use 27,750 trials from subjects 1, 2, 5, and 7. Among them, 2.770 trials, including 982 COCO images, are used as the test set, while the remaining trials are used as the training set. Note that the 982 test stimuli are shared among all four subjects.

---

Published as a conference paper at ICLR 2026
BOtted TRAS STRETT with an unbrella
BICE Dhan around a sabl eating food
pracer rg food
shiiig On Bench
COART MEE
With piaze
Auly Spuct with
younn bobling
trachaion
hand
folid
lable
Skzs down GInOo Prl
Must
Tet NITH burrhh
STOUP DWOps
Narara CITIO DA NED
hold
Mack
FACOU
bolding racquae
and cadl otfer
Sip lend
Sukking
VolV elephare resl weathe
of streett
SAIE SPACK FOAT
THE giv Earchig FOOT
Figure 4: Qualitative results of brain-to-text reconstruction for one subject (S1). For each group, the image on the left is the ground truth stimulus. Imperfect captions are highlighted in Orange, and semantic errors are marked Red.
Evaluation Metrics. Following previous works (Shen et al. 2024 Chen et al. 2025a c. we evaluate
brain-to-text performance using the following standard metries: BLEU-1 (B 1), BLEU-2 (B 2), BLEU-3 (B @ @), BLEU-4 (B @4) (Papineni et al. 2002). METEOR Denkowski & Lavie 2014
ROUGE (Lin & Hovy 2003), CIDEr /Vedantam et al. 2015) and SPICE (Anderson et al. 2016
These metrics are commonly used evaluate both n- n-gram overlap and semantic relevance between
generated and reference texts.
4.3 COMPARISON WITH STATE- OATE--FF-TEE-AT
Qualitative Analysis. First of all, qualitatively analyze brain-to-text reconstruction to illustrate the behavioral patterns and performance gains of our Yo" Mind, in comparison with the most relevant existing approaches, Mind-SA (Chen al. (025c) and MindGPT (Chen al., 2025a). Qualitative results are shown in Fig. 4
Intuitively, we observe that Mind is able to effectively capture and summarize the details the entire visual scene. In contrast, Mind-SA and MindGPT produce imperfect text reconstructions, such semantic mismatches incomplete descriptions.
For example, we see that Yo' Mind can accurately infer the detail "a woman walking down umbrella", even though the visual content appears small scale in
street with the image (col 1, row 1). Although Mind-SA fails to reconstruct the main semantic content, nevertheless captures peripheral details "a man sitting". In contrast, MindGPT yields com- pletely inaccurate reconstruction. In the second case (col 2, row 1), although the image exhibits high visual complexity, Yo" Mind is capable of generating satisfactory semantic reconstruction
7

---

Published as a conference paper at ICLR 2026
9.0000001
P.c00.0001
P::0.0001
P-c.0.000:
P.c.0.0001
5.0000001
0.000001
55
BLEU1
50-
BLEU
BLED
MET TCTOOO
%
ROUGE
50
WNodd
Mond S4
35
Mine d4
18
SA
Wo Hi
35
Ho
Host
35
25
Mint
Figure 5: Statistical analysis of model performance. Yo Mind significantly outperforms Mind-SA across all metries (two-tailed paired r-test, P < 0.0001; error bars denote 99% confidence intervals).
that encompasses most key details such as "a group of people". "sitting around a table", and "eating food". . By contrast, alternative methods manage reconstruct only par- tial semantic content, iee,, "table" and "food". Other similar cases include Yo Mind captur- ing "a city street filled with cars and trucks" (col 1, row 2), and "a baby elephant standing next to another elephant" (col 2, row 5). These preliminary findings highlight the robustness of Yo" Mind in generating coherent and semantically faithful de- scriptions, significantly reducing issues of semantic mismatches.
Quantitative Comparison. In this part, to enable a comprehensive evaluation, we quantitatively compare the brain-to-tent reconstruction performance of Yo' Mind against a wide range of baselines, including Mai & Zhang (2023); y Ferrante al. (2023); Takagi & Nishimoto (2023b); Chen et al. 2025a); Shen et al. (2024); Xia et al. (2024): Chen et al. (2025c). The reconstruction results on NSD (SI, S2, S5, and S7) are summarized in Tab. [ As shown in Tab. [ we find that Yo' Mind achieves the highest performance across six metrics when compared with 8 state-of-the-art methods.
Compared to the MindGPT baseline, which can considered vanilla method without semantic pruning strategy. Yo" Mind achieves substantial performance gains: 91.6% on METEOR, 31.6% on BLEU-4, and 28.7% on CIDEr. Importantly, our method consistently outperforms recent Mind-SA, demonstrating a clear and substantial advantage. In general, these quantitative results show that captions produced by Yo' Mind achieve higher quality descriptions, which further demonstrates the effectiveness of the personalized multimodal semantic masking module. Beyond the COCO caption (Lin al. 2014), we also reran the reconstruction experiments using Harmon (Qwen2.5)-generatte captions (Wu al. 2025), which provide more detailed description of images, as ground truth (see [ottom). The resu is demonstrate superior performance of Yo" Mind in decoding fine-grained text, suggesting that can produce not only simple captions but also rich, detailed descriptions. We also provide additional qualitative comparisons on DIR dataset (Shen et al. 2019) in the Appendix.
Statistical Analysis. To evaluate whether the performance gains of Yo' Mind are statistically sig- nificant, we conduct a statistical analysis on BLEU, METEOR, ROUGE, and CIDEr scores using bootstrap resampling and a two-tailed paired r-test. Specifically, we perform 100 bootstrap resam- ples of the test set and recomputing the metrics for both Yo" Mind and Mind-SA on each resample (see Fig. 5). Results show that Yo" Mind consistently outperforms Mind-SA, and the improvements statistically significant (P < 0.0001 across all evaluation metrics).
4.4 REVEALING BRAIN- PREFERRED SEMANTICS
The success of Yo Mind in brain-to-text decoding can be attributed to the appropriate modeling of brain- brain-preferred semantic elements. To better understand how the proposed OT-driven semantic masking module enhances brain decoding, we investigate its role from two perspectives: visualiza- tion of salient regions and comparison with eyc-tracking data.
Visualization of Salient Regions. As illustrated in Fig. Ga, we visualize salient regions by project- ing the partial assignment matrix of the fMRI [CLS] embedding onto the image patches. We observe that in relatively simple images, which typically contain single-object scenes, the brain-preferred se- mantic elements highly consistent. In the first two examples, the preferred semantic content is consistently concentrated on the "cat" and the "skier" (from left to right are subjects 1, 2. 5, and 7). In contrast, for complex visual images, the brain-preferred semantic content exhibits clear
8

---

Published as a conference paper at ICLR 2026
bemee dovn
a bed
of aed
lcding on bed
lavigg hi bed
moebbord in the ordw
Hew
S1
S2
S5
beach
beach
dag
with
beack froit
sheat uetondle
street
Vorloeed IDamd
Mieit OCEV DOON
West No LOOW (901)
Minil OCENT STin
bennur hagphg From
ghop planding hand
grop poods mhonding
hoble
bolket goids
Figure 6: a. The role of the OT-driven semantic masking mechanism. The first row shows examples of intra-subject consistency, and the second presents examples of inter-subject variability. Only the top 80% salient attention patches are highlighted for visualization, and reconstruction differences are marked in Red. b. Comparison of different masking mechanisms. Notably, the hard masking mechanism limits the generalization ability of Mind-SA (Chen et al. 2025c).
Language Similarity Metries
Subject
ROIs
Voxel number
Masking
B@4
METEOR
ROUGE
CIDEr
ventral
8,590
/
17.64
35,99
41.88
46.24
ventral
8,590
X
15.90
21.05
38.98
43.68
ventral, lateral
17,223
/
19.42
37.68
43.41
52.80
ventral, lateral
17,223
16.58
22.36
39.90
45,47
ventral, lateral, parictal
21.721
/
21.78
38,98
45.82
58.87
ventral, lateral, parictal
21,721
X
16.37
21.26
38.06
46.87
Table 2: Effects of brain regions and the semantic masking module on reconstruction performance.
variability. For example, in the third case, subject appears to focus the action of "flying kite", whereas subjects 1, 5, and 7 concentrate on the "boy" and the "dog". Finally, we present Mind-SA. As shown in Fig. 6 b,
several examples illustrate the superiority of method Mind-SA fails to effectively process complex images, mainly because its fixed image patch masking restricts the model's ability capture rich visual-semantic cues.
Comparison with Eye-Tracking Data. Using eye-tracking to verify whether Yo' Mind truly cap- tures subjects' visual atlention may appear be straightforward strategy. However, in NSD, all subjects were instructed to maintain central fixation most of the time Allen et al. (202)). As a result, the recorded gaze locations are tightly clustered near the central fixation point and do not reflect the attention when viewing the images. Therefore, we only select the limited cases where the recorded eye positions deviated noticeably from the central fixation and examined their OT-driven semantic allocation plans (see Appendix Fig. A7). Qualitative analysis provides preliminary evidence that Mind is indeed sensitive to subjects' actual viewing behavior.
It is important to note that gazing at fixed point does not necessarily imply uniform semantic attention. A large body of work in psychology and neuroscience, such as Posner's covert attention paradigm (Posner, 1980), has shown that subjects flexibly shift high-level semantic attention without moving their eyes. Validating this would require new datasets that explicitly measure covert attention, which will be important direction for our future work.

---

Published as a conference paper at ICLR 2026
Language Similarity Metries 1
Subject
ROIs
Voxel number
Visual
Text
Bo04
METEOR
ROUGE
CIDEr
veotral, lateral, parietal
21,721
x
21.45
38.07
44.97
57,24
ventral, lateral, parietal
21,721
X
/
18.35
32.95
41.87
45.02
ventral, lateral. parietal
21,721
7
21.78
38.98
45.82
58.87
Table 3: Effects of multimodal semantic set on reconstruction performance.
Languuge Similarity Metries
Subject
ROIs
Voxel number
Visual
BB4
METEOR
ROUGE
CIDEr
ventral, lateral. parietal
21.721
CLIP
SMALLCAP
21.36
38.14
44.56
57.21
ventral, lateral. parictal
21.721
CLIP
38.46
45,97
58.47
ventral, lateral. purietal
21,721
CLIP
Harmon
21.78
38,98
45.82
S8.87
Table 4: Effects of image captions generated from different models on reconstruction performance.
4.5 ABLATION STUDIES
Here we explore effects of different components or configurations on the performance of Yo Mind.
OT-Driven Masking Strategy. First, we explore the effects of personalized multimodal seman- tic masking on the decoding, as well as investigate the performance across different brain regions (ROIs). Tab. 2 presents the results of Yo" Mind, conditioned on three ROI combinations, with and without the semantic masking strategy. Results show that OT-driven semantic masking significantly enhances decoding performance when using voxels from the ventral, lateral, and parietal streams.
A possible explanation is that different brain regions encode different semantic information. For example, the ventral stream ("what" pathway) exhibits relative specialization in object recognition such as shape and category, whereas the dorsal stream ("where" pathway) is mainly involved in processing spatial location and motion-related information. In other words, different brain regions (voxels) exhibit distinct semantic selectivity. Under this condition, the OT-based semantic masking strategy can assign different semantic elements to different brain areas, thus enabling region -specific alignment (precise matching of specific voxel subsets to corresponding semantic elements).
Multimodal Alignment. Next, we report the impact of multimodal (image and text) alignment on the reconstruction performance, as summarized Tab. 3 The results show that the text description facilitates brain- maahine aligmment, as evidenced by improved reconstruction performance, which in line with previous findings such as et al. (2023).
Image Captioning, Beyond the above analysis, we further evaluate how the quality of image de- scriptions affects the performance of Yo' Mo'Mi In addition to Harmon, we leverage another ad- vanced multimodal large language model, BI.IP3o (Chen et al., (025d), as well as SMALLCAP [Ramos et al. 2023). a representative classical image captioning model, to provide textual supervi- sion during the alignment process (see Tab. We find that using captions generated by Harmon and BLIP3o as supervisory signals yields better reconstruction performance, suggesting that richer and more structured textual semantics lead to more effective brain decoding.
CONCLUSION
Our Yo Mind introduces novel OT-driven, self-adaptive supervision scheme that reshapes the de- coding target space to preserve semantic consensus between brain and machine representations during brain-to-text reconstruction. without the need for any extra annotations or hyperparameter tuning. From the perspective of mitigating semantic bias, Yo' Mind enhances the recent Mind-SA model via personalized multimodal semantic masking. Technically, we use OT tools to avoid fixed hard masking strategies, enabling our Yo'Mind to automatically adapt to visual scenes of varying complexity. In addition, the flexibility of OT allows our method to be seamlessly extended to mul- timodal alignment and cross-subject decoding. On the other hand, our method holds promise neuroscience tool for analyzing intra-subject consistency and inter-subject variability. Nonetheless, such analysis still requires reliable biological validation. This is also a potential limitation of our work. In future studies, we plan to incorporate additional physiological signals, such as eye-tracking to further validate the prefe-pe semantic elements decoded by Yo" Mind.
10

---

Published as a conference paper at ICLR 2026
ETHICS STATEMENT
This study relies entirely on publicly available fMRI datasets (NSD and DIR) and does not involve the collection of any new neural recordings. Our analyses use no personally identifiable information. We follow dataset usage policies and established ethical guidelines to ensure responsible application of brain decoding methods.
ACKNOWLEDGMENTS
This work was supported in part by the Science and Technology Innovation 2030 Major Projects (No. 2021 1000000004),) Zhejiang Provincial Natural Science Foundation of China (No. LR24F020002), Natural Science Foundation of China (NSFC) (No. 624B2127), and the Qiushi Rising Star Program of Zhejiang University.
REFERENCES
Emily J Allen, Ghislain St- Yves, Yihan Wu, Jesse L Breedlove, Jacob S Prince, Logan T Dowdle, Matthias Nau, Brad Caron, Franco Pestilli, Ian Charest, et al. A massive 7t fmri dataset to bridge cognitive neuroscience and artificial intelligence. Nature Neuroscience, 25(1 35(1):1 16-1226, 2022.
Peter Anderson, Basura Fernando, Mark Johnson, and Stephen Gould. Spice: Semantic propo- sitional image caption evaluation. In European Conference on Computer Vision, pp. 382-398. Springer, 2016.
Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. Language models are few-shot learners. Advances in Neural Information Processing Systems, 33:1877-1901, 2020.
Jiaxuan Chen, Yu Qi, and Gang Pan. Rethinking visual reconstruction: Experience- con- tent completion guided by visual cues. In Proceedings of the 40th International Conference on Machine Learning, volume 202 of Proceedings of Machine Learning Research, pp. 4856-4866. PMLR, 23-29 Jul 2023a.
Jiaxuan Chen, Yu Qi, Yueming Wang, and Gang Pan. Bridging the semantic latent space between brain and machine: Similarity is all you need. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pp. 11302-11310, 2024a.
Jiaxuan Chen, Yu Qi, Yueming Wang, and Gang Pan. Mind artist: Creating artistic snapshots with human thought. In Proceedings of the IEEE/VV Conference on Computer Vision and Pattern Recognition, pp. 27207-27217, 2024b.
Jiaxuan Chen, Yu Qi, Yueming Wang, and Gang Pan. Mindgpt: Interpreting what you see with non-invasive brain recordings. IEEE Transactions on Image Processing. 34:3281-3293, 2025a.
Jiaxuan Chen, Yu Qi, Yueming Wang, and Gang Pan. Human-like cognitive generalization for large models via brain-in-the-loop supervision. arXiv preprint arXiv:2505. 09085, 2025b.
Jiaxuan Chen, Yu Qi, Yueming Wang, and Gang Pan. Bridging the gap between brain and machine in interpreting visual semantics: Towards self-adaptive brain-to-tex decoding. In Proceedings of the IEEE/CVF International Conference on Computer Vision, 2025c.
Jiuhai Chen, Zhiyang Xu, Xichen Pan, Yushi Hu, Can Qin, Tom Goldstein, Lifu Huang, Tianyi Zhou, Saining Xie, Silvio Savarese, et al. Blip3-o: A family of fully open unified multimodal models-architecture, training and dataset. arXiv preprint 2505v:2505.09566 2025d.
Zijiao Chen, Jiaxin Qing. Tiange Xiang, Wan Lin Yue, and Juan Helen Zhou. Seeing beyond the brain: Conditional diffusion model with sparse masked modeling for vision decoding. In Pro- ceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 22710- 22720, 2023b.
11

---

Published as a conference paper at ICLR 2026
Marco Cuturi. Sinkhorn distances: Lightspeed computation of optimal transport. Advances in
Neural Information Processing Systems, 26, 2013.
Michael Denkowski and Alon Lavie. Meteor universal: Language specific translation evaluation for any target language. In Proceedings of the ninth workshop on statistical machine translation, pp- 376-3800 2014.
Daniel DeTone, Tomasz Malisiewicz, and Andrew Rabinovich. Superpoint: Self-supervised interest point detection and description. In Proceedings of the IEEE/CVE Conference Computer Vision and Puttern Recognition Workshops, pp. 224-236, 2018
Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, Jakob Uszko- reit, and Neil Houlsby. An image worth 16x 16 words: Transformers for image recognition scale. In International Conference on Learning Representations, 2021.
Changde Du, Kaicheng Fu, Jinpeng Li, and Huiguang He. Decoding visual neural representations by multimodal learning of and Machine Intelligence, pp. 1-17, 2023. doi: 10. 20000// 222.3331
linguissii features. IEEE Transactions on Pattern Analysis
Edgar A Duéñez-Guzmin Suzanne Sadedin, Jane X Wang, Kevin R McKee, and Joel Z Leibo. A social path to human-like artificial intelligence. Nature machine intelligence, 5(11):1181-1188 2023.
Matteo Ferrante, Furkan Ozcelik, Tommaso Boccato, Rufin VanRullen, and Nicola Toschi. Brain captioning: Decoding human brain activity into images and text. arXiv preprint arXiv:2305.11550. 2023.
Tomoyasu Horikawa, Masako Tamaki, Yoichi Miyawaki, and Yukiyasu Kamitani. Neural decoding of visual imagery during sleep. Science, 340(6132):639 642, 2013.
Eric Jang. Shixiang Gu, and Ben Poole. Categorical reparameterizzatoo with gumbel-softmax. arXiv preprint arXiv: 1.01 144, 2016.
Kendrick N Kay, Thomas Naselaris, Ryan J Prenger, and Jack I. Gallant. Identifying natural images from human brain activity. Nature, 452(7185):352-3355 2008
Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXiv preprint 1rX2v:1412.660 6980, 2014.
Junnan Li, Dongxu Li, Caiming Xiong, and Steven Hoi. Blip: Bootstrapping language-image pre- training for unified vision-language understanding and generation. In International Conference on Machine Learning, pp. 12888-12908 PMLR, 2022.
Chin-Yew Lin and Eduard Hovy. Automatic evaluation of summaries using n-gram co-oceurrence statistics. In Proceedings of the 2003 Human Language Technology conference of the North American chapter of the Association for Computational Linguistics, pp. 150-157, 2003.
Tsung-Yi Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and C Lawrence Zitnick. Microsoft coco: Common objects in context. In European Conference Computer Vision, pp. 740-755. Springer, 2014.
Yizhuo Lu, Changde Du, Qiongyi Zhou, Dianpeng Wang, and Huiguang He. Minddiffuser: Con- trolled image reconstruction from human brain activity with semantic and structural diffusion. In Proceedings of the 31st ACM International Conference on Multimedia, pp. 1899-5908, 2023.
Weijian Mai and Zhijun Zhang. Unibrain: Unify image reconstruction and captioning all in one diffusion model from human brain activity. arXiv preprint arXiv:2308.07425 2023.
Yoichi Miyawaki, Hajime Uchida, Okito Yamashita, Masa-aki Sato, Yusuke Morito, Hiroki C Tan- abe, Norihiro Sadato, and Yukiyasu Kamitani. Visual image reconstruction from human brain activity using combination of multiscale local image decoders. Neuron, 60(5):915-929, 2008.
12

---

Published as a conference paper at ICLR 2026
Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. Bleu: method for automatic evaluation of machine translation. In Proceedings of the 40th anmual meeting of the Association for Computational Linguistics, pp. 311-318, 2002.
Gabriel Peyré, Marco Cuturi, et Computational optimal transport: With applications to data science. Foundations and Trends@ in Machine Learning, 11(5-6)355-607, 2019.
Michael Posner. Orienting of attention. Quarterly Journal of Experimental Psychology, 32(1):
3-25, 1980.
Weikang Qiu, Zheng Huang, Haoyu Hu, Aosong Feng, Yujun Yan, and Rex Ying. Mindllm: A subject-agnostic and versatile model for fmriito-text decoding. arXiv preprint arXiv:2502.15786, 2025.
Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, et al. Language models are unsupervised multitask learners. OpenAl blog, 1(8):9, 2019.
Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, al. Learning transferable visual models from natural language supervision. In International Conference Machine Learning, pp. $748-8763. PMLR, 2021.
Zarina Rakhimberdina, Quentin Jodelet, Xin Liu, and Tsuyoshi Murata. Natural image reconstruc- tion from fmni using deep learning: A survey. Frontiers in neuroscience, 15:795488, 2021.
Rita Ramos, Bruno Martins, Desmond Elliott, and Yova Kementchedjhieva. Smallcap: lightweight image captioning prompted with retrieval augmentation. In Proceedings of the IEEE/CVF Con- ference Computer Vision and Pattern Recognition, pp. 2840-2849, 2023.
Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bjöm Ommer. High- resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF Con- ference on Computer Vision and Pattern Recognition, pp. 10684-10695, 2022.
Paul-Edouard Sarlin, Daniel DeTone, Tomasz Malisiewicz, and Andrew Rabinovich. Superglue: Learning feature matching with graph neural networks. In Proceedings of the IEEE/CVF Confer- on Computer Vision and Pattern Recognition, pp. 4938-49417 2020.
Sanne Schoenmakers, Markus Barth, Tom Heskes, and Marcel Van Gerven. Linear reconstruction of perceived images from human brain activity. Neurolmage, 83:951-961, 2013.
Paul Scotti, Atmadeep Banerjee, Jimmie Goode, Stepan Shabalin, Alex Nguyen, Aidan Dempster, Nathalic Verlinde, Elad Yundler, David Weisberg, Kenneth Norman, et al. Reconstructing the mind's eye: fmmrito-image with contrastive learning and diffusion priors. Advances in Neural Information Processing Systems, 36, 2024.
Paul Steven Scotti, Mihir Tripathy, Cesar Torrico, Reese Kneeland, Tong Chen, Ashutosh Narang, Charan Santhirasegaran, Jonathan Xu, Thomas Naselaris, Kenneth A Norman, et al. Mindeye2: Shared-subject models enable fmri-to-image with hour of data. In Forty-first International Conference Machine Learning.
Guobin Shen, Dongeheng Zhao, Xiang He, Linghao Feng, Yiting Dong, Jihang Wang, Qian Zhang, and Yi Zeng. Mision language: Enhancing brain recording based visual reconstruction and language interaction. Advances in Neural Information Processing Systems, 012998883981111 2024.
Guohua Shen, Tomoyasu Horikawa, Kei Majima, and Yukiyasu Kamitani. Deep image recon- struction from human brain activity. PLOS Computational Biology, 15(1):1-23, 01 2019. doi: 10. /0.oral/ 1006633.
Yu Takagi and Shinji Nishimoto. High-resolution image reconstruction with latent diffusion models from human brain activity. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 14453-14463, 2023a.
13

---

Published as a conference paper at ICLR 2026
Yu Takagi and Shinji Nishimoto. Improving visual image reconstruction from human brain activity using latent diffusion models via multiple decoded inputs. arXiv preprint arXiv:2306. 1536, 2023b.
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need. Advances in Neural Informa- tion Processing Systems, 30, 2017.
Ramakrishna Vedantam, C Lawrence Zitnick, and Devi Parikh. Cider: Consensus- based image description evaluation. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pp. 4566-4575, 2015.
Aria Y Wang, Kendrick Kay, Thomas Naselaris, Michael J Tarr, and Leila Wehbe. Better models of human high-level visual cortex emerge from natural language supervision with large and diverse dataset. Nature Machine Intelligence, 5(12): 1415):4415-426, 2023.
Jianfeng Wang, Zhengyuan Yang, Xiaowei Hu, Linjie Li, Kevin Lin, Zhe Gan, Zicheng Liu, Ce Liu, and Lijuan Wang. Git: generative image-to-text transformer for vision and language. arXiv preprint arXiv:2205. 1ati:22055.110,, 2022.
Shizun Wang. Songhua Liu, Zhenxiong Tan, and Xinchao Wang. Mindbridge: A cross-subject brain decoding framework. In Proceedings the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 11333-11344 2024.
Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pierric Cistac, Tim Rault, Remi Louf, Morgan Funtowicz, Joe Davison, Sam Shleifer, Patrick VOn Platen, Clara Ma. Yacine Jernite, Julien Plu, Canwen Xu, Teven Le Scao, Sylvain Gugger, Mariama Drame, Quentin Lhoest, and Alexander Rush. Transformers: State-of-the-ar natural language processing. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations, pp. 38 38-45, Online, October 2020. Association for Computational Linguisties.
Size WU, Wenwei Zhang, Lumin Xu, Sheng Jin, Zhonghua Wu, Qingyi Tao, Wentao Liu, Wei Li, and Chen Change Loy. Harmonizing visual representations for unified multimodal understanding and generation. arXiv preprint 1999, 2025.
Weihao Xia, Raoul de Charette, Cengiz Oztireli, and Jing-Hao Xue. Umbrae: Unified multimodal brain decoding. In European Conference on Computer Vision, pp. 242-259. Springer, 2024.
14

---

Published as a conference paper at ICLR 2026
A
APPENDIX
32
$5
kitchen preparing pizza
group flying in sky
group people flying holding deughnut
S1
32
S3
S5
Figure A7: Top: Comparison with eye-tracking. For each group, we visualize the transport plan weights and highlight the top 10 patches with the highest weights. Note that the captions here reconstructed from Yo' Mind model. Bottom: 2D histograms of gaze positions (log scale) for all subjects in NSD. These results are derived from Allen et al. (2022).
Stimull
Mind
Stimuli
Yo Yo'Mind
Figure A8: Image reconstruction examples of Yo' Mind.
A.1 EFFECTS OF THE NUMBER OF FMRI PATCHES
In this study, divide the fMRI voxels into 8 patches, following the underlying cortical organi- zation of the NSD dataset: early visual, midventral, ventral, midlateral, lateral, midparietal, and parietal regions, with the early visual cortex naturally spanning two patches due its large number
15

---

Published as a conference paper at ICLR 2026
Figure A9: Visualization the OT plan for the early visual cortex. On the left each example, we
highlight the top 80% of the mass.
Language Similarity Metrics
Dutaset
ROIS
Number
METEOR
ROUGE
CIDEr
NSD
early, midventral, ventral, midlaterul, lateral, midpurictal, purictal early, midventral, ventral, early, midventral,
34.99
40.13
NSD
midparietal, parietal midparietal, parietal
38.40
37.73
Table A5: Effects of the number of fMRI patches on reconstruction performance.
of voxels. To ensure that each patch has the same dimensionality, we pad the voxel vectors with zeros where necessary. The OT-based semantic allocation performed independently within each patch, meaning that the transport plan is constrained solely by the voxel representations in that re- gion. Consequently, visual cortex), the OT solution naturally assigns nearly zero mass to high-level semantic components (see Fig. A9 for examples). To further test effects of fMRI voxel split on the reconstruction perfor- mance, conduct ablation study on the number of fMRI patches. Note that for these alternative divisions, we first concatenated the voxels from the early visual, midventral, ventral, midlateral, lateral, midparietal, and parietal regions, and then evenly split the concatenated voxels into the spec- ified number of patches. Results (see Tab, A5) show that coarser or finer fMRI splits lead to a drop performance, which indicates that the IMRI split strategy is crucial for OT-guided alignment.
a patch does not contain high-level semantic information (such as the early
A.2 IMAGE RECONSTRUCTION
The proposed Yo' Mind focuses on brain-to-text semantic decoding, and image reconstruction is beyond the scope of this work, as they correspond to different tasks and model designs. Most existing prain-to-image reconstruction methods such as Chen et al. (2024b) adopt two-stream architecture, where one stream extracts high-level semantic representations and the other preserves low-level perceptual or spatial structures. other words, if image reconstruction desired, Yo' Mind can be naturally integrated with an additional stream dedicated to extracting low-level visual features from the fMRI data.
Here we report several IMRI-to-image decoding instances in Fig. A8 Following Chen et al. (2024b), we use L2-regularized linear regression to predict low-level latent representations images. The predicted latent representations, together with the decoded image captions, are then fed into Stable Diffusion (version 1.4) (Rombach et al. 2022 for reconstruction. Qualitative results demonstrate that this extension enables the reconstructed images preserve high fidelity to the original stimuli, both in terms of high-level semantic content and low-level perceptual details.
A.3 RESULTS ON THE DIR DATASET
To further validate the effectiveness and robustness Yo Mind, we perform brain-to-ter text decoding on the DIR dataset (Shen et al.) 2019), which is significantly smaller than NSD (Allen al. 2022 Note that the image stimuli in DIR originate from ImageNet. Therefore, following Chen et al. 2024b), we leverage SMALLCAP (Ramos al. 2023 to produce the pseudo labels. Results (see
16

---

Published as a conference paper at ICLR 2026
Language Similarity Metrics
Method
Subject
ROIs
Dataset
Text
B64
METEOR
ROUGE
SPICE
MindGPT (Chen stal.
2025a
SMALLCAP
10.3
MindAm Chen al.
2024b
VI-V4, FFA, PPA,
DIR
SMALLCAP
16.0
13.1
37.2
11.3
Yo Mind (Ours)
V1-V4, FFA. PPA, LOC
DIR
SMALLCAP
18.3
15.3
39.1
12.0
Table A6: Quantitative results of brain-to-text reconstruction on the DIR dataset.
Language Similarity Metrics
Method
Subject
ROLs
Dataset
Text
SSIM
FID.
CLIP
CLIP, @
SDDeecon Takag & Nishimato 20233
VI-4, FFA, PFA, LOC
DIR
SMALICAP
0.152
17.4
0.372
24,6%%
MindGPT
SMALLCAP
MindAn Chen et 40246
VI.4,, PEA, LOC
DIR
SMALICAP
0.223
287
0.622
40.3%
(Ouns)
SMALTCAP
Table A7: Quantitative results of train-to-image reconstruction on the DIR dataset. CLIP
denotes 10-way CLIP image-text scores.
Tab. A6 show that our method remains effective even when trained on small-scale fMRI data. We also report the visual reconstruction results on DIR as reference (see Tab. A7).
A.4 COMPUTATIONAL COST AND SCALABILITY
In our framework, we use the Sinkhorn algorithm Cuturi 2013) to solve the assignment problem. The Sinkhorn algorithm relies solely on matrix-vector products (with complexity O(nm)), allowing efficient GPU implementation. Moreover, given a fixed number of iterations and fMRI patch em- beddings, the computational cost of Sinkhorn grows linearly with the total number of image patch and word embeddings (N M).
17


Reviews:
[{"review_id": "bSZXjzaqlc", "reviewer": "Reviewer_HQz8", "summary": "This paper focuses on the task of reconstructing language from fMRI signals evoked by visual stimuli.\nThe authors focus on the fMRI representation learning stage of this task. Their main idea is that human subjects tend to pay attention to local regions within an image rather than processing the entire image uniformly.\nAccording to the authors, a previous approach enables the removal of unattended image patches under end-to-end optimization. The remaining patches (or tokens) are then used to compute the CLIP loss together with the fMRI embeddings.\nIn this paper, in addition to aligning the fMRI embeddings with a subset of image patches, the authors also introduce alignment between fMRI embeddings and a subset of tokens from the original image caption. Moreover, they improve the process of discarding unattended image and text tokens by leveraging the Optimal Transport (OT) framework.\nThe authors conduct experiments on the Natural Scenes Dataset (NSD), and their results outperform the previous related works they follow.", "questions": "1. Why does Figure 5 only show 80% of the retained patches? This kind of selective visualization does not represent the actual results produced by the model.\n\n2. I’m quite curious why the proposed method and the baseline model MindSA were not evaluated on the fMRI-to-image reconstruction task. If the trained fMRI representation model can truly capture the regions that subjects focus their attention on, then the reconstructed images should semantically correspond to the remaining (attended) image patches. Such an evaluation would provide stronger evidence for the effectiveness of the proposed approach.", "limitations": "", "rating": 4, "confidence": 3, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "1. This paper provides a well-organized and comprehensive review of the related work.\n\n2. The method proposed in this paper outperforms the baseline models.", "weaknesses": "1. Since the model is optimized in an end-to-end manner, its loss function is computed between the reconstructed text and the ground-truth (GT) captions provided in the MSCOCO dataset. Such a supervision signal seems to encourage the model to extract information from the fMRI signals that is relevant to the GT text, and thus, during the removal of image/text tokens, it appears to discard those unrelated to the GT captions. This training process, therefore, might not actually capture the image regions that the subject’s attention was focused on. In fact, the retained image and text tokens are those that are highly correlated with the ground-truth (GT) captions, which is also consistent with the visualization results shown in Figure 5. Therefore, in my view, the method proposed in this paper does not truly capture the regions that the subjects focused on during the visual perception process.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "yAywtnMyZb", "reviewer": "Reviewer_P2dE", "summary": "This paper presents Yo'Mind, an Optimal Transport-driven personalized multimodal semantic masking framework for brain-to-text decoding. The work addresses the critical challenge of semantic mismatch between machine representations and human brain signals during neural decoding. The proposed method introduces a dynamic semantic pruning mechanism that adaptively masks redundant visual semantic components based on individual neural responses, without requiring additional human supervision or hyperparameter tuning. The authors demonstrate state-of-the-art performance on the Natural Scenes Dataset while improving interpretability of the decoding process.", "questions": "Please see the weakness above.", "limitations": "", "rating": 6, "confidence": 2, "soundness": 2, "presentation": 3, "contribution": 3, "strengths": "1. **Novel Theoretical Framework:** The integration of Optimal Transport theory into brain signal decoding represents a significant innovation, providing a mathematically rigorous approach to address semantic mismatch. The virtual dustbin mechanism elegantly resolves the mass conservation constraint in traditional OT, allowing for adaptive semantic filtering.\n2. **Biologically-Inspired Design:** The multimodal semantic set construction combining visual patches and textual descriptions aligns well with human cognitive processing. The soft assignment mechanism better reflects the brain's distributed representation compared to hard masking approaches.\n3. **Comprehensive Experimental Evaluation:** The paper includes thorough comparisons with 8 state-of-the-art methods across multiple standard metrics (BLEU, METEOR, ROUGE, CIDEr, SPICE), demonstrating clear performance advantages. The ablation studies on brain regions and multimodal components provide valuable neuroscientific insights.\n4. **End-to-End Differentiable Architecture:** The seamless integration of OT modules with fMRI encoding and text decoding enables joint optimization while maintaining differentiability through Sinkhorn iterations, supporting both intra-subject consistency and inter-subject variability modeling.", "weaknesses": "1. The selection of cosine distance as the cost metric lacks sufficient biological or theoretical justification. Similarly, critical hyperparameters (ε=1 for Sinkhorn, K=8 patches) appear to be set empirically without sensitivity analysis.\n\n2. Results lack statistical significance testing (p-values, confidence intervals), and evaluation is confined to the NSD dataset without cross-dataset validation. The fixed loss weight (10 for alignment loss) seems arbitrary and may not generalize.\n\n3. The computational overhead of Sinkhorn iterations and OT optimization is not quantified, despite being crucial for practical applications. There's no analysis of training/inference time, memory requirements, or computational complexity.\n\n4. While attention visualizations are provided, deeper analysis linking the learned representations to established neuroscientific principles is lacking. The choice of standard ViT as fMRI encoder lacks biological adaptation reasoning.\n\n5. Insufficient details about implementation specifics hinder reproducibility, and the paper lacks discussion of brain privacy concerns and potential misuse of decoding technology.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "ULTyQbwePL", "reviewer": "Reviewer_AFHn", "summary": "This paper proposes Yo’Mind, a new framework for brain-to-text decoding using fMRI. The key idea is to reduce the semantic mismatch between brain activity and machine representations. The authors argue that current large vision–language models encode all semantic content in an image, whereas the human brain selects only the salient elements, and this selection differs across individuals. It introduces OT-driven personalized multimodal semantic masking, where an optimal-transport assignment dynamically determines which semantic components (image patches and text attributes) align with brain responses. The method is tested on NSD Dataset and achieves SOTA performance on most captioning metrics (BLEU, METEOR, CIDEr, SPICE).", "questions": "1. Robustness to textual supervision\nWhat happens if you generate attribute text from a smaller model or random captions?\nIs the gain from text semantics inherent or model-dependent?\n\n2. Can Mind-SA perform competitively with adaptive masking?\nA stronger baseline would be a dynamic masking version of Mind-SA (not fixed patch count).\n\n3. Statistical significance\nPlease provide confidence intervals or hypothesis tests for BLEU/CIDEr results.\n\n4. Scalability\nHow does complexity scale when increasing #patches / #text tokens? Is OT a bottleneck?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 2, "presentation": 3, "contribution": 3, "strengths": "1. Motivation is strong and well-founded.\nThe paper explicitly addresses a subtle but fundamental issue identified in Mind-SA (ICCV'25): neural decoding suffers from semantic mismatch because machine representations capture full-scene semantics while the brain encodes only selective ones.\n\n2. Improved interpretability.\nVisualizations of “brain-preferred semantic regions” show intra-subject consistency and inter-subject variability, which is a compelling neuroscientific insight.\n\n3. Extensive quantitative comparison.\nIncludes multiple recent baselines (MindGPT, Mind-SA, UMBRAE, Neuro2Language).\nResults show consistent improvements across most metrics.\n\n4. Novel formulation but simple integration.\nThe OT masking module is differentiable and can be plugged into any brain-to-text model, not just MindGPT.", "weaknesses": "1. Evaluation is restricted to one dataset (NSD).\nNSD is the standard benchmark, but the claim of general fine-grained multimodal alignment would be more convincing with additional datasets (other extended fMRI tasks).\n\n2. Dependency on generated textual semantics may bias supervision.\nSince text semantics are produced by Harmon/Qwen2.5, the reconstruction target space partly inherits the biases of that MLLM. No robustness study is provided to show how performance varies across different captioning models or prompts.\n\n3. Statistical significance not reported.\nImprovements over Mind-SA are meaningful (≈2 BLEU / ≈2 CIDEr), but there are:\nno confidence intervals,\nno statistical significance test (e.g., bootstrap / paired permutation test).\n\n4. OT computational cost and scalability not thoroughly analyzed.\nSinkhorn is run for 100 iterations, but no discussion on: efficiency impact when using more image patches, O(N×M) scaling when adding more semantic text attributes.\n\n5. The novelty is incremental, not groundbreaking.\nThe paper improves Mind-SA’s masking strategy, but still inherits the architecture/flow of MindGPT. The conceptual leap may be viewed as methodological refinement rather than a new brain decoding paradigm.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "Ps15FPMMrI", "reviewer": "Reviewer_eqj6", "summary": "This paper addresses the semantic mismatch in brain-to-text decoding and proposes Yo’Mind, an optimal transport-driven personalized multimodal semantic masking framework that dynamically prunes redundant semantic components. Its key innovations include introducing a virtual \"dustbin\" for soft-selection masking, unifying brain-visual-linguistic alignment, and enabling cross-subject decoding without extra supervision or hyperparameter tuning. On the NSD dataset, Yo’Mind outperforms baselines like Mind-SA across metrics and achieves better fine-grained decoding.", "questions": "none", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. Unlike prior fixed-patch masking methods, Yo’Mind introduces a virtual \"dustbin\" into the OT framework to realize soft, adaptive semantic pruning, which automatically assigns irrelevant visual-textual components to the dustbin based on individual fMRI responses, enabling fine-grained alignment with human selective attention.\n2. This paper creatively integrates CLIP-derived visual patch embeddings and MLLM-generated textual embeddings into OT-driven alignment, extending traditional brain-visual mapping to brain-visual-linguistic alignment.\n3. Rigorous experiments (NSD dataset, 8 metrics, ablations on brain regions/masking) and clear qualitative analyses confirm Yo’Mind’s superiority over SOTA.", "weaknesses": "1. The study only used fMRI data from 4 out of 8 subjects, 1, 2, 5, 7, in the NSD dataset, which may not enough to fully prove its cross-subject decoding works well for everyone. It should test the other 4 subjects too, to see if Yo’Mind stays reliable across more diverse people.\n2. It only used one MLLM, Qwen2.5, to make textual semantic cues. We can’t tell if the better performance comes from mixing visual and text data, or just because Qwen2.5 is good. It should try other common MLLMs, better or worse, to show the design works no matter which MLLM is used.\n3. This paper says it captures “brain-preferred semantics,” but there is no check with real physiological data like eye-tracking. If it compared the model’s focused semantic areas to where subjects actually looked, that would make the claim way more believable.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "UVf0zJQh5j", "reviewer": "Reviewer_4kw3", "summary": "The authors propose a novel semantic decoding model Yo'Mind that utilizes Optimal Transport (OT) theory to allocate semantic components to fMRI patches. Using this framework, the paper implements cross-subject fine-grained semantic-level brain decoding. Experimental results confirm the powerful capability of the proposed Yo'Mind in semantic decoding.", "questions": "See weaknesses", "limitations": "", "rating": 2, "confidence": 4, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "(1) The authors introduce OT theory into the brain decoding field and propose a novel subject-adaptive and fine-grained alignment semantic decoding framework.\n\n(2) Experimental results validate that the proposed model has strong semantic decoding capabilities.", "weaknesses": "(1) **Unfair comparison**. Although the proposed model achieves superior performance in Table 1, this comparison is not fair. As shown in Table 2, performance significantly improves when the voxel number is increased. Some baselines, such as UMBRAE [1], used fewer voxels (subj01: 15,724), while the author compared it with more voxels (subj01: 21,721). Additionally, I am curious why the performance of UMBRAE slightly differs from the report in the original paper.\n\n(2) **Lack of low-level decoding ability validation (image reconstruction task)**. The paper only validates the model on the brain caption task, lacking validation on direct image reconstruction. Image reconstruction tasks can assess the model’s performance at the low-level. Note that both low-level (e.g. perceptual) and high-level (e.g. semantic) capture capabilities are equally important for brain decoding [2]. I am concerned that by using OT to allocate different semantic components to fMRI patches, the model may disrupt the spatial information of the original visual stimuli, which could result in a lack of low-level decoding ability. Therefore, the proposed model may have gained high-level decoding ability at the cost of sacrificing low-level decoding ability. The low-level decoding ability needs to be measured through the metrics of image reconstruction tasks (e.g., SSIM, FID, PSNR).\n\n(3) **Biologically unreasonable OT-driven semantic allocation**. The author uses OT to allocate different semantic components to fMRI patches from different brain regions. However, this operation may violate the biological response of the human brain to visual stimuli. In fact, some lower-level brain areas (such as V1-V4) are responsible for processing raw optical information, while higher-level brain areas handle color and texture, and even higher-level areas are responsible for semantic understanding [3]. But the authors allocate semantic components to all brain regions using OT, including those that are biologically not involved in semantic understanding. Therefore, I question the validity of this OT-based approach. In addition, the authors fix the number of fMRI patches at 8, but there is a lack of further explanation (or ablation) for this choice.\n\n(4) **Incomplete validation of the motivation and model efficacy**. The author’s motivation is that different subjects focus on different areas when viewing the same image. Although the author provides a visualization of the OT semantic mask in Figure 5, there is no evidence to prove that the proposed OT-driven method truly captures the subjects’ attention. It would be interesting if this could be confirmed (for example, through eyetracking data provided by NSD).\n\n\n\n(5) **Some expressions in the paper are not clear enough**. (a) In Figure 5a, the author states, \"The first row shows examples of intra-subject consistency, and the second row presents examples of inter-subject variability.\" However, the figure lacks detailed annotations, so we cannot clearly understand what the four images for the same sample represent (four different subjects? or inferences with different seeds from the same subject?). (b) The citation format throughout the paper is incorrect, which makes it difficult to read. Please use the `\\citep` command. (c) The mathematical expressions are confusing. In Line 210, what do you mean by $\\min_{\\mathbf\\Gamma\\in\\Pi}$? What is $\\Pi$? It has not been described earlier in the text. In Equation 2, what is $\\mathbf 1_{N+M}$ and $\\mathbf 1_{K}$? ... These are just examples of the issues I mentioned.\n\n[1] Xia et al., Umbrae: Unified Multimodal Brain Decoding. ECCV, 2024.\n\n[2] Scotti et al., Mindeye2: Shared-subject models enable fmri-to-image with 1 hour of data. ICML, 2024.\n\n[3] Grill-Spector et al., The Human Visual Cortex. Annu. Rev. Neurosci., 2004.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)