AttTok: Marrying Attribute Tokens with Generative Pre-trained Vision-Language Models towards Medical Image Understanding
Authors: Hualiang Wang
Xinyue Xu
Lehan Wang
Bin Pu
Xiaomeng Li
Venue: ICLR 2026 Poster (2026)
Keywords: Medical generative pre-trained models
medical Multi-Modal alignment
medical VQA
instruction tuning
Primary area: applications to computer vision, audio, language, and other modalities

Abstract: Recent generative pre-trained vision–language (GPTv) models have achieved remarkable success in multi-modal understanding, inspiring their adaptation to medical imaging tasks such as disease diagnosis and visual question answering (VQA). However, current instruction-tuned GPTv models suffer from two key challenges: (1) medical attributes (e.g., disease names, severity grades) are encoded as plain text tokens, collapsing semantically distinct concepts into nearly identical textual sequences; and (2) inadequate textual supervision weakens visual representation learning, leading to severe inter-attribute confusion and misaligned vision–language embeddings. To address these limitations, we introduce attribute tokens (AttTok), a set of pre‑defined special tokens that uniquely encode clinical attributes (e.g., imaging modality, diagnosis, severity) within a structured token space. Complemented by attribute‑centric embedding books, AttTok serves as anchor points for aligning both visual and textual modalities into a shared, discriminative representation space. Building on this foundation, we design two key components: an attribute‑centric cross attention (ACC) adapter, which breaks the vision‑to‑text information‑flow bottleneck and enriches the visual encoder with discriminative attribute knowledge, and an attribute‑centric matching (ACM) loss, which enforces robust multi‑modal alignment centered on the attribute tokens. Extensive experiments on five medical classification benchmarks and three VQA datasets demonstrate that AttTok substantially improves both discriminative accuracy and medical knowledge reasoning, establishing a new paradigm for medical GPTv models with clinically discriminative understanding.

Published as a conference paper at ICLR 2026
ATTToK: MARRYING ATTRIBUTE TOKENS WITH GEN-
ERATIVE PRE-TRAINED VISION-LANGUAGE MODELS
TOWARDS MEDICAL IMAGE UNDERSTANDING
Hualiang Wang, Xinyue Xu, Lehan Wang, Bin
Department of Electronic and Computer Engineering The Hong Kong University of Science and Technology
Xiaomeng Li
Department of Electronic and Computer Engineering The Hong Kong University of Science and Technology eexml cemliuust... .hk
ABSTRACT
Recent generative pre-trained vision-language (GPTv) models have achieved re- markable success in multi-modal understanding, inspiring their adaptation to medical imaging tasks such as disease diagnosis and visual question answering (VQA). However, current instruction-tuned GPTv models suffer from two key challenges: (1) medical attributes (e.g., disease names, severity grades) en- coded as plain text tokens, collapsing semantically distinct concepts into nearly identical textual sequences; and (2) inadequate textual supervision weakens vi- sual representation learning, leading to severe inter-attribute confusion and mis- aligned vision-language embeddings. To address these limitations, we introduce attribute tokens (AttTok), a set of pre-defined special tokens that uniquely encode clinical attributes (e.g., imaging modality, diagnosis, severity) within a structured token space. Complemented by attribute-centric embedding books, AttTok serves as anchor points for aligning both visual and textual modalities into shared, dis- criminative representation space. Building this foundation, design two key components: an attribute-centrie cross attention (ACC) adapter, which breaks the vision-to-text information-flow bottleneck and enriches the visual encoder with discriminative attribute knowledge, and an attribute-centric matching (ACM) loss, which enforces robust multi-modal alignment centered on the attribute tokens. Extensive experiments on five medical classification benchmarks and three VQA datasets demonstrate that AutTok substantially improves both discriminative accu- racy and medical knowledge reasoning, establishing new paradigm for medical GPTv models with clinically discriminative understanding. Codes.
INTRODUCTION
Recent advances in generative pre-trained lasion-language (GPTv) models, such as GPT-4o (Hurst al. 2024) and Qwen2.5-VL (Bai et al., (002)), have demonstrated remarkable progress across wide range of universal visual understanding tasks (Peng al. 2023 Liu et al. 2024a b). These achievements have spurred growing interest in developing unified medical GPTv models, resulting in systems such as VLaMMM (Li al. 2023). HealthGPT (Lin et al.) 2025), and Lingshu (Xu et al.) 2025). Despite their general capabilities, current medical GPTv models still fall short in discase diagnosis and classification tasks, particularly when required to deliver precise -rccsion-makingg on extensive clinical attributes (e.g., disease names and lesion types) across diverse medical imaging modalities, as evidenced by the limited zero-shot performance reported in Table
Instruction tuning (Liu et al. 2023b) provides a feasible solution by representing medical attributes as specialized textual terms within instruction-tuning datasets. GPTv models learn to predict the
"Corresoonding author

---

Published as a conference paper at ICLR 2026
vision reput
quection prompt
ondeer nesponse
vil mpp
question prompt
ortiee response
Mild bR
"Whet conden be
conditicn derecred"
nnd
bever OR"
Moderate be
Weker it ccoiio "askff Seveer?
Wil o snerched
BR
GPTV
ottriuue
GPTV
dieass takes
(nl) Vonilo
(oé) GPTv nedel
Mackcol
enbeddinge
textual
otte
SPTV AfTTak
Figure I: Comparison between (al) vanilla medical GPTv with only textual response tokens and (b1) our proposed GPTv model with discriminative attribute tokens. (al) The vanilla model pro- duces plain textual answers (e.g., "mild DR". "severe DR"), which may lack explicit contextual or clinical meanings. (b1) By introducing attribute tokens and multi-modal token books, model augments responses with discriminative features. (a2) and (b2) exhibit the embeddings of disease- level visual tokens (o with different colors) and disease tokens (* with different colors) from two GPTv models instruction tuned on DR grading dataset. The embeddings are projected into 2D space via t-SNE (Maaten Hinton 2008). The disease tokens are the textual discase names for vanilla GPTv models, and attribute tokens for GPTv models with AttTok.
textual sequence in generative manner, i.e., next-token prediction paradigm (Radford al. 2018). Despite offering flexibility in representing diverse clinical concepts through word combinations, this paradigm a double-edged sword, exposing two critical limitations:
(1) Encoding medical attributes as simple word combinations fails to reflect their underlying clinical semantics and inter-attribute distinctions. For instance, conditions such as mild diabetic retinopathy (DR) and severe diabetic retinopathy are represented by nearly identical textual sequences (Figure I (al)) rather than as clinically meaningful categories with distinct pathological characteristics. As illustrated by the DR grading example in Figure [ (a2), the token visualization reveals substantial confusion among textual tokens corresponding different diseases, (marked via * with different colors and zoomed out here for clearer display) for the instruction-tuned medical GPTv model.
(2) More critically, owing to the causal information flow (Radford et al., 2018) in GPTv models (vi- sion- - text), inadequate textual encoding of medical attributes provides only weak supervision back to the visual modality. Consequently, the model fails to acquire discriminative visual representations required to capture clinically salient features. This shortcoming results in pronounced inter-attribute confusion within visual features and vision-text alignmennmn,, as illustrated in Figure (a2).
In essence, the generative paradigm provides flexibility but at the expense of reducing the discrim- inative power of tokens. As a result, it produces ambiguous representations of clinical attributes, entangled visual features, and weakened vision-languse alignment. This raises natural question:
Can discriminative medical attributes be explicitly and precisely incorporated into the generative paradigm of GPTv models?
Motivated by this, introduce novel concept, Attribute Tokens (AttTok), a of special tokens that uniquely convey the pre-defined clinical attributes. For instance, the token < ff > specifies the imaging modality (fundus imaging) and the diagnostic finding (severe DR). Further- more, attribute-centi embedding banks are constructed to provide representative features, using attributes as anchors to facilitate the alignment of multi-modal information.
On top of the attribute tokens and embedding books, (1) an attribute-centri cross attention (ACC) adapter introduced to infuse clinical attribute knowledge from the embedding books into the visual encoder. ACC enhances visual saliency to perceive corresponding clinical attributes while eliminat- ing the vision-to-tex unidirectional bottleneck inherent in GPTv models. (2) an attribute-centric matching (ACM) loss is designed to enhance GPTv models by aligned and discriminative feature learning through intra-atrribute consistency and anter-atrribuee separability.
To evaluate the effectiveness of our AttTok, establish comprehensive evaluation framework that systematically assesses both discriminative and generative capabilities of instruction-tumi GPTv models. The evaluation covers five publicly available datasets spanning diverse medical
2

---

Published as a conference paper at ICLR 2026
imaging modalities, including dermatology (Derma), fundus photography (Fundus), optical coher- ence tomography (OCT), radiography (X-ray), and pathology (Path), addressing discriminative dis- case/lesion diagnosis and classification tasks. Furthermore, three commonly used medical VQA benchmarks, encompassing diverse medical knowledge, are employed to evaluate performance on medical knowledge understanding.
In summary, our contributions are:
We introduce attribute tokens and attribute-centrr embedding books explicit carri- ers of domain-specific attributes. These constructs provide GPTv models with structured access to discriminative medical semantics, thereby moving beyond surface-level word memorization.
Conceptual:
Technical: We develop an attribute-centric cross attention adapter to highlight atrrbutee-rleeann
visual embeddings, and attribute-centrie matching loss align multi-modality information into a unified embedding space. These components jointly enable GPTv models to explicitly integrate discriminative medical knowledge into the generative paradigm, enhancing both semantic grounding and discriminative capacity.
2
RELATED WORK
2.1
GENERAL GPTV MODELS
The development generative pre-trained vision-language (GPTv) models has drawn increasing attention in both computer vision and natural language processing communities. Broadly, prior efforts can be grouped into three threads.
Large multimodal transformers. Recent large-scale models such as GPT-4o (Hurst et al. 2024) and Qwen2.5-VE VL (Bai et al., (025) integrate visual and textual streams within unified transformer architectures. These systems leverage massive web-scale corpora and joint optimization strategies to acquire strong perception, reasoning, and generation abilities across diverse modalities. A related line of work, such Flamingo (Alayrae al., (022), Kosmos-1 I (Huang al. (023), and PaLM- E (Driess et al. 2023), explores cross- modal fusion with frozen backbones. auxiliary adapters, or joint pretraining strategies, further demonstrating the scalability of multimodal LLMs.
Instruction-tunee VL models. Instruction tuning has proven to be an important technique for aligning multimodal models with user-centric tasks. Representative models include LLaVA (Liu et al. (023a) and InstructBLIP (Dai et al. 2023). which adapt pretrained VL backbones with natural language instructions to support multi-turn dialogue, visual reasoning, and open-ended question answering. These approaches highlight the effectiveness of aligning general-purpose GPTv models with task-specific instructions while maintaining generalization capabilities.
Task-specitic extensions. Beyond general reasoning, several models target structured multimodal tasks such as captioning, retrieval, or visual grounding. For example, PaLI (Chen et al., 2022) demonstrates unified performance captioning and translation, whereas specialized models extend GPTv with external memory, retrieval- -ugmeeted features, or multi- caalltigraaularityy embeddings to im- prove interpretability robustness. While successful in natural image understanding, these methods emphasize linguistic fluency and broad coverage rather than fine-grained attribute discrimination.
2.2 MEDICAL GPTv MODELS
The success of GPTv models general domains has motivated their extension into medical appli- cations, where multimodal reasoning is essential. This body of work can be divided into text-only and vision- languae branches.
Medical language models. Early efforts such BioGPT (Luo al. 2022). PubMedGPT (Bolton ct al.) 2022), and PMC-LLaMA (Wu et al. 2024) focus on text-only corpora, capturing biomedical terminology and clinical reasoning through large-scale language modeling. More recently, Med- PaLM Tu et al., 2024) and Med-PaLM (Singhal et al. 2023) incorporate instruction tuning on curated QA datasets, yielding LLMs capable of expert-level responses in complex medical contexts.
Multimodal medical GPTy models. To further integrate visual signals, recent works such as Hu- aTuo (Wang al. 2023), HealthGPT (Lin al. 2025), and Lingshu (Xu al. 2025) adapt multi-
3

---

Published as a conference paper at ICLR 2026
modal instruction tuning pipelines to train GPTv variants medical image text pairs. These sys- tems enable models to answer clinical visual questions. identify diseases, and provide textual expla- (Moor
nations. In another direction, models such as ChatCAD (Tang al. 2025) and et al., 2023) incorporate conversational interactions, retrieval mechanisms, or large-scale clinical datasets to improve alignment with radiology pathology domains. These advances mark signifi- cant progress toward general-purpose medical GPTv assistants.
Despite these efforts, most medical GPTv models inherit the training paradigm from general GPTv models, representing medical attributes (e.g., disease names, severity scales, anatom- ical structures) as plain text tokens. This introduces two fundamental limitations: (1) semantically distinct medical concepts are encoded as highly similar textual sequences, weakening discrimina- tive capacity; and (2) the causal prediction objective offers weak supervision to the visual encoder, resulting in entangled features and ambiguous cross-modal alignment. Addressing these issues re- quires attribute-awar representations that explicitly capture clinical semanties while strengthening visual-textual alignment.
Limitations.
PRELIMINARY
The canonical generative pre-trained vision-language (GPTv) model is typically composed of three key components.
(1) Vision encoder. A vision encoder & (typically instantiated as VIT Dosovitskiy et al. 2020 maps an input image I into a sequence of visual token embeddings:
E. E R.N.dd
(1)
where N, denotes the number visual tokens and is the embedding dimension.
(2) Text tokenizer and embedding. A tokenizer T converts input text into sequence of discrete token indices, which are subsequently projected into the embedding space by a token embedding layer &. The embedding layer maintains set of token weights {e. with each e, R xx corresponding to the i-th entry in the vocabulary and M indicating the total number of cadidate tokens. Given word token from the textual questions Q answer response R, its index and embedding are computed as:
= T(),,
Eg(3) = Ey:
where y E {I...., .M3 } is the token index and f E 1xd is the corresponding textual embedding, in terms of the word from Q and R.
(3) Decoder-only language model. A decoder-only large language model Dr consumes a concate- nated sequence of visual embeddings F" (totally N, tokens), question embeddings F (totally Ng answer token, calculated in Equ 2), and previously generated response embeddings F (tokens be- fore the i-th token, calculated in Eqn 2) It predicts the probability distribution of the next answer token of as:
(I F i i [1,
where of is the ground-truth index of the i-th token in the response sequence with totally N,, tokens.
Training objective. Training follows the next-token prediction paradigm, where the model is opti- mized with cross-entropy loss over the answer sequence:
CNTP
1 N
Σ log plyl
Inference. At test time, the sequence of the input image and question prompt is provided, and the model autoregressively generates the response tokens y" using the next-token prediction. This unified training-inference formulation ensures coherent answer generation.
4

---

Published as a conference paper at ICLR 2026
Input Image
Question
Ánswer
visual emb.
emb, books
What condition could be detected?
fIfuddus_sde severe DR.
Store
pooling
ema update
VIT
Text Tokenizer
(b) updating embeclding books
Embedding Loyer
visual emb, att-centric emb.
(c) ACC
OOOk K.
Cross-Attn.
emb, books
Large-Language Model
visual emb.
Centrute-Centric CA (ACC)
(b) updating embedding books
attribute emb.
(d) ACM loss
PoSITIVe emb
X
emb books
NTP loss
emb books
x
negative
(a) The workflow of AttTok
(d) Attibute- Centrrc Matching Loss
attribute-centric embeddings
current embeddings
Figure 2: The architecture AttTok. (a) The workflow integrates visual, textual, and attribute embeddings, processed through VIT. textual tokenizers, and embedding layers, and optimized with Next Token Prediction Loss and attribute-centric matching loss. (b) The updating strategy for attribute-centric embedding books of pre-defined attributes. (c) The attribute-centric cross attention module enhances visual embeddings via discriminative embeddings from medical attributes. (d) The attribute-centric matching loss enforces alignment between positive attribute-centréc embeddings while separating negative ones.
ATTRIBUTE TOKENS FOR GPTv MODELS
In this section, introduce our framework for incorporating clinical attribute tokens into GPTv models. For each attribute token, we construct embedding book that optimizes and stores dis- criminative visual, attribute and textual representations.
Building upon these token books, we further design two key modules: (1) Attribute-Centric Cross Attention (ACC) adapter, which enriches visual encoders with attribute- perception; and (2) Attribute-Centric Matching (ACM) loss, which enforces cross-modal alignment among visual, attribute, and textual tokens better capture inter-attribute clinical salience and distinctions. The overall pipeline is illustrated in Figure 2
4.1 INITIALIZING AND UPDATING ATTRIBUTE TOKENS AND TOKEN BOOKS
Initializing Attributes. Here, we formally define attribute token as a new pre-defined token used represent a specialized clinical concept. For example, < Ifundus. sdr denotes that the input modality is fundus imaging and the associated clinical concept is severe diabetic retinopathy (DR). In general, such concepts can correspond a disease, a lesion, or an anatomic site.
These attribute tokens are generated alongside the instruction-tuning datasets. For classification- based datasets, the attributes are naturally determined by the category labels. contrast, for VQA datasets, the relevant attributes are pre-processed and extracted directly from the question-answer texts. Specifically, we employ open-source GPT models to summarize keywords from all ques- tion-answer pairs. Once the keywords question question-answer answer pair, which serves as the ground-truth keyword annotation. These keywords are then organized into tokens following the format < |modality concept) >. For example, keyword
determined, further GPT assign them each
5

---

Published as a conference paper at ICLR 2026
is "severe DR" and attribute token is fundusssdr) aforementioned. A detailed workflow provided in Appendix A.2.
Initializing and Updating Attribute Token Books. To incorporate attribute information into GPTv models, we expand the embedding layer to Je U {a:) where is the token em- bedding of the i-th attribute token, with totally attribute tokens.
For each attribute token, we construct multi-modal discriminative embedding book used each at- tribute as anchor. The embedding book for the k-th attribute is centered on corresponding attribute token and contains: (1) the attribute token itself, agi (2) the textual token, (md(k)) corresponding to the attribute's ' keyword (where (k) denotes the index of the textual tokens associated with at- tribute k); and (3) the visual prototype token, fü derived from the average visual tokens belonging attribute Formally, By (ax: Eind(k)) }i:
To smoothly estimate the visual prototype for each attribute, we employ an exponential moving average (EMA) to iteratively update the prototype fü. Given all visual embeddings associated with attribute k in the current training batch, the visual prototype token is updated
ufk + (1 p1) No JEN.
Sj. SJ.
where N, denotes the number visual embeddings belonging to attribute k the batch. μ is typically set value close 1, Here set it as 0.99. If no samples of attribute & are present in current batch, the corresponding visual prototype remains unchanged. Since both ag and Cin((k) are learnable parameters, they jointly optimized during the training of the GPTv model.
4.2 ATTRIBUTE-CHE CENTRIC CROSS ATTENTION
In GPTv models, the causal information flow proceeds from vision text, preventing the vision encoder from directly accessing textual information. However, pure textual tokens lack discrimina- tive signals to effectively backpropagate supervision the vision encoder. This limitation results in poor visual-text alignment.
To address this weakness, we propose Attribute-Centric Cross Attention (ACC) adapter, which explicitly injects discriminative attribute knowledge into visual token representations. Essentially, the ACC module establishes skip route that delivers attribute signals directly the vision encoder.
Given the visual embeddings F" of the visual encoder, we enhance it via cross-attention over the attribute embedding books B {Bx / of all K attribute. Formally, the total 3K tokens from B serve as the key and value embeddings and the visual embeddings F is treated as query tokens, the adapted visual embeddings F" are calculated as:
(F") (BWv),
Aut Fe, B) Softmax
Vd F. F + A
where Wo. WK. Wy and Wo are learnable projection matrices. 7 is a weighting factor that controls the strength of the ACC adapter. Here set y 0.1. Since direct access to ground-truth responses unavailable in visual feature learning, all attributes are employed as both keys and values, enabling ACC to adaptively learn the correct activations.
The enhanced representation F" inherits discriminative guidance from both textual attributes and visual prototype tokens. This design enables the model to learn attribute-ware perception, strength- ening the alignment between medical concepts and visual structures, and effectively breaks the vision-to-text bottleneck inherent in standard GPTv models.
4.3 ATTRIBUTE-CENTI MATCHING LOSS
While the cross-attention mechanism enriches visual tokens with attribute- aware information, it is still essential to explicitly enforce the alignment among visual representations, the corresponding

---

Published as a conference paper at ICLR 2026
attribute tokens and their textual counterparts. To this end, introduce Ctrribute-eettrie Matching (ACM) loss that leverages the attribute token books as anchors to drive discriminative and modallty-aligneee representation learning.
Positive and Negative Sets. Given image, the GPTv model predicts the embedding vector s corresponding to the attribute k, The positive embedding book By provides three forms of positive supervision: (1) the attribute embedding. (2) the textual embedding of the attribute keywords, and (3) the visual prototype. Conversely, negatives are drawn from all other books Bj, o k.
Similarity Measurement. quantify alignment, we employ a Linear layer, 8() (IR lxd - RIId), to project embeddings into unified space to measure the cosine similarity:
((a) ((b) s(a b) = [1((a) (laaa)lll@bb) [[b][
(8)
This symmetric metric encourages tokens from the same attribute (regardless of modality) to lie close in the embedding space, while pushing tokens from different attributes apart.
Multi-modality Matching Objective. Building on this definition, we construct matching loss that promotes alignment of the predicted attribute embedding f with all positives while contrasting against negatives:
Σ
exp
<(f".p)
(f)) loog
PEB,
Σ PEBL
exp
pf) 7
exp
sif .n)
BEB
where 7 temperature parameter controlling concentration. Intuitively, this formulation performs soft classification of fa with respect to the anchors in the attribute book. ACM loss integrates multi- modal (visual, textual, and attribute) supervision. thereby producing a richer and more discriminative signal for embedding alignment. Importantly, attribute books serve consistent anchors across modalities, ensuring robust cross-modal coupling and disentanglement distinct attributes.
Finally, the ACM loss is combined with the standard Next Token Prediction (NTP) loss to jointly optimize the model given an input sample (I, Q. R). The total loss objective is LNTP + ACACM, where À is hyperparameter balancing the contribution of the attribute-centric matching loss. The NTP loss ensures faithful generation of responses, while the ACM loss enforces discriminative attribute-aware representations, enabling the model to achieve both accurate medical understand- ing and coherent textual generation.
EXPERIMENT
5.1 EXPERIMENTAL DETAILS
Baselines. To comprehensively evaluate the performance of our AttTok various medical bench- marks, we compare it against diverse set of baseline models, encompassing both proprietary mod- els and open-source GPTv models from both general and medical domains. For medical classi- fication and diagnosis tasks, we further compare AttTok with CLIP-based models, the discrimi- native models. Specifically. our evaluation includes the following models. (1) Medical GPTv models include Med-R1 (Lai et al.) 2025). MedVLM-RI (Pan et al., (2025), HuatuoGPT-V (Wang al. 2023). HealthGPT (Lin al. 2025). LLaVA-Med (Li et al. 2023) and Lingshu (Xu et al. 2025 (2) General-purposs GPTv v models consist of GPT-4.1 version Hurst et al.] (2024) and Gemini-2.5-Flash (Comanici et al.) 2025), which are the most representative proprictary models available with strong general visual understanding capabilities. Q
(Bai et al.)
2025) and InterVL33 (Zhu et al. 2025), which are two state-of-the-a open- spen-source models, (3)
CLIP-based models include CLIP (Radford et al., 2021), SigLIP2 (Tschannen et al., 2025) and PubMedCLIP (Eslami et al., 2023). These discriminative vision- language models are used to give the comparision reference medical classification tasks.

---

Published as a conference paper at ICLR 2026
Table 1: Performance comparison of our methods with other GPTv models and fine-tuning methods on medical disease diagnosis and classification tasks across five modalities. The reported metrics close-end and open-end accuracies. "_" presents the unanswerable or extremely low perfor- mance on open-end zero-shot inference for precise disease diagnosis.
Models
Denma
Fundus
OCT
X-ray
Path.
Avg.
open
close
open
close
open
close
open
close
open
close
open
close
ClJP-hased Discriminative Models
CLIP
68.4
62.6
89.8
93.5
89.8
80.8
SigLIP2
65.3
86.7
92.9
89.1
80.0
PuhMedCTJJP
65.
59.8
89.6
90.2
88.2
78.6
Zero-shot GPTv Models
HeallhGPP.L
13.9
28.4
29.1
65.9
34.7
34.4
HudTwoGPT-35B
78.3
Swen2.5-VL-77
14.2
25.7
24.2
$1.9
30.8
29.4
Lingshu-7B
41.9
33.2
25.5
85.0
56.8
48.5
Instruction-Tuni GPTV Models
Qwen2.5-VI.
74.9
90.5
75.2
84.4
78.4
Lingshi:B
550.3
B6.8
74.7
76.9
88.
73.5
34.5
67.2
76.9
85.3
92.1
77.5
88.0
80.9
Instruction tuning. For all models, the default technique is LoRA which is common-used adap- tation for the parameter efficiency. Besides we also compare AttTok with different alignment tech- niques such as vision-text alignment CCL (Jiang et al. 2025) and visual-only alignment. For training data of AttTok, all attribute tokens are placed in front of the original text and serve as discriminative guidance.
Benchmarks. To validate the effectiveness of our proposed AttTok in enhancing medical image understanding tasks, conduct experiments on two kinds of benchmarks.
(1) Medical image diagnosis and classification tasks selected from five common medical imaging modali- ties, including ISIC-2018 (Milton) 2019) for seven-class
Table 2: Performance comparison on medical VQA tasks.
skin lesion diagnosis. Optical coherence tomography
(OCT) images (Kermany et al. 2018) for retinal diseases comprises of diagnosis categories. NCT-CRC-HE. 100K of histological images (Chen & Krishman (022),
Rad-VQA
PaiNQA
32.5
LLaVA-Med-71
48.0
38.8
468
comprises of types of tissues. MessiDR (Lepetit- Aimon
HuatooGPE-V. V-78
67.0
67.3
48.0
609
et al. 2024) is used to conduct DR grading task. X-Ray dataset (Yang et al. 2023) for pneumonia classifications
1 Langshn-78
67.9
83.1
61.9
71.0
Instraction Tuned GPTV Models
69.5
X3.1
62.3
71.6
is also employed.
Dars
84.0
63.5
(2) Medical VQA tasks include Rad- (Lau et al, 2018), SLAKE (Liu al. (021). and Path PathVQA (He al., 2020). These datasets cover CT, MRI, Chest-Xray, and Pathology modalities on questions i location, modality and normal/abnormal judgment.
84.6
61.1
Ours
71.4
85.8
64.7
34.0
Evaluation. (1) DiaagnssiCCassificattinnn Datasets are organized as two formats including the open-ended questions (without refereed category candidates in questions), and close-ended ques- tions (with refereed category candidates in questions).
(2) VQA. For open-ended questions, we first apply strict, rule-based evaluation to check for exact matches (i.e., accuracy) between the model prediction and the ground-truth answer. When the rule- based method does not find exact match, an LLM--as---jdde strategy is used to judge whether the prediction semantically equivalent to the ground truth. For close-ended question, the rule-based accuracy is reported.
8

---

Published as a conference paper at ICLR 2026
5.2 PERFORMANCE COMPARISON
Medical Classification and Diagnosis. Table reports the results of medical classification across five modalities. We observe that: (1) The Necessity of instruction tuning for precise medical imaging classification and diagnosis. It is witnessed by the notably low zero-shot performance of GPTv models this domain. medical-specific GPTv models. This is evidenced by AttTok's consistent improvements across five modalities on both types of GPTv models, yielding least a 2% increase accuracy. (3) Compa- rable to discriminative vision-languase models. Benefiting from the multi-modal discriminative capacity introduced by AttTok, GPTv models are able to achieve performance comparable to, and in skin lesion, DR grading and X-ray normal/abnormal status classification tasks surpassing, CLIP- based models, while still operating within a generative framework.
Consistent and significant gains across general-purpon and
Medical VQA. The results on multi-modal VQA benchmarks are given in Table The improve- ments from instruction tuning Lingshu-7B (Xu et al. (2025) are relatively modest (compared to zero-shot version), this can be explained by the fact that Lingshu has already undergone extensive pretraining large-scale medical data, including multiple rounds of VQA and report generation tasks, which leaves limited room for further gains. Nevertheless, even under this circumstance, our AttTok still achieves accuracy improvements via 0.5%, 1.2% and 0.6% accuracy improvement. This particularly notable given that the attribute definitions and initializations (details in Appendix(A.2) in VQA datasets are relatively coarse compared to the more explicit labels used in diagnostic and classification tasks.
5.3
ABLATION STUDY
(a) Dermatology
(b) Fundus
(c) OCT
(d) X- X-Ray
(c) Pathology
Figure 3: The effectiveness of ACC and ACM modules on five modality medical imagings.
Effectiveness of ACC and ACM Modules. Figure reveals the effectiveness of the proposed components. (1) The attribute-centric cross attention adapter (ACC) breaks the bottleneck of exist- ing GPT-v models, where information typically flows only from vision to text. By injecting attribute information into ViT, ACC enables richer cross-modal interaction and consistently boosts perfor- mance. (2) The attribute-cnttric matching loss (ACM) explicitly unifies clinical attributes in both visual and textual spaces, aligning them into a shared representation that captures domain- specific semantics. This design achieves substantially higher gains than using ACC alone.
(3) When jointly applying ACC and ACM, the model achieves the strongest results, showing their complementary benefits and further validating the superiority of our attribute-centric alignment framework.
Effectiveness Different Token Alignment Strategies. As shown in Table 4 both v.-t. align- ment (directly aligning vision and text tokens, meth- ods from (Jang et al., 2025)) and v. alignment (clus- tering visual tokens) bring limited or unstable gains compared with the baseline. In particular, align- ment even degrades performance on certain modali- ties (e.g., Fundus, OCT), indicating that naive visual token clustering loses critical fine- -ra-grained informa- tion. In contrast, marks, achieving the top accuracy on every modality (e.g., +3.3 on Dermatoscopy, +3.2 on Fundus, and +2.7 OCT over the best baseline).
Table 3: Performance on different token alignment strategies.
Method
Derma
Pundus
OCT
X-Ray
Puth.
Lingsbu 7B
72.8
63.7
76.7
89.1
81.3
76.3
85.8
aligo.
73.1
74.2
84.6
Ours
76.1
79.0
87.2
proposed strategy consistently outperforms all alternatives across five bench-

---

Published as a conference paper at ICLR 2026
Additional experiments, including feature visualization analyses and extensions imbalanced learn- ing problems, are presented in Appendix A.3 A discussion of the limitations of our work and potential directions for future research can be found in Appendix
A.5 A.5
5.4 SCALING AND COMPUTATIONAL ANALYSIS MORE COMPLEX TASKS
Here, we further provide results on more challenging settings with more fine-grained or complex disease spaces: the Fitzpatrick177 7k (F17K) dataset with 113 fine- aneegrained skin conditions, and the RFMiD dataset with 28 retinal conditions including comorbid combinations (diseases with less than 10 images are merged as other diseases). Note that for RFMiD, each image can be associated with multiple attribute tokens due to discase co-occurrence. We demonstrate the feasibility and necessity from two perspectives: (1) performance gains and (2) computational overhead.
To be specific, as the number of fine-grained diseases and comor- bid combinations increases, the task becomes substantially more challenging, especially for GPT-v models, which are not inher- ently strong discriminative learners. Even the medically special- ized Lingshu-7B model continues to struggle these large, fine- grained label spaces after fine-tuning. In contrast, when augmented with our AttTok strategy, its performance improves substantially, from 27% to 1% accuracy on FI7K and from 25% to 36% mAP on RFMiD, indicating that AttTok particularly beneficial in precisely the regime of more complex disease diagnosis tasks. More impor- tantly, with about 1-3% additional FLOPs introduced, the above performance improvements are obtained at nearly zero computational overhead.
Table 4: Performance on F17K and RFMID.
Models
(F1
mAP (REMID)
CLIP-B16
0.40
0.45
BioMedLLIP
0.34
0.47
PMC-CLIP
0.45
0.49
Lingsbu-7B
0.27
0.25
ARTDE
(7.4)
0..66
Impeuved.
New Flogs
2.5%
1.45
For tasks with more than 1K diseases or concepts, assuming an input resolution of (224 224), the vision encoder (VE) of (tngshht77B/Qwe2255-VL LLM computation (which remains almost identical with and without AttTok) already exceeds 900 GFLOPs solely for encoding the visual tokens. The additional cost of the ACC module with (K) attributes is approximately ((13 + 0.16K) GFLOPs. Thus, even when scaling to 1000 attributes, the computational overhead remains merely 14% additional FLOPs.
requires approximately 340 GFLOPs. The
5.5 SENSITIVITY ANALYSIS OF ATTRIBUTE KEYWORD QUALITY
Robustness across different LLMs. To assess the robustness of this process and the impact of potential keyword noise, we conduct an ablation in which three different LLMs (Qwen-3, InternLM, and GPT5) are independently used to derive the attribute book. The very small variance across LLMs (85.6%, 85.9% and 85.7%) indicates that the keyword extraction process stable and not sensitive to the specific LLM used.
6 CONCLUSION
In this work, we identified two major limitations of instruction-tuned GPTv models for medical imaging: the inadequate encoding of clinical attributes plain text tokens and the resulting inaceu- racies in vision-language alignment. To overcome these challenges, we proposed AttTok, a unified framework that introduces structured attribute tokens, attribute-centric embeddings, and alignment mechanisms tailored for clinical concepts. Comprehensive evaluations across diverse benchmarks confirm its effectiveness, demonstrating consistent accuracy gains and improved medical reasoning.
ACKNOWLEDGMENTTS
This work was partially supported by a research grant from the Research Grants Council (RGC) of Hong Kong (Project No. R6005-24); a grant from the Joint Research Scheme (JRS) jointly funded by the National Natural Science Foundation of China (NSFC) and the RGC of Hong Kong (Project No. N_HKUST654/24); additional funding from the RGC of Hong Kong (Project Nos. AoE/E-601/24-N and 16500825); and in part by a research grant from the National Natural Science Foundation of China (Grant No. 62306254).
10

---

Published as a conference paper at ICLR 2026
REFERENCES
Lean-Baptistt Alayrac, Jeff Donahue, Pauline Luc, Antoine Miech, lain Barr, Yana Hasson, Karel Lenc, Arthur Mensch, Katherine Millican, Malcolm Reynolds, et al. Flamingo: a visual language model for few-shot learning. Advances in neural information processing systems, 35:23716- 23736, 2022.
Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Sibo Song, Kai Dang. Peng Wang, Shijie Wang, Jun Tang, al. Qwen2. 5-vl technical report. arXiv preprint 2rxvv:2502.13923 2025.
Elliot Bolton, David Hall, Michihiro Yasunaga, Tony Lee, Chris Manning, and Percy Liang. Stan- ford crfm introduces pubmedgpt 2.7 b. Stanford
Artificial Intelligence, 2022.
Richard J Chen and Rahul G Krishman. Self-supervised vision transformers learn visual concepts in histopathology. arXiv preprint arXiv:2203.00585 2022.
Xi Chen, Xiao Wang, Soravit Changpinyo, Anthony Piergiovanni, Piotr Padlewski, Daniel Salz, Sehastian Goodman, Adam Grycner, Basil Mustafa, Lucas Beyer, et al. Pali: A jointly-scaled multilingual language-image model. arXiv preprint (rXXi:2209.06794 2022.
Gheorghe Comanici, Eric Bieber, Mike Schaekermann, Ice Pasupat, Noveen Sachdeva, Inderjit Dhillon, Marcel Blistein, Ori Ram, Dan Zhang, Evan Rosen, al. Gemini 2.5: Pushing the frontier with advanced reasoning, multimodality, long context, and next generation agentic capa- bilities. arXiv preprint arXiv:2507.0626/ 2025.
Wenliang Dai, Junnan Li, Dongxu Li, Anthony Tiong, Junqi Zhao, Weisheng Wang, Boyang Li, Pascale N Fung, and Steven Hoi. Instructblip: Towards general-purpose vision-language models with instruction tuning. Advances in neural information processing systems, 36:49250-49267 2023.
Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, et al. An image is worth 16x16 words: Transformers for image recognition at scale. arXiv preprint arXiv:2010.I 11929, 2020.
Danny Driess, Fei Xia. Mehdi SM Sajjadi. Corey Lynch, Aakanksha Chowdhery, Ayzaan Wahid, Jonathan Tompson, Quan Vuong, Tianhe Yu, Wenlong Huang, et al. Palm-e: An embodied mul- timodal language model. 2023.
Sedighch Eslami, Christoph Meinel, and Gerard De Melo. Pubmedelip: How much does clip benefit visual question answering in the medical domain? In Findings of the Association for Computa- tional Linguistics: EACL 2023, pp. 1151-1163, 2023.
Xuehai He, Yichen Zhang, Luntian Mou, Eric Xing, and Pengtao Xie. Pathvqa: 30000+ questions for medical visual question answering. arXiv preprint arXiv:2003. 10286, 2020.
Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, Weizhu Chen, et al. Lora: Low-rank adaptation of large language models. ICLR, 1(2):3 2022.
Shaohan Huang, Li Dong, Wenhui Wang, Yaru Hao, Saksham Singhal, Shuming Ma, Tengchao Lv, Lei Cui, Owais Khan Mohammed, Barun Patra, et al. Language is not all you need: Align- ing perception with language models. Advances in Neural Information Processing Systems, 36: 72096-72109, 2023.
Aaron Hurst, Adam Lerer, Adam Goucher, Adam Perelman, Aditya Ramesh, Aidan Clark, AJ Os- trow, Akila Welihinda, Alan Hayes, Alec Radford, et al. Gpt-4o system card. arXiv preprint arXiv:2410.21276 2024.
Songtao Jiang, Yuxi Chen, Sibo Song. Yan Zhang, Yeying Jin, Yang Feng, Jian Wu, and Zuozhu Liu. Knowing or guessing? robust medical visual question answering via joint consistency and contrastive learning. In International Conference on Medical Image Computing and Computer- Assisted Intervention, pp. 325-335. Springer, 2025.

---

Published as a conference paper at ICLR 2026
Daniel S. Kermany, Kang Zhang, and Michael H. Goldbaum. Labeled optical coherence to- mography (oct) and chest x-ray images for classification. 2018. URL
semanticscholeee org. ottp:/orrops.:
Yuxiang Lai. Jike Zhong, Ming Li, Shitian Zhao, and Xiaofeng Yang. Med-rl: Reinforce- ment learning for generalizable medical reasoning in vasion-language models. arXiv preprint arXiv:2503. 1 13939, 2025.
Jason J Lau, Soumya Gayen, Asma Ben Abacha, and Dina Demner- generated visual questions and answers about radiology images. Scientific data, 5(1):1-10, 2018.
Demner-Fushman. Fushman, dataset of clinically
Gabriel Lepetit-Aimon, Clément Playout, Marie Carole Boucher, Renaud Duval, Michael H Brent, and Farida Cheriet. Maples-dr: Messidor anatomical and pathological labels for explainable screening of diabetic retinopathy. Scientific Data, 11(1):914, 2024.
Chunyuan Li, Cliff Wong, Sheng Zhang, Naoto Usuyama, Haotian Liu, Jianwei Yang, Tristan Nau- mann, Hoifung Poon, and Jianfeng Gao. Llava-med: Training a large language-and-visin sistant for biomedicine in one day. Advances in Neural Information Processing Systems, 36: 28541-28566 2023.
Tianwei Lin, Wenqiao Zhang. Sijing Li, Yuqian Yuan, Binhe Yu, Haoyuan Li, Wanggui He, Hao Jiang. Mengze Li, Xiaohui Song, et al. Healthgpt: A medical large vision-language model for unifying comprehension and generation via heterogeneous knowledge adaptation. arXiv preprint arXiv:2502.09888, 2025.
Bo Liu, Li-Ming Zhan, Li Xu, Lin Ma, Yan Yang, and Xiao-Ming Wu. Slake: A semantically- labeled enowledge-enhanced dataset for medical visual question answering. In 2021 IEEE 18th international symposium on biomedical imaging (ISBI), pp. 1650-1654. IEEE, 2021.
Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. Visual instruction tuning. Advances in neural information processing systems, 36:34892-34916 2023a.
Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. Visual instruction tuning. Advances
in neural information processing systems, 36:34892-34916, 2023b.
Shilong Liu, Zhaoyang Zeng. Tianhe Ren, Feng Li, Hao Zhang, Jie Yang, Qing Jiang, Chunyuan Li, Jianwei Yang, Hang Su, al. Grounding dino: Marrying dino with grounded pre-training for open-set object detection. In European conference computer vision, pp. 38-55. Springer, 2024a.
Yuanxin Liu, Shicheng Li, Yi Liu, Yuxiang Wang, Shuhuai Ren, Lei Li, Sishuo Chen, Xu Sun, and Lu Hou. Tempcompass: Do video 11ms really understand videos? arXiv preprint arXiv:2403.00476, 2024b.
Rengian Luo, Liai Sun, Yingee Xia, Tao Qin. Sheng Zhang, Hoifung Poon, and Tic- Yan Liu. Biogpt: generative pre-traing -rained transformer for biomedical text generation and mining. Briefings in bioin- formatics, 23(6)bbbc4409 2022.
Laurens van der Maaten and Geoffrey Hinton. learning research, 97Nov)2557--2605 2008.
Visualizing data using t-sne. Journal of machine
Md Ashraful Alam Milton. Automated skin lesion classification using ensemble of deep neural networks in isic 2018: Skin lesion analysis towards melanoma detection challenge. arXiv preprint 11X1v:1901.10802, 2019.
Michael Moor, Qian Huang, Shirley Wu, Michihiro Yasunaga, Yash Dalmia, Jure Leskovec, Cyril Zakka, Eduardo Pontes Reis, and Pranav Rajpurkar. Med-flaming flamingo: a multimodal medical few- shot learner. In Machine Learning for Health (ML4H), pp. 553-367. PMLR, 2023.
Jiazhen Pan, Che Liu, Junde Wu, Fenglin Liu, Jayuan Zhu, Hongwei Bran Li, Chen Chen, Cheng Ouyang, and Daniel Rueckert. Medvlm-rl: Incentivizing medical reasoning capability of vision- language models (vlms) via reinforcement learning. International Conference on Medical Image Computing and Computer-Assisted Intervention, pp. 337-347. Springer, 2025.
12

---

Published as a conference paper at ICLR 2026
Zhiliang Peng, Wenhui Wang. Li Dong, Yaru Hao, Shaohan Huang, Shuming Ma, and Furu Wei. Kosmos-2: Grounding multimodal large language models to the world. arXiv preprint TrXii:2306.14824. 2023.
Alec Radford, Karthik Narasimhan, Tim Salimans, Ilya Sutskever, et al. Improving language under- standing by generative pre-training. 2018.
Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, al. Learning transferable visual models from natural language supervision. In International conference on machine learning, pp- 8748-8763. PmLR, 2021.
Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi. Haowei Zhang, Mingchuan Zhang, YK Li, Yang Wu, et al. Deepseekmath: Pushing the limits of mathemati- cal reasoning in open language models. arXiv preprint arXiv:2402.03300, 2024.
Karan Singhal, Tao Tu, Juraj Gottweis, Rory Sayres, Ellery Wulczyn, Mohamed Amin, Le Hou, Kevin Clark, Stephen Pfohl, Heather Cole-Lewis, et al. Toward expert-level medical question answering with large language models. Nature Medicine, (1(3))943-995) 2025.
Jing Tang, Hongru Xiao, Xiang Li, Wei Wang, and Zeyu Gong. Chatead: An mllm-guided frame- work for zero-shot cad drawing restoration. In ICASSP 2025-2025 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 1-5. IEEE, 2025.
Michael Tschannen, Alexey Gritsenko, Xiao Wang, Muhammad Ferjad Naeem, Ibrahim Alabdul- mohsin, Nikhil Parthasarathy, Talfan Evans, Lucas Beyer, Ye Xia, Basil Mustafa, et al. Siglip 2: Multilingual vision-language encoders with improved semantic understanding, localization, and dense features. arXiv preprint arXiv:2502. 14786, 2025.
Tao Tu, Shekoofeh Azizi, Danny Driess, Mike Schackermann, Mohamed Amin, Pi-Chuan Chang, Andrew Carroll, Charles Lau, Ryutaro Tanno, Ira Ktena, et al. Towards generalist biomedical ai. Nejm Ai, 1882300133,, 2024.
Haochun Wang, Chi Liu, Nuwa Xi, Zewen Qiang, Sendong Zhao, Bing Qin, and Ting Liu. Huatuo: Tuning Iaama model with chinese medical knowledge. arXiv preprint arXiv:2304.06977. 2023.
Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue. Anthony Moi, Pierric Cistac, Tim Rault, Rémi Louf, Morgan Funtowicz, Joe Davison, Sam Shleifer, Patrick von Platen, Clara Ma, Yacine Jemite,, Julien Plu, Canwen Xu, Teven Le Scao, Sylvain Gug- ger, Mariama Drame, Quentin Lhoest, and Alexander M. Rush. Transformers: State-of-the-art natural language processing. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations, pp. 38-45, Online, October 2020. As- sociation for Computational Linguistics. URL https: : /www. a acl web. 2020 emn de-deoss.
ogy/
Chaoyi Wu, Weixiong Lin, Xiaoman Zhang, Ya Zhang, Weidi Xie, and Yanfeng Wang. Pme-Ilama: toward building open-source language models for medicine. Journal of the American Medical Informatics Association, (1) 1833-1843 2024.
Weiwen Xu, Hou Pong Chan, Long Li, Mahani Aljunied, Ruifeng Yuan, Jianyu Wang. Cheng- hao Xiao, Guizhen Chen, Chaoqun Liu, Zhaodonghui Li, et al. Lingshu: A generalist foun- dation model for unified multimodal medical understanding and reasoning. arXiv preprint arXiv:2506.00 07044, 2025.
An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, et al. Qwen3 technical report. arXiv preprint arXiv:2505.09388 2025.
Jiancheng Yang, Rui Shi, Donglai Wei, Zequan Liu, Lin Zhao, Bilian Ke, Hanspeter Pfister, and Bingbing Ni. Medmnist v2-a large-scale lightweight benchmark for and 3d biomedical image classification. Scientific Data, 10(1):41, 2023.
13

---

Published as a conference paper at ICLR 2026
Yaowei Zheng, Richong Zhang, Junhao Zhang, Yanhan Ye, Zheyan Luo, Zhangchi Feng, and Yongqiang Ma. Llamafactory: Unified efficient ine-tuning of 100+ language models. In Pro- ceedings of the 62nd Anmual Meeting of the Association for Computational Linguistics (Volume
3: System Demonstrations), Bangkok, Thailand, 2024. Association for Computational Linguis- 13372
tics.
Jinguo Zhu, Weiyun Wang, Zhe Chen, Zhaoyang Liu, Shenglong Ye, Lixin Gu, Hao Tian, Yuchen Duan, Weijie Su, Jie Shao, et al. Internv1:: Exploring advanced training and test-time recipes for open-source multimodal models. arXiv preprint arXiv:2504. 10479, 2025.
14

---

Published as a conference paper at ICLR 2026
A APPENDIX
A.1 EXPERIMENTAL DETAILS
For instruction tuning, we adopt LoRA (Hu et al, 2022) as the default adaptation technique with rank 16. The model is optimized using AdamW with learning rate of x 10-4, weight decay of 1 x 10 a cosine learning rate schedule, and a warmup phase over the first 10% of training iterations. We train with batch size of 32 on NVIDIA A800 GPUs for epochs. The weight of the ACM loss is initially set to and gradually decays to 0 during training. The in is set as 1.
By default, both the vision tower and the LLM tower are frozen, and LoRA is only applied to linear layers. To preserve the original encoding space of GPT-v, we keep the embedding and classifier weights for all tokens except the newly introduced attribute tokens unchanged by zeroing their gradients. This is implemented via PyTorch hook functions
The ablation study is conducted on Lingshu-7B by default, and the performance is evaluated on open-ended diagnosis and classification fasks.
A.2 WORKFLOWS FOR DATA PROCESSING AND ATTRIBUTE DEFINITION
Dataset processing. Given the complexity of GPTv training pipelines (especially built on the highly integrated Transformers (Wolf et al. 2020) and Llama-Factonn (Zheng et al.) (024) libraries), it is impractical to perform online image augmentation as commonly used in conventional training for CLIP-based discriminative models (Radford et al. 2021). To address this, we adopt offline classification tasks. Specifically, we apply dass-balanced sampling together with random cropping and horizontal flipping to generate aug- mented samples. Each augmented instance is stored as a new image, ensuring both diversity and balance in the training set while remaining compatible with the discriminative models.
augmentation strategy tailored for disease diagnosis/Cassiifcaatoo
Attribute definition and assignment for VQA datasets. Given a VQA dataset, we first input each sample's question, answer, and auxiliary information into Qwen-3 (Yang et al.] 2025). and instruct it to reformulate the QA pair into
freice
Carrent Rlyeent X-Roy, Lung, Abnamal
Acd foad Ditt:
bost wn Leeg
declarative sentence and extract key concepts as candidate keywords. After aggregating all can- didates, we filter out non-discriminative terms (e.g., spatial positions, shapes, colors, sizes), redundant concepts, and low-frequency words. In other word, we only focus on words present- ing discriminative concepts, i.e., modality, dis- case status inormal/abnormal, disease names), anatomical site. Given that we explicitly define the clinical concepts of interest, the scope of extracted keywords is constrained to dozen categories. These keywords are restricted to conclusive terms, ensuring that the assigned keywords focus on diagnostically meaningful semantic.
Chucked leyword
Homry ndl-
Figure 4: The workflow generate the attribute words for samples from VQA datasets.
The resulting refined keyword set is then used as a prompt, and Qwen-3 is further applied to assign appropriate keywords back to each sample. For Vat since the dataset is exclusively composed of pathological images, the modality keyword attribute is omitted in this process.
A.3 ADDITIONAL EXPERIMENTAL ANALYSIS
The importance of attribute-centric alignment. The Figure presents the visualization of visual and attribute tokens across multiple datasets, demonstrating that the proposed ArrTok substamtially improves the discriminabliity of learned feature representations. We further examine two comple- mentary settings: (1) image generation using the atrTok-aggmented Lingshu-7B model, and (2)
regisstr..
hook. html
15

---

Published as a conference paper at ICLR 2026
(b) Fundus
(d) X-Ray
(6) Path.
Figure 5:
The feature visualization for five modalities, tuned via AtrTok. Visual tokens (a with different colors) and attribute tokens (* with different colors) are projected into 2D space via t-
SNE (Maaten & Hinton 2008). AttTok achieves clear attribute-centric molliittddlltyy alignment.
feature retrieval with attribute tokens employed as classifier weights. As summarized in Table 5 attribute tokens provide a more accurate characterization of class-centered semanties. Leveraging this property, AttTok can be utilized "teacher" module to distill discriminative capability into "student" GPTv models, thereby facilitating more robust multimodal alignment.
Cls Acc.
Derna
Fundus
OCT
X-Ray
Path.
Retrieval via AuTTo
82.5
70.6
90.2
95.8
93.3
GPTv with AutTok
76.1
68.4
79.0
92.4
87.2
Table Performance comparison between direct retrieval using attribute tokens as classifier weights and generative results from the fine-tuned GPTv model.
Model
MEL(111)
NV (6705)
BCC (514)
AKIEC (327)
BKL (1099)
DF (115)
VASC (142)
w/o AutTok
35.8
82.2
69.2
25.4
25.4
11.8
48.7
AtTTok $3.3
70.4
35.4
35.4
31.4
$2.2
Table Lese classification performance (F1 (Fl-score) on the ISIC-2018 (Milton 2019 dataset. The numbers in parentheses denote the number of original training samples per category.
Towards more discriminative diagnosii/classification tasks. Here, we extend AttTok to tackle more complex and challenging discriminative diagnosis and classification tasks, imbalanced data problems which are commonly encountered in real-world clinical practice. As described in the dataset processing stage, we employ offline augmentation strategies to balance the training data. In this analysis, however, we do not use pre-processing in order to expose the inherent imbalance of the skin lesion classification dataset, ISIC-2018 (Milton, 2019). We report the class-wiss wise FI as Table (6. Except for the NV category, which dominates the dataset with overwhelming number of training samples, most classes suffer from severe data scarcity. Under this highly imbalanced setting, the discriminative features introduced by AuTTok substantially improve the Fl scores of all lample categories. This observation highlights the potential of AttTok for handling the imbalanced data distributions that are commonly encountered in medical datasets.
Here, we provide the abla- tion study on the scale and X, testing on open-ended DR grad- ing tasks. As shown in Table ? larger leads to over-fit on extra knowledge from attribute books, the empirical suitable values are in the range of 10.1, 0.5]. Similarly, larger À will hinder the learn- ing of NTP and smaller one will shrink the impact discriminative feature learning. A weight decay is needed.
Ablation on key
7-00
7-00
77
60.3
60.1
58.6
56.6
X--05"
X= E.6 " 2.0
58.2
57.5
60.3
59.8
Table 7: Ablation on y and A.
A.4 VISUALIZATION ANALYSIS ON ACC AND ACM
To illustrate the effectiveness of the proposed ACC module, a visu- alization analysis was conducted in F which demonstrates the class activation maps for fea-
16

---

Published as a conference paper at ICLR 2026
Original Image
wio ACC
w/ ACC
Original Image
wio ACC
ACC
Figure 6:
Visualization of ACC module's effectiveness. Left: ACC directs attention to correct visual cues (e.g.. hemorrhage, exudates). Right: ACC mitigates erroneous background focus, despite not fully capturing foreground cues.
from visual encoder trained DR grading tasks. This module utilizes prototype embeddings from attribute books to guide the visual encoder toward learning discriminative features. The left side of the visualization exhibits examples where the ACC module successfully directs the model's attention to appropriate visual cues, such as abnormal signals like hemorrhage and exudates. The right side demonstrates three cases where the ACC module effectively clears erroneous focus on the background, despite not yet achieving full concentration on correct foreground cues.
Furthermore, we analyze the feature correlations of the model trained without AttTok. When only textual features used as guidance together with purely causal information flow strategy, the visual and textual signals for each pair of diseases become highly correlated and indistinguish- able. In contrast, training with AttTok explicitly guides the model through attribute tokens, which substantially enhances feature discriminability. At the classification level, this manifests as clear reduction in confusion between different grading categories. To test class-wise metrics, sample 100 testing images for each grading via data augmentation strategies.
A.5 DISCUSSION OF LIMITATIONS AND FUTURE EXPLORATIONS
Text fluency, interpretability, and controllability. As illustrated in Figure 1 and and described in Section 4.1, all attribute tokens (multiple for VQA and multi-label classification tasks) are in- serted in the front of the textual input sequence. In other words, the attribute tokens form short, structured prefix that naturally transitions into the subsequent free-form text, without disrupting lin- guistic fluency changing the conversational style. This design also makes the behavior of the model controllable: by learning attribute tokens firstly, we can explicitly steer the model's focus before generates its response.
From an interpretability perspective, this setup leverages the causal information flow GPT-v: tokens that appear earlier in the sequence causally influence the hidden states and attention distribu- tions that determine all later tokens. Consequently, the attribute tokens act as explicit, discriminative guidance signals for the downstream text response. This core contribution of our work.
Compositional attribute management. Because one image can correspond to multiple questions, single image may be associated with multiple attribute tokens. We treat these in multi-label man- ner, allowing attributes to be combined orthogonally (e.g., organs, findings, and modalities together), which enables scaling more attributes without conflict while preserving semantic completeness and flexible expression. This design ensures that our attribute space is both scalable and composi- tional, aligning with the multi-faceted nature of medical images.
17

---

Published as a conference paper at ICLR 2026
Figure 7: From left to right: The feature similarities for average visual features and textual features among five DR grading; the feature similarities for average visual features and attribute features among five DR grading; the confusion matrices for models trained w/o AttTok; the confusion ma- trices for models trained w/ AttTok.
Limitations. Although integrating discriminative attributes into GPTv models has been shown enhance performance in diagnostic, classification, and VQA tasks, where short, conclusive an- swers are expected, the extension and validation of AtrTok in long-text generation and fine-graing -gaiinddddd chain-of-thougle reasoning tasks remain largely unexplored. Addressing these challenges will likely require not only the construction of AttTok but also more sophisticated attribute relations and even knowledge graph-based modeling, enabling AttTok to better handle longer and more complex tex- tual scenarios.
Future work. this study, we demonstrated the value of discriminative attributes for medical image diagnosis, thereby opening new research exploration on hybrid discriminative and generative representations on GPTv models.
Specifically, our findings point the potential of GPTv models in broader range of discrim- inative tasks, including (but not limited to) multi-label classification, few-shot learning, and long- tail problems. In parallel, they highlight promising opportunitics explore AttTok-based unified frameworks in segmentation, detection, and other discriminative low-level tasks.
Moreover, following the standard evolution route of GPTv models (Hurst al., 2024 Bai et al, 2025 Xu et al. (2025), the stage after instruction tuning typically involves reinforcement learn- driven preference optimization and logical reasoning. Within this pathway, the AttTok frame- work presents promising opportunities for integration with online preference alignment strategies and the design of AttTok-informed reward models such as GRPO (Shao et al.) (024)), paving the way for deeper explorations in task alignment and reasoning optimizatiin. A feasible direction is to explore sentence-lews modeling, extending AttTok from token-level discriminative representations to long-form representations. For example, cassonig-sspecffi tokens could be introduced to guide a clearer chain-of of-huuh,, enabling GPTv models to generate more structured and interpretable reasoning paths.
B LLM USAGE
Large Language Models (LLMs) were employed exclusively assist with manuscript preparation. Their role was limited to refining language and improving readability. Specifically, the LLM contributed tasks such as rephrasing sentences, correcting grammar, and smoothing the overall flow of the text.
The LLM played part in formulating research questions, designing methodologies, conducting experiments, or analyzing results. All scientific ideas, experimental designs, and analyses were conceived and executed entirely by the authors. The authors accept full responsibility for the manuscript's content, including any text edited with LLM assistance. Care was taken to ensure that LLM-ddrived text complies with ethical standards and does not introduce plagiarism or scientific misconduct.
18


Reviews:
[{"review_id": "Fk0ZU8wKBF", "reviewer": "Reviewer_UEQz", "summary": "The paper proposes AttTok, a framework that introduces attribute tokens (e.g., 
) and a multi-modal attribute dictionary (embedding book) containing textual keywords, learned embeddings, and EMA-updated visual prototypes.\nTwo main modules are designed:\n\n* Attribute-Centered Cross-Attention (ACC): Allows visual features to cross-attend to attribute embeddings, breaking the traditional unidirectional “vision → text” flow in generative vision-language (GPT-v) models.\n\n* Attribute-Centered Matching (ACM): A triplet-style objective aligning visual, textual, and attribute embeddings to improve intra-class consistency and inter-class separability.", "questions": "* Can new attributes (e.g., new diseases or grading levels) be added incrementally to the dictionary without retraining the entire model? How is representation drift prevented? \n\n* Does AttTok handle combinations (e.g., “chest X-ray + mild infiltration + suspected heart failure”) via atomic tokens or compositional embeddings? How is combinatorial explosion managed?\n\n* Since VQA accuracy is determined via LLM-based semantic matching, has the team validated with human or rule-based evaluation? How consistent are scores across evaluators?\n\n* How does AttTok behave under domain shifts, image noise, or out-of-distribution (OOD) cases? Can attribute tokens support uncertainty estimation or refusal to answer?", "limitations": "", "rating": 4, "confidence": 4, "soundness": 3, "presentation": 2, "contribution": 2, "strengths": "* The attribute-token and dictionary act as explicit anchors; ACC and ACM target information flow and alignment respectively. The design is intuitive and easy to integrate into existing GPT-v pipelines (e.g., LoRA fine-tuning with frozen backbones).\n\n* Demonstrates stable improvements across diverse medical modalities (fundus, dermoscopy, OCT, X-ray, pathology) and VQA tasks, showing strong generalizability and robustness even on advanced medical VL backbones.", "weaknesses": "* As the attribute space grows (rare diseases, fine-grained grades, comorbid combinations), the dictionary (~3K entries) and ACC computation may become computationally expensive. The paper lacks analysis of complexity, efficiency, or EMA prototype stability as K increases.\n\n* For VQA, attribute keywords are extracted using Qwen-3. The paper does not assess label noise, propagation effects, or how mis-specified keywords may misguide alignment.\n\n* Results mainly report accuracy without confidence intervals, AUROC, calibration, or significance testing. The use of LLM-based semantic scoring for VQA may introduce bias.\n\n* It remains unclear how introducing attribute tokens affects text fluency, interpretability, or controllability, especially when encountering unseen or composite attributes.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "iXkrDFC0Bs", "reviewer": "Reviewer_AcFf", "summary": "This paper addresses the failure of generative vision-language (GPTv) models in medical imaging, where encoding clinically distinct attributes (e.g., \"mild\" vs. \"severe\" disease) as similar plain text causes semantic ambiguity and poor vision-language alignment. The proposed solution, AttTok, introduces a set of pre-defined, special \"attribute tokens\" to uniquely represent these clinical concepts. This framework uses these tokens as anchors in two new components: an Attribute-centric Cross Attention (ACC) adapter to enrich the visual encoder with attribute knowledge, and an Attribute-centric Matching (ACM) loss to enforce robust multi-modal alignment via contrastive learning. Trained jointly with the standard next-token prediction loss, experiments across five medical classification and three VQA datasets demonstrate that AttTok substantially improves discriminative accuracy over baselines.", "questions": "1. Can you please address the apparent contradiction in Table 1? For the Lingshu-7B model, your method improves the average open-ended accuracy (67.2% $\\rightarrow$ 80.6%) but seems to harm the closed-ended accuracy (76.9% $\\rightarrow$ 71.6%). This contradicts the claim of \"consistent improvements\". Is this a typo in the table, or does AttTok introduce a trade-off between closed-ended and open-ended performance for this specific model?\n2. The method relies on pre-defining $K$ attribute tokens. How do you see this approach scaling to a much larger and more complex attribute space (e.g., thousands of findings, locations, and modifiers)? Would the ACC module and ACM loss remain effective and computationally tractable as $K$ increases significantly?\n3. The process for defining VQA attributes uses a GPT model to extract and assign keywords . How robust is this? Did you manually verify the quality of these extracted attributes? Given the \"coarse\" nature of these attributes, do you believe the modest gains on VQA are a limitation of the AttTok method itself or a result of noisy/imprecise attribute definitions?\n4. Your method adds a strong discriminative objective (ACM loss) to the generative NTP loss. Did you perform any qualitative or quantitative analysis on whether this discriminative pressure harms the model's generative fluency, coherence, or its ability to generate nuanced, long-form explanations?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The paper clearly identifies and targets a core weakness of medical GPTv models: the poor discriminative ability of plain-text encodings for clinically distinct attributes.\n2. The proposed solution is well-designed. The two main components, ACC and ACM, directly target the two identified weaknesses (information bottleneck and misalignment). The ACC adapter is an effective mechanism to inject top-down attribute knowledge into the visual encoder . The ACM loss is a well-founded contrastive objective to enforce multi-modal alignment\n3. The evaluation is moderately thorough. The authors test on varied datasets covering diverse medical modalities. \n4. The analysis in Table 5, showing that AttTok improves F1-scores for low-sample categories on an imbalanced dataset, is a strong point.", "weaknesses": "1. This is the most significant weakness. In Table 1, the average results for the Lingshu-7B + Ours model show a decrease in closed-ended accuracy (from 76.9% down to 71.6%) while improving open-ended accuracy (from 67.2% up to 80.6%). This directly contradicts the text, which claims \"consistent and significant gains\". This discrepancy is also seen in individual results (e.g., Lingshu-7B on Derma, where 'open' drops, and on Fundus, where 'close' drops). This confusion undermines the central claims and must be clarified.\n2. The entire framework hinges on a set of \"pre-defined special tokens\". This raises critical questions about scalability. The method for defining attributes for VQA (using a GPT to extract keywords) seems ad-hoc and potentially brittle; the paper itself calls these attributes \"relatively coarse\". It is unclear how this approach would scale to thousands of fine-grained attributes or complex attribute combinations. The attribute books $\\mathcal{B}$ would become very large, potentially making the ACC cross-attention and ACM matching computationally expensive.\n3. As the authors acknowledge, the performance improvements on the three VQA benchmarks are modest (e.g., +0.8% average gain for Lingshu-7B). While the provided explanation (the baseline is already heavily pretrained) is reasonable, it limits the demonstrated impact of AttTok on tasks requiring more complex reasoning beyond classification.\n4. The paper focuses on tasks with short, conclusive answers (classification and VQA). The authors concede in Appendix A.4 that the method's utility for long-text generation or chain-of-thought reasoning is \"largely unexplored\". It is unknown if adding the strong discriminative ACM loss harms the generative fluency, coherence, or nuance of the base LLM.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "UiqC7XdOnJ", "reviewer": "Reviewer_UJRS", "summary": "The paper proposes AttTok, a unified framework to enhance medical GPTv models through learnable attribute tokens and embedding books that explicitly encode clinical concepts. It introduces an Attribute-Centric Cross Attention (ACC) adapter and Attribute-Centric Matching (ACM) loss to improve visual–text alignment, achieving consistent gains across multiple medical VQA benchmarks.", "questions": "A few questions below, details refer to weakness.\n1) How does AttTok differ conceptually and technically from prototype-based learning methods?\n2) How are attribute tokens and embedding books defined—disease-specific or dataset-dependent—and what effort is required to construct them?\n3) How sensitive is the model to attribute definition errors or GPT-generated keyword noise?\n4) Can the authors provide ablation or scalability analysis on key hyperparameters (λ, γ) and large attribute sets?", "limitations": "", "rating": 4, "confidence": 3, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "1) This manuscript clearly addresses the semantic collapse problem in medical GPTv models, motivation is clear.\n2) The methodologically seems sound, with explicit formulation of attribute tokens, ACM loss, and prototype stabilization.\n3) The end-to-end integration supporting both discriminative and generative paradigms.\n4) Empirical evidence across diverse modalities and clear ablation validation supports the claims.", "weaknesses": "1) The approach builds on existing prototype-based and token-alignment ideas without substantial theoretical innovation, which limits its technical novelty \n2) The method resembles prototype-guided learning. The paper should clarify how the attribute-token design and embedding book differ from prototype representations.\n3) The paper lacks details discussion on the effort, criteria, and consistency in defining attributes, e.g., whether they are disease-specific or dataset-dependent. Furthermore, attribute tokens may encode explicit cues, inflating results.\n4) Missing comparisons / discussions of the difference with existing medical multimodal models, such as BioMedCLIP and MedCLIP.\n5) ACM loss lacks formal analysis or guarantees on improved alignment.\n6) Evaluation is limited to classification and closed-form VQA, may perform additional tests on open-ended or noisy data.\n7) Minor - next-token prediction paradigm (Radford et al.), lack of publication year.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "Otqx6alUxu", "reviewer": "Reviewer_ZHpa", "summary": "This study focuses on enhancing the fine-grained attribute discrimination capabilities of pre-trained GPTv models through \"instruction tuning\" (i.e., post-training). It addresses a limitation where existing GPTv models struggle with semantically distinct, fine-grained medical concepts due to their encoding as plain text tokens, leading to diminished discriminative power and misaligned vision-language embeddings. To overcome this, the authors propose **AttTok**, a framework that introduces specialized attribute tokens and associated attribute-centric embedding books for unique clinical attribute encoding. AttTok further integrates ACC adapter to infuse \"discriminative attribute knowledge\" into the visual encoder, and ACM loss to enforce robust \"multi-modal alignment\" centered on these attribute tokens. Extensive experiments across five medical classification benchmarks and three VQA datasets consistently demonstrate the effectiveness of AttTok. However, the analysis does not sufficiently clarify whether the observed performance gains genuinely stem from its attribute-centric design.", "questions": "N/A", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "AttTok addresses the practical limitation that current GPTv models struggle with fine-grained medical attribute recognition, which is an important component of clinical reasoning. \n\n- The proposed attribute-centric post-training strategy offers an efficient way to improve attribute sensitivity without modifying the base architecture. \n\n- The combination of attribute tokens, embedding tables, an ACC adapter, and ACM loss is well motivated and leads to consistent performance gains across multiple medical benchmarks.", "weaknesses": "1. Ambiguity in Baseline Configuration: It is not clearly stated whether the \"Instruction-Tuned GPTv Models\" baselines (e.g., Lingshu-7B (r=16)) are trained on the exact same datasets as AttTok-enhanced models, but without AttTok’s attribute modeling components (attribute tokens, ACC adapter, ACM loss). Without explicit confirmation, one cannot fully exclude the possibility that performance gains originate from differences in the post-training setup rather than AttTok’s specific contributions.\n\n2. Lack of Dedicated \"Fine-grained Attribute Discrimination\" Evaluation: Although AttTok aims to capture subtle inter-attribute distinctions, the paper does not provide a quantitative evaluation that directly assesses fine-grained discrimination capabilities. More detailed analyses such as confusion matrices or direct comparisons of highly similar clinical attributes (e.g., mild DR vs. severe DR) would help verify that AttTok truly improves sensitivity to fine-grained variations.\n\n3. Limited Qualitative Evidence and Visual Insights: The paper lacks qualitative demonstrations that illustrate how AttTok influences attribute-aware perception within the visual encoder. Visualizations such as heatmap examples showing shifts in attention guided by attribute tokens could provide more interpretability. Case studies highlighting where baseline models fail but AttTok correctly identifies fine-grained findings would further reinforce the method’s clinical relevance.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)