FLARE: Fully Integration of Vision-Language Representations for Deep Cross-Modal Understanding
Authors: Zheng Liu
Mengjie Liu
Jingzhou Chen
Jingwei Xu
Bin CUI
Conghui He
Wentao Zhang
Venue: ICLR 2026 Poster (2026)
Keywords: Vision Language Model
Cross-Modal Alignment and Integration
Text-Guided Vision Encoding
Context-Aware Decoding
Dual-Semantic Mapping Loss
Text-Driven VQA Synthesis
Primary area: applications to computer vision, audio, language, and other modalities

Abstract: We introduce FLARE, a family of vision language models (VLMs) with a fully vision-language alignment and integration paradigm. Unlike existing approaches that rely on single MLP projectors for modality alignment and defer cross-modal interaction to LLM decoding, FLARE achieves deep, dynamic integration throughout the pipeline. Our key contributions include: (1) Text-Guided Vision Encoding that incorporates textual information during vision encoding to achieve pixel-level alignment; (2) Context-Aware Alignment Decoding that aggregates visual features conditioned on textual context during decoding for query-level integration; (3) Dual-Semantic Mapping Loss to supervise feature mapping from both modalities and enable modality-level bridging; and (4) Text-Driven VQA Synthesis that leverages high-quality text to generate VQA pairs and synthesize corresponding images, enabling data-level optimization. We train FLARE at 3B and 8B scales under both fixed and dynamic resolution settings, demonstrating that our full-modality alignment significantly outperforms existing methods while maintaining strong generalizability. FLARE 3B surpasses Cambrian-1 8B and Florence-VL 8B using only 630 vision tokens. Ablation studies reveal that FLARE achieves superior performance over existing methods with minimal computational cost. Even without dynamic resolution, FLARE outperforms LLaVA-NeXT, validating the effectiveness of our approach.

Published as a conference paper at ICLR 2026
FLARE: FULLY INTEGRATION OF VISION-LANGUAGE REPRESENTATIONS FOR DEEP CROSS-MODAL UN- DERSTANDING
Zheng Liu Mengjie ? Jingzhou Chen 12, Jingwei Xu², Bin Cui', Conghui He?,
Wentao Zhang 1.3.4,
Peking University, SShanghai Al Laboratory, 'Zhongguaneun Academy, 'Beijing Key Laboratory of Data Intelligence and Security (Peking University), Nanjing University
ABSTRACT
We introduce FLARE, a family of vision language models (VLMs) with a fully vision-language alignment and integration paradigm. Unlike existing approaches that rely on single MLP projectors for modality alignment and defer cross- modal interaction to LLM decoding. FLARE achieves deep, dynamic integration throughout the pipeline. Our key contributions include: (1) Text-Guided Vision Encoding that incorporates textual information during vision encoding to achieve pixel-level alignment; (2) Context-Awaid Alignment Decoding that aggregates visual features conditioned on textual context during decoding for query-level in- tegration; (3) Dual-Semantic Mapping Loss supervise feature mapping from both modalities and enable modality-level bridging; and (4) Text-Driven VQA Synthesis that leverages high-quality text generate VQA pairs and synthe- size corresponding images, enabling data-level optimization. We train FLARE at 3B and 8B scales under both fixed and dynamic resolution settings, demonstrat- ing that our full-modality alignment significantly outperforms existing methods while maintaining strong generalizability. FLARE 3B surpasses Cambrian-1 8B and Florence-V VI. 8B using only 630 vision tokens. Ablation studies reveal that FLARE achieves superior performance over existing methods with minimal com- putational cost. Even without dynamic resolution, FLARE outperforms LLaVA- NeXT, validating the effectiveness our approach.
INTRODUCTION
passive process like camera capturing reality but rather an active, interpretative mechanism shaped by factors such as language and envi- ronment. In (Gary] 2012), the authors demonstrated that hearing the name of target object before searching for it significantly improved the speed and accuracy of object detection. This underscores the deep integration of vision and language, suggesting that linguistic input facilitating more effec- tive visual processing by directing attention and helping the brain prioritize relevant features.
Research has shown that human visual perception
Despite the fundamental role of vision-language interaction human perception, current VLMs fail to capture this relationship. Traditional models (Liu et al.) 2023 Chen al. 2023) process vision features independently via encoders and fuse them with text during decoding. This approach results
in lack of centralized encoding of vision features, leading to limited useful information available for interaction. Recent advancements (Liu et al.] 2024a] Bai et al. [025 Li al. 2024) enrich visual representations through dynamic resolution multiple encoders. However, these improvements fo- cus on enhancing visual encoding itself, overlooking the deeper, bidirectional integration necessary for effective vision-language fusion. Our alignment visualization
Figure [ further confirms this issue: LLaVA (Liu et al.) (023) and LLaVA-NeXT (Liu et al., 2024a), lacking semantic alignment across modalities, fail achieve proper feature mapping after the projector. At the decoding stage, both show weak attention, with LLaVA-NeXTT excessive vision tokens further dispersing focus.
Corresponding Author

---

Published as a conference paper at ICLR 2026
LLAVA
Whate floer
Irput
PLARE
Projector
dlge
medalttylleeel dlge
quary-loval dlge
Figure 1: We visualize modality alignment by computing attention maps between image and query at pixel/query levels, and cosine similarity in the LLM space at modality level. The results indicate that our model achieves consistent and progressively enhanced mossl alignment and integration.
achieving deep integration between vision and language throughout the pipeline. Previous studies have explored varied integration strategies, such as text-based QFormer integration (Dai et al. 2023), and entity-enhanced modality alignment (Yin et al. (024). Nevertheless, these models lack a comprehensive algorithm for aligning and integrating modalities. A key unresolved issue is embedding misalignment, where inherent discrepancies between vision and text embeddings hinder seamless integration. Moreover, current models lack high-quality training data designed to guide the alignment and integration process.
This challenge serves as the core motivation for our work:
In this paper, we push the boundaries of vision-language fusion by proposing sophisticated algo- rithm that realizes a deeper, more interactive integration of visual and textual. Our approach consis- tently integrates textual guidance throughout the visual encoding. We also introduce self-supervised guidance modality space mapping and dynamically aggregate vision features based on textual context during decoding. Additionally, we introduce a new data synthesis method that generates QA pairs from text, guiding the model toward more efficient feature integration. Our method enables more cognitively inspired multimodal learning paradigm. Overall, our contributions are as follows:
Text-Guided Vision Encoding: We project text embeddings into visual space and jointly perform attention with visual representations to enhance pixel-level integration.
Context-Aware Alignment Decoding: We introduce mentext-award latent tokens and se- mantic exchange layer that aggregate visual features based on textual context during LLM decoding, enabling fine-grained, query-level alignment and integration.
Dual-Semantic Mapping Loss: We propose bidirectional reconstruction loss, mitigating semantic discrepancies between modalities, achieving modality-leve alignment.
Text-Driven VQA Synthesis: We develop a new method that prioritizes constructing high- quality QA pairs while leveraging generative models to produce images. We synthesize a large-scale, diverse QA dataset, enabling data-level optimization.
Building on our exploration, we present new family of VLMs that achieve leading performance across diverse benchmarks. To foster further research, we will release our code, data, and models.
PRELIMINARIES AND RELATED WORK
2024a) are de- Vision Language Models. Vision Language Models (VLMs) (Liu al. 2023
signed to process information from multiple modalities, primarily combining visual and textual data. Typically, images are divided into patches and processed by vision encoders to obtain visual representations, which are then projected into the LLM space and fused with text tokens during de- coding. Recent advances (Li et al. 2024 Bai et al. 2025 Tong et al. 2024a Shi et al. 2025) have explored dynamic resolution and multiple vision encoders, enhancing the precision of visual encod- ing. However. these methods primarily emphasize visual encoding quality, neglecting deeper and more interactive vision vision-language dynamics, thus hindering the full potential of multimodal learning.
2

---

Published as a conference paper at ICLR 2026
Input
Space
Encoding
Vision Map
Eroaded Emage R?"
joyce
Encoder
Sam my Loes froz
MLPv2t
concat
Vision Encoder
Image
TartiWisinn Ig
Brooded Kg"
Rebuld Tragg E?
QU: What the title of the book next to the cat? Q2: What are the specific colors on the
Space
MLP12+
MLP-er
Text Mop
Rebali:: Query
MLP12v
Text Brheet
Sirilantt Losil
Query
LLM
Figure 2: Illustration of Text-Guided Vision Encoding and Dual-Semantic Mapping Loss. The query is projected into vision space and processed jointly with image. The extracted features are then mapped into text space for decoding. To enhance mapping reliability, we reconstruct text and image tokens and enforce similarity with raw tokens, promoting structural alignment across modalities.
Modality Alignment and Integration. Recent studies have explored different alignment strategies. (Dai et al. 2023) incorporates text guidance by embedding textual tokens into QFommer, enabling language-driven visual feature projection. (Yin et al. 2024) introduces contrastive loss in the projector during pretraining, strengthening the association between visual features and entity repre- sentations. More recently, (Chen et al.) 2024b) proposed a generative vision encoder that leverages multiple textual prompts to integrate diverse feature types, achieving state-of-the-ar performance. Nonetheless, a key challenge persists: the inherent disparity between visual and textual feature spaces makes robust alignment difficult, often resulting in inconsistent multimodal representations.
Instruction Tuning Data. Existing works (Li et al., 2024 Tong et al. 2024a Shi et al. 2025) rely on visual question answering (VQA) benchmarks to construct training recipes. Other efforts (Wu et al. 2023; Chen et al.) 2024a) leverage GPT-4V to generate high-quality QA pairs from im- ages. While these datasets improve multimodal understanding, they remain vision-centrie, pairs are grounded mainly in image content. This reliance limits their capacity to support complex, instruction tasks across diverse domains. A promising alternative (Liu al. 2025) seeks to overcome this limitation by first synthesizing high-quality captions and then generating corre- sponding images via diffusion models. However, this approach is still confined to pretrain, and the dataset scale remains small, restricting its effectiveness for large-scale multimodal training.
QA
3 METHOD
3.1 TEXT-GUIDED VISION ENCODING
As first step, fundamentally revise the vision encoding paradigm by incorporating textual infor- mation directly into the vision encoder to enhance multimodal integration. We design specialized fusion strategy that balances semantic refinement with visual detail preservation.
As shown in Figure 2, embeddings T, from encoder and LLM,, and map T, into vision space obtain Text2Vision V,-
given an image and query Q, we extract visual embeddings V; and textual
Vi VisionEmbed(I)), Tg LLMEmbed(()) V, MLP:2.
The mapped Visi2Vision V,, and visual embeddings V; are then jointly processed in encoder, en- abling mutual refinement of multimodal features. Each layer updates these features as follows:
(V. V*) Encoderl LarooderLLyye((V" -V.t1 v*k-1), 1,..,., N.
where N represents the total number of encoder layers, with Vo = V, and V 9 V,,
Since lower-layer visual features lack semantic information, we mask disual-to-texttr attention in the first half of the encoder layers, allowing early layers focus on learning visual representations.
3

---

Published as a conference paper at ICLR 2026
Vision2Text Tokens 7r
Semantic Exchange Layer
Decoding Layer
Semantic Exchange Layer
Cross Attention
Decoding Layer
Latera Takens 7
concaT
capy
Vidan2Tont Takers
Lanent Tokam! Textual Tokane To Latentt Tokane T, Textual Toker
Contect-AAee QUIT
didden Stene R,
Illustration of Context-Aware Alignment Decoding, For query textual tokens T2( high- lighted in yellow), we prepend set of context-aware latent tokens Ty (highlighted in green). Addi- tional semantic exchange layers introduced between decoding layers, where Vision2Text image tokens T; interact with both latent tokens and textual tokens at a query-level granularity.
Figure 3:
After encoding, we average features in shallow(fist-hhlf) and deep(second-hall) layers to capture both coarse, predominantly vision-only information and fine- ane-grained, richly text-enhanced details:
N
(V Vi) N Σ (V. V.), (V.d V =
2
Σ k=
(V.
V*A.
(3)
N
Instead of using only final-layer features as in prior work, concatenate both vision-only and text-enhanced features, providing a balance between unimodal fidelity and cross-modal enrichment:
V Concat(V,", V.d: dimm=hhanell) V Concat(V,7 Vid dim=channel).
(4)
Finally, visual features V are mapped into LLM space, yielding Vision2Text image tokens T, Textual embeddings T, serve as textual tokens, interacting with T; through causal mask in decoder.
T: MLP (V²))
(5)
3.2 CONTEXT-AWARE ALIGNMENT DECODING
Conventional approaches defer cross-modality interaction to LLM decoding, where the causal mask restricts such interaction to unidirectional flow. To overcome this, we propose novel token orga- nization strategy, as shown in Figure 3 For each query, we prepend context-aware latent tokens Tı, which serve as intermediaries for cross-modal integration. We further introduce semantic exchange layers between LLM decoding layers, where latent tokens Tr, refine visual features under textual context, enabling bidirectional and fine-grained cross-modal interaction during decoding.
We employ windowed attention in semantic exchange layers to achieve efficient modality interac- tion. Let T2, x denote the latent tokens, where I is the spatial resolution and D the embed- ding dimension. The Vision2Text image tokens are denoted as T; E mx serving as keys and values. Window sizes m/l, n// restrict attention computation within local regions.
During decoding, given textual query Q, we first identify the position P corresponding to the end of the query textual tokens. The hidden state Hp I x31 which aggregates all preceding context, is extracted and concatenated with each latent token Tr.r, c] to form context-aware queries:
TQ rr c] (L rr, Tr. [r. c))) IRD
where r, E 1,. 1} are the row and column indices, with being its spatial resolution.
Next, use Vision2Text image tokens T; keys and values. Latent token Trlr, c) is updated
Ty Ty(r , c] = softmax Q(r, c) c/7
Vlr ER!x))
VD
4

---

Published as a conference paper at ICLR 2026
All GALLERY
Count
Color
Artwork
Celebrity
Landmark
Text
ALUU
The image showortes a modern and vibrant living room with striking color contrast. At the center, there is bright yellow two seater sofa with classic rolled arms plush cushions, giving cozy yet contemporary Text: The image captures serene lakeside scene featuning sign for art gallery nomed "Shister Cove." The sign is mounted wooden pole and has and gold section with bold white lettering that reads "ART GALLERY,"
Color:
Llama3
Curated Caption Pool
Llama3-700 708
General Q
Recsoning
8 MultiChoice
Q: What are some elements in the image
Q: What is the purpose of the sign?
Q: What is written the sign? ART GALLERY COVE' RYAN STIL'S MASTERFIECES.
contribute peaceful and picturesque setting?
A) Advertise art gallery galleryy
The image includes calm lake with still small boot floating gently, and wooden docks extending into the water...
B) Give lake safety warnings
Provide road directiors
D) Announce weather updates
Figure 4: Overview of Text-Driven VQA Synthesis framework. We leverages high-quality captions as the foundation for both image generation via diffusion models and diverse QA pair construction.
where
pfr. c) w [o]r,c],
K[r, c] W Tir- wi (r+1) 1) ww,
Ah], K[r,c] E g(wwh)xD Vrr, c] = w Tirr- w (r) 1), e.h: (c+ 1) h), E (wohixD
Here, Wa, W WV are learnable projection matrices.
In essence, each semantic exchange layer extracts most relevant visual features for current query and embeds them into latent tokens. In the subsequent decoding layer, these enriched tokens are integrated with query tokens, enabling bidirectional modality interaction.
3.3 DUAL-SEMANTIC MAPPING LOSS
In the previous sections, visual and textual features are frequently mapped and interact. To better guide feature mapping, we introduce Dual-Semantic Mapping Loss based on two complementary projectors: M2. and MI. The illustration of our reconstructed loss is shown Figure
For MLP u2r. it is crucial to ensure that the mapped visual features align closely with the feature space of LLM. We shift our focus to Ve, which represents textual tokens processed by the vision encoder in the visual space. Consequently, after mapping Ve through LLP2,, its rebuilt textual representation T should closely resemble the LLM-based textual representation Tg. To quantify the alignment of MLP compute the cosine similarity-based loss between these two features:
T, T [Tyl [Tă
T MLPP2c( Loz
Analogous L.2r, we T; evaluate the effectiveness of M M.P2. An optimal MLP22 should transform T; into rebuilt image representation V that closely resembles vision feature V, in the visual space. We again employ cosine similarity loss assess the quality of M LP
VTT (T), Liz. = 1/2 V
(9)
V.I IV7I
The training objective combines cosine similarity losses with the traditional cross-entropy loss, bal- anced by weighting parameter:
Liotat entrroy-ntropy X(Luze L12w).
(10)
5

---

Published as a conference paper at ICLR 2026
stort cetios
Syta CALDO KJ
Spnth
SyatTwokk
Synniny 2010
Tviskemee Viaind
Ofrersaniinn
SyraSooee (250
9.7M
LLa W...AttaiSSSS
4.8%
beakggia 1e00 K)
Capcion
Selence
Speik Log
ALLaYA-cage 10.00
Spenotabkheee 2002
A
Spriskeneens (00,
Qpe
OCR
Sptr
SowtCanting (150u K)
kreejls7 D.3
ShavCPTeY (P32
SywhCannhhnn 12500
ChencTree22 D30
Paaiig
Yo ic
DeNN K)
LJ A.AAR20 K)
OerOA (1380
UNQAAa000 KG
Stage2
SEVVAII DXC
Nath Viaaacs KC
Va
CecaNirat XD
TAlyOAA/180 K:
Chemp K)
AI2D
LauigAAZ2 K3
% Sceanc
Apwi KG
HITAACSS K0
AT QNNORRO
Figure 5: FLARE-IOM and FLARE-I2M. We collected total of 10M samples for pretraining (Stage 1) and 12M samples for fine-tuning (Stage 1.5 and Stage 2). The circle illustrates the data distribution, with the right side of the circle showing all the utilized data and their proportions.
The intuition behind Luz is to first derive visual features corresponding to textual tokens via Text- Guided Vision Encoding, and then project them back into the textual space for alignment with the original textual embeddings. L22v is defined a symmetrical manner. Importantly, these loss func- tions require no additional input and can be seamlessly integrated into the overall model architecture.
3.4 TEXT-DRIVEN VQA SYNTHESIS
The cross-modality integration components we proposed require diverse combinations of text and images for effective training. However, existing datasets construct QA pairs based images, where the constraints of fixed visual content lead to monotonous and limited textual diversity. A more direct approach is first ensure textual diversity, then generate corresponding visual content to match the rich variety of QA pairs. Based on this principle, we propose text-driven VQA synthesis, as shown in Figure which shifts the emphasis from visual content textual richness and diversity.
Our process begins with carefully selecting high-quality captions. These captions are then enriched with LLM, producing detailed and nuanced textual descriptions that capture various visual and con- textual attributes. These enriched descriptions serve prompts for diffusion model, which generates images closely aligned with the textual context. Simultaneously, we utilize the descriptions as input content, leveraging the LLM again to construct diverse QA pairs, ensuring broad coverage across multiple-choice, multi-turn dialogues and reasoning tasks. This step distinguishes our method from traditional first MLLM-based approaches. Directly leveraging comprehensive textual data significantly enhances the model's understanding and representation of visual contexts.
In Appendix D we provide a detailed breakdown of our data synthesis process. Our approach com- bines prompt engineering with a rigorous, multi-stage filtering pipeline to design diverse QA tasks, ensuring high standards of quality and alignment across modalities. Through this way, model is equipped with various QA data, enabling comprehensive training of cross- cross-modalls integration.
4 EXPERIMENTS
Training Strategies. We propose a three-stage training framework for vision-text alignment and integration: 1) Stagel-Foundations Semantic Alignment: pretrain the vision encoder jointly han- dle visual and textual representations; 2) Stage 1.5 1.5-Contextual Multimodal Fusion: train on diverse QA and caption data to strengthen cross- modal alignment in varied contexts; 3) Stage2-Visual In- struction Tuning: finetune on visual task datasets for effective downstream question answering.
Data Synthesis Details. For Stagel, we generated five datasets- SynthColor, SynthCount, SynthSpatial, SenthText, and caanSScnn--yllddin 1.5M image-caption pairs. In Stage 1.5, seven categories were Synieed--nSntMMMtihhii..... SynthConwLong. SynthConvShort, SynthReasoning.

---

Published as a conference paper at ICLR 2026
Table 1: Model comparison on 16 Benchmarks. # Vis tok. denotes the roughly averaged number of visual tokens across all benchmarks.
Model
General
OCR & Char
Knowledge
Vision Centric
# Vis tok.
MMB PN
MMB
POPE
MM-
MME
MMB
Soad-mmage
MMStar
TeuVQA
OCRBench
ChariQA
AED
Vashi Vista
MMMU
SQA
RWQA
CVBone
MMVP
Method
<-48 Model Comparioon
MiniCPM-V22.0 3B
400
69.1
62.6
86.3
41.0
15314
322.8
67.1
39.1
74.1
60.5
75.7
62.9
39.8
38.2
80.7
55.8
FlorenceVVL 3B
576
71.6
60.8
88.3
51.0
1498.7
403.9
70.6
44.9
69.1
63.0
70.7
73.8
52.
41.8
84.6
60.4
70.2
64.7
3B
1400
61.4
1592.4
607.5
79.7
84.0
81.4
62.3
65.4
75.5
76.7
WWSSeckkVVL-Taae
1500
74.6
72.1
52.5
1548.3
357.1
72.3
45.9
80.7
80.5
81.0
71.6
53.6
40.7
64.2
Phi 3,5-Vision
2100
75.5
64.2
82.2
46.5
1473.4
412.1
69.9
49.0
61.1
59.8
72.0
77.4
43.3
89.0
53.5
69.3
67.7
FLARE-L 3B (ours)
630
79.6
74.7
88.8
59.1
1588.3
425.6
74.2 50.5
73.3
63.8
74.2
79.4
54.0
44.6
86.6
62.1
78.2
76.3
FLARE-X 3R (ours)
1400
36.3
88.6
61.9
1612.4
437.2
77.2
67.4
79.3
81.2
57.2
46.3
63.7
80.1
79.8
7- Model Comparison
MiniCPM-Llanme V-2.5 8B
400
77.6
73.8
86.7
52.8
1604.8
398.5
72.3
54.8
76.6
72.5
79.7
78.4
54.5
45.8
89.2
63.5
Cumbrain
576
87.4
1547.1
62.4
73.3
73.0
49.0
72.2
$1.3
Florence-VV VL 8B
576
76.2
69.5
88.4
56.3
1560.0
381.1
74.9
50.0
74.2
63.4
74.7
74.2
55.5
43.7
85.9
64.2
73.4
73.3
Eagle 8B
1024
75.9
1589.0
76.3
77.1
62.6
80.1
76.1
52.7
43.8
84.3
66.5
71.6
Molmo 7B
1200
73.6
75.2
89.0 41.5
1536.1
341.6
74,9 52.7
80.4
65.6
80.4
81.0
48.7
49.1
74.6
71.1
Owen2.5VL. 7D
1400
1652.7
88.5
89.5
83.4
68.4
81.1 80,9
LLaVA-One Vision 7B
2400
81.7
78.0
87.2 58.8 1626.0
483.0
74,8 60.9
78.5
69.7
78.8
81.6
56.1
47.7
96.6
65.5
FLARE-L 8B (ours)
630
81.8
76.2
88.7
59.6
1596.8
347.3
77.6
54.5
76.2
66.3
77.9
81.6
57.5
43.1
89.7
65.4
79.5
78.6
FLARE-X 8B (ours)
1400
62.8 1639.1
378.9
78.7
56.4
79.7
83.4
83.6
61.1
45.6
66.9
81.5
79.7
SynthTexIQA, SynthContrastL.ong and SynthContrattShho--ppddcccc 1M QA pairs. Most were derived from captions to produce descriptions, QA pairs and images, while SynthTexiQA required multi-step pipeline to ensure the presence of text elements. Stage 2 followed a similar process with redesigned prompts, generating 0.9M QA pairs across SynthMuliiCooice, SynthConvLongg and SynthConvShort. All the prompts are listed in Tables 14 29 Detailed synthesis procedures are provided in Appendix E, along with filtering, evaluation, and t-SNE visualizations.
Data Collection Details. The data used is summarized in Figure 5 In Stage 1, we adopt CC12M Singla et al. 2024), ShareCaptioner (Chen et al.) 2023). URSA-Alignman
(Luo et al. 2025 to-
gether with T 1.MM synthesized pairs. In Stage 1.5, the primary objective is to enhance question di- versity. We combines Pixmo (Deitke al. 2024). QInstruct (Wu et al. (023), AVIS-Inntruce (Wang) cct al., (023)) GPT4Rcwrite (Tong et al.] 2024a). along with IM synthesized QA pairs. In Stage 2, the focus shifts towards vision-centric instruction tuning. We mainly uses Cambrian-1 7M (Tong let al. 2024a), MMathCoT (Luo et al. 2025), MulBerry (Yao et al. (024a). DocMatix (Laurençon al. 2024). and 0.9M synthesized QA pairs. In both Stage 1.5 and Stage 2, we incorporate pure text data to maintain language ability and adapt LLM decoding.
Backbone. 1) LLM: We select Phi-3.5-mini-instruet (bbdi et al. 2024 and ILLMA3.1-881 instruct (AI@ Meta 2024) 2) Vision Encoder: We use SigLIP2-
PPT-Patch16-38. (Tschannen al. 2025). 3) Projector: M LP22. and M LP are both implemented as two-layer MLPs.
Implementation Details. In Context-Aware Alignment Decoding, a semantic exchange layer is inserted every three decoding layers. Furthermore, we adopt a dynamic token mechanism, randomly sampling latent tokens from {4, 16, 36, 64, 144) per batch, which improves training stability while maintaining effectiveness. Latent tokens and semantic exchange layers are randomly initialized. For Dual-Semantic Mapping Loss, the balancing coefficient X is set to 0. During both training and evaluation, we set random seeds to 42 to ensure stability and reproducibility. Comprehensive ablation studies examining the effects of every key parameter settings are provided in Appendix C
We train two versions: FLARE-L and FLARE-X. For FLARE-L, we adopt fixed resolution with an average of 630 vision tokens (including context-aware latent tokens), designed for lower com- putational cost. For FLARE-X, we follow LLaVA-NeXT (Liu et al. 2024a), employing dynamic resolution for stronger performance. In this setting, we perform Text-Guides Vision Encoding and Sual-Semantic Mapping Loss computation for each subfigure. For Context-Award Alignment De- coding, we concatenate all subfigures to serve keys and values in the attention process.
7

---

Published as a conference paper at ICLR 2026
Table 2:
Architectural Comparison with Qwen2.5VL using identical backbones. FLARE demon-
strates superior performance on most metrics, highlighting the effectiveness of our architecture.
Seed Seed- Image
OCRBench
MMBEN
MMB
MM-Vet
MME
MMEC
MMStar
TexiQQ
ChanQA
Math Vista
MMMU
RWQA
CVBench
MMVP
POPE
AED
SQA
Model
ViT NoViT+Qwen2.55
5.53B
Qwen2.5VL. 3B
79.1 78.1 87.3 61.4 1592.4 607.5 74.0 55.9 79.3
79.7
84.0
81.4
62.3
51.2
81.4
65.4
75.5
76.7
FLARE 3B
83,2 77.7 89.0 63.8 1637.1 473.1 77.1 56.1
80.8 73.2
82.2
82.9
59.9
50.8
88.7
64.5
81.2 80.4
7B
Qwen2.5VL
7B
83.2
82,8 85.9 69.7 1652.7
633.4
77.0
64.1
83.5
88.5
89.5
83.4
68.1
58.0
89.0
68.4 4
81.1 80.9
FLARE 7B 86.0 83.3 89.8 68.2 1665.4 508.5 79.7 62.4
85.2 78.4 88.3
84.2
65.9
54.4 92.2
71.1 82.8 83.3
Table 3: Performance analysis of model components. We denote Text-Guided Vision Encoding,
Mapping Loss, Context-Aware Alignment Decoding as A, B, and C, respectively.
Seed Image
MMB
MMB
MM--et
MME
MMEC
MMSta
TexVQA
ChanQA
MMMU
RWQA
CVBench
MMVP
POPE
A12D
SQA
A
LLXV
74,9
72.4
87.5
43.1
1562.4
361.9
74.2
47.1
70.8
66.2
72.8
43.5
78.9
60.4
70.4
69.5
Baseline
72.5
70.4
87.2
40.9
1531.7
358.2
71.7
43.9
61.5
57.7
71.0
38.3
76.2
58.3
68.3
66.
73.5
71.4
88.0
43.2
1543.2
388.5
72.8
45.0
64.4
60.2
72.1
41.
77.4
59.8
70.7
67.6
/
73.7
71.5
87.5
42.0
1554.3
354.7
72.8
45.4
62.6
72.5
39.8
77.9
68.4
/
74.5
72.6
87.5
42.3
1574.2
374.9
73.7
44.1
63.2
60.3
73.5
41.0
79.7
59.7
70.7
70.3
/
74.4
72.3
87.9
43,7
1566.8
367.2
73.7
43.8
64.7
61.1
72.9
42.6
78.5
61.0
71.2
69.1
/
/
/
75.3
73.4
88.0
43.4
1583,9
355.8
74.6
45.5
65.6
61.7
73.7
42.4
80.3
60.8
71.7
69.8
In each stage, we unfreeze all components to ensure comprehensive optimization. For all models, we use a global batch size of 256 with a cosine-decay learning rate of le-4 pretraining. In Stage 1.5, the global batch size is 128 and the learning rate is 2e-5. In Stage 2, we maintain a global batch size of 128 with learning rate of 1e-5, We trained all models epoch on 64 NVIDIA A100 GPUs.
Comparison Models. We compare our models with the following baselines across different param- eter scales. For 3B models, we compare FLARE against MiniCPM-V-2. 3B (Yao et al., 20246), Florence- VI. 3B (Chen et al.) 2024b). Qwen2.5VI. 3B (Bai et al. 2025), DeepSeek- VI.2- 1 Tiny (Wu
ct al. 2024), and Phi Vision (Abdin et al. 2024). For 7B models, we compare FLARE against
MinCCPM-Llama--Va. 5-22. 8B Yao et al. (024b), Cambrian-1 8B (Tong et al. 2024a). Florence-V VL
8B (Chnn et al. 2024b), Eagle 8B (Shi et al. (025). Molmo 7B (Deitke et al. 2024 Qwen2.5VL
7B (Bai al. (025). and LLa -n-One Vision 7B (Li al. (024)).
We evaluate the performance on benchmarks with four categories. 1) General: MMBench (EN and CN) (Liu et al., 2024b), POPE (Fan et al., 2023). MM-Vet (Yu et al.
Evaluation.
2024
MME Perception (Fu et af. 2024). MME Cognition (Fu et al. 2024). SeedBench (Li et al.
2023
MMStar (Chen et al. (022c)).
OCR Chart: TextVQA (Sidorov et 2020), OCRBench (Liu
ct al. 2024c), ChartQA (Masry et al. 2022). 3) Knowledge based: A12D (Hippala et al.
2020).
Math Vista (Lu al. 2024), MMMU
Yue al. 2024) and ScienceQA (Lu al. 2022)
Vision
Centrie: MMVP (Tong et al. 2024b). RealworldQA (x.al 2023) and CV-Bench (Tong et al.
2024a
Result. As shown in Table I our model achieves performance comparable or exceeding state- of-the-art art models with only fraction of training resources. FLARE-L surpasses most models with a comparable number of vision tokens, and even achieves a large margin over MiniCPM-V while using only about 1/100 of its training data. Moreover, FLARE-L 3B outperforms Cambrian-1 8B and Florence-Vi VL SB. On larger scales, FLARE-L 8B reaches performance on par with LLaVA- Vie 7B despite requiring merely one-fourth of its vision token count. For FLARE-X. it achieves performance comparable to or even exceeding Qwen2.5VL on nearly half of the bench- marks, despite utilizing merely 1/1000 of the training data. These results demonstrate how carefully designed modality fusion strategies, coupled with high-quality data synthesis, can yield substantial improvements in both efficiency and overall performance.
8

---

Published as conference paper at ICLR 2026
Table 4: Fair comparison between different models. We conduct fair evaluations on Open-LLaVA-
NeXT-IM and Cambrian-1 7M dataset with the same LLM backbone.
Seed Image
NMBBE
MMB
POPE
MM-Vet
MME
MMEC
MMStar
TexVVQ
CharQQ
A12D
MMMU
SQA
RWQA
CVBeach
MMVP
Models
Open
NeXT
Dataset
Florence-VL VL
72.9
70.9
87.8
41.5
1547.2
366.8
72.1
44.8
65.7
60.2
70.7
40.5
76.3
57.7
68.6
66.4
Cambrian-1 8B
73.8
71.1
87.5
42.8
1558.8
347.1
72.9
46.3
64.4
62.8
71.2
42.1
78.3
59.9
71.8
67.9
Eagle 8B
74.1
71.9
87.4
43.8
1577.3
364,9
73.7
47.3
68.8
66.1
72.9
43.0
78.7
62.6
70.9
70.0
LLaVA-NeXT 8B
74.9
72.2
87.5
43.1
1562.4
361.9
74.0
47.1
69.8
66.2
72.8
43.5
78.9
60.6
70.4
69.5
FLARE-L 8B
74.5
72.4
87.7
44.3
1566.4
353.2
74.2
44.4
66.9
62.1
73.0
42.8
60.3
70.7
69.6
FLARE-X
75.5
87.8
362.4
71.5
Cambrion
7M Datase!
Florence-Vi 88
74.8
72.1
88.5
46.7
1554.2
363.4
73.4
49.7
71.0
71.6
72.5
43.4
84.8
63.6
71.8
69.1
Cambrian-1 8B
75.6
72.4
87.9
46.4
1562.1
389.5
74.4
52.2
71.8
72.2
74.3
45.7
83.3
64.7
74.2
70.6
Fagle 8B
76.6
72.3
87.8
48.6
1566.4
334,9
74.6
55.7
75.4
77.2
75.2
44.8
82.7
66.2
74.1
72.4
LLaVA-NeXT
73.2
1549.2
394.5
75.1
75.8
76.6
44.5
66.7
75.2
72.8
FLARE-L 8H
77.8
72.1
89.0
46.5
1569.6
398.4
74.7
55.2
74.4
74.2
74.8
42.4
83.8
65.7
73.2
72.9
FLARE-X 8B
78.4
74.0
88.8
51.2
1556.3
370.8
75.7
54.5
78.4
79.6
77.9
44.4
85.1
66.5
76.0
74.2
ABLATION STUDY
5.1 UNLOCKING FLARE'S FULL POTENTIAL.
To showcase FLARE's full potential, we removed the vision token constraint and used the same Qwen2.5VL backbones for fair comparison. As shown in Table 2 FLARE excels the major- ity of general visual understanding benchmarks, despite using only fraction of its training data. FLARE 3B even surpasses the larger Qwen2.5VL 7B benchmarks like MMBench, POPE and CVBench. This experiment highlights the superiority of the FLARE architecture.
5.2 COMPONENT ANALYSIS MODALITY FUSION
We adopt FLARE-L architecture to investigate the impact of different components. All models are trained on Open-LLa VA-NeXT IM dataset. Results Table show that each component con- tributes to overall performance. The integration of Seal--eeantic Mapping with Text-Guided Vision Encoding yields significant performance boost, underscoring the importance modality alignment and the effectiveness of our loss design. Context-Aware Alignment Decoding substantially improves OCR accuracy and mitigates hallucinations. When integrated collectively, our model outperforms the LLaVA-NeXT dynamic resolution baseline while using only half of the vision tokens.
5.3 FAIR COMPARISON WITH EXISTING MODELS
To highlight the superiority of our architecture while removing effects of training data size and stronger backbone models, conduct controlled comparisons. All models use LLaMA3.1- 8B-instruct as LLM backbone and are trained on Open-LLaVA-NNee 1M and Cambrian-1 7M. Specifically, for FLARE, effects. The results are shown in Table On Open-LLaVA-NeXTTIMM FLARE-X surpasses existing models like Florence- VL and Cambrian-1. Notably, FLARE-L achieves performance comparable to Eagle and LLaVA-NeXT with much fewer vision tokens. While Cambrian-1 and Eagle require four vision encoders, FLARE-L attains competitive results with a single encoder. On the larger Cambrian-1 1 7M dataset, FLARE demonstrates even stronger gains, highlighting its scaling ability.
the SigLip model the vision encoder to better isolate architectural
5.4 COMPUTATIONAL CoST ANALYSIS
We evaluate the computational cost of FLARE components under different resolution strategies and compare FLARE-L with models in Section 5.3 using same configurations. All models are trained on Open-LLLaVA-NeTT-IN with 32 A100 GP Us and evaluated single A 1100. As shown

---

Published as a conference paper at ICLR 2026
Table Components- wise Computational Cost Analysis. We denote Texi-Guided Vision Encoding, Dual- Dual-Semantic Mapping Loss, and Context-Aware Alignment Decoding as A, B, and C.
Fixed Resolution
Dynamic Resolution
buseline
+A
+A.B
+A.
+A.BC
baseline
+A
+A.B
+AC
+A.B.C
Train
13.0
13.3
16.3
28.2
28.8
Infer (s/Ik sample) F1. (llpppapllp
171
176
176
196
197
313
323
324
357
359
6.56
6.64
6.64
8.44
8.44
12.32
12.41
12.41
15.39
15.40
Table 6: Model-wise Computational cost analysis. under the same model configurations and dataset.
We compare FLARE with existing methods
FLARE KB
Florence VL
Cumbrian 8B
Eagle 8B
LLaVA
(h)
14.
19.1
18.2
infer (s/Ik sample)
197
183
232
294
313
7.46
9.77
11,16
Table Impact of synthetic data on performance. We gradually incorporate our generated data into
training data, showing the improvement in performance on various tasks.
Seed Seed-Imen Image
MMB4N
MMBCN
POPE
MM-Ner
MME
MMS
MMStar
TorVQQ
Charqq
AIDD
MMMU
SQA
RWQA
CVBench
MMVP
Models
Syoth Data
FLARE-L 8B
79.4
75.1
88.8
56.4
1582.4
363.4
76.1
54.6
75.3
76.1
78.2
42.8
87.0
65.7
76.8
75.5
FLARELL 8B
81.8
76.2
88.7
59.6
1596.8
347.3
77.6
54.5
76.2
77.9
81.6
43.1
89.7
65.4
79.5
78.6
FLARE-X 8B
81.1
76.7
88.7
61.7
1600.4
403.6
76.9
56.2
78.8
80.3
81.1
44.7
88.6
67.0
79.2
76.3
FLARE-X 8B
83.6
78.3
89.1
62.8
1639.1
378.9
78.7
56.4
79.7
83.4
83.6
45.6
91.2
66.9
81.5
79.7
in Table 5 Text-Guided Vision Encoding and Dual-Semantic Mapping Loss incur only 2% cost, while Context-Aware Alignment Decoding adds 25% in training and 10% in inference, remain- ing manageable due windowed attention. Compared with dynamic essoltiio(LLLaV-Nees or Cultiiencoder(Cambriann
Fagle-X) in Table 6 FLARE-L achieves much lower computational cost. Combined with the performance in Table 4 FLARE-L demonstrates dual advantages: SOTA performance with reduced overhead. FLARE-L proves that without increasing the number of vision tokens or encoders, superior performance can be attained through effective modality integration.
5.5 INFLUENCE OF SYNTHETIC DATA ON PERFORMANCE
While our architecture provides substantial improvements, another key contributor to the SOTA per- formance is the use of synthetic data. To evaluate its impact, conduct experiments by adding generated samples to training corpus. As shown in Table synthetic data significantly enhance per- formance in targeted downstream tasks such as MMBench, SQA, CVBench, and MMVP. Notably, when combined with dynamic resolution, FLARE-X achieves even greater performance gains.
In essence, the architecture determines the performance ceiling, while high-quality synthetic data enables the model to realize this potential. As shown in Tables [ and [7] FLARE shows strong performance on benchmarks such as MMBench, MMVP, SQA, and Seed-Image. These datasets often contain abundant textual cues (keywords in questions or multichoice options), which provide rich context for modality fusion. Our synthetic data is further tailored this process: through prompt engineering, we construct diverse QA pairs to help the model learn more effective fusion.
CONCLUSION
We present FLARE, a family of VLMs that rethinks how vision and language should be integrated. Rather than treating vision as static input, our model enables interaction at every phase- from vision encoding guided by text, to decoding that aligns with visual features. We also propose dual-semantic loss that enforces consistency between modality mapping. Beyond architecture, generation method that synthesizes QA pairs from text. Our experiments show that FLARE achieves superior performance with far fewer vision tokens, highlighting both its efficiency and accuracy.
introduce data
10

---

Published as a conference paper at ICLR 2026
ACKNOWLEDGMENTS
This work is supported by National Natural Science Foundation of China (92470121, 62402016), National Key R&D Program of China (2024YFA101400)) Zhongguancun Academy (Grant No.s C20250204, C20250602), and figh-peeformance Computing Platform Peking University.
REFERENCES
Marah Abdin, Jyoti Aneja, Hany Awadalla, Ahmed Awadallah, Ammar Ahmad Awan, Nguyen Bach, Amit Bahree, Arash Bakhtiari, Jianmin Bao, Harkirat Behl, Alon Benhaim, Misha Bilenko, Johan Bjorck. Sébastien Bubeck, Martin Cai, Qin Cai, Vishrav Chaudhary, Dong Chen, Dong- dong Chen, Weizhu Chen, Yen-Chun Chen, Yi-Ling Chen, Hao Cheng, Parul Chopra, Xiyang Dai, Matthew Dixon, Ronen Eldan, Victor Fragoso, Jianfeng Gao, Mei Gao, Min Gao, Amit Garg, Allie Del Giorno, Abhishek Goswami, Suriya Gunasekar, Emman Haider, Junheng Hao, Russell J. Hewett, Wenxiang Hu, Jamie Huynh, Dan Iter, Sam Ade Jacobs, Mojan Javaheripi, Xin Jin, Nikos Karampatziakis, Piero Kauffmann, Mahoud Khademi, Dongwoo Kim, Young Jin Kim, Lev Kurilenko, James R. Lee, Yin Tat Lee, Yuanzhi Li, Yunsheng Li, Chen Liang, Lars Liden, Xihui Lin, Zeqi Lin, Ce Liu, Liyuan Liu, Mengchen Liu, Weishung Liu, Xiaodong Liu, Chong Luo, Piyush Madan, Ali Mahmoudzadeh, David Majercak, Matt Mazzola, Caio César Tcodoro Mendes, Arindam Mitra, Hardik Modi, Anh Nguyen, Brandon Norick, Barun Patra, Daniel Perez- Becker, Thomas Portet, Reid Pryzant, Heyang Qin, Marko Radmilac, Liliang Ren, Gustavo de Rosa, Corby Rosset, Sambudha Roy, Olatunji Ruwase, Olli Saarikivi, Amin Saied, Adil Salim, Michael Santacroce, Shital Shah, Ning Shang, Hiteshi Sharma, Yelong Shen, Swadheen Shukla, Xia Song. Masahiro Tanaka, Andrea Tupini, Praneetha Vaddamanu, Chunyu Wang. Guanhua Wang, Lijuan Wang, Shuohang Wang, Xin Wang, Yu Wang, Rachel Ward, Wen Wen, Philipp Witte, Haiping Wu, Xiaoxia Wu, Michael Wyatt, Bin Xiao, Can Xu, Jiahang Xu, Weijian Xu, Ji- long Xue, Sonali Yadav, Fan Yang, Jianwei Yang. Yifan Yang, Ziyi Yang. Donghan Yu, Lu Yuan, Chenruidong Zhang, Cyril Zhang, Jianwen Zhang, Li Lyna Zhang, Yi Zhang, Yue Zhang, Yunan Zhang, and Xiren Zhou. Phi-3 technical report: A highly capable language model locally on your phone. arXiv preprint arXiv:2404. 14219, 2024.
AI@Meti Llama 3 model card.
2024.
URL hitps: httts://github.commmeeee www.:/o/mmttaalaaaaaaa.aaaaaaaaa
1lam3/bbob/main/ /MODEL CARD md
Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Sibo Song, Kai Dang, Peng Wang, Shijie Wang, Jun Tang, Humen Zhong, Yuanzhi Zhu, Mingkun Yang, Zhaohai Li, Jianqiang Wan, Pengfei Wang, Wei Ding, Zheren Fu, Yiheng Xu, Jiabo Ye, Xi Zhang, Tianbao Xie, Zesen Cheng, Hang Zhang, Zhibo Yang, Haiyang Xu, and Junyang Lin. Qwen2.5-vl technical report. arXiv preprint ar 2rXiv22502.33223 2025.
Guiming Hardy Chen, Shunian Chen, Ruifei Zhang, Junying Chen, Xiangbo Wu, Zhiyi Zhang, Zhi- hong Chen, Jianquan Li, Xiang Wan, and Benyou Wang. Allava: Hamessing gpt4v-synthesized data for lite visionelanguage models. arXiv preprint 2402: 1684, 2024a.
Jiuhai Chen, Jianwei Yang, Haiping Wu, Dianqi Li, Jianfeng Gao, Tianyi Zhou, and Bin Xiao. Florence-vl: Enhancing vision-language models with generative vision encoder and depth- breadth fusion. arXiv preprint arXiv:2412.0444 2024b.
Lin Chen, Jisong Li, Xiaoyi Dong, Pan Zhang, Conghui He, Jiaqi Wang, Feng Zhao, and Dahua Lin. Sharegpt4v: Improving large multi-modal models with better captions. arXiv preprint wrXiv:2311.1279. 2023.
Lin Chen, Jinsong Li, Xiaoyi Dong, Pan Zhang, Yuhang Zang, Zehui Chen, Haodong Duan, Jiaqi Wang, Yu Qiao, Dahua Lin, and Feng Zhao. Are we on the right way for evaluating large vision- language models? arXiv preprint arXiv:2403.20333 2024c.
Wenliang Dai. Junnan Li, Dongxu Li, Anthony Meng Huat Tiong, Junqi Zhao, Weisheng Wang, Boyang Li. Pascale Fung, and Steven Hoi. Instructblip: Towards general- purpose -aiounlanguaage models with instruction tuning. arXiv preprint 2305::230000000 2023.

---

Published as a conference paper at ICLR 2026
Matt Deitke, Christopher Clark, Sangho Lee, Rohun Tripathi, Yue Yang, Jae Sung Park, Moham- madreza Salchi, Niklas Muennighoff, Kyle Lo, Luca Soldaini, Jiasen Lu. Taira Anderson, Erin Bransom, Kiana Ehsani, Huong Ngo, YenSung Chen, Ajay Patel, Mark Yatskar, Chris Callison- Burch, Andrew Head, Rose Hendrix, Favyen Bastani, Eli VanderBilt, Nathan Lambert, Yvonne Chou, Arnavi Chheda, Jenna Sparks, Sam Skjonsherg. Michael Schmitz, Aaron Sarnat, Byron Bischoff, Pete Walsh, Chris Newell, Piper Wolters, Tanmay Gupta, Kuo-Hao Zeng, Jon Bor- chardt, Dirk Groeneveld, Crystal Nam, Sophie Lebrecht, Caitlin Wittlif, Carissa Schoenick, Os- car Michel, Ranjay Krishna, Luca Weihs, Noah A. Smith, Hannaneh Hajishirzi, Ross Girshick, Ali Farhadi, and Aniruddha Kembhavi. Molmo and pixmo: Open weights and open data for State-of-the-art vision-language models. arXiv preprint arXiv:2409. 17146, 2024.
Zhiwen Fan, Panwang Pan, Peihao Wang, Yifan Jiang, Dejia Xu, Hanwen Jiang, and Zhangyang Wang. Pope: 6-dof promptable pose estimation of any object, in any scene, with one reference. arXiv preprint 2riiv2205.11577,, 2023.
Chaoyou Fu, Peixian Chen, Yunhang Shen, Yulei Qin, Mengdan Zhang, Xu Lin, Jinrui Yang, Xiawu Zheng, Ke Li, Xing Sun, Yunsheng Wu, and Rongrong Ji. Mme: A comprehensive evaluation benchmark for multimodal large language models. arXiv preprint arXiv:2306. /3394, 2024.
Samir Yitzhak Gadre, Gabriel Ilharco, Alex Fang, Jonathan Hayase, Georgios Smyrnis, Thao Nguyen, Ryan Marten, Mitchell Wortsman, Dhruba Ghosh, Jieyu Zhang. Eyal Orgad, Rahim En- tezari, Giannis Daras, Sarah Pratt, Vivek Ramanujan, Yonatan Bitton, Kalyani Marathe, Stephen Mussmann, Richard Vencu, Mehdi Cherti, Ranjay Krishna, Pang Wei Koh, Olga Saukh, Alexan- der Ratner, Shuran Song, Hannaneh Hajishirzi, Ali Farhadi, Romain Beaumont, Sewoong Oh, Alex Dimakis, Jenia Jitsev, Yair Carmon, Vaishaal Shankar, and Ludwig Schmidt. Datacomp: In search of the next generation of multimodal datasets. arXiv preprint arXiv:2304.1 14108, 2023.
Lupyan Gary, Linguistically modulated perception and cognition: The label-feedback hypothesis.
Frontiers in Psychology, 2012.
Tuomo Hiippala, Malihe Alikhani, Jonas Haverinen, Timo Kalliokoski, Evanfiya Logacheva, Sera-
fina Orekhova, Aino Tuomainen, Matthew Stone, and John A. Bateman. Ai2d-rst: a multimodal corpus of 1000 primary school science diagrams. http://dx.doi.or///00 htw.://////1..0001100000010 10770000000779- 55(3):661-688 December 2020. ISSN 1574-0218. doi: 10. 10.10007/110579-00000993
Hugo Laurençon, Andrés Marafioti, Victor Sanh, and Léo Tronchon. Building and better under- standing vision-language models: insights and future directions. 2024.
Bo Li, Yuanhan Zhang. Dong Guo, Renrui Zhang, Feng Li, Hao Zhang, Kaichen Zhang, Peiyuan Zhang, Yanwei Li, Ziwei Liu, and Chunyuan Li. Llava-onevision: Easy visual task transfer. arXiv preprint 4rXhv22408.03326, 2024.
Bohao Li, Rui Wang, Guangzhi Wang, Yuying Ge, Yixiao Ge, and Ying Shan. Seed-bench: Bench- marking multimodal Ilms with generative comprehension. arXiv preprint arXiv:2307.16125, 2023.
Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. Visual instruction tuning. arXiv preprint arXiv:2304.0844 2023.
Haotian Liu, Chunyuan Li, Yuheng Li, Bo Li, Yuanhan Zhang, Sheng Shen, and Yong Jae Lee. Llava-next: Improved reasoning, ocr, and world knowledge. http://www......igg......bbbbbba 2024a.
Yuan Liu, Haodong Duan, Yuanhan Zhang, Bo Li, Songyang Zhang, Wangbo Zhao, Yike Yuan, Jiaqi Wang, Conghui He, Ziwei Liu, Kai Chen, and Dahua Lin. Mmbench: Is your multi-modal model an all-around player? arXiv preprint urXiv:2307.060 06281, 2024b.
Yuliang Liu, Zhang Li, Mingxin Huang, Biao Yang, Wenwen Yu, Chunyuan Li, Xu-Cheng Yin, Cheng-Lin Liu, Lianwen Jin, and Xiang Bai. Ocrbench: on the hidden mystery of ocr in large mul- timodal models. http://dx..d 1869-1919. doi: 10.1007/s1 432-0774551422-222442333----
67(12), December 2024c. ISSN
12

---

Published as a conference paper at ICLR 2026
Zheng Liu, Hao Liang, Bozhou Li, Tianyi Bai, Wentao Xiong, Chong Chen, Conghui He, Wen- tao Zhang, and Bin Cui. Synthvlm: High-efficiency and high-quality synthetic data for vision language models. arXiv preprint arXhv:2407.20756 2025.
Pan Lu, Swaroop Mishra, Tony Xia, Liang Qiu, Kai- Wei Chang, Song-Chun Zhu, Oyvind Tafjord, Peter Clark, and Ashwin Kalyan. Learn to explain: Multimodal reasoning via thought chains for science question answering. arXiv preprint 2e:::2229.091333 2022.
Pan Lu, Hritik Bansal, Tony Xia, Jiacheng Liu, Chunyuan Li, Hannaneh Hajishirzi, Hao Cheng, Kai- Wei Chang, Michel Galley, and Jianfeng Gao. Mathvista: Evaluating mathematical reasoning of foundation models in visual contexts. arXiv preprint arXiv:2310.0223. 2024.
Ruilin Luo, Zhuofan Zheng, Yifan Wang. Yiyao Yu, Xinzhe Ni, Zicheng Lin, Jin Zeng, and Yujiu Yang. Ursa: Understanding and verifying chain-of-thorn of-thougttt reasoning in multimodal mathematics. arXiv preprint Arv:::2001 250.004866. 2025.
Ahmed Masry, Do Xuan Long, Jia Qing Tan, Shafiq Joty, and Enamul Hoque, Chartqa: A bench- mark for question answering about charts with visual and logical reasoning. arXiv preprint arXiv:2203. 10244, 2022.
Min Shi, Fuxiao Liu, Shihao Wang. Shijia Liao, Subhashree Radhakrishnan, Yilin Zhao, De-An Huang. Hongxu Yin, Karan Sapra. Yaser Yacoob, Humphrey Shi, Bryan Catanzaro, Andrew Tao, Jan Kautz, Zhiding Yu, and Guilin Liu. Eagle: Exploring the design space for multimodal Ilms with mixture of encoders. arXiv preprint arXiv:2408. 15998, 2025.
Oleksii Sidorov, Ronghang Hu, Marcus Rohrbach, and Amanpreet Singh. Textcaps: a dataset for image captioning with reading comprehension. arXiv preprint arXiv:2003.12462, 2020.
Vasu Singla, Kaiyu Yue, Sukriti Paul, Reza Shirkavand, Mayuka Jayawardhana, Alireza Ganjdanesh, Heng Huang, Abhinav Bhatele, Gowthami Somepalli, and Tom Goldstein. From pixels prose: A large dataset of dense image captions. arXiv preprint arXiv:2406. /0328, 2024.
Shengbang Tong. Ellis Brown, Penghao Wu, Sanghyun Woo, Manoj Middepogu, Sai Charitha Akula, Jihan Yang, Shusheng Yang, Adithya Iyer, Xichen Pan, Ziteng Wang, Rob Fergus, Yann LeCun, and Saining Xie. Cambrian-1: fully open, vision-centric exploration of multimodal llms. arXiv preprint arXiv:2406. 16660, 2024a.
Shengbang Tong. Zhuang Liu, Yuexiang Zhai, Yi Ma, Yann LeCun, and Saining Xie. Eyes wide shut? exploring the visual shortcomings of multimodal Ilms, 2024b. URL https:://aaaat org/ abs 2401.441100 06209
Michael Tschannen, Alexey Gritsenko, Xiao Wang, Muhammad Ferjad Naeem, Ibrahim Alabdul- mohsin, Nikhil Parthasarathy, Talfan Evans, Lucas Beyer, Ye Xia, Basil Mustafa, Olivier Hénaff, Jeremiah Harmsen, Andreas Steiner, and Xiaohua Zhai. Siglip 2: Multilingual vision-language encoders with improved semantic understanding, localization, and dense features. arXiv preprint arXiv:2502.14786, 2025.
Junke Wang. Lingchen Meng, Zejia Weng, Bo He, Zuxuan Wu, and Yu-Gang Jiang. To see is to believe: Prompting gpt-4v for better visual instruction tuning. arXiv preprint 1..052231 2023.
Haoning WU, Zicheng Zhang, Erli Zhang, Chaofeng Chen, Liang Liao, Annan Wang, Kaixin Xu, Chunyi Li, Jingwen Hou, Guangtao Zhai, Geng Xue, Wenxiu Sun, Qiong Yan, and Weisi Lin. Q-instruct: Improving low-level visual abilities for multi-modality foundation models. arXiv preprint arXiv:2311.06788 2023.
Zhiyu Wu, Xiaokang Chen, Zizheng Pan, Xingchao Liu, Wen Liu, Damai Dai, Huazuo Gao, Yiyang Ma, Chengyue Wu, Bingxuan Wang, Zhenda Xie, Yu Wu, Kai Hu, Jiawei Wang, Yaofeng Sun, Yukun Li, Yishi Piao, Kang Guan, Aixin Liu, Xin Xie, Yuxiang You, Kai Dong, Xingkai Yu. Haowei Zhang, Liang Zhao, Yisong Wang, and Chong Ruan. Deepseck-v12. Mixture- of-experts language models for advanced multimodal understanding. arXiv preprint arXiv:2412. 10302, 2024.
13

---

Published as a conference paper at ICLR 2026
x.ai. Grok 1.5v: The next generation of ai. 2023. Accessed: 2024-07-26.
Huanjin Yao, Jiaxing Huang, Wenhao Wu, Jingyi Zhang, Yibo Wang, Shunyu Liu, Yingjie Wang, Yuxin Song, Haocheng Feng, Li Shen, and Dacheng Tao. Mulberry: Empowering mllm with ol-like reasoning and reflection via collective monte carlo search. arXiv preprint
arXiv:2412.18319, 2024a.
Yuan Yao, Tianyu Yu, Ao Zhang, Chongyi Wang, Junbo Cui, Hongji Zhu, Tianchi Cai, Haoyu Li, Weilin Zhao, Zhihui He, Qianyu Chen, Huarong Zhou, Zhensheng Zou, Haoye Zhang, Shengding Hu, Zhi Zheng, Jie Zhou, Jie Cai, Xu Han, Guoyang Zeng, Dahai Li, Zhiyuan Liu, and Maosong Sun. Minicpm-v: A gpt-4v level milm on your phone, 2024b. URL https: /ttp://aaxvvvioio
abs/2408 01800
Yuanyang Yin, Yaqi Zhao, Yajie Zhang, Ke Lin, Jiahao Wang, Xin Tao, Pengfei Wan, Di Zhang, Baoqun Yin, and Wentao Zhang. Sea: Supervised embedding alignment for token-level visual- textual integration in milms. arXiv preprint arXiv:2408. 1813, 2024.
Weihao Yu, Zhengyuan Yang, Linjie Li, Jianfeng Wang, Kevin Lin, Zicheng Liu, Xinchao Wang, and Lijuan Wang. Mm-vet: Evaluating large multimodal models for integrated capabilities. arXiv preprint arXiv:2308 02490, 2024.
Xiang Yue, Yuansheng Ni, Kai Zhang, Tianyu Zheng, Ruoqi Liu, Ge Zhang, Samuel Stevens, Dongfu Jiang, Weiming Ren, Yuxuan Sun, Cong Wei, Botao Yu, Ruibin Yuan, Renliang Sun, Ming Yin, Boyuan Zheng, Zhenzhu Yang, Yibo Liu, Wenhao Huang, Huan Sun, Yu Su, and Wenhu Chen. Mmmu: A massive multi-discipline multimodal understanding and reasoning benchmark for expert agi. arXiv preprint 1. 2024.
14

---

Published as a conference paper at ICLR 2026
Appendix
A REPRODUCIBILITY STATEMENT
detail experimental configurations, hyperparameterr, and training procedures in Section We release our code, model weights, and datasets in http:://tttbb..ooommmuuvvvttt32f
B USE OF LARGE LANGUAGE MODELS
We employed Large Language Models solely for refining the linguistic presentation and improving the clarity of our manuscript. LLMs were not involved in the conception of architectural innovations, experimental design, any technical contributions presented in this work.
C KEY PARAMETERS ABLATION STUDIES
In this section, conduct additional ablation experiments to perform deeper investigations into design choices. For quick validation, we adopt relatively smaller model configurations: Vicuna 7B CLIP and LLaMA3 8B CLIP.
C.1 IMPACT OF THE NUMBER AND PLACEMENT OF SEMANTIC EXCHANGE LAYERS
We analyze the impact of both the number and insertion positions of semantic exchange layers on model performance and computational efficiency. We explore insertion strategies: inserting semantic exchange layers every 2 decoding layers, every 3 decoding layers and every 6 decoding layers. For both Vicuna and LLaMA models, these strategies correspond to approximately 15, 10 and 5 insertion points, respectively. At each insertion point, we experiment with inserting 1, 2, or 3 semantic exchange layers, while ensuring that the total number of semantic exchange layers does not exceed 20 to maintain resource consumption. The experimental results summarized in Table When the number of semantic exchange layers per insertion point is held constant, inserting every 3 decoding layers consistently outperforms inserting every 6 decoding layers across most evaluation metrics. On the other hand, increasing the number of semantic exchange layers per insertion point significantly raises computational costs without yielding substantial performance improvements. Notably, inserting 3 semantic exchange layers at each position leads to pronounced degradation in overall model performance. Based these observations, we adopt final configuration that inserts a single semantic exchange layer every 3 decoding layers, achieving a favorable trade-off between performance and efficiency.
C.2 IMPACT OF SAMPLE STRATEGY OF LATENT TOKENS
In the main results, set the number of latent tokens to {4. 16, 36, 64, 144), randomly sam- pling from this set for each training batch. We find that this random selection strategy balances the strengths of different token quantities and accelerates model convergence. As shown in Table evaluate both fixed numbers of latent tokens and dynamic sampling from different ranges. Our re- sults show that increasing the number of latent tokens leads to notable improvements in OCR tasks. Conversely, reducing the number of latent tokens yields slight performance gains on general and vision-centric benchmarks. Furthermore, when extending the sampling range {16, 36, 64, 144, 576}. the model does not exhibit significant performance improvements. We hypothesize that, with 576 latent tokens, the use of localized attention limits each operation to only one token, which may reduce information density and consequently limit performance gains.
C.3 IMPACT OF THE MAPPING LOsS WEIGHTT ON MODEL PERFORMANCE
In this section, we investigate the effect of the weighting parameter in the Oual-Semantic Mapping Loss, specifically associated with the reconstruction cosine similarity component. By varying the value A, we aim to understand its influence on modality alignment and overall model performance. Figures 6al and Figure 6b illustrate the training loss and reconstruction similarity of the LLaMA3
15

---

Published as a conference paper at ICLR 2026
Table Effects of the number and placement of semantic exchange layers. We investigate the effect
of insertion strategies. Specifically, we experiment with three insertion intervals(2.3.6) and three number of semantic exchange la,er)(1,2.3).
SeedImuge
interval
per layers
MMB
MMB
VizWiz
POPE
MM-
MMR
MME
ILAVA
CVBench
MMVP
AI2D
MMMU
SQA
Tex
ChanQA
Vicuna-7B
+ Clip
69.4
61.6
58.4
86.7
38.3
1536.2
331.7
68.6
77.5
65.6
66.0
64.3
36.9
70.1
60.8
51.8
20.0
62.1
59.9
87.2
38.9
1564.1
326.4
69.8
76.4
65.8
65.7
64.9
36.8
71.4
61.2
52.2
3
2
68.8
61.1
59.3
86.9
37.9
1521.9
312.4
70.1
78.2
63.5
65.3
64.2
35.9
70.3
60.2
51.3
1
70.4
61.2
56.4
86.5
39.2
1537.2
356.3
68.1
74.4
66.4
64.9
64.8
35.2
70.9
60.0
52.4
2
69.5
61.6
58.8
87.0
37.4
1552.3
307.9
68.9
69.7
65.7
66.1
64.6
36.9
72.0
59.8
50.7
3
68.1
60.7
56.4
85.9
39.0
1498.6
300.7
67.9
71.4
63.7
64.9
62.7
35.0
69.2
61.6
52.7
LLMMA388B
+ Clip
73.7
72.4
87.3
1569.7
352.3
73.4
78.8
69.9
71.9
77.8
55.8
74.9
72.2
61.2
87.9
44.8
1558.3
3618
73.0
82.7
69.7
70.0
71.7
39.2
78.3
63.6
55.5
2
73.8
71.6
58.8
87.7
45.3
1537.6
334.5
72.1
84,2
67.8
70.2
71.2
39.9
77.7
62.2
54.3
1
73.6
73.3
61.4
87.5
43.5
1551.2
383,9
71.9
79.4
69.9
68.5
70.8
39.4
78.0
62.8
55.5
2
73.1
72.0
59.2
87.0
45.0
1533.1
349.8
72.7
75.8
68.4
69.3
72.0
37.3
78.4
61.9
55.4
3
72.5
70.8
$7.5
86.9
43.3
1518.7
322,4
71.1
77.5
67.6
69.0
69.7
38.5
78.4
62.4
53.6
Table 9: Impact of sample strategy of latent tokens. We examine how different configurations of latent tokens affect model performance. Our experiments include both fixed-length sample settings (64, 144, 256) and dynamic-length sample configurations (ranging from to 256, and 16 to 576 tokens).
SeedInagg
MMB
MMB
Vi2Wzz
POPE
MM-Ver
MMBE
MME
LLaVA
CVBench
MMVP
A12D
MMMU
SQA
TeaVQA
ChurQA
Latent Token
Vicuna- 7 Clip
69.2
60.1
38.8
1548.2
322.7
69.6
66.3
65.4
36.9
59.9
144
69.5
61.9
60.3
38.4
1557.1
335.8
70.2
65.4
65.6
63.7
37.7
70.2
60.7
52.0
576
68.8
61.2
58.7
87.0
40.1
1538.5
331.5
69.4
69.6
64.3
63.8
64.2
34.6
68.8
62.1
51.8
4.16,36.64,144
70.2
62.4
59.9
87.3
39.2
1573.2
342.1
70.1
75.4
66.2
65.1
64.9
37.2
71.7
60.7
52.6
16,36,64,144,576
69.3
61.1
57.6
87.3
39.9
1547.2
324.5
69.7
71.3
65.3
64.4
64.3
35.3
69.6
61.1
52.2
LLaMA3-8B
Clip
73.8
61.4
45.6
1535.9
356.8
72.7
68.8
71.1
41.0
62.9
$4.0
144
74.0
71.1
59.7
87.7
45.5
1579.8
351.4
72.6
81.8
69.0
69.3
70.6
41.5
77.5
62.6
54.3
576
72.0
69.9
60.7
87.7
44.9
1522.6
313.9
71.3
73.6
67.5
69.4
71.0
38.5
75.4
65.2
57.9
4.16,36.64,144
74.7
72.6
61.2
87.8
45.1
1562.5
340.6
72.7
79.9
69.7
70.3
71.7
39.1
78.4
64.4
57.7
16.36.64.144.576
72.2
70.5
60.6
87.6
45.9
1548.2
336.9
71.9
69.1
69.1
39.9
76.8
62.6
58.1
8B + CLIP model under different settings. Additionally, Table 10 summarizes the performance metrics corresponding to each À configuration. As increases, the overall training loss also tends to increase, with loss curves shifting horizontally along the x-axis, indicating consistent trends across different settings. The reconstruction similarity exhibits similar pattern. However, the results in Table I suggest that setting À either too small or too large leads to suboptimal performance. When is too small, the alignment between modalities is weak; conversely, an excessively large À causes the model overfit to the modality alignment objective. Empirically, we find that setting À = 0.1 yields the best performance, balancing alignment and generalization effectively.
C.4 EFFECTIVENESS OF TRAINING STRATEGY
In our main experiments, we unfreeze all parameters every training stage. To assess the signif- icance of this strategy, we conducted ablation studies by selectively freezing certain components of the model during different training phases. To reduce computational cost, we adopt the Phi-3.5- mini-instruutt+igl...
configuration and train each variant for 8000 steps in each stage. As shown in Table IT fully unfreezing all parameters in each stage consistently results in superior performance. We attribute this the fact that framework departs from the conventional usage patterns of both
16

---

Published as a conference paper at ICLR 2026
Thaninn LEs DurvEs for Verbou
Thaning gpeeen Crsane
(a) Training loss for different X.
(b) Alignment similarity for different A.
Figure Comparative analysis of training loss and alignment cosine similarity with varied A.
Table 10: Impact of the mapping loss weight. We vary the coefficient À of the Mapping Loss to investigate how the degree of modality reconstruction and alignment affects overall model perfor- mance.
SoedImage
MMB
MMBON
VizWiz
MM-Vet
MMEE
MMEC
LLAVA
CVBench
MMVP
MMMO
TeuVQA
ChanQA
POPE
AI2D
SQA
Vicuno-7B + Clip
68.8
60.4
87.0
37.9
1550.3
322.1
69.0
74.2
65.0
64.4
36.4
70.9
52.4
69.5
87.4
38.1
1576.8
322.4
71.0
64.3
63.6
37.5
70.3
51.6
70.0
62.1
59.9
87.2
38.9
1564.1
326.4
69.8
76.4
65.8
65.7
64.9
36.8
71.4
61.2
52.2
0.5
68.9
61.7
57.3
87.0
39.5
1536.8
348.3
70.2
72.0
66.0
63.2
64.3
36.1
69.5
60.3
52.7
1.0
67.9
60.5
55.2
86.4
37.7
1511.4
308.2
68.3
68.8
63.7
64.1
62.9
35.6
69.9
58.2
50.1
LLoMA3-8B
Clip
73.8
71.4
58.9
87.6
45.6
1568.6
350.4
72.3
78.9
67.0
68.3
71.2
40.0
76.2
63.0
$4.9
0.01
75.2
71.8
62.0
87.6
45.3
1546.3
357.1
72.2
79.8
70.3
70.2
71.1
38.6
77.9
63.3
54.5
0.1
74.9
72.2
61.2
87.9
44.8
1558.3
361.8
73.0
82.7
69.7
70.0
71.7
39,2
78.3
63.6
55.5
0.5
74.1
71.3
59.3
87.2
45.1
1556.3
372.9
71.3
77.6
67.5
67.4
72.1
38.8
77.1
62.4
54.6
1.0
72.6
70.8
58.7
86.9
43.3
1532.9
337.6
69.9
79.3
65.8
68.2
70.3
37.7
75.3
63.2
55.9
the vision encoder and the language model. As a result, the model must continually adapt new processing paradigm, which requires end-to-end optimization across all components.
D MORE ALIGNMENT VISUALIZATION
To provide more intuitive understanding of how FLARE's components work in synergy achieve deep cross-modal alignment, we present another detailed visualization in Figure 7 This example illustrates the model's dynamic, query-aware attention mechanism when presented with the same image but different questions. highlights the progressive refinement of cross-modal interaction throughout our pipeline, from initial encoding final decoding.
At the Pixel Level (Vision Encoder). The visualization clearly shows that vision-language inter- action begins at the earliest stage. When the query "Is there a snowboard in the image?", our Text-Guided Vision Encoding directs the encoder to focus its attention specifically the snow- board. Conversely, when asked about the "person," the attention dynamically shifts to concentrate on the skier's body. This makes the visual feature extraction process itself context-dependent and query-aware, rather than static, one-size-fitee fits-al operation.
At the Space Level (Projector). As the features pass through the projector for modality-level alignment, the semantic focus is maintained and sharpened. The attention maps show that the fea-
17

---

Published as a conference paper at ICLR 2026
Table 11: Effectiveness our training strategy. We validate the effectiveness of our training strategy by comparing different wwaamtter-freeiigg schemes. Specifically, in Stage 1, when parameters are not fully unfrozen, only the projector and interaction layer are updated. In Stages 1.5 and 2, partial unfreezing involves training the projector, interaction layer, and the LLM, while keeping encoder fixed.
Stagel
Stage
Stage2
MMB
MMBOX
VizWiz
POPE
MM-Vet
MMEE
MMB
LLAVA
CVBench
MMVP
A12D
MMMU
SQA
TexIVQ
CharQQA
70.4
62.6
51.9
85.6
41.1
1396.8
272.4
72.7
63.2
66.8
69.5
38.9
73.4
63.9
62.2
70.0
62.4
51.4
85.9
40.2
1379.1
307.9
70.6
62.5
66.9
69.2
38.1
72.6
63.3
61.9
69.3
62.8
51.2
85.1
39.8
1365.8
288.2
71.8
61.4
67.3
68.8
39.2
72.1
62.9
61.6
68.9
62.7
51.3
85.1
37.7
1354.2
276.4
71.5
61.6
66.7
68.7
38.7
71.6
62.1
61.9
68.7
62.8
51.1
85.3
35.9
1368.6
275.1
69.1
60.7
66.5
68.4
38.9
71.2
61.9
61.6
tures corresponding the queried object-whether the "snowboard" or the "person"-an clearly distinguished from the background snow. This indicates a successful mapping into shared semantic space where the core concepts are well-represented and aligned across modalities.
At the Query Level (LLM). Finally, during the LLM decoding phase, our Context-Aware Align- ment Decoding mechanism produces highly focused attention map. The model's attention is now strongly concentrated on the most relevant visual evidence required to answer the question. The attention for the "snowboard" query precisely on the board, while the attention for the "person" query covers the skier, validating the effectiveness of our approach in achieving the fine-grained, question- level aligmment necessary for accurate reasoning.
SYNTHESIZED LANGUAGE-DRIVEN FRAMEWORK PROCESS
E. PROMPT DESIGN AND DATA GENERATION
We present the data generation prompts used in both the pretraining and finetuning stages. For the pretraining data, we generated five distinet types of data: SynthColor, SynthCount, SynthSpatial, SynthText, and SynthScene. Each of these datasets was generated using a single prompt, and the details of these prompts are provided in Table 14 Table I5 Table 16
Table 17 Table [8
In the finetuning phase, specifically during Stage 1.5, we generated seven categories of data, includ- ing SynthuuliChoice, SynthConvLowd SynthConvShoor, SynthReasoning. SynthTextQA, Synth- ContrastLong, and SynthContrastthort. Among these, SynthMultiChoice, SynthConvLong. Synth- ConvShort, and SynthRcasoning were first created based on captions to generate descriptions. These descriptions were then used to generate the corresponding QA pairs. The details of these prompts are provided in Table[24] Table [2] T9 A more complex procedure was employed for SynthTextOS Given the complexity of the OCR task and the limitations posed by caption elements, we first generated SynthText from the caption. Then, SynthText was used to generate Descriptions, and finally, the QA pairs were produced. This approach ensures that the generated descriptions con- tain text elements, resulting in higher-qualiyy QA pairs. The prompts are provided in Table 28 29 For the SynthContrastLong and SymthContrastShort categories, the challenge was highlight the detailed contrast between similar images. To address this, we carefully designed a prompt capable of generating two descriptions, each corresponding to an image, along with QA pairs that emphasize the differences between these two descriptions. The design of these prompts are presented in Table 27 26 The primary focus of the prompts in this phase was to prioritize variability in QA pairs, thereby enhancing the model's generalization across diverse multimodal contexts.
In Stage 2, we generated SynthMultiChoice, SynthConyLong and SynthConvShort, using a similar process to Stage 1.5. The details of these prompts presented in Table 25 21 23 The design of the prompts in this stage focused on enhancing the model's 's ability to follow specific instructions for various tasks.
18

---

Published as a conference paper at ICLR 2026
snowboard
showboard
snanboard
there person the
Input
Vision Encoder pizel-level align
Projectorr space-level dlign
LLM
question-level
Figure 7: Visualization of query-aware attention maps at different stages of the FLARE pipeline. The top row corresponds to the query about the "snowboard," and the bottom row corresponds to the query about the "person." The heatmaps show that attention dynamically shifts and sharpens based on the textual context, demonstrating true vision-language integration.
We present specific examples and forms of the different data types in Figures 14
TO
For all image generation, employ FLUX-dev with guidance steps set to 50. For all descriptions and question-anse answering tasks, we utilize LLAMA3.1-70B -00tiitutttttt The use of open-source models ensures that entire pipeline is easily reproducible.
E.2 QA PAIRS DATA FILTERING
Generative models inherently introduce challenges, including ambiguity, missing information, and inconsisten cies in generated content. To address these issues, implement rigorous, multi-stage filtering process.
The pretraining data filtering process adopt follows the approach outlined in (Liu et al. 2025 We primary focus on the selection of finetuning data. Our filtering procedure consists of four stages, each designed ensure the quality and relevance of the data used in subsequent training steps.
E.2.1 CAPTION FILTERING
The first step focuses selecting high-quality captions for generating the QA pairs. The captions are sourced from the Datacomp (Gadre al., 2023) dataset, which contains significant amount of low-quality or irrelevant data. Prior to generating descriptions, we conduct filtering process to remove low-quality captions, such as those containing watermarks, advertisements, or non- -nnlisl text. Initially, rule-based filtering methods applied, including:
Removal of character-level and word-level repetitions,
Removal of captions with high perplexity values,
Ensuring strong image-text alignment between the caption and its corresponding original image.
This step ensures that the selected captions are better aligned with real-world images and more easily extensible. Subsequently, we use LLM to score these captions, ensuring their semantic correctness and syntactic richness. The scoring process emphasizes the diversity of the elements in the captions. The prompt, along with the specific rule-based methods, is outlined in Table 12 For the filtering process, we use Data-Juicer for rule-based filtering and LLaMA3.1 70B for scoring. After this step, approximately 20 million captions retained.
19

---

Published as a conference paper at ICLR 2026
Table 12: Metric and Prompt used for Caption Filtering
Caption Filtering
## Based Metrics
Alhanumerii Filter: Tokenization: false, Min ratio: 0.60
Character Repetition Filter: Rep length: 10, Max ratio: 0.09373663
Flagged Words Filter: Language: en, Tokenization: false, Max ratio: 0.0
Perplexity Filter: Language: en, Max perplexity: 5500.0
Special Characters Filter: Min ratio: 0. 0.16534802, Max ratio: 0.42023757
Word Repetition Filter: Language: en, Tokenization: false, Rep length: 10, Max ratio:
0.03085751
Image-Text Matching Filter: HF BLIP: salesforcchhip-im--b------ baa--ooo-oa Min score: 0.8,
Max score: 1.0. Horizontal flip: false, Vertical flip: false, Reduce mode: avg. Any all:
any, Mem required: 1500MB
Image-Tex Text Similarity Filter: HF CLIP: openailclii-viibbsseeaathhh2 Min score: 0.28
## Prompt
Assume you are an expert the field of Al image generation. Your goal is to select high-descriptive prompts that will enable the successful generation of images. will provide you with a specific de- scriptive prompt, and your task is to evaluate it thoroughly. Consider the prompt's level of detail, its logical coherence, and the clarity with which it describes the desired image. is essential to whether the prompt contains sufficient information to guide the diffusion model effectively, ensuring that it can produce an image that meets expectations. You should only respond with Yes or No.
E.2.2 DESCRIPTION FILTERING
The second step focuses on selecting high-quality descriptions for generating QA pairs and images. The 10 million captions selected Step are input into LLaMA3.1 70B to generate corresponding descriptions. A similar filtering approach is employed for descriptions in Step 1, with the key difference being the exclusion of image-text matching calculations. After filtering, approximately 12 million descriptions are retained.
E.2.3 IMAGE GENERATION FILTERING
This step aims to ensure the quality of the generated images and their alignment with the corre- sponding descriptions.
Let represent the generated image, and D represent the corresponding description. We first caleu- late the CLIPScore between the description D and the generated image to measure the alignment.
To assess the image quality, we utilize the Structural Similarity Index (SSIM) metric. As described in the main paper, I will be resized and processed by Siglip. Additionally. 1 will be divided into four non-overlapping sub-images, each processed individually through Siglip. The SSIM computed both between the resized image and its original version, as well as between each sub-image and its corresponding resized sub-image.
Let the height and width of the image be denoted H and W, respectively, and let the crop size used Siglip be S. The first step to calculate the SSIM between the original image / and the resized image:
SSIM,, = SSIM (7, Resize (Resize (T,, (H, W)))
(11)
For the four sub-images, we first partition the original image into four non-overlapping regions. Specifically, each sub-in sub-image Iub(ay) corresponds section of the image, where i and j {1,2) indicate the row and column of the sub-image. The extraction of the (i,j)-th sub-image is defined
20

---

Published as a conference paper at ICLR 2026
Table 13: Evaluation results across synthesized and existing datasets.
Dataset
CLIPScore
Human Evaluation
GPT-40 Evaluation
Image
Text
Image- Text
Image
Text
Image-
SynthMultiChoice
0.373
4.6
6.9
7.3
5.8
6.8
7.0
SynthConvShore
0.382
5.3
6.3
6.9
5.7
7.1
7.3
Lynth
0.379
5.1
7.1
7.2
6.1
7.9
7.6
SynthContrastthort
0.346
5.7
7.6
7.6
5.3
7.9
6.8
SynthContrasttLong
0.341
5.5
8.0
7.4
5.9
8.6
7.4
LLaVA665k
0.332
7.8
7.1
8.3
8.2
7.4
7.2
ShareGPT4V
0.313
7.7
6.8
7.2
7.9
6.6
8.1
LVIS- Instrctt
0.337
7.1
6.7
7.9
8.5
7.8
7.7
as:
H
H
W
W
Iab(t.))
1)
(j
1):
(12)
2
2
The SSIM between each sub-image and its resized version computed as:
SSIM,(I,)) SSIM (Tsab(t.))>
(13)
И W
Resize [Resize(/sub(ij)- (S,
2
Next, compute the overall SSIM score for the entire image by combining the SSIM of the full image (SSIM,)) and the SSIM values for each of the four sub-images (1,):
SSIM, x SSIM +
U2 x SSIM,
(14)
where W1 is the weight for the SSIM of the full image and ty is the weight for each sub-image's SSIM. These weights are chosen to appropriately balance the contribution the full image and sub-images to the overall image quality score.
Finally, the total score for the image computed by combining the CLIPScore with the overall SSIM score SSIM,, using weighting factor X:
Total Score CLIPScore (I, D) + À X SSIM
(15)
In our implementation, wi is set to and w2 is set to 0.25. set to 0.5 to banlance the importance of the SSIM score and CLIPScore. After this filtering step, retain approximately 6 million images.
E.2.4 QA PAIR FILTERING
The final step involves ensuring the quality and relevance the generated QA pairs. We begin by using MILM to verify the correctness of the answers within each QA pair, filtering out all incorrect QA pairs. Subsequently, we convert each QA pair into a declarative sentence and compute CLIPScore between the statement and the corresponding image. The final score for QA pair is the average the CLIPScores for each declarative sentence within the QA pair:
N Σ CLIPScore(QA.
1
Final Score
(16)
where N is the number of questions in the QA pair. The QA correctness is evaluated using the LLa -LaVA--OnViiioon 72B, while the transformation to declarative sentences is carried out with LLLMA3.1-70B. After filtering, retain approximately 4 million QA pairs.
21

---

Published as a conference paper at ICLR 2026
(a) Visualization of text embeddings in pretrain.
(b) Visualization of image embeddings in pretrain.
Figure 8: TSNE visualizations of synthetic and real datasets during pretraining for text and image modalities.
(a) Visualization embeddings in finetune.
(b) Visualization of image embeddings in finetune.
Figure 9: TSNE visualizations of synthetic and real datasets during fine-tuning for text and image modalities.
E.3 SYNTHESIZED DATA EVALUATION
We assess the quality of our synthesized datasets using a combination of heuristic metrics, human evaluation, and automated model-based scoring, with comparisons existing datasets. Follow- ing our data filtering procedure, QA pairs are reformulated into declarative statements to facilitate image-text alignment evaluation via CLIPScore. To further evaluations and leverage GPT-4o to score each sample on image quality, text quality, and amage-text alignment, using 0-10 rating scale.
data quality, we conduct human
The synthesized datasets include SynthMultiChoice, SynthConvShort Short, SynthConyLong, SynthCon- trastLong, and SynthContrastShor., For comparison, we use existing datasets such as LLa VA665k, ShareGPT4V, and GPT-4o evaluations are conducted on 50 randomly sampled examples per dataset.
CLIPScore is calculated across the full dataset, while human and
As summarized in Table 13 our synthesized datasets achieve higher CLIPScores than existing datasets. This result is closely linked our filtering process, which selects for better-aligned im- age-text pairs
Regarding text quality, synthesized data performs comparably to real datasets, suggesting that our generation process maintains strong linguistic fluency and coherence. In particular, the Synth- Contrastive subsets exhibit notably strong performance in text quality evaluations, indicating that contrastive instruction formats may enhance the diversity of generated content by encouraging ied and contextually nuanced textual outputs. For image-text alignment, synthesized data shows slightly lower scores than real data, likely due limitations in image generation models capturing
22

---

Published as a conference paper at ICLR 2026
fine-grained visual details. Notably, image quality remains key weakness: synthesized images are consistently rated lower than real images, which typically consist of high-resolution, naturally captured content.
While our synthesis pipeline effectively produces aligned and coherent multimodal data, improve- ments in image generation needed to match the quality of real datasets. Future work may incor- porate higher-capacity generative models, visual post-processing or human-in-the-lop, refinement bridge this gap.
E.4 T-SNE VISUALIZATION
We employ t-SNE to reduce the dimensionality of text and image embeddings, enabling comparison between synthetic and real datasets. Figures Sa 8b 9a 9b visualize both pretraining and fine-tuning distributions for text and image modalities.
For clarity, we first focus our analysis on text embeddings. In Figure Sa synthetic datasets cluster closely with their real counterparts. indicating strong semantic alignment. Notably, SynthTextVQA extensively intermixes with LLaVA-558K, and ShareCaptioner, suggesting that the generated data ef- fectively capture the linguistic style and content distribution of existing data. SynthSpatial exhibits slightly broader dispersion beyond the densest regions of CC12M and ShareCaptioner. In Fig- ure 9a dialogue-style datasets (SynthConvLonn, SynthConvShorr) align tightly with LLaVA-665k and LVISInstruut44-2220K indicating successful emulation of conversational exchanges. Multiple- choice synthetics (SynthChoice) partially overlap with LLa VA-665K, confirming stylistic and struc- tural consistency. The distinct distributional patterns of contrastive prompts, compared to existing data types, indicate their potential to enrich the fine-tuning corpus with novel and semantically con- sistent textual variations.
The image embeddings exhibit analogous patterns. For the pretraining stage, our synthetic images align closely with LLaVA-558k and CC12M, demonstrating effective replication of the distribu- tional properties found large-scale image-caption datasets. For the fine-tuning stage, the synthetic image data show strong overlap with COCO and SAM, indicating consistency with real-world visual instruction datasets.
However, we observe notable distributional gaps between our synthetic images and datasets such as ScienceQA, suggesting room for improvement capturing the oppeif- visual features.
Overall, the t-SNE visualizations confirm that our synthetic text datasets closely replicate the mantic structures of real corpora while introducing complementary variations.
23

---

Published as a conference paper at ICLR 2026
Table 14: Prompt used for SynthColor
SymthColor
Role
Assume you are Al visual assistant.
## Goal
will provide you with brief description image. primary is accurately construct this image in your mind. Based on your understanding and the basic image, you need to adjust the colors of objects to enhance the expressiveness and visual information of the image. If the original description is vague or insufficient, creatively adjust or add color elements to improve clarity and impact.
## Rule
To maintain harmony:
The number of color types should not exceed three
The overall description should be concise
Colors should be diverse, avoiding repetition
After modifying. you need provide a new description that specifies the color of each significant object. Ensure that the new description is:
Logically consistent
Overall harmonious
Focused on visual information
Minimizing irrelevant content
No more than 12 words
nn Output Format Present the description content directly, without any extrancous information.
Table 15: Prompt used for SynthCount
SynthCount
## Role
Assume you are Al visual assistant.
## Goal
will provide you with a brief description of an image. Your primary task to accurately construct this image in your mind. Based on your understanding and basic image, you need modify visual information the image.
quantity objects to enhance the expressivcness original description is vague insufficient, ereatively adjust or add elements improve elarity and impact.
## Rule
To maintain harmony:
The quantity of each object type should not exceed three
The total number of objects should not exceed six
After modifying. you need provide description that quantifies each signiticant object type. Ensure that the description
Logically consistent
Overall barmonious
Focused on visual information
Minimizing irrelevant
No more 12 words
Output Format
Present the description content directly.
some possible formats:
1. Three balloons float above table where two children play board games.
2. Two dark mountains loom above the water with three people at shoreline.
24

---

Published as a conference paper at ICLR 2026
Table 16: Prompt used for SynthSpaital
Santal
## Role
Assume you Al visual assistant.
## Goal
will provide you with brief description an image. Your primary task is to accurately construct this image in your mind, focusing particularly on the spatial arrangement and relative positions of objects within it. Based on your understanding, you need to modify the positions of objects to highlight their interactions and improve spatial coherence. If the original description lacks detail on positioning. creatively adjust add elements clarify spatial relationships and enhance visual impact. ## Rule
Ensure that arrangement is logical and maintains aesthetic harmony. After modifying, you need provide new description quantifies each significant object type. Ensure the new description is:
Logically consistent
Overall harmonious
Focused on visual information
Minimizing irrelevant content
No more than words
## Output Format
Present the description content directly. Here
possible formats:
1. Bicycle leaned against the left side of park bench under a tree.
2. Bed against the right wall, with luggage beside it, near curtained windows.
Table 17: Prompt used for Synth Text Text
TexthText
## Role
Assume you are Al visual assistant.
## Goal
will provide you with brief description of image. Your primary task is to accurately construct this image in your mind. Based on your understanding and the hasic image, you need to add textual content enhance the expressiveness and visual information of the image, and these texts should be closely related to the theme and visual elements of the image.
## Rule
The textual information can include:
Words and numbers on signs
Billboards
Posters
Slogans
Additionally, you can add other types of text needed to ensure the diversity and richness of the text.
You also create visual content appropriately if the given description is not suitable for adding text. After adding text, you need to provide new description that includes the text you added. Remember:
Your response must contain text
You must specify the type for each text
The description should be logically consistent
No more than 12 words
Output Format
Present the description content directly. Here are some possible formats:
1. A vibrant mural displays Dream Big!With colorful butterflies and stars.
street sign reads Main Stwith red arrow pointing right.
25

---

Published as a conference paper at ICLR 2026
Table 18: Prompt used for SynthScene
SynthSeene
Role
Assume you are an Al visual assistant.
## Goal
I will provide you with brief description of image. Your primary task is to accurately construct this image your mind. Based on your understanding and the basic image, you need adjust or enhance the scene elements to improve the expressiveness visual information the image.
## Rule
If the original description is vague or insufficient, creatively adjust or add scene elements such as:
Landmarks
Iconic buildings
Natural landscapes
Urban settings
Specific environments
Historical sites
Famous monuments
Picturesque vistas
Distinctive locations
Remember:
Each caption must contain only one scene element
Scene elements should be diverse (avnid repetition like multiple Eiffel Towers)
Description should be logically consistent and harmonious
Focus on visual information, minimize irrelevant content
Response must no more than 12 words
## Output Format
Present the description content directly, without any extraneous information.
Table 19; Prompt used for SynthReasoning
Reasoning
## Role
Suppose you
Al visual assistant, viewing detailed description of image.
## Goal
The task is to use the provided description of an image, create some plausible questions about the im- age, and provide the answers in detail. You can create complex questions beyond describing the scene. Make the question challenging by not including the visual content details in the question so that the user needs reason about that first. To answer such questions, you should require first understanding the visual content, then based on the background knowledge, think through the solution step by step. Either explain why the things are happening that way, or provide guides and help to user's 's request. ## Rule
Please give the Q&A content directly without any extrancous words and separate questions and an- swers with Q and A. Ensure that the answer cach question less than 100 words. Additionaly, you should control the QA pairs not exceeding 6 pairs to preserve visual clarity.
26

---

Published as a conference paper at ICLR 2026
Table 20: Prompt used for Stage1.5 SynthConvLong
Stage Stagel.5--yymiie Sy5-Syythooooooooo
## Role
Suppose you Al visual assistant, viewing detailed description of image. ## Goal
Design a conversation where user asks wide range of questions about an image, and the Al visual assistant responds based on what it sees. The questions should cover a broad spectrum of aspects related to the image, ensuring variety and inclusivity of different types of inquiry. The aim is to explore the image from multiple angles and include different kinds of questions to create diverse, dynamic conversation.
## Rule
The questions should explore the following aspects:
Object types: What objects are visible in the image? What are their characteristics?
Counting objects: How many of a certain type of object or entity can be seen in the image?
Object actions: Are objects being used manipulated in any way?
Object locations: Where are specific objects located relative to one another in the image?
Relative positions between objects: How are objects positioned relation one another in scene?
Background Elements: What is the hackground environment like? What elements fea- are visible in the backdrop?
Lighting & Color: What kind of lighting or color schemes dominate the image? How does the lighting affect the mood or feel of the scene?
Textures and Details: Are there any significant textures small details the image that stand out?
Ensure that only questions with clear, confident answers included:
1. The content referenced in the question is clearly visible in the image, and the answer can be confidently provided.
2. clear from image that content not present.
What's 's more, the conversation should span these different dimensions, including variations in the types of questions
the dialogue explores rich variety of observations about the image. Ensure the conversation logically consistent, focuses on describing the visual information. and avoids unnecessary distractions. Keep the number of QA pairs to no more than 8 maintain clarity. ## Example Output Format
Q: How many people visible in the image? A: there two people visible, sitting desk and the other standing the window.
Q: What are the people doing in the image?
A: The person sitting at the desk appears to be typing on laptop, while the other person is holding cup. likely standing near the coffee machine.
Q: What is background the image?
A: In the background, there's a large window with some greenery visible outside, suggesting it's sunny day.
Q: How room lit?
A: The room is brightly lit by natural sunlight streaming through the window, with soft shadows from the furniture.
Is there any or decoration the walls?
A: Yes, there is large modern painting hanging on above the desk.
27

---

Published as a conference paper at ICLR 2026
Table 21: Prompt used for Stage2 SynthConvL.ong
Stage2-SynthConvLoong
## Role
Suppose you are Al visual assistant, viewing a detailed description of an image. ## Goal
Design conversation where user asks questions about an image, and the Al visual assistant responds with high precision and specificity, based solely what is clearly visible in the image. Each answer should be directly aligned with the visual content, ensuring that is 100% accurate and reflects the scene as it appears. The questions should be carefully framed avoid any speculative or unclear answers.
## Rule
Key aspects to focus
Object types: Identify objects and details based on their distinct visual attributes, ensuring the description matches what is observable in the image.
Counting objects: Provide exact numbers of objects or entities visible, with guesswork.
Object actions: Only describe actions that can be clearly observed in the image, with no assumptions.
Object locations:
Describe precise spatial relationships between objects based on their po- sitioning in the image.
Relative positions between objects: Indicate objects are arranged in relation to one another, using clear visual evidence from the image.
Ensure that only questions with clcar, confident answers are included:
1. The content referenced the question is clearly visible in the image, and the answer can confidently provided.
2. It is clear from the image that the content not present.
What's more, avoid asking questions that cannot be confidently answered from the image. The con- versation include more complex questions, such asking about the background knowledge of the objects in the image discussing events happening in the image. However, make these questions are based on clear visual evidence from the image and not uncertain speculation. Ensure that con- versation logically consistent, focuses on describing the visual information, and avoids unnecessary distractions. Keep the number of QA pairs to no more than 8 to maintain clarity. ## Example Output Format
How many lamps are visible A: here are two lamps, both on either
image?
room, near the walls.
Q: What color is the couch?
A: The couch is deep brown color, with smooth leather texture.
Q: Where is the desk located the image?
A: The desk is located against the far wall of the room, with a chair placed in front of it.
Q: What is the shape the rug?
A: The rug rectangular, positioned underneath the coffee table.
Q: Is there a TV in the room?
A: No, there is television visible in the room.
28

---

Published as a conference paper at ICLR 2026
Table Prompt used Stagel.5 SynthConvShort
Stage 1.5- S5nSyythhoooSSorrt
## Role
Suppose you are an Al visual assistant, viewing a detailed description of an image. ## Goal
Design conversation where user asks questions about an image, and the Al visual assistant responds with brief, concise, and to to-the-point answers. The focus is providing short responses that directly match what visible avoiding long explanations or unnecessary details. The questions should cover a broad spectrum of aspects related to the image, ensuring variety and inclusivity of different types of inquiry. The aim is explore the image from multiple angles and include different kinds of questions to create diverse, dynamic conversation.
image,
elaboration.
should answer clearly
simply,
## Rule
The questions should explore the following:
Object types:
What objects visible in the image? What are their characteristics?
Counting objects: How many of a certain type object or entity can be seen in the image?
Object actions: Are objects being used manipulated in any way?
Object locations: Where are specific objects located relative to one another in the image?
Relative positions between objects: How are objects positioned in relation another in scene?
What is the background environment like? What elements fea- tures are visible in the backdrop?
Background Elements:
Lighting Color: What kind of lighting or color schemes dominate the image? How does the lighting affect the mood or feel of the scene?
Textures and Details: Are there any significant textures small details the image that stand out?
Ensure that only questions with clear, confident answers are included:
1. The content referenced the question is clearly visible in the image, and the answer can be confidently provided.
2. It is clear from the image that the content not present.
What's 's more, the conversation should span these different dimensions, including variations in the types of questions to ensure the dialogue explores rich variety of observations about the image. Ensure that the conversation is logically consistent, focuses describing the visual information, and avoids unnecessary distractions. Keep number of QA pairs to more than maintain clarity. ## Example Output Format
Q: What furmiture is in the room?
A: Sofa and table
Q: What is placed on the table?
A: Coffee cup
Q: What is the color of the carpet?
Grey
Q: Where is the clock located?
A: On the wall
29

---

Published as a conference paper at ICLR 2026
Table 23: Prompt used for Stage2 SynthConvShhott
Stage2-SynthConnSShorr
Role
Suppose you are an Al visual assistant, viewing a detailed description of an image. ## Goal
Design conversation where asks questions about image, and the Al visual assistant responds with high precision and specificity, based solely what is clearly visible in the image. The answers should as brief as possible, ensuring that they directly reflect visible elements Each answer should be directly aligned with the visual content, ensuring that is 100% accurate and reflects the scene as it appears. The questions should be carefully framed to avoid any speculative or unclear answers.
image.
## Rule
Key aspects to focus on:
Object types: Identify objects and details based their distinet visual attributes, ensuring the description matches what is observable the image.
Counting objects: Provide exact numbers of objects or entities visible, with no guesswork.
Object actions: Only describe actions that can be clearly observed in the image, with assumptions.
Deseribe precise spatial relationships between objects based on their po- sitioning in the image.
Object locations:
Relative positions between objects: Indicate how objects are arranged in relation to one another, using clear visual evidence from the image.
Ensure that only questions with clear, confident answers are included:
1. The content referenced in the question is clearly visible in the image, and the answer can be confidently provided.
2. It is clear from the image that the content is not present.
What's more, avoid asking questions that cannot be confidently answered from the image. The versation can include more complex questions, such as asking about background knowledge objects in the image or discussing events happening in the image. However, make sure these questions are based on clear visual evidence from the image and not uncertain speculation. Ensure that the con- versation logically consistent, focuses deseribing the visual information, and avoids unnecessary distractions. Keep the number of QA pairs no more than 8 maintain clarity. Example Output Format Q: Is the laptop open or closed? A: Open
Q: How many chairs are around the table?
A: Four
Q: Is there dog visible in the image?
No
Q: What is the person doing?
A: Reading
30

---

Published as a conference paper at ICLR 2026
Table 24: Prompt used for Stage1.5 1.5 SynthMuliChooie
Stage Saael..-SyytiMMtihhhiie
Role
Imagine you are an Al visual assistant tasked with analyzing a detailed description of an image.
## Goal
through understanding the image by asking multiple-choiee and yes/no ques- tions. Each question should help the explore the image from variety of perspectives. The goal to generate dynamic conversation that covers as many aspects the visual content as possible, ensuring diverse and varied inquiries.
You guiding
## Rule
The types of questions should involve range of topics, including but limited
Object types: What types of objects can be seen in the image? What are their characteris- tics?
Counting objects: How many instances a particular object entity are visible?
Object actions: Are objects being manipulated, used, or interacted with?
Object locations: Where are specific objects or entities located the image?
Relative positions between objects: How are the objects positioned in relation to one an- other in the scene?
Background Elements:
What does the background consist of? Are there any environmental features or structures visible?
Lighting Color: What types of lighting are visible? How does the lighting affect the visual atmosphere of scene?
Textures and Details: Are there any notable textures, intricate details, small visual ele- the image?
Ensure that only questions with clear, confident answers are included:
1. The content referenced the question is clearly visible in the image, and the answer can be confidently provided.
2. It is clear from the image that the content is not present.
Ensure each question brings out a different angle image, covering wide range of obser- vations and keeping the conversation engaging and informative. The questions should vary in scope, from general descriptions to more specific details, allowing the user to explore various facets of the image. Ensure that the conversation is logically consistent, focuses describing the visual informa- tion, and avoids unnecessary distractions. Keep number of QA pairs to more than to maintain clarity.
## Example Output Format
Q What is the object located in the center of the image? A. A vase B. sculpture C. A pillow D. A mirror
Q: Is the room illuminated by sunlight?
A: Yes
Q: What type of floor covering visible? A. Carpet Wood Tiles Concrete
A: A
Q: What color the chair in the corner of the room? A. Red B. Black C. Green D Yellow
D
Q: Is the person holding something in their hand?
A: NO
Q: Where is large plant in room positioned? Next to window Near door C. Beside the desk D. In the corner of the room A: C
31

---

Published as a conference paper at ICLR 2026
Table 25: Prompt used for Stage2 SynthMultiChoice
Stage2-SynthMuatiChoiee
## Role
Suppose you Al visual assistant, viewing detailed deseription of image.
#i Goal
Design a conversation where user asks questions about an image, and the AI visual assistant responds with high precision and specificity, based solely on what is clearly visible in the image. The focus is on high-quality matching where cach answer must be directly tied to the observable content in the image. The questions should ensure that the answers are clear, precise, and grounded in visible evidence, and should avoid any speculative responses.
## Rule
Key aspects to focus on:
Object types: Identify objects details based on their distinct visual attributes, ensuring
the description matches what is observable in the image.
Counting objects: Provide exact numbers of objects entities visible, with guesswork.
Object actions: Only describe actions that can be clearly observed in the image, with no assumptions.
Object locations:
Describe precise spatial relationships between objects based on their po- sitioning in the image.
Relative positions between objects: Indicate how objects arranged in relation to another, using clear visual evidence the image.
Ensure that only questions with clear, confident answers are included:
The content referenced the question is clearly visible in image, answer can confidently provided.
2. It is clear from the image that the content is not present.
What's more, each question must avoid ambiguity and the answers are firmly grounded in visual ev- idence. The conversation can include more complex questions, such as asking about the background knowledge of the objects in the image discussing events happening the image. However, make these questions based clear visual evidence from the image and uncertain speculation. Ensure that the conversation is logically consistent, focuses on describing visual information, and avoids unnecessary distractions. Keep the number of QA pairs to no more than 8 to maintain clarity. ## Example Output Format
Q: How many people are visible in the image? A. One B. Two C. Three D. None
Q: there door visible in the image?
A: Yes
Q: What the color the table in image? A. Black B, Brown White D. Blue
A:
Q: What is the object on the desk? A. A phone B. A laptop C. A coffee cup D. A lamp
A: C
Q: What the object heside the couch? A. A small table B. A lamp C. A plant D. A book
A: C
Q: Are there any curtains visible?
No
32

---

Published as a conference paper at ICLR 2026
Table 26: Prompt used for SynthContrastShort
SymthContrastShor
## Role
Suppose you Al visual assistant, viewing a detailed description of image.
#i Goal
Your primary task is to accurately construct basic visual basic visual, you need to create two descriptions of images. Both descriptions should stem from the description provided but must differ distinctly in aspects such as color, quantity, type of objects, or Then you need to create set of questions and answers that highlight the
image in your mind. Based on this
their placement and differences between the images.
Rule
Instructions for Image Descriptions:
Imagel Description: Provide detailed depiction of imagel, focusing visual elements such as the types of objects, their quantities, actions, locations, and their relative positions to one another.
Image2 Description: Develop contrasting depiction of image2 by altering key visual ele- ments. Changes can include similiar objects, differrnt color scheme, different locations, varying quantities or types objects, or a new arrangement.
Instructions for Questions and Answers:
Create a set of questions and answers that effectively highlight the differences between the two images. For clarity, in QA Section, refer to them as the 'left' (Imagel) and 'right"
(Image2) images.
The questions should inquire about specific differences in visual aspects such as color, quan- tity, or arrangement of objects.
Answer the questions using single word phrase.
## Example Output Format
Output Format:
Imagel: The image depicts [Detailed visual description the imagel], ensuring the de- scription is no less than 120 words and no more than 180 words.
Image2: The image describes [Detailed visual description of the image2], ensuring it clearly differs from Image the description is no than 120 words no more than words.
QA Section Example: (You should focus more the visual differences between the two images)
Q: What objects are present in both the left right images?
A: Trees and house
Q: What objects are left image but not the right image?
A: A sedan car
Q: What objects are right image but not left image?
A: A bicycle
Q: How do the colors objects differ between the two images?
A: Left image has green trees, right image has palm trees in blue
Q: Does the image have more trees than right image?
A: No
Q: the quantity of objects different between the two images?
A: Yes
Q: Do both images feature pedestrians?
A: Yes
can design other questions to explore additional aspects, such as context objects, their arrangement, any textual elements, or the mood conveyed by the colors and lighting in each image. Additionally, you should control the QA pairs not exceeding pairs preserve visual clarity. Please present the descriptions and QAs directly, without any extraneous information.
33

---

Published as a conference paper at ICLR 2026
Table 27: Prompt used for SynthContrast..ng
SynthContrastL.ng
## Role
Suppose you are an Al visual assistant, viewing a detailed description of an image. ## Goal
Your primary task is to accurately construct the basic visual of the image in your mind, Based on this basic visual, you need create descriptions of images. Both descriptions should from the description provided but must differ distinctly in aspects such color. quantity, type of objects, or their placement and so on. Then you need to create set of questions and answers that highlight the differences between the two images.
## Rule
Instructions for Image Descriptions:
Imagel Description: Provide a detailed depiction of imagel, focusing visual elements such as the types of objects, their quantities, actions, locations, and their relative positions to another.
Image2 Description: Develop a contrasting depiction of image2 by altering key visual elc- ments. Changes can include similiar objects, different color scheme, different locations, arrangement.
varying quantities types of objects,
Instructions for Questions and Answers:
Create set of questions and answers that effectively highlight the differences between the two images. clarity, QA Section, refer them as the 'left' (Imagel) and 'right' (Image2) images.
The questions should inquire about specific differences in visual aspects such color, quan- tity, or arrangement objects.
The answers should provide clear and concise explanations, directly referencing the visual disparities between the left and right images.
Example Output Format
Output Format:
Imagel: The image depicts [Detailed visual description of the imagel], ensuring the de- scription no less than 120 words and no more than 180 words.
Image2: The image descrihes [Detailed visual description of the image2], ensuring it clearly differs from Image the description is no less than 120 words and more than 180 words.
QA Section Example:
Q: What the differences in the objects found in the left and right images?
A: The left image contains [specific objects and features in Imagel], while the right image includes [specific objects and features in Image2].
Q: What the color difference hetween objects in the left and right images?
A: The left image features [description of colors in Imagel], whereas the right image has [description of colors in Image2].
Q: How does the quantity of objects differ hetween the two images?
A: In the left image, there [number and types of objects in Imagel], compared to [number and types of objects Image2] in the right image.
You can design other questions to explore additional aspects, such as the context of the objects, their arrangement, any textual elements, the mood conveyed by the colors and lighting in each image. Additionally, you should control the QA pairs not exceeding pairs preserve visual clarity. Please present the descriptions and QAs directly, without any extraneous information.
34

---

Published as a conference paper at ICLR 2026
Table 28: Prompt used for SynthTex(QA-Stepl
SynthTex(QA-Stepl
## Role
Suppose you are Al visual assistant, viewing a detailed description of an image.
## Goal
Your primary task accurately construct the image your mind. Based on your understanding and basic image, you need further expand and enrich content the image.
construction
## Rule
You need to add textual content to enhance the expressiveness and the visual information of the image, ensuring that the text becomes the main subject of the image and is closely related the theme and visual elements the image. The textual information can include:
Words and numbers on signs
Billboards
Posters
Digital displays
Graffiti
Slogans
Traffic directions
Road signs
Historical markers
Famous quotes
To maintain harmony:
The image contains no more than three textual elements.
The total word count shuold not exceed twelve words.
After adding text, you need to provide new image description that includes text you added. En- sure that the new description is logically consistent, overall harmonious, and focuses on describing the visual information in the image, minimizing the interference of irrelevant content. Your description must be thorough and specific enough to abling it accurately reproduce the imagined scene.
visual guide for image generation model,
## Example Output Format
Please ensure your response is no less than 110 words and more than 150 words, and present the description content directly, without any extraneous information. Here are some possible formats:
1. A pink heart button displays "Love Yourself" cursive, bold font, against pastel-colored background. On a nearby wall, motivational poster titled "Self-Love Journcy" in modem,, sans-serif font, features stylized illustration of a person embracing their reflection. Below the poster, small sticker adding a touch whimsy the scene. The surrounding environment minimalistic, with few scattered flowers and a faint, gradient glow, emphasizing the uplifting and empowering message of the image.
laptop reads "You Are Enough" in playful, handwritten font,
A serene and vibrant painting titled "Sweet Dreams" dominates wall, featuring an array of colorful flowers and puffy white clouds that evoke a sense of peacefulness. The title "Sweet Dreams" is written in clegant, cursive script at the top of the painting, while small plaque the bottom reads "Artist: Lily Rose" in simple, yet refined font. A tiny inscription the frame states "Dream Big" in delicate letters, further emphasizing the whimsical and soothing ambiance
artwork.
3. A beautiful, rustic wooden sign with the words "Welcome to New Hope Church" in elegant, golden letters hangs above the entrance of quaint, countryside church. On the church door, smaller sign reads "Sunday Service 10am" in simple, yet inviting font, providing essential information to visitors. In the foreground, a stone path is lined with vibrant flowers and a directional signpost that states "Peace Garden" with an aroww pointing towards serene garden area to the side, further emphasizing the church's 's peaceful atmosphere and inviting all explore its tranquil grounds.
35

---

Published as a conference paper at ICLR 2026
Table 29: Prompt used for SynthTex(QA-Step2
SynthTex1QA-SSep2
## Rolc
Suppose you Al visual assistant provided with description of image containing textual elements.
Goal Your task to design a conversation hetween yourself and a person asking ahout these textual elc- ments.
#i Rule
1. Conversation Format:
Present the conversation directly, separating questions and answers with and more than 5 QA pairs maintain clarity.
Limit the conversation
2. Tone and Style:
Answer in the tone of visual AT assistant observing and describing the image. Ensure answers are based solely on the visible content of the image.
3. Content Include:
All textual elements mentioned in the image description must be covered in the con- versation.
4. Question Design:
Diversify the way questions asked, especially when they of the same type.
Answer the question using single word phrase.
For questions focusing specific texts, vary the format:
Half of the time, answer only with the text.
The other half, describe the text.
Ensure you ask about each textual element in the description.
Examples:
Q: What's written on the wonden sign?
A: Welcome Home
Q: Could you describe the text on the taxi's side panel?
A: NYC Taxi
Q: What inscription is on the traditional Japanese lantem?
Konnichiwa
5. Optional Questions:
You may include additional questions exploring aspects like the location font style the text, not mandatory.
can also design other questions explore additional aspects.
But also answer the question using single word phrase.
6. Avoiding Repetition:
For similar questions, avoid repeating the same phrasing.
Design questions that explore different aspects to keep the conversation engaging.
36

---

Published as a conference paper at ICLR 2026
3 What color the gin''s hair?Arswer the question using single word or phrase.
Human
Brown
What is the girl holding?
Human
Teddy bear
What color the bow tie around the teddy bear's s neck?
Hommn
Red
What the central element in the londscape? Answer the question using single word or phrose.
Haman
89
E What is mooned along the loke's edge?
Wooden boat
+ What covers part of the boat?
Human
Figure 10: Illustration of our SynthConsShort
What the main subject image?
Human
main subject image is Taylor performer standing at the center of the stage.
What is Toylor Swift wearing?
Toylor Swift weaning dozzling, long sleeved, silver jumpsuit with intricate, sparkl kinking details.
E What disployed on the mossive screen behind Taylor Swift?
Human
The mossive screen behind Taylor Swift disploying a vibront, pulsing light show that synchronizes with the music
What is the woman in the image holding her hands?
Haman
The womon in the image is holding a korge, round birthday cake.
@ What is written on the birthday cake bold, cursive script?
The rumber \'30V" is written on the birthday cake bold, cursive script.
30
+ Whot is the woman weoning on her hands while holding the cake?
Htman
The womon is wecnring oven mitts on hen hands while halding the
cake.
Figure 11: Illustration of our SynthConvLong
37

---

Published as a conference paper at ICLR 2026
What is the primary difference in the number of smokestocks between the two images? Arswer the question using single word phrase.
Wi/ht difference
the smokestocks between
the two images?
Gray Brown
1O
2 Are there any workars present in the right image?
Human
OD
What primory diff oefnceence cooking between the images? Answer the question using a single word or phrose.
Hunsa
Open range oven vs. induction cooktop
OD
How de the color schemes of the two images differ?
Worn tones vs. cool tones
DOD
What type of matorial used the island surface in the image?
Human
Wood
Figure 12: Illustration of our SynthContrastShort
+ What are the dif ferences in the presentation formot between the left and right images?
Hurnnn
left image feotures cigitol presentation loptop screen, while right image shows o traditional, hond-drown presentation on poper beard
E What is the color diff fiffneence between the objects in the left and right images?
1
Human
The image features predominantly
green The right image has bold, ned and black color scheme.
whereas
does the quantity of visuel clements diff or
the imayes?
Is the left image, there one flweer visus elements, with focus os dlean lines and simple graphics, componed the right image, which hor mons vicee elomeets including hand- drown charts and a dusteeed backgraand
180
differences objects found and right images?
The laft mage partially constructed bost, boxes, welding
Humsa
mochines, workbench verious parts, includes completed bost, fishing with fishing pear.
right image jockets, and small wooden
What is the color differance between the objects in the left and right images?
The left image features silver bast and bright orange vests, wheneas the night image weathered, bronze beot and faded denim jocket.
How does the size of the boot differ betweer the two imoges?
Homan
image, fest in longth in right inage.
lergth, compored about
Figure 13: Illustration of our SynthContrastL.ong
38

---

Published as a conference paper at ICLR 2026
What the woman holding above her head? Answer with the
option' from given choices directly. D. flower
A. A book A hat C, stuffed onimol
Human
10
Where woman'' hand some side os bear's ear?
A. Grosping the bean ear Supporting bear's head
8. In her pocket
Holding body
Homaa
What the primory octivity depicted left photo? Answer with option's letter from the given choices directly.
Hiking Swimming
Skinng D. Cycling
00
What the upperent Terroin the background of the right photo?
A.
valley range
JOIN OUR EPIC CYCLING AND HIKING ADVENTURE
Human
O
REGISTER NOW!
Figure 14: Illustration of our SynthMultiChoice
Render clear and concise summory of the photo.
Human
Woman in long, navy coat and orey hat, against showy white backdrop.
o0
Present compact description the photo's key features
Homan
white windows, warm beige walls, and rich wood floor
o0
Figure 15: Illustration of our SynthColor
39

---

Published as a conference paper at ICLR 2026
Provide brief description the given image.
Human
Two seagulls fly above the beach with Belmont Castle in the background.
00
Write terse but informative summory of the picture.
Human
Three flowers bloom on the grass near children Wisconsin,
O0
Figure 16: Illustration our SynthCount
Render a clear and concise summary of the photo.
Human
People with snow machines near the majestic Mount Fuji in green meadow.
00
@ Create compact narrative representing the image presented.
Human
Vintage lantern illuminates Louis XVI desk Eiffel.
10
Figure 17: Illustration of our SynthScene
40

---

Published as a conference paper at ICLR 2026
Write a terse but informative summary of the picture.
Human
Person stands behind counter, glass in front of them, bottle to the night.
00
2 Describe the image concisely.
Human
Home the foreground, car parked left, teddy bear on seat, under cloudy sky.
Oo
Figure 18: Illustration of our SynthSpatial
Describe the image concisely.
Relax & Uniiidd
Human
ENJY THE VIEW
boat hot tub sign reads, "Relax Unwind Enjoy the View!
o0
G Relay brief, clear account the picture shown.
Human
Achieve Your Goals!
Woman holding small business growth chart with AAchieve Your Goals!" slogan.
Ooo
Figure 19: Illustration of our SenthText
41


Reviews:
[{"review_id": "fr0JI3Zy8v", "reviewer": "Reviewer_NPmU", "summary": "Vision-Language Models (VLMs) often rely on shallow fusion, aligning visual and textual features only during decoding, which limits accurate cross-modal understanding. This paper presents FLARE, a framework that integrates several techniques to achieve deeper and more dynamic vision-language interaction. FLARE introduces Text-Guided Vision Encoding to incorporate textual cues into visual feature extraction, a Dual-Semantic Mapping Loss for bidirectional modality-level alignment, and Context-Aware Alignment Decoding that aggregates visual information conditioned on text for fine-grained query-level fusion. Additionally, a Text-Driven VQA Synthesis method is employed to augment training data with synthetic question-answer pairs. Experiments across multiple benchmarks show that FLARE performs competitively with existing models while often using fewer visual tokens. The paper also presents ablation studies, fair comparisons under controlled settings, and a cost analysis showing the benefits of the proposed FLARE.", "questions": "1. Section 5.2 presents controlled comparisons using identical backbones and datasets, but it remains unclear whether the proposed Synthesis augmentation was applied in these experiments.\n2. The paper highlights the efficiency in using fewer visual tokens. However, this design constraint also raises an important question: if the visual token limit were relaxed, could FLARE outperform existing systems?\n3. Figure 1 is intended to visualize cross-modal alignment but is difficult to interpret without clear ground-truth references.\n4. The training pipeline is described as stages 1, 1.5, and 2, rather than 1, 2, and 3. It would be helpful to explain the reason.", "limitations": "", "rating": 4, "confidence": 4, "soundness": 1, "presentation": 3, "contribution": 2, "strengths": "1. The key concept of achieving deeper cross-modal alignment through mutual reconstruction, mapping text to image and image to text, is intuitive and well-motivated. It aligns with the goal of reducing modality gaps and provides a more integrated framework than typical unidirectional projection methods.\n2. The proposed data synthesis offers a practical way to expand multimodal training data using LLM-generated question-answer pairs. As shown in Table 6, this augmentation leads to measurable performance gains. The release of the code, data, and models could benefit broader vision-language research. \n3. The main experiments are complete and cover a wide range of multimodal benchmarks and baselines. The paper also includes ablation studies, fair comparisons, and cost analyses, which together strengthen the empirical evaluation. These analyses clarify the contribution of each module and demonstrate the efficiency of the model.\n4. The paper is clearly written and well-structured. Figures, such as those showing cross-modal alignment and architecture flow (e.g., Figures 2 and 3), effectively communicate complex ideas.", "weaknesses": "1. As shown in Table 1, FLARE performs competitively but does not substantially outperform recent multimodal baselines of similar capacity. Claims such as \"achieves leading performance across all benchmarks\" are overstated given the mixed improvements and the complexity of the proposed pipeline. The cost of full training may not justify the small performance gains.\n2. Some architectural decisions are only heuristically motivated. For example, it is proposed to mask visual-to-textual attention in the lower encoder layers. The paper asserts that early visual features lack semantics, but provides no ablation or analysis verifying that this masking improves stability or alignment.\n3. The idea of reconstructing the representation of one modality from the other is conceptually reasonable and aligns with the goal of deeper cross-modal integration. However, in FLARE, the reconstruction is performed mainly using questions rather than captions. Reconstructing a question from an image may not yield semantically grounded alignment. In contrast, using captions or paired statements would provide a more consistent and general signal for vision-language correspondence, especially considering the proposed similarity loss.\n4. The proposed data synthesis is basically an LLM-based data augmentation method, and its key implementation details are missing. The paper claims that only prompt engineering is required, which oversimplifies the complexity of generating consistent and high-quality image-text pairs. In practice, diffusion-based image generation often struggles with fine-grained accuracy (e.g., hands, text, and object details), yet these limitations are not discussed. In short, while the additional data may be valuable, the approach itself is not that novel and could introduce a potentially unfair advantage in comparison with the other existing models.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "yLSmml42Uw", "reviewer": "Reviewer_usqb", "summary": "The paper proposes FLARE, a system-level approach for fully integrating heterogeneous components (hardware accelerators, runtime scheduler, and compiler toolchain) to improve end-to-end performance and developer productivity for ML workloads. The authors present the system design, an implementation supporting several accelerators, and an evaluation on representative models and benchmarks showing improved throughput and reduced end-to-end latency compared to a baseline composed of independently developed components. The paper claims contributions in modular integration APIs, cross-layer optimizations, and an evaluation demonstrating practical gains on real workloads.", "questions": "1.\tWhich specific baseline toolchains and vendor stacks were used for comparison, and why were these chosen over others?\n\n2.\tCan you provide an ablation study that separately measures the impact of compiler optimizations, runtime scheduling, and accelerator-specific backends?\n\n3.\tHow does FLARE handle dynamic workloads with runtime shape changes or mixed-precision operations; are there cases where integration harms performance?\n\n4.\tWhat are the compile-time and memory overheads introduced by the integrated toolchain compared to the baseline? Please provide quantitative measurements.\n\n5.\tHow difficult is it to add support for a new accelerator backend; what is the required engineering effort and the expected common failure modes?", "limitations": "", "rating": 4, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The paper identifies the practical pain points of mismatched interfaces across compiler, runtime, and accelerator stacks and motivates why full-stack integration yields benefits.\n\n2. The modular architecture and defined integration API are well described and appear reusable for multiple accelerators.\n\n3. A working prototype that integrates compiler passes, a runtime scheduler, and at least two accelerator backends strengthens the paper’s practicality.\n\n4. Results include latency and throughput measurements on relevant models, showing consistent improvements over the baseline and an analysis of where gains arise (e.g., operator fusion, memory reuse).\n\n5. The paper is mostly well organized with diagrams that clarify the cross-layer interactions and a reproducible experimental setup description.", "weaknesses": "1. The evaluation compares against a narrow set of baselines; it is unclear how FLARE performs versus other recent integrated stacks or optimized vendor toolchains.\n\n2.  While gains are reported, the paper lacks deeper ablation isolating the contribution of each integration component (compiler vs runtime vs scheduler).\n\n3.  The current implementation targets a limited set of accelerators and ML model classes; scalability to diverse models (e.g., large sparse models, non-neural workloads) is not demonstrated.\n\n4.The paper does not fully quantify engineering or runtime overheads introduced by the integration (e.g., compile time, memory footprint, scheduler overhead under dynamic workloads).\n\n5.Some implementation details and tuning knobs used in experiments are missing, which could hinder reproduction by other researchers.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "LyBush2rFB", "reviewer": "Reviewer_NAvg", "summary": "This paper introduces FLARE, vision-language models with several designs to enable cross-modal alignment and interaction across in visual encoders and LMM backbones. Besides architectural innovation, a data synthesis strategy is proposed to construct text-centric data. Experimental results validate the effectiveness of key components in FLARE.", "questions": "see weaknesses", "limitations": "", "rating": 4, "confidence": 5, "soundness": 3, "presentation": 2, "contribution": 2, "strengths": "* This paper is strongly motivated, compared with prior work’s preliminary explorations, this paper conducts a comprehensive exploration of how to enhance text-guided visual encoding.\n* The proposed Context-Aware Alignment Decoding module is interesting, providing a novel solution to refine in-context visual features layer by layer.\n* Experimental results support the effectiveness of FLARE, the authors also provide analysis on additional computational cost, which is important to analyze the introduced modules.", "weaknesses": "* Presentation of this paper can be improved:\n  1. Horizontal lines and citations are missing in Table 1\n  2. In Equation 3, would it be $(V_i^s, V_q^s)$ rather than $V_i^s(V_q^s)$?\n  3. Lack of explanation for $r$ and $c$ in Equation (6, 7) \n\n* Missing important details:\n  * How to collect the curated caption pool in Figure 4\n  * The number of latent tokens is randomly sampled during training, how about evaluation?\n  * Except for Context-Aware Alignment Decoding, the novelty of Text-Guided Vision Encoding and Text-Driven data synthesis is limited:\n    * Text-Guided Vision Encoding is quite similar to Q-Former in InstructBLIP.\n    * Utilizing synthesized images for VLM training is not a novel solution, for example, SynthVLM [1].\n* There are several important points to address for the method:\n  * From my perspective, data filtering is important for the synthesized data, which is not included in this paper\n  * Most VLMs adopt different strategies to control trainable parameters in different stages, which is not considered in FLARE\n  * How could the designed module be applied to more complex scenarios like videos, multi-images, and excessively long text contexts?\n  * Fore FLARE-X, why not apply the dynamic-resolution strategy that is utilized in NaViT and Qwen2.5-VL, which seems more suitable for your method compared to the subfigure-based solution.\n* Conducted experiments are limited, in terms of:\n  * Lack of analysis on the quality of synthesized images\n  * Experiments in Table 3 do not sound fair to me, the superior performance of FLARE may come from the SigLIP2, which is better than vision encoders used by other models.\n  * Current analysis in the main paper focuses on quantitative performance on benchmarks, detailed analysis is expected to reveal how the proposed modules help vision-language interaction. \n\n\n[1] SynthVLM: Towards High-Quality and Efficient Synthesis of Image-Caption Datasets for Vision-Language Models (https://arxiv.org/abs/2407.20756)", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "mOeSTuv2EI", "reviewer": "Reviewer_1a7W", "summary": "This paper introduces FLARE, a new family of vision-language models (VLMs) that aim for a deeper integration of vision and language representations. Unlike previous methods that often treat vision features independently and only fuse them with text during decoding, FLARE proposes a fully vision-language alignment and integration paradigm. With extensive experiments verified its effectiveness.", "questions": "follow weakness", "limitations": "", "rating": 4, "confidence": 4, "soundness": 3, "presentation": 2, "contribution": 2, "strengths": "1. The paper is clearly structured and easy to follow, making the proposed methodology and experimental results comprehensible.\n2. The authors have conducted extensive experiments across various benchmarks and scales, demonstrating the effectiveness and generalizability of FLARE.", "weaknesses": "1. The effectiveness of a simple MLP projecting text tokens into vision space for a vision encoder with significantly fewer parameters than the LLM is unclear. I'm curious about how the vision encoder can effectively process these projected text tokens.\n2. Equation 3 appears to have typos. Should it be $(V_i,V_q)$ instead of $V_i(V_q)$?\n3. While Qwen2.5VL 7B is presented as a strong baseline, it would strengthen the paper to include experiments based on Qwen2.5 model series to further demonstrate the superiority of the proposed method.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)