MindOmni: Unleashing Reasoning Generation in Vision Language Models with RGPO
Authors: Yicheng Xiao
Lin Song
Yukang Chen
Yingmin Luo
Yuxin Chen
Yukang Gan
Wei Huang
Xiu Li
XIAOJUAN QI
Ying Shan
Venue: NeurIPS 2025 poster ()
Keywords: Unified MLLM
Reasoning Generation
Reinforcement Learning
Primary area: deep_learning

Abstract: Recent text-to-image systems face limitations in handling multimodal inputs and complex reasoning tasks.
We introduce MindOmni, a unified multimodal large language model that addresses these challenges by incorporating reasoning generation through reinforcement learning. MindOmni leverages a three-phase training strategy: i) design of a unified vision language model with a decoder-only diffusion module, ii) supervised fine-tuning with Chain-of-Thought (CoT) instruction data, and iii) our proposed Reasoning Generation Policy Optimization (RGPO) algorithm, utilizing multimodal feedback to effectively guide policy updates.
Experimental results demonstrate that MindOmni outperforms existing models, achieving impressive performance on both understanding and generation benchmarks, meanwhile showcasing advanced fine-grained reasoning generation capabilities,  especially with mathematical reasoning instruction. All codes will be made public.

MindOmni: Unleashing Reasoning Generation in Vision Language Models with RGPO
Yicheng Xiao 2,, Lin Song Yukang Chen Yingmin Luo", Yuxin Chen', Yukang Gan', Wei Huang", Xiu Li Xiaojuan Qi Ying Shan
Tsinghua Shenzhen International Graduate School, Tsinghua University
ARC Lab, Tencent PCG
The Chinese University of Hong Kong xiaoyc23@mails tsinghua. edu.
"The University of Hong Kong rommysongdtencent.
Reasoning Generation
Vision Editing
User: Generate on image of animal with
User: An elephant rabbit stand both
User: pool of minus degrees
User: cIMG> Add a pair sunglaases to the dog on the right side.
seesow.
User: Change this <IMG> info the style Water Color.
image multiple apples, the quantity of apples the solution equation" 11".
User:
image China's national tressure animal.
Sydney Opera House when New York at
User: <IMG> Find in the photo and outline them
pones
OO X O
OO
User: <IMG> Add the piece cell.
X
X ROX X
X O
User: cIMSxGenerate on image onimal growing up.
User: <IMG> Remove the piece
and add to c5 cell.
The of our MindOmni:
<tiink (conten)] </lintt canswers (Conlent) <IMS> clunswers
Figure 1: Core capabilities of proposed unified model, MindOmni. i) Multimodal Reasoning
Generation: MindOmni incorporates our proposed RGPO reinforcement learning algorithm to improve Chain-of- Chain-of-Thought generation, resulting in more interpretable and responsible reasoning outputs, thereby advancing reasoning generation. ii) Beyond style transfer, MindOmni preserves low-level details, such texture, posture, and spatial layout, from the reference image, ensuring high-fidelity visual editing. Detailed reasoning outputs can be found in Appendix A.3, while qualitative comparison with prior methods, including GPT-4o and Gemini-2.5, is presented in Fig. 6.
Project Lead. s Corresponding author.
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

Abstract
Recent text-lo-image systems face limitations in handling multimodal inputs and complex reasoning tasks. We introduce MindOmni, unified multimodal large language model that addresses these challenges by incorporating reasoning genera- tion through reinforcement learning. MindOmni leverages three- three-phase training strategy: i) design of a unified vision language model with a decoder-only diffu- sion module, ii) supervised fine-tuming with Chain-of-Thougeee data, and iii) our proposed Reasoning Generation Policy Optimization (RGPO) algorithm, utilizing multimodal feedback effectively guide policy updates. Exper- imental results demonstrate that MindOmni outperforms existing models, achiev- ing impressive performance on both understanding and generation benchmarks, meanwhile showcasing advanced fine-grained reasoning generation capabilities, especially with mathematical reasoning instruction. All codes will be made public at https://githu..com http://gtttubboooooo//ccetAAAAA TencentARR//iinddnn
(CoT) instruction
Introduction
Advanced text-to-image systems such SD-XL [32], FLUX [21], and Sana [57] generate images by leveraging textual descriptions as conditioning signals. However, they encounter two key challenges: i) The use of fixed text encoder restricts the model textual instructions, thereby limiting its capacity to incorporate multimodal inputs like images or audio. ii) These methods are incapable of producing explicit inferences in response instructions that involve world knowledge, mathematical logic. or spatiotemporal perception (see Fig. 1), which is defined as reasoning generation capacity. Previous studies build unified MLLMs capable of both understanding and generation to address the first challenge. These approaches employ learnable queries [11, 8, 30] or direct projector [45] to implicitly extract semantics from VLMs, or integrate text token autoregressive and image generative processes within single transformer [58, 64, 5] to enable early fusion. Although these approaches support multimodal inputs, they still struggle address scenarios that require reasoning.
Building on early explorations of Chain-of-Thouuge (CoT) reasoning in VLMs [59, 33] and LLMs [37, 50], recent works [61, 9] propose generating textual layout plans by guiding large models with customized CoT prompts for image synthesis. While there has been progress on the second challenge, these approaches are constrained spatial relations and hindered by intricate separation processes and inflexible CoT templates, limiting their applicability to more complex reasoning scenarios, Following the success of Deepseek-RI [14], subsequent research [65, 4] has shown that incorporating the Group Relative Policy Optimization (GRPO) [37] algorithm into textual reasoning pipelines significantly enhances VLM performance on mathematical and coding tasks by automatically producing accurate and diverse CoT trajectories. This observation prompts a fresh perspective: Can reinforcement learning be leveraged to unleash the reasoning generation capabilities of unified VLMs?
To address the issue, we introduce a three-phase training strategy. In the initial stage, we propose a new multimodal large language model, named MindOmni, for unified vision understanding and generation, which includes a vision language model [i]. a lightweight connector, a text head, and a decoder-only diffusion module [52]. Prior studies [14, 65] have shown that applying reinforcement learning following supervised fine-tuning can significantly enhance algorithmic performance. Therefore, we construct a series of coarse-to-fine CoT instruction data and then train our model with supervised tuning in the second stage to effectively initiate the reinforcement learning algorithm later. In the third stage, we propose the Reasoning Generation Policy Optimization (RGPO) algorithm and collect a curated corpus of tlain-text reasoning sequences to guide the model in generating precise
X2Image Generation
Unified
End-to-End Pipeline
Fine-grained Image Editing
Explicit CoT
RL Augmented
Method
Und. & Gen.
Janus-Pro [5]
X
x
X
MctaMorph [45]
x
x
X
GoT [9]
X
V
MindOmni
V
Table 1: Characteristics comparison with other counterparts.
2

---

CoТ reasoning content. In contrast to the original GRPO algorithm, which relies solely on text representations, RGPO generates multimodal feedback signals from both image and text features to guide policy updates. In addition to the format reward function that enforces adherence to the correct CoT structure, we introduce a consistency reward function to evaluate visual-linguistic alignment. Tailored for reasoning generation, we introduce two independent Kullback-Leible divergence regularizers into RGPO. They are applied separately to text and visual rollouts during policy updates, to stabilize training and mitigate knowledge forgetting. Ultimately, our MindOmni obtains obvious advantages over previous counterparts as shown in Table
Fig, 1 illustrates the effectiveness of MindOmni across various complex reasoning generation scenar- ios, notably its strong performance on mathematical reasoning instruction. Furthermore, showeases advanced fine- grained editing capabilities, covering general edits as well as computer vision tasks like instance segmentation. Remarkably, after fine-tuning with some specific data, MindOmni can accurately play chess and tic-tac-toe according to user inputs, establishing foundation for future research. We also conduct extensive experiments to validate the effectiveness of MindOmni on both understanding and generation benchmarks. Our method surpasses LLaVA-One-Vise [22] by 2.4% on MMBench [62] and boosts Janus- sarus-Pro f [5] by 10.6% on MMMU [63]. Furthermore, we achieve ofte--ofttheaan performance with 71% overall score the reasoning generation benchmark WISE [27], which outperforms previous methods with a remarkable margin, as well as 83% overall score on the basic text-to-imane generation benchmark, GenEval [12].
fine-grained
2 Related Work
2.1 Unified Model for Understanding and Generation
Unified multimodal LLMs have demonstrated the ability to perform both understanding and genera- tion tasks within a single framework. Early research DreamLLM [8] and Emu-series [42, 41] predict the next multimodal element by regressing visual embeddings and integrate a separate diffusion U-Net to achieve image generation. Chameleon [43] and Emu3 [48] discretize visual features and train token-based autoregressive models on mixed image-text data. Furthermore, TransFusion [64] and Show-o [58] integrate the denoising mechanism and autoregressive approaches within a single transformer model. However, many of these unified models still fall short of the performance of task-specific architectures. In an effort to bridge this gap, Janus [51] uses distinct tokenizers for understanding and generation tasks, whereas TokenFlow [34] employs dual codebooks with shared mappings to facilitate the integration of low- and high-level features. Some methods [18, 23] aim to incorporate unified visual tokenizer to employ a more advanced VLM, along with a separate diffusion DiT, achieve improved performance.
2.2 Reasoning in Large Models
Recently, there has been surge in research both in academia and industry regarding the reasoning ca- pabilities of large language models. OpenAl-ol [28] makes a significant breakthrough by enhancing reasoning performance through extended Chain-of- Thought (CoT) reasoning framework and intro- ducing inference-ting scaling. Many studies have explored reinforcement learning [37, [0J. majority voting [49], and search algorithms [19, 10] ways to scale test-time compute and improve reasoning performance. Notably, DeepSeek-RI R1 [14] demonstrates exceptional reasoning performance after only few thousand steps of RL training, leveraging the GRPO [37] algorithm. Furthermore, several studies have also applied GRPO to multimodal large models, improving visual reasoning capabilities across various tasks, including mathematical reasoning [65, 4] and reference segmentation [26].
2.3 Image Generation with Reasoning
Reasoning-drit driven image generation has made significant strides in recent years, garnering consid- erable attention from the academic community. PARM [15] utilizes an iterative reasoning process to enhance image quality by combining clarity and potential evaluation. RPG-Diffusinn Difffussiionnt [61] employs a multimodal learning model (MLLM) for text-to-image diffusion, ensuring coherence in the generated output by breaking prompts into multiple sub-regions. GoT [9] introduces a struc- tured language reasoning framework to build a generative thinking chain, first analyzing semantic relationships and spatial arrangements, and then generating and editing images. However, GoT's
3

---

fins
the seste
is Us ausan The nassress procso and sower endousl with vieece ottek Citacc-aaaer chenaee.
about nosoring
then the witherrcss promsi
Rephoc
CAEE 169;, alc
Al Acmaania trgm- Sucee Peeeeppp..eee P.
User Geseraic imge the entmall prodtngan
+
Text Telesier
VIT Saundrr
Toloniier
Eacoder
Teleroot
Multi-Modal Large Language Model
Concicmaa:
New
Toa
Tokenkez
Takuu
Ausblaatt:
Amsiitat: stteeke aenenda sourding
intructiors
onea grown into frog. proveeen feature tadpole hase dikagosored as the fagg devdlope.. c/tbiee canteer For the image fualf. you sauld depái a trog atting rack, possibly near water, glen that trogs andessments
read condder what the tadpele
with i., enneecee
whith
Function Vision Understanding
Fanction Muttimddl Reasoning Gemeration
Function C: Vision Editing
Figure 2: Overview of our inference framework. MindOmni accomplishes vision understanding, multimodal reasoning generation, and vision editing tasks in unified large model.
independent structure limits its flexibility and adaptability. Meanwhile, SimpleAR [47] demonstrates the effectiveness of reinforcement learning algorithms discrete autoregressive generative models. However, previous approaches remain focused on strengthening the consistency between images and instructions using MLLM, often overlooking the use of MLLM's powerful logical reasoning capabilities for genuine reasoning generation. In this work, we aim to address this gap by introducing the RGPO reinforcement learning algorithm into unified multimodal large model (MLLM) for both generation and understanding, enabling deeper reasoning-die generation.
3 Method
In this section, we first present the pipeline formulation and describe our proposed model, MindOmni, for unified vision understanding and generation. Subsequently, we introduce our proposed Reasoning Generation Policy Optimization (RGPO) algorithm in the RL process to achieve impressive reasoning. guided image generation, with the detailed training procedure provided at the end.
3.1 Pipeline Formulation
A unified multimodal large model is capable of processing both visual and textual inputs simulta- neously. Separate encoders map each type of input into common text feature space. Meanwhile, diffusion decoder conditions the MLLM's output feature sequence to denoise the latent noise into a real image, enabling multimodal generation and vision editing. However, prior methods have primarily used multimodal models as semantic feature extractors, overlooking the inherent reasoning capabilities of LLMs. As a result, they underperform on reasoning generation and editing tasks. Recent research on the reasoning ability of large models [65, 4, 14] has inspired us to introduce reinforcement learning algorithms into our framework to unleash the reasoning generation power.
To build unified MLLM for reasoning generation, we introduce three-stage training pipeline and propose the RGPO reinforcement learning algorithm, which enables the model to explicitly express reasoning via a chain-of Mlain-of-tie of-thought process.
3.2
Our Model
Recent vision language models [1, 6, 22] show substantial world knowledge and eross-men mross-modal reason- ing skills but lack effective generation capabilities. In contrast, existing generative models [32, 21] produce high-quality renderings but struggle to interpret complex semantic controls, such as logical reasoning or multimodal inputs. Furthermore, significant structural differences between VLMS and DiTs [31] also hinder smooth mode integration. To address it, we develop our MindOmni based on Qwen2.5-VL [1] and OmniGen [52]. The former is advanced vision language model, and the latter

---

Digiusenn Lort
Tolaa Misur avaienee
Thunhhaii
Reforence Model
the skit Sprie
Calozino
Hujamilg costice luok ont
inture deowntt the
Transformet Block
Cennzieen Block
KL Divergeseo
desiven our biocc
Drodle
Thes appeenci
bod and divel Buckinn
slop
Vont
Diflasan
Connector
Todkel Mestare
Pretruined
Input
MILM
Gray Companiion
Stage 1: Pretrain with tditing and caption cata.
Pulicy Medel Uo
Reward Model
Stage data.
Text Takam
Fastumm
Viee Fasture
"We Image of tadpolo after growing up"
(a) Stage
Stage 2: Supervised Tuning
(b) Stage 3: Reinforcement Learning with RGPO
Figure 3: Overview of Training Pipeline. We propose three-stage training framework comprising
pretraining, instruction-based supervised fine-tuning, and reinforcement learning with RGPO.
diffusion decoder that offers a promising X2Image generation performance with streamlined, LLM-style network design. We utilize a connector comprising two standard LLM decoder layers bridge two models. Additionally, by leveraging our RGPO reinforcement learning algorithm, MindOmni can generate more precise chains of thoughts through an autoregressive paradigm to guide reasoning generation shown in Fig.
Vision Understanding. Given an image I and a corresponding user linguistic request for vision understanding, MindOmni first extracts continuous vision features by pretrained ViT of Qwen2.5- VL and encodes the text into n discrete tokens T = (4,, .,.,,,, in Function A of Fig. Subsequently, the model samples the output text embedding fx+1 and detokenizes the final output according to the conditional probability:
...,..n) (n) with an LLM tokenizer as shown
n+1
P(t Plt,.tg...,,t (1) II 1=1
(I)
Reasoning Generation and Editing. As depicted in function of Fig. 2, when the user provides a tadpole image and asks for prediction of its appearance as it matures, we first capture the current visual semantics and steer the model's reasoning by adding system prompt. The autoregressive generation process is also based the conditional probability in Eq. (1). When the generation process encounters a special token, we use the connector to align the last hidden states of the previous tokens with the dimension of the diffusion decoder. Then concatenate time token (representing the time step of denoising process) Ftime along with m noise latent tokens Fmoiie Fnoise ~ NO0, 1) mm
the prior tokens Foond which consists vision and text features. For the vision editing task, we recognize that relying solely on VIT to extract image semantic features may cause a loss of low-level information, such as texture and layout, which can result in the inability to preserve the fine details of the reference image during editing. To address this, we incorporate the VAE features to retain these crucial details as shown Function of Fig. 2. At the end, the entire multimodal sequence injected into the diffusion decoder for N loops of denoising.
3.3 Stage 1 & Stage 2: Supervised Tuning
To implement our model, we develop three-stage training pipeline as shown in Fig. 3. In the first two stages, we enable the model initially achieve basic text-to-image generation and editing capabilities. In the first stage, we only train the connector using image-text pairs [39, 36] and X2I data pairs [52]. ensuring that the diffusion decoder can seamlessly process the semantic representation from the MILMM. We deploy diffusion loss and KI. distillation loss objective functions to optimize our model. Following [21, 52], we use rectified flow to optimize the model. Specifically, given a ground truth image zo and random time step t, sample the noise data N 1) and get x; which is
5

---

GenEval
DPG-Bench
Type
Model
Size
Single Obj Two Obj
Counting Colors Position Color Attri Overall Global Relation Overall
LLaMAGen [40]
8.0B
0.98
0.71
0.34
0.81
0.17
0.21
0.32
63.2
Emu3-Gen [48]
8.0B
0.811
0.391
0.45
0.66
81.6
SDv1-5 [35]
0,98
0.97
0.38
0.35
0.76
0.04
0.06
0.43
74.6
73.5
63.2
PixArt
0.68
0.98
0.50
0.44
0.80
0.08
0.07
0.48
75.0
82.6
71.1
Gen. Only
SDXL [32]
2,68
0.98
0.74
0.39
0.85
0.15
0.23
0.55
83.3
86.6
74.7
SimpleAR
8.0B
0.90
0.28
0.45
0.63
88.0
88.3
82.0
Flux [2]]
12B
0.67
84.0
[7]]
(Bavelline) 3.8B
0.99
0.55
81.2
Chameleon [45]
7.0B
0.39
Show-o [58]
0.95
Janus [5]]
0.97
79.7
Janus-Pro [5]
7.08
0.99
0.59
0.90
0.79
0.66
84.2
TokenFlow-XLj,
10B
0.97
0.66
0.40
0.84
0.17
0.26
0.55
78.7
85.2
73.4
TokenFlow [4]
14B
0.931
0.721
0.451
0.821
0.451
0.42
0.63
78.7
85.2
73.4
Und. and Gen.
GOT[9]
7,98
0.99
0.69
0.67
0.85
0.34
0.27
0.64
MetaQuary-XL[30
7.08
0.801
82.1
BL [1]
7.0B
0.83
80.7
BAGEL|71
7,0B
0.82
MindOmnn (ours)
7,0B
0.99
0.97
0.77
0.89
0.59
0.64
0.81
89.7
88.7
83.0
MindOmni (ours)
7.08
0.99
0.90
0.71
0.71
89.1
89.2
82.5
Table 2: Performance comparison on GenEval and DPG-Bench. "Und." and "Gen." denote "understanding" and "generation", respectively. indicates using the rewritten prompts, which may improve the accuracy significantly. "Size" unified models refers to the size LLM backbone. MindOmni" denotes the variant trained with higher-quality data [29, 2]. is the released version.
defined as: T2 = (1 1)e + toog Consequently, we denote diffusion loss as:
Laiffuison = EE T 1, Froood - (r0 : =
where indicates the parameters our model and π is our MindOmni. To mitigate the adverse effects of potential distribution shift, we employ KL divergence to enforce consistency with the teacher model f in modeling the vector field during the denoising process. Distillation loss formulated as:
log(()(T)((re Froond)) HH (0r,,, Frond)).
Edistillation
where n² and u² indicate the noise latent feature of i-th layer in our model and teacher model, respectively. represents the entropy and ő indicates the feature mapping. In the second stage, we collect captions of varying granularity to construct CoT instruction data, using coarse-grained descriptions answer content and fine-grained descriptions reasoning content. We also utilize leading text-to-image models [21, 57] to retrieve corresponding high-quality images. The connector and diffusion decoder are trained with diffusion loss only.
3.4 Stage 3: Reasoning Generation Policy Optimization
At this stage, we propose the RGPO reinforcement learning algorithm to enhance the model's reasoning generation capabilities by explicitly generating logical chain of thought inspired by DeepSeek-R1 [14]. To achieve this, we first construct a plain-text training dataset comprising user instructions, ground-truth prompts, and corresponding explanations. Additionally, we design reasoning -oriatttioooooeeeddd system prompt following DeepSeek-RR to guide our model in generating Chain-of- ofain-of-Thought content.
During rollout process, policy model Ta first samples G groups of results {o for each request g, each comprising reasoning chain of and a corresponding image of. To enhance the quality of the generated reasoning process, we introduce two types of reward functions to guide the policy model toward producing coherent and effective outputs: i) Format reward evaluates
Reward Functions.

---

Spatio-Temporal
Natural Science
Type
Model
Size
Cultural
Overall
Time
Space
Biology
Physies
Chemistry
Emu3-Gen [48]
8.0B
0.34
0.45
0.48
0.41
0.45
0.27
0.39
SDv1-5 [35]
0.98
0.34
0.35
0.32
0.28
0.29
0.21
0.32
PixAn [1]
0.68
0.45
0.50
0.48
0.49
0.56
0.34
0.47
Gen. Only
SDXL [32]
2.68
0.43
0.48
0.47
0.44
0.45
0.27
0.43
FLUX [21]
2.715
0.48
0.58
0.62
0.42
0.51
0.35
0.50
OmniGen (Baseline)
3.8B
0.40
0.38
0.51
0.27
0.51
0.59
0.44
Chamelcon [43]
7.0B
0.39
Show-o [58]
1.5B
0.28
0,40
0.48
0,30
0.46
0.30
0.35
Janus [51]
1.5B
0.16
0.26
0.35
0.28
0.30
0.14
0.23
Janus- Pro
7.0B
0.30
0.37
0.49
0.36
0.42
0.26
0.35
Meta(uury--LL331
7.0B
0.55
Und and Gen.
BI.IP-30[2]
7.0B
BAGEL[]]
7.0B
0.75
0.65
0.58
MindOmni (w/o thinking)
7.0B
0.40
0.38
0.62
0.36
0.52
0.32
0.43
MindOmni (ours)
7.08
0.60
0.62
0.64
0.56
0.68
0.48
0.60
MindOmni" (ours)
7.0B
0.75
0.70
0.76
0.76
0.72
0.52
0.71
Table 3: Comparison with ttate-of-the-arts on WISE [27] benchmark. "Und." and "Gen." denote "understanding" and "generation". respectively. MindOmni denotes the variant trained with higher- quality data [29, 2] and evaluated with thinking mode.
whether the chain of thought adheres to the expected structure, returning 1 if the content is enclosed within <think> and <answer> tags, and 0 otherwise. ii) Consistency reward measures the semantic alignment between the generated image and the reference ground-truie -trut prompt using cosine similarity derived from CLIP image and encoders. Then the advantage A, of of the i-th output is computed by all reward values {ri) which is formulated as:
A
Fi Var(r))
while == &C Var(r) =
I G &L 7)2.
(4)
KL Regularizer. During reinforcement learning, we incorporate two KL divergence-based distilla- tion strategies, DiL for text generation and Diu for image generation to penalize large deviations between the reference model Taf and previous policy, thereby promoting smoother policy transitions and mitigating the risk of forgetting previously learned knowledge. We calculate two distillation functions of o as:
Jnf (0,
Tef (o] (q) 1, To of 19)
DI (Tel TTe))
log
(5)
(of
( (of (q)) Dá. (Tol(Tref)= = E o(To(o!) (o)
(6)
MT (o (q))
Finally, we optimize our policy model by minimizing the objective function LORPO as follows:
LORPO 1E q~~ P(Q), f (00)( Li=1 (min noforlo A,, clip To(a,
(7)
EC G
1 + +e) A.) -Be -3-
Fu (o.10]
where e, 8 and 3 are hyper-parameters.
Experiments
We conduct extensive experiments to evaluate the effectiveness of our MindOmni and compare it to the widely adopted large language models across diverse multimodal understanding and generation benchmarks under fair evaluation setting.
7

---

(a) Consistency Reward
(b) Format Reward
(c) Completion Length
0.00
020
630
282
200
300
Sceps
Steps
Steps
Figure 4: Curves of different Metric in RGPO. "Completion Length" indicates the output length of the policy model during rollout process.
Model
Size MMMU MMB RWQA
Und. only
59.6
LLaVA-OV [2]
7.0B
48.8
80.8
66.8
Enm3-Chat [48]
8.0B
31,6
58.5
57.4
58.8
Qwen2.5 VL.
7.0B
51.1
83.4
68.9
500-
58.0
Und. and Gen.
Janus Pm [5]
7.0B
41.0
79.2
Training step
TrkenFloowXDI XI. [ [34]
14B
38.7
68.9
58.3
090
MetaMorph [45]
8.0B
41.8
75.2
68.8
MindOmni (ours)
7.0B
51.6
83.2
68.1
Figure 5: Scatter plot showing the performance on WISE Benchmark. The Y and X axes represent the overall score and training step during RL, respec- tively. The size of each circle reflects the completion length of our model in a certain step.
Table 4: Performance Comparison Vi- sion Understanding Benchmarks. "Und." and "Gen." denote "understanding" and "generation," respectively.
4.1 Training Details
We utilize the open-sourced image-caption pairs [36, 39] and in-house data as training data. Group Number G, KL coefficient 3 and 3 in RGPO are set as 4, 0.06 and 0.01 by default. Due to the limited space, detailed contents are present in Appendix A.I.
4.2 Main Results
Image Understanding and Generation. Our performance on image understanding is shown in Ta- ble 4. We first evaluate MindOmni understanding benchmarks (MMMU [63], MMBench [62] and RealworldQA). Due to the inclusion of the text KL regularization term, the model's ' performance after reinforcement learning fine-tuning remains nearly identical to the original VLM backbone, with only a 0.1 % average difference. MindOmni boots previous unified counterparts, Janus-Pro by 10.6% on MMMU and MetaMorph by 9.8% MMBench. Furthermore, text-to-image generation ability on GenEval [12] and DPG-Bench [16]. MindOmni achieves state-of- the-art performance on GenEval in unified MLLM with 83% overall score as shown in Table 2. We build MindOmni based on OmniGen [52], which outperforms MetaQuery [30] with Sana [57] as the generator on DPG-Bench by 0.4% as shown in Table
validate our common
Reasoning Generation. To evaluate the reasoning aware generation ability of our proposed Min- dOmni, we benchmark it against a broad set of existing generative-only and unified models using comprehensive evaluation suite [27] that spans cultural knowledge, spatio-temporal reasoning (time and space), and natural science domains (biology, physics, and chemistry). As shown in Table 3, MindOmni outperforms all methods across nearly all reasoning subcategories, achieving an overall score of 0.71 with the thinking mode. Specifically, MindOmni surpasses prior unified models such as MetaQuery-XX [30] (0.55 overall) by notable margins both time reasoning and physies reasoning. Compared to leading generative-only models such as FLUX (0.50 overall) and PixArt (0.47 overall), MindOmni demonstrates consistent advantages across every category, reflecting the benefit of joint cewraandddiingggeattiiio training and CoT-guided reinforcement learning. Additionally, Fig.

---

Only Generation Model
Unitied Model Understanding Generation
An image of an animal with
6+
An elephant and rabbit stand sensow
Scene in Sydney Opera House when New York
China's treasure animal.
image multiple apples, quantity apples solution to the equation
St3ee-darge
Janus-Pro-7B
MindOmai
Figure 6: Qualitative comparison among leading models on reasoning-awarr image generation.
Stage Stage 2 Stage 3 GenEval
WISE
Layer Number
Type
Param.
#FLOPs GenEval
0.73
0.42
0.81
0.54
MLP
0.1%
2G
0.63
0.72
0.49
1.5%
31G
0.68
Decoder
0.81
0.60
3.0%
60G
0.73
Table 5: Effectivvness of the training stages.
Table 6: Impact of different connectors.
illustrates the progression of various metrics throughout the reinforcement learning process, with steady increases in both the consistency reward and completion length. However, as shown Fig. 5, merely increasing the length of output thinking content does not lead to better end results.
4.3 Qualitative Results
We visualize and compare advanced eext-to-image methods [21. 38, 57] and unified systems on the reasoning generation task, including GPT-4o [29] and Gemini-2.5 [13] as depicted Fig. 6 and Fig. In the absence of reasoning mode, GPT-4o is unable to perform reasoning generation for scenarios involving mathematical logic and physical perception. More basic text-to-image results are shown in Fig. 8 in Appendix.
4.4 Ablation Study
Effectiveness of Training Stage. We conduct an ablation study to assess the contribution of each training stage. As shown in Table 5, stage I (pre-training) initially expands the generation capability of MindOmni. Adding Stage 2 (thinking template SFT) brings significant boost, especially on WISE (+0.12). In contrast, skipping Stage 2 and applying RGPO alone leads drop in GenEval (0.72) and a smaller WISE boost (0.49), suggesting the necessity of CoT instruction SFT.
Impact of Different Connectors. We briefly compare 2 types of projectors and find that using the decoder projector can better convey the semantic from VL.M.
We evaluate our algorithm's performance under various settings for the RGPO stage, including the KI. coefficient, group numbers, and reward strategies as shown Table 7, Table and Table 9, respectively. Consistency reward plays key role in steering reasoning generation, and
Ablation on RGPO.

---

Show a girl holding this plant the autumn.
Generate an image of the animal growing up.
Multi-Modal User Input
BAGEL
Gemini2.5 Flaash
GPT-4o
GPT-4o m2: MindOmni (our)
Figure 7: Qualitative comparison among leading models on reasoning-aware image generation with multimodal user input.
gr
g
GenEval
WISE
Group Number
Format Reward
Consistency Reward
GenEval
WISE
WISE
0.01
0.79
58.8
0.01
0.06
0.81
59.8
0.79
59.2
56.1
0.004
0.01
0.80
58.9
0.81
59.8
59.4
0.01
0.78
58.2
6
0.79
59.3
59.8
Table 7: Effectiveness of differ- ent KL coefficient.
Table 8: Effeetiveness of dif- ferent group numbers.
Table 9: Effectiveness of dif- ferent reward strategies.
the inclusion of KL divergence for the visual modality helps the model maintain its fundamental text-to-image generation abilities.
5 Conclusion
In this paper, we introduce MindOmni, unified MLLM that enhances reasoning generation through reinforcement learning. Our three-phase training strategy significantly achieves impressive perfor- mance on both vision understanding and text-to-image generation tasks, outperforming existing counterparts. Furthermore, MindOmni demonstrates strong reasoning-aware generation capabili- ties, including mathematical problem-solving and time-space perception, paving the way for future advancements in multimodal AI systems.
Acknowledgments and Disclosure of Funding
This work was partly supported by Shenzhen Key Laboratory of next generation interactive media innovative technology Nb0ZZDBXYS222210023292200000 and National Natural Science Foundation of China (No.62293544, 62425117).
References
[1] Bai, S., Chen, K., Liu, X., Wang, J., Ge, W. Song, S., Dang, K., Wang, P., Wang, S., Tang, J., et al.: Qwen2. 5-vl technical report. arXiv preprint 23Xi (2025)
[2] Chen, J., Xu, Z., Pan, X., Hu, Y., Qin, C., Goldstein, T., Huang, L., Zhou, T., Xie, S., Savarese, S., et al.: Blipš-o: A family of fully open unified multimodal models-architecture, training and dataset. arXiv preprint arXiv:2505 09568 (2025)
[3] Chen, J., Yu, J., Ge, C., Yao, L.,, Xie, E., WU, Y., Wang. 7., Kwok, J., Luo, P., Lu, H., et al.: Pixart-alpha: Fast training of diffusion transformer for photorealistic text-to-image synthesis. arXiv preprint arXiv:2310.004:
(2023)
[4] Chen, I.., Li, L., Zhao, H., Song, Y.: Vinci. rl-v: Reinforcing super generalization ability in vision-language models with less than 3
10

---

[5] Chen, X., Wu, Z., Liu, X., Pan, Z., Liu, W. Xie, Z., Yu, X., Ruan, C.: Janus proo Unified multimodal understanding and generation with data and model scaling (2025)
[6] Chen, Z., Wang, W., Tian, H., Ye, S., Gao, Z., Cui, E., Tong, W., Hu, K., Luo, J., Ma, Z., et al.: How far are we to gpt-4v? closing the gap to commercial multimodal models with open-source suites. Science China Information Sciences 67(1)), 220101 (2024)
[7] Deng, C., Zhu, D., Li, K., Gou, C.,, Li, F., Wang. Z., Zhong, S., Yu, W., Nie, X., Song, Z., Shi, G., Fan, H.: Emerging properties in unified multimodal pretraining. arXiv preprint arXiv:2505. 14683 (2025)
[8] Dong, R., Han, C., Peng, Y. Qi. Z., Ge, Z., Yang, J., Zhao, L., Sun, J., Zhou, H., Wei, H., et al.: Dreamllm: Synergistic multimodal comprehension and creation. arXiv preprint arXiv:2309.1 1499 (2023)
[9] Fang, R., Duan, C., Wang, K., Huang, L., Li, H., Yan, S., Tian, H., Zeng, X., Zhao, R., Dai, J., et al:: Got: Unleashing reasoning capability of multimodal large language model for visual generation and editing, arXiv preprint arXiv:2503.10633 (2025)
[10] Feng, X., Wan, Z.,, Wen, M., McAleer, S.M., Wen, Y., Zhang, W., Wang, J.: Alphazero-like tree- search can guide large language model decoding and training. arXiv preprint arXiv:2309.17179 (2023)
[11] Ge. Y.. Zhao, S.. Zhu, J., Ge, Y., Yi, K., Song, L., Li, C.. Ding, X., Shan. Y.: Seed-x: Multimodal models with unified multi uulti-granularity comprehension and generation. arXiv preprint arXiv:2404.144 14396 (2024)
[12] Ghosh, D., Hajishirzi, H., Schmidt, L.: Geneval: An object-focused framework for evaluating text-to-image alignment. Advances in Neural Information Processing Systems pp. 521 52132-52152 (2023)
[13] Google: Gemini 2.5: Our most intelligent ai model (2025), https://blog.goon technology/ wtwgg/ooolledemenng
[14] Guo, D., Yang. D.,, Zhang, H., Song, Zhang, R., Xu, R., Zhu, Q.. Ma. S., Wang, P., Bi, X., et al.: Deepseck-rl: Incentivizing reasoning capability in Ilms via reinforcement learning, arXiv preprint arXiv:2501. 12948 (2025)
[15] Guo, Z., Zhang, R., Tong, C., Zhao, Z., Gao, P., Li, H., Heng, P.A.: Can we generate images with cot? let's verify and reinforce image generation step by step. arXiv preprint arXiv:2501.13926 (2025)
[16] Hu, X., Wang, R., Fang, Y., Fu, B., Cheng, P., Yu, G.: Ella: Equip diffusion models with Ilm for enhanced semantic alignment. arXiv preprint arXiv:2403.05135 (2024)
[17] Huang, J., Xu, Z., Zhou, J., Liu, T., Xiao, Y., Ou, M., Ji, B., Li, X., Yuan, K.: Sam-rl: Leveraging sam for reward feedback in multimodal segmentation via reinforcement learning. arXiv preprint arXiv:2505. 225966 (2025)
[18] Huang, R., Wang, C., Yang. J., Lu, G., Yuan, Y., Han, J.. Hou, L., Zhang, W., Hong. L., Zhao, H.. et al.: Illume+: Illuminating unified milm with dual visual tokenization and diffusion refinement. arXiv preprint arXiv:2504.0199 ..1944 (2025)
[19] Khanov, M., Burapacheep, J., Li, Y.: Args: Alignment as reward- guided search. arXiv preprint arXiv:2402.01694 (2024)
[20] Kumar, A., Zhuang, V., Agarwal, R., Su, Y., Co-Reyes, J.D., Singh, A., Baumli, K., Iqbal, S., Bishop, C., Roelofs, R., et al.: Training language models to self-correct via reinforcement learning, arXiv preprint 1xXv:22409.12917 (2024)
[21] Labs, B.F.: Flux. https://githut http:://gittuuu.ccoobccaa.. wtw.ccom//b//.eee forest- httpc//alle---oreetaaaaa/////// (2023)
[22] Li, B., Zhang, Y., Guo, D., Zhang, R., Li, F. Zhang, H., Zhang, K., Li, Y., Liu, 7., Li, C.: Llava-onevision: Easy visual task transfer. MrXiv:2408.03326 (2024)
11

---

[23] Lin, H., Wang, T., Ge, Y.. Ge, Y., Lu, Z., Wei, Y., Zhang, Q. Sun. Z., Shan, Y: Toklip: Marry visual tokens to clip for multimodal comprehension and generation. arXiv preprint arXiv:2505.05222 (2025)
[24] Lin, H., Xu, H., Wu, Y., Guo, Z., Zhang, R., Lu, Z., Wei, Y., Zhang, Q. Sun, Z.: Quantization meets dilms: A systematic study of post-training quantization for diffusion llms. arXiv preprint arXiv:2508.1 148966 (2025)
[25] Liu, S., Han, Y., Xing, P., Yin, F., Wang, R., Cheng, W., Liao, J., Wang, Y., Fu, H., Han, C.,, et al.: Steplx-edit: A practical framework for general image editing. arXiv preprint arXiv:2504.17761 (2025)
[26] Liu, Y., Peng, B., Zhong, Z., Yue, Z., Lu, F., Yu, B., Jia, J.: Seg-zero: Reasoning- guided segmentation via cognitive reinforcement, arXiv preprint arXiv:2503.06522 (2025)
[27] NIu, Y., Ning, M., Zheng, M., Lin, B., Jin, P., Liao, J., Ning, K., Zhu, B., Yuan, L.: Wise: A world whnowledge-ifformedd semantic evaluation for text-to-image generation. arXiv preprint arXiv:2503.07265 (2025)
[28] OpenAl: Openai ol. https//openai..com// (2024)
[29] OpenAl: Introducing 4o image generation (2025), https://openai: com
[30] Pan, X., Shukla, S.N., Singh, A., Zhao, Z., Mishra, S.K., Wang. J.. Xu, Z., Chen, J., Li, K., Juefei- Xu, F., et al.: Transfer between modalities with metaqueries. arXiv preprint arXiv:250.00255 (2025)
[31] Peebles, W. Xie, S.: Scalable diffusion models with transformers. In: Proceedings of the IEEE/CVF International Conference on Computer Vision. pp. 4195 4205 (2023)
[32] Podell, D., English, Z., Lacey, K., Blattmann, A., Dockhorn, T., Müller, J., Penna, J., Rombach, R.: Sdxl: Improving latent diffusion models for high-resolution image synthesis. arXiv preprint arXiv:2307.01952 (2023)
[33] Qiao, Y., Duan, H., Fang, X., Yang, J., Chen, L., Zhang, S., Wang, J., Lin, D., Chen, K.; Prism: A framework for decoupling and assessing the capabilities of vlms. arXiv preprint arXiv:2406. 14544 (2024)
[34] Qu, L., Zhang. H., Liu, Y., Wang, X., Jiang, Y., Gao, Y., Ye, H., Du, D.K., Yuan, Z., Wu, X.: Tokenflow: Unified image tokenizer for multimodal understanding and generation. arXiv preprint arXiv:2412.08 9.99009 (2024)
[35] Rombach, R., Blattmann, A., Lorenz, D., Esser, P., Ommer, B.: High-resolution image synthesis with latent diffusion models. In: Proceedings of the IEEE/CVF conference computer vision and pattern recognition. pp. 10684-10695 (2022)
[36] Schuhmann, C., Köpf, A., Vencu, R., Coombes, T., Beaumont, R.: Laion coco: 600m synthetic captions from laion2b-en. URL https://laion. ai/blog/laion-cooo (2022)
[37] Shao, Z., Wang, P., Zhu, Q.. Xu, R., Song, J., Bi, X., Zhang, H., Zhang, M., Li, Y., Wu, Y., et al.: Deepseekmath: Pushing the limits of mathematical reasoning in open language models. arXiv preprint arXiv:2402.03300 (2024)
[38] stability.ai: Introducing stable diffusion 3.5 (2024), https://stability.aiii ai ai/news/ introducing-stee stable- diffusion-3--
[39] Sun, K. Pan, J., Ge, Y., Li, H., Duan, H., Wu, x., Zhang, R., Zhou, A., Qin, Z., Wang, Y., et al.: Journeydb: benchmark for generative image understanding. Advances in Neural Information Processing Systems 36 (2024)
[40] Sun, P., Jiang, Y., Chen, S., Zhang, S., Peng, B., Luo, P., Yuan, Z.: Autoregressive model beats diffusion: Llama for scalable image generation. arXiv preprint arXiv:2406.0652. (2024)
12

---

[41] Sun, Q. Cui. Y.. Zhang, X., Zhang, F., Yu, Q., Wang, Y., Rao, Y., Liu, J., Huang, T., Wang,
X.: Generative multimodal models are in-context learners. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp.
(2024)
[42] Sun, Q. Yu, Q. Cui, Y., Zhang, F. Zhang, X., Wang, Y. Gao, H., Liu, J. Huang, T., Wang, X.: Emu: Generative pretraining in multimodality. arXiv preprint arXiv:2307 0.200222222222 (2023)
[43] Team, C.: Chameleon: Mixed-modal early-fusion foundation models. arXiv preprint arXiv:2405.0981 (2024)
[44]
Team, Q.: Qwen3: Think deeper, act faster. https://wwwaangggabbbbbbblggg// (2025)
[45] Tong, S., Fan, D., Zhu, J., Xiong, Y., Chen, X., Sinha, K., Rabbat, M., LeCun, Y., Xie, S., Liu, Z.: Metamorph: Multimodal understanding and generation via instruction tuning. arXiv preprint arXiv:2412.144 14164 (2024)
[46] Wang. J., Pu, J., Qi. Z., Guo, J., Ma, Y., Huang. N., Chen, Y., Li, X., Shan, Y.: Taming rectified flow for inversion and editing. arXiv preprint 1:Xiv:2411.04744, (2024)
[47] Wang, J., Tian, Z., Wang, X., Zhang, X., Huang, W., Wu, Z., Jiang, Y.G.: Simplear: Pushing the frontier of autoregressive visual generation through pretraining, sft, and rl. arXiv preprint arXiv:2504.1 1455 (2025)
[48] Wang. X., Zhang, X., Luo, Z., Sun, Q., Cui, Y., Wang. J., Zhang, F., Wang. Y., Li, Z., Yu, Q., et al.: Emu3: Next-token prediction is all you need. arXiv preprint arXiv:2409. 18869 (2024)
[49] Wang, X., Wei, J., Schuurmans, D., Le, Q., Chi, E., Narang, S., Chowdhery, A., Zhou, D.: Self-consistency improves chain of thought reasoning in language models. arXiv preprint arXiv:2203.1 1171 (2022)
[50] Wei, J., Wang, X., Schuurmans, D., Bosma, M., Xia, F,, Chi, E., Le, Q.V., Zhou, D., al.: Chain-of-thought prompting elicits reasoning in large language models. Advances in neural information processing systems 35, 24824-2483 (2022)
[51] Wu, C., Chen, X., Wu, Z., Ma, Y., Liu, X., Pan, Z., Liu, W., Xie, Z., Yu, X., Ruan, C., al.: Janus: Decoupling visual encoding for unified multimodal understanding and generation. arXiv preprint arXiv:2410. 13848 (2024)
[52] Xiao, S., Wang, Y., Zhou, J., Yuan, H., Xing, X., Yan, R., Wang. S., Huang, T., Liu, Z. Omnigen: Unified image generation. arXiv preprint arXiv:2409. 11340 (2024)
[53] Xiao, Y., Luo, Z., Liu, Y., Ma, Y., Bian, H., Ji, Y., Yang, Y., Li, X.: Bridging the gap: A unified video comprehension framework for moment retrieval and highlight detection. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 18709-18719 (2024)
[54] Xiao, Y., Song, L., Wang, J., Song, S., Ge, Y., Li, X., Shan, Y., et al.: Mambatree: Tree topology all you need in state space model. Advances in Neural Information Processing Systems 37, 75329- 75354 (2024)
[55] Xiao, Y. Song, L., Yang, R., Cheng, C., Ge, Y., Li, X., Shan, Y.: Lora-gen: Specializing large language model via online lora generation. arXiv preprint arXiv:2506. 11638 (2025)
[56] Xiao, Y., Song, I., Yang, R., Cheng, C., Xu, Z., Zhang, Z., Ge, Y., Li, X., Shan, Y.: Haploomni: Unified single transformer for multimodal video understanding and generation. arXiv preprint arXiv:2506.0297. (2025)
[57] Xie, E., Chen, J., Zhao. Y., Yu, J., Zhu, L.. Wu, C., Lin. Y., Zhang, Z., Li, M., Chen, J., et al.: Sana 1.5: Efficient scaling of training-time time and inference-time compute in linear diffusion transformer. arXiv preprint MxXiv:2501.1844 (2025)
[58] Xie, J.. Mao, W. Bai, Z., Zhang, D.J., Wang, W., Lin, K.Q., Gu, Y., Chen, Z., Yang, Z., Shou, M.Z.: Show-o: One single transformer to unify multimodal understanding and generation. arXiy preprint arXiv:2408 12528 (2024)
13

---

[59] Xu, G., Jin, P., Hao, L., Song, Y., Sun, L., Yuan, L.: Llava-ol: Let vision language models reason step-by-step. arXiv preprint arXiv:2411.10444 (2024)
[60] Xu, Z., Yu, Z., Zhou, Z., Zhou, J., Jin, X., Hong, FTT., Ji, X., Zhu, J., Cai, C., Tang, S., et al.: Hunyuanportrait: Implicit condition control for enhanced portrait animation. In: Proceedings of the Computer Vision and Pattern Recognition Conference, pp. 19909-15913 (2025)
[61] Yang. L., Yu, Z., Meng, C., Xu, M., Ermon, S., Cui, B.: Mastering text-to-image diffusion: Recaptioning, planning, and generating with multimodal llms. In: Forty-first International Conference on Machine Learning (2024)
[62] Yuan, L., Haodong, D., Yuanhan, Z. Bo, L., Songyang, Z., etc.: Mmbench: Is your multi-modal model an all-around player? AXXi:23307.0628 (2023)
[63] Yue, X., Ni, Y., Zhang, K., Zheng. T., Liu, R., Zhang, G., Stevens, S., Jiang, D., Ren, W., Sun, Y., et al.: Mmmu: A massive multi-discipline multimodal understanding and reasoning benchmark for expert agi. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 9556-9567 (2024)
[64] Zhou, C., Yu, L., Babu, A., Tirumala, K., Yasunaga, M., Shamis, L., Kahn, J., Ma, X., Zettlemoyer, L., Levy, O.: Transfusion: Predict the next token and diffuse images with one multi-modal model. arXiv preprint arXiv:2408. 11039 (2024)
[65] Zhou, H., Li, X., Wang, R., Cheng, M., Zhou, T., Hsieh, C.J.: R1-zero's" aha moment" in visual reasoning on 2b non-sft model. arXiv preprint arXiv:2503.05132 (2025)
[66] Zhou, J., Li, J., Xu, Z., Li, H., Cheng. Y., Hong, F.T., Lin, Q., Lu, Q., Liang, X.: Fireedit: Fine-graine instruction- based image editing via region- aware vision language model. In: Proceedings of the Computer Vision and Pattern Recognition Conference, pp. 13093-13103 (2025)
14

---

A Appendix
Figure 8: More qualitative results across different resolutions.
A.1 Implementary Details
In the first stage of training, we utilize the Laion COCO [36] and Journey [39] image-caption pairs as foundational text2image data. Additionally, we incorporate the X2I dataset [52], which includes tasks for computer vision [54], in-context learning, multi-modal instructions, and subject-driven. To further enrich our dataset, we leverage Sana [57] and Flux [21] to generate high-quality images based on the existing prompt. During training, we use constant learning rate scheduler with an initial rate of x 10 and weight decay of 0.05. The model is trained with a batch size of 1024 using
15

---

images at resolution of 256 x 256. In the second stage, we employ Qwen2.5-VL [1] to generate CoT instruction data. At this stage, we progressively increase the training resolution to 512 x 512. while reducing the learning rate to 5 x 10-5. In the final stage, we utilize Qwen3 [44] to generate dataset of logical reasoning texts, which serve as training data for reinforeement learning. We adopt a cosine scheduler following previous works [52, 53, 55] during the reinforcement learning phase.
We evaluate our method during multimodal understanding and various generation benchmarks. Specifically, our model is evaluated on MMMU [63], MMBench [62] and RealworldQA for image understanding. As for basic image generation tasks, we evaluate our model on GenEval [12], which involves various metrics such as counting, colors, and position. We also evaluate our text-to- image generation capability on DPG- PrGBBench [16] following previous methods [5, 34]. For reasoning generation evaluation on WISE [27] Benchmark, we perform our MindOmni into thinking mode and generate the corresponding image with 1024 x 1024.
A.2 Image Editing Performance
We conduct evaluations GEdit [25] to verify the performance on image editing task of our method. In detail, under the fair setting as previous models [25, 66], our model achieved SC and PQ scores of 6.55 and 6.92, respectively, with overall accuracy of 5.95.
A.3 Detailed Response of Our MindOmni
For the user input of different reasoning generations in Fig. 1, the detailed response of our model MindOmni is shown in Fig. 9, Fig. 10, Fig. 11 and Fig. 12.
User: Generate an image of an animal with [3 lives. MindOmmi: <thinle generate image based on the instruction "An Image an animal with [3 6) lives," first to interpret the mathematical expression given: WB3+ 6 9W This suggests that the animal should have nine lives, which common characteristic associated with mythical creatures
User: An elephant and a rabbit stand on both sides of seesaw. MindOmni: <think generate an illustration of an elephant and rabbit standing both sices a seesaw, need consider a key elements: the size cifference between the elephant and the rabbit, the balance and movement implied the seesew, and the positioning of the animals. Elephants
dragons.
larger then rabbits, they would be seesow, while the rabbit would likely be the other, opposite end.
the
Given this interpnetation, I would suggest creating an image of cat, which known having lives, The image could digital drawing or illustration to reflect its enduring and resilient nature.
The typically represented horiontal balanced central
cat, possibly a playful and whimsical style
fulcrum. The elaphanes or platform-lie structure that eatends from either side of this central bar. The elephant's end would be lower because rabbit's. The rabbit's end would likely be higher due to balancing effect of the seesaw.
rabbit would each standing on a platform
The instruction does specify the cat when reforring the concept "nine lives' due to popular belief that a eat can survive falls from great heights, often attributed to its ability land its feet.
animal further, but commonly,
is significantly greater than
For the image:
terms of style, the illustration could cartoonish for simple, child- friendly look, or more realistic If aiming for cetailed and lifelike depiction. The background can minimal, perhaps just suggesting an open space, keeps
The cat could be depicted in simple yet vibrant drawing.
it might be shown in dynamic pose, perhaps mid leap or after a sudden
the subjects.
The cot could have a light, fluffy aopesrance with lorge eyes and a small body, capturing the classic cartoonish representation a cat.
To create such an image, one might start with a simple sketch, than add details like elephant's trunks the rabbit's long ears, point, and the platforms.. Color choices could include earthy tones the elephant (lke brown) and lighter, more playful colors for the rabbit llike pink yellow. </thinko
balance
This approach aligns with the whimmical imaginative nature typically associeted with the idea of cat having nine siesik/
instructions, you could generate a digital drawing or illustration of cat. The cat should have playul and whimsscaa style, possibly depicted in dynamic such leap or after sudden
<answers create image according
<answor-Creeating an image an elophant and rabbit a secsaw involves drawing or digitally rendering two large elephants and a small rabbit placed opposite ends of seesow. The elephant should be standing near one
platform
should be the opposite end. The seesaw should be shown balancing in the middle point, with the elephant's end appearing lower due to its larger mass and the rabbit's end appearing higher.
seesaw,
The cat could drawn with light, fluffy appearance, lange eyes, and small body, capturing the classic cartoorish representation
image would visually represent the concept the cat having nine lives, as traditionally associated with feline resilience agility <IMG> saanwer>
For a simple Illustration:
1. Sketch the outtine of the seesow with the fulceum in center.
2. Add the elephant on one side with Its trunk possibly hanging down or interacting, and the rabbit on the other side.
3. Covor the elephant in darker shades, the rabbit in lighter, playful colors.
4. Ensure the elephant's lower and the rabbit's </answer>
balanced correctly, with the
Figure 9: Detailed response of our MindOmni.
16

---

User: pool minus ten degrees Calius. MindOmni: <think generate image of pool minus ten degrees Celiks,, one should consider the freezing point of water (o'C or 32"F) and the comsequences of the water freezing below this temperature. At ice, forming
User: Scone the Sydney Opera House when New York is at noon. MindOmni: TthinkoTo generate image of Sydney Opera House New York at noon, we need to consider the geographical and time difference between Sydney (which in the Castern Australia Time Zonel and New York (which in Eastern Standard Time). The difference between these two locations is approximataly 15 hours, with New York being 15 hours ahead of Sydney, Therefore, when it's noon in New York, would be approxlmately AM Sydney.
minus ten degrees Celsius, water would indeed be frozen a solid surface. The appearance would likely include clear, uncolored ice covering the pool's water, which, due the cold, would have glassy, possibly cloudy texture. image would depiit a scene entire body of water is frozen, creating a hard, loy surface on top.
The image should reflect the interior of the Sydney Opera House during this the right early moming, distinctive curved shell structures characteristic of the Opera House would likely be illuminated by the artificial lighting inside the building, which would contrast with the dark surroundings outside. The seating areas, corridors, and architectural features would be highlighted.
create such image, It might involve drawing or digitally rendering rectangular or drrular pool shape filled with smooth, fat, transporent ice layer. The background could show the outline of the pool with visiblle edges, suggesting boundary depicted in light blue or white color, with subtle shading to indicate the thickness of the ice and any slight irregularities that could be present in real-life scenarios. The image minus ten degrees Celsius, the water has already froze..
area.
surface
not include any liquid water,
Given the specific time would close dawn, there might subtle hint light or the appearance of sunrise through windows or doors leading the interior.
The visual representation
static the image would not capture movement; would facus salely on the frozen state of the water, highlighting the transformation from liquid solid under extreme cold conitoors</thinkk
In terms of capturing this scene artistically, the focus would be on the interplay shadows and light within the interior design the Opera House, modern sesthetic the structure, overal
emphas futuristic atmosphere would be calm and sarone, with the dariness outside juxtaposed against the bright interior l/ghts,</think> canswere could best conweyed through a digital Illustration or photo composition. You might use software like Adobe Photoshop or digital drawing tools where you draw composite elements of the interior the Sydney Opera House, specfically highlighting its iconic architectural festures like the white curves and the rows of seating, al rendered Inside the building.
canswer> To create the image, start by drawing a rectangular or circular outline representing the pool. Fill the pool with flat, transparent ice rendered or white color, with siiht variations in hue to suggest thicknoss and the potential for small bubbles impurities trapped the ice. Ensure the ice layer smooth but shows some variation texture to reflect its real- world appearance.
layer covers entrre surface.
The overall image should emphasiee the immobility and stiliness the froze with benesth key surface should be depicted as hard, solid cover over the entire pool area, with no thawing or image. surrounding the pool should be kept neutral to avaid drawing attention from the main subject- frezen pool, <IMG>> </answer>
The scene would be set in still, minimalist ervironment mimicking AM, with ambient lighting creating soft pattorns across the surfaces of the seats and the ceiling. You could add faint glow from the windows showing the beginning daylight outside to provide a sense of the time. The lighting within the Opera House would depict a variety light sources: warm accent lights dramatic affect, and the soft, scattered lights from a dim evening ambiance. <IMG> </answer>
indication liquid
The
melting
environment
Figure 10: Detailed response of our MindOmni.
A.4 Linitation&Futtun Work
In this work, our Th-ougghht (CoT) content is presented in plain text, while multimodal CoT processes will be addressed in future research. Furthermore, enhancing reasoning efficiency [24] and expanding the unified model to support additional modalities and tasks [56, 17, 60, 66, 46) represent promising prospects for future investigation.
A.5 More Qualitatvie Results
We provide more qualitative results about text2image generation of our MindOmni as shown in Fig. 8.
17

---

An China's national treasure animal. MindOmal: <thinko gonerate an image China's national treasure animal, one should consider which animal is commonly recogrized as the "national treasure" Chinese culture. This is often the glant panda (Alluropoda melanolouca). Pandas loonic symbols
User:
An image multiole apples, the quantity apples is the solution equation 2
MindOmni: <thinko generate the image, we need to solve the given equation for first Vxx2 2+2 2 111. Subtracting 2 from both sicees Vee2 - 91). Taking square root both sides, we get Vix \pma)) Since the number of apples must be positive integer, we consider only the positive solution, which 3. Therefore, the image should depict three apples.
and are known for their distinctive black- andhk-en- fur pattern.
instruction does not spectly the medium style image, we can create a digita illustration of panda. Pandas typically have a round face, large dark and a body that mostly white with black markings around their eyes, cars, logs, shoulders.
The next could be a simple drawing or digital Illustration showing three apples. The apples can be depicted in any style, such as cartconish realistic representation, but the key point is that there should be exactly three apples in image-x/thee //thnkk
image that visually represents this solution. This
For the style, simple cartoon- like realistic style could be used, depanding the intended audience and contest. image be educational or for children, a cartoonish style might be more appropriste. For a more formal or artistic representation, a more detailed realistic depiction would suitable.</hhinke
canswer> To treate the image, you would drow or digitally render three apples. The aoples can be placed in row, stacked, or arranged any other way clearly to convey the solution to the equation 2 -11 by visually
image should simple enough
canswer> To create an image China's national treasure animal, such a giant panda, could drawing characteristic features: round face, large dark eyes, and a body that predeminantly white with black patterns around eyes, ears, legs, shoulders. simple cartoon style or more detailed, realistic rendering can both capture the essence of panda. <IMG> </answer>
representing the quantity of apples, which 3.
You might use a white background with the apples natural color like red your prefarence. you're using digital tool, you can draw the apples directly use ciip art to place them the simplictty ensure viewer eastly understand that the image represents the solution to the equation, which is three apples, <IMG> </answer>
depending
The focus should be clarity
Figure 11: Detailed response of our MindOmni.
User: <MGG Generate an image animal growing up,
MindOmai: cthinio To generate an image according given instruction, we need to consider what the teopole will look like once has into frog. Typically, frag characterited its four legs, flat body, and prominent head with eyes top. tail, which distinct feature of the tedpole stage, will have disappeared as the frog develops. The transit ion from tadpole to frog involves sigiffiant physical changes that include the development
before the frant the growth of the lungs for breathing air, and the traneformation of the tall into hind legs. The skin also thickens, becoming smoother and drier.
Given the instruction to generate an image the animal after growing up, will create a depiction of a fully developed loped frog, which is the adult form of the frog species depicted here. </thinke-ranswer- To
image of frog after the tadpole stage, might imagine frog robust bocl, smooth dry skin, and rell-developed and strong enough to support the frog's wright. The frog's head would large and round, with huiging eyes. It would likely have a strardinee body shape and a broad snout with moist appearance.
sturdy legs, inclucing
legs
image you depict frog sitting on
possibly near water, given
most frogs or near water sources during their tadgole phase but transition to terrestrial environments sdults. The frog's co coloration can vary greatly depending on the species, but many frogs have patterns that them blend into their environment, such green, brown,
Since you didn't specity any particular species frog, the image could represent gar ganeric which would typically these characteristk features.
specity
Peese note that this description a general representation based on common frog dharacteriatlss you're looking for specific species, ddditional detalls about the frog's pattem, would be necessary accurately depict the frog.
such coloration and
<IMG2> /answer>
Figure 12: Detailed response of our MindOmni.
18

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: The claims in the abstract and introduction accurately reflect motivation and contribution.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
is fine to include aspirational goals as motivation as long as is clear that these goals not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations the work performed by the authors?
Answer: [Yes]
Justification:
We have discussed the limitations of our work.
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't 't acknowledged judgment and recognize that individual actions in favor of transparency play impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
the paper. The authors should use their best
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [NA]
19

---

Justification:
The paper does not include theoretical results.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced the statement of any theorems.
The proofs can either appear the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged to provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Question:
Answer: [Yes]
Justification: We describe the experimental details both the main paper and the appendix. We will make our code publicly available, along with detailed instructions for reproduction.
Guidelines:
The answer NA means that the paper does not include experiments. If the paper includes experiments, No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make it clear how reproduce that algorithm. (b) If the contribution is primarily
model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution is a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome describe the particular way they provide for reproducibility, In the case of closed-source models, it may be that access to the model is limited in some way (e.g., registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
20

---

Answer: [Yes]
Justification:
We provide the core part of our code in the supplementary material.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nipp cc/
for more details.
While we encourage the release of code and data, we understand that this might not be possible, "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (c.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https: /Codeeuomiisseeeeeeee
/nips. wwtppclcccc/ccccccccccccccccclct /odee ssssssoliccyyyy for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible supplemental material (appended the paper) is recommended, but including URLs to data and code is permitted.
6. Experimental setting/details
Does the paper specify all the training and test details (c.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary understand the results?
Question:
Answer: [Yes]
Justification: The complete experiment details are introduced in the main paper and the appendix.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, as supplemental material.
7. Experiment statistical significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer:
[Yes]
Justification:
We provide the statistical analysis results in the appendix.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the enror bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
21

---

It should be clear whether the error bar is the standard deviation or the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out range (e.g. negative еπхо rates).
If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification: We have described the resources required to perform our experiments in the appendix.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary failed experiments that didn't make into the paper).
9. Code ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurlPS Code of Ethics https:
Answer: [Yes]
Justification: Our work conforms with the NeurIPS Code of Ethics in every respect
Guidelines:
The answer NA means that the authors have reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make to preserve anonymity (e.g.. if there is special consid- cration due to laws or regulations their jurisdiction..
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [No]
Justification: Our work mainly focuses on the research of basic neural network architecture. We have identified no potential negative social impacts.
Guidelines:
The answer NA means that there no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
22

---

The conference expects that many papers will be foundational research and not tied particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms monitor how system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification: Our paper poses such risks.
Guidelines:
The answer NA means that the paper poses such risks.
Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
The creators or original owners of assets (e.g., code, data, models), used the paper, properly credited well as the license and terms of explicitly are mentioned and properly respected.
Justification:
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset used and, possible, include URI.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (c.g.. website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms use in package should be provided. For popular datasets, paperswi twwode com datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
23

---

For existing datasets that are re-packaged, both the original license and the license of the derived asset (if has changed) should be provided.
If this information is not available online, the authors are encouraged to reach out to the asset's s creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer:
[NA]
Justification:
No new assets introduced.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/moddl as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset used.
At submission time, remember to anonymize your assets (if applicable). You can either create anonymized URL or include anonymized zip file.
14. Crowdsourcing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?
Answer: [NA]
Justification: Our work does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According the NeurIPS Code of Bthies, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: Our work does involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurlPS Code of Ethics and the guidelines for their institution.
For initial submissions, not include any information that would break anonymity (if applicable), such the institution conducting the review.

---

16. Declaration of LLM usage
Does the paper describe the usage of LLMs if it is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, or formatting purposes and does not impact the core methodology. scientific rigorousness, originality of the research, declaration is not required.
Question:
Answer: [NA]
Justification: The core method development in this research does not involve LLMs as any important, original, or non-standard components.
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, or non-standard components.
Please refer to our LLM policy (https: / /ttp///ersp////// theurips.cc/Confeeee /eeeneer///eee22//////// 20LM/LLM) for what should or should be described.
25


Reviews:
[{"review_id": "Mtq2z0XMDh", "reviewer": "Reviewer_s7fx", "summary": "The paper introduces RG-VLM, a unified multimodal large language model that advances reasoning generation in vision-language tasks by integrating reinforcement learning via the Reasoning Generation Policy Optimization (RGPO) algorithm. The approach involves a three-phase training strategy: the creation of a decoder-only diffusion-based unified architecture, supervised fine-tuning using Chain-of-Thought (CoT) instruction data, and further training via RL with multimodal feedback rewards. RG-VLM demonstrates strong performance in both image understanding and generation tasks, outperforming recent unified models on established benchmarks, and excels on complex reasoning and editing scenarios. The paper includes empirical evidence from several public benchmarks and qualitative results across a spectrum of vision-language scenarios.", "questions": "How does RG-VLM compare (quantitatively) with the very latest closed and proprietary models (e.g., Gemini-2.5, GPT-4o) on publicly available benchmarks, given that only qualitative results (Figure 6) are provided? Are there any plans or ongoing work to provide more direct numerical comparisons with these models?", "limitations": "yes", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths:\n1. Reinforcement Learning for Reasoning Generation: The integration of RGPO, a reinforcement learning algorithm with multimodal feedback and explicit Chain-of-Thought supervision, is well-motivated and distinguishes RG-VLM from models relying solely on supervised or prompt-based approaches. The reward shaping strategy is explained in detail, offering insights into how multimodal consistency and CoT structure are operationalized during policy optimization.\n2. Image Editing/Generation with Reasoning: Figure 1 and Figure 6 visually demonstrate RG-VLM’s strengths in both plain text reasoning (with CoT) and complex image editing/generation. For instance, Figure 1’s concrete reasoning scenarios and vision editing examples illustrate capabilities not present in competitor models.\nWeaknesses:\n1. Limited Multimodal CoT: The current CoT process is stated (Appendix A.3, but addressed in main text as well) to operate only in plain text; true multimodal reasoning chains (i.e., with explicit image intermediate steps or edits) are left for future work. This constrains the novelty and applicability of the reasoning generation claim, as current generation is essentially textual CoT guiding generation, rather than fully realized multimodal reasoning sequences.", "was_revised": true, "final_justification": "I appreciate the authors providing a thorough response to my concerns. Most of my concerns have been addressed.", "rebuttal": "We sincerely appreciate the reviewer for the time and comments and will address the concerns raised.\n\n**W1: Limited multimodal CoT.**\n\n**Ans:**\nOur work aims to move beyond the limitations of conventional fine-tuning and prompt rewriting strategies, which often fail to fully harness the reasoning potential of large models due to incorrect text guidance or inefficient prompt following execution. We introduce a reinforcement learning-based approach that unleashes the reasoning generation capabilities under our developed unified VLM. While current CoT reasoning remains text-based, we have shown it can substantially improve performance, laying a foundation for future multimodal CoT extensions research.\n\n| **Method** | **WISE Overall Score** |\n|------------|:------------:|\n| Baseline  | 0.44 |\n| Prompt-Rewrite by Qwen2.5  | 0.49 |\n| RG-VLM (w/ CoT) | 0.61 |\n\n**Table 10: Effectiveness of the CoT of our model.**\n\n**Q1: Closed models' performance.**\n\n**Ans:**\nWe perform iterative training using open-source data and evaluate GPT-4o on the WISE benchmark. As shown in Table 11, our model significantly narrows the performance gap with closed-source counterparts compared to open models.\n\n| **Method** | **WISE Overall Score** |\n|------------|:------------:|\n| Janus-Pro  | 0.35 |\n| MetaQuery-XL  | 0.55 |\n| RG-VLM (w/ CoT) | 0.71 |\n| GPT-4o | 0.80 |\n\n**Table 11: Performance Comparison.**"}, {"review_id": "gfnPwPO0BU", "reviewer": "Reviewer_Joq5", "summary": "The paper proposes RG-VLM, a unified VLM.  RG-VLM attaches an OmniGen diffusion decoder to a Qwen-2.5 backbone via lightweight connectors and fine-tunes it with Reasoning-Generation Policy Optimisation (RGPO): an RL scheme that extends GRPO with a CLIP-based “consistency” reward and dual KL regularisers. RG-VLM delivers SoTA or near-SoTA scores on six benchmarks spanning multimodal understanding (MMMU, MMBench, RealWorldQA) and reasoning-centric generation (GenEval, DPG-Bench, WISE), while preserving baseline comprehension performance.", "questions": "The output images from RG-VLM appear to have simpler backgrounds, saturated colors, and reduced aesthetic appeal. Could this be due to the reward design? What might improve this?", "limitations": "The limitations described in the paper seem more like future work. Insufficient attention is paid to the real limitations.", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 2, "strengths_and_weaknesses": "Strengths:\n- Brings chain-of-thought (CoT) reasoning into image generation/editing, advancing multimodal agent capabilities.\n- RGPO’s extension over GRPO is well documented; ablations (Tables 6-9) isolate each reward/regulariser’s effect.\n- Six quantitative benchmarks plus qualitative comparisons with GPT-4o and Gemini 2.5 demonstrate versatility.\n- Appendices list datasets, hyper-parameters, and training stages, making replication easier.\n\nWeaknesses:\n- RGPO adds a CLIP reward and dual KL loss to GRPO; similar CLIP-guided RL appeared in SimpleAR (2025), not much has changed conceptually.\n- All alignment metrics are CLIP-based; no user study confirms that generated images faithfully reflect the model’s textual reasoning.\n- Consistency reward is computed on low-resolution latents, potentially encouraging adversarial CLIP features rather than genuine semantics.\n- The number of available examples is limited. Including more would help reinforce the paper's claims.", "was_revised": true, "final_justification": "I appreciate the added experiments, but concerns about color saturation and aesthetics remain. No concrete evidence shows improved color fidelity or appeal. I keep my rating.", "rebuttal": "We sincerely appreciate the reviewer for the precious time and valuable comments and will address the concerns raised.\n\n**W1: The difference from SimplerAR​.**\n\n**Ans:** \nOur contributions lie in two key areas: (1) constructing a unified VLM capable of both understanding and generation, and (2) leveraging reinforcement learning to unleash reasoning generation. To the best of our knowledge, SimpleAR and our work are conducted concurrently. But, unlike SimpleAR, which is an image generation expert method and employs a single KL constraint, our method supports both understanding and generation, as well as integrates a dual-KL formulation. We have included comprehensive ablation studies as shown in Tables 7-9 of the manuscript to validate its effectiveness within a unified architecture.\n\n**W2: Lack of user studies.**\n\n**Ans:** \nWe conduct a user study as shown in Table 12 below, with 50 participants following the previous method. Participants are required to select the most preferred results from a set of methods, including only-support generation, FLUX.dev, and the unified understanding and generation counterparts, Janus-Pro and Bagel.\n\n| **Method** | **User Study** |\n|------------|:------------:|\n| FLUX.dev  | 0.32 |\n| Janus-Pro  | 0.10 |\n| Bagel | 0.28 |\n| RG-VLM (ours) | 0.30 |\n\n**Table 12: User Study.**\n\n**W3: Questions about consistency reward.**\n\n**Ans:**\nPotential misunderstandings may arise here. In the third stage, our consistency reward is calculated on the generated RGB image rather than the low-level latent representation, as detailed in Section 3.4 of the manuscript. Additionally, we conduct ablation studies to evaluate the effects of different reward models under consistent settings as shown in Tables 7-9 of the manuscript. Switching to a VLM-based reward model directly offers no substantial advantage compared to the CLIP model. We will further investigate the role of the reward model in the reinforcement learning process in our future work.\n\n| **Reward Model** | **WISE Overall Score** |\n|------------|:------------:|\n| CLIP-ViT  | 0.598 |\n| VisualPRM  | 0.606 |\n\n**Table 13: Ablation of different reward models.**\n\n**W4: The Number of available examples is limited.**\n\n**Ans:**\nWe have presented some text-controlled image generation results in Figure 7 in the appendix part of the manuscript, but due to rebuttal policy limitations, we cannot provide more at this rebuttal period. We will release our model publicly and update the revised version with more diversity visualizations.\n\n**Q1: The output images have simpler backgrounds.**\n\n**Ans:**\nIf the reviewer is referring to the example in Figure 1, the observed content stems from the model-generated CoT, which includes certain stylistic descriptions, which are illustrated in Figures 8–10 in the appendix. Our model exhibits strong prompt-following capabilities, which likely accounts for this phenomenon. By tuning the temperature and sampling configurations of the VLM, one can control the variability of the generated outputs. What's more, additional results of text-controlled generation are presented in Figure 7.\n\n**Q2: Insufficient attention is paid to the real limitations.**\n\n**Ans:**\nThe primary limitation at present lies in the reward function. The CLIP-score-based consistency reward does not adequately capture low-level image details or numerical information. We plan to further explore this direction in future work."}, {"review_id": "9EJOvpWvCR", "reviewer": "Reviewer_BGbw", "summary": "The paper introduces RG-VLM, an MLLM model that relies on reinforcement learning in visual reasoning generation process. The authors propose Reasoning Generation Policy Optimization (RGPO) algorithm to effectively guide multimodal policy update. RG-VLM is compared with several counterpart approaches, showing its clear reasoning advantage.", "questions": "1. Are there any other limitations of RG-VLM besides the one mentioned in Appendix A3?\n2. It will be interesting to see how effective would RG-VLM be in solving „classical” abstract reasoning tasks, such as Raven’s Progressive Matrices or Bongard Problems.\n3. How the RG-VLM Chain-of-Thought reasoning conceptually differs from that of DeepSeek?", "limitations": "Discussion of limitations is extremely brief (just one sentence).", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths:\n1. Introduction of RG-VLM, and MLLM dedicated to solving visual reasoning tasks and showing strong performance across diverse benchmark datasets.\n2. Proposal of RGPO algorithm that has proven very effective in the Chain-of-Thought reasoning. \n3. The research is very well motivated and the main ideas are clearly presented.\n\nWeaknesses:\n1. The experimental evaluation completely omits abstract visual reasoning benchmarks, i.e. Raven’s Progressive Matrices, Bongard Problems, Visal Analogy Problems, and other.\n2. A discussion on RG-VLM limitations is extremely brief.", "was_revised": true, "final_justification": "My concerns are mostly addressed, however, I still believe RG-VLM requires more extensive experimental evaluation on other types of reasoning tasks to confirm its universality and overall strength.", "rebuttal": "We sincerely appreciate the reviewer for the constructive comments and will address the concerns raised.\n\n**W1&Q2: Abstract visual reasoning benchmark.**\n\n**Ans:** \nWe appreciate the reviewer’s valuable suggestion. While our work mainly centers on reinforcement learning strategies for image reasoning generation, our dual KL approach ensures that model performance does not deviate significantly from the original policy. Accordingly, as shown in Table 14, our RG-VLM performs similarly to Qwen2.5-VL on the visual reasoning benchmark mentioned by the reviewer.\n\n|      **Model**    | **Raven** | **Bongard Problems** | **Visal Analogy Problems** |\n|-------------------|:---------:|:---------:|:--------------:|\n| Qwen2.5-VL        |    0.10   |   0.16    |       0.193    |\n| RG-VLM (ours)     |    0.10   |   0.15    |       0.184    |\n\n**Table 14: Training GPU Hours comparison for unified models.**\n\n**W2&Q1: Discussion on RG-VLM limitations is extremely brief.**\n\n**Ans:**\nAn additional limitation at present lies in the reward function. The CLIP-score-based consistency reward does not adequately capture low-level image details or numerical information. We plan to further explore this in future work.\n\n\n**Q3: Conceptual differences between RG-VLM's and DeepSeek's Chain-of-Thought reasoning.**\n\n**Ans:**\nRG-VLM is capable of both multimodal input and output, whereas DeepSeek is limited to textual processing. Moreover, the Chain-of-Thought (CoT) in RG-VLM serves as a reasoning mechanism to enhance visual generation, while DeepSeek’s CoT is designed solely for textual reasoning tasks."}, {"review_id": "Py6pBVLr90", "reviewer": "Reviewer_Si6L", "summary": "The paper presents MindOmni, a unified multimodal large language model that enhances reasoning generation in vision-language models via the proposed Reasoning Generation Policy Optimization (RGPO) algorithm. It employs a three-phase training strategy, combining a unified vision-language model with a decoder-only diffusion module, supervised fine-tuning with Chain-of-Thought data, and RGPO to leverage multimodal feedback for policy updates. Experiments demonstrate that MindOmni achieves impressive performance on both understanding and generation benchmarks, particularly in mathematical reasoning and spatial-temporal tasks, demonstrating its advanced fine-grained reasoning generation and image editing capabilities. The work advances the field by enabling deeper reasoning-driven generation in unified multimodal models, bridging the gap between vision understanding and complex reasoning tasks.", "questions": "* The claim that the model can \"accurately play chess\" is a particularly strong statement. However, the paper presents no experimental results, evaluation protocols, or qualitative/quantitative evidence to support this assertion. Additional experiments and analyses are needed to substantiate this claim.\n\n*  Policy collapse, entropy collapse, and reward collapse are common issues during RLVR training. Although the authors propose a dual KL-divergence regularization technique as an interesting stabilization method, the paper lacks an in-depth analysis of its effectiveness, such as a thorough comparison with standard single-KL regularization. Moreover, the phenomenon of \"overthinking\" revealed in the results is not discussed in the context of existing literature.", "limitations": "Yes", "rating": 4, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "**Strengths**\n* This paper is clearly written and easy to follow.\n* The core contribution of this work lies in the application of a class of policy gradient reinforcement learning algorithms—specifically, the proposed RGPO—to unified multimodal generation models. By explicitly modeling and optimizing the CoT reasoning process, the authors provide a strong baseline for enhancing the controllability and complex instruction-following abilities of multimodal models. Furthermore, this approach offers potential insights for the development of o4-like models and world models.\n\n**Weaknesses**\n* The consistency reward based on CLIP similarity represents a major limitation of RG-VLM. Extensive prior work has demonstrated that CLIP performs poorly in evaluating complex spatial, physical, or logical relationships. For example, while CLIP can recognize objects within an image, it often fails to verify the correctness of their spatial arrangements. As a result, the reward signal serves as a very coarse and unreliable proxy for reasoning ability. An unhackable and trustworthy reward is arguably the most crucial factor for RLVR training effectiveness and stability.\n\n* In the critical second stage of the pipeline, the paper uses another model (Qwen2.5-VL) to synthesize CoT data, yet provides no description of its quality control, diversity assurance, or bias filtering mechanisms. This raises concerns that the model’s reasoning ability may be inherently limited by the teacher model’s upper bound, and that it may end up merely imitating the specific \"thinking style\" of the teacher model rather than acquiring generalized reasoning skills. In current industry practice, there are primarily two cold-start strategies: distillation or zero-shot self-bootstrapping followed by filtering. If the CoT data is generated by a model lacking genuine reasoning capacity, it is unlikely to exhibit robust reasoning patterns.", "was_revised": true, "final_justification": "The authors have clarified the issues regarding the acquisition of Chain-of-Thought abilities and the details of the RLVR reward. I recommend this paper for acceptance.", "rebuttal": "We begin by sincerely appreciating the reviewer for the respectful and patient feedback. We completely understand the concerns raised and will address them individually for further clarification.\n\n**W1: Question about the CLIP reward.**\n\n**Ans:** \nWe agree with the reviewer that CLIP-based similarity metrics have known limitations in capturing complex spatial or logical relationships. However, our main contributions are the proposal of a unified reasoning-generation model and a training RL framework that enhances its reasoning generation ability. While we initially chose CLIP rewards to discourage benchmark hacking, we also explored a more capable reward model that better aligns with fine-grained reasoning. This alternative yielded a 0.8-point performance improvement under the same settings, as shown in Table 13 (in the response to reviewer Joq5’s W3).\n\n**W2: Question about the training stage 2.**\n\n**Ans:** \nOn the one hand, the CoT data in stage 2 is composed of short, coarse image captions as input prompts, with the corresponding “reasoning” content being detailed and descriptive long-form captions, which are generated by Qwen2.5VL-72B. On the other hand, we utilize this data to train only the generation decoder, enabling it to adapt specifically to Chain-of-Thought contextual inputs. The VLM component remains frozen throughout this phase, thereby preserving its general reasoning capabilities and ensuring that the model is not constrained by the limitations of the additional model. If this stage is omitted, the model's performance on the WISE benchmark will drop from 60% to 49% as shown in Table 5 in the manuscript.\n\n**Q1: The claim that the model can ”accurately play chess” is a particularly strong statement.**\n\n**Ans:**\nWe regret any potential misunderstanding. Our goal was to underscore the model’s ability to perform fine-grained editing tasks, such as manipulating chess pieces according to specific instructions. We create a chessboard test dataset to assess this functionality and also evaluate the model on public editing benchmarks (GEdit and ImgEdit) as shown in Table 15 below. This fine-grained control is essential for enabling future research into adaptive chess playing. We will include these experimental results and the corresponding benchmark descriptions in the revised version.\n\n|                | **Chess** | **GEdit** | **ImgEdit** |\n|----------------|:------------:|:----------:|:--------:|\n| baseline       |      0.45    |    0.503   |   0.296  |\n| RG-VLM (ours)  |      0.52    |    0.599   |   0.333  |\n\n**Table 15: Performance of editing task.**\n\n**Q2: Question about training stage 3.**\n\n**Ans:**\nWe have provided extensive ablation studies in Tables 7–9 of the manuscript, evaluating different aspects of the RGPO reinforcement learning strategy, including KL regularization schemes, group numbers, and reward strategies. Specifically, Table 7 shows that standard single-KL term ($\\beta^{I}$=0) reduces performance by 1.6 points on the WISE benchmark, underscoring the effectiveness of our dual KL approach. Furthermore, Figure 4 illustrates stable reward behavior throughout training, suggesting that KL regularization effectively mitigates reward collapse. We acknowledge the overthinking phenomenon observed during our RL process and note that it aligns with findings in prior works [1,2]. We will incorporate a discussion of this effect and relevant citations in the revision.\n\n[1] Mitigating Overthinking in Large Reasoning Models via Manifold Steering\n\n[2] Think How to Think: Mitigating Overthinking with Autonomous Difficulty Cognition in Large Reasoning Models\n\nPlease let us know if you have any unsolved or other concerns."}]

Author rebuttal:

Meta-review: This work proposes a unified visual understanding and generation framework and enhances the reasoning capacity of the model by reasoning generation policy optimization (RGPO) algorithm. The study is well motivated, and the empirical result is impressive. All reviewers have positive scores and concerns, such as the acquisition of Chain-of-Thought abilities, and the details of the RLVR reward, were addressed after rebuttal. Overall, the work demonstrates a promising direction. Please incorporate the comments from all discussions and improve the revision with more extensive experimental evaluation as suggested, e.g., different reasoning tasks, complex background, etc.
Decision: Accept (poster)