UniGen: Enhanced Training & Test-Time Strategies for Unified Multimodal Understanding and Generation
Authors: Rui Tian
Mingfei Gao
Mingze Xu
Jiaming Hu
Jiasen Lu
Zuxuan Wu
Yinfei Yang
Afshin Dehghan
Venue: NeurIPS 2025 poster ()
Keywords: Unified Multimodal Understanding and Generation
Multimodal Foundation Model
Multimodal LLM
Primary area: deep_learning
TL;DR: We introduce UniGen, a unified multimodal large language model capable of image understanding and generation.

Abstract: We introduce UniGen, a unified multimodal large language model (MLLM) capable of image understanding and generation. We study the full training pipeline of UniGen from a data-centric perspective, including multi-stage pre-training, supervised fine-tuning, and direct preference optimization. More importantly, we propose a new Chain-of-Thought Verification (CoT-V) strategy for test-time scaling, which significantly boosts UniGen’s image generation quality using a simple Best-of-N test-time strategy. Specifically, CoT-V enables UniGen to act as both image generator and verifier at test time, assessing the semantic alignment between a text prompt and its generated image in a step-by-step CoT manner. Trained entirely on open-source datasets across all stages, UniGen achieves state-of-the-art performance on a range of image understanding and generation benchmarks, with a final score of 0.78 on GenEval and 85.19 on DPG-Bench. Through extensive ablation studies, our work provides actionable insights and addresses key challenges in the full life cycle of building unified MLLMs, contributing meaningful directions to future research. Code is available at https://github.com/apple/ml-unigen.

UniGen: Enhanced Training & Test--Tmme Strategies for Unified Multimodal Understanding and Generation
Rui Tian 20, Mingfei Gao2 Mingze Xu
Jiaming Hu?, Jiasen Lu², Zuxuan Wu' Yinfei Yang", Afshin Dehghan
"Institute of Trustworthy Embodied AI, Fudan University AApll minaae2,minnzze.e adehghan]@apple.,, com,
com, (rtian23, 2xwu))Fuanne First authors; *Corresponding authors
Abstract
We introduce UniGen, a unified multimodal large language model (MLLM) capable of image understanding and generation. We study the full training pipeline of Uni- Gen from data data-centric perspective. including multi-stage pre- pre-training, supervised fine-tuning. and direct preference optimization. More importantly, we propose a new Chain-of- Thoughht Verification (CoT-V) strategy for test- scaling, which significantly boosts UniGien's s image generation quality using simple Best-of-N test-time strategy. Specifically, CoT-VV enables UniGen to act as both image genera- tor and verifier test time, assessing the semantic alignment between a text prompt and its generated image in step-by-step CoT manner. Trained entirely open- source datasets across all stages, UniGen achieves state-of-the-arr performance range of image understanding and generation benchmarks, with a final score 0.78 on GENEVAL and 85.19 on BPG-BENCH. Through extensive ablation studies, work provides actionable insights and addresses key challenges in the full life cycle of building unified MLLMs, contributing meaningful directions to future research. Code is available at https: / gi thub. com/ apple/ml unigen.
Show-o
Janus
Janus-Pro
UniGen
100
84
76.5 75.5
78.4
78
79.7
68.3
70.8
73
83.7
61.5
63.7
53
52
49
44
42.7
36
36.2
38.7
22.1
MathVista
ScienceQA
Seedbench
GenEva
Understanding
Generation
Figure I: Comparison against state-of-the-art unified MILMs. UniGen-I.5B outperforms Show- o-1 1-1.3B, Janus- 1.3B and Janus-Pro- Prass-Pro-l.a 1.5B across understanding and generation benchmarks.
1 Introduction
Unifying understanding and generation within a single framework represents key step toward general-purpose artificial intelligence models [53]. Pioneering work [9, 16, 67, 83, 84, 87, 96] has made encouraging progress but relies on distinct training recipes and internal datasets. More
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

importantly they have yet to demonstrate good practice in wisely collaborating these two capabilities within a unified architecture achieve substantial performance gains. We advance the development of unified multimodal large language models (MI.LMs) by carefully studying the impact of their training recipes across different stages and proposing optimizations to improve both image understanding and generation. We further explore leveraging test-time interaction between understanding and generation tasks, selecting images with higher quality by using our unified MLLM as the velfvveriierr
Specifically, we introduce UniGen, unified MLLM for image understanding and generation. To shed light on the impact of different training stages, we walk through the entire life cycle of the model development, including multi-stage pretraining, supervised fine-tuning [40, 54, 56], and direct preference optimization [60. 78]. We ablate the impact of each training stage and their design choices from a data-centric perspective, and draw insightful lessons for building advanced unified MLLMs. Unlike state-of-the-art models [9, 47, 87, 84] that rely on large-scale internal datasets, we curate new data mixtures across training stages by using only open-source images. We show that models trained on publicly available data can also achieve competitive results.
To further enhance image generation quality, we propose new Chain-of-Thoughee Verification (CoT- v) strategy for test-time scaling. The key idea is to leverage UniGen's inherent understanding ability as a self-verifier to assess the quality of its own generated images. Specifically, during inference, UniGen produces N images for a given text prompt, while CoT-V progressively evaluates semantic coherence between each image-text pair and selects the best. With only lightweight fine-tuning (e.g., 500 training steps), UniGen is able to achieve the reasoning capability, thinking step-by-ster to verify each atomic fact according to the prompt and each generated image. Importantly, this CoT verification seamlessly enhances UniGen's 's image generation quality while preserving its general understanding performance. In this way, we collaborate the understanding and generation capabilities within a unified MLLM, substantially boosting the text-to-image generation quality using a simple Best-of-N strategy [79, 106] and self-verification [82, 7, 24]. Our experiments show that UniGen's s performance is consistently improved across various image generation benchmarks.
We evaluate UniGen on various understanding and generation tasks, as shown in Fig. 1. For image understanding, UniGen outperforms comparable unified MLLMs (e.g. Show-o [87] and Janus- Pro [9]) across benchmarks and even ties with some strong understanding specialist models. such as LLaVA-OV [32] and MM1 .5 [102], as displayed in Table 1. For text-to-image generation, UniGen obtains 0.78 on GENEVAL and 85. 19 on DPG-BENCE using only open-source data, surpassing state-of-the-ar unified MIL.MS [87, 84, 9] by a clear margin.
2 Related Work
Multimodal Large Language Models (MLLMs) have advanced significantly in image [1. 11, 40, 41, 49, 77, 106] and video understanding [42, 103, 89, 90, 101, 108]. Their architecture typically consists of a vision encoder [59, 100, 72] to extract visual features, a projector [34, 2] to align image-text embeddings, and large language model (LLM) [1, 71, 92, 10] to generate responses. Early work focuses on pre-training using large-scale vision-language corpus [48, 69], then moves to carefully curated instructional datasets for supervised fine-tuning [32, 102] and reinforcement learning [27, 60]. Recently, enabling MLLMs to output explicit reasoning trajectories has become a promising research direction [52. 19, 68]. They explore strategies, such as
(CoT) prompting [13, 95], reinforcement learning [66, 88], and test-time scaling [106, 79] to enhance the visual reasoning capabilities of MLLMs.
Unified Understanding and Generation aims to combine visual understanding and generation within a single MLLM framework [51, 67, 58, 81, 44, 43, 8, 12, 35]. This is often achieved by jointly optimizing LLMss with multimodal objectives and generation
losses, such as autoregressive decoding [84], diffusion [105], flow-matching [47], and masked image prediction [94, 87]. Visual tokenizers [14, 45, 50, 73, 75, 98, 104] are critical for enabling both semantic understanding and high-fidelity generation. Recent efforts explore both decoupled encoders [70, 84) and unified tokenizers [29, 58, 85] for better task balancing. Integrating CoT into visual generation emerges as a promising strategy. PARM [20] scales test-time computation by introducing new verification process. MINT [8] ImageGen-Cort [38], and Got [15] leverage multimodal reasoning to perform prompt planning. generation, reflection, and refinement. Despite these advances, using chain-of- thought for unified understanding and generation remains underexplored. In this work, UniGen incorporates a Cos-b eelf-verification strategy via Best-of-N selection during test-time scaling, which leads to substantial improvements in image generation performance.

---

3 Recipe for Building UniGen
3.1 Architecture
As shown in Fig. 2, we unify the image un- derstanding and generation tasks into pre- trained LLM. Motivated by prior work [84], we separate visual encoding for understand- ing and generation into continuous and dis- crete embedding spaces, respectively.
TEXT
Gen Decoder *
LLM
Projector
Projector
For image understanding, we follow the LLaVA [40] workflow and adopt the next- token prediction paradigm. Given an in- put image X", the understanding encoder Enc" (e.g.. SigLIP [100J) extracts its fea- ture as a vector of continuous tokens XO Enc" (x"). The projector P aligns the image and text embeddings into the same space, then the embeddings are fed into LLM as inputs. We compute the understanding loss using the vanilla autoregressive training objective Lund. To preserve the LLM's language modeling capability, we also train UniGen with text-only data and backpropagate the corresponding loss Liezt-
TEXT
Encoder +
Und Encoder
+
Figure 2:
The architecture of UniGen, which is based on an autoregressive LLM and decoupled vision en- coders for image understanding and generation tasks.
For text-to-image generation, we employ the masked token prediction [5] as our training objectives. Unlike the autoregressive decoding for text tokens, this paradigm enables models generate multiple image tokens in parallel, significantly accelerating the generation process. During training, for each image X E #V the generation encoder Enc (e.g., MAGVIT-v2 [97]) tokenizes it into sequence of discrete tokens X2 of length N H/d,- W/d5, where d, refers to the spatial down- sampling factor of Enc Then, given a masking ratio 7j according to the scheduling function 7(-), we randomly sample a binary mask M(n)) = [mo.- m where * N positions are uniformly set to 1 and others are set to 0. For each position where my equals to 1, we replace its corresponding discrete image token N,C with special mask token [MASK] to form the final input image sequence. Finally, we prepend the textual tokens (e.g.. image classes or captions) with the masked sequence X During inference, the image generation starts with all masked tokens X [[ASK], (MASKK]) and gradually fills up the latent representation with scattered predictions in parallel.
3.2 Pre-Training (PT)
The goal of pre-training is develop UniGen's visual generation capability while preserving its potential for multimodal understanding. Thus, we only optimize the generation projector and the LLM with other parameters frozen. We also include image-to-text and text-only pre-training to keep UniGen's 's language modeling capability. To encourage a better alignment between discrete image tokens and the text, we directly use the generation encoder for understanding tasks but only in this stage. We empirically find that this design can significantly improve the image generation performance. Specifically, we employ an "easy-to-difficul difffcuul strategy through two-stage process.
Pre-training Data. We generate fine-grained captions for images from ImageNet [62], CC-3M [63], CC-12M [6] and SAM- [31] dataset using Qwen2.5-VI--7B [3] form 40M image-text pair corpus. For text-only pre-training, we use RefinedWeb [55].
PT-1 Stage seeks to align the image and text embeddings and predict the distribution of basic visual concepts. Similar to prior works [84], we employ ImageNet for generation pre-training warmup and leverage the full 40M image-text pairs for the understanding task. However, propose that using image captions, rather than image categories, for text-to-image generation leads to better convergence.
PT-2 Stage further facilitates UniGen generalize to wider visual generation capabilities. We expand the text-to-ine to-image training dataset to the full 40M image-text pairs, while using the same image-to-text and text-only ones. We argue that training data with a richer distribution enables more accurate control over generation patterns. We name the model trained in this stage as UniGen-PT.
3

---

Prompt
understanding To photo of dining toble abave suitcose"
+ UniGen Generation
CoT-V Steps
thane
CoT-V Prompts
Candidates
Fiiving ger
sultoose7
Yes
1.0
the diening
Und Encoder
lobk abows the sutrose?
Yes
UniGen
Understanding
there aatrooue?
Yes
0.3
the dhenteg
toli above
No
0.3
Uee austusure
Figure 3: The workflow of UniGen using test-time scaling and CoT-V. Left: Illustration of Best- of-N selection with CoT-V with N Right: Visualization of the step-by-step reasoning process in CoT-V for computing the final quality score.
3.3 Supervised Fine-Tuning (SFT)
the SFT stage, UniGen jointly trained on the image understanding and generation tasks. We fine- tune the generation projectors, understanding projectors, and the LLM, while still keeping the vision encoders frozen. For image understanding. we notice that the knowledge-centee is limited during pre-training stages. To enhance related capabilities, we adopt the strong image mixture from SlowFFst-LLaVA-LE [90]. which was carefully curated from open-source datasets with 4.67M multimodal VQA samples. For image generation, prior work [9] uses high-quality synthetic data that can enable fast and robust training convergence. We share this observation by using the JourneyDB [64] and text-2-image-21 [28] improve the aesthetic quality of our generated images. We name the model trained in this stage as UniGen-SFT.
3.4 Direct Preference Optimization (DPO)
We further enhance UniGen by aligning its outputs with human preference through DPO. We first discuss how construct our synthetic preference dataset, then describe DPO algorithm.
Preference Dataset. We leverage UniGen-SFT to generate images for our preference dataset. For given prompt, 20 images generated. A preferred and rejected sample pair is constructed by evaluating the coherence between each image and the prompt. To improve the data robustness, we collect 6k short prompts from PARM [20], 6k medium-length prompts from T21-Comp [25] training set, and 6k long prompts from re-annotated SA1B to generate training image candidates.
For short prompts, we use the GENEVAL metrics to evaluate the generation quality. For prompts of medium or long lengths, we decompose each prompt into fine-grained visual questions with Qwen2.5- 7B. Then, we assess image-prompt consistency by feeding Qwen2.5VL. pair. An output "yes" indicates the image aligns with the description, and "no" otherwise. The final consistency score S is averaged from these answers. For each prompt, we sample one highest-scored example as the preferred image and the lowest one as the rejected image. Prompts with no clear preference are filtered out. Finally, we obtain around 13k triplets for training.
with the image-question
DPO Training. We adopt the vanilla DPO training loss and freeze the understanding encoder and projector in this stage. The training ends in one epoch with batch size of 64 and learning rate of 5 We empirically find that this DPO training does not impair UniGen's understanding performance. We name the model trained in this stage as UniGen-DPO.

---

nor strankors there Yea
above tol No there <tthine
No
0.0
Yes
0.66
0.66
UniGen
UniGen
UniGen
Und Encoder
Und Encoder
Und Encoder
This image gamersted by à prompt puodo of on soak obove is Does this imag acourctdly ropresent the prompt? Pleas ansver you
La therwaa o
The image is generated by promot a phaco spple above o. Plaase isset image genecation quelity step by step, First breakdown the promot mo muliple....
apple?
Figure An example of using different image verification methods: (a) Outcome Verification, (b) Rule-based Verification and (c) Chain-of- Chain-of-Thought Verification.
3.5 Teme Scaling
Recent studies have shown the effectiveness of test-time scaling on improving both image understand- ing [79, 106] and generation [20, 86). We employ the Best-of-N evaluation strategy and leverage UniGen's understanding ability to conduct self-critique for image generation verification. The general workflow is illustrated in Fig. First, UniGen generates N candidate images for a given prompt. Second, we input each generated image along with its prompt into UniGen, which evaluates the alignment between the image and its textual description and outputs a quality score S. Third, return the image with the highest score. We propose three verification methods as shown in Fig. 4.
Outcome Verification (OV) simply prompts UniGen to directly judge the coherence of the input prompt and each image candidate, giving a binary score (i.e. "yes" for good match and "no" for a failure generation). We randomly select one if there are candidates with the same score.
Rule-based Verification (RV) breaks down each prompt into several atomic questions based on pre-defined rules, then sequentially feeds them with the generated image into UniGen for quality verification. The results of all sub-questions
averaged as the final quality score.
Shain-of-Though Verification (CoT-V) instructs the model to think step-by-step and verifies each atomic fact according the prompt and each generated image, following the CoT format: sthink_start<pe ? A1; ... Qa? An <think_end> We compute the final quality score S by parsing the CoT outputs. Specifically, given a text prompt and generated image I, CoT- V produces a list of visual questions Q {Q1.-.. ... Qn and their corresponding answers A [A1..... }. The final score S is computed by averaging the scores in the answer list.
OV relies on UniGen's 's pattern matching capabilities without intermediate reasoning. RV incorporates rule-driven reasoming process into test-time scaling. Although effective on well-structured prompts, RV struggles with free-form complex instructions, such as those in DPG-Bench [23]. CoT-V leverages the strengths both approaches, enabling reasoning-driven image verification without the need for manual prompt decomposition. Thus, we use V our default verification method.
3.5.1 CoT-V Post-Tree Training
UniGen has not been precisely trained to generate CoT responses. Here we introduce a lightweight post-training strategy upon UniGen-DPO, equipping it with the ability of CoT-bbsee verification.
Data. To construct the CoT-V post-training data, we reuse the image-text pairs collected during the DPO stage (See. 3.4). For prompts sourced from PARM, extract the question-amm ansser pairs via rule-based matching, since they are built upon a clear structure [17]. For prompts from T2I-Comp that are more complicated, then query Qwen2.5-76 5-------- 7--VB VL with each image-question pair to obtain their binary pseudo labels. We exclude the prompts from SA- 1B due to the lower quality of the decomposed visual questions. We empirically find that most of the decomposed questions do not fully cover the visual concepts of the original caption. We totally sample 20K image-question-anse
first guide Qwen2.5-7D [91] to generate series of atomic questions,
triplets from both prompt sources.
Training. We format the above 20K training pairs as fostrwction-folloviing conversations, and feed them into UniGen-DPO for supervised fine-tuning, In this stage, we only optimize the understanding
5

---

Table 1:
Comparison with state-of-the-art models image understanding benchmarks. denotes reproduced results and RW-QA denotes RealWorld-QA.
Model
Paramm
Res.
AI2D
GQA
POPE
MMMU Math RW-QA ScienceQA
Seedbench
Understanding MLLMs
LLaVA-OV [32]
0.58
AnyRes AnyRes
57.1
31,4
34.8
55.6
67.2
65.5
MML.5 [102]
IB
59.3
88.1
35.8
37.2
53,3
82.1
70.2
LLaVA [39]
7B
336
55.1*
62.0
86.1
36.3*
26.7*
55.8*
66.8
66.1
Unified MLLMs
Show-o [87]
1.3B
336
36.2*
61.0
84.5
27.4
22. 1
48.5
42,7
61.5*
Janus [84]
1.38
49.0*
87.0
30.5
33.7*
48.4*
76.5*
63.7
Janus-Pro [9]
1.5B
384
63.7*
59.3
86.2
36.3
36.8
51.1
75.5
68.3
Vila-U [85]
7B
384
60.8
85.8
56.3
MMAR [93]
7B
256
83.0
64.5
UniToken [29]
7B
384
68.7
32.8
38.5
69.9
UniGen
1,5B
384
67.4
62.3
87.8
32.3
44.6
56.7
79.4
70.8
projector and the LLM. To ensure not impairing UniGen's general understanding capabilities, we fine-tune UniGen on this CoT-V dataset for only 500 steps using small learning rate of 1 10-5. The model trained after this stage is our final model, and we name it as UniGen.
4 Experiments
4.1 Implementation Details
We use 32 H100-80G GPUs for pre- pre-trrinnng stages and 8 H100-80G GPUs for the others. UniGen is built upon the pre-trained Qwen2.5-1.5B [91]. We adopt MAGVITv2 from Show-o [87] as our discrete visual encoder with input resolution of 256 256 and SigLIP [100] as our continuous visual encoder. As discussed in Sec 3.1, we use MAGVITv2 for both understanding and generation in PT-1 and PT-2, and keep using SigI.I as the understanding encoder after SFT.
Training. We follow Show-o [87] to use a bidirectional attention mask within image tokens, but keep the causality within text tokens and between multimodal tokens. Detailed hyperparameters for each training stage are described in Appendix Table 17 with more details in Appendix Sec. E.0.2.
Inference and Evaluation. We follow the common practice of image generation guidance [22] and set the scale to 5.0. In addition, we follow MaskGIT [5] to adopt the cosine masking scheduler in inference and set the default number of steps T 50. We MAGVITv2 decoder to project the visual tokens back to the pixel space. For test-time scaling with CoT-V, we generate N 20 image candidates per text prompt and select top-K K (K out of them, sending for evaluation on GENEVAL and DPG-BENCH.
classifier-free
4.2 Main Results
We report the performance of UniGen on various benchmarks (details are discussed in Appendix Sec. A) and show qualitative results in Fig 5. We mainly compare UniGen with state-of-the-an unified LLMs in Table and 2, but also reference strong specialist models to understand our position in the whole picture of MLLMs. Here we highlight the following observation.
First, UniGen achieves state-of-the-art results across understanding benchmarks compared to existing unified MLLMs. Specifically, UniGen outperforms Janus-Pro on Real World-r.. A12D and Math Vista by +5.6%, +3.7%, and +7.8%, respectively. We believe our improvements are mainly driven by using (i) the decoupled generation and understanding encoders and (ii) the stronger SFT data mixture. Notably, UniGen is even comparable with some strong understanding-onl MLLMs, such LLaVA-OV-0.5B and MM1.5-IB, even though they use much higher input resolutions.
Second, UniGen significantly outperforms existing unified MLLMS and strong generation- only models on text-fo-image benchmarks. Using GENEVAL in Table 2 as an example, UniGen achieves the overall score of 0.78, significantly outperforming Janus-Pre Pro by 0.05. Besides, our model demonstrates an overwhelming advantage on the "Counting" task by +0.27 higher than Janus-Pro. UniGen even beats a range of superior generation-only models (e.g., outperforming DALLE-2, and Emu3 by +0.26 and +0.24, respectively). even though they are with much larger model sizes. Similarly, UniGen outperforms existing models by a clear margin on DPG-BENCH as shown in Table 2, outperforming Show-o and Janus-Pro by +13. 49 and +2.56, respectively.

---

Table 2: Comparison with state-of-the-art models GENEVAL and DPG-BENCH benchmark.
GenEval
DPG-Bench?
Model
Params
Two Ohj
Counting
Position
Color Attri.
Overall
Global
Relation
Overall
Text-to-Image Generation Models
[61]
DALLE-3(4]
0.87
0.47
0.43
0.45
0.67
90.97
90.58
83,50
Emu3 [80]
0.71
0.34
0.17
0.54
85.21
90.22
80.60
SDXL [57]
2.6B
0.74
0.39
0.15
0.23
0.55
83.27
86.76
74.65
SimpleAR [76]
1.5B
0.90
0.28
0.45
0.63
87.97
88.33
81.97
Infinity [21]
2B
0.85
0.49
0.57
0.73
93.11
90.76
83.46
Unified MLLMs
Show-o [87]
1.3B
0.52
0.49
0.11
0.28
0.53
80.39*
83.36"
71.70*
Janus 184)
1.3B
0.68
0.30
0.46
0.42
0.61
82.33
85.46
79.68
Janus: Pro [9]
1.5B
0.82
0.51
0.65
0.56
0.73
87.58
88.98
82.63
ILLUME [74]
7B
0.86
0.45
0.39
0.28
0.61
UniToken [29]
0.80
0.35
0.38
0.63
VARGPT-eL.. [107]
9B
0.53
0.48
0.13
0.21
0.53
84.83
88.13
78.59
TokenFlow [58]
UniGen
1.5B
0.92
0.68
0.48
0.52
0.74
91.53
91.09
84.89
UniGen CoT-V
1.5B
0.94
0.78
0.57
91.95
92.04
Table 3: Ablation of different stages of our model on image understanding benchmarks.
Model
Stage
GenEval
DPG-Bench
A12D
GQA
POPE
MMMU
MathVVsta
RW-QA Science
Seedbeuch
PT-1
0.53
78.14
UniGen
79.7
84.89
67.9
88.0
45.0
79.5
71.0
CoT-V
85.19
79.4
4.3
Ablation Studies
4.3.1
Impact of Different Training Stages
We examine training pipeline by showing the understanding and generation performance after each stage in Table 3. Here we highlight some key observations.
First, UniGen demonstrates consistent improvements in generation performance across dif- ferent training stages, as indicated by the increasing numbers of GENEVAL and DPG-BENCIL The pre-training stages aim to warm up the generation capability of UniGen. The SFT boosts the GENEVAL and DPG- BENCIENCI by using high-quality generation datasets. With the effectiveness of preference data, the DPO stage significantly improves GENEVAL and DPG-BENCH to 0.73 (+0.10) and 84.89 (+2.14), respectively. CoT-V further enhances the scores to 0.78 (+0.05) and 85.19 (+0.3) via test-time scaling.
Second, UniGen's 's strong understanding capability stimulated in the SFT stage and can be maintained in the following stages. The SFT stage promotes the instruction following capability of UniGen that leads to strong performance on understanding benchmarks. In the DPO stage, UniGen successfully maintains the strong understanding capability. CoT-V contains an additional lightweight fine-tuning to encourage the CoT verification during test-time scaling. The results show that it does not sacrifice the general understanding capability, except for slight regression on Real ReallWrld-OL We attribute this regression to the distribution gap between CoT-V's V's synthetic training data and the real-world images in RealWorld- QAQ.
4.3.2 Ablation of CoT-V
Here we evaluate different verification methods discussed in Sec. 3.5 with the following highlights.
First, CoT verification achieves the best performance and prompting UniGen's thinking process important. As shown in Table 4, using Outcome verification shows no improvement, while using CoT thinking obtains a significant boost of generation performance on both GENEVAL and DPG-BENCH. We also observe that Rule-based verification also effective, leading to 0.75 on GENEVAL However, it is not general enough to be used on free-form prompts. Comparing the results from Co7 Verification and Rule- based Verification, we can see that prompting the model itself to think is beneficial for more reliable critique.
7

---

a photo of four books
photo of three
photo dining
photo right
photo of an orange skate board and pink
photo kuiffe
old red track, its body covered in of rust abandoned open field. The tuck's white stand contrast running across glass.
Beneath the expansive flag adorning the pinnacle of towering lighthouse drifts gently in ovening broeze, its haes subdued by the soft silver glow the moon... Al the water's odgs. rough texture of the rocks internittenlly made
sprinkled with myriad twinkling
shatlered, spickrweb empty, and are worn, hinting many years service and neglect.
Figure 5: Visual examples of UniGen's 's results using CoT-V. The first three rows show examples for counting, position, and color attribute, respectively, and the last row shows images generated by free-form prompts. The first column contains images selected by UniGen as the test-time verifier.
Table 4: Ablation of verification methods.
Table 5: Ablation of CoT-V post-training.
Method
Outcome
Rule
CoT
GenEval
DPG-Bench
Method
CoT-V Post-train
GENEVAL DPG-BENCH
x
X
X
0.74
85.02
X
0.64
76.32
Show-o
V
X
X
0.74
85.00
0.66
77.09
UniGen
X
0.74
84,89
x
X
V
0.78
85.19
UniGen
0.78
Second, CoT-V post-training is essential for strong test-time verification. As shown in Table 5, directly using UniGen without CoT-V post-train leads to notable performance drop, especially for GENEVAL. This comparison demonstrates that CoT-V post-train is pivotal for CoT verification.
Third, CoT-V can effectively generalize to other models. We finetune Show-o with DPO and CoT-V with our generated data to boost its generation performance. Results in Table 5 show that CoT-V is general technique that can also enhance Show-o's generation performance.
4.3.3 Ablation of DPO
We ablate the contribution of each data source and demonstrate the effectiveness of our DPO data on other unified models.
Table Ablation study of DPO. The results are from UniGen-DPe without test-time scaling.
Method
PARM
T21-Comp
SAIB
GenEval
DPG-tench
First, every prompt source contributes pos- itively to generation performance. Table shows that adding only PARM DPO data results in remarkable improvements (rowl vs. row2), while further adding T2I-Comp mainly benefits OPG-BENCE (row2 vs. row3). UniGen-DPO with all of three prompts, introduces the best overall performance (row3 vs. row4).
x
x
0.63
82.75
x
0.73
83.48
x
0.72
84.09
0.74
84,89
0.56
71.70
Show-o
/
v
/
0.64
76.32
Second, our DPO data also largely improves Show-o, showing that it is generalizable to other unified models. When fine-tuning Show-o directly with our DPO data, we also observe a notable gain, from 0.56 to 0.64 GENEVAL and from 71.70 to 76.32 DPG-BENCH shown in Table 6.

---

4.4 Ablation of SFT
Table 7: Ablation of SFT stage. PT-2 Data denotes the training data used in the PT-2 Stage. JD and TI denote JourneyDB and text- 2ert-2-image-M-- respectively. The results are from UniGen-SFT.
Und Date
Gen Data
GenEval
DPG-Beach
AI2D
GOA
POPE
MMMU
MathVista
RW-QA
ScienceAA
Seedhench
Und Ang.
FE22 Data
0.56
79.67
68.3
62.4
87.5
33.3
42.2
54.4
79.6
76.7
62.3
JD=I1
0.63
K2.77
08.0
02.5
87.4
32.4
45.2
58.0
79.7
71.1
63.1
ILAVAI5
ID-TI
0.4
51.82
467
628
87.4
27.1
22.1
53.7
55.5
64.0
52.7
By default, use the image mixture from SlowFast-LLaVA-155 [90] as understanding datasets and JourneyDB and text-2-image-2M as the generation datasets. In this section, we ablate the datasets in Table 7 to evaluate their impacts and draw the following conclusion.
First, using high-quality generation data is necessary for further lifting generation results. JourneyDB and text-2-image-2MM have much higher quality compared the generation data used during the PT-2 stage. Table (row1 1 vs. row2) shows that using high-quality generation data in the SFT stage results in better image generation performance.
Second, using a stronger data mixture crucial to improve the understanding performance, which is also helpful for fine-grained text-to-image generation. As shown in Table 7 (row2 vs. row3), replacing FoowFas--LLLV--.. mixture with LLaVA115 1.5's induces much worse understanding performance. Also, training with Faaa-ast----------- data produces higher results DPG-BENCE BENCH. We believe better understanding capability is important for comprehending the complex text prompts of DPG-BENCE that can eventually be beneficial for better text-to-image toagg generation.
4.5
A Alltioon of PT-1 and PT-2
Table Impact of using understanding task in PT stages. The results are from UniGen-SFF
Und Data
PT-1
Und Data
GenEvaal
DPG-hench
AI2D
GQA
POPE
MathVistu
RW-QA
ScienceQA
Seedbench Avg.
x
0.61
82.51
60.5
59.6
87.4
30.9
38.1
49.0
72.0
66.1
$8.0
0.63
82.75
68.0
62.5
87.4
32.4
45.2
58.6
71.1
63.
Table Ablation of PT-1 stage. Cls and Recap indicate class names and high-quality captions are used for generating images, respectively. The results are from UniGen-SF7:
Stage 1
Gen Data
GenEval DPG-bench
AI2D
GQA
POPE
MMMU
MathVista RW-QA
ScienceQA
Seeclbench
Und Avg.
0.64
82.26
70.3
62.5
87.9
33.7
45.7
54.2
80.5
21.7
63.3
ImageNatCls) (mageNot(Roccap)
88.0
31.4
53.6
61.8
82.75
68.0
62.5
87.4
32.4
45.2
58.6
79.7
71.1
63.1
We explore the necessity and key factors of UniGen's 's pre-training stages. We first discuss whether we should include the understanding dataset in the pre-training stages as shown in Table 8. Second, we ablate the impact of the generation datasets to both generation and understanding performance in Table 9 (for PTT-)) and Table 10 (for PT-2). Since PT-1 and PT-2 are early stages in UniGen's 's training pipeline, continue the training the SFT stage to verify their impact the final performance more reliably. Unless noted otherwise, all ablations this section use UniGen's 's default SFT settings. Here highlight the following observations.
First, including understanding data in pre-training stages is crucial for both generation and understanding performance. In Table 8's row 1, we keep the default setting and only remove the understanding loss from the training objectives. We observe a significant performance decrease across generation and understanding benchmarks at the SFT stage. We attribute this to the fact that understanding data is important for better vision-language alignment early training stages, which is helpful for both image-to-text and text-to-image tasks.
Second, the high-quality text-to-image task is more effective than the de facto class-to-image task in PT-1. One common practice of unified MLLMs for pre-training is using the class-to- to-ima task with ImageNet [84, 87]. However, we find that using ImageNet with fine-grained captions leads to better performance for understanding tasks in the UniGen-SFT stage as shown in Table (row2 vs. row3). This result of better languae alignment introduced by the detailed caption-to- tortinn-o--maae mapping.

---

Table 10: Ablation of PT-2 stage.
The reported results are from UniGen-SFF
Stage II
Gen & Und Data
GGenEval DPGG-bench
AI2D
GQA
POPE
MMMU
MathiVisa
RW-QA
ScienceQA
Seedbench
Und Avg.
0.58
79.25
70.2
61.9
87.3
31.6
47.0
54.9
82.5
71.2
63.3
CC+SA-IMN
80.64
40.9
RResp)
0.64
82.63
6722
62.4
87.5
31.0
40.8
56.5
79.7
71.1
62.0
(CC-IMNKReca))
0.63
82.75
67.9
62.1
87.8
31.6
42.0
58.2
80.3
70,8
62.6
(CC+SA/RRccap
0.62
82.34
68.6
02.2
87.4
30.6
45.4
59.3
80.5
70.8
63.1
(CC+SA+IMNNRRccap)
0.63
82.75
68.0
62.5
87.4
32.4
45.2
58.6
79.7
71.1
63.1
Third, to maintain high performance on generation, we need both PT-1 and PT-2. According to Table 9 (rowl vs. row3) and Table 10 (row1 vs. row6), we notice that completely removing PT-1 or PT-2 stage will largely decrease the generation metries. Especially, eliminating PT-2 has much bigger negative impact, leading to a dramatic drop of numbers on both GENEVAL and DPG-BENCH. Excluding PT-1 will not apparently affect GENEVAL, but hurts DPG-BENCH. This is because the prompts of DPG-BENCH BENCH are more complicated, thus more pre-training helps our model to better comprehend their semantics.
Fourth, to keep a strong understanding performance, we need at least one of the PT-1 and PT-2. According to Table 8, we infer that the understanding performance will be destroyed if we remove both PT-1 and PT-2. However, discarding PT-1 or PT-2 in Table and Table 10 does not impact understanding numbers. As a result, recommend keeping at least one of them for good understanding capability and leveraging both of them for the best generation and understanding performance if the compute budget allows.
Fifth, using high-quality captions in PT-2 is important for understanding and generation performance. Table 10 (row2 vs. row6) demonstrates that using high-quality image captions results in stronger performance in both understanding and generation tasks. This is due to the better text-to-image and image-to-text alignment learned from the fine-grained captions.
Sixth, each data source of PT-2 has meaningful contributions. We remove each data component from the training set of PT-2 and observe that retaining all of them leads to the best performance as shown in Table 10 (row3 to row6). This finding supports the usefulness of each dataset we curated.
5 Conclusion
We present UniGen, MLLM for unified multimodal understanding and generation. We discuss the key factors along the entire training pipeline and propose optimization methods to improve the performance. We also make the first attempt to collaborate UniGen's 's understanding and generation capabilities, by enabling UniGen to perform as both image generator and verifier during test-time scaling. As a result, successfully further boost the image generation quality by a clear margin. Trained with only open-source datasets, UniGen achieves the state-of-thed of--th--ttttttt performance across extensive understanding and generation benchmarks. We hope our exploration and ablation studies provide insights into the future development of strong unified MLLMs.
Limitation.
First, we instantiate UniGen with only 1.5B model, since larger scales will impose much higher demands on the computational cost. However, larger models have been shown effective for improving both understanding and generation performance [9]. Second, our generation capability targets at promoting semantic alignment between the input text prompt and the generated image, therefore we only focus on resolution of 256 x 256. We plan support higher resolution image generation, such as 480p or even 1080p, which is valuable for improving the visual fidelity. Third, although achieving convincing results on DPG-Bench, CoT-V is still limited for complicated text prompts, due to the noisy CoT data generated by Qwen2.5VL. as a pseudo labeler. This could be largely relieved by using a stronger pseudo labeler or leveraging human filtering in the future. Equipping UniGen with stronger reasoning and CoT capabilities in an earlier stage is also a promising direction.
Broader Impact. Unified MLLMS offer scientific benefits by enabling human-A AI interaction and advancing general-purpose multimodal understanding. There are many real-world applications, such as design assistants, education, and collaborative robots. However, there could be unintended usages and we advocate responsible usage complying with applicable laws and regulations.
Acknowledgment. We thank Haiming Gang, Jesse Allardice, Shiyu Li, and Yifan Jiang for their kind help.
10

---

References
[1] Marah Abdin, Jyoti Aneja, Hany Awadalla, Ahmed Awadallah, Ammar Ahmad Awan, Nguyen Bach, Amit Bahree, Arash Bakhtiari, Jianmin Bao, Harkirat Behl, et al. Phi-3 technical report: A highly capable language model locally on your phone. arXiv:2404. 14219, 2024.
[2] Jean- Bann-Battstte Alayrac, Jeff Donahue, Pauline Luc, Antoine Miech, Iain Barr, Yana Hasson, Karel Lene, Arthur Mensch, Katherine Millican, Malcolm Reynolds, al. Flamingo: visual language model for few-shot learning. In NeurIPS, 2022. 2
[3] Shuai Bai, Keqin Chen, Xuejing I. Liu, Jialin Wang, Wenbin Ge, Sibo Song, Kai Dang, Peng Wang, Shijie Wang, Jun Tang, et Qwen2. 5-vl technical report. arXiv:2502. 13923, 2025.
[4] James Betker, Gabriel Goh, Li Jing, Tim Brooks, Jianfeng Wang, Linjie Li, Long Ouyang. Juntang Zhuang, Joyce Lee, Yufei Guo, et al. Improving image generation with better captions. Computer Science., 2023. 7, 26
[5] Huiwen Chang, Han Zhang, Lu Jiang, Ce Liu, and William T Freeman. Maskgit: Masked generative image transformer. In CVPR, 2022. 3, 6
[6] Soravit Changpinyo, Piyush Sharma, Nan Ding, and Radu Soricut. Conceptual 12m: Pushing web-scale image-text pre-training recognize long-tail visual concepts. In CVPR, 2021.
[7) Jiefeng Chen, Jie Ren, Xinyun Chen, Chengrun Yang, Ruoxi Sun, and Sercan Ö Anı. Sets: Leveraging self-verification and self-correction for improved test-time scaling. arXiv:2501. 19306, 2025. 2
[8] Jiuhai Chen. Zhiyang Xu, Xichen Pan, Yushi Hu, Can Qin, Tom Goldstein, Lifu Huang, Tianyi Zhou, Saining Xie, Silvio Savarese, et al. Blip3-o: A family of fully open unified multimodal models-architecture, training and dataset. arXiv preprint arXiv:2505.0940 099668 2025. 2
[9] Xiaokang Chen, Zhiyu Wu, Xingchao Liu, Zizheng Pan, Wen Liu, Zhenda Xie, Xingkai Yu, and Chong Ruan. Janus-pro: Unified multimodal understanding and generation with data and model scaling. arXiv:2501.17811, 2025. 1, 1, 4, 6, 7, 10, 25, 26
[10] DeepSeek-AI. Deepseek-v3 technical report. arXiv:2412. 19433 2024.
19437,
[11] Matt Deitke, Christopher Clark, Sangho Lee, Rohun Tripathi, Yue Yang, Jae Sung Park, Mohammadreza Salehi, Niklas Muennighoff, Kyle Lo. Luca Soldaini, et al. Molmo and pixmo: Open weights and open data for tfate-of-the-ar multimodal models. arXiv:2409.17166, 2024.
[12] Chaorui Deng, Deyao Zhu, Kunchang Li, Chenhui Gou, Feng Li, Zeyu Wang, Shu Zhong, Weihao Yu, Xiaonan Nie, Ziang Song, et al. Emerging properties in unified multimodal pretraining. arXiv preprint arXiv22505.14685 2025.
[13] Yuhao Dong, Zuyan Liu, Hai-Long Sun, Jingkang Yang, Winston Hu, Yongming Rao, and Ziwei Liu. Insight-v: Exploring long-chain visual reasoning with multimodal large language models. arXiv:2411. 22241...1443.. 2024. 2
[14] Patrick Esser, Robin Rombach, and Bjorn Ommer. Taming transformers for high-resolution image synthesis. In CVPR, 2021.
[15] Rongyao Fang. Chengqi Duan, Kun Wang, Linjiang Huang, Hao Li, Shilin Yan, Hao Tian, Xingyu Zeng, Rui Zhao, Jifeng Dai, et al. Got: Unleashing reasoning capability of multimodal large language model for visual generation and editing. arXiv:2503.1065 2025.
[16] Yuying Ge. Sijie Zhao, Jinguo Zhu, Yixiao Ge. Kun Yi, Lin Song, Chen Li, Xiaohan Ding, and Ying Shan. Seed-x: Multimodal models with unified multi-granularity comprehension and generation. arXiv:2404. SKtV:2240.143960 14396, 2024.
[17] Dhruba Ghosh, Hannaneh Hajishirzi, and Ludwig Schmidt. Geneval: An focuct-focused frame- work for evaluating text-to-image alignment. Advances in Neural Information Processing Systems, 2023, 5, 25

---

[18] Grok- 1.5, https::/w.ai/neews/rrokk/ 2024. 25
[19] Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, et al. Deepseek-rl: Incentivizing reasoning capability in Ilms via reinforcement learning. arXiv:2501. /2948, 2025. 2
[20] Ziyu Guo, Renrui Zhang, Chengzhuo Tong, Zhizheng Zhao, Peng Gao, Hongsheng Li, and Pheng-Ann Heng. Can generate images with cot? let's verify and reinforce image generation step by step. arXiv:2501. 13926, 2025. 2, 4,5
[21] Jian Han, Jinlai Liu, Yi Jiang, Bin Yan, Yuqi Zhang, Zehuan Yuan, Bingyue Peng, and Xiaobing Liu. Infinity: Scaling bitwise autoregressive modeling for high-resolution image synthesis. arXiv:2412.04431, 2024. 7.26
[22] Jonathan Ho and Tim Salimans. Classifier-free diffusion guidance, arXiv:2207112595 2022.
[23] Xiwei Hu, Rui Wang, Yixiao Fang, Bin Fu, Pei Cheng, and Gang Yu. Ella: Equip diffusion models with Ilm for enhanced semantic alignment. arXiv:2403.05115, 2024. 2,
[24] Chengsong Huang, Langlin Huang, Jixuan Leng, Jiacheng Liu, and Jiaxin Huang. Efficient test-time scaling via self-calibration. arXiv:2503 00031, 2025.
[25] Kaiyi Huang, Kaiyue Sun, Enze Xie, Zhenguo Li, and Xihui Liu. T2i-compbench: A comprehensive benchmark for open-world compositional lext-to-image generation. In NeurIPS, 2023.
[26] Drew A Hudson and Christopher D Manning. Gqa: A new dataset for real-world visual reasoning and compositional question answering. In CVPR, 2019. 25
[27] Hamish Ivison, Yizhong Wang, Jiacheng Liu, Zeqiu Wu, Valentina Pyatkin, Nathan Lambert, Noah A Smith, Yejin Choi, and Hanna Hajishirzi. Unpacking dpo and ppo: Disentangling best practices for learning from preference feedback. In NeurlPS, 2024. 2
[28] jackyhate. text-to-imaae-2m:: high-quality, diverse text-to-image training dataset. https:
/ //wggggngface...ee// cot.aaaasss///ttteet////.sssss/s jackyaate/teet teatttto-imngee 2M, 2024. 4
[29] Yang Jiao, Haibo Qiu, Zequn Jie, Shaoxiang Chen, Jingjing Chen, Lin Ma, and Yu-Gang Jiang. Unitoken: Harmonizing multimodal understanding and generation through unified visual encoding. arXiv:2504. 04423, 2025. 6, 26
[30] Aniruddha Kembhavi, Mike Salvato, Eric Kolve, Minjoon Sco, Hannanch Hajishirzi, and Ali Farhadi. A diagram is worth dozen images. In ECCV, 2016. 25
[31] Alexander Kirillov, Eric Mintun, Nikhila Ravi, Hanzi Mao, Chloe Rolland, Laura Gustafson, Tete Xiao. Spencer Whitehead, Alexander C Berg, Wan-Yen Lo, et al. Segment anything. In ICCV, 2023. 3
[32] Bo Li, Yuanhan Zhang, Dong Guo, Renrui Zhang, Feng Li, Hao Zhang, Kaichen Zhang, Peiyuan Zhang, Yanwei Li, Ziwei Liu, et al. Llava-onevision: Easy visual task transfer. arXiv:2408.00 03336, 2024. 2, 6
[33] Bohao Li, Rui Wang, Guangzhi Wang, Yuying Gc, Yixiao Ge, and Ying Shan. Seed-bench: Benchmarking multimodal Ilms with generative comprehension. arXiv:2307.16125, 2023, 25
[34] Junnan Li, Dongxu Li, Silvio Savarese, and Steven Hoi. Blip-2: Bootstrapping language-image pre-training with frozen image encoders and large language models. In ICMI., 2023. 2
[35] Yanghao Li, Rui Qian, Bowen Pan, Haotian Zhang, Haoshuo Huang, Bowen Zhang, Jialing Tong, Haoxuan You, Xianzhi Du, Zhe Gan, et al. Manzano: simple and scalable unified multimodal model with hybrid vision tokenizer. arXiv preprint 25xiv:250. 2025. 2
[36] Yifan Li, Yifan Du, Kun Zhou, Jinpeng Wang, Wayne Xin Zhao, and Ji-Rong Wen. Evaluating object hallucination in large vision-language models. In EMNLP, 2023. 25
12

---

[37] Zhimin Li, Jianwei Zhang, Qin Lin, Jiangfeng Xiong, Yanxin Long, Xinchi Deng, Yingfang Zhang. Xingchao Liu, Minbin Huang, Zedong Xiao, et Hunyuan-dit: A powerful multi- resolution diffusion transformer with fine-grained chinese understanding. arXiv:2405.087453 2024. 26
[38] Jiaqi Liao, Zhengyuan Yang, Linjie Li, Dianqi Li, Kevin Lin, Yu Cheng, and Lijuan Wang. Imagegen-cot: Enhancing text-to-image in-context learning with chain-of- of-thought reasoning. 1rXiv:2503,19312, 2025.
[39] Haotian Liu, Chunyuan Li, Yuheng Li, and Yong Jae Lee. Improved baselines with visual instruction tuning. In CVPR, 2024. 6
[40] Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. Visual instruction tuning. 2023.
2, 3
[41] Zhijian Liu, Ligeng Zhu, Baifeng Shi, Zhuoyang Zhang, Yuming Lou, Shang Yang, Haocheng Xi, Shiyi Cao, Yuxian Gu, Dacheng Li, et al. Nvila: Efficient frontier visual language models. wxXiv:242.004468 2024. 2
[42] Zuyan Liu, Yuhao Dong, Ziwei Liu, Winston Hu, Jiwen Lu, and Yongming Rao. Oryx MILM: On-demand spatial-temporal understanding at arbitrary resolution. ICLR, 2025.
[43] Jiasen Lu, Christopher Clark, Sangho Lee, Zichen Zhang, Savya Khosla, Ryan Marten, Derek Hoiem, and Aniruddha Kembhavi. Unified-io 2: Scaling autoregressive multimodal models with vision, language, audio, and action. arXiv:2312.17172, 2023. 2
[44] Jiasen Lu, Christopher Clark, Rowan Zellers, Roozbeh Mottaghi, and Aniruddha Kembhavi. Unified-io: unified model for vision, language, and multi-modal tasks. arXiv:2206. 08916, 2022. 2
[45] Jiasen Lu, Liangchen Song, Mingze Xu, Byeongjoo Ahn, Yanjun Wang, Chen Chen, Af- shin Dehghan, and Yinfei Yang. Atoken: A unified tokenizer for vision. arXiv preprint arXiv:2509. 14476, 2025.
[46] Pan Lu, Hritik Bansal, Tony Xia, Jiacheng Liu, Chunyuan Li, Hannaneh Hajishirzi, Hao Cheng, Kai-Wei Chang, Michel Galley, and Jianfeng Gao. Mathvista: Evaluating mathematical reasoning of foundation models in visual contexts. ICLR, 2024. 25
[47] Yiyang Ma, Xingchao Liu, Xiaokang Chen, Wen Liu, Chengyue Wu, Zhiyu Wu, Zizheng Pan, Zhenda Xie, Haowei Zhang, Liang Zhao, et al. Janusflow: Harmonizing autoregression and rectified flow for unified multimodal understanding and generation. arXiv:2411.00975, 2024.
2
[48] Brandon McKinzie, Zhe Gan, Jean- -han-philippe Fauconnier, Sam Dodge, Bowen Zhang, Philipp Dufter, Dhruti Shah, Xianzhi Du, Futang Peng, Anton Belyi, et al. Mml: methods, analysis and insights from multimodal IIm pre-training. In ECCV, 2024. 2
[49] Lingchen Meng, Jianwei Yang, Rui Tian, Xiyang Dai, Zuxuan Wu, Jianfeng Gao, and Yu- Yu-Gang Jiang. Deepstack: Deeply stacking visual tokens is surprisingly simple and effective for Imms. In NeurIPS, 2024.
[50] Fabian Mentzer, David Minnen, Eirikur Agustsson, and Michael Tschannen. Finite scalar quantization: Vq-vae made simple. arXiv:2200. 15505, 2023. 2
[51] David Mizrahi, Roman Bachmann, Oguzhan Fatih Kar, Teresa Yeo, Mingfei Gao, Afshin Dehghan, and Amir Zamir. 4M: Massively multimodal masked modeling. In NeurlPS, 2023.
[52] OpenAI. Openai-ol. 2
[53] OpenAL. Gpt-4o, 2024.
[54] Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. Training language models to follow instructions with human feedback. In NeurIPS, 2022. 2
13

---

[55] Guilherme Penedo, Quentin Malartic, Daniel Hesslow, Ruxandra Cojocaru, Alessandro Cap- pelli, Hamza Alobeidli, Baptiste Pannier, Ebtesam Almazrouei, and Julien Launay. The refinedweb dataset for falcon IIm: outperforming curated corpora with web data, and web data only. arXiv:2306.00 16, 2023. 3
[56] Wujian Peng, Lingchen Meng, Yitong Chen, Yiweng Xie, Yang Liu, Tao Gui, Hang Xu, Xipeng Qiu, Zuxuan Wu, and Yu-Gang Jiang. Inst-it: Boosting multimodal instance understanding via explicit visual prompt instruction tuning. arXiv preprint arXiv:2412.03555, 2024.
[57] Dustin Podell, Zion English, Kyle Lacey, Andreas Blattmann, Tim Dockhorn, Jonas Müller, Joe Penna, and Robin Rombach. Sdxl: Improving latent diffusion models for high-resolution image synthesis. In ICLR, 2024. 7,26
[58] Liao Qu, Huichao Zhang, Yiheng Liu, Xu Wang, Yi Jiang, Yiming Gao, Hu Ye, Daniel K Du, Zehuan Yuan, and Xinglong Wu. Tokenflow: Unified image tokenizer for multimodal understanding and generation. arXiv:241200066, 2024. 2, 7, 26
[59] Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, al. Learning transferable visual models from natural language supervision. In ICML, 2021. 2
[60] Rafael Rafailov, Archit Sharma, Eric Mitchell, Christopher D Manning, Stefano Ermon, and Chelsea Finn. Direct preference optimization: Your language model secretly reward model. In NeurlPS, 2023. 2
[61] Aditya Ramesh, Prafulla Dhariwal, Alex Nichol, Casey Chu, and Mark Chen. Hierarchical text-conditional image generation with clip latents. arXiv:2204.06125, 2022. 7, 26
[62] Tal Ridnik, Emanuel Ben-Baruch, Asaf Noy, and Lihi Zelnik-Manor. Imagenet- pretraining for the masses. 2XX1:22004.19972, 2021. 3
[63] Piyush Sharma, Nan Ding, Sebastian Goodman, and Radu Soricut. Conceptual captions: A cleaned, hypernymed, image alt-text dataset for automatic image captioning. In ACL, 2018.
[64] Keqiang Sun, Junting Pan, Yuying Ge, Hao Li, Haodong Duan, Xiaoshi Wu, Renrui Zhang, Aojun Zhou, Zipeng Qin, Yi Wang, et al. Journeydb: A benchmark for generative image understanding. In NeurlPS, 2023. 4
[65] Peize Sun, Yi Jiang, Shoufa Chen, Shilong Zhang, Bingyue Peng, Ping Luo, and Zehuan Yuan. Autoregressive model beats diffusion: I.lama for scalable image generation. 45X55:2406.06555, 2024. 25
[66] Zhiqing Sun, Sheng Shen, Shengcao Cao, Haotian Liu, Chunyuan Li, Yikang Shen, Chuang Gan, Liang-Yan Yan Gui, Yu-Xiong Wang. Yiming Yang, Kurt Keutzer, and Trevor Darrell. Aligning large multimodal models with factually augmented rlhf. arXiv:2309. 145255, 2023. 2
[67] Chameleon Team. Chameleon:
Mixed-modal carly-fusion foundation models.
arXiv:2405.09816 2024. 1,2
[68] Kimi Team, Angang Du, Bofei Gao, Bowei Xing, Changjiu Jiang, Cheng Chen, Cheng Li, Chenjun Xiao, Chenzhuang Du, Chonghua Liao, et al. Kimi k1.5: Scaling reinforcement learning with llms. arXiv:2501. 12599, 2025. 2
[69] Peter Tong, Ellis Brown, Penghao Wu, Sanghyun Woo, Adithya Jairam Vedagiri IYER, Sai Charitha Akula, Shusheng Yang. Jihan Yang, Manoj Middepogu, Ziteng Wang, al. Cambrian-1: A fully open, vision-centric exploration of multimodal 1lms. In NeurIPS, 2024. 2
[70] Shengbang Tong, David Fan, Jiachen Zhu, Yunyang Xiong, Xinlei Chen, Koustuv Sinha, Michael Rabbat, Yann LeCun, Saining Xie, and Zhuang Liu. Metamorph: Multimodal understanding and generation via instruction tuning. arXiv:2412.144 14/66, 2024. 2
[71] Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timo- thée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, et al. Llama: Open and efficient foundation language models. 27Xiv: 2023. 2
14

---

[72] Michael Tschannen, Alexey Gritsenko. Xiao Wang, Muhammad Ferjad Naeem, Ibrahim Alabdulmohsin, Nikhil Parthasarathy, Talfan Evans, Lucas Beyer, Ye Xia, Basil Mustafa, et al. Siglip 2: Multilingual vision-language encoders with improved semantic understanding, localization, and dense features. arXiv:2502. 14786, 2025.
[73] Aaron Van Den Oord, Oriol Vinyals, et al. Neural discrete representation learning. In NeurIPS, 2017.2 2
[74] Chunwei Wang, Guansong Lu, Junwei Yang, Runhui Huang, Jianhua Han, Lu Hou, Wei Zhang. and Hang Xu. Illume: Illuminating your ilms to see, draw, and self-enhance. arXiv:2412.06663,
2024. 7, 26
[75] Junke Wang, Yi Jiang, Zehuan Yuan, Bingyue Peng, Zuxuan Wu, and Yu-Gang Jiang. Omnito- kenizer: joint image-video tokenizer for visual generation. In NeurIPS, 2024. 2
[76] Junke Wang, Zhi Tian, Xun Wang, Xinyu Zhang, Weilin Huang, Zuxuan Wu, and Yu-Gang Jiang. Simplear: Pushing the frontier of autoregressive visual generation through pretraining, sft, and rl. arXiv:2504.1 1455, 2025. 7, 26
[77] Peng Wang, Shuai Bai, Sinan Tan, Shijie Wang, Zhihao Fan, Jinze Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, al. Qwen2-vl: Enhancing vision-language model's perception of the world at any resolution. arXiv:2409.1211 2024. 2
[78] Weiyun Wang, Zhe Chen, Wenhai Wang. Yue Cao, Yangzhou Liu, Zhangwei Gao, Jinguo Zhu, Xizhou Zhu, Lewei Lu, Yu Qiao, et al. Enhancing the reasoning ability of multimodal large language models via mixed preference optimization. arXiv:2411.1044/ 2024. 2
[79] Weiyun Wang, Zhangwei Gao, Lianjie Chen, Zhe Chen, Jinguo Zhu, Xiangyu Zhao, Yangzhou Liu, Yue Cao, Shenglong Ye, Xizhou Zhu, et al. Visualprm: An effective process reward model for multimodal reasoning. M.ii:25033.1291, 2025. 2,5
[80] Xinlong Wang, Xiaosong Zhang, Zhengxiong Luo, Quan Sun, Yufeng Cui, Jinsheng Wang, Fan Zhang, Yueze Wang, Zhen Li, Qiying Yu, et al. Emu3: Next-token prediction is all you need. arXiv:2409. 18869, 2024. 7, 26
[81] Yi Wang, Mushui Liu, Wanggui He. Longxiang Zhang. Ziwei Huang, Guanghao Zhang, Fangxun Shu, Zhong Tao, Dong She, Zhelun Yu, et al. Mint: Multi-modal chain of thought in unified generative models for enhanced image generation. NMt::250330/298,, 2025. 2
[82] Yixuan Weng, Minjun Zhu, Fei Xia, Bin Li. Shizhu He, Shengping Liu, Bin Sun. Kang Liu, and Jun Zhao. Large language models are better reasoners with self-verification. In EMNLP, 2023. 2
[83] Zejia Weng, Xitong Yang, Zhen Xing, Zuxuan Wu, and Yu-Gang Jiang. Genrec: Unifying video generation and recognition with diffusion models. In NeurIPS, 2024. 1
[84] Chengyue Wu, Xiaokang Chen, Zhiyu Wu, Yiyang Ma. Xingchao Liu, Zizheng Pan, Wen Liu, Zhenda Xie, Xingkai Yu, Chong Ruan, et al. Janus: Decoupling visual encoding for unified multimodal understanding and generation. arXiv:2410. 13848, 2024. 1, 3, 6, 7, 9, 25, 26
[85] Yecheng Wu, Zhuoyang Zhang, Junyu Chen, Haotian Tang, Dacheng Li, Yunhao Fang, Ligeng Zhu, Enze Xie, Hongxu Yin, Li Yi, et al. Vila-u: a unified foundation model integrating visual understanding and generation. ICLR, 2025. 2,
[86] Enze Xie, Junsong Chen, Yuyang Zhao, Jincheng Yu, Ligeng Zhu, Chengyue Wu, Yujun Lin, Zhekai Zhang, Muyang Li, Junyu Chen, al. Sana 1.5: Efficient scaling of training- -imm time and inference-time compute in linear diffusion transformer. arXiv preprint arXiv:2501. 18427, 2025. 5
[87] Jinheng Xie, Weijia Mao, Zechen Bai, David Junhao Zhang, Weihao Wang, Kevin Qinghong Lin, Yuchao Gu, Zhijie Chen, Zhenheng Yang, and Mike Zheng Shou. Show-o: One single transformer to unify multimodal understanding and generation. arXiv:2408. 12528, 2024.
6, 7, 25, 26
15

---

[88] Tianyi Xiong, Xiyao Wang, Dong Guo, Qinghao Ye, Haoqi Fan, Quanquan Gu, Heng Huang, and Chunyuan Li. Llava-critic: Learning to evaluate multimodal models. arXiv:2410.02712. 2024. 2
[89] Mingze Xu, Mingfei Gao, Zhe Gan, Hong-You You Chen, Zhengfeng Lai, Haiming Gang, Kai Kang, and Afshin Dehghan. flowfast-llavat A strong training-free baseline for video large language models. arXiv:2407, 15844, 2024. 2
[90] Mingze Xu, Mingfei Gao. Shiyu Li, Jiasen Lu, Zhe Gan, Zhengfeng Lai, Meng Cao, Kai Kang, Yinfei Yang, and Afshin Dehghan. Slowfast-llav--ss A family of token-efficient video large language models for long-form video understanding. arXiv:2503. 18943, 2025, 2, 4, 9, 28
[91] An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, et al. Qwen2. 5 technical report. arXiv:2412.15115, 2024. 6
[92] An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li. Dayiheng Liu, Fei Huang, Haoran Wei, Huan Lin, Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Yang, Jiaxi Yang, Jingren Zhou, Junyang Lin, Kai Dang, Keming Lu, Keqin Bao, Kexin Yang, Le Yu, Mei Li, Mingfeng Xue, Pei Zhang, Qin Zhu, Rui Men, Runji Lin, Tianhao Li, Tingyu Xia, Xingzhang Ren, Xuancheng Ren, Yang Fan, Yang Su, Yichang Zhang, Yu Wan, Yuqiong Liu, Zeyu Cui, Zhenru Zhang, and Zihan Qiu. Qwen2.5 technical report. 1rXiv:2412.55115. 2024. 2
[93] Jian Yang, Dacheng Yin, Yizhou Zhou, Fengyun Rao, Wei Zhai, Yang Cao, and Zheng- Jun Zha. Mmar: Towards lossless multi-modal auto-regressive probabilistic modeling. arXiv:2410. 10798, 2024.
[94] Ling Yang. Xinchen Zhang, Ye Tian, Chenming Shang, Minghao Xu, Wentao Zhang, and Bin Cui. Hermesflow: Seamlessly closing the gap in multimodal understanding and generation. arXiv:2502. 2025. 2
[95] Huanjin Yao, Jiaxing Huang, Wenhao Wu, Jingyi Zhang, Yibo Wang, Shunyu Liu, Yingjie Wang, Yuxin Song, Haocheng Feng, Li Shen, al. Mulberry: Empowering milm with ol-like reasoning and reflection via collective monte carlo tree search. arXiv:2412.18319, 2024. 2
[96] Hanrong Ye, De-An Huang, Yao Lu, Zhiding Yu, Wei Ping, Andrew Tao, Jan Kautz, Song Han, Dan Xu, Pavlo Molchanov, et al. X-vila: Cross-modality alignment for large language model. arXiv:2405. 19335, 2024.
[97] Lijun Yu, Jose Lezama, Nitesh Bharadwaj Gundavarapu, Luca Versari, Kihyuk Sohn, David Minnen, Yong Cheng, Agrim Gupta, Xiuye Gu, Alexander G Hauptmann, et al. Language model beats diffussin-ttoeenizer is key visual generation. In ICLR, 2024.
[98] Lijun Yu. José Lezama, Nitesh B. Gundavarapu, Luca Versari, Kihyuk Sohn, David Minnen, Yong Cheng, Agrim Gupta, Xiuye Gu, Alexander Hauptmann, Boqing Gong, Ming- Yang, Irfan Essa, David A. Ross, and Lu Jiang. Language model beats diffusion tokenizer is key to visual generation. arXiv:2310.05737 2023. 2
[99]
Xiang Yue, Yuansheng Ni, Kai Zhang, Tianyu Zheng, Ruoqi Liu, Ge Zhang, Samuel Stevens, Dongfu Jiang, Weiming Ren, Yuxuan Sun, Cong Wei, Botao Yu, Ruibin Yuan, Renliang Sun, Ming Yin, Boyuan Zheng, Zhenzhu Yang, Yibo Liu, Wenhao Huang, Huan Sun, Yu Su, and Wenhu Chen. Mmmu: A massive -ulti-discipline multimodal understanding and reasoning benchmark for expert agi. In CVPR, 2024. 25
[100] Xiaohua Zhai, Basil Mustafa, Alexander Kolesnikov, and Lucas Beyer. Sigmoid loss for language image pre-training ICCV, 2023. 2, 3,
[101] Boqiang Zhang, Kehan Li, Zesen Cheng, Zhiqiang Hu, Yuqian Yuan, Guanzheng Chen, Sicong Leng, Yuming Jiang, Hang Zhang, Xin Li, VideoL Frontier multimodal foundation models for image and video understanding, 1.X1::2501.13106, 2025. 2
16

---

[102] Haotian Zhang, Mingfei Gao, Zhe Gan, Philipp Dufter, Nina Wenzel, Forrest Huang, Dhruti Shah, Xianzhi Du, Bowen Zhang, Yanghao Li, et al. Mml. 5: Methods, analysis & insights from multimodal IIm fine-tuning. arXiv:2409.20566 2024. 6
[103] Yuanhan Zhang, Jinming Wu, Wei Li, Bo Li, Zejun Ma, Ziwei Liu, and Chunyuan Li. Video
instruction tuning with synthetic data. arXiv:2410.02713, 2024. 2
[104] Yue Zhao, Yuanjun Xiong, and Philipp Krähenbühl. Image and video tokenization with binary spherical quantization. 2r0iv:240.00755 2024. 2
[105] Chunting Zhou, Lili Yu, Arun Babu, Kushal Tirumala, Michihiro Yasunaga, Leonid Shamis, Jacob Kahn, Xuezhe Ma, Luke Zettlemoyer, and Omer Levy. Transfusion: Predict the next token and diffuse images with one multi-modal model. arXiv:2408. 11099, 2024. 2
[106] Jinguo Zhu, Weiyun Wang. Zhe Chen, Zhaoyang Liu, Shenglong Ye, Lixin Gu, Yuchen Duan, Hao Tian, Weijie Su, Jie Shao, et al. Internv13: Exploring advanced training and test-time recipes for open-source multimodal models. arXiv:2504. 10479, 2025. 2,5
[107] Xianwei Zhuang. Yuxin Xie, Yufan Deng, Dongchao Yang. Liming Liang, Jinghan Ru, Yuguo Yin, and Yuexian Zou. Vargpt-v1. 1: Improve visual autoregressive large unified model via iterative instruction tuning and reinforcement learning. 250.009904 2025. 7, 26
[108] Orr Zohar, Xiaohan Wang, Yann Dubois, Nikhil Mehta, Tong Xiao, Philippe Hansen-Estruch, Licheng Yu, Xiaofang Wang, Felix Juefei-Xu, Ning Zhang, et al. Apollo: An exploration of video understanding in large multimodal models. arXiv:2412 10360, 2024."
17

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: The abstract and introduction have accurately claimed the contributions of this work, including the UniGen model and the CoT-V test-time scaling strategy.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
is fine to include aspirational goals as motivation as long as is clear that these goals not attained by the paper.
2. Limitations
Question:
Does the paper discuss the limitations the work performed by the authors?
Answer: [Yes]
Justification:
Limitations have been discussed in Sec.5.
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't 't acknowledged judgment and recognize that individual actions in favor of transparency play impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
the paper. The authors should use their best
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [NA]
18

---

Justification:
This paper does not include theoretical results.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced the statement of any theorems.
The proofs can either appear the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged to provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Question:
Answer: [Yes]
Justification: The paper fully discloses the information needed to reproduce the main experimental results in main paper's Sec. and Sec. 4.1.
Guidelines:
The answer NA means that the paper does not include experiments. If the paper includes experiments, No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make it clear how reproduce that algorithm. (b) If the contribution is primarily
model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution is a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome deseribe the particular way they provide for reproducibility, In the case of closed-source models, it may be that access to the model is limited in some way (e.g., registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
19

---

Answer: [Yes]
Justification:
The paper plans to open-source the code and data after the internal review.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines
public/gu httlc//guiees//oeeeeeeee www.ubmisssoooooioioooooooo for more details.
While we encourage the release of code and data, we understand that this might not be possible, "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (c.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https: //nips. wttpccccppcccccccccccccccccccccc wwwgggiiiiiie wwwuudes/oooooo des/CodeSubbi ssionPolicy for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible supplemental material (appended the paper) is recommended, but including URLs to data and code is permitted.
6. Experimental setting/details
Does the paper specify all the training and test details (c.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary understand the results?
Question:
Answer: [Yes]
Justification:
The paper has specified the training and test details in Sec. 4.1.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core the paper to a level of detail that is necessary appreciate the results and make sense of them.
The full details can be provided either with the code, appendix, as supplemental material.
7. Experiment statistical significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [No]
Justification: Following common practice in the multimodal learning literature, do not report error bars in this paper because of the heavy computation overheads.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the enror bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
20

---

It should be clear whether the error bar is the standard deviation or the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables figures symmetric error bars that would yield results that are out range (e.g. negative error rates).
error bars are reported in tables or plots, The authors should explain the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification:
The paper has provided the computation information in Sec. 4.1.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make into the paper).
9. Code of ethics
Question: Does the research conducted the paper conform, in every respect, with the NeurlPS Code of Ethics https: http://eeeriiippeee httper/rppp....oooooo/ccccccciee
Answer: [Yes]
Justification:
The paper is under the NeurlPS Code of Ethics.
Guidelines:
The answer NA means that the authors have not reviewed the NeurlPS Code of Ethics.
If the authors answer No, they should explain the special cireumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., if there is special consid- cration due to laws or regulations their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification: Unified MLLMs offer significant scientific benefits by enabling more intuitive human-Al interaction and advancing general-purpure multimodal understanding. There many real-world application scenarios, such as design assistant, education, and collaborative robots. However, there could be unintended usages and advocate responsible usage complying with applicable laws and regulations.
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact why the paper does not address societal impact.
21

---

Examples of negative societal impacts include potential malicious unintended uses (e.g., disinformation, generating fake profiles, surveillance), faireess considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms monitor how system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper deseribe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification: The paper poses no such risks.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification: We properly cite the original assets in the paper.
Guidelines:
The answer NA means that the paper does use existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset used and, possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website). the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms use in package should be provided. For popular datasets, papersw thcode. com com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
22

---

For existing datasets that are re-packaged, both the original license and the license of the derived asset (if has changed) should be provided.
this information is not available online, the authors are encouraged to reach out to the asset's s creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes]
Justification: The details of the newly contributed dataset/code/mmodl have been discussed in Sec. of the main paper.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/moodl as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset used.
At submission time, remember to anonymize your assets (if applicable). You can either create anonymized URL or include anonymized zip file.
14. Crowdsourcing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification:
The paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included the main paper.
According to the NeurIPS Code Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: The paper does not involve crowdsoureing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere the NeurIPS Code of Ethics and the guidelines for their institution.
23

---

For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
16. Declaration of LLM usage
Question: Does the paper describe the usage of LLMs if it is important, original, or non-standard component of the core methods this research? Note that the LLM is used only for writing, editing, formatting purposes and does not impact the core methodology, scientific rigorousness, or originality of the research, declaration is not required.
Answer: [Yes]
Justification: The LLM is used to labeling the training data. We clarified their usage in the main paper.
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, non- non-standard components.
Please refer our LLM policy (https: http:::////uuiippee htttss////errrpsccc///oreerrrrrs /L/ for what should or should not be described.
24

---

A Benchmarks and Evaluation Protocol
For image understanding, we include widely-used (i) general VQA benchmarks, such as GQA [26].
RealWorld-O QA [18], and Seedbench [33]. (ii) knowledge- benchmarks, such as A12D [30],
MMMU [99], and Math [46], and (iii) hallucination benchmarks, such as POPE [36]. We leverage the Imms--evll toolkit to compute the results for the above benchmarks.
For text-to-image generation benchmarks, we report results on GENEVAL [17] and DPG- BENCH [23] to comprehensively evaluate the semantic alignment between text prompt and the generated images. To fairly compare with recent unified MILMs [84, 9, 87], our results are obtained using the official evaluation repository of GENEVAL and DPG-BENCI".
B More Ablation Studies
Table 11: Ablation of Best-of-N strategy. The results are from UniGen with CoT-V.
Table 12: Ablation of the order of CoT-V post- training and DPO training. The results are from UniGen with CoT-V.
N
GenEval DPG-Bench Speed (s/img)
Sequence
Und Avg. GenEval DPG-bench
0.73
84.89
3.55
3
0.77
85.13
16.59
DPO + CoT-V Post-train
62.7
0.78
85,19
5
0.78
85.19
27.22
CoT-V Post-train DPO
62.7
0.76
85.20
Choosing N for Best-of-N strategy. We ablate N for Best-of-NV selection with CoT-V. As shown in Table 11, using larger N consistently yields higher performance on GENEVAL benchmark. To assess the efficiency trade-off, we further measure the inference speed of CoT-V on the same H100 GPU, considering both MILMM execution time and tokenizer decoding overhead. The average speed is computed across all GENEVAL samples using one prompt per batch. Under the default setting of (N=5), the total inference time is approximately 8x slower than standard inference, highlighting the trade-off between increased computational cost and improved accuracy.
Switching the order of CoT-V post-training and DPO training. We reverse the order of DPO training and CoT-V post-training and present the results in Table 12. The findings indicate that the training order has minimal impact, with the default setting showing slight advantage on GENEVAL.
Table 13: Ablation of visual tower for under- standing. The results are from UniGen-Sie SFT.
Table 14: Ablation of visual tokenizer for gen- eration. The results are from UniGen-Sar SFT.
Visual Encoder Und Avg.
GenEval DPG-Bench
Tokenizer
GenEval
DPG-bench
Freeze Unfreeze
63.11
0.63
82.75
MAGVTTv2
0.63
82.75
63.16
0.65
82.71
VQ-16
0.62
82,93
Freezing the visual tower for understanding during supervised fine-tuning achieves performance on par with the unfrozen setting understanding benchmarks. As reported in Table 13, unfreezing the visual tower leads to a 2% improvement on GENEVAL, while freezing the encoder reduces computational cost. These results indicate that the frozen design leads to more cost-efficient option with only minor performance differences.
Changing the discrete visual tokenizers. By default, we adopt the MAGViTv2 implementation from Show-o". To ablate the impact of discrete visual tokenizer on generation, we further experiment with the VQ-16 tokenizer from LLamaGien [65]. As shown in Table 14, both tokenizers achieve comparable performance on GENEVAL and DPG-BENCH after supervised fine-tuning, demonstrating the robustness and generalizability of our training framework.
P "hups://github.co http:///hhbbbeomm/m/o/omoeee//// httpp://wwggigghtcec.o///bbbbbbt
25

---

C More Results
We present the breakdown comparison of UniGen against state-of-the-at models on GENEVAL and DPG-BENCU in Table 15 and Table 16.
Table 15: Comparison with state-of-the-art models on the GenEval benchmark.
Model
#Params
Single Obj.
Two Obj.
Counting
Colors
Position
Color Attri
Overallf
Text-to-lmage Generation Models
DALLE-2 (61)
6.5B
0.94
0.66
0.49
0.77
0.10
0.19
0,52
DALLF-33[4]
0.96
0.87
0.47
0.83
0.43
Emu3 [80]
SDXL [57]
2.6B
0.98
0.74
0.39
0.85
0.15
0.23
0.55
SimpleAR [76]
1.5B
0.90
0.28
0.45
0.63
Infirity [21]
2B
0.85
0.49
0.57
0.73
Unified MLLMs
Show-o [87]
1.3B
0.95
0.52
0.49
0.82
0.11
0.53
Janus [84]
0.84
Janus-Pro [9]
1.5B
0.98
0.82
0.51
0.89
0.65
0.56
0.73
I.IUME [74]
7B
0.99
0.86
0.45
0.71
0.39
0.28
0.61
UniToken [29]
7B
0.99
0.80
0.35
0.84
0.38
0.39
0.63
VARGPT-v1.1 [107]
9B
0.96
0.53
0.48
0.83
0.13
0,21
0.53
TokenFlow-XX XI. [58]
13B
0.93
0.72
0.82
0.45
UniGen
0.87
UniGen CoT-V
1.5
1.00
0.94
0.78
0.87
0.57
0.54
0.78
Table 16: Comparison with state-of-the-art models on the DPG-bench benchmark.
Model
#Params
Attrihute Text-to-Image Generation Models
Global
Entity
Relation
Other
Overall
Hunyuan-DiT [37]
84.59
80.59
88.01
74.36
86.41
78.87
DALLE-3[[4]
90.97
89.61
88.39
90.58
89.83
83.50
Emu3 [80]
85,21
86.68
86.84
90.22
83.15
80.60
SDXL [57]
83.27
82.43
80.91
80.41
74.65
SimpleAR [76]
87.97
81.97
Infinity [21]
93.11
90.76
83.46
Unified MLLMs
Show-o [87]
1.3B
80.39
82.17
83.36
82.88
71.70
Janus [84]
1.3B
82.33
87.38
87.70
85.46
86.41
79.68
Janus-Pro [9]
87.58
88.63
88.17
88.30
82.63
VARGPT-v1.1 [107]
84.83
82,80
84.95
87.70
78.59
TokenFlow-XL [58]
13B
78.72
79.22
81.29
85.22
71.20
73.38
UniGen
1.5B
91.53
90.39
90.30
90.86
84.89
UniGen + CoT-V
1.5B
91.95
89.68
90.90
90.91
85.19
D Details of Test-Time Strategies
D.0.1 Prompts of different verifications for test-time inference
Prompt. 1: Chain-of- Thought Verification
(image) This image is generated by prompt: (prompt). Please assess the image generation quality step by step. First, breakdown the prompt into multiple visual questions and iteratively answer each question with Yes or No between <think start> <think <think_end>> end>. Questions should cover all-round details about whether the image accurately represents entity categories, counting of entities, color, spatial relationship in the prompt. Next, output the final result between <answer_ start> <answer end>, Output Yes multi-choice answers equal yes to show the image has accurate alignment with the prompt. Otherwise answer with No.
26

---

Prompt. 2:
Outcome Verification
(image) This image is generated by prompt: (prompt). Does this image accurately represent the prompt? Please answer yes or no.
Prompt. 3: Rule-based Verification
(image) (question) Please answer yes no with detail explanation.
E Details of Training
Table 17: Hyperparameter setup for different training stages of UniGen. Data ratio refers the ratio of image understanding data, pure text data, and image generation data.
Hyperparameters
PT-1
PT-2
SFT
DPO
CoT-V Post-Training
Learning
1.0 10
1.0 10
1.0 10
1.0 x 10
1.0 x 10
LR scheduler
Cosine
Cosine
Cosine
Cosine
Cosine
Weight decay
0.01
0.01
0.05
0.05
0,05
Gradient clip
1.0
1.0
1.0
1.0
1.0
Optimizer
AdamW
Adam W
Adam W
AdamW
AdamW W
Warm-up steps
6000
5000
1000
500
Training steps
150k
400k
146k
1.6k
0.5k
H100 hours
1.0k
2.8k
0.7
Batch size
896
512
64
80
64
Data ratio
2:1:4
2:1:4
4:1:3
1:--
E.0.1
Training Parameters
Details of hyperparameters during each training stage are presented in Table 17.
E.0.2
Training Data Overview
We list the datasets used in our training stages in Table 18. Refer to Appendix E.0.4 and Ap- pendix E.0.5 for more details about the preference data and CoT-V data.
E.0.3 Prompts for Generating Pre-Train Data
We use Prompt. prompt Qwen2.5VLL-7B for generating fine ane-grained captions for CC-3M, CC-
12M, SA-IB and ImageNet that are used in pre-training stages as shown in Table 18.
Prompt. Re-caption
(image) What the content of this image?
E.0.4 Preference Data Generation for DPO
PARM. GENEVAL metric used to rate each generated image candidate per prompt. The highest and lowest rated ones are used as the preferred and rejected samples for this prompt.
T2I-Comp and SA-1B. These prompts are more complex than the prompts of PARM, therefore. is difficult to rate each generated image using rule-based metrics. We adopt two-step approach to evaluate the coherence between image and the prompt. First, we use Qwen2.5-7B to decompose each text prompt into atomic facts represented as questions using Prompt. 5. Then, the image with each decomposed question fed into Qwen2.5VL-77 with Prompt. 6. The model responds with yes if visual generation passes the fact-check and with no otherwise. The final score of an image is calculated by averaging results of fact-checks. take the most and least aligned images per prompt as the preferred and rejected sample pairs.
27

---

Prompt. 5: Visual Questions Generation
Now you need to convert image description into fine-grained, related visual questions. The questions should comprehensively cover detailed visual facts of entities, attributes (c.g.. color, count, texture, shape, and size), and relationships (e.g., spatial and non-spatial) between the entities mentioned in the description. Please complete the task by analyzing each clause in the sentence step by step. For each clause, first raise questions about whether each mentioned entity exists in the image. Then, raise questions about whether the attributes or relationships of the entities are accurately represented in the image. For an image accurately aligned with the description, all questions should be answered with "yes"; otherwise, they should be answered with "no".
Make sure all questions are able to be responded with yes or no and are connected with semicolon, Here are examples:
Example 1:
description: three black keys, four chickens and a fabric blanket
output: Are there keys?; Are there three keys?; Are the keys black?; Are there chickens?; Are there four chickens?; Is there a blanket?; Is the blanket fabric? Example 2:
description: A person in blue shirt and red and black apron is using a power tool, likely drill, to assemble white cabinet shelving unit indoors. The floor is covered with light-colored wood or laminate material.
outpur: Is there person?; Is the person wearing shirt; Is the shirt blue?; Is the person wearing apron?; Is the apron red and black?; Is the person using a drill?; Is there white cabinet or shelving unit?: the person using the drill indoors?; Is there light-colored wood on the floor?; Is there laminate material on the floor? Example 3:
description: large Ferris wheel with digital clock showing the time as 1 11:00. The Ferris wheel is located an urban area, as indicated by the modern buildings in the background. There is also a tree on the left side of the image, partially obscuring the view of the Ferris wheel. The sky appears clear, suggesting sunny day.
output: Is there a Ferris wheel?; Is there digital clock?; ; Is the digital clock on the Ferris wheel?; Is the digital clock showing the time as 11:00?; Is the Ferris wheel located in an urban area?; Are there modern buildings in the background?; Is there tree on the left side?; the sky clear and sunny?
Please convert this image description: (eterippiion:deerpppppiiiii (description iinto fine-grained related visual questions.
Prompt. 6: Visual Fact-Check
(image) (question) Please answer yes without explanation.
We display some visual results of DPO preference data in Fig.
Table 18: Training data overview. CC, SA, IMN, JD, T2I indicate CC-3M&CC- 12M, SA-1B, ImageNet, JourneyDB, text-2-image-2 respectively. Recap denotes that the images are re-captioned using Qwen2.5VL-7B.
Stage
Gen Data
Und Data
Text-only
PT-1 IMN (Recap)
(CC+SA+IMN) (Recap)
RefinedWebb
PT-2
(CC+SA+IMN) (Recap)
(CC+SA+IMN) (Recap)
RetinedWeb
SFT
JD+T2I Preference Data
SF-LLaVA115 (Image Mixture) [90]
RefinedWeb
DPO CoT-V
CoT-V data
E.0.5
CoT-V Post-Training Data
We sample 20K preference data from PARM and T21-Comp in Appendix E.0.4 to construct our CoT-V post-train data and use Prompt. to encourage UniGen generate CoT reasoning during training. To supervise the training process, construct the CoT reasoning labels based on decomposed
28

---

Eatity
Entity: semantics
Attribute: counting
Attribute: shape
Relation: spatial
Preferred
Rejected
Five Individuals shand stone wall. cundoons with clear blus sky in the buckgrourd One person holde a dog on leash. and the sotting sugpeess an anciont histnicaal sile
Two black vehscles are poriedd in front at oenate building, which fentunes sistues and inscriptions abowe the entrane:: The car on the left is hearne with a flural arangeeman on moof and the car on the right is standard sedar.
Tembstunes and mamoleemm are decernted with flown and candllss A graaey hill with a clock lower is visille the background
Small animal tracks are scostuned acrons light- coloral send, siighlly dieker than the barkground.
Skrwers of glaved meniballe are arniggd en surfice with grazn ketture underneell.
Prompt
Figure 6: Visual examples of our generated preference data for DPO training.
atomic question-answer pairs corresponding to visual facts presented in the image. For PARM, separate each prompt into fine-grained sub-questions according to the templates originally used for generating the prompt. Rules of GENEVAL image with yes or no. For T2I-Comp, we directly use the decomposed question-answer from the preference data. The final answer is yes if all the sub-questions are answered with yes and is otherwise. To form the CoT label, the separated question-answer answers are treated thinking process enclosed within special tokens sessvrrsstar>>annsiee
used label each sub-question corresponding the
end>, and the final answer resides within
F
More Qualitative Results
Paraed CoT-V Quectian
Prefereed
Rejvcted
You
foar
Betr Oor achnnc
No
lehera bridd
The inif browy?
Yes
No
ledira dising abie?
plote oc's diaing table right of ovee
luther exen?
You
Yeu
Ir daing vala righ of ewer?
Yeu
No
Yoa
à these surtbeas!
Yes
green orange
the gress
Figure 7: Successful examples and CoT-V verification on GENEVAL.
29

---

Parsed Co Questions
Prefered
Dones?
vizuaa umsannent of blke yellow Bownes, wibh delicte poad one has gree stosa, placed a die plas vas, The uae pelished thariiating the Amarel cavca akding sncc saarsi curns to the sating
light for eanmal
glms?
Ao fhe Teweess bly!
Are the Forere yollie??
Do pasking deciiossel Mack wiik?
erticing painting domenated by dtata and white, wold and Blver out, cabing color the
a flower?
accertoonatt burigruun The recure the stokes is toctle quilis
glving pleciig
imag yolkow Ind henouhh the cobo. de uestur sell dark dath cue be atsorval providing contua
cabc?
conwtieed thac
odters tedadhp cooel the trone a nendescnpr acc object to pumally vistle, adding de comporttoos
In de clibh?
is the cabe roiing on clock
de bowoth the sco?
Aie Dhire twuic!
alecks dark weeden beets resing clens reflees the riborri dhy chese and the buch gowwary semeerinng the labels wlps. Poeikumaa ooe to oiher the heatt sacked sdle vuleo.
10
0.93
jcurncy
Figure 8: Successful examples and CoT-V verification on CBNCHE
False Positive Answers
False Negative Answers
Is there an buw!?
Yes
Is there bottle?
Yes
a photo of red
there sink?
Ves,
photo of a bottle right of train
Is there train!
Yes
bowl and pisk
Is howl red?
Ves
Is the bottlc right of the
No
Is the sink pink?
Yes
Fube Pursed CoT-V Questions
rustic, toned wooden table holds white ceramic plate piled high with steaming dumplings, the pleats carefully crimped, indiciting handerafted care, Next it sits round, earthy-tomed how! filled with ripe, perple plems their skins glessy and butl The gentle glow the sotting
buwl?
Yes
there bowl of plamss
Yes
light over scene. illiinatiin the golden wheat fields and a distact barn in the backdrop, painting picturesque countryside uhlles.
dumplings with plums?
On a smooth, polished weoden desk, there are five small square shaped power converters with metallic finish neatly placed beside sach other. Behund
crasers round?
Yes
grain, slighlly showing signs of use with puncil sha vins nearby. by: The desktop itself is batbed in the warn glow of the room's i lighting, highlightng the gentle textures of the wood and the matt suriaces of the converters and crasers.
li
there pencil shavings?
Are there five power converien Mereeiieeniislle
N/A
Figure 9: Failure cases GENEVAL and DPG-BENCH. The top half of the image shows failed examples of CoT-V on short prompts. The bottom half of the image shows additional cases with bad or missing questions when CoT-V parses the complicated and long prompts.
We present qualitative results in Fig. 7 and Fig. 8. They indicate CoT-V's effectiveness selecting images that accurately convey the entities, color, counting and spatial relation. However, as failure cases shown in Fig. 9, CoT-V may struggle with hallucination in more difficult cases. Particularly, we acknowledge that UniGen still falls short of generating an accurate reasoning process given free-form complex prompts. We posit that scaling up the model size or improving our CoT training via reinforcement learning algorithms could improve the capability of reasoning and image generation, and consequently enhance the overall performance of CoT-V.
30


Reviews:
[{"review_id": "WZohRb5agz", "reviewer": "Reviewer_wZHu", "summary": "# Summary\n\n---\n\nThis work introduces a novel 1.5B-parameter model that unifies both image understanding and generation within a single framework. The model is trained through a comprehensive pipeline, including two-stage pre-training, SFT, DPO, and an additional CoT-V post-training phase to enable inference-time scaling with CoT-V techniques. For model design, the authors employ MAGVIT-v2 as the image tokenizer and utilize a cosine masking schedule for image decoding following MaskGIT. The image understanding encoder uses the widely adopted SigLIP. Both the encoder and decoder are kept frozen throughout all training stages, with only the LLM and connector modules being trainable. Notably, during pre-training, the image tokens are also optimized on understanding tasks to enhance the semantics of the image tokens.", "questions": "# Questions (bonus)\n\n---\n\n- I am also curious about the scaling performance of UniGen. In your scaling experiments (if any), did you encounter any bottlenecks caused by the generation decoder? Specifically, as the model size increases, did you observe that the image generation ability tends to saturate? If so, is this likely due to limitations of the image tokenizer, especially the generation decoder?\n- How do the authors recap the PT-1 stage data using Qwen2.5-VL-7B? What prompts did you use for this process?", "limitations": "Yes.", "rating": 4, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 3, "significance": 2, "originality": 2, "strengths_and_weaknesses": "# Strength\n\n---\n\n- The model demonstrates impressive performance despite its small 1.5B parameter size.\n- The CoT-V inference-time scaling technique is both innovative and interesting.\n- The implementation is solid, with comprehensive training stages that span from pre-training to preference learning.\n\n# Weakness\n\n---\n\n1. **What are the differences between the PT-1 and PT-2 stages?** Section 3.2 does not clearly state the distinction between these two stages. The datasets, augmented datasets, and tasks (t2i, i2t, t) all appear to be the same. Further clarification on how PT-1 and PT-2 differ would be helpful.\n2. **What is the key novelty of UniGen compared to prior work?** UniGen seems to combine multiple existing modeling designs. For example, its architecture is highly similar to Janus, with both featuring decoupled generation (MAGViT) and understanding (SigLIP) encoders, as well as similar training strategies. While some differences exist, such as the use of a cosine masking schedule for parallel visual decoding, multi-stage pretraining, and frozen encoders throughout all stages, there is a lack of ablation studies to assess the impact of these techniques. The explanations for improved performance (lines 240–241) do not sufficiently address differences compared to Janus, leaving the true reasons for UniGen’s superior performance underexplored.\n3. **Regarding Table 2**: Does only UniGen utilize BoN samples for image generation? If so, this could make the comparison unfair, as such techniques like contrastive reranking [1] can significantly affect generation performance.\n4. **Additional ablation studies are needed**: Ablations should cover not only different training stages (PT-1, PT-2, SFT, DPO, CoT-V post-training), but also the key novelties of this work. For example:\n- The effect of the generation encoder and decoder (vs. diffusion-based or flow matching-based methods)\n- The impact of masking-based parallel visual decoding vs. causal decoding [2, 3]\n- Frozen vs. tunable visual encoders during SFT (for comparison with Janus)\n- The effect of different tokenizers\n- The effect of not using the generation encoder for understanding tasks during pre-training\n- (At least) Comparisons among OV, RV, and CoT-V\n- Among others.\n5. **Lack of evaluation on text-only tasks**: The paper does not report results on text-only tasks. How do the authors draw conclusions regarding preserved text performance?\n\n[1] Scaling Autoregressive Models for Content-Rich Text-to-Image Generation\n\n[2] Making LLaMA SEE and Draw with SEED Tokenizer\n\n[3] MIO: A Foundation Model on Multimodal Tokens", "was_revised": true, "final_justification": "The authors’ responses have addressed most of my concerns. The additional ablation studies further enhance the depth and impact of this work, and several of my questions have been clarified in the supplementary materials. The practices and recipes presented serve as valuable guidelines for future research. However, the novelty of this work remains unclear, as CoT-V is essentially a BoN technique, which has already been explored in many previous studies.", "rebuttal": "**To Reviewer wZHu**\n\nThanks for the valuable suggestions. We added comprehensive experiments/ablations to address your concerns. Please see the detailed responses below. We are looking forward to seeing you increase the score. Please let us know if there are any more concerns.\n\n**Q1:** What are the differences between the PT-1 and PT-2 stages? Section 3.2 does not clearly state the distinction between these two stages. The datasets, augmented datasets, and tasks (t2i, i2t, t) all appear to be the same. Further clarification on how PT-1 and PT-2 differ would be helpful. \n\n**A1:** The major difference of PT-1 and PT-2 is the datasets used in those stages (see Table 14 in supplementary materials). \nIn PT-1, our goal is to learn generating basic visual concepts for warming up, so we train with ImageNet. \nIn PT-2, we want to learn broader visual concepts, so we make the T2I datasets more diverse by adding CC3M, CC12M and SA1B. \nWe will make this more clear in the final version.\n\n**Q2:** What is the key novelty of UniGen compared to prior work? UniGen seems to combine multiple existing modeling designs. For example, its architecture is highly similar to Janus, with both featuring decoupled generation (MAGViT) and understanding (SigLIP) encoders, as well as similar training strategies. While some differences exist, such as the use of a cosine masking schedule for parallel visual decoding, multi-stage pretraining, and frozen encoders throughout all stages, there is a lack of ablation studies to assess the impact of these techniques. The explanations for improved performance (lines 240–241) do not sufficiently address differences compared to Janus, leaving the true reasons for UniGen’s superior performance underexplored.\n\n**A2:** We would like to kindly clarify that the novelty of this work does not lies in building new architectures. \nWe claim our contributions in the following: First, with only publicly accessible training data, we walk through the entire training cycle and transparently show how metrics of each benchmark moves across different stages. \nHopefully, the comprehensive ablation would share some useful tips to the community for building future unified MLLMs. \nSecond, UniGen is the first unified MLLM that explicitly shows the generation and understanding capabilities can collaborate together for boosting performance via our proposed CoT verification.\n\n**Q3:** Regarding Table 2: Does only UniGen utilize BoN samples for image generation? If so, this could make the comparison unfair, as such techniques like contrastive reranking [1] can significantly affect generation performance.\n\n**A3:** Our baselines do not use BoN. We softly argue that each method utilizes their own strengths for boosting their results. \nFor example, Janus-Pro uses much more training data including internal high quality data. Similarly, test-time scaling via CoT-V (BoN) is one of our contributions that idenfity an effective way to further boost the generation performance for unified MLLMs. \nTo futher reduce the concern, we compare UniGen (without BoN) to one of the most top-performing Unified MLLM (Janus-Pro) as well as the best text-to-image generation model (infinity) as follows. \nThe results show that even without BoN, UniGen performs better than these strong baselines.\n\n| Model          | GenEval | DPG-Bench |\n| -------------- | ------- | --------- |\n| Infinity       | 0.73    | 83.46     |\n| Janus-Pro      | 0.73    | 82.63     |\n| UniGen-1.5B w/o BoN | 0.74    | 84.89     |\n\n**Q4:** Additional ablation studies are needed.\n\n**Q4.1:** The effect of the generation encoder and decoder (vs. diffusion-based or flow matching-based methods)\n\n**A4.1:** Thanks for the helpful suggestion. \nIncorporating diffusion/flow matching in UniGen requires a lot of infra changes that is challenging to accomplish within the rebuttal time. \nA concurrent work BLIP3o is based on diffusion model with flow matching. Comparison with BLIP3o may be helpful to understand the pros and cons of diffusion based models. \nWe scale up our model and compare with BLIP3o in the following table. Note that BLIP3o utilized GPT4o labeled data to boost performance and we incoporate their dataset in SFT stage for fair comparison. \nThe results show that our method performs significantly better on GenEval and DPG-Bench. \nOne interesting finding is that our UniGen demonstrates strong advantage on DPG-Bench. Our 1.5B model significantly outperforms BLIP3o-8B by ~3.6 points. This may imply that our method has much better instruction following capability for generation, since most of the prompts in DPGBench are long/complicated with many fine-grained elements.\n| Model          | GenEval | DPG-Bench |\n| -------------- | ------- | --------- |\n| UniGen-1.5B | 0.78    | 85.19     |\n| BLIP3o-8B     | 0.84    | 81.6     |\n| UniGen-7B* | 0.90    | 86.38     |\n\n[1] BLIP3-o: A Family of Fully Open Unified Multimodal\nModels—Architecture, Training and Dataset\n\n**Q4.2:** The impact of masking-based parallel visual decoding vs. causal decoding.\n\n**A4.2:** To answer your question, we add experiments by replacing masked based decoding with the causal decoding. The comparison after SFT stage is shown as follows. \nAs we can see, on average, the causal decoding baseline performs much worse on both understanding and generation benchmarks. This implies that full attention across vision tokens has advantages over the casual attention in our design.\n\n|Method| Und. Avg   | GenEval | DPGbench |\n| ----- | ----- | ------- | -------- |\n|mask decoding| 63.11 | 0.63    | 82.75    |\n|causual decoding| 61.94 | 0.53    | 80.99    |\n\n**Q4.3:** Frozen vs. tunable visual encoders during SFT (for comparison with Janus)\n\n**A4.3** We add an experiment by unfreezing the visual encoder during SFT and report the results as follows. The results demonstrate that unfreezing visual encoder shows on-par performance compared to the frozen one, except that there is notable 2% improvements on GenEval. \nSince freezing visual encoder is more cost friendly, we would suggest to go with the frozen design.\n\n|Visual Encoder| Und. Avg   | GenEval | DPGbench |\n| ----- | ----- | ------- | -------- |\n|Frozen| 63.11 | 0.63    | 82.75    |\n|Unfreeze| 63.16 | 0.65    | 82.71    |\n\n**Q4.4:** The effect of different tokenizers\n\n**A4.4:** We add an experiment by replacing the default tokenizer (MAGViTv2) with VQ-16 from LLamaGen[2]. The comparison after SFT stage is shown as follows. \nWe can see that VQ-16 leads to a similar performance as MAGViTv2, verifying the generalizability of our training framework.\n\n| Tokenizer |GenEval | DPGbench |\n| ----- | ------- | -------- |\n| MAGViTv2| 0.63    | 82.75    |\n| VQ-16 |  0.62  |  82.93  |\n\n[2] Autoregressive Model Beats Diffusion: Llama for Scalable Image Generation\n\n**Q4.5:** The effect of not using the generation encoder for understanding tasks during pre-training\n\n**A4.5:** We have shown in Table 8 (supplementary materials) that completely removing understanding tasks during pretraining will negatively impact understanding performance. \nMoreover, we add one more experiment that uses a separate SigLIP encoder for understanding during pretraining. The results after SFT are shown as follows. The results indicate that using a separate SigLIP encoder during pretraining will significantly improve understanding performance while sacrificing generation performance.\n\n| Enc. for Und. | Und. Avg   | GenEval | DPGbench |\n| ----- | ----- | ------- | -------- |\n| MAGViTv2|  63.11| 0.63    | 82.75    |\n| SigLIP | 65.02 |  0.61|    79.82  |\n\n**Q4.6:** (At least) Comparisons among OV, RV, and CoT-V\n\n**A4.6:** For comparisons among the three verification methods, we have provided detailed analysis in section 4.3.2 and Table 4. \n\n**Q4.7:** Lack of evaluation on text-only tasks: The paper does not report results on text-only tasks. How do the authors draw conclusions regarding preserved text performance?\n\n**A4.7:** Thanks for the suggestion. It would be optimal, but not trivial to have no regression on text-only task. \nWe follow prior works (Janus-pro, Show-o) that do not focus on text-only tasks. We believe it will be an interesting future work.\n\n**Q5:** I am also curious about the scaling performance of UniGen. In your scaling experiments (if any), did you encounter any bottlenecks caused by the generation decoder? Specifically, as the model size increases, did you observe that the image generation ability tends to saturate? If so, is this likely due to limitations of the image tokenizer, especially the generation decoder?\n\n**A5:** We add an experiment that scales up the model size of UniGen to 7B under our default training pipeline and data mixtures. \nNote that due to the limited time, we use the DPO and CoT-V data generated by UniGen-1.5B for training UniGen-7B. We expect more increase if we use UniGen-7B for creating its own data. \nNevertheless, we observe increase on both understanding and generation benchmarks. We believe better image tokenizer leads to better generation quality. \nImproving tokenizer is one promising future work in our roadmap.\n\n| Model          | Und Avg.|GenEval | DPG-Bench |\n| -------------- | ------- |------- | --------- |\n| UniGen-1.5B | 62.66| 0.78    | 85.19     |\n| UniGen-7B | 63.78| 0.79    | 85.79     |\n\n**Q6:** How do the authors recap the PT-1 stage data using Qwen2.5-VL-7B? What prompts did you use for this process?\n\n**A6:** We have provided the detailed prompts in the supplementary materials. Please refer to E03."}, {"review_id": "gg7sF6Vfxy", "reviewer": "Reviewer_XW9f", "summary": "This paper proposes a unified model for multimodal understanding and generation. To improve image generation quality, the authors introduce a method called Chain-of-Thought Verification (COT-V) for use during the inference phase. The model first generates a set of candidate images, then utilizes its multimodal understanding capabilities to score each image and selects the one with the highest score as the final output.", "questions": "Please see the weaknesses.", "limitations": "yes.", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 2, "originality": 3, "strengths_and_weaknesses": "Strengths:\n- The paper is clearly written and easy to follow. The motivation is reasonable, and the method is simple and reproducible.\n- Leveraging the chain-of-thought reasoning process of multimodal models to evaluate image quality is an interesting idea.\n\nWeaknesses:\n- The core contribution, Chain-of-Thought Verification, does not appear to emerge naturally from the model but instead relies on fine-tuning with specific datasets. These datasets are based on rule-based scoring, which may limit the potential of the proposed method.\n- There is randomness in the model's output during the chain-of-thought scoring process. How can one ensure that the output format consistently matches the expected structure, such as the format shown in Line 187?", "was_revised": false, "final_justification": "", "rebuttal": "**To Reviewer XW9f**\n\nWe thank the reviewer for their valuable feedback. Please see our response below. We are looking forward to seeing you increase the score. Please let us know if there are any more concerns.\n\n**Q1:** The core contribution, Chain-of-Thought Verification, does not appear to emerge naturally from the model but instead relies on fine-tuning with specific datasets. These datasets are based on rule-based scoring, which may limit the potential of the proposed method.\n\n**A1:** We argue that the instruction following capability emerges by only finetuning on the CoT-V training set for 500 steps. The light finetuning implies the naturalness of CoT-V for the model. \nFor the concern of the rule-based scoring, we argue that rule-based scoring could also be generalizable given the recent success of RL with verifiable reward functions (rule-based) [1]. \nBesides UniGen, we also finetuned Show-o with our dataset and showed that the Show-o can also be significantly improved (see Table 5). This also suggests the generated datasets are general.\n\n[1] DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning\n\n**Q2:** There is randomness in the model's output during the chain-of-thought scoring process. How can one ensure that the output format consistently matches the expected structure, such as the format shown in Line 187?\n\n**A2:** We calculate the percentage of data that follows the expected format and observe that UniGen sucessfully followed the instruction of CoT-V for 100% samples on GenEval, and only failed with one image-prompt input (0.13% in total) on DPGBench. The numbers suggest that our outputs consistently follow the expected format."}, {"review_id": "Huy3jPoqvl", "reviewer": "Reviewer_WbDR", "summary": "This paper introduces a unified image understanding and generation model UniGen. This model borrows the ideas from some established models (e.g., Janus, Show-o) to constitute its architecture and undergoes five training stages: pre-training, supervised fine-tuning (SFT), direct preference optimization, and test-time scaling learning. The authors propose a post-training approach named Chain-of-Thought Verification, which uses UniGen itself as the verifier to evaluate the generated images and produce CoT data. They then conduct SFT on the CoT data to endow the model with thinking ability. In experiments, UniGen surpasses other MLLMs by notable margins.", "questions": "See weaknesses.", "limitations": "Yes.", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 4, "strengths_and_weaknesses": "> Strengths:\n\n1. Unifying image understanding and generation with one MLLM has been a hot and interesting topic in recent years. This paper provides a new unified MLLM with SOTA performance for the community, which can advance this field.\n2.  The proposed CoT-V method is novel and effective. The idea of asking the model to answer a series of atomic questions to serve as the thinking process is interesting and can intuitively enhance model generation performance.\n3. The effectiveness of CoT-V is not only grounded in UniGen but also in unified models, indicating this technique’s generalizability.\n4. The constructed preference dataset would be a valuable contribution to the community if published.\n\n> Weaknesses:\n\n1. The resolution of Fig. 1 is low. Please make a refinement.\n2. In Introduction, the authors state that “we walk through the entire life cycle of the model development” and “draw insightful lessons”. However, in experiments, the authors seem not to show what these “lessons” are. They primarily focus on demonstrating the effectiveness of their specific model design or training strategies, rather than introducing some globally useful knowledge or experiences that can benefit the community in designing such sophisticated models. Can the authors share some experiences (the learned “lessons”) regarding training or model design from a more global perspective?\n3. It is unclear whether the model weights and curated dataset in this paper will be published. If so, the authors should make a clear statement somewhere in the main text.\n4. In DPO training, what is the size of preferred samples $y_w$ and rejected samples $y_l$ used in experiments?\n5. In L193, the authors say that “RV struggles with free-form or complex instructions, such as those in DPG-Bench [21].” and choose CoT-V as a better option. The description is too limited to demonstrate RV’s drawbacks. Since the Best-of-N strategy can also be combined with RV and achieve similar functionality, the superiority of CoT-V over RV remains unclear. Can the authors give some examples of free-form or complex instructions to better illustrate RV’s limitation and give more details regarding CoT-V’s superiority?\n6. The authors use UniGen-DPO as a self-verifier to produce training data for test-time scaling. The model can generate low-quality or erroneous data, and it seems that no human correction has been applied to this data. How do the authors ensure data quality, and will low-quality data hamper model learning?\n7. During training, UniGen first undergoes DPO and then the test-time scaling training. However, it seems that the model could first undergo the test-time scaling stage and then DPO, since training with CoT-V is simply supervised fine-tuning. This also makes sense to first foster the model’s CoT-based reasoning ability and then optimize its preference. What is the reason behind choosing the original order? Will this lead to better model performance? I recommend the authors provide more clarification on this point.\n8. typo: L257: stages -> stages.", "was_revised": true, "final_justification": "Most of the concerns are addressed. Therefore, the rating will be maintained.", "rebuttal": "**To Reviewer WbDR**\n\nThanks for your suggestions. Please see our responses below.\n\n**Q1:** The resolution of Fig. 1 is low. Please make a refinement.\n\n**A1:** Thanks for the suggestion. We will adjust it in the final draft.\n\n**Q2:** In Introduction, the authors state that “we walk through the entire life cycle of the model development” and “draw insightful lessons”. However, in experiments, the authors seem not to show what these “lessons” are. They primarily focus on demonstrating the effectiveness of their specific model design or training strategies, rather than introducing some globally useful knowledge or experiences that can benefit the community in designing such sophisticated models. Can the authors share some experiences (the learned “lessons”) regarding training or model design from a more global perspective?\n\n**A2:** Thanks for pointing this out. We have provided valuable lessons in the supplementary materials (section B). We highlighted the lessons we want to share in bold font. I summarize them in the following. \n\nL1: using high-quality generation data is necessary for further lifting generation results (line 932-935).\n\nL2: using a stronger data mixture is crucial to improve the understanding performance, which is also helpful for fine-grained text-to-image generation (line 936-941).\n\nL3: including understanding data in pre-training stages is crucial for both generation and understanding performance (line 950-955).\n\nL4: the high-quality text-to-image task is more effective than the de facto class-to-image task in PT-1 (line 956-961).\n\nL5: to maintain high performance on generation, we need both PT-1 and PT-2 (line 962-968).\n\nL6: to keep a strong understanding performance, we need at least one of the PT-1 and PT-2 (line 969-974).\n\nL7: using high-quality captions in PT-2 is important for understanding and generation performance (line 975-978).\n\nL8: each our selected data source of PT-2 has meaningful contributions (line 979-981). Future research is encouraged to start from this mixture.\n\n\n**Q3:** It is unclear whether the model weights and curated dataset in this paper will be published. If so, the authors should make a clear statement somewhere in the main text. \n\n**A3:** We will open source the code and the curated datasets, and will add the statement to the main text.\n\n**Q4:** In DPO training, what is the size of preferred samples and rejected samples used in experiments?\n\n**A4:** As stated in line 150/154/159, we collected 8k short, 8k medium and 8k long prompts, respectively. After filtering, we employ around 13k triplets of prompts, perferred images and rejected images. We wil add more details in the final draft.\n\n**Q5:** In L193, the authors say that “RV struggles with free-form or complex instructions, such as those in DPG-Bench [21].” and choose CoT-V as a better option. The description is too limited to demonstrate RV’s drawbacks. Since the Best-of-N strategy can also be combined with RV and achieve similar functionality, the superiority of CoT-V over RV remains unclear. Can the authors give some examples of free-form or complex instructions to better illustrate RV’s limitation and give more details regarding CoT-V’s superiority?\n\n**A5:** For almost all the examples in DPGBench, the prompts are free-form and complicated (see a typical example below). It is impossible to design a general rule that can decompose the prompts into atomic questions.\n\n```\nThree sleek remotes are aligned perfectly next to a simple black picture frame, which encloses a monochrome photograph. These objects rest on the glossy surface of a rich mahogany coffee table. Surrounding them, the soft glow of a nearby lamp illuminates the living room's plush, earth-toned furniture, casting subtle shadows that contribute to a peaceful evening ambiance.\n```\n\n**Q6:** The authors use UniGen-DPO as a self-verifier to produce training data for test-time scaling. The model can generate low-quality or erroneous data, and it seems that no human correction has been applied to this data. How do the authors ensure data quality, and will low-quality data hamper model learning?\n\n**A6:** We kindly remind that when labeling training data, we used a much stronger pseudo labeler Qwen2.5VL-7B, not UniGen-DPO. \nWe empirically find that the data quality is very high. To show this qualitatively, we randomly selected around 700 examples and ask human to do the annotation. \nThen, we compare the model labeled results and human labeled ones. We find that only 77 out of 692 (11%) samples have different results. \nThis indicates that our model labeled data has very good quality.\n\n**Q7:** During training, UniGen first undergoes DPO and then the test-time scaling training. However, it seems that the model could first undergo the test-time scaling stage and then DPO, since training with CoT-V is simply supervised fine-tuning. This also makes sense to first foster the model’s CoT-based reasoning ability and then optimize its preference. What is the reason behind choosing the original order? Will this lead to better model performance? I recommend the authors provide more clarification on this point.\n\n**A7:** The reason we chose the original order is that the test-time scaling is the last stage in our pipeline and putting CoT-V tunning at the end would more likely ensure the model to follow the thinking instruction.\n\nWe agree that it is interesting to try the other order. We flip the order of DPO and CoT-V in the following table. The results show that both strategies work and the original order has slight advantage on GenEval.\n\n|Method| Und. Avg   | GenEval | DPGbench |\n| ----- | ----- | ------- | -------- |\n|DPO->CoT-V| 62.7 | 0.78    | 85.19    |\n|CoT-V->DPO| 62.7 | 0.76  | 85.20    |\n\n**Q8:** typo: L257: stages -> stages.\n\n**A8:** Thanks. We will fix the typo."}, {"review_id": "aPxzjwKVnI", "reviewer": "Reviewer_CnR8", "summary": "The paper presents UniGen, a unified multimodal large language model (MLLM) for image understanding and generation. It proposes a comprehensive training pipeline including pretraining, supervised fine-tuning, and direct preference optimization. A test-time Chain-of-Thought Verification (CoT-V) method is introduced to enhance generation quality through self-verification and Best-of-N selection.", "questions": "1. **Comparison Between Model Scaling and Test-Time Scaling:** I’m curious about the trade-offs between scaling the model size (e.g., training a larger MLLM) versus applying test-time strategies like CoT-V with a smaller model. How do they compare in terms of performance gains and computational overhead during inference?\n\n2. **Open-Source Concern:** Since a major contribution of this work is the unified training pipeline and the use of open-source data (with additional synthetic annotations), there is a concern about whether the authors will release the training code and datasets. Making these resources publicly available would greatly benefit reproducibility and further research.", "limitations": "Yes", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 2, "strengths_and_weaknesses": "Strengths:\n\n1. **Clear Pipeline:** The paper outlines a clear and systematic methodology, covering multi-stage pretraining, supervised fine-tuning, preference optimization, and test-time reasoning. This end-to-end design provides practical insights into constructing unified multimodal large language models.\n\n2.  **Good Empirical Results:** UniGen achieves notable improvements over prior unified MLLMs on multiple benchmarks, including GENEVAL and DPG-BENCH, demonstrating the effectiveness of its design and training strategies.\n\nWeaknesses:\n\n1. **Limited Scale:**  The current model is constrained to 1.5 billion parameters. While the results are competitive, extending the framework to larger model sizes could further strengthen the contribution.\n\n2. **High Test-Time Overhead:** The proposed Chain-of-Thought Verification (CoT-V) strategy involves evaluating multiple candidates per prompt, which may be computationally expensive. A detailed analysis of test-time cost and latency would improve the practicality assessment of the method.", "was_revised": false, "final_justification": "", "rebuttal": "**To Reviewer CnR8**\n\nThanks for your helpful comments. Please see our responses below.\n\n**Q1:** Limited Scale: The current model is constrained to 1.5 billion parameters. While the results are competitive, extending the framework to larger model sizes could further strengthen the contribution.\n\n**A1:** To resolve your concern, we scale up our model to 7B parameters. As shown in the table, we observe improvement on both understanding and generation benchmarks with the larger model size. Note that due to the limited time, we use the DPO and CoT-V data generated by UniGen-1.5B for training UniGen-7B. We expect more increase if we use UniGen-7B for creating its own data. \n\n| Model          | Und Avg.|GenEval | DPG-Bench |\n| -------------- | ------- |------- | --------- |\n| UniGen-1.5B | 62.66| 0.78    | 85.19     |\n| UniGen-7B | 63.78| 0.79    | 85.79     |\n\n**Q2:** High Test-Time Overhead: The proposed Chain-of-Thought Verification (CoT-V) strategy involves evaluating multiple candidates per prompt, which may be computationally expensive. A detailed analysis of test-time cost and latency would improve the practicality assessment of the method.\n\n**A2:** We test the speed of image generation with different inference strategies on the same H100 GPU, covering both the MLLM running time and the tokenizer decoding cost. \nWe compute the averaged speed on all samples of GenEval by feeding our model with one prompt in each batch. \nFor CoT-V, we output N candidate images for each prompt, and perform self-verification by sequentially evaluating each image-prompt pair. \nThen, we select the best image out of N candidates. By default (N=5), the total inference time for picking the best image cost around 27.22s. As shown in the table, we observe improved performance as we using larger N. This aligns with the goal of test-time scaling that is trading compute for accuracy. \n\n| Model          |   N for BoN    | GenEval | DPG-Bench | Speed (s/img) |\n| -------------- | ------- | ------- | --------- | ---- |\n| UniGen-1.5B |   1   |  0.73   |   84.89    |  3.55   | \n| UniGen-1.5B |   3   |  0.77   |       85.13     |  16.59  | \n| UniGen-1.5B |   5   |  0.78   |   85.19    |  27.22  | \n\n**Q3:** Comparison Between Model Scaling and Test-Time Scaling: I’m curious about the trade-offs between scaling the model size (e.g., training a larger MLLM) versus applying test-time strategies like CoT-V with a smaller model. How do they compare in terms of performance gains and computational overhead during inference?\n\n**A3:** We add the experiment of 7B model. As shown in the table below, with N set to 3, CoT-V effectively bridges the gap between 7B and 1.5B model with slightly slower inference speed than the 7B model.\n\n| Model          |   N for BoN    | GenEval | DPG-Bench | Speed (s/img) |\n| -------------- | ------- | ------- | --------- | ---- |\n| UniGen-1.5B |   1   |  0.73   |   84.89    |  3.55   | \n| UniGen-1.5B |   3   |  0.77   |       85.13     |  16.59  | \n| UniGen-1.5B |   5   |  0.78   |   85.19    |  27.22  | \n| UniGen-7B   |   1   |  0.75   |   85.71    |  12.92  |\n\n**Q4:** Open-Source Concern: Since a major contribution of this work is the unified training pipeline and the use of open-source data (with additional synthetic annotations), there is a concern about whether the authors will release the training code and datasets. Making these resources publicly available would greatly benefit reproducibility and further research.\n\n**A4:** We will open source the code, the curated dataset and the script for genearting synthetic annotations."}]

Author rebuttal:

Meta-review: This paper contributes UniGen, a 1.5B sized VLM trained for both image understanding and generation. The model design is based on existing architectures and components, such as MAGViT-v2 for tokenization, MaskGIT schedule for generation, and SigLIP for understanding. At its core sits an autoregressive LLM that operates on tokens and is the main focus of training. The paper also contributes Chain-of-Thought Verification (CoT-V,) an inference-time scaling technique that leverages the model's ability to act as a critic to improve its generations.

Overall, the reviewers are satisfied with the contribution, highlighting its significance, and value for other researchers to continue from.
Reviewers see the value both in showcasing how to train a large VLM and in CoT-V, which makes progress on an important problem. 
A reviewer concern was the lack of novelty over prior approaches, such as Janus, which is insufficiently compared to. In response, the authors argue that their contribution lies in demonstrating how to walk through the entire training process using publicly available data, and CoT-V, which shows how to leverage generation and understanding capabilities in a single model. Another concern is that for CoT-V to work, targeted finetuning is needed (relying on a rule-based verifier for scoring), somewhat limiting the naturalness of the approach. It is argued that this later might be resolved via RL, and the datasets used are general enough to improve another model in this regard as well. Other issues raised by the reviewers were sufficiently addressed by the authors in their response. 

From the reviewer scores, this paper is somewhat borderline. One reviewer, recommending LR did not participate in the rebuttal, which was taken into account. Other reviewers recommend LA, though none of the reviewers is willing to champion the work. In the AC-reviewer discussion a concern was raised that CoT-V is too similar to relevant prior work on BoN sampling for pure image generation. Overall my assessment is that there is enough of a contribution here and I will mark as accept for now.
Decision: Accept (poster)