T2I-R1: Reinforcing Image Generation with Collaborative Semantic-level and Token-level CoT
Authors: Dongzhi Jiang
Ziyu Guo
Renrui Zhang
Zhuofan Zong
Hao Li
Le Zhuo
Shilin Yan
Pheng-Ann Heng
Hongsheng Li
Venue: NeurIPS 2025 poster ()
Keywords: Autoregressive Image Generation; Chain-of-Thought (CoT) Reasoning; Multimodal Large Language Model
Primary area: deep_learning

Abstract: Recent advancements in large language models have demonstrated how chain-of-thought (CoT) and reinforcement learning (RL) can improve performance. However, applying such reasoning strategies to the visual generation domain remains largely unexplored. In this paper, we present **T2I-R1**, a novel reasoning-enhanced text-to-image generation model, powered by RL with a bi-level CoT reasoning process. Specifically, we identify two levels of CoT that can be utilized to enhance different stages of generation: (1) the semantic-level CoT for high-level planning of the prompt and (2) the token-level CoT for low-level pixel processing during patch-by-patch generation. To better coordinate these two levels of CoT, we introduce **BiCoT-GRPO** with an ensemble of generation rewards, which seamlessly optimizes both generated CoTs within the same training step. By applying our reasoning strategies to the baseline model, Janus-Pro, we achieve superior performance with 13% improvement on T2I-CompBench and 19% improvement on the WISE benchmark, even surpassing the state-of-the-art model FLUX.1. All the training code is in the supplementary material and will be made public.

T2I-R1:
Reinforcing Image Generation with Collaborative Semantic-level and Token-level CoT
Dongzii Jiang"), Ziyu Guo",, Renrui Chang-11s; |1=, Zhuofan Zong', Hao Li
Le Zhuo 1.3 Shilin Yan, Pheng-Ann Heng", Hongsheng Li
CUHK MMLab
CUHK IMIXR
Shhanhai Al Laboratory
'CPII under InnoHK
{dzjiang, ziyuguo, renruizhang) hslißee cuhk. edu. hk
www.iklukkkk.k....... edu.hk
Equal Contribution
Project Leader
*Corresponding author
Abstract
Recent advancements in large language models have demonstrated how chain-of- thought (CoT) and reinforcement learning (RL) can improve performance. How- ever, applying such reasoning strategies to the visual generation domain remains largely unexplored. In this paper, we present T2I-R1, a novel reasoning-enhanced text-to-image generation model, powered by RI. with a bi-level CoT reasoning process. Specifically, different stages of generation: (1) the semantic-level CoT for high-level planning of the prompt and (2) the token-level CoT for low-level pixel processing during patch-by-pated generation. To better coordinate these two levels COT, we intro- duce BiCoT-GRPO with ensemble of generation rewards, which seamlessly optimizes both generated CoTs within the same training step. By applying our reasoning strategies the baseline model, Janus-Pro, we achieve superior perfor- mance with 13% improvement on 12l-CompBench and 19% improvement on the WISE benchmark, training code and data are available at https: // ///ggttubbb... Cattuuu.oo///aaaa/////a/
identify two levels of CoT that can be utilized to enhance
surpassing the state-of-the-art model 1LUX.1. All the
Introduction
The emergence of advanced Large Language Models (LLMs) [62, 64, 81, 95], such OpenAI ol [65] and DeepSeek-R1 [20], has demonstrated considerable reasoning capabilities across domains including mathematies [1, 27, 55, 54] and coding [8, 30, 53]. Through reinforcement learning (RL) [72, 73, 52], these models analyze problems progressively with a comprehensive Chain-of- Thought (CoT) [84, 36, 24, 32, 105, 23] before providing answers, significantly enhancing output accuracy.
The CoT reasoning strategies have also been extended to the visual domain. Recent Large Multi-modal Models (LMMs) [7, 59, 103, 92] have adapted the paradigm to accommodate the visual understanding task [51, 105, 32, 22]. These advanced LMMs can jointly process images and their associated textual queries, performing step-by-step analyses of visual details and integrating them with reasoning steps to derive final answers. Concurrently, CoT-like reasoning has been initially investigated in the visual generation task [70, 99, 74, 37, 35, 10, 34, 100], particularly in autoregressive text-to- image generation. The pioneering work, 'Image Generation with Co7" [24], regards the progressive generation of the image tokens as kind of CoT analogous to that of the text tokens, and proposes to optimize this intermediate process enhance the image quality.
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

CoT Image Understanding
Question
Recunotig Preceus
ohe Spms,
dicrse of arsi sight sagens deger of eugh
How obiaiin the fona anzwer?
Jedieaikk tu dedammmmraas and 200C = Powe if
OBC
ummnl (00 on serweitk is
diveriey cA08 300 -Iar
Auwwr.
CoT Image Generation
Semantic-Level Col
Recoriing Proccss
Image Promat
How does the whole image look like!
Baachar with aoaadd fioer o'a boowe masi with yoirs and moteiing atar.
sesll froed with and die Busigrtand appaar lea lusé garden was vilvase flomor and fillgs
ou gool)/ gi
'A Bicc and brown
Semante
survearmnd the baech
v Highslool Planning
Token-level Col
Image Prompt
How does the nest
black and
Local Detalls
Low-loeell Pracsesing
Figure 1: The Illustration of CoT in Image Understand and Generation Tasks. In the image understanding task, the CoT the textual reasoning process. the autoregressive visual generation task, we identify two levels of CoT: the semantic-level and token-level COT. The semantic-level CoT the high-level planning prior to the image generation, the form of text. The token-level CoT is the intermediate mauch-by-patee generation process, focusing on the local pixel details within patch, in the form of image tokens.
Despite these advances, the exploration of CoT for image generation remains preliminary. Unlike image understanding, image generation requires the complex interpretation of cross-modal alignment and the synthesis of fine-grained visual details. To address these challenges, we identify two distinct levels of CoT reasoning that can be leveraged to enhance image generation, as illustrated in Fig. 1:
Semantic-level CoT is the textual reasoning about the image to generate, which is introduced prior to the image generation. The semantic-level CoT' designs the global structure of the image, e.g., the appearance and location of objects. In case the prompt requires reasoning shown in Fig. 2, the semantic-level CoT also helps to deduce the objects to generate. Optimizing the semantic-level CoT could explicitly decouple the planning and reasoning of the prompt from the subsequent image tokens generation, making the generation easier.
Token-level CoT is the intermediate patch-by-patch generation process of the image, as originally introduced [24]. This process could be viewed as a form of CoT as it outputs each subsequent token conditioned on all previous tokens within discrete space, similar to the textual CoT. Unlike semantic-level CoT, token-level CoT focuses on low-level details like pixel generation and maintaining visual coherence between adjacent patches. Optimizing the token-level CoT can enhance both the generation quality and the alignment between prompt and the resulting images.
Despite recognizing these two levels of CoT, critical question remains unaddressed: How can we enhance and coordinate them for text-to-image generation? Current mainstream generative models [76, 79, 70, 37] are trained exclusively on generation targets, lacking the explicit textual understanding required for semantic-leved CoT reasoning. Although introducing a separate model (e.g., an LLM) specifically for prompt interpretation [13] technically feasible, this approach would significantly increase computational costs, complexity, and deployment challenges. Recently, trend has arisen to merge visual understanding and generation within a single model. Building upon LMMs, these unified LMMs (ULMs) [86, 93, 107, 9] could not only understand the visual inputs but also generate images from text prompts. However, their two capabilities are still decoupled, typically pre-trained in two independent stages, with no clear evidence that the understanding capabilities can

---

Prompt
Semantic-level CoT
Token-level CoT
T2I-R1
Janus-Pro
snucture made of fis, mudinionally buil by the Insst people
The image depicts a boqutital and intricate iglee...
Thes aploo has aerudynamic dome slape, showasingg... surface uppxars smooth translacent. The entrance to ulso cousists ofa geatly sloping tunnel.
A speglic type of flower caltivated is the country where Amsientam located
The image portrays
grown
where Aunsterdars located. ranging yellow
The purple..
At image of ciraslar embleen repeoouiing
cirsular eablen depicting the Iulance and of Taiist philocophy features the Vin gybbil prominenity
balovar and harmany
Thoist phnlooophy
The showcases hammonious arrangementt.
Underneath the trinn on the botoommnstt part the platforns lies they pi: with trownngellrsshh mat. The pie curled medting pescelallyy csmounded by the iron structure of the trui tracks and the dimy in plattorn.
a pig the bettone a thale
Figure 2: Visualization of the Image Generation Process of T21-R1. All the prompts need reasoning contain an uncommon scenario. We observe that T2I-R1 successfully deduces the true intention behind the prompt or provides a sensible imagination (highlighted in the text) to produce a satisfying result compared with the baseline model, Janus-Pro.
benefit generation. Given these potentials and issues, we start from a ULM and enhance it to unite both the semantic-level and token-level CoT into one framework for text-to-image generation.
To fulfill our target, we introduce BiCoT-GRPO, an RL method to jointly optimize the two levels of CoT for ULM. We opt for RL instead of supervised fine-tuning (SFT) for two reasons: First, the
ULM has possessed the fundamental ability needed for the semantic-level and token-level CoT: our goal is only to elicit the fusion of these two abilities by guiding the model's ''s self-exploration. Second, RL methods have proven highly effective for enhancing reasoning capabilities, which are essential for both levels of COT.. Specifically, we first instruct the ULM to imagine and plan the image based on the prompt to obtain the semantic-level COT. Then, we feed it into the ULM as the condition for the subsequent image generation for token-level COT. We simultaneously generate multiple images from each prompt and then compute group-relative reward to optimize both levels of CoT within the same iteration. Unlike understanding tasks, where clearly defined rules for rewards exist, image generation lacks such standardized rules. Therefore, we propose to utilize an ensemble of diverse vision experts [90, 82, 49, 24] as reward models. This reward design serves two critical purposes: it evaluates generated images from multiple dimensions to ensure reliable quality assessment, while also functioning as a regularization method prevent the ULM from hacking single reward model.
Through the proposed reasoning strategies, we obtain T2I-R1, the first Geasoning-enhaneee text-to- image model combining the semantic-level and token-level COT. Empirical results show that our approach outperforms baseline models by 13% and 19% improvements on the T2I-CompBench and WISE benchmark, and even surpasses the previous state-of-the-art model FLUX.1. Qualitative analysis reveals that our method empowers the model to generate more human- -lligned results by reasoning about the true intentions behind the prompt and demonstrates enhanced robustness when dealing with uncommon scenarios.
Our contributions are summarized as follows:
1. We identify dual-level reasoning process in the autoregressive image generation task by introducing the semantic-level and token-level CoT, which decouple high-level image planning from low-level pixel generation for more reliable generation.
2. We develop BiCoT-GRPO, both levels of CoT reasoning, seamlessly integrating the understanding capabilities of ULMSs for image generation. For reward modeling, we investigate robust reward system utilizing an ensemble of vision experts.
reinforcement learning framework that jointly optimizes
3. Our resulting model, T21-R1, incorporates both levels of CoT using BiCoT-GRPO and demonstrates significant quantitative and qualitative improvements, surpassing FL.UX.1 across multiple established benchmarks.
3

---

2 Method
2.1 Preliminary
Recently, the employment of reinforcement learning has been the dominant approach to elicit the reasoning capability of the large models. [73] introduces GRPO, enhancing PPO by eliminating the value function and estimating the advantage in group-relative manner. For specific prompt-answer pair (p, a), a group of individual responses {o is sampled from the old policy Tew- Each response is then input to reward function to obtain the individual reward R, Then, the advantage of the i-th response is calculated by normalizing the rewards {R, of the group:
A; R; mean (R.);-1)
stal (R)
GRPO adopts a clipped objective similar to PPO. Besides, KL penalty term between the current policy Ty and the reference model The is directly added in the loss function:
(0) D..a)//,,D...
G lo.1
min (r.,(0)A,, clip (0). E, -,,++) - -BDRL ( [ ]
ΣΣ
Ei=1 |0:1
where (0) is the ratio between the probabilities of Th and Tew for outputting the current token:
Ti.j (0) = To (oi. q, 0r,41)
(2)
To (Oi, 9: Oi,<))
In text reasoning tasks like mathematical problem solving, the model is instructed follow the pre-defined template to output the reasoning process and final answer. The reward functions are rule-based rewards that only check the correctness the final answer and the output format.
2.2 Semantic-level and Token-level CoT
In the autoregressive text generation tasks of LLMs and LMMs, CoT occurs in the textual reasoning format. However, in autoregressive image generation tasks, we identify two distinct types of CoTT that could enhance the image generation at different abstraction levels:
Semantic-level CoT. Semantic-level CoT is defined as the textual reasoning that precedes image generation, serving mirrors human artistic creation: when given a brief prompt, an artist first thinks about the scene construction, considering object attributes, spatial relationships, and interactions. In addition to the planning for common prompts, we also observe the semantic- If the prompt does not directly depict the object to generate, the semantic-level CoT reason about the true intention from the user's prompt. providing more aligned images. As illustrated in Fig. 2, the semantic-level CoT reasons that the flower cultivated in the country where Amsterdam is located tulip. Without this semantic-level CoT, Janus-Pro fails to provide valid results. Additionally, the semantic-level CoT demonstrates importance when handling unusual or potentially ambiguous scenes. In the bottom example of Fig. 2, when given the prompt 'A pig on the bottom of train', semantic- level CoT introduces the action 'lying" for the pig. creating more sensible scenario. In contrast, direct generation without this interpretive imagination creates significant confusion for Janus-Pras Formally, each semantic-level CoT s; is composed of |s text tokens {st,1 81,2 ...,...,, $1,1s;13-
overall semantic planning stage for the intended image. This process
CoT benefits two other scenarios.
Token-level COT. Unique to the image generation task, token-level step- ceep-by-step thinking exists in the image generation process. The generation of image tokens much resembles a chain of thought: the image tokens are generated patch by patch, where the current patch is generated based on the previous ones. We define the sequential generation of image tokens as token-level COT. This process parallels how an artist progressively fills canvas, with the generated patches forming visual reasoning chain. The reasoning content the choice of the specific visual token for each patch, which corresponds to the patch coherence, object appearance, lighting conditions, and other visual details. Note that,

---

Step 1
Image Prompt
Reasoning Instruction
Group Relative Policy Optimization
pink rabéiz right grom landl
"Proside a bris, pricie vinsaizatiin elomeess the pronpt....
Advaniages
painting grond."
Group Computation
Unified LMM
Rewards
Semantic level CuT
Step
"4 piuk nubbet stands to the right of gross band, their cafors couttnn visid costrant Two background polntinng odd artisti flair with varied
Ensemble
Image Prompt
The mubli distme from the bow), baloncing composttion.
la piuk rabbit right...
Human Preference
Object Detector
VQA Model
Output Reward Model
Model
Unified LMM
Token-level CoTl
Decode
<ing start
Figure 3: Framework of BiCoT-GRPO. In step 1, we instruct the model to generate the semantic- level CoT based on the image prompt. In step 2, images are generated conditioned on both the image prompt and semantic-level CoT, with the intermediate generation process serving token-level CoT. The resulting images are evaluated by an ensemble of vision experts to obtain rewards. We generate N images from each prompt to compute the group-relative reward and perform GRPO training.
the reasoning occurs simultaneously with final output generation, rather than separate steps. The model reasons and generates in paraliel, integrating its thinking directly into the creation process. The generated chain patches is later reshaped a 2D grid G RAxuxx and input to an image decoder D to obtain the image. Unlike semantic-level CoT, which addresses global planning, token-level CoT focuses on local details and visual coherence across the image space. Formally, each token-level CoT t; consists of M image tokens {fi,1, 42,,, fi, J. where M represents the resolution of the generated image, i.c.,
X w.
2.3 BiCoT-GRPO
GRPO has been proven to be highly effective for exploring the reasoning capability of the LLMs and LMMs. To accommodate both semantic-level and token-level CoT in image generation, propose BiCoT-GRPO, where the model reasons twice a single generation process. We instruct the model first perform semantic-level CoT for global planning, and then dive into the local details by performing token-level COT.
However, compared with the task of text generation, great pipeline challenge is posed for incorpo- rating two levels CoT for image generation. Limited by the training paradigm, most current ULMSs cannot generate interleaved images and text themselves. A manual signifier is often needed to instruct the model on which task to perform, either text generation or image generation. For Jamus- Pro to generate image, which is the ULM use in this work, need to manually concatenate image start token (<img_start>> to explicitly instruct the model to start generating image tokens.
To tackle this problem, we propose novel pipeline to facilitate ULM in generating images with two levels of CoT, as shown in Fig. Specifically, our pipeline composed of a two-step generation process. The first step is to generate the semantic-level COT. We input the image prompt and instruct the model to imagine and reason about the details of the image to generate semantic-level CoT {s, )x,)__- The second stage focuses on the token-level CoT generation. We input the image prompt, the generated semantic-level CoT in the first stage, and the image start token to the UL.M for generating image tokens {t Then, the image tokens input to the image decoder to obtain the
5

---

Image
Prompt
Human Preference Model
Prompt Alignment?
Reward:
0.27
& Image
Aesthetic Appeal?
Object
Object Detector
Object Existence?
Reward:
Prompt
& Image
Spatial Relotion?
1.00
blue butterfly the left of green candle," "
Object w/ Attribute
Hlggh "les" Proh
GPT-4o mini
Blue Butterffy?
Reward:
VQA Model
& Image
Green Candle?
"Yes"
0.6
Object
Butterfly
Candle
Object w/ Attribute
Prompt
Output Reward Model
Reward:
Green Candle
Bloe Butterfly
& Image
Prompt Prompt Alignment?
Prah
0.5
Figure 4: Illustration of the Ensemble of Generation Rewards. We use GPT-4o mini to extract objects and their attributes before training. Each specialized reward model receives customized information inputs for the reward calculation. We take the average of all the rewards as final reward.
image I. Since there exist two types of CoT our method, first the semantic-level CoT and then the teken-level COT. Each response 0; composed of two parts, namely o (8i, 4.)- In this sense, the ri (0) is converted to:
no (ol 19.80.22) Thau (as (s)
S
Ta.j(()
To (Oi.j q. 0s,<,)
mo(te (essstsses) Tuau (45 (o:seff
|s;| < S
|s;
+ M
Then, update the ULM by maximizing Equation 2.1. In practice, we incorporate the token-level policy gradient loss in [101], where the loss term normalized over all generated tokens to balance the reward on overly long semantic-level COT.
2.4 Ensemble of Generation Rewards
Unlike DeepSeek-RI with the rule-based reward, assessing the images based on pre-defined rules infeasible. The assessment of the image includes various aspects, including the aesthetic appeal and objects' existence, attributes, and relationships. Considering the complexity, we introduce an ensemble of vision experts to judge the generated image from multiple aspects. Meanwhile, the use of multiple reward functions also serves as regularization method to prevent the ULM from hacking into a specific reward model. As shown in Fig. the ensemble contains the following experts:
Human Preference Model. Human preference models (HPMs), such HPS [90] and ImageRe- ward [94], are trained to simulate human aesthetic preferences. These models are developed using datasets of human rankings synthetic images, where annotators evaluate and compare generated outputs. During inference, these models assess both the aesthetic quality and prompt alignment of generated image, producing a composite human preference score RuPM- This expert provides holistic reward signal from general perspective.
Object Detector. Another option of the reward model an object detector, e.g., Ground- ingDINO [49] and YOLO-world [12]. These open-vocabulary detection models accept an image along with object queries as input and output both the spatial positions and confidence scores for detected objects. This kind of vision expert serves ideal tool to evaluate the object's existence

---

Janus-Pu Pro
Only
Oale
Only
Only
The plarehene ueate of fosser
te lanes
Pejing
Genner as deugr
godginn in fuilles
kndiaa day misk Airinger hilee
Apport dal
fonmaamu grsholing
buad
und lork 185 cnaay aod permppplty
costy Niw Tod
Figure 5: Visualization Results. We provide the image generation results of the same prompt from four models: base model, the model with only semantic-level CoT optimized, the model with only token-level CoT optimized, and the model with both levels of CoT optimized.
and relationship concerning space and numbers. For implementation, we extract all objects {obj, i=1 from the training image prompts, where represents the total number of objects. We then query the object detector identify these objects within the generated image. For each object, assign a binary existence score (1 detected, 0 otherwise) and average these scores across all objects in the prompt. If the prompt contains a spatial relationship, we further leverage the detected location validate its correctness. We calculate the relative distance and intersection over union (IoU) between the objects for the spatial score Rypatill If the number of the object Uobj; is specifically pointed out in the prompt, we compare the number with the detected number of the object Robj, The reward from the object detector Re is determined as:
R. + (1 - ax) E l(obj; detected), spatial relationship in the prompt, + i=1 [[Mobi Noob)))
RDet
if number in the prompt, else.
[[obj, detected),
where Ryputial is the relative distance between the objects is larger than a threshold and the direction is right. If the direction is wrong, the reward is O. Otherwise, reward. We set a as 0.6 to encourage the correctness the spatial relationship.
the IoU as the spatial
Visual Question Answering Model. The visual question answering (VQA) models are trained to answer questions based on the image input. The VQA models include earlier models prior to LLM, e.g., BLIP [42] and GIT [82], and LMMs like LLaVA [47]. We leverage these models to judge the existence and attributes of the objects. For example, if the image prompt is red dog and yellow cat, we first reformat each individual object with its attribute as a question to the VQA model, i.e., red dog? and yellow cat?. Then, we record the probability for the model to answer Yes as Pi and
No as P The reward for a prompt is calculated as: RvOA = K Σ +7%
Output Reward Model. Lastly, we also employ the output reward model (ORM) proposed in [24] reward model. The ORM is fine-tuned from an LMM (c.g.. LLaVA-One Vision [39]) specifically for evaluating the alignment between the prompt and the image. The fine- sine-tunigg is instruct the model to output Yes if the image perfectly aligns with the image and No otherwise. We calculate RORM using the methodology similar to RvqA- except that we input the whole image prompt to the ORM instead of reformatting the prompt. The major difference between the ORM and HPMs is that the ORM model incorporates extensive world knowledge inside the LMM while HPMs mainly focus on the human preferences including prompt-image alignment and aesthetic appeal.
We can choose one or multiple reward functions illustrated above, and take the average as the final reward for specific sample. The detailed experiments of reward model are shown in Table 3.
7

---

Table 1: T21-CompBench Result. The best scoré in blue, with the second-best score in green.
Model
Attribute Binding
Object Relationship
Complext
Color 1
Shapet
Texturet
Spatialt
Non-Spatialf
Diffusion Models
SD-v1.5 [70]
0.3758
0.3713
0.4186
0.1165
0.3112
0.3047
PixArt-a [6]
0.6690
0.4927
0.6477
0.2064
0.3197
0.3433
CoMat [31]
0.7827
0.5329
0.6468
0.2428
0.3187
0.3680
ID-XL-bascc-L0 [66]
0.5879
0.4687
0.5299
0.2131
0.3119
0,3237
FLUX.1 [37]
0.7407
0.5718
0.6922
0.2863
0.3127
0.3703
AntoRegressive Models
Show-o [93]
0.5623
0.4178
0.4641
0.2015
0.3067
0.2992
Show-o PARM [24]
0.7549
0,5632
0.6684
0.2971
0.3126
0.3701
EMU3 [83]
0.7544
0.5706
0.7164
Monus-Pro-7BB [9] (Baseline)
0.6359
0.3528
0.4936
0.2061
0.3085
0.3559
T2I-RI (Ours)
0.8130
0.5852
0.7243
0.3378
0.3090
0.3993
3
Experiment
In this section, we first provide the main results of T21-R1 in T21-CompBench [28], WISE [61] and GenAl-Bence [45] in Section 3.1. Then we present the results of different reward function combinations in Section 3.2 and the ablation study of the effectiveness of two levels of CoT in Section 3.3. Please refer to the Appendix for TIIF-Bench [85] results, detailed experiment setup, and more visualizations.
3.1 Main Results
We compare T2I-R1 with leading text-to-image diffusion and autoregressive models the T2I- CompBench and WISE benchmarks (in Table 1, and 4). We also provide the qualitative results in Fig. 5. Our method demonstrates substantial improvements enhancements of 13% and 19% on 121-CompBench and WISE, respectively. On 121-CompBench, the most significant gains appear in attribute binding, with average improvement of 19%. For the WISE benchmark, improvements are more evenly distributed across categories. When compared to the more powerful state-of-the-art diffusion models, T21-RI achieves superior or comparable results across both benchmarks. Notably, on T2l-Chmpeench, our method leads in five of six subtasks, with exceptional performance in the spatial subtask (0.3378), surpassing previous SOTA results by over 5%. Similarly, for WISE, T2I-R1 excels in four of seven subtasks and achieves the highest overall score of 0.54, outperforming the robust FLUX.1-dev by 4%. Remarkably, our approach consistently achieves the leading results across all subtasks in both benchmarks when compared to other autoregressive models. Remarkably, the improvement on T21-Comphench benefits from the planning ability brought by the semantic-level COT, which designs the complex scenarios before generation. While the enhancement of WISE due to the reasoning capability from the semantic- level CoT, which deduces the true object place depicted behind the prompt. For GenAl-Bench, T2I-RI largely improves the baseline model, and in the meantime, achieves the highest overall score on both the basic and advanced prompts. Again, T2I-R1 surpasses FLUX.1 [37] in both types of prompts and showcases remarkable margin the advanced prompt, probably attributed to the high-level reasoning capability granted by semantic-level CoT.
the baseline model, with average
3.2 Reward Analysis
In this section, we experiment with the choice of reward functions and their combinations. We hope to provide some insights into how to choose the reward functions and combine them. Our results are shown in Table 3. We first experiment with the individual reward model. HPM (H) demonstrates superior performance in attribute binding but shows limited effectiveness in object relationships, likely due to its weak relation comprehension capabilities. The object detector (D) yields the least improvement in attribute binding, which aligns with expectations since our detector- reward functions do not explicitly evaluate attributes. The improvements observed stem solely from enhanced object existence ratios in the prompts. We observe that VQA model (V) and ORM (0)

---

Table 2: WISE Result. The best score is in blue, with the second-best score in green
Spatio-Tempora
Natural Science
Model
Cultural
Overall
Time
Spacef
Biology
Physiest
Chemistryf
Diffusion Models
PixArt-Alpha [6]
0.45
0.50
0.48
0.49
0,56
0.34
0.47
Playground-ve -2.5 [40]
0.49
0.58
0.55
0.43
0.48
0.33
0.49
SD-vI-5 [70]
0.34
0.35
0.32
0.28
0.29
0.21
0.32
SD-XL-base-009 [66]
0l43
0.48
0.47
0.44
0.45
0.27
0.43
FLUX. [37]
0.48
0.58
0.62
0.42
0.51
0.35
0.50
AstoRegressive Models
Emu3 [83]
0.34
0.45
0.48
0.41
0.45
0.27
0.39
Show-o [93]
0.28
0.40
0.48
0.30
0.46
0.30
0.35
VILA-U [91]
0.26
0.33
0.37
0.35
0.39
0.23
0.31
Canus-Pro-7B [9] (Baseline)
0.30
0.37
0.49
0.36
0.42
0.26
0.35
T21-RI (Ours)
0.56
0.55
0.63
0.54
0.55
0.30
0.54
Table 3: 121-CompBench Results with Different Reward Models. 'Det" stands for object detector.
Model
Reward Model
Attribute Binding
Object Relationship
Complest
Visual Quality!
HPM
Det
VQA
ORM
Colur
Shape1
Texture
Spatial!
Non-Sputial?
Pro-7B
0.6359
0.3528
0.4936
0.3085
0.3559
0.8134
0.5048
0.7311
0.2383
0.3012
0.3899
0.7422
0.5140
0.6494
0.3044
0,3100
0.3872
V
0.7819
0.5638
0,7010
0.3301
0,3103
0.3959
1.775
0.8210
0.6074
0,7440
0.3189
0.3076
0.4005
1.942
T2I-RI
0.8130
0.5852
0.7243
0.3378
0.3090
0.3993
2.063
V
0.7599
0,5742
0.8902
0.2796
0.3070
0.3921
both effective reward models with distinet strengths: the VQA model excels at improving attribute binding, while ORM demonstrates superior performance in relationships. Then experiment with multiple reward models. We start from the composition of HPM and object detector (H + D), and progressively incorporate other reward models. Our findings indicate that both the HPM-object detector combination (H + D) and the three-mode integration (H + D V) deliver balanced and satisfactory results in both attribute and relationship tasks. To obtain the optimal choice of reward models, we conduct a human study to evaluate the visual quality, detailed in Appendix C.2. We adopt the combination of the highest visual quality, the ensemble of three reward models (H + D + V) for our final model.
HPM, object detector, and VQA
Token-level CoT Only
Semantic-level Token-level CoT
black squirrel and brown nut
Figure 6: Visualization Result the Image Diversity of a Single Prompt. We showcase the result of only token-level CoT optimized and both semantic-level and token-level CoT optimized.
3.3 Ablation Study
To validate the effectiveness of the semantic-level CoT, we compare T21-R1 with a baseline method that generates images using only the token-level CoT optimized with the GRPO method. This is the default text-to-image generation setting in Janus, whose result is shown the third row Table Comparing the third and fourth row in the table, we find that semantic-level CoT generally brings performance improvements across both benchmarks tested. We witness a particularly significant gain the WISE benchmark. This enhanced performance can be attributed to the textual reasoning

---

Table 4: GenAI-Bench Evaluation Results. The best score is in blue with the second best score in green
Basic Prompt
Advanced Prompt
Method
Attribtet
Scene?
Relution Action
Overall!
Count?
Dütfert
Compare?
Logical
Overall!
Spriill
Part
Negate
Universlf
Piffucson Madele
SD 12.1100
0.50
0.79
0.76
0.77
0.50
0.78
0.68
0.70
0.68
0.54
0.54
0.62
SD-XL [66]
0.89
0.73
0.60
Midjourae vé [60)
0.88
0.87
0.87
0.87
0.91
0.87
0.78
0.78
0.79
0.50
0.76
0.69
BLUX 1-ber (87)
Regresslve
1WM [16]
0.63
0.62
0.65
0.63
0.70
0.63
0.59
0.58
0.54
0.69
0.53
0.53
Show-o [03]
0.72
0.72
0.70
0.70
0.75
0.70
0.70
0.62
0.71
0.51
0.65
0.60
ViI [-11
0.78
0.78
0.77
0.78
0.79
0.76
0.70
0.71
0.74
0.53
0.56
0.64
Liquid [87]
0.73
UniTek [56]
0.76
0.76
0.79
0.45
0.73
0.67
Mogao 7B [44]]
0.77
0.74
0.77
0.53
0.71
0.65
Janus Pro 7B ( (Daseline)
0.85
0.87
0.85
0.84
0.55
0.84
0.73
0.73
0.71
0.45
0.50
0.65
T21-R1 (Ours)
Table 5: Ablation Experiments on the Effectiveness of the Two Levels of CoT.
Model
Optimized CoT
T2I-CompBench
WISE
Diversity?
Semantic-level
Token-level
Color?
Shape?
Texture
Culture?
Spatio-Temporal
Sciencef
Janus-Pro-
0.6359
0.3528
0.4936
0.3000
0,4232
0.3467
6.976
0,8082
0.5684
0.7219
0.5900
0,5599
04367
8.177
0.7752
0.5849
0.7451
0.3500
0.4732
0.3900
6.255
T2-R1
0.5852
0.7243
0.5600
0,5855
0.4633
8,203
capabilities inherent in semantic-level COT. As illustrated in Fig. 5, our method could first clearly reason about the objects or phenomena described in the prompt through semantic-level COT. This effectively decouples the reasoning and generation processes and thereby facilitates superior results. We also observe that training solely with token-level CoT substantially reduces the diversity of generated images, demonstrated in Fig, 6, 7, 13, and 14. To quantify this effect, we evaluate image diversity by reusing the generated images from Comppnnnenn,, where each prompt generates ten images. We compute the Vendi Score [18] across the ten images for each prompt. Results indicate that GRPO training without semantic-level CoT decreases the diversity score, whereas incorporating semantic-level CoT significantly improves diversity through varied textual planning.
We also consider another situation validate the effectiveness of token-level CoT: the semantic-level CoT is incorporated in the image generation process, as T2I-R1, but GRPO only optimizes the semantic-level CoT without the token-level COT. This can be viewed as only enhancing the model's high-level planning capabilities. The second row of Table presents the result. The results show that optimizing semantic-leve CoT exclusively yields smaller improvements compared to the joint optimization approach. Additionally, much better aesthetic quality compared with optimizing semantic-leve CoT only, as shown in Fig. 5. This indicates the necessity to jointly optimize both levels of COT.
find that optimizing both CoT types produces images with
Conclusion
In this paper. we introduce T21-R1, the first reasoning-enhanceed text-to-image model powered by a bi-level CoT reasoning process. We identify the semantic-level CoT for high-level planning and the token-level CoT for patch-by-pater generation. We further integrate them through our proposed BiCoT-GRPO, an RL framework incorporating two levels of CoT within the same training step. By leveraging ULM capable of both visual understanding and generation, our approach eliminates the need for separate specialized models while achieving significant performance improvements, +1 +133% on T21-CompBench and +19% on the WISE benchmark. Our qualitative analysis demonstrates that T21-R1 better understands complex prompts, reasons about user intentions, and handles uncommon scenarios with greater robustness, establishing paradigm for text-to-image generation tasks.
10

---

Acknowledgments
This study was supported in part by National Key R&D Program of China Project 20222D0161 100, in part by the Centre for Perceptual and Interactive Intelligence, a CUHK-led InnoCentre under the InnoHK initiative of the Innovation and Technology Commission of the Hong Kong Special Administrative Region Government, in part by NSFC-RGC Project N CUHK499/24, and in part by Guangdong Basic and Applied Basic Research Foundation (No. 10231111130000 XW).
Additional support was provided by the Research Grants Council of the Hong Kong Special Adminis- trative Region, China, under Project 14201321 and Project 14200824.
References
[1] Amini, A., Gabriel, S., Lin, P., Koncel-Kedziorski, R., Choi, Y., Hajishirzi, H.: Mathqa: Towards interpretable math word problem solving with operation formalisms. arXiv preprint arXiv:1905. 13319 (2019)
[2] Austin, J., Odena, A., Nye, M.I., Bosma, M., Michalewski, H., Dohan, D., Jiang, E., Cai, C.J., Terry, M., Lc, Q.V., Sutton, C.: Program synthesis with large language models. CoRR abs/2108.07732 (2021)
[3] Bai, S., Chen, K., Liu, X., Wang. J., Ge, W., Song, S., Dang, K., Wang, P., Wang. S., Tang, J., Zhong, H., Zhu, Y. Yang, M.. Li, Z., Wan, J.. Wang, P., Ding. W., Fu, Z., Xu, Y., Ye, J., Zhang. X.. Xie, T., Cheng, Z., Zhang, H.. Yang, Z., Xu, H., Lin, J.: Qwen2.5-vl technical report. arXiv preprint arXiv:2502. 13923 (2025)
[4] Bar-Tal, O., Chefer, H., Tov, O., Herrmann, C., Paiss, R., Zada, S., Ephrat, A., Hur, J., Liu, G., Raj. A.. et al:: Lumiere: A space-time diffusion model for video generation. In: SIGGRAPH Asia 2024 Conference Papers. pp. 1-11 (2024)
[5] Chen, J., Xue, L., Shu, M.. Zhang, Y., Zhang, Y., Xu, R., Wang, X.E., Xiong, C.: Blip3-o: A family of fully open unified multimodal models architearc training and dataset. arXiv preprint mtXiv:2505.09566 (2025)
[6] Chen, J., Yu, J., Ge, C., Yao, L., Xie, E., Wu, Y., Wang, Z., Kwok, J., Luo, P., Lu, H., Li, Z.: Pixart-o: Fast training of diffusion transformer for photorealistic cext-to-image synthesis (2023)
[7] Chen, L., Li, L., Zhao, H., Song, Y., Vinci: R1-v: Reinforeing super generalization ability in vision-language models with less than S3, https: //gi:::////ttt com/Deep Dewp///eepppgg////eeee /Rint (2025), accessed: 2025-02-02
[8] Chen, M., Tworek, J., Jun, H., Yuan, Q., de Oliveira Pinto, H.P., Kaplan, J., Edwards, H., Burda, Y., Joseph, N., Brockman, G., Ray, A., Puri, R., Krueger, G., Petrov, M., Khlaaf, H., Sastry, G.. Mishkin, P., Chan, B., Gray, S., Ryder, N., Pavlov, M., Power, A., Kaiser, L., Bavarian,
M. Winter, C., Tillet, P., Such, F.P., Cummings, D., Plappert, M., Chantzis, F., Barnes, E., Herbert- Voss, A., Guss, W.H., Nichol, A., Paino, A., Tezak, N., Tang, J., Babuschkin, I., Balaji, S., Jain, S., Saunders, W., Hesse, C., Carr, A.N., Leike, J., Achiam, J., Misra, V., Morikawa, E., Radford, A., Knight, M., Brundage, M., Murati, M., Mayer, K., Welinder, P., McGrew, B., Amodei, D., McCandlish, S., Sutskever, I., Zaremba, W.: Evaluating large language models trained on code. CoRR abs/2107.03374 (2021)
[9] Chen, X., WI, Z.,, Liu, X., Pan, 7., Liu, W., Xie, 7.,, Yu, X., Ruan, C.: Janus-pro: Uni- fied multimodal understanding and generation with data and model scaling. arXiv preprint 1XXiv2250117811 (2025)
[10] Chen, X., Ge, J., Zhang, T., Liu, J., Zhang, S.: Learning from mistakes: Iterative prompt relabeling for text-to-image diffusion model training, arXiv preprint arXiv:2312.1622 (2023)
[11] Chen, X., Zhang, R., Jiang, D., Zhou, A., Yan, S., Lin, W., Li, H.: Mint-cot: En- abling interleaved visual tokens mathematical chain-of-thought reasoning. arXiv preprint rxXiv:2506.05331 (2025)

---

[12] Cheng, T., Song, L., Ge, Y., Liu, w., Wang, X., Shan, Y: Yolo-world: Real-time open- vocabulary object detection. In: Proceedings the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 16901-16911 (2024)
[13] Datta, S., Ku, A., Ramachandran, D.. Anderson, P.: Prompt expansion for adaptive text- to-image generation. In: Ku, L.W., Martins, A., Srikumar, V. (eds.) Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). pp. 3449-3476. Association for Computational Linguistics, Bangkok, Thailand (Aug 2024). htps:://oi.orr// http:///oooorro..... wttpo/orrrr11..5555...1 www.oo/o.......onoooooo 189, httpps//acclaat /atp//holaaattoooo...oo.oooo..a. acl acl-long. 189/
[14] Deng, Y., Bansal, H., Yin, F., Peng, N., Wang, W., Chang, K. W.: Openvithinker: An early exploration to complex vision-language reasoning via iterative self-improvement (2025), attps://arxiva wwwp:/orr//////aaaeeeeeee 17352
[15] Dong, R., Han, C., Peng, Y., Qi, Z., Ge, Z., Yang, J., Zhao, L., Sun, J., Zhou, H., Wei, H., et al.: Dreamllm: Synergistic multimodal comprehension and creation. arXiv preprint arXiv:2309. 1199 (2023)
[16] Esser, P., Rombach, R., Ommer, B.: Taming transformers for high-resolution image synthesis. In: Proceedings of the IBEE/CVT conference computer vision and pattern recognition. pp. 12873-12883 (2021)
[17] Fang, R., Duan, C., Wang, K., Li, H., Tian, H., Zeng. X., Zhao, R., Dai, J., Li, H., Liu, X.: Puma: Empowering unified milm with multi-granular visual generation. arXiv preprint arXiv:2410.1386 (2024)
[18] Friedman, D., Dieng, A.B.: The vendi score: A diversity evaluation metric for machine learning. arXiv preprint arXiv:2210.0241 (2022)
[19] Gemini Team, G.: Gemini: a family of highly capable multimodal models. arXiv preprint arXiv:2312.1 1805 (2023)
[20] Guo, D., Yang, D., Zhang, H., Song, J., Zhang, R., Xu, R., Zhu, Q. Ma. S., Wang, P., Bi, X., et al.: Deepseek-rl: Incentivizing reasoning capability in ilms via reinforcement learning. arXiv preprint arXiv:2501. 12948 (2025)
[21] Guo, Z., Lin, H., Yuan, Z., Zheng, C., Qiu, P., Jiang, D., Zhang, R., Feng, C.M., Li, Z.: Pisa: A self-augmented data engine and training strategy for 3d understanding with large models. arXiv preprint arXiv:2503. 10529 (2025)
[22] Guo, Z., Chen, X., Zhang, R., An, R., Qi, Y., Jiang, D., Li, X., Zhang, M., Li, H., Heng, P.A.: Are video models ready as zero-shot reasoners? empirical study with the mme-cof benchmark. arXiv preprint arXiv:2510. 268002 (2025)
[23] Guo, Z., Zhang, R., Chen, H., Gao, J., Jiang, D., Wang. J., Heng, P.A.: Sciverse: Unveiling the knowledge comprehension and visual reasoning of Imms on multi-modal scientific problems. arXiv preprint arXiv:2503. 10627 (2025)
[24] Guo, Z., Zhang, R., Tong, C., Zhao, Z.,, Gao, P., Li, H., Heng, P.A.: Can we generate images with cot? let's verify and reinforce image generation step by step. arXiv preprint 1rXiv:2501.13926 (2025)
[25] Han, J., Liu, J., Jiang, Y., Yan, B., Zhang, Y., Yuan, Z., Peng, B., Liu, X.: Infinity: Scaling bitwise autoregressive modeling for high-resolution image synthesis (2024)
[26] He, J., Li, H., Hu, Y., Shen, G., Cai, Y., Qiu, W., Chen, Y.C.: Disenvisioner: Disentangled and enriched visual prompt for customized image generation. arXiv preprint 100.022410.02067 (2024)
[27] Hendrycks, D., Burns, C., Kadavath, S., Arora, A., Basart, S., Tang, E., Song, D., Steinhardt, J.: Mcasuring mathematical problem solving with the math dataset. NeurlPS (2021)
12

---

[28] Huang, K., Sun, K., Xie, E., Li, Z., Liu. X.: T2i-compbench: A comprehensive benchmark for open-world compositional eoxt-toimage generation. Advances in Neural Information Processing Systems 36, 78723-78747 (2023)
[29] Huang, W., Jia, B., Zhai, Z., Cao, S., Ye, Z., Zhao, F., Hu, Y., Lin, S.: Vision-rl: Incentivizing reasoning capability in multimodal large language models. arXiv preprint arXiv:2503.06749 (2025)
[30] Jain, N., Han, K., Gu, A., Li, W., Yan, F., Zhang, T., Wang, S., Solar-Lezama, A., Sen, K., Sto- ica, I: Livecodebench: Holistic and contamination free evaluation of large language models for code. CoRR abs/2403.07974 (2024), https: //wii org/10. hwwgg/...8550000 arxiv. 2403. 07974
[31] Jiang, D., Song, G., Wu, X., Zhang, R., Shen, D.. Zong, Z., Liu, Y., Li, H.: Comat: Aligning text-to-image diffusion model with image-to-text concept matching. arXiv preprint arXiv:240400665 (2024)
[32] Jiang, D., Zhang, R., Guo, Z., Li, Y., Qi, Y., Chen, X., Wang, L., Jin, J., Guo, C.,, Yan, S., et al.: Mme-cot: Benchmarking chain-of-thought in large multimodal models for reasoning quality, robustness, and efficiency, arXiv preprint arXiv:2502.0962 (2025)
[33] Jiang, D., Zhang, R., Guo, Z., Wu, Y., Lei, J., Qiu, P., Lu, P., Chen, Z., Song, G., Gao, P., et al.: Mmsearch: Benchmarking the potential of large models multi-modal search engines, arXiv preprint arXiv:2409. 12959 (2024)
[34] Jiang, D., Zhang, R., Li, H., Zong, Z., Guo, Z., He, J., Guo, C., Ye, J., Fang, R., Li, W., et al.: Draco: Draft cot for text-to-image preview and rare concept generation. arXiv preprint arXiv:2512.055 (2025)
[35] Jiang, D., Zhang. R., Shu, Y., Peng. Y., Zong, Z., Duan, Y., Wang. Z., Liu, J., Chen, H., Guo, Z., Ye, J., Liu, R., Heng, P.A., Zhang, S., Li, H.: Ulmevalkit: An open- toolkit for evaluating unified large multi-modal models and generative models (October 2025), https: //github. com/ UULEEalEEa--- AtLMAaaaa/MMMMaaai
[36] Kojima, T., Gu, S.S., Reid, M., Matsuo, Y., Iwasawa, Y.: Large language models are zero-shot reasoners. Advances neural information processing systems 35, 22199-22213 (2022)
[37] Labs, B.F. Flux. https::///// //gt thub. com
(2024)
[38] Lei, J., Zhang, R., Hu, X., Lin, W., Li, Z., Sun, W., Du, R., Zhuo, L., Li, Z., Li, X., al.: Imagine-e: Image generation intelligence evaluation of state-of-the-art text-to-image models. (2025)
arXiv preprint arXiv:2501.1399
[39] Li, B., Zhang, Y., Guo, D., Zhang, R., Li, F., Zhang, H., Zhang, K., Li, Liu, Z., Li, C.: Llava-onevision: Easy visual task transfer. arXiv preprint arXiv:2408.03322 (2024)
[40] Li, D., Kamko, A., Akhgari, E., Sabet, A., Xu, L., Doshi, S.: Playground v2. 5: Three insights towards enhancing aesthetic quality in text-to-image generation. arXiv preprint arXiv:2402.17245 (2024)
[41] Li, H., Tian, C., Shao, J., Zhu, X., Wang, Z., Zhu, J., Dou, W., Wang, X., Li, H., Lu, L., et al.: Synergen-vl: Towards synergistic image understanding and generation with vision experts and token folding, arXiv preprint arXiv:2412.096 (2024)
[42] Li, J., Li. D., Xiong, C., Hoi. S.: Blip: Bootstrapping language-iman pre-training for unified langunelanguage understanding and generation. In: International Conference on Machine Learning, pp. 12888-122 18888-12900. PMLR (2022)
[43] Li, Y., Zhang, Y., Wang, C., Zhong, 7., Chen, Y., Chu, R., Liu, S., Jia, J.: Mini-gemini: Mining the potential of multi-modality vision language models. arXiv: 2403. 18814 (2024)
[44] Liao, C., Liu, L.,, Wang, X., Luo. Z., Zhang, X., Zhao, W., Wu, J., Li, L., Tian, Z., Huang, W.: Mogao: omni foundation model for interleaved multi-modal generation. arXiv preprint arXiv:2505.05472 (2025)
13

---

[45] Lin, Z., Pathak, D., Li. B., Li, J., Xia, X., Neubig, G., Zhang, P., Ramanan, D.; Evaluating text-to-visual generation with image-to-tex generation. In: European Conference on Computer Vision. pp. 366-384. Springer (2024)
[46] Liu, H., Yan, W., Zaharia, M., Abbeel, P.: World model on million-length video and language with ringattention. arXiv e-prints pp. arXiv 2402 (2024)
[47] Liu, H., Li, C., Wu, Q., Lee, YJ..: Visual instruction tuning. In: NeurlPS (2023)
[48] Liu, J., Chen, H., An, P., Liu, Z., Zhang, R., Gu. Li, X., Guo, Z., Chen, S., Liu, M., et al.: Hybridvla: Collaborative diffusion and autoregression in unified ussio-languagg-acttin (2025)
model. arXiv preprint arXiv:2503.
[49] Liu, S., Zeng, Z., Ren, T., Li, F., Zhang, H., Yang. J. yue Li, C., Yang, J., Su, H., Zhu, J.J., Zhang. L.: Grounding dino: Marrying dino with grounded pre-training for open-ser ppen-set object detection. ArXiv abs/2303.05499 (2023)
[50] Liu, Y., Peng, B., Zhong, Z., Yue, Z., Lu, F., Yu, B., Jia, J.: Seg-zero: Beasoning-chall guided segmentation via cognitive reinforcement. arXiv preprint arXiv:2503.0652) (2025)
[51] Lu, P., Bansal, H., Xia, T., Liu, J., yue Li, C., Hajishirzi, H., Cheng, H., Chang, K.W., Galley, M., Gao, J.: Mathvista: Evaluating math reasoning in visual contexts with gpt-4v, bard, and other large multimodal models. ArXiv abs/2310.02255 (2023)
[52] Lu, Z., Ren, H., Yang, Y., Wang, K.. Zong. Z., Pan, J., Zhan, M., Li, H.: Webgen- agent: En- hancing interactive website generation with multi-level feedback and step-level reinforcement learning (2025), https: : //trxivavvvv///////////////aaaal org/ abs ablaabaa2/2// 2509 22644
[53] Lu, Z., Yang. Y., Ren, H., Hou, H., Xiao, H., Wang, K., Shi, W., Zhou, A., Zhan, M., Li, H.: Webgen-bench: Evaluating ilms on generating interactive and functional websites from scratch (2025), https: //trxxaxvvv///////////eee/eaaaa/ org/aagg/o wwg/abs////eeoaoooeeo.goooaaaeea 2505 03733
[54] Lu, Z., Zhou, A., Ren, H., Wang. K., Shi, W., Pan, J., Zhan, M., Li, H.: Mathgenie: Generating synthetic data with question back-translation for enhancing mathematical reasoning 1lms (2024), https: : //arxi/ax.i..////a org/ abs 2202 16352
[55] Lu, Z., Zhou, A., Wang, K., Ren, H., Shi, W., Pan, J., Zhan, M., Li, H.: Mathcoder2: Better math reasoning from continued pretraining on model-translated mathematical code (2024), https://arxis
orgoor////oo barg/abs//ae /aab///2400 200.s/0110011011666 08196
[56] Ma, C., Jiang, Y., Wu, J., Yang, J., Yu, X., Yuan, Z.,, Peng, B., Qi, X.: Unitok: A unified tokenizer for visual generation and understanding. arXiv preprint arXiv:2502.20321 (2025)
[57] Ma, Y., Liu, X., Chen, X., Liu, W., Wu, C., Wu, Z., Pan, Z., Xie, Z., Zhang, H., Zhao, L., et al.: Janusflow: Harmonizing autoregression and rectified flow for unified multimodal understanding and generation. arXiv preprint Ttt::2411.09977 (2024)
[58] MAA: American invitational mathematics examination aime. In: American Invita- tional Mathematics Examination AIME 2024 (February 2024), https://nat org/ math competitions iemeeetiiiiinnieeem ameri can invitatiiiii teaittatinlllett.. mataaallmaaaaaiiiicmmmmmmmm www.miaaaaaiiieeeeeee..e.....aae ion aime
[59] Meng, F., Du, L., Liu, Z., Zhou, Z., Lu, Q., Fu, D., Shi, B., Wang, W., He, J., Zhang, K., et al.: Mm-eureka: Exploring visual aha moment with rule-based large-scale reinforcement learning. arXiv preprint arXiv:2503.07663 (2025)
[60] Midjourney:
Midjourney v6.1. httph:tt///. /wwww//wwww// midj oudjuuree..... com/ (2024)
[61] Niu, Y., Ning, M., Zheng, M., Lin, B., Jin, P., Liao, J., Ning, K., Zhu, B., Yuan, L.: Wise: A world knowledge-inforreed semantic evaluation for text-to-image generation. arXiv preprint arXiv:2503.07265 (2025)
[62] OpenAl: Chatgpt. https: /ttp///caattt openai. com (2023)
[63] OpenAl: GPT-4V(ision) system card (2023), https::/ com/research/
gpt-4rr-ysstem-card
14

---

[64] OpenAI: Hello gpt-4o. https://opents wwwminneee/heelloogggg gpt- 40/ (2024)
[65] OpenAI: Introducing openai ol, 2024. (2024), https://openai. com/o1/
[66] Podell, D., English, Z., Lacey, K., Blattmann, A., Dockhorn, T., Müller, J., Penna, J., Rombach, R.: Sdxl: Improving latent diffusion models for high-resolution image synthesis. arXiv preprint MiXiv:2307001522 (2023)
[67] Qu, I.., Zhang, H., Liu, Y., Wang, X., Jiang, Y., Gao, Y., Ye, H., Du, D.K., Yuan, Z., Wu, X.: Tokenflow: Unified image tokenizer for multimodal understanding and generation. arXiv preprint arXiv:2412.0300 (2024)
[68] Radford, A., Kim, J.W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., Sastry, G., Askell, A., Mishkin, P., Clark, J., Krueger, G., Sutskever, I.: Learning transferable visual models from natural language supervision. In: International Conference on Machine Learning (2021), oorpa//occccc.aoooorr/rrraaaa 2tte//ooppsSDD221511155 231591445
https://api..
[69] Rein, D., Hou, B.L., Stickland, A.C., Petty, J., Pang, R.Y., Dirani, J., Michael, J., Bowman, S.R.: GPQA: graduate-leve woogle-proot q&a benchmark. arXiv preprint arXiv:2311.12022 (2023)
[70] Rombach, R., Blattmann, A., Lorenz, D., Esser, P., Ommer, B.: High-resolution image synthesis with latent diffusion models. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. pp. 10684-10695 (2022)
[71] Ronneberger, O., Fischer, P., Brox, T.: U-net: Convolutional networks for biomedical image segmentation. In: Medical image computing and computer-asiistd intervention-MICCA. 2015: 18th international conference, Munich, Germany, October 5-9, 2015, proceedings, part III 18. pp. 234-241. Springer (2015)
[72] Schulman, J., Wolski, F., Dhariwal, P., Radford, A., Klimov, O.: Proximal policy optimization algorithms. arXiv preprint T.XXvv11077003440 (2017)
[73] Shao, Z., Wang, P., Zhu, Q.. Xu, R., Song, J., Zhang, M., Li, Y., Wu, Y., Guo, D.: Deepseek- math: Pushing the limits of mathematical reasoning open language models. arXiv preprint arXiv:2402.03300 (2024)
[74] Shen, D., Song, G., Zhang, Y., Ma, B., Li, L.,, Jiang, D., Zong. Z., Liu, Y.: Adt: Tuning diffusion models with adversarial supervision, arXiv preprint arxiv:2504.1 11423 (2025)
[75] Song, W., Wang, Y., Song. Z., Li, Y., Sun, H., Chen, W., Zhou, Z., Xu, J., Wang. J., Yu, K.: Dualtoken: Towards unifying visual understanding and generation with dual visual vocabularics. arXiv preprint MtXiv22503.14324 (2025)
[76] Sun, P., Jiang, Y., Chen, S., Zhang. S., Peng, B., Luo, P., Yuan, Z.: Autoregressive model beats diffusion: Llama for scalable image generation. arXiv preprint ar arXiv:2406.06525 (2024)
[77] Sun, Q., Cui, Y., Zhang, X., Zhang, F., Yu, Q., Luo, Z., Wang, Y., Rao, Y., Liu, J., Huang, T., Wang, X.: Generative multimodal models are in-context learners. arXiv: 2312.13288 (2023)
[78] Sun, Q., Yu, Q. Cui, Y., Zhang, F., Zhang, X., Wang, Y., Gao, H. Liu, J., Huang, T., Wang, X.: Generative pretraining in multimodality, arXiv: 2307.05222 (2023)
[79] Tian, K., Jiang,
Y., Yuan, Z.,, Peng, B., Wang, L.: Visual autoregressive modeling: Scalable
image generation via next-scale prediction. Advances in neural information processing systems 37, 34839-84865 (2024)
[80] Tong, S., Fan, D., Zhu, J., Xiong, Y., Chen, X., Sinha, K., Rabbat, M., LeCun. Y.. Xie, S., Liu. Z.: Metamorph: Multimodal understanding and generation via instruction tuning. arXiv preprint arXiv:24 14164 (2024)
[81] Touvron, H., Lavril, T., Izacard, G., Martinet, X., Lachaux, M.A., Lacroix, T., Rozière, B., Goyal, N., Hambro, E., Azhar, F,, et al.: Llama: Open and efficient foundation language models. arXiv preprint arXiv:2302.1399 13971 (2023)
15

---

[82] Wang, J., Yang, Z., Hu, X., Li, L., Lin, K., Gan, Z., Liu, Z., Liu, C., Wang. L.: Git: A generative image-to-tex transformer for vision and language. arXiv preprint arXiv:2205.14100 14100 (2022)
[83] Wang. X., Zhang, X., Luo, Z., Sun, Q. Cui, Y., Wang. J., Zhang. F., Wang. Y., Li, Z., Yu, Q.. et al.: Emu3: Next-token prediction is all you need. arXiv preprint arXiv:2409. 18869 (2024)
[84] Wei, J., Wang, X., Schuurmans, D., Bosma, M., Xia, F., Chi, E., Le, Q.V., Zhou, D., et al.: Chain-of-thouglh prompting elicits reasoning in large language models. Advances in neural information processing systems 35, 24824-24837 (2022)
[85] Wei, X., Zhang, J., Wang, Z., Wei, H., Guo, Z., Zhang, L.: Tiff-bench How does your t2i model follow your instructions? arXiv preprint arXiv:2506.0216 (2025)
[86] Wu, C., Chen, X., Wu, Z., Ma, Y., Liu, X., Pan, Z., Liu, W., Xie, Z., Yu, X., Ruan, C., et al.: Janus: Decoupling visual encoding for unified multimodal understanding and generation. arXiv preprint arXiv:2410. 13.144488811188888 (2024)
[87] Wu, J., Jiang, Y., Ma, C.. Liu, Y., Zhao, H., Yuan, Z., Bai, S., Bai, X.: Liquid: Language models are scalable multi-modal generators. arXiv preprint arXiv:2412.04332 (2024)
[88] Wu, W., Li, Z., He, Y., Shou, M.Z., Shen, C., Cheng, L., Li, Y., Gao, T., Zhang. D., Wang, Z.: Paragraph- to- im-ae generation with information-emrichedd diffusion model. International Journal of Computer Vision 133, 5413-5434 5434 (2025). htpp:///bi.org110.000///1111 02435-1
[89] Wu, X., Bai, Y., Zheng, H., Chen, H.H., Liu, Y.. Wang, Z.. Ma. X., Shu, WJ.. Wu, X., Yang, H., Lim, S.N.: LightGen: Efficient Image Generation through Knowledge Distillation and Direct Preference Optimization (Mar 2025). https://doi.org// 0eee/10.4555000 Xiv.5//a
[90] Wu, X., Hao, Y., Sun, K., Chen, Y., Zhu, F., Zhao, R., Li, H.: Human preference score v2: A solid benchmark for evaluating human preferences of text-to-image synthesis. arXiv preprint arXiv:2306.0934 (2023)
[91] Wu, Y., Zhang. Z.. Chen, J.. Tang, H., Li, D., Fang, Y., Zhu, L., Xie, E., Yin, H. Yi, L., et al.: Vila-u: a unified foundation model integrating visual understanding and generation. arXiv preprint arXiv:2409.044 04429 (2024)
[92] Xiao. H., Wang, G.. Chai, Y., Lu, Z., Lin, W., He, H., Fan, L., Bian, L., Hu, R., Liu, L., et al.: Ui-genie: self-improving approach for iteratively boosting milm-based mobile gui agents. arXiv preprint arXiv:2505.2 22221164 (2025)
[93] Xie, J., Mao, W., Bai, Z., Zhang, DJJ., Wang, W., Lin, K.Q.. Gu, Y., Chen, Z., Yang, Z., Shou, M.7.: Show-o: One single transformer to unify multimodal understanding and generation. arxiv preprint ar Xiv: 12528 (2024)
[94] Xu, J., Liu, X., WU, Y., Tong, Y., Li, Q., Ding, M., Tang, J., Dong, Imagereward: learning and evaluating human preferences for text-to-image generation. In: Proceedings of the 37th International Conference on Neural Information Processing Systems. pp. 15903-15935 (2023)
[95] Yang, A., Yang, B., Hui, B., Zheng, B., Yu, B., Zhou, C., Li, C., Li, C., Liu, D., Huang, F., Dong, G., Wei, H., Lin, H., Tang, J., Wang. J., Yang, J.. Tu, J., Zhang, J., Ma, J., Xu, J., Zhou, J., Bai, J., He, J., Lin, J., Dang. K., Lu, K., Chen, K., Yang, K., Li, M., Xue, M., Ni, N., Zhang, P., Wang, P., Peng. R., Men, R., Gao, R., Lin, R., Wang, S., Bai, S., Tan, S., Zhu, T., Li, T., Liu, T., Ge, W., Deng, X., Zhou, X., Ren, X., Zhang, X., Wei, X., Ren, X., Fan, Y., Yao, Y., Zhang, Y., Wan, Y., Chu, Y., Liu, Y. Cui, Z., Zhang, Z., Fan, Z.: Qwen2 technical report. arXiv preprint arXiv:2407.1067. (2024)
[96] Yang, L.,, Yu, Z.,, Meng, C., Xu, M., Ermon, S., Cui, B.: Mastering text-to-image diffusion: Recaptioning, planning, and generating with multimodal llms. In: International Conference on Machine Learning (2024)
16

---

[97] Yang, Y., He. X., Pan, H., Jiang, X., Deng, Y. Yang, X., Lu, H., Yin, D., Rao, F,, Zhu, M., al.: RI-onevision: Advancing generalized multimodal reasoning through cross-modal formalization. arXiv preprint arXiv:2503. 10615 (2025)
[98] Ye, J., Jiang, D., He, J., Zhou, B., Huang, Z., Yan, Z., Li, H., He. C., Li, W.: Blink-twice: You see, but do you observe? a reasoning benchmark on visual perception. arXiv preprint mmXiv25510.9336 (2025)
[99] Ye, J., Jiang, D., Wang, Z., Zhu, L.., Hu, Z.,, Huang, Z.., He, J., Yan, Z., Yu, J., Li, H., et al.: Echo-4o: Harnessing the power of gpt-4o synthetic images for improved image generation. arXiv preprint arXiv:2508.0900 099877 (2025)
[100] Ye, J., Zhu, L., Guo, Y., Jiang, D., Huang, Z., Zhang, Y. Yan, Z., Fu, H., He, C., Li, W.: Realgen: Photorealistic text-to-image generation via detector -erector-guieed rewards. arXiv preprint arXiv:2512.00473 (2025)
[101] Yu, Q. Zhang, Z., Zhu, R., Yuan, Y., Zuo, X., Yue, Y., Fan, T., Liu, G., Liu, L.,, Liu, X., et al.: Dapo: An open-source Ilm reinforcement learning system at scale. arXiv preprint arXiv:2503. 14476 (2025)
[102] Zhan, J. Ai, Q. Liu, Y., Pan, Y., Yao, T., Mao, J., Ma, S., Mei, T.: Prompt refine- ment with image pivot for text-to-image generation. In: Ku, L.W., Martins, A., Sriku- mar, (eds.) Proceedings of the 62nd Annual Meeting of the Association for Computa- tional Linguistics (Volume 1: Long Papers). pp. 941-954. Association for Computational Linguistics, Bangkok, Thailand (Aug 2024). htts://doi.org/100 https: bttps://aclanthon
org/ 2224.0222.... acl- ael-long.555
[103] Zhang. J., Huang, J., Yao, H., Liu, S., Zhang, X., Lu, S., Tao, D.: RI-vl: Learning reason with multimodal large language models via step-wise group relative policy optimization. arXiv preprint arXiv:2503. 12937 (2025)
[104] Zhang, R., Han, J., Liu, C., Zhou, A., Lu, P., Qiao, Y., Li, H., Gao, P.: Llama-adapter: Efficient fine-tuning of large language models with zero-initialized attention. In: ICLR 2024 (2024)
[105] Zhang. R., Jiang, D., Zhang, Y., Lin, H., Guo, Z., Qiu, P.. Zhou, A., Lu, P., Chang, K.W. Gao, P., et al.: Mathverse: Does your multi-modal I1m truly see the diagrams in visual math problems? ECCV 2024 (2024)
[106] Zhang, R., Wei, X., Jiang, D.,, Zhang, Y., Guo, Z., Tong, C., Liu, J., Zhou, A., Wei, B., Zhang, S., et al.: Mavis: Mathematical visual instruction tuning, arXiv preprint arXiv:2407.087.9 (2024)
[107] Zhou, C., Yu, L., Babu, A., Tirumala, K., Yasunaga, M., Shamis, L., Kahn, Ma, X., Zettlemoyer, L., Levy, O.: Transfusion: Predict the next token and diffuse images with one multi-modal model. arXiv preprint arXiv:2408. 1039 (2024)
[108] Zong, Z., Jiang, D., Ma, B., Song, G., Shao, H., Shen, D., Liu, Y., Li, H.: Easyref: Omni- generalized group image reference for diffusion models via multimodal Ilm. arXiv preprint arXiv:2412.09611 (2024)
[109] Zong, Z., Ma, B., Shen, D., Song, G., Shao, H., Jiang, D., Li, H., Liu, Y. Mova: Adapting mixture of vision experts multimodal context, arXiv preprint arXiv:2404. 13046 (2024)
17

---

A Related Work
Unified Generation and Understanding LMM. Recently, the effort to unify image generation and understanding in a single LMM has attracted much attention. Building upon large language models (LLMs), it is natural for the LMMs to understand the image and output the text [63, 39, 109, 19, 104, 33]. However, the method of how to generate an image from LMM is still under exploration. The image generation method diverges into different branches. One line of the method relies exterior image generation model to complete generation [15, 78, 77, 43, 80, 17, 108, 38). The generator often utilizes text-to-image diffusion models [70, 66] due to its powerful generation capability. To deliver the generation information, the LMM passes either the implicit conditional feature or the explicit image prompt to the generator. For example, EMU [78] first trains the LMM to output CLIP [68] image features identical to that input to the LMM. Then, pretrained UNet [71]
of Stable Diffusion [70] receives the output feature as the condition to generate an image. Another line of the method seeks to train the LMM generate discrete tokens produced by VQGAN [16] to eliminate the need for an additional generator. [83, 41] directly adopts the VQGAN encoder as the image tokenizer for LMM. However, the VQGAN encoder only pretrained on the image reconstruction task and thereby generates visual tokens less helpful for image understanding. To improve the understanding capability, [86, 9, 57, 48] proposes to tackle the understanding and generation tasks with different vision encoders separately. The CLIP encoder deals with image input for understanding, while the VQGAN encoder is responsible for generation. Moreover, some works [91, 67, 75] attempt to empower the vision encoder with both the understanding and the generation capability. VILA-U [91] trains a vision encoder with both the contrastive loss [68] for text-image understanding and reconstruction loss [16] for image detail preserving. Thanks to the joint pretraining, the vision encoder could generate text-aligned discrete visual tokens. The LMM is then trained to receive the discrete tokens for image understanding and predict them for image generation.
Reinforcement Learning for Large Reasoning Models. The emergence of OpenAI ol [65] has gained tremendous attention in developing the reasoning capability of large language models. Later, DeepSeek-R1 [20] proposes rule-based reward and GRPO training method. The introduced method instructs the model to perform an extensive reasoning process before generating the final answer. The reward only focuses
the correctness of the final answer and the following of the pre-defined format. Recently, a number of works have applied this method to multi-modal large language models [7, 59, 97, 103, 14, 29, 11] with task-specific rewards like correctness and IoU [50]. This training paradigm largely helps various bassoning-intensin tasks [69, 32, 23] like mathematical problem-solving [27, 58, 51, 105, 106], code generation [8, 2, 30J, and complex scene understanding [98].
B More Experiment Details
B.1 Experiment Setup
Training Settings. Our training dataset comprises text prompts sourced from the training set of T21-CompBench [28] and [24], totaling 6,786 prompts with no images. Prior training, we use GPT- 4o mini to extract the objects and their attributes from the prompts to facilitate computing the rewards. We use Janus-Pro-7B reward model, we choose HPS [90] as the human preference model, GroundingDINO [49] as the object detector, and GIT [82] as the VQA model. For the ORM, we finetune LLa VA-One Vi in the same manner as [24].
the base model. We use a learning rate of le-6 and beta of 0.01, For the
Benchmark. We test on T21-CompBench [28], WISE [61]. GenAl-Bench [45], and THF-- Bench [85] validate the effectiveness of our method. Del-ComBBench comprises 6,000 com- positional text prompts evaluating three categories (attribute binding, object relationships, and complex compositions) and six sub-categories (color binding, shape binding, texture binding, spatial relationships, non-spatial relationships, and complex compositions). WISE consists of 1,000 text prompts spanning three categories (cultural common sense, spatial-temporal reasoning. and natural science) for evaluating world knowledge of the text-to-image models. To correctly generate an image, the model needs to reason about what the exact object or scenario is depicted in the prompt. We slightly modify the reasoning instruction on the WISE benchmark for more aligned results. GenAl-
18

---

Token-level CoT Only
Semantic-level + Token-level CoT
fabric bag and glass
key on the right of a dog
Figure 7: More Visualization Result of the Image Diversity of Single Prompt. We showcase the result of only token-level CoT optimized and both semantic-level and token-level CoT optimized.
Table 6: TIIF-Bench Testmini Subset Evaluation Results. The best score is in blue, with the second-best score in green
Busic Fellowing
Advance Folloiing
Dedigeer
Ovanal
Arg
Annhhrn
Relation
Reaoming
Arg
Altrinti +Relation
Airihiic Reesceing
Relation +Reoscring
Style
Tixtt
Real World
sbazt
Liarsgen
41.67
35.22
50.00
48.33
50.32
$1.07
47.50
31.57
40.54
33.33
Lightoon [o]
Stowo [9]]
Infanty [25]
62.07
62.32
73.08
75.41
04.38
73.57
12.07
71.82
60.44
55.57
23.33
10.83
23.83
Imna--Po (+]
65.02
79:33
18:15
03.32
79.07
55.07
56.20
T2E-I (Ouro)
66.59
67.39
82.50
51.63
86.50
43,00
33.47
79.45
78.73
K2
69.05
68.00
71.64
00.47
72:45
09.95
69.00
70.00
60.00
63.33
27.60
26.24
07.54
00.45
Bench is a benchmark containing 1,600 complex, real-world text prompts collected from professional designers, which covers broad spectrum of compositional text-to-visual generation elements, from basic aspects like scenes, attributes, and relationships to more professional ones, including counting. comparison, differentiation, and logical reasoning. TIIF-Bench comprehensive benchmark for fine-graine fine-grained text-to-image model evaluation, featuring 36 novel prompt combinations across six compositional dimensions and 100 real-world designer-level prompts with rich aesthetic judgment. We follow the official evaluation setting of all the benchmarks.
C
More Experiment Results
C.1 More Results
We provide the experiment results on TIIF-Bench in Table and more qualitative examples in Fig.
Finally, we discuss the zero-shot potential of the baseline model to perform both semantic-level and token-level reasoning. Specifically, we apply the same image generation process of T2I-R1 directly to the baseline model, where the baseline model is first instructed to output the semantic- CoT and then the teken-level COT. We term this method of generation as 'Janus-Pro w/ zero-shot semantic- CoT" in Figure 9-12. As shown in the figure, zero-shot semantic-leva CoT brings very marginal improvement, while T2I-RI demonstrates a satisfying result. The reasons are twofold: (1) Zero-shot semantic-level CoT misses critical objects in the original prompt. As shown in Figure 12, the zero-shot semantic-level CoT misses the bird in the original prompt. (2) Zero-shot semantic-level CoT does not fit the model's generation ability provide useful information for generation. Although
19

---

T21-R1
T21-R1
Janus-PPe Pro
Only Semantie-lvee
Only Token-levl
(emmntic-leve +Token-level)
Janus-Pro Pro
Only Semantic-level
Only Token-levd
(Smantic-level TToken-evvel)
chawiesss perferty comoylaged spetur brows fegf
A chancloo peofent amanfaag spiner lag"
A Epgeyll typ atrinf and in World War
A,frm prrsoal commonication lçfore adfest toigheoss
Aeneus peshillss
tranpessniee Eayst
Cryper aspead air
Bong
with calurall opersesting party beaity
figys
Figure 8: More Visualization Results. We provide the image generation results of the same prompt from four models: base model, the model with only semantic-level CoT optimized, the model with only token-level CoT optimized, and the model with both levels of CoT optimized.
the semantic-level CoT in Figure 9-11 includes all the objects and relationships, the baseline model still fails to generate a satisfying result. This highlights the necessity of our proposed BiCoT-GRPP training method to build the synergy between the levels of CoT and make them work together.
C.2 More Details about Reward Analysis
We conduct human study to evaluate the visual quality of the generated images. Specifically, select four options of reward models (V, O, H + D, and H + D + V) to generate an image from the same prompt. Then we ask humans to rank the four images and for 3 points, rank 2 for 2 points, and so on). We ask the annotators to judge only according to the visual appeal. We provide examples of an unsatisfactory generation, including different aspects: 1) Corrupted floating words, 2) Distortion like melting effect of faces, hands, or other objects, 3) Extra or missing hands, legs, tails, etc., Object merging, frequently observed for compositional prompts, 5) Low-level aspects including over exposure or over saturation, 6) Blurred areas, especially in fine details like keyboard. The instruction the annotator highlights that the highly ranked images should show or fewer problems compared with others. Eight graduate students are employed to conduct the study eliminate individual bias. We randomly choose 30 prompts from each of the subtasks from the Del-ChmBBench. The result is shown in the visual quality column in Table 3. We observe that ensemble rewards achieve better visual quality, with H + This improvement could be attributed to the implicit regularization provided by multiple rewards, preventing overfitting to single reward model. Conversely, individual reward models fail to provide satisfactory quality despite high benchmark scores.
them according to the rank (rank
obtaining slightly superior results.
20

---

C.3 Comparison with Prompt Rewriting Methods
The prompt rewriting [96, 102, 26, 88] method to leverage an external LLM to generate an enriched version of the prompt and use this version for the text-to-image model to produce the images. Our semantic-level CoT design resembles the high-level idea of this method, but there exist several key differences. First, the semantic-level CoT is generated from the image generation model itself, while the prompt enrichment leverages an extra I.L.M. Second, our design facilitates joint-optimization of the prompt design and the image generation. On the contrary, the prompt enrichment is generation- peceerttiooomdel-aaaeee -gnessag, which means that although the prompt enriched with more details, it is not necessary that the generation model can generate better image based on this enriched prompt. We validate this claim in Figure 9-11. Although the zero-shot semantic-level CoT (enriched prompt) all correctly includes all the key objects mentioned in the prompt and adds more details, the model still cannot generate satisfying result. Simply enriching the prompt is not sufficient to bring notable improvements.
C.4 Comparison with Training with RL and Supervised Finetuning
In our experiments, we directly employ reinforcement learning. An alternative to first conduct supervised finetuning (SFT) and then continue with RL training. Our findings show that a cold start stage with SFT seems to be detrimental for the final performance. Specifically, we use the open-source high-quality text-to-image dataset BLIP3o-60K [5] as our SFT dataset. We employ Swen2.5-VL-72B. Instruct [3] to generate the semantic-level CoT with the image and the original short prompt input. We finetune the model for one epoch and then conduct RL finetuning following the same training setting of T2I-R1. The results are shown in Table below:
Table 7: Model Performance Comparison Between RL and SFT.
Training Method
Color
Shape
Texture
Spatial
Non-Spatial
Complex
Janus-Pro
0.6359
0.3528
0.4926
0.2061
0.3085
0.3559
SFT Only
0.7035
0.5217
0.6423
0.2775
0.3068
0.3626
Hybrid optimization
0.7765
0.5832
0.6981
0.3327
0.3092
0.3949
T2I-RI (RL Only)
0,8130
0,5852
0.7243
0.3378
0.3090
0.3993
The key findings are two-fold. First, high-quality SFT is beneficial. The curated BLIP3-60K dataset yields significant performance gains. However, SFT performance substantially lags behind RL training. We hypothesize this occurs because SFT constrains the model to replicate the training distribution rather than leveraging its inherent capabilities. Specifically, when presented with valid semantic-leve CoT,, SFT forces the model to generate the exact corresponding training image, even when the model could produce alternative valid outputs. This constraint introduces unnecessary training complexity. Second, RL after high-quality SFT is still inferior to direct RL training. While high-quality SFT pre-training improves subsequent RI. performance, the combined approach remains mostly inferior to or merely comparable with direct RI. training.
C,5 Choice of Reward Weights
In Table the weight of all the rewards are set 1. We find that the weight of the reward model has little influence on the final training result. Here we provide the detailed study to illustrate the method is stable to the weight of the reward model. We follow the training setting of T21-R1, where three reward models, HPS, GroundingDINOO, and GIT are employed. In our experiments, we multiply the weight of each reward model by respectively, while maintaining the weight of the other two reward models. Apart from these three experiments, we also conduct an experiment where we compute the relative reward of each reward model inside the group, and then sum up the normalized reward as the final reward for the sample. This eliminates the mean and variance difference among the reward models (termed as Normalized Reward). The results are shown in Table 8 below.
C.6 Hyperparameters
All of our experiments are conducted on 8 H800. Our training procedure lasts about 16 hours. We provide the detailed training hyperparameters in Table 9.
21

---

Table 8: Comparison of Different Reward Weights.
Reward Design
Color
Shape
Texture
Spatial
Non-Spatial
Complex
5*HPS, 1sothers
0.8215
0.5915
0.7337
0.3051
0.3074
0,4051
S+GDINO, *otthrs
0.7951
0.5520
0.7100
0.3313
0,3104
0.3916
5+GIT, +otters
0.7972
0.5620
0.7149
0.3357
0.3111
0.3934
Normalized Reward
0.8106
0.5820
0.7142
0.2940
0.3072
0.3993
T2I-R1 (Equal Weights)
0.8130
0.5852
0.7243
0.3378
0.3090
0.3993
Table 9: T21-R1 training hyperparameters.
Name
Learning rate
le-6
Beta 8
0.01
Group Size G
8
Classifier-Fres Guidance Scale
Max Gradient Norm
1.0
Batchsize
8
Training Steps
1,600
Gradient Accumulation Steps
2
Image Resolution h w
384 384
D Limitations and Future Work
While this work explores the text-to-image generation task, requires more exploration on how to apply this paradigm to other modalities like video generation [4] or 3D [21] tasks. Specifically, video generation tasks are more complex regarding the reward design and the base model. For the reward design, how to apply dense rewards on each generated frame is still an open question. Besides, there exists no understanding and generation unified model for videos, so BíCoT-GRPO cannot be used directly. Meanwhile, the current inference time of video generation is too long for the current GRPO paradigm. How to balance the training time and effect needs further study.
22

---

Prompt
a red train, a blue bowl, green flower, and yellow cake
Janus-Pro w/ zero-shot semantic-level CoT
Janus-Pro
T2I-R1
Zero-shot Semantic-level CoT
Underneath blue sky, a train moving along the railway track. Next to it, green flower stands tall, while yellow cake sits on a nearby table.
In the image, the red train occupies the left side of the image, with sleck and polished metallic finish. Beside it, a blue bowl sitting gently on wooden table, its surface reflecting light, and there lies delicate green flower the table, its petals spreading like butterfly's wings. In the background, an appetizing yellow cake with intricate icing decoration stands next to table on the right side of the scene. The train appears slightly ahead the bowl, while the cake sits further behind the table and flower, creating an interesting spatial arrangement of objects and colors against bright backdrop.
T2I-R1 Semantic-level CoT
Figure 9: Visualization Results of Semantic-level CoT. We provide the image generation results of the same prompt from three settings: base model, base model with zero-shot semantic-level CoT, and T21-R1. For the setting of base model with zero-shot semantic-level CoT, pipeline of T2I-RI directly on the base model. We employ the same prompt of T2I-R1 to instruct the base model to generate zero-shot semantic-level CoT, which visualize in the figure and provide a comparison of the semantic-leve CoT generated by T2I-R1.
the same generation
23

---

Prompt
three paintings and one knife
Janus-Pro w/ zero-shot semantic-level CoT
Janus-Pro
T2I-R1
Three paintings hung the wall, and in the middle stood sharp- looking knife. The three paintings were of varying nature: one depicted sunflowers in vibrant yellow, the second portrayed a serene countryside with soft blues and greens, while the third had an abstract, multicolored design.
Zero-shot Semantic-level CoT
The image displays three paintings hung on a wall, each characterized by vibrant colors such pink, blue, and yellow, reflecting various artistic styles. front of the paintings, there is single, slender knife with metallic polish, positioned horizontally and lying polished wooden surface. The knife's reflective glare contrasts with the richness of the paintings, creating an intricate visual interplay. The paintings provide backdrop the knife, and their colors accentuate the sharpness of the blade.
T2I-R1 Semantic-level CoT
Figure 10: Visualization Results of Semantie-level CoT. We provide the image generation results of the same prompt from three settings: base model, base model with zero-shot semantic-level CoT, and T21-R1. For the setting of base model with zero-shot semantic-level CoT, pipeline of T2I-RI directly on the base model. We employ the same prompt of T21-R1 to instruct the base model to generate zero-shot semantic-level CoT, which visualize in the figure and provide a comparison of the semantic-leve CoT generated by T2I-R1.
the same generation
24

---

Prompt
playful waiting dog behind the door
Janus-Pro w/ zero-shot semantic-level CoT
Janus-Pro
T2I-R1
A dog is standing behind door, with brown fur, dark eyes, a black collar, and white patch on its face. The door is wooden, with metal handle and hinges. The dog's ears are perked up, and looks curious or alert through the door.
Zero-shot Semantic-level CoT
The image shows brown and white dog peeking its head through a small gap the door, with gentle and curious expression etched onto its face. The door is slightly ajar, and the dog's body is barely visible beyond the edge of the frame, hinting game. The room behind the door dimly lit, creating an eerie yet mission or has been
T2I-R1 Semantic-level CoT
intimate atmosphere as if the dog is guarding the entrance for some time. The door itself is wooden with simple metal handle. The overall tone of the visual emphasizes curiosity and anticipation.
Figure Visualization Results of Semantic-level COT. provide the image generation results of the same prompt from three settings: base model, base model with zero-shot semantic-level CoT,, and T2I-R1. For the setting of base model with zero-shot semantic-level CoT, we use the same generation pipeline of T2I-R1 directly on the base model. We employ the same prompt of T2I-R1 to instruct the base model to generate a zero-shot semantic-level CoT, which we visualize the figure and provide comparison of the semantic-level CoT generated by T2I-R1.
25

---

Prompt
dog, bird, a lake, and building
Janus-Pro w/ zero-shot semantic-level CoT
Janus-Pro
T2I-R1
Zero-shot
Semantic-level CoT
A dog wags its tail, plays fetch at the lake, and sits in front the building.
A fluffy white dog with bright amber eyes sits by the edge of serene lake, observing vibrant black and white bird soaring above. Nearby, classic building in shades brown and cream stands tall with numerous windows, giving it an old-world charm. Together, they create a picturesque scene that evokes peace and charm.
T2I-RI Semantic-level CoT
Figure 12: Visualization Results of Semantic-level CoT. We provide the image generation results of the same prompt from three settings: base model, base model with zero-shot semantic-level CoT,, and T2I-R1. For the setting of base model with zero-shot semantic-level CoT, we use the same generation pipeline of T2I-R1 directly on the base model. We employ the same prompt of T21-R1 to instruct the base model to generate a zero-shot semantic-level CoT, which we visualize the figure and provide comparison of the semantic-level CoT generated by T2I-R1.
26

---

Janus- Pro
T2I-R1
origami cranes unfolding into real birds during cherry blossom seuson
Figure 13: More Visualization Result of the Image Diversity of Single Prompt. We showcase the result the bascline model, Janus-Pro, and T2I-R1.
27

---

Janus-Pro
T21-R1
paint drops falling from brushes creating flowers cunvas below
Figure 14: More Visualization Result of the Image Diversity of Single Prompt. We showcase the result the baseline model, Janus-Pro, and T21-R1.
28

---

NeurIPS Paper Checklist
The checklist is designed to encourage best practices for responsible machine learning research, addressing issues of reproducibility, transparency, research ethics, and societal impact. Do not remove the checklist: The papers not including the checklist will be desk rejected. The checklist should follow the references and follow the (optional) supplemental material. The checklist does NOT count towards the page limit.
Please read the checklist guidelines carefully for information on how to answer these questions. For each question in the checklist:
You should answer [Yes], [No], or [NA].
[NA] means either that the question is Not Applicable for that particular paper or the relevant information is Not Available.
Please provide short (1-2 sentence) justifiicton right after your answer (even for NA).
The checklist answers are an integral part of your paper submission. They are visible to the reviewers, area chairs, senior area chairs, and ethics reviewers. You will be asked to also include (after eventual revisions) with the final version of your paper, and its final version will be published with the paper.
The reviewers of your paper will be asked to use the checklist as one of the factors in their evaluation.
While "[Yes] " is generally preferable to "[No]", ", it is perfectly acceptable to answer "[No] provided proper justification is given (e.g., "error bars are not reported because it would be too computationally expensive" or "we were unable to find the license for the dataset we used"). In general, answering "[No] or "[NA] is not grounds for rejection. While the questions are phrased in binary way, we acknowledge that the true answer often more nuanced, so please just use your best judgment and write justification to elaborate. All supporting evidence can appear either in the main paper or the supplemental material, provided in appendix. If you answer [Yes] to a question, in the justification please point to the section(s) where related material for the question can be found.
IMPORTANT, please:
Delete this instruction block, but keep the section heading "NeurIPS Paper Checklist",
Keep the checklist subsection headings, questions/answers and guidelines below.
Do not modify the questions and only use the provided macros for your answers.
1. Claims
Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Question:
Answer: [Yes]
Justification: The claims made in the abstract and introduction accurately reflect this paper's contributions and scope.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made the paper and important assumptions and limitations. A No or
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
It is fine to include aspirational goals as motivation as long as is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors? Answer: [Yes]
29

---

Justification:
We discuss the limitations of the work in Appendix.
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model vell-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested few datasets with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community, Reviewers will be specifically instructed not penalize honesty concerning limitations.
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?
Answer: [NA]
Justification:
The paper does not include theoretical results.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that affects the main claims and/or conclusions of the paper (regardless whether the code and data are provided or not)?
Answer: [Yes]
Justification: Please see Section Experiments, Appendix, and the supplementary material.
Guidelines:
The answer NA means that the paper does not include experiments.
30

---

If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) the contribution is primarily a new algorithm, the paper should make clear how reproduce that algorithm. (b) If the contribution is primarily new model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution is new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g.. with an open-source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors welcome describe the particular way they provide for reproducibilityy In the case of closed-source models, it may be that access to the model is limited in some way (e.g., registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [Yes]
Justification: We will provide open access to the data and code, with sufficient instructions to faithfully reproduce the main experimental results.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://nips cc/
for more details.
While we encourage the release of code and data, we understand that this might not be possible, "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (c.g., for a new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https: /ttpsscccppbblcc////eeccppcpppcp hetpublcccgguieeeee /CodeSubmiii ssionPolicy for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
31

---

Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: Please see Section Appendix.
Guidelines:
The answer NA means that the paper does include experiments.
The experimental setting should be presented the core the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, as supplemental material.
7. Experiment statistical significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [No]
Justification: We conduct experiments only once and report the accuracy of the best model, and it would be too computationally expensive to conduct the pre-training multiple times.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
It should be clear whether the error bar is the standard deviation or the standard error of the mean.
is OK to report I-sigma error bars, but one should state it. The authors should preferably report a 2-simma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out range (e.g. negative error rates).
If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time exccution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification: Please see Section Appendix.
Guidelines:
The answer NA means that the paper does include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
32

---

The paper should provide the amount of compute required for each of the individual experimental runs as well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary failed experiments that didn't make into the paper).
9. Code of ethics
Question: Does the research conducted the paper conform, in every respect, with the NeurlPS Code of Ethics pettpa:/nneurppe: cttplass...cc.aeeeeeoee http//ullc//Btthee csGuieellnnnn
Answer: [Yes]
Justification: The research conducted in the paper conforms, in every respect, with the NeurIPS Code of Ethics.
Guidelines:
The answer NA means that the authors have reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., if there is special consid- eration due to laws or regulations in their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [No]
Justification: This work focuses on academic, publicly-available datasets including T2I- CompBench, WISE, and GenAl-Bench This work is not related any private or personal data, and there's 's no explicit negative social impacts.
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of MI.).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (c.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
33

---

Justification:
The paper poses no such risks.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards challenging, and many papers do not require this, but encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification: Please see Section Appendix.
Guidelines:
The answer NA means that the paper does not existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset used and, possible, include URL.
The name of the license (c.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website). the copyright and terms of service of that source should be provided.
assets are released, the license, copyright information, and terms use in the package should be provided. For popular datasets, haperswithcome com/ datasets has curated licenses for some datascts. Their licensing guide can help determine the license of dataset.
For existing datasets that re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
If this information is not available online, the authors are encouraged to reach out to the asset's creators.
13. New assets
Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Question:
Answer: [Yes]
Justification:
Please see supplementary material.
Guidelines:
The answer NA means that the paper does not release assets.
Researchers should communicate the details of the dataset/code/mooel as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
34

---

Answer: [Yes]
Justification:
Please see Section Appendix.
Guidelines:
The answer NA means that the paper does involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According the NeurIPS Code of Ethies, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals equivalent for research with human subjects
Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Question:
Answer: [NA]
Justification: The paper does not involve research with human subjects.
Guidelines:
The answer NA means that the paper does involve crowdsourcing nor research with human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurlPS Code of Ethics and the guidelines for their institution.
For initial submissions, not include any information that would break anonymity (if applicable), such as the institution conducting the review.
16. Declaration of LLM usage
Question: Does the paper describe the usage of LLMs if is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing. editing, or formatting purposes and does not impact the core methodology, scientific rigorousness, originality of the research, declaration is not required.
Answer: [NA]
Justification: LLM is used only for writing, editing, or formatting purposes and does not impact the core methodology, scientific rigorousness, originality of the research.
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, non non- non-standard components. Please refer to our LLM policy (https : pettts://neur/ee //euripi......// cc/Conf Cocoooofeeenee////aoo /202 /LLM) for what should or should not be described.
35


Reviews:
[{"review_id": "vNQgfEXpMH", "reviewer": "Reviewer_ypQJ", "summary": "This paper introduces T2I-R1, which aims at performing text-to-image generation with bi-level CoT reasoning process. Specifically, the semantic-level CoT captures high-level semantics while the token-level CoT is for low-level information modelling. The proposed method BiCoT-GRPO boosts the performance of Janus-Pro by 13% on T2I-CompBench. The ablation verifies the effectiveness of different levels of the CoT reasoning process.", "questions": "The topic of the paper is worth investigating, and the paper well illustrates the benefit of RL in text-to-image generation. While some weaknesses above, I still think this paper is acceptable.", "limitations": "yes.", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "**Strengths**\n1) The motivation is clear, and the exploration of RL into text-to-image generation is worthwhile. The proposed methods sound technically correct. The CoT reasoning-based generation helps improve fidelity significantly. \n2) The BiCoT-GRPO unifies the semantic-level CoT and low-level CoT in a sequential manner, which leverages the semantic-enriched prompt for generation, exhibiting high quality.\n3) The ensemble of generation reward models, including the object detector, VQA, HPS, and ORM helps comprehensively improve the generation quality in different aspects.\n\n**Weakness**\n1) In Table 3, I think the evaluation of the object detector on T2I-CompBench can not fully showcase the benefit of the reward design. I suppose the metrics counting in GenEval can better illustrate its effectiveness.\n2) In Figure 5, if only included token-level CoT is included, the generated image is relatively similar to the original Janus-Pro output. It seems like the semantic-level CoT benefits more.\n3) The function of the human Preference Model and Output Reward Model seems to overlap.", "was_revised": true, "final_justification": "I appreciate the detailed rebuttal, and my concerns have been addressed. Thanks for the effort, and I will maintain my original score.", "rebuttal": "> **Q1: The evaluation of obj detector cannot be fully showcased in T2I-CompBench. Need metrics of counting in Geneval to better illustrate.**\n\nThanks for your advice. **We provide the results comparison of T2I-R1 with the baseline model on the counting tasks in three different benchmarks**: GenEval, T2IComp-Bench++, and GenAI-Bench:\n\n||Baseline|T2I-R1|\n|-|-|-|\n|T2IComp-Bench++-numeracy|0.4132|0.6097|\n|GenEval-counting|0.5375|0.5688|\n|GenAI-counting|0.73|0.81|\n\nAs the results shown, **T2I-R1 largely surpasses the baseline model on all of the three benchmarks.** We believe this could validate the effectiveness of the object detector in counting tasks. Note that, we cannot reproduce the result of the baseline model Janus-Pro-7B in their paper. For fair comparison, we evaluated its performance on our own, ensuring that both the baseline model and T2I-R1 used the same random seed during generation.\n\n> **Q2: In Fig. 5, token-level CoT is similar to the baseline output. It seems like the semantic-level CoT benefits more.**\n\nSorry for the confusion. We would like to address your concerns from the following two aspects:\n\n1. **Qualitative Aspect (Figure 5)**\n\n   The prompt in Fig. 5 are sourced from WISE, which requires a strong reasoning capability. We do not inlcude this kind of data in our training dataset. Therefore, without the semantic-level CoT helping analyze the prompt, this prompt is out of distribution for the model, causing the baseline model and model with only token-level CoT optimised generating similar non-sense images. \n\n   To observe the gain brought by the token-level CoT, **we should include the semantic-level CoT when comparing, i.e., comparing the second and forth column in Fig. 5**. The improvement brought by the token-level CoT is obvious. For example, with the top left prompt *\"Show a plant that is asymbol of good fortune inIrish culture, and is knownfor its three-lobed leaves\"*:\n\n   + **Semantic-level only (Column 2)**: Correctly identifies \"clover\" but produces a flat, textureless image with poor lighting and plain white background\n   + **With token-level CoT (Column 4)**: Generates vibrant colors, natural lighting, realistic textures, and professional depth-of-field that makes the subject prominent\n\n2. **Quantitative Aspect (Table 4)**\n\n   We also illustrate the effectiveness of the token-level CoT quantitatively. \n\n   First, **for the compositional ability** illustrated in Tabel 4, it can be shown that only optimising the token-level CoT brings large improvement, with 20% on T2I-CompBench and 5% on WISE benchmark.\n\n   Second, **we provide a quantitative metric to measure the improvement of visual quality brought by the token-level CoT.** Specifically, we employ a CLIP-based aesthetic predictor, which receives the image and outputs a score. The predictor does not require prompt input so that the score is purely based on the visual appealing. We input all of the generated images when testing T2IComp-Bench to this predictor and record the average score.\n\n   |Model|Semantic-level CoT|Token-level CoT|Aesthetic Score|\n   |-|-|-|-|\n   | Janus-Pro-7B |||5.41|\n   |-|$\\checkmark$||5.58|\n   |-||$\\checkmark$|5.68|\n   |T2I-R1|$\\checkmark$|$\\checkmark$|5.76|\n\n   The improvement from Line 2 to Line 4 and from Line 1 to Line 3 both demonstrate the effectiveness of token-level CoT in visual quality. We will include the illustration of Fig. 5 and the aesthetic score in the final version to make the effectiveness of token-level CoT more obvious.\n\n> **Q3: Function of HPS and ORM is overlap**\n\nSorry for the confusion. While both models evaluate prompt alignment, they also serve other distinct purposes:\n\n1. ORM focuses exclusively on prompt-image alignment, whereas **HPS incorporates human aesthetic preferences alongside alignment assessment**, as demonstrated in Figure 4 and Line 167. For the training data of ORM, the label of an image is *Yes* whenever it is aligned with the prompt. For HPS, the model is trained on the human preference data [1], which contains both the alignment and visual appealing aspects.\n2. **ORM leverages multimodal large language models, thereby preserving extensive world knowledge**, while HPS relies on CLIP-based architectures. With the knowledge from LLM, ORM is expected to provide a more faithful score when encourtering a unusual prompt or generated image.\n\nWe will clarify these distinctions in the text in our final version.\n\n[1] Wu, Xiaoshi, et al. \"Human preference score: Better aligning text-to-image models with human preference.\" Proceedings of the IEEE/CVF International Conference on Computer Vision. 2023."}, {"review_id": "PXIKGL1AJ4", "reviewer": "Reviewer_7zUq", "summary": "This paper proposes T2I-R1, a reasoning-enhanced text-to-image generation framework that integrates reinforcement learning (RL) with bi-level Chain-of-Thought (CoT) reasoning. It introduces two levels of CoT: semantic-level CoT for high-level prompt planning, and token-level CoT for fine-grained pixel-wise generation. To unify these two reasoning processes, the authors design BiCoT-GRPO, a training mechanism with an ensemble of generation rewards to jointly optimize both levels. Applied to the Janus-Pro baseline, T2I-R1 achieves significant performance improvements, outperforming prior state-of-the-art methods by 13% on T2I-CompBench and 19% on WISE.", "questions": "see Weakness.", "limitations": "see Weakness.", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 2, "originality": 3, "strengths_and_weaknesses": "Strengths \n\n1. The paper introduces a novel bi-level reasoning paradigm for image generation, distinguishing between semantic-level and token-level CoT processes.\n\n2. It proposes a reward-driven optimization strategy for RL-based image generation, which is well-aligned with the model's hierarchical reasoning structure.\n\n3. The approach is empirically validated on multiple benchmarks, demonstrating performance improvements over strong baselines.\n\n\nWeaknesses\n\n1. The semantic-level CoT aims to enrich the initial prompt by imagining more detailed aspects of the target image. However, this idea closely relates to prior work on prompt enrichment in text-to-image generation, which is not discussed.\n\n- Mastering Text-to-Image Diffusion: Recaptioning, Planning, and Generating with Multimodal LLMs\n- Prompt Refinement with Image Pivot for Text-to-Image Generation\n- DisEnvisioner: Disentangled and Enriched Visual Prompt for Customized Image Generation\n- Paragraph-to-Image Generation with Information-Enriched Diffusion Model\n\n2. The paper assumes a patch-by-patch generation process similar to auto-regressive models and maps this to token-level CoT. However, this assumption may not hold for diffusion-based generation, which follows a denoising trajectory rather than token-wise generation. Whether token-level CoT is meaningful or effective under diffusion paradigms is unclear. Alternative strategies, such as coarse-to-fine generation (e.g., as in VAR or Next-Scale Prediction), might be more appropriate and should be discussed.\n\n- Scalable Image Generation via Next-Scale Prediction\n\n3. The role of reinforcement learning in this framework needs to be further justified. It’s unclear whether RL is essential for the proposed model or whether similar gains could be achieved through supervised or hybrid optimization.\n\n4. Experimental results suggest that most performance gain comes from the semantic-level CoT. However, it remains unclear how much of this improvement comes purely from prompt enrichment, without involving RL or token-level reasoning. A fairer comparison would include a model with enriched prompts but no further optimization, especially in the context of diffusion-based generators.", "was_revised": true, "final_justification": "The authors have provided a thorough rebuttal with detailed responses and additional experiments that effectively address all of my concerns. The proposed approach is novel and well-motivated. I recommend acceptance.", "rebuttal": "> **Q1: Semantic-level CoT is to enrich the initial prompt by adding more details. However, this idea is closely related to prompt enrichment, which is not discussed.**\n\nThanks for pointing out. After thoroughly reading the related papers mentioned, **we acknowledge that the idea of semantic-level CoT is related to the prompt enrichment work**. However, we would like to **highlight some key differences and advantages between our method and the prompt enrichment technique**:\n\n1. The semantic-level CoT is generated from the image generation model itself, while the prompt enrichment leverages an extra LLM. **Our design facilitates a joint-optimization of the prompt design and the image generation. On the contrary, the prompt enrichment is generation-model-agnostic**, which means that although the prompt is enriched with more details, it is not necessary that the generation model could generate a better image based on this enriched prompt. \n\n   To validate this claim, we conduct the following experiments on T2IComp-Bench:\n\n   ||Prompt Generation Model|Image Generation Model|\n   |-|-|-|\n   |*Exp1*|Janus-Pro-7B|Janus-Pro-7B|\n   |*Exp2*|Qwen2.5-7B-Instruct|Janus-Pro-7B|\n\n   Note that, T2I-R1 is obtained after *Exp1* is trained with our proposed BiCoT-GRPO. The results are shown below:\n\n   ||Color|Shape|Texture|Spatial|Non-Spatial|Complex|Aesthetic|\n   |-|-|-|-|-|-|-|-|\n   |Janus-Pro|0.6359|0.3528|0.4926|0.2061|0.3085|0.3559|5.41|\n   |*Exp1*|0.6451|0.3773|0.5262|0.2298|0.3049|0.3578|5.31|\n   |*Exp2*|0.7004|0.4111|0.5892|0.2522|0.3088|0.3597|5.30|\n   |T2I-R1|0.8130|0.5852|0.7243|0.3378|0.3090|0.3993|5.76|\n\n   As shown, using the prompt enriched from either Qwen2.5 or Janus-Pro-7B itself still far lag behind T2I-R1 in both compositional abilities and aesthetics.\n\n   Specifically, to evaluate aesthetics, we employ a CLIP-based aesthetic predictor, which receives the image and outputs a score. The predictor does not require prompt input so that the score is purely based on the visual appealing. We input all of the generated images when testing T2IComp-Bench to this predictor and record the average score.\n\n2. **Qualitative examples shown in Figure 9 to 12 also shows that our method goes beyond prompt enrichment.** Although the zero-shot semantic-level CoT (enriched prompt) in Fig. 9 to 11 all correctly include all the key objects mentioned in the prompt and add more details, the model still cannot generate satisfying result. Simply enriching the prompt is not sufficient to bring notable improvements.\n\n3. Since the semantic-level CoT and the image are generated from the same model, **the deployment cost is much smaller than the prompt enrichment method:**\n\n   || Memory (GB)|\n   |-|-|\n   |T2I-R1|16.657|\n   |*Exp2*|30.501|\n\nWe will include the discussion of prompt enrichment and cite all the works you mentioned in the related work in our final version.\n\n> **Q2: Whether token-level CoT is meaningful under diffusion paradigms is unclear. Alternative strategies, such as VAR should be discussed.**\n\nThank you for raising this important point. **As illustrated in Line 27, the focus of our paper is the autoregressive text-to-image generation**, where the images are generated token by token in a discrete space. \n\n+ **For the diffusion-based models**, they employ iterative denoising in continuous space that are totally different from our settings. While some notable parallels exist (e.g., the generation process of diffusion model resembles a type of CoT happening in the noise space instead of the token space), this would require a complete theoretical reformulation. We will leave this for future work and will further clarify our focus in the introduction.\n\n+ **As for VAR**, we believe our method could be directly applied. Techinically, VAR also adopts an autoregressive generation manner and the generation image encoder is also discrete.  The only difference is that VAR first generates the coarse tokens then the fine-grained ones, but this does not conflict with our motiavation or method. In fact, it is interesting that this hierarchical reasoning from the coarse structure to fine details perfectly matches the spirit of CoT reasoning. We will also include this discussion into the future work part in our final version.\n\n> **Q3: Role of RL should be justified. It's unclear whether RL is essential or whether similar gains could be achieved through supervised or hybrid optimization.**\n\nThanks for your advice. **We conduct a supervised finetuning (SFT) and a hybrid optimization (SFT then RL) to validate the advantage of direct RL training.**\n\n+ **SFT:** We first source 200K high-quality image-text pairs. Then we use a MLLM to generate a detailed thinking process based on the image and original short prompt, which is used as the semantic-level CoT in the SFT training. During training, we jointly train two tasks together:\n\n  1. Input: Image Prompt + Reasoning Instruction; Output: Semantic-level CoT\n  2. Input: Image Prompt + Semantic-level CoT + <img_start>; Output: Token-level CoT. \n\n  This design strictly follows the generation manner of T2I-R1 for a fair comparison. We train the model for one epoch with learning rate of 1e-5.\n\n+ **Hybrid Optimization:** After the SFT training, we follow the same setting of T2I-R1 to train the SFT model with reinforcement learning.\n\nThe results are shown below:\n\n||Color|Shape|Texture|Spatial|Non-Spatial|Complex|\n|-|-|-|-|-|-|-|\n|SFT|0.6385|0.4698|0.5429|0.1950|0.2987|0.3305|\n|Hybrid optimization|0.8003|0.5764|0.7012|0.3011|0.3082|0.3878|\n|T2I-R1 (RL)|0.8130|0.5852|0.7243|0.3378|0.3090|0.3993|\n\n**Key Findings:**\n\n+ **Both SFT and hybrid optimization fails to outperform training only with RL.** This likely stems from the model's existing proficiency of generating semantic-level and token-level CoT. The additional SFT process forces the model to learn from data that is misaligned with its current capabilities, resulting in minimal or even detrimental improvements.  Furthermore, SFT constrains the model's generation capabilities to match the training data distribution, reducing output diversity and limiting the exploration space during the subsequent RL phase.\n\n+ **The training cost of SFT and hybrid optimization are also larger than exclusive RL training.** We showcase the total training hours and the data that needs annotation. SFT requires annotating semantic-level CoT based on existing text-image pairs datasets while RL needs no data annotation.\n\n  ||Total Training Hours|Needed Annotation|\n  |-|-|-|\n  |SFT|4.3|Semantic-level CoT|\n  |Hybrid optimization|29.0|Semantic-level CoT|\n  |T2I-R1 (RL)|14.5|None|\n\nWe will include this experiment and the corresponding conclusion in our final version to stength the role of reinforcement learning.\n\n> **Q4: Most performance comes from semantic-level CoT. It remains unclear how much of this improvement comes purely from prompt enrichment, without involving RL or token-level reasoning. A fairer comparison would include a model with enriched prompts but no further optimization, especially in the context of diffusion-based generators.**\n\nThanks for your valuable advice. We want to address your concerns from three aspects:\n\n1. **Token-level CoT contributes substantially to performance improvement.** We show the benefit obtained from token-level CoT from two aspects:\n\n   1. **Qualitative Aspect (Figure 5)**\n\n      The ablation visualizations in Figure 5 clearly demonstrate token-level CoT's crucial role. . Comparing columns 2 (semantic-level CoT only) and columns 4 (semantic-level and token-level CoT), the improvement brought by the token-level CoT is obvious. For example, with the top left prompt *\"Show a plant that is asymbol of good fortune inIrish culture, and is knownfor its three-lobed leaves\"*:\n\n      + **Semantic-level only (Column 2)**: Correctly identifies \"clover\" but produces a flat, textureless image with poor lighting and plain white background\n      + **With token-level CoT  (Column 4)**: Generates vibrant colors, natural lighting, realistic textures, and professional depth-of-field that makes the subject prominent\n\n   2. **Quantitative Aspect (Table 4)**\n\n      We also illustrate the effectiveness of the token-level CoT quantitatively. \n\n      First, **for the compositional ability** in Tabel 4, it can be shown that only optimising the token-level CoT brings large improvement, with 20% on T2I-CompBench and 5% on WISE benchmark.\n\n      Second, **we provide a quantitative metric to measure the improvement of visual quality brought by the token-level CoT:**\n      \n      |Model|Semantic-level CoT|Token-level CoT|Aesthetic Score|\n      |-|-|-|-|\n      | Janus-Pro-7B |||5.41|\n      |-|$\\checkmark$|| 5.58|\n      |-||$\\checkmark$|5.68|\n      |T2I-R1|$\\checkmark$|$\\checkmark$|5.76|\n\n2. **Ablation of Prompt Enrichment.** \n\n   The qualitative results from Figure 9 to 12 in the appendix showcase that **a valid prompt enrichment (zero-shot semantic-level CoT in the figure) does not necessarily produce aligned images**. Without learned coordination between planning and generation, the model fails to follow the enriched prompt.  In contrary, by our optimization, T2I-R1 could leverage and understand this enriched prompt to generate satisying results.\n\n   The quantitative results from *Exp1* and *Exp2* have also shown that the improvement of our method has only partly come from the prompt enrichment. \n\n3. While we focus on autoregressive models, we understand the reviewer's interest in broader applicability. However, **testing prompt enrichment on diffusion models seems to provide very limited insights**. There are huge differences including theoretical foundations, architecture, and training method between these two kinds of models. This cross-paradigm results wouldn't validate or invalidate our autoregressive findings.\n\n   Nevertheless, we're open to discussing specific experimental designs if the reviewer believes certain diffusion experiments would strengthen our analysis."}, {"review_id": "marKJ729tr", "reviewer": "Reviewer_1t3d", "summary": "This paper introduces T2I-R1, the first reasoning-enhanced text-to-image generation model that applies chain-of-thought (CoT) reasoning to visual generation. This work bridges the gap between language model reasoning capabilities and visual generation, showing that explicit reasoning processes can significantly enhance image generation quality and prompt alignment.", "questions": "1. The computational overhead and complexity of the two-step generation process and ensemble reward system, which may limit practical deployment and accessibility. \n2. The work lacks insights into RL training and optimization. Overall, it appears to be a straightforward application of the GRPO algorithm to text-to-image tasks, lacking fundamental innovation.", "limitations": "No", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 2, "strengths_and_weaknesses": "Strengths:\n1. First to identify and formalize two distinct types of reasoning (semantic-level and token-level) that mirror human artistic creation process. The dual-level approach addresses both global planning and local detail generation.\n2. The proposed approach achieves significant quantitative improvements: +13% on T2I-CompBench, +19% on WISE benchmark.\n\nWeaknesses:\n1. The two-step generation process requires generating semantic COT, then image tokens, increasing inference time. Doesn't compare against using a dedicated LLM for planning + separate image generator\n2. Ensemble of multiple expert models is applied to prevent reward hacking. However, it requires careful tuning of multiple reward models and their combinations. The importance of each reward model is not clear in Table 3.\n3. Multi-stage training with RL makes reproduction challenging.\n4. The work lacks insights into RL training and optimization. Overall, it appears to be a straightforward application of the GRPO algorithm to text-to-image tasks, lacking fundamental innovation.", "was_revised": true, "final_justification": "Thanks for the clarifications. The authors solved most of my concerns. I am willing to raise my final rating to 4 (Borderline accept).", "rebuttal": "> **Q1: Two-step generation requires semantic-level CoT + image tokens, which increases inference time. Doesn't compare a dedicated llm for planning + seperate image generator.**\n\nThanks for you valuable advice. To address your concerns, we conducted additional experiments comparing our method against dedicated planning approaches and applying prompt enrichment on other backbones:\n||Prompt Generation Model|Image Generation Model|\n|-|-|-|\n|*Exp1*|Janus-Pro-7B|Janus-Pro-7B|\n|*Exp2*|Qwen2.5-7B-Instruct|Janus-Pro-7B|\n|*Exp3*|Qwen2.5-7B-Instruct|Show-o|\n\nWe show the results on T2IComp-Bench, the aesthetic score and the inference cost below:\n||Color|Shape|Texture|Spatial|Non-Spatial|Complex|Aesthetic|Memory (GB)|Latency (sec/sample)|\n|-|-|-|-|-|-|-|-|-|-|\n|Janus-Pro|0.6359|0.3528|0.4926|0.2061|0.3085|0.3559|5.41|15.847|6.67|\n|*Exp1*|0.6451|0.3773|0.5262|0.2298|0.3049|0.3578|5.31|16.153|7.92|\n|*Exp2*|0.7004|0.4111|0.5892|0.2522|0.3088|0.3597|5.30|30.501|8.32|\n|*Exp3*|0.6759|0.4700|0.5265|0.2603|0.3017|0.3082|-|-|-|\n|T2I-R1|0.8130|0.5852|0.7243|0.3378|0.3090|0.3993|5.76|16.657|8.57|\n\n**Key Findings:**\n\n+ **Beyond Prompt Enrichment**: While dedicated planning models (*Exp1-3*) show modest improvements over the baseline, all fall significantly short of T2I-R1's performance. This demonstrates that our method's effectiveness extends well beyond simple prompt enrichment and highlights the crucial role of our joint RL optimization framework.\n\n+ **Aesthetic Enhancement**: Our approach not only improves compositional capabilities across all categories but also enhances aesthetic quality (5.76 vs. baseline 5.41), indicating holistic image generation improvement.\n\n+ **Excellent Trade-off**: Our method demonstrates remarkable efficiency by adding only modest computational overhead—0.8 GB additional memory and 1.9 seconds per sample—when incorporating semantic-level CoT. With an average CoT length of just 97.1 tokens, our approach achieves a 16% average improvement across benchmarks. This efficiency significantly outperforms existing RL-enhanced reasoning methods that require thousands of reasoning steps to achieve comparable performance gains (as demonstrated in Figures 2 and 3 of the DeepSeek-R1 paper [1]).\n\nWe will include results from these three experiments in the final version to provide a more comprehensive comparison.\n\n[1] Guo, Daya, et al. \"Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning.\" arXiv preprint arXiv:2501.12948 (2025).\n\n> **Q2: Ensemble of multiple expert models requires careful tuning of multiple reward models and their combinations. The importance of each reward model is not clear in Table 3.**\n\nThanks for pointing out! In Table 3, the weight of all the rewards are set to 1. Our preliminary experiment shows the weight of the reward model causes little influence to the final training result. Here **we provide a detailed study to illustrate the our method is stable to the weight of the reward model.** We follow the training setting of T2I-R1, where three reward models, HPS, GroundingDINO, and GIT are employed.\n\nIn our experiments, we multiply the weight of each reward model by 5 respectively, while maintaing the weight of other two reward models. Apart from these three experiments, we also conduct an experiment where we compute the relative reward of each reward model inside the group, and then sum up the normalized reward as the final reward for the sample. This eliminates the mean and variance difference among the reward models (termed as *Normalized Reward*). The results are shown below:\n|| Color | Shape | Texture | Spatial | Non-Spatial | Complex |\n|-|-|-|-|-|-|-|\n| *5\\*HPS, 1\\*others*| 0.8215 | 0.5915 | 0.7337 | 0.3051 | 0.3074 | 0.4051 |\n| *5\\*GDINO, 1\\*others*| 0.7951 | 0.5520 | 0.7100 | 0.3313 | 0.3104 | 0.3916 |\n| *5\\*GIT, 1\\*others*| 0.7972 | 0.5620 | 0.7149 | 0.3357 | 0.3111 | 0.3934 |\n| *Normalized Reward* | 0.8106 | 0.5820 | 0.7142 | 0.2940 | 0.3072 | 0.3993 |\n| T2I-R1 (*Equal Weights*)| 0.8130 | 0.5852 | 0.7243 | 0.3378 | 0.3090 | 0.3993 |\n\nThe result showcases that the training procedure is relatively stable to the weights of the reward model, with an average range of 2.51% fluctuation. We acknowledege that the choice of the model weights are of great importance and we will study how to adaptively find the best weights in our future work.\n\n> **Q3: Multi-stage training with RL makes reproduction challenging.**\n\nSorry for the confusion. We would like to clarify that the stage in the paper does not mean different training phases. **Our method requires only one stage of training.** The step shown in the Fig. 3 refers to the sequential generation process of semantic-level CoT and token-level CoT. Detailedly, since the baseline model Janus-Pro-7B does not possess the ability to generate the image start token autonomously, we employ a two-step generation method. The two steps are:\n\n1. First step: we input the image prompt and the reasoning insturction to generate the semantic-level CoT.\n\n2. Second step: we input the image prompt, previously generated semantic-level CoT and the image start token to obtain the token-level CoT.\n\nWe will further clarify this concept and replace the word stage with step to avoid causing confusion in the final draft. \n\n> **Q4: The work lacks insights into RL training and optimization. Overall, it appears to be a straightforward application of the GRPO algorithm to text-to-image tasks, lacking fundamental innovation.**\n\nWe respectfully disagree with the assessment for the following reasons:\n\n1. **Our BiCoT-GRPO method provides the first insight into leveraging GRPO to jointly optimize the CoT from both the text and image modalities in a single iteration.**\n2. Besides, **we propose a novel ensemble of reward models for the reward modeling in the GRPO training**. Previous methods only adopt GRPO in the understanding tasks, where the reward functions are mostly rule-based and format rewards. We first study the reward functions in GRPO for the image generation tasks. Our ensemble approach combines multiple reward perspectives to deliver holistic evaluation while preventing reward hacking.\n3. Finally, as illustrated in Line 2-9, the main focus of our paper is to explore how what types of CoT process could be utilized in the text-to-image generation task. We identify that RL is an ideal training method for eliciting the reasoning capability of the base model to generate two types of CoT. Here, **GRPO is simply the tool that enables our investigation, much like how supervised learning enables many fundamental discoveries without being the focus itself.**\n\n> **Q5: The computational overhead and complexity of the two-step generation process and ensemble reward system, which may limit practical deployment and accessibility.**\n\nThanks for your valuable advice. We address each aspect below:\n\n1. **The semantic-level CoT generation introduces minimal computational overhead**, as demonstrated in the comparison below:\n\n||Training Time Per Step (sec/step)|Inference GPU Memory (GB)|Inference Latency (sec/sample)|\n|-|-|-|-|\n| Token-level CoT only| 11.4| 15.847| 6.67 |\n| Qwen2.5-7B (planning) + Janus-Pro-7B (generation) |-| 30.501| 8.32 |\n| T2I-R1 (Semantic-level CoT + Token-level CoT)|16.3|16.657|8.57|\n\nGiven the substantial improvement across benchmark results, this minor overhead is practically acceptable compared to the gains achieved.\n\n2. **The ensemble of the reward model is only used in the training time and cost-free for the inference.**\n3. Compared to alternative two-step generation approaches that require separate models (e.g., a dedicated LLM for planning plus an independent image generator), **our unified approach requires only a single model, effectively reducing the GPU memory requirements by half during inference**.\n\n> **Q6: The work lacks insights into RL training and optimization. Overall, it appears to be a straightforward application of the GRPO algorithm to text-to-image tasks, lacking fundamental innovation.**\n\nPlease refer to our reponse to Q4."}, {"review_id": "drruprKkVy", "reviewer": "Reviewer_FgvF", "summary": "This paper introduces T2I-R1, a novel text-to-image generation model that pioneers a bi-level Chain-of-Thought (CoT) reasoning process. By effectively decoupling high-level semantic planning from low-level, patch-by-patch synthesis and optimizing both via a robust reinforcement learning framework (BiCoT-GRPO), the authors present a highly effective and well-validated method. The work's core concept is innovative, its experimental results are compelling—achieving state-of-the-art performance on multiple benchmarks.", "questions": "# Questions\n1. About Semantic-Level CoT\n\nFor simple, unambiguous prompts (e.g., 'a red apple'), could the authors discuss how the model avoids 'over-planning'—where the generated textual details might unnecessarily constrain the model's stochastic creativity in aspects like lighting, texture, and composition?\n\n2. About the analogy on 'Token-Level CoT'\n \nThe paper terms the patch-by-patch generation process 'Token-Level CoT,' drawing an analogy to textual CoT. Could the authors further elaborate on what distinguishes this process from standard autoregressive generation, beyond the inherent sequential dependency? For instance, textual CoT often involves intermediate steps that are human-interpretable and logically verifiable. How does this core aspect of 'thought' translate to the token-level visual generation? A deeper conceptual clarification would help solidify the paper's contribution in this regard.\n\n3. About Visual Quality\n\nTo further strengthen the analysis on the impact of BiCoT-GRPO on Visual Quality, could the authors provide more details on the human study protocol? The paper's primary contribution is enhancing compositional reasoning. Could the authors offer a more explicit discussion on how the BiCoT-GRPO framework specifically impacts aesthetics as a distinct objective? For example, is there a risk that the RL process might overly prioritize compositional correctness at the expense of aesthetic quality in some cases, and how does the ensemble reward system mitigate this potential conflict?", "limitations": "yes", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 4, "significance": 3, "originality": 3, "strengths_and_weaknesses": "# Strengths\n1. **Novel Concept**: Introduces an innovative \"bi-level Chain-of-Thought\" (CoT) framework.\n2. **Thorough Experiments**: Features comprehensive ablation studies validating each component.\n3. **Smart Reward System**: Uses a robust ensemble of \"vision experts\" to guide RL training effectively.\n\n# Weaknesses\nThe term \"Token-Level CoT\" for the standard autoregressive, patch-by-patch generation process can be seen as a loose analogy. While the authors define it based on its sequential and conditional nature the \"thought\" aspect is less explicit compared to the semantic-level CoT. Since there is no explicit thinking process, just a patch-by-patch image generation process.", "was_revised": true, "final_justification": "I appreciate the authors' clarifications. I think the quality of this paper is sufficient for acceptance at NeurIPS.  I will therefore maintain my positive rating.", "rebuttal": "> **Q1 & Q3: The term \"Token-Level CoT\" for the standard autoregressive, patch-by-patch generation process can be seen as a loose analogy. While the authors define it based on its sequential and conditional nature the \"thought\" aspect is less explicit compared to the semantic-level CoT. Since there is no explicit thinking process, just a patch-by-patch image generation process. & How does this core aspect of 'thought' translate to the token-level visual generation? A deeper conceptual clarification would help solidify the paper's contribution in this regard.**\n\nThank you for your valuable advice. We acknowledge that our terminology extends the CoT concept beyond its traditional textual reasoning context. We would like to clarify the concept of token-level CoT and address your concerns:\n\n1. First, **we believe there must exist a type of reasoning in the process of image generation**, otherwise, we will not observe model with only token-level CoT optimized beats the baseline model on the same prompt or even in the setting where the top half image is given and the model is asked to generate the remaining ones.\n\n2. Then, **we will revisit the definition of CoT in the textual context, and lists where the autoregressive image generation bears the similarity.** \n\n   For the textual CoT, it is defined as the intermediate natural language reasoning steps that lead to the final output in [1]. The CoT contains a long reasoning process with problem analysis and logical inference, then followed by a short final answer. It is commonly acknowledged that the part of the long output corresponds to the part of extensive thinking. The 'thought' process is reflected in two ways: \n\n\n   + Decomposition of the problem to steps\n   + Logical coherence within the step \n\n\n   Since the model outputs the tokens in the language space, this process can be easily interpreted by the human. The two aspects correspond to the logical path of solving and the correctness of each reasoning step.\n\n\n   **For the token-level CoT, the key insight is that the model's \"thought process\" or intermediate reasoning occurs simultaneously with final output generation, rather than as separate steps.** The model reasons and generates in parallel, integrating its thinking directly into the creation process. Different from textual CoT, the token-level CoT is much longer compared and also holds a strict sequential and conditional nature. The two 'thought' process become:\n\n\n   + The logical path of solving is determinant in the context of token-level CoT, which is the raster-scan order. \n   + The correctness of each reasoning step corresponds to the choice of specific visual tokens, which is what our method tries to optimize. This correctness includes patch coherence, object appearance, lighting conditions, and other visual details. These intermediate steps could also be verified and interpreted by humans by decoding the visual tokens into images.\n\n3. In summary, the modality difference between autoregressive text and image generation causes the \"thought\" happens in the different place. We draw an analogy between the two key aspects between the textual CoT and token-level CoT. **The token-level CoT refers to the 'thought' process we identify in the autoregressive generation.** We focus on optimizing the aspect on choosing the correct visual token.\n\nWe will include this clarification and comparison in our final version.\n\n[1] Wei, Jason, et al. \"Chain-of-thought prompting elicits reasoning in large language models.\"\n\n> **Q2: About Semantic-level CoT: unambiguous prompts (e.g., 'a red apple'), could the authors discuss how the model avoids 'over-planning'—where the generated textual details might unnecessarily constrain the model's stochastic creativity in aspects like lighting, texture, and composition?**\n\nThank you for this insightful question. We would like to address your concerns both qualitatively and quantitatively:\n\n1. **Qualitatively (Figure 13 to 14)**\n\n   Our experiment analysis in Figure 13 and 14 in the appendix reveals that **our introduced semantic-level CoT actually enhances the diversity of the generated images**. As shown in Figure 13, the base model Janus-Pro possess relatively low diversity. The generated images of Janus-Pro are basically the same: there is a tree branch on the bottom left corner and color of the origami cranes are all white with same zoom-in scale. While T2I-R1 generates much more diverse images with different backgrounds, cranes colors, and zoom-in scales. The same conclusion holds for Figure 14.\n\n2. **Quantitatively**\n\n   **We further conduct an experiment to validate this conclusion.** We generate 300 unambiguous prompts based on the example *'a red apple'*. We enumerate 150 common objects and 10 colors and then randomly choose 2 colors to combine with each object. Then we use the base model Janus-Pro, the model trained only with the semantic-level CoT, and T2I-R1 to generate 10 images on each prompt. Here we include the model trained only with the semantic-level CoT to eliminate the potential influence on the diversity from the token-level CoT. The results are as follows:\n\n   |Model|Diversity|\n   |-|-|\n   |Janus-Pro-7B|7.460|\n   |Only Semantic-level CoT optimized|8.612|\n   |T2I-R1|8.519|\n\n   The results clearly demonstrate that **semantic-level CoT increases output diversity by 15.4% instead of harming it**.\n\n> **Q4: To further strengthen the analysis on the impact of BiCoT-GRPO on Visual Quality, could the authors provide more details on the human study protocol? The paper's primary contribution is enhancing compositional reasoning. Could the authors offer a more explicit discussion on how the BiCoT-GRPO framework specifically impacts aesthetics as a distinct objective? For example, is there a risk that the RL process might overly prioritize compositional correctness at the expense of aesthetic quality in some cases, and how does the ensemble reward system mitigate this potential conflict?**\n\nThanks for this important question regarding the balance between the compositional generation and aesthetic quality.\n\n1. **Human Study Protocol**\n\n   We ask the annotators to judge only according to the visual appeal. We provide examples of an unsatisfactory generation, including different aspects:\n\n   + Corrupted floating words \n   + Distortion like melting effect of faces, hands, or other objects\n   + Extra or missing hands, legs, tails, and etc.\n   + Object merging, frequently observed for compositional prompts\n   + Low-level aspects including over exposure or over saturatation\n   + Blurred areas, especially in fine details like keyboard\n\n   The instruction to the annotator highlights that the highly ranked images should show no or less problems above compared with others.   \n\n2. **Aesthetics Quality**\n\n   We would like to address your concern of the aesthetics of the generated images:\n\n   **HPS score contains extra judgment based on the human aesthetic preferences**, as shown in Fig. 4 and Line 167. For the training data of HPS [1], the model is trained on the human preference data, which contains both the alignment and visual appealing aspects. Therefore, **the images generated by T2I-R1 actually shows better visual quality compared with the baseline model**.\n\n   We employ a different image quality judgement model to score the generated images from both Janus-Pro and T2I-R1 to further showcase the superior visual quality:\n\n   |Model|Aesthetic Score|\n   |-|-|\n   |Janus-Pro-7B|5.41|\n   |T2I-R1|5.76|\n\n   Specifically, we employ a CLIP-based aesthetic predictor, which receives the image and outputs a score. The predictor does not require prompt input so that the score is purely based on the visual appealing. We input all of the generated images when testing T2IComp-Bench to this predictor and record the average score. The result showcases that T2I-R1 improves the generation quality of the baseline model.\n\n[1] Wu, Xiaoshi, et al. \"Human preference score: Better aligning text-to-image models with human preference.\" Proceedings of the IEEE/CVF International Conference on Computer Vision. 2023."}]

Author rebuttal:

Meta-review: This paper introduces T2I-R1, a reasoning-enhanced text-to-image generation model integrating reinforcement learning with bi-level Chain-of-Thought (CoT) reasoning: semantic-level for high-level planning and token-level for low-level generation, optimized via BiCoT-GRPO. Reviewers praised its novel bi-level framework, effective reward ensemble, and significant performance gains (13% on T2I-CompBench, 19% on WISE). However, concerns were raised: under-discussed connections to prior prompt enrichment work, ambiguity in token-level CoT’s relevance to diffusion models, RL’s necessity, computational overhead, and limited insights into reward model tuning. The authors addressed key concerns in their rebuttal with clarifications and additional experiments. Despite minor limitations, T2I-R1 makes valuable contributions by bridging language reasoning and visual generation, with strong empirical support. The AC concurs this is a worthwhile advancement and recommends acceptance.
Decision: Accept (poster)