HermesFlow: Seamlessly Closing the Gap in Multimodal Understanding and Generation
Authors: Ling Yang
Xinchen Zhang
Ye Tian
Shiyi Zhang
Chenming Shang
Minghao Xu
Wentao Zhang
Bin CUI
Venue: NeurIPS 2025 poster ()
Keywords: Unified Multimodal Understanding and Generation
Primary area: deep_learning

Abstract: The remarkable success of the autoregressive paradigm has made significant advancement in Multimodal Large Language Models (MLLMs), with powerful models like Show-o, Transfusion and Emu3 made notable strides in unified image understanding and generation. For the first time, we uncover a common phenomenon: the understanding capability of MLLMs is usually stronger than their generative capability, with a significant gap between them. Building on this insight, we propose HermesFlow, a simple and general framework designed to seamlessly bridge the gap between understanding and generation in MLLMs. Specifically, we take the homologous data as input to curate homologous preference data of both understanding and generation. Through Pair-DPO and self-play iterative optimization, HermesFlow effectively aligns multimodal understanding and generation using homologous preference data. Extensive experiments demonstrate the significant superiority of our approach over prior methods, particularly in narrowing the gap between multimodal understanding and generation. These findings highlight the potential of HermesFlow as a general alignment framework for next-generation multimodal foundation models.

HermesFlow: Seamlessly Closing the Gap in
Multimodal Understanding and Generation
Ling Yang! !+t Xinchen Zhang?*
Ye Tian
Shiyi Zhang? Chenming Shang?
Minghao Xu Wentao Zhang
Bin Cui
Princeton University
Tsinghua University
3 Peeknng University
https://g hhb.bb...t.t...// com/Gen oerse/o
Abstract
The remarkable success the autoregressive paradigm has made significant ad- vancement in Multimodal Large Language Models (MLLMs), with powerful mod- els like Show-o, Transfusion and Emu3 achieving notable progress in unified image understanding and generation. For the first time, uncover common phenomenon: the understanding capabilities of MLLMs are typically stronger than their generative capabilities, with significant gap between the Building this insight, we propose HermesFlow, a simple yet general framework designed seamlessly bridge the gap between understanding and generation in MILMs. Specifically, we take the homologous data as input to curate homologous prefer- ence data of both understanding and generation. Through Pair-DPO and self-play iterative optimization, HermesFlow effectively aligns multimodal understanding and generation using homologous preference data. Extensive experiments demon- strate the sigmificant superiority of our approach over prior methods, particularly in narrowing the gap between multimodal understanding and generation. These findings highlight the potential of HermesFlow as general alignment framework for next-generation multimodal foundation models.
Introduction
The rapid advancement of Large Language Mod- els (LLMs) [24] 9 55
74] has driven sig- nificant development in both multimodal un- derstanding [22
53
f
MLLM
und
Preference cota
MLLM
Prefererce desa
72 19 and autoregressive im- age generation
Recent studies focused on developing unified system capable of both multimodal un- derstanding and generation. Powerful Multi- modal Large Language Models (MLLMs) like Show-o 48 Transfusion [69], and Emu3 41. employ a single transformer to unify these tasks, demonstrating remarkable performance across both domains.
Training
Captson
Troining
36
18 45
Steratioe petintratin
MLLM
Und
CInage, Caption)
praferatce
Poir
MLLM Sensiiin
MLLMM
cratie
(c)
Figure 1: Architecture comparison between (a) DPO training improve multimodal understanding 771 M. (b) DPO training improve multimodal
Recently, there has been growing interest in ex- ploring the synergy between multimodal under- standing and generation
generation [1 and (c) our HermesFlow.
45
40
3. Liquid
45 demonstrates that these two tasks are mutually
"Contributed equally.
'Corresponding authors.
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

0.646
Ieagel
Isage3
0.6
A = 0.169
0.6
0.599
0.182
Understanding Score
0.477
0.417
0.4
Prompt1: A hedgehog wreoped in a blanket, holding cup of hot chocolate, sitting "fireplace" rade pebbles.
MLLM
Generation Score
0.3
Understanding Generation VILA-U
0.3
Understanding Generation Janus
0.6
0.6
kitten sliding down an aqua stide. excited.
Proupt2:
44 -4-
Score
0.520
0.087
0.5-
0.533
4
0.036
0.433
Prompt3: blue Porsche 356 parked in front of yellow brick sall.
0.3
Understanding Generation
Understanding Generation Ours
Show-o
(a)
(b)
Figure 2: Motivation of HermesFlow. (a) A general pipeline to quantitatively assess the MLLM's performance multimodal understanding and generation. (b) The imbalance between understanding and generation capabilities is a common phenomenon in MLLMs, and our method significantly narrows this disparity. For detailed descriptions, please refer to Section 5.2
expanding the data for either understanding or generation enhances the performance the other. Furthermore, MetaMorph [40) reveals that understanding data is more effective than generation data in improving both understanding and generation performance. However, these works jointly improve the understanding and generation capabilities of MILMs from a data-level perspective but fail to consider the gap between them. remains unclear whether capability gap exists between them.
beneficial:
Regarding both task difficulty and architectural constraints, there exists significant gap between multimodal understanding and generation. In terms of task difficulty, generation maps textual features visual domain, requiring both semantic accuracy and high-fidelity details. However, converting compressed text into high- gigh-dimensional visuals inherently loses features, making generation harder. In contrast, understanding tasks involve compressing rich visual information into textual domain, typically focusing on low-frequency semantics. It is important to clarify that visual understanding broad concept; in this paper, specifically refer to the basic feature understanding of natural images. Accordingly, generation is fundamentally more challenging. For architectural constraints, image generation struggles with autoregressive modeling because visual spatiality conflicts with sequential token prediction, yielding suboptimal results. In contrast, understanding tasks benefit from natural alignment with autoregressive text serialization. Thus, generation also encounters greater architectural challenges.
To quantitatively assess the performance of multimodal understanding and generation, we design general pipeline, as illustrated in Figure 2 (a). For any pretrained MLLM, imput consists of (image, prompt/caption) pairs. For understanding tasks, MLLM presented with multiple questions related to each image, and the final understanding score is calculated as the average accuracy of its answers. MLLM generates an image for each prompt, and these images are evaluated by posing the same set of questions using GPT-4o [16), with the final generation score calculated based on the average accuracy of GPT-4o's answers. We employed this pipeline evaluate multiple MLLMs. As demonstrated in 2 (b), unified models like VILA-U [47), Janus [44] and Show-o [88 exhibit notably stronger understanding capabilities compared to their generation capabilities. Our experiments highlight recurring phenomenon: MILMs consistently demonstrate superior understanding abilities over generation abilities, with a significant gap between them.
Figure
In the pretraining of MLLMs, simply increasing the training data for understanding or generation does not yield proportional improvements in both aspects 44), leaving significant gap between their understanding and generation capabilities. A truly unified MLLM should excel in both under- standing and generation, striking balance rather than favoring one over the other. However, current unified MLLMs struggle to achieve this equilibrium. To bridge the gap between understanding and generation in MILMs, we propose HermesFlow a self-improvement framework that collects paired understanding and generation preferences from homologous input data, and then employ novel Pair-DPO post-training framework seamlessly bridge the gap through the paired preference data. To curate understanding preference data, enable MLLM to generate multiple captions for single

---

input image and filter paired understanding preference data using BERT similarity scores. To curate generation preference data, we prompt MLLM generate multiple images from a single prompt and employ self-critic-like approach to evaluate the images through self-VQA scoring, thereby filtering and selecting the paired generation preference data. Finally, we design Pair-DPO for preference alignment of homologous paired data, and through iterative optimization to simultaneously and progressively reduce the gap between understanding and generation following the same approach. We achieve the self-improvement of both understanding and generation of MLLM without incorporating any external high-quality training data. As shown in Figure 2 (b), based on Show-o, HermesFlow not only reduces the gap between understanding and generation, but also improves both capabilities, demonstrating the necessity of bridging the gap os these two abilities.
We compare HermesFlow with previous work in Figure I and summarize our contributions as follows:
An insightful discovery regarding significant gap between the understanding and generation abilities of MLLMs, with understanding consistently outperforming generation.
We propose a general multimodal self-improvement framework, HermesFlow, using Pair- DPO based on homologous data to seamlessly close the gap between multimodal under- standing and generation.
Self-play iterative optimization paradigm is highly compatible with the multi-round en- hancement of MLLMs. HermesFlow has potential general alignment framework for next-generation multimodal foundation models.
Extensive qualitative and quantitative comparisons with previous powerful methods, such as Show-o, Janus and VILA-U, demonstrate the effectiveness and superiority of our method.
2
Related Work
2.1
Unified Multimodal Understanding and Generation
In recent years, growing number studies E
46
57
23
32 have explored unified multimodal
models capable of both visual understanding 110
51 and generation [65 52 Early 73 63 leveraged diffusion models as external tools, where MLLMs generate conditions for visual generation [54] 39) without having direct generative capabilities. For instance, DreamLLM B introduces learnable embeddings called dream queries, which encapsulate the semantics encoded by MILMss and serve conditions for the diffusion decoder. More recently, inspired by the success of autoregressive paradigms, many studies [36 48, 669 [26 [4 [59]
62
66
31
methods [3 40 6 35
1 61
have shifted focus representing and generating images using discrete visual tokens within single
transformer framework. For instance, Emu3 [41] is trained solely with next-token prediction on mixture of multimodal sequences using single transformer. Janus 44 separates visual encoding into distinct pathways for multimodal understanding and generation while maintaining unified transformer architecture. However, no existing research has focused on the relationship between the strengths of understanding and generation capabilities in MLLMs, which is essential for the balanced and sustainable development of these models.
2.2 DPO in Multimodal LLMs
enhances the performance of multimodal LLMs through the post- post-training process. In Figure | we categorize these approaches into three types. Some methods 770 II 60 utilize DPO to enhance understanding capability, as shown in Figure (a). For instance, CSR 71 enables the model to self-improve by iteratively generating candidate responses, evaluating the reward for each response, and curating preference data for finetuning. Other methods [4] improve the generation capability of MLLMs through DPO as illustrated in [ Emu3 41 generates a data pool and constructs a preference dataset through manual ranking, which is then used to optimize the model's generation capabilities via DPO. However, these models focus exclusively on enhancing either understanding or generation capabilities. In contrast, our approach uses Pair- DPO to effectively narrow the gap between the two, achieving mutual improvement.
Direct Preference Optimization (DPO) 64
56
53
3

---

BEAT Sun
Selt-Play Iterative Optintzation
sndyevoy
Little nedgevvg
Tterative ustate
Und
Multimodal LLM
Paired preferunce data
patri
Captions
Inages
hesgetig Lasas
Size capliee for image
blacker
Und & Gen
Multimodat LLM
padri
rargetos wropoed
Tagt Dusstions
Ing2 tues
Viai
Lonel
Pair-DPO Training
Und
Multimodat
LLM
Caption::
Hermesflow
Ing2 occs
0.8
8.6
Pipeline of HermesFlow. We begin by curating paired data that captures both understanding and generation preferences from homologous input data. Leveraging this homologous preference data, design Pair-DPO and employ self-play iterative optimization to seamlessly bridge the gap between multimodal understanding and generation.
Figure 3:
3
Preliminary
3.1 Next Token Prediction
Next token prediction is fundamental task in sequence modeling, where the goal is to estimate the conditional probability of the next token de given its preceding context &ec = {2i, ; 14-1
Formally, for sequence {01, 22. ..,,..,......................... FT the joint probability is factorized as:
P(x)
II P P( 21,
II P(xe(x<)) 1=1
This factorization relies on the autoregressive assumption, where each token depends solely on its preceding tokens. During training, the model optimized by minimizing the negative log-likelihood loss over the dataset:
1 L C= --
Σ log P(x.(ae))
In autoregressive models, next-token prediction facilitates sequential generation by iteratively sam- pling tokens from the learned distribution P(x This approach is widely applicable multimodal domains such as visual understanding and visual generation.
3.2 Direct Preference Optimization
Direct Preference Optimization (DPO) provides straightforward and efficient method by directly utilizing pairwise preference data to optimize the policy model. Specifically, given an input prompt x, and preference data pair (yw, M1), DPO aims to maximize the probability of the preferred output yo and minimize that of the undesirable output yr- The optimization objective is formulated as:
To( x)) Teet Tre |x)
To(un (x)
]
(f) - -E log G
3 log
Tiet (1
where D is the pair-wise preference dataset, is the sigmoid function, Te (- (=))) is the policy model to be optimized, (-e((-12)) (z) is the reference model kept unchanged during training. and the hyperparameter 3 controls the distance from the reference model.
4 Method
this section, we present our method, HermesFlow, which curates pairwise preference data for both multimodal understanding and generation using homologous images and prompts, and seamlessly bridging the gap of multimodal understanding and generation through Pair-DPO training. overview of HermesFlow is illustrated in Figure 3 In Section 4.1 detail the methods for

---

curating homologous preference data for multimodal understanding and generation, respectively. In Section 4.2 we propose the Pair-DPO training strategy bridge the gap between multimodal understanding and generation. In Section 4.3 we introduce self-play iterative optimization, enabling the self-impprvement of MI.LM over multiple iterations.
4.1 Curating Homologous Preference Data
Homologous Input Data The curation of both multimodal understanding and generation preference data begins with homologous data (I. where y represents the caption prompt of the image x.
Understanding Preference Data We focus on the image captioning task to collect understanding preference data, which reflects the ability of MLLMs capture visual features, including object attributes, spatial relationships, and detailed elements of both the subject and background. Give an image v, pretrained MLLM is used to generate different captions according the input prompt: "Give caption for this image.". ". We then calculate the BERT similarity scores [2] ( x) between the original prompt and each the n captions. The caption with the highest BERT similarity score is selected as the winning sample yu. while the one with the lowest score is chosen as the losing sample yu- Following this process, we construct the pairwise understanding preference data.
Generation Preference Data Starting with the caption or prompt y, we use the pretrained MLLM randomly generate n images. Given that MLLM's understanding abilities surpass its generation capabilities, we apply a self-critique or self-selection method for choosing the generated data.
Specifically, given the prompt y. use TIFA [14 to generate q visual question-answee answerr pairs, denoted as {(11, A1), (Q2, A2)...., (Q,, A,)) For each generated image, we evaluate them based on the accuracy of the VQA responses provided by the MILM:
Acc(o;) =
T( Ad), = 11=1,2.....
(4)
R3 = MLLM(Z,, Qi,i)
(5)
where Ry.s represents the response of MILM according to the input of image x, and question Qja- We select the image with the highest accuracy as the winning sample zw and the one with the lowest accuracy as the losing sample Hú, while also ensuring that the highest accuracy exceeds 0.6. Using this process, we construct the pairwise generation preference data.
Homologous Output Preference Data After curating understanding and generation preference data from homologous input (I, y) as mentioned above, where y represents the caption or prompt of the image v, we obtain the homologous output preference data D, denoted as (ar, y, Thoy Fi, Ywo y1)-
In this paper, we focus on exploring a preliminary method to achieve the self-improvement and self-alignment framework for understanding and generation in MLLMs, without the need for any external data as supervision. Although we curate preference data for understanding and generation using different methods, both are designed to capture the model's inherent biases accurately. These preference data do not represent two unrelated tasks, but rather offer two distinct perspectives on the same multimodal semantics.
4.2 Unified Enhancement with Pair-DPO
Homologous preference paired data of understanding and generation indicate the optimized directions for both capabilities of pretrained MLLM within the same semantic space. To achieve joint opti- mization and alignment of understanding and generation, we introduce Pair-DPO. The optimization objective of Pair-DPO can be formulated as:
Lpair-Do (0) = E. (... 2w) [log a (Aund
To (hh गea (Yw
x))
To (y)
x))
AUnd = B log
-3 log
(7)
x)
Tee (31 x)
no (Fer
4)
my (ar y)
AGen = 3 log
|3 log
(8)
neer Fe
1)
n (i) y)
5

---

Algorithm The pseudocode of HermesFlow
Input: Homologous data (I, v), pretrained model MILM,, with parameters 0
for =0 -....., iler do
2:
if O then
3:
Yuu Y1 = MLLM, (a) // Und preference data
4:
Two 21 MILM, 3 (y) Gen preference data
5:
else
6:
vi 1/. ................................ yh MLLM, (x)
7:
Vmnx = arg max.e(1.....? 8yy
8:
Update und-preference data using Equation (10)
9:
2 22.
MLLM3 (v)
10:
xinx = arg (I..e( Acc(at)
Update gen sen-preference data using Equation (10)
12:
end if
Optimize MLLM MILM using Equation
14:
end for
where AGen and Aund represent the preference differences in generation and understanding of MLLM, respectively. By using Pair-DPO to optimize homologous preference data jointly, we not only ensure mutual improvement in the understanding and generation capabilities of MLLM but also effectively narrow the gap between them. We provide the detailed derivation of the Pair-DPO optimization objective in Appendix
4.3 Self-Play Iterative Optimization
To achieve comprehensive optimization and achieve a convergence gap in understanding and genera- tion of MLLMs, we introduce novel yet easy self-play iterative optimization using Pair-DPO with multiple turns.
Take understanding preference data as an example. We denote the preference data curated in round i in Section 4. as (yt "/ 11). ). In the optimization of round i, the optimized MLLM generates n new captions (vi, v/....... ...) from the input of image x. The preference data is selected based on the following rules:
Vinx arg
max ke (I....))
s(yh. y)
[lmmax y (-11 if s y) > a(y'11,2)
(y vi)
(10)
(Vimax Vi otherwise
where s(yk. y) denotes the BERT similarity score between the generated caption vf and the homolo- gous input caption y. Select the caption Vinux with the highest similarity score, which represents the local upper bound the optimized MILM's understanding capability. If s y) > (yyi 1.)) .y), MLLM has effectively learned preference knowledge from the previous round. Therefore, it needs to be updated and further optimized using the higher-quality sample y'nxx as the benchmark. Conversely, if syyiaxx y) s(yi=1 1. v). effective optimization was not achieved in the previous round. In this case, it is necessary to update with simpler and clearer preference data yinx as the winning sample to provide smoother learning gradient. Through iterative optimization, achieve self-improvement of MLLM without relying on any external high-quality training data.
Experiments
5.1 Experimental Setup
33 as our Training Setup We randomly select 5,000 image-caption pairs from JourneyDB
homologous input data. For the Visual Question Answering (VQA) data corresponding to each pair, we combine the VQA from JourneyDB with the VQA generated from TIFA [14) for comprehensive evaluation. Our HermesFlow trained upon Show-o (8), using a batch size of for both caption and generation data over 3,000 steps. We employ the Adam optimizer with weight decay of 0.01,

---

Table 1: Evaluation multimodal understanding benchmarks. The baseline data is quoted from
Show-o 48
Model
Params
POPET
MMET
Flickr30kt
VQAv2nea1
GQAt
MMMUT
Gemini-N Nano-1
1.8B
26.3
Emu
13B
$7.2
NEXT-GPT 46
13B
84.5
66.7
SEED-X 6
17B
84.2
1435.7
52.3
47.9
35.6
Chameleon 36
34B
74.7
66.0
Show-o S8
1.3B
80.0
1232.9
67.6
74.7
61.0
27.4
HermesFlow (Ours)
1.3B
81.4
1249.7
69.2
75.3
61.7
28.3
Table 2: Evaluation on visual generation benchmarks: GenEval
7 and DPG-Bench
13
GenEvall Colors
DPG-Bencht
Methods
oparams
Single Obj.
Two Obj.
Counting
Position
Color Attr.
Overall
Average
Diffusion Model
0.23
0.37
DALL-E2
20
4.28
0.94
0.66
0.49
0.77
0.10
0.19
0.52
1.5
30
860M
0.94
0.37
0.27
0.72
0.05
0.07
0.40
63.18
SD 2.1
30
865M
0.97
0.50
0.46
0.80
0.07
0.14
0.49
68.09
Autoregnessivee
LlamaGin
775M
0.87
0.25
0.23
0.51
0.06
0.04
0.32
65.16
Emu 35
148
0.34
0.26
0.56
0.07
0.06
0.36
Chameleon
26
34B
0.89
0.39
0.28
0.66
0.08
0.07
0.40
LWM 21
7B
0.46
0.79
0.09
0.15
0.47
SRED-X 0
17R
0.97
0.58
0.26
0.80
0.19
0.14
0.49
Show 48
1.3B
0.77
0.58
0.81
0.23
0.44
0.64
67.48
Janus
1.3B
0.97
0.68
0.30
0.84
0.46
0.42
0.61
HermesFlow (Ours)
1.3B
0.98
0.84
0.66
0.82
0.32
0.52
0.69
70.22
and initial learning rate of 2e-5 with a cosine scheduling. The parameter / for Pair-DPO to 0.2. All experiments are conducted under NVVVIDIA A100 GPUs.
Notably, while use JourneyDB's prompts and images solely construct preference data, do not treat them as direct supervision targets. Instead, our supervision stems from preference pairs generated by the model itself, making HermesFlow self-improvemmet framework for MIL.Ms. The JourneyDB data merely serves as model input, while training and alignment are driven by the model's own output quality. This design enables the framework to work with arbitrary image-prompt pairs, enhancing its flexibility and applicability.
Evaluation Metrics To assess multimodal understanding capabilities, evaluate using POPE 20)) MME (5) Flickr30k (25). VQAv2 g GQA [15,, and MMMU [58]. For visual generation capabilities, GenEval ( and DPG-Bench [33 to evaluate the model's prompt-image aligmment. We further assess image fidelity with FID [12] and CLIP-Score (27). Additionally, we conduct a comprehensive user study objectively compare model with other baselines.
5.2 Main Results
Multimodal Understanding Performances Table summarizes the comparison between our method and other leading MLLMSs on multimodal understanding benchmarks. Notably, HermesFlow achieves similar or superior understanding performance compared to larger models like SEED-X and Chameleon, using less than 1/10 of the parameters. Additionally, HermesFlow demonstrates significant strengths across all metrics compared to Show-o, indicating that Pair-DPO effectively reduces the understanding-gert -enerrrttnnnnnnninnnnnnnno gap while maintaining or even enhancing understanding ability. For more qualitative examples on multimodal understanding, please refer to Section D
Image Generation Performances As shown Figure HermesFlow achieves superior generation results compared to three powerful Multimodal LLMs: VILA-U [7]. Janus 44 and Show-o [48 Compared to backbone, Show-o, HermesFlow demonstrates superior performance generating object attributes and accurate counting. This improvement stems from its stronger understanding

---

VILA-U
Janus
Show-o
Ours
VILA-U
Janus
Show-o
Ours
charmiing red bicycle with basket of Flowers adds splash of color to quaint cobo Lestone street In picturesque village.
picturesque Parisian cafe terrace offers stunning view of the Eiffel Tower franed by vibrant flowers and lush
Four dogs gather under the shade of grand tree, vintage clock stands elegantly on a rantel, Lanked by three softly gloving candless.
enbodying friendship and tranqui Lity in nature.
peaceful dinosaur roans anidst prehistoric paradise of cascading waterfalls and vibrant blooms.
A lush Japanese garden in spring, with blooming cherry blossons and tranquil koi pand.
A mysterious detective exanines cryptic letters by candlelight, deep in thought as shadows dance around the roon.
Standing tall ageinst the storny skies, the Lighthouse guides the way with its unavering beacon of hope.
Figure 4: Qualitative comparison between our HermesFlow and three outstanding Multimodal LLMs VILA-U [47], Janus (44), and Show-o 488. Colored text denotes the advantages of HermesFlow in generated images.
Table 3: MSCOCO zero-shot FID and CLIP-Score.
Method
Params
FID,
CLIP-Scoret
Ours Show-o
LDM 30
1.4B
12.64
69%
10%
21%
DALL- 20
6.5B
10.39
SD 1.5 30
860M
9.62
30.23
Ours vs VILA-U
69%
7%
244%
2.1
8.03
30.87
LlanaGen
77SM
9.45
29.12
Ours vs Chamelon
733
16%
Emu 35
28.98
80%
100%
LWM 21
12.68
SEED-X 6
17B
14.99
Show-o 48
1.3B
9.24
30.63
HermesFlow (Ours)
1.3B
9.07
31.08
Figure 5: Results of user study.
capabilities, which are utilized to filter generated images and achieve mutual refinement through Pair-DPO iteratively.
We compare HermesFlow with other visual generation models on GenEval 7 and DPG-Bench [
as shown in Table Compared to the diffusion- generative model SD 2.1 30), HermesFlow demonstrates remarkable performance across all benchmarks. Furthermore, surpasses larger autoregressive models, such Chameleon Rd and LWM [21]. When compared to Show-o [48]. HermesFlow cxhibits significant strengths in object counting and positions, this is attributed to the critique of its superior understanding capability, which greatly enhances its visual generation performance in aspects such as object quantity, location, and attributes. We present the zero-shot FID [2] and CLIP-Score ( of HermesFlow on MSCOCO-30K in Table 3 The results clearly show that after the iterative optimization with Pair-DPO, HermesFlow achieves improved performance in both image fidelity and prompt- alignment.
also conducted comprehensive user study to evaluate the effectiveness of HermesFlow in visual generation. As illustrated in Figure we randomly selected 25 prompts for each comparison, and invited 35 users from diverse backgrounds to vote on image generation quality, collecting total of 3,500 votes. Alignment between the generated images and the prompts was used as the

---

Table 4: Quantitative assess of MILM's Understanding and Generation Gap.
Method
Params
Understanding Score?
Generation Score t
Gap L
VILA-U 47
48
7B
0.646
0.477
0.169
Janus 44
1.3B
0,599
0.417
0.182
Show-o 48
1.3B
0.520
0.433
0.087
HermesFlov (Ours)
1.3B
0,533
0.497
0.036
Table 5: Comparison of Pair-DPO vs. DPO and the Effect of Pair- DPO Iterations.
Understanding Bench
Generation Bench
Methods
POPET
MMEt
MMMU1
GenEval (Overall)
DPG-Bench (Average)
Show-o 48
80.0
1232.9
27.4
0.64
67.48
DPO (Understanding)
80.8
1242.2
27.8
0.58
67.88
DPO (Generation)
80.5
1239.3
27.5
0.70
70.03
Pair-DPO (Iter. (Show-o)
80.0
1232.9
27.4
0.64
67.48
Pair-DPO (Iter.
81.1
1246.7
28.0
0,68
70.19
Pair-DPO (Iter.
81.3
1248.3
28.1
0.69
70.21
Pair-DPO (Iter. 3)
81.4
1249.7
28.3
0.69
70.22
primary evaluation criterion, with aesthetic quality and detail completeness considered under the same conditions. The results demonstrate that HermesFlow received widespread user approval in visual generation.
As shown in Figure 2 we homologous data consisting of caption/prompt y and image input to evaluate the capability of understanding and generation respectively. The homologous data randomly selected from JourneyDB 333 For the understanding task, to ensure comprehensive and high-quality question- answer (QA) pairs, we first use TIFA [4] to generate QA pairs based on the image and caption. These are then augmented with QA pairs from JourneyDB create more thorough and in-depth dataset. The final understanding score is calculated as the average accuracy of the answers. For the generation task, images are evaluated by posing the same set of questions to GPT-4o [16], with the final generation score determined by the average accuracy of GPT-4o's answers. Since the generation capabilities of MLLMs are relatively limited, strict evaluation criteria are applied in cases of severe object blurring or significant loss of details. Therefore, GPT-4o required carefully analyze the completeness and authenticity of the objects involved in each question before providing answers. This evaluation pipeline was applied to multiple MILMs, with the results presented in Table
Quantitative assess of MLLM's Understanding and Generation Gap
the prompt input generate image for each prompt. These generated
is clear that a significant gap exists between multimodal understanding and generation in MLLM. HermesFlow seamlessly bridges this gap through self-play iterative optimization using Pair-DPO from homologous preference data.
5.3 Ablation Study
Pair-DPO vs. DPO Pair-DPO can simultaneously enhance both the understanding and generation capabilities of multimodal LLMs. As shown in Table compared to DPO methods that rely solely on understanding or generation preference, single round of Pair-DPO achieves superior performance by jointly optimizing both capabilities through the use of multimodal preference data. Furthermore, we observed that when using preference data from only one modality, whether understanding or generation, the capability of the other modality also improves. This demonstrates the same findings MetaMorph 400] and Liquid 45) that multimodal understanding and generation are synergistic.
Self-play Iterative Optimization As shown in Table we conducted an experimental analysis to examine the impact of iterations in self-play iterative optimization. It evident that the first round of iterative optimization yields the most significant improvements in both understanding and generation. This is because the notable gap between the understanding and generation capabilities of MILMss is most effectively bridged in the initial iteration. When the number of iterations exceeds 2,

---

Und-MIE -IME
Und-Baseline Gen-DPCBench
Gen-DPGBench
Und Und-MMN MdM-MM
Gen-Baseline
Numbers of generated samples
Figure 6: Influence of the richness of each preference sample.
observed that understanding ability continues improve slightly, while generation ability remains almost stable. We argue that since generation is tine-grained visual task, crossccapability transfer has limited impact on further enhancing generation ability in subsequent iterations.
The Impact of Each Preference Sample Richness The performance of Pair-DPO is largely influenced by the number of generated samples n for both understanding and generation data. We conducted experiments to analyze the impact of n on both understanding and generation in MLLMs, with results shown in Figure 6 The dashed lines represent the performance of the baseline model, Show-o 488.
When n is too small, the model's understanding and generation performance decline. This is due to the insufficient number of samples and the limited capability of the baseline model, which leads to noisy preference dataset and significantly impacts the results. However, the sample size increases, enables more accurate identification the model's optimal local upper bound, which in turn facilitates the curation of higher-quality preference data, leading to noticeable improvements in the understanding and generation capabilities of MILMs.
Furthermore, Figure 6 reveals that achieving performance comparable to the baseline in generation requires more sampling data that understanding. This indicates that the generation capabilities of MLLMs are more sensitive to noise in the preference data, highlighting a greater need for high-quality generation data.
Conclusion
In this paper, we present new MLLM alignment paradigm, HermesFlow, seamlessly bridge the gap between multimodal understanding and generation. By iterative optimized with Pair-DPO using homologous preference data, HermesFlow successfully Improve the capabilities of both multimodal understanding and generation while narrowing the gap between them. Our extensive empirical evaluations across diverse understanding and generation benchmarks demonstrate the effectiveness of HermesFlow. However, due to current limitations in the number and capabilities of open- open-source MLLMs, HermesFlow has not yet been optimized across wider range of backbones. FlomesFlow has the potential as general alignment framework for next-generation multimodal foundation models. In the future, we plan to extend this framework: (i). Extending HermesFlow to diffusion foundation models like MMaDA [52) to evaluate its generalization and performance various backbones; (ii) Combining with more advanced RL algorithms 44 to enhance multimodal inteligence for different scenarios.
Acknowledgement
This work is supported by National Natural Science Foundation of China (U22B2037), Beijing Municipal Science and Technology Project (7221 100010323002, research grant No. SH-2024JK29, PKU- AKU-Tencent joint research Lab, and High-performance Computing Platform of Peking University.
10

---

References
[1] Gong Cheng, Pujian Lai, Decheng Gao, and Junwei Han. Class attention network for image recognition. Science China Information Sciences, 66(3):132055, 2023.
[2] Jacob Devlin. Bert: Pre-training of deep bidirectional transformers for language understanding, arXiv preprint arX1v:1810004805, 2018.
[3] Runpei Dong, Chunrui Han, Yuang Peng, Zekun Qi, Zheng Ge, Jinrong Yang. Liang Zhao, Jianjian Sun, Hongyu Zhou, Haoran Wei, et al. Dreamllm: Synergistic multimodal comprehension and creation. arXiv preprint arXiv: 1499, 2023.
[4] Lijie Fan, Tianhong Li, Siyang Qin, Yuanzhen Li, Chen Sun, Michael Rubinstein, Deqing Sun, Kaiming He, and Yonglong Tian. Fluid: Scaling autoregressive text-to-image generative models with continuous tokens. arXiv preprint 24Xi::2400.18600 2024.
[5] Chaoyou Fu, Peixian Chen, Yunhang Shen. Yulei Qin, Mengdan Zhang, Xu Lin. Jinrui Yang, Xiawu Zheng. Ke Li, Xing Sun, et al. Mme: A comprehensive evaluation benchmark for multimodal large language models. arXiv preprint arXiv:2306. 13394, 2023.
[6] Yuying Ge, Sijie Zhao, Jinguo Zhu, Yixiao Ge. Kun Yi, Lin Song, Chen Li, Xiaohan Ding, and Ying Shan. Seed-x: Multimodal models with unified multi-granulee gulti-granularity comprehension and generation. arXiv preprint arXiv:2404. 14396. 2024.
[7] Dhruba Ghosh, Hannaneh Hajishirzi, and Ludwig Schmidt. Geneval: An object-focused framework for evaluating text-to-image alignment. Advances in Neural Information Processing Systems, 36, 2024.
[8] Yash Goyal, Tejas Khot, Douglas Summers-Stay, Dhruv Batra, and Devi Parikh. Making the in vqa matter: Elevating the role of image understanding in visual question answering. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 6904-6913, 2017.
[9] Daya Guo, Dejian Yang. Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, et al. Deepscek-rl: Incentivizing reasoning capability in Ilms via reinforcement lcarning. arXiv preprint arXiv:2501. 2..2:2250...948,. 2025.
[10] Dong Guo, Faming Wu, Feida Zhu, Fuxing Leng, Guang Shi, Haobin Chen, Haoqi Fan, Jian Wang. Jianyu Jiang, Jiawei Wang, al. Seed1. 5-vl technical report. arXiv preprint arXiv:2505.07063 2025.
[11] Jiayi He, Hehai Lin, Qingyun Wang, Yi Fung, and Heng Ji. Self-correction is more than refinement: A learning framework for visual and language reasoning tasks. arXiv preprint 2rXiv:24/0.0400) 2024.
[12] Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter. Gans trained by two time-scale update rule converge to a local nash equilibrium. Advances in neural information processing systems, 30, 2017.
[13] Xiwei Hu, Rui Wang, Yixiao Fang. Bin Fu, Pei Cheng, and Gang Yu. Ella: Equip diffusion models with Il for enhanced semantic alignment. arXiv preprint arXiv:2403.05// 2024.
[14] Yushi Hu, Benlin Liu, Jungo Kasai, Yizhong Wang, Mari Ostendorf, Ranjay Krishna, and Noah Smith. Tifa: Accurate and interpretable text-to-image faithfulness evaluation with question answering. In Proceedings of the IEFF/CVF International Conference on Computer Vision, pages 20406-20417, 2023.
[15] Drew A Hudson and Christopher D Manning. Gqa: A new dataset for wallwwrldd visual reasoning and compositional question answering. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 6700-6709, 2019.
[16] Aaron Hurst, Adam Lerer, Adam P Goucher, Adam Perelman, Aditya Ramesh, Aidan Clark, AJ Ostrow, Akila Welihinda, Alan Hayes, Alec Radford, et al. Gpt-4o system card. arXiv preprint 211222240.... 2024.
[17] Jing Yu Koh, Daniel Fried, and Russ R Salakhutdinov. Generating images with multimodal language models. Advances in Neural Information Processing Systems, 36:21487-21506, 2023.
[18] Hao Li, Changyao Tian, Jie Shao, Xizhou Zhu, Zhaokai Wang, Jinguo Zhu, Wenhan Dou, Xiangang Wang, Hongsheng Li, Lewei Lu, et al. Synergen-vl: Towards synergistic image understanding and generation with vision experts and token folding, arXiv preprint 24
2024.
[19] Junnan Li, Dongxu Li, Silvio Savarese, and Steven Hoi. Blip-2: Bootstrapping language -magg pre- me-training with frozen image encoders and large language models. In International conference on machine learning, pages 19730-19742. PMLR, 2023.

---

[20] Yifan Li, Yifan Du, Kun Zhou, Jinpeng Wang. Wayne Xin Zhao, and Ji- Ji-Rong Wen. Evaluating object hallucination in large vision-language models. arXiv preprint arXiv:2305.10335 2023.
[21] Hao Liu, Wilson Yan, Matei Zaharia, and Pieter Abbeet. World model million-length lenghh video and language with blockwise ringattention. CoRR, 2024.
[22] Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. Visual instruction tuning. Advances in neural information processing systems, 36, 2024.
[23] Yiyang Ma. Xingchao Liu, Xiaokang Chen, Wen Liu, Chengyue Wu, Zhiyu Wu, Zizheng Pan, Zhenda Xie, Haowei Zhang, Liang Zhao, et al. Janusflow: Harmonizing autoregression and rectified flow for unified multimodal understanding and generation. arXiv preprint (rXxv:2411007777 2024.
[24] OpenAI. Openai ol system card. preprint, 2024.
[25] Bryan A Plummer, Liwei Wang, Chris M Cervantes, Juan C Caicedo, Julia Hockenmaier, and Svetlana Lazebnik, Flickr30k entities: Collecting region-to-phrase correspondences for richer image-to-sentenee models. In Proceedings of the IEEE international conference on computer vision, pages 2641-2649, 2015.
[26] Liao Qu, Huichao Zhang, Yiheng Liu, Xu Wang. Yi Jiang. Yiming Gao, Hu Ye, Daniel K Du, Zehuan Yuan, and Xinglong Wu. Tokenflow: Unified image tokenizer for multimodal understanding and generation. arXiv preprint arXiv:2412.03069, 2024.
[27] Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transfcrable visual models from natural language supervision. In International conference on machine learning, pages 8748-8763, PMLR, 2021.
[28] Rafael Rafailov, Archit Sharma, Eric Mitchell, Christopher D Manning, Stefano Ermon, and Chelsea Finn. Direct preference optimization: Your language model is secretly a reward model. Advances in Neural Information Processing Systems, 36, 2024.
[29] Aditya Ramesh, Prafulla Dhariwal, Alex Nichol, Casey Chu, and Mark Chen. Hierarchical text-conditional image generation with clip latents. arXiv preprint arXiv:2204.06123 06125, 1(2)3, 2022.
[30] Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bjöm Ommer. High-resolution image synthesis with latent diffusion models. In Proceedings of the IEFF/CVF conference on computer vision and pattern recognition, pages 10684-10695, 2022.
[31] Chufan Shi, Haoran Yang. Deng Cai, Zhisong Zhang, Yifan Wang, Yujiu Yang, and Wai Lam, A thorough examination of decoding methods in the era of llms. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 8601-8629, 2024.
[32] Weijia Shi, Xiaochuang Han, Chunting Zhou, Weixin Liang, Xi Victoria Lin, Luke Zettlemoyer, and Lili Yu. Llamafusion: Adapting pretrained language models for multimodal generation. arXiv preprint arXiv:2412.15188, 2024.
[33] Keqiang Sun. Junting Pan, Yuying Ge. Hao Li. Haodong Duan, Xiaoshi Wu, Renrui Zhang, Aojun Zhou, Zipeng Qin, Yi Wang, et al. Journydh:: A benchmark for generative image understanding. Advances in Neural Information Processing Systems, 36, 2024.
[34] Peize Sun, Yi Jiang, Shoufa Chen, Shilong Zhang, Bingyue Peng, Ping Luo, and Zehuan Yuan. Autore- gressive model beats diffusion: Llama for scalable image generation. arXiv preprint arXiv:2406.0r 065255 2024.
[35] Quan Sun, Yufeng Cui, Xiaosong Zhang, Fan Zhang, Qiying Yu, Yueze Wang. Yongming Rao, Jingjing Liu, Tiejun Huang, and Xinlong Wang. Generative multimodal models are in-context learners. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 14398-14409, 2024.
[36] Chameleon Team. Chameleon: Mixed- modal early-fusion foundation models.
arXiv preprint
arXiv:2405.0983 2024.
[37] Gemini Team, Rohan Anil, Sebastian Borgeaud, Jean- can-BBatiste Alayrac Jiahui Yu, Radu Soricut, Johan Schalkwyk, Andrew M Dai, Anja Hauth, Katie Millican, et al. Gemini: a family of highly capable multimodal models. arXiv preprint arXiv:2312. 11805, 2023.
[38] Keyu Tian, Yi Jiang, Zehuan Yuan, Bingyue Peng, and Liwei Wang. Visual autoregressive modeling: Scalable image generation via next-scale prediction. arXiv preprint 2rXiv:2404.02905. 2024.
12

---

[39] Ye Tian, Ling Yang. Haotian Yang, Yuan Gao. Yufan Deng, Jingmin Chen, Xintao Wang, Zhaochen Yu. Xin Tao, Pengfei Wan, et al. Videotetris: Towards compositional text-to-video generation. arXiv preprint urXiv:2406.04271 2024.
[40] Shengbang Tong, David Fan, Jiachen Zhu, Yunyang Xiong. Xinlei Chen, Koustuv Sinha, Michael Rabbat, Yann LeCun, Saining Xie, and Zhuang Liu. Metamorph: Multimodal understanding and generation via instruction tuning. arXiv preprint arXiv:2412. 14164, 2024.
[41] Xinlong Wang. Xiaosong Zhang, Zhengxiong Luo, Quan Sun, Yufeng Cui, Jinsheng Wang. Fan Zhang, Yueze Wang, Zhen Li, Qiying Yu, al. Emu3: Next-token prediction all you need. arXiv preprint arXiv:2409. 18869, 2024.
[42] Yinjie Wang, Ling Yang, Bowen Li, Tian, Ke Shen, and Mengdi Wang. Revolutionizing reinforcement learning framework for diffusion large language models. arXiv preprint (:XXi:2559.06944 2025.
[43] Yinjie Wang, Ling Yang, Ye Tian, Shen, and Mengdi Wang. Co-evolving IIm coder and unit tester via reinforcement learning. arXiv preprint ar0iv:2506. 03136, 2025.
[44] Chengyue Wu, Xiaokang Chen, Zhiyu Wu, Yiyang Ma, Xingchao Liu, Zizheng Pan. Wen Liu, Zhenda Xie, Xingkai Yu, Chong Ruan, et al. Janus: Decoupling visual encoding for unified multimodal understanding and generation. arXiv preprint arXiv:2410. 13848, 2024.
[45] Junfeng Wu, Yi Jiang, Chuofan Ma, Yuliang Liu, Hengshuang Zhao, Zehuan Yuan, Song Bai, and Xiang Bai. Liquid: Language models are scalable multi-modal generators. arXiv preprint arXiv:2412.04332 2024.
[46] Shengqiong Wu, Hao Fei, Leigang Qu, Wei Ji, and Tat-Seng Chua. Next-gpt: Any-to-any multimodal I1m. arXiv preprint 2rXiv:2309.05519, 2023.
[47] Yecheng Wu, Zhuoyang Zhang, Junyu Chen, Haotian Tang, Dacheng Li, Yunhao Fang. Ligeng Zhu, Enze Xie. Hongxu Yin, Li Yi, et al. Vila-u: a unified foundation model integrating visual understanding and generation. arXiv preprint arXiv:2409.044
2024.
[48] Jinheng Xie, Weijia Mao, Zechen Bai, David Junhao Zhang, Weihao Wang. Kevin Qinghong Lin, Yuchao Gu, Zhijie Chen, Zhenheng Yang, and Mike Zheng Shou. Show-o: One single transformer to unify multimodal understanding and generation. arXiv preprint arXiv:2408. /2..22288..228.... 2024.
[49] Rongchang Xie. Chen Du, Ping Song, and Chang Liu. Muse-vl: Modeling unified vim through semantic discrete encoding. arXiv preprini 17X72 2024.
[50] Wulin Xie, Yi-Fan Zhang. Chaoyou Fu. Yang Shi, Bingyan Nie, Hongkai Chen, Zhang Zhang, Liang Wang. and Tieniu Tan. Mme-unify: A comprehensive benchmark for unified multimodal understanding and generation models. arXiv preprint arXiv:2504.03641. 2025.
[51] Cheng Yang, Chufan Shi, Yaxin Liu, Bo Shui, Junjie Wang, Mohan Jing. Linran Xu, Xinyu Zhu, Siheng Li, Yuxiang Zhang, et al. Chartmimic: Evaluating Imm's cross-modat reasoning capability via chart-to-code generation. arXiv preprint arXiv:2406.09961, 2024.
[52] Ling Yang. Ye Tian, Bowen Li. Xinchen Zhang, Ke Shen, Yunhai Tong, and Mengdi Wang. Mmada: Multimodal large diffusion language models. arXiv preprint arXiv:2505. 15809, 2025.
[53] Ling Yang, Zhaochen Yu. Bin Cui, and Mengdi Wang. Reasonflux: Hierarchical IIm reasoning via scaling thought templates. arXiv preprint 2rXiv:2502.0667.. 2025.
[54] Ling Yang. Zhaochen Yu, Chenlin Meng. Minkai Xu, Stefano Ermon, and CUI Bin. Mastering text-to- image diffusion: Recaptioning, planning, and generating with multimodal llms. Forty-first International Conference on Machine Learning, 2024.
[55] Ling Yang, Zhaochen Yu, Tianjun Zhang, Shiyi Cao, Minkai Xu, Wentao Zhang, Joseph E Gonzalez, and Bin Cui. Buffer of thoughts: augmented reasoning with large language models. Advances in Neural Information Processing Systems, 2024.
[56] Ling Yang, Zhanchen Yu, Tianjun Zhang, Minkai Xu, Joseph Gonzalez, Bin Cui, and Shuicheng Yan. Supercorrect: Supervising and correcting language models with error-driven insights. In International Conference on Learning Representations, 2025.
[57] Hanrong Ye, De-An Huang, Yao Lu, Zhiding Yu, Wei Ping. Andrew Tao, Jan Kautz, Song Han, Dan Xu. Pavlo Molchanov, et al. X-vila: Cross- modality alignment for large language model. arXiv preprint arXiv:2405. 19335, 2024.
Cross-modality
13

---

[58] Xiang Yue. Yuansheng Ni. Kai Zhang, Tianyu Zheng, Ruoqi Liu, Ge Zhang, Samuel Stevens, Dongfu Jiang, Weiming Ren, Yuxuan Sun, et al. Mmmu: A massive multi-discipline multimodal understanding and reasoning benchmark expert agi. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 9556-9567, 9567, 2024.
[59] Bo Zhang, Jun Zhu, and Hang Su. Toward the third generation artificial intelligence. Science China Information Sciences, 66(2):121101, 2023.
[60] Di Zhang, Jingdi Lei, Junxian Li, Xunzhi Wang, Yujie Liu, Zonglin Yang, Jiatong Li, Weida Wang. Suorong Yang. Jianbo Wu, et al. Critic-v: VIm critics help catch vim errors in multimodal reasoning. arXiv preprint arXiv:2411. 1tfi::2411.822221 18203, 2024.
[61] Jiawei Zhang, Zijian Wu, Zhiyang Liang, Yicheng Gong, Dongfang Hu, Yao Yao, Xun Cao, and Hao Zhu. Fate: Full-head gaussian avatar with textural editing from monocular video. arXiv preprint arXiv:2411.15600 2024.
[62] Shuoshuo Zhang. Zijian Li, Yizhen Zhang, Jingjing Fu, Lei Song. Jiang Bian, Jun Zhang. Yujiu Yang, and Rui Wang. Pixelcraft: A multi-agent system for high-fidelity visual reasoning on structured images. arXiv preprint arXiv:2509. 2815, 185, 2025.
[63] Xinchen Zhang. Ling Yang, Yaqi Cai. Zhaochen Yu, Kai-Ni Wang. Ye Tian, Minkai Xu, Yong Tang. Yujiu Yang, CUI Bin, et al. Realcompo: Balancing realism and compositionality improves text-to-image diffusion models. In The Thirty-eighth Annual Conference on Neural Information Processing Systems, 2024.
[64] Xinchen Zhang, Ling Yang, Guohao Li, Yaqi Cai, Jiake Xie, Yong Tang, Yujiu Yang. Mengdi Wang, and Bin Cui. Itercomp: Iterative composition-aware feedback learming from model gallery for text-to-image generation. arXiv preprint arXiv:2410.07117, 2024.
[65] Xinchen Zhang, Xiaoying Zhang, Youbin Wu, Yanbin Cao, Renrui Zhang, Ruihang Chu, Ling Yang, and Yujiu Yang. Generative universal verifier as multimodal 2025.
arXiv preprint arXiv:2510. 13804,
[66] Yizhen Zhang, Yang Ding, Shuoshuo Zhang, Xinchen Zhang, Haoling Li, Zhong-zhi Li, Peijie Wang. Jie Wu, Lei Ji, Yelong Shen, et al. Perl: Permutation-enhannee reinforcement learning for interleaved vision-language reasoning. arXiv preprint arXiv:2506. 149077 2025.
[67] Chujie Zheng. Shixuan Liu, Mingze Li, Xiong-Hui Chen, Bowen Yu, Chang Gao, Kai Dang, Yugiong Liu, Rui Men, An Yang, et al. Group sequence policy optimization. arXiv preprint arXiv:2507.1800 2025.
[68] Kaizhi Zheng, Xuchai Hc, and Xin Eric Wang. Minigpt-5: Interleaved vision-and-language generation via generative vokens. arXiv preprint arXiv:2310.02239) 2023.
[69] Chunting Zhou, Lili Yu, Arun Babu, Kushal Tirumala, Michihiro Yasunaga, Leonid Shamis, Jacob Kahn, Xuezhe Ma, Luke Zettlemoyer, and Omer Levy. Transfusion: Predict the next token and diffuse images with one multi-modal model. arXiv preprint arXiv:2408. / 1039, 2024.
[70] Yiyang Zhou, Chenhang Cui, Rafael Rafailov, Chelsea Finn, and Huaxiu Yao. Aligning modalities in vision large language models via preference fine-tuning. arXiv preprint arXiv:2402.11411 2024.
[71] Viyang Zhou, Zhiyuan Fan, Dongjie Cheng, Sihan Yang, Zhaorun Chen, Chenhang Cui, Xiyao Wang, Yun Li, Linjun Zhang, and Huaxiu Yao. Calibrated self-rewarding vision language models. arXiv preprint arXiv:2405.14622 14622. 2024.
[72] Deyao Zhu, Jun Chen, Xiaoqian Shen, Xiang Li, and Mohamed Elhoseiny. Minigpt-4; Enhancing vasion-language understanding with advanced large language models. arXiv preprint 230::33334.0052 2023.
[73] Yiyu Zhuang, Yuxiao He, Jiawei Zhang, Yanwen Wang, Jiahe Zhu, Yao Yao, Siyu Zhu, Xun Cao, and Hao Zhu. Towards native generative model for 3d head avatar. arXiv preprint arXiv:2410.0122 2024.
[74] Jiaru Zou, Ling Yang, Jingwen Gu, Jiahao Qiu, Ke Shen, Jingrui He, and Mengdi Wang. Reasonflux-prm: Trajectory-awace prms for long chain-of-thoughh reasoning in 1ms.. arXiv preprint 188. 2025.
14

---

A Derivation of the Pair-DPO Optimization Objective
Considering that the optimization objective of standard Direct Preference Optimization is:
LDPO - log Blo (e x) 3 log ( (x)
(11)
Tree (yo (x) Tire (3/)
Pair-DPO simultaneously optimizes understanding and generation using pairedepreférence data, with its loss function comprising these two components:
CPai--DP(()) Luna(0) LGen
To ( 23) (Vw (x)
no ( |z) Teet(y/
(o ( Teer (y)
(x2 13) TTcr (B)
logo Blo
log
+ log o Blog
B log
-E( (a:
Te (B x))
log no ( (x)
)
To (Zw 18)
log To (x (3/) Teer (1)
log a B Blog
log
That (ve lx)
Teer (31 x))
Tea (
(12)
Here, Aund and AGen are defined as:
To (Jw
x)
To (y/ Tet (3
x))
Aund
B log
B log
(13)
Tet y/
x)
x)
To (F
1))
Ta (z) y))
AGen = 3 log
B log
(14)
Feet Tww
y)
ree (xz y)
Substituting these definitions, the final Pair-DPO objective can be expressed as:
L
[log (AUnd
(15)
B More Quantitative Results
Table 6: Evaluation on MME-Unify (50
Understanding MITIU
Generation TIG
MME-U U Score
Methods
#params
SIPU
VPU
Avg
Avg
GILL 117
7B
22.18
6.00
3.56
10.58
46.60
6.12
MiniGPT-5 68
7B
19.25
10.92
15.93
15.37
35.48
7.09
Show-o 48
1.3B
32.47
34.75
25.66
30.96
43.54
12.74
Emu3
41
8B
45.75
30.50
23.32
33.19
49.08
13.79
HermesFlow (Ours)
1.3B
41.49
33.00
28.32
34.27
46.48
14.01
To more comprehensively evaluate HermesFlow's capabilities in both understanding and generation, present results the more extensive MME-Unify (50) benchmark. As shown in Table 6 with the backbone Show-o, HermesFlow achieves significant improvements both understanding and generation, indicating that approach not only reduces the gap between the two but also facilitates their joint enhancement. Despite using fewer parameters (1.3B), HermesFlow outperforms larger models such as Emu3 with 7B parameters, further demonstrating its potential general post-training framework for MLLMs.
15

---

C Example of Paired Preference Data
Homologous Input Data:
Prompt/Caption
Image:
Two samurals, one black and the other white, samurai
fiercely ly engage combat as the skillfully blocks an incoming sword attack amidst city backdrap, creating scene of intense and tense action.
Sampled data (n=10):
Win Lose
Sampled Captions for Understanding:
stand front city, holding swords. Caption2: Two warriors in armor, one with a sword and the other with spear, are standing
Captionl: Two warriors
front city.
platform with other characters. Caption4: A scene of two warriors standing in front of a city. One warrior has a sword in
Caption3: character the game Shadowrun stands
each hand, they facing other.
Caption5: Two warriors in armor, one with sword and the other with spear, are standing front city.
Caption6: knights
standing front of
Caption7: The two knights are in a dark city, standing on a street with swords in hand.
They seen have just come out from the shadows, ready for battle. with sword and the other with spear, standing
Caption8: Two warriors in armor, front of city.
Caption9: Two warriors armor stand platform, holding swords. One the warriors has horned helmet, the other has helmet with horns, city background. Caption10: image shows two knights on the edge cliff. They have two swords and are facing off, preparing fight. Sampled Images for Generation:
Imagel
Image2
Image3
Inage4
Image5
Image6
Inage7
Image8
Image9
Image10
Paired Preference Data:
Understanding:
Caption_wind wiin:wwwinno Two warriors in armor stand on a platform, holding swords. One of the warriors
helmet, the other helnet with horns, city background. Caption_lose: A character in the game Shadowrun stands on platforn with two other characters.
Generation:
Image_widd win:
Image_Lose: lose:
Figure 7: An example of the curation of paired preference data.
16

---

D
Qualitative Examples on Multimodal Understanding
Qualitative Example of Multimodal Understanding (1)
HermesFlow:
The image features rusty, old-fashioned, and somewhat eerie-looking metal sculpture of hand holding flower. The sculpture placed on patch of dirt, surrounded by barren landscape. The hand appears to be holding a flower, which is a single, wilted flower. The overall scene gives off sense of abandonment and decay, the sculpture seems to be relie from bygone era.
Prompt:
Please describe this image
detail.
Qualitative Example of Multimodal Understanding (2)
HermesFlow:
The image features an astronaut floating in the ocean, with large planet visible in the background. The astronaut is wearing spacesuit and appears to be floating the surface of the water. The scene set outer space, with star filled sky visible in the background. The astronaut seems to be gazing at the planet, possibly contemplating their journey or the vastness of the universe.
Prompt:
Please describe this image
detail.
17

---

Qualitative Example of Multimodal Understanding (3)
HermesFlow:
The image features a small, adorable bunny wearing a red sweater, sitting on a white surface. The bunny is surrounded by beautiful arrangement of white daisies, with some of the flowers positioned behind the bunny, creating visually pleasing scene. The bunny appears he the main focus of the image, with its cute appearance and the contrasting red sweater standing out against the white daisies.
Prompt:
Please describe this image in
detail.
Qualitative Example of Multimodal Understanding (4)
HermesFlow:
The image features woman with blonde hair, wearing black dress, and looking out of a window. She is positioned in the center of the frame, with her gaze directed towards something outside the window. The woman wearing large earrings, which are noticeable window. The scene appears to set indoors, with the woman standing near a window, possibly in room.
looks out the
Prompt:
Please deseribe this image in
detail.
18

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: Please see the details in the abstract and introduction sections.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results
be expected to generalize other settings.
is fine to include aspirational goals as motivation as long is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification: Please refer to the conclusion part.
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those not discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification. asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low images are taken in low lighting. Or speech-to-text system might be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach address problems of privacy and faimess.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [NA]
19

---

This paper introduces model training and inference algorithms for applications without theoretical outputs.
Justification:
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proof's can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should properly referenced.
4. Experimental result reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Question:
Answer: [Yes]
Justification:
Justification: Please refer to the details in Section
Guidelines:
The answer NA means that the paper does not include experiments. If the paper includes experiments, No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make it clear how reproduce that algorithm. (b) If the contribution is primarily
model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution is a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome describe the particular way they provide for reproducibility, In the case of closed-source models, it may be that access to the model is limited in some way (e.g., registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
20

---

Answer: [Yes]
Justification: Please see the details supplemental material.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nips.cm
crriiinnooccyll for more details.
While we encourage the release of code and data, we understand that this might not be possible, so "No" an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https
/ cppppi.ccppulli//uuuia wwwpc//ggue////eeeeeoooc CoodeSubeiiieeee ssi wswonnolli onnolicy)))) for more details.
The authors should provide instructions on data access and preparation, including how access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: Please refer to the details in Section 5 Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, or as supplemental material.
7. Experiment statistical significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [Yes]
Justification: Please refer to the details in Section 5
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, statistical significance tests, at least for the experiments that support main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
should be clear whether the error bar is the standard deviation the standard error of the mean.
21

---

It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out range (e.g. negative error rates).
If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification: Please refer the details in Section S
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g.. preliminary or failed experiments that didn't make into the paper).
9. Code of ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https: h/te::///euiis..eeeee http://///////ttcccciiieeeeeeeee
Answer: [Yes]
Justification: Please the whole paper and supplemental material.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special cireumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., if there is a special consid- eration due to laws or regulations their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [NA]
Justification: There is societal impact of the work performed.
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No. they should explain why their work has no societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that improvement in the quality of generative models could be used to
22

---

generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
there are negative socictal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification: The paper poses such risks.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification: Please see the whole paper and supplemental material.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (c.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, wwppppsippteeeee www.rrewwthhhe....c. com/ datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if has changed) should be provided.
If this information is not available online, the authors are encouraged to reach out to the asset's is creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
23

---

Answer: [Yes]
Justification: Please see the whole paper and supplemental material.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset used.
At submission time, remember to anonymize your assets (if applicable). You can either create anonymized URL or include an anonymized zip file.
14. Crowdsourcing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification:
The paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included the main paper.
According to the NeurIPS Code of Ethics, workers involved in data collection, curation, other labor should be paid at least the minimum wage in the country of the data collector.
Institutional review board (IRB) approvals or equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: The paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent)
may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
16. Declaration of LLM usage
Question: Does the paper describe the usage of LLMs if it is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, or formatting purposes and does not impact the core methodology, scientific rigorousness, originality of the research, declaration is not required.
Answer: [NA]
24

---

The core method development in this research does not involve LLMs as any important, original, or non-standard components.
Justification:
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, or non- standad components.
Please refer to our LLM policy hattps://neurips... 200p://ee////oofeeeeee///22///// for what should or should not be described.
25


Reviews:
[{"review_id": "VyMtOqDF1r", "reviewer": "Reviewer_KKjB", "summary": "This paper identified a performance gap between multimodal understanding and generation, and proposed a framework to reduce this gap.", "questions": "What are the designing philosophies of the generation score and understanding score? Are there any alternative approaches to implement them?", "limitations": "Discussed briefly in conclusion.", "rating": 4, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 2, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths:\n- Strong motivation. The problem of exploring the performance gap between multimodal understanding and generation is important and under-explored.\n- The proposed method successfully reduced the gap between understanding score and generation score.\n\nWeaknesses:\n- The definition of generation score may bring noises and may be hacked by 4o model. The generation score, which is important for calculation the performance gap, is heavily relied on the gpt 4o model to answer questions based on generated images. However, it's unclear how these visual information are important in answering these questions. Thus this score may not faithfully reflect the generation performance. Some more widely-used generation power metric or more grounded metrics may be needed.\n- In Table 5, GenEval of Pair-DPO (Iter. 3)  should not be bolded as it is not the best one.\n- Figure 3 is difficult to understand as it lacks some visual orders or focuses. Some interaction or mentioning of Fig 3 in the texts may be needed. It can also be removed if the main information are all included in the text.\n- Results are largely missing in Table 1.\n- The paper lacks evaluation on the quality of the grouped winning and losing samples, and how they differ.", "was_revised": true, "final_justification": "Although the VQA-based evaluation is my biggest concern, but this paper is introducing a new important problem and conducted comprehensive experiments. I vote for a borderline acceptance.", "rebuttal": "*We sincerely thank you for your time and efforts in reviewing our paper and for your valuable feedback. We are glad to see that the motivation of our paper is strong and important, and the proposed method is effective. Please see below for our responses to your comments.*\n\n\n> **Q1: The definition of generation score may bring noises. It's unclear how these visual information are important in answering these questions.** \n\n**A1:** Sorry for the confusion. The generation score refers to the average score computed by GPT-4o through Visual Question Answering. The VQA questions used here are limited to **basic visual perception tasks**, such as object existence, attributes, and correctness of spatial relationships. GPT-4o has a **significant advantage in handling these relatively simple perception-based questions**. Therefore, using GPT-4o to compute the generation score does not introduce significant errors. In addition, we also conducted experiments **using Gemini-2.5-Pro(0605)**, the current state-of-the-art vision-language model. The results are as follows:  \n\n**Table1: Comparison between Using GPT-4o and Gemini2.5-Pro to Calculate Generation Score.**\n|Method|# Params|Generation Score(GPT-4o)|Generation Score(Gemini-2.5-Pro)|\n|-|-|-|-|\n|VILA-U|7B|0.477|0.473|\n|Janus|1.3B|0.417|0.410|\n|Show-o|1.3B|0.433|0.438|\n|HermesFlow|1.3B|0.497| 0.495|\n\nSince these VQA questions focus on simple visual perception tasks, the experimental results show that GPT-4o and Gemini-2.5-Pro exhibit very similar performance. Therefore, using GPT-4o to calculate the generation score does not introduce significant errors. Thank you for your valuable feedback. \n\n> **Q2: In Table 5, GenEval of Pair-DPO (Iter. 3) should not be bolded as it is not the best one.** \n\n**A2:** Thank you for pointing out this issue. We will update the manuscript and ensure that similar problems do not appear in other parts of the paper. \n\n> **Q3: Figure 3 is difficult to understand as it lacks some visual orders or focuses.**\n\n **A3:** We apologize for any inconvenience caused. We will make sure to present Figure 3 more clearly in the revised manuscript, including aspects such as visual order and focus. Here, I’d like to help clarify the workflow of Figure 3:\n\nHermesFlow consists of two main components: \n\n- The curation of homologous preference data (left half of Figure 3) \n- Pair-DPO training (right half of Figure 3) \n\nRegarding the construction of homologous preference data, we **use homologous data (i.e., image-caption pairs) as input**. Given the image, we use the MLLM's understanding capability to perform the image captioning task, forming the **understanding pair**. Given the image caption, we use the MLLM's generation capability to perform the text-to-image task, forming the **generation pair**. In this way, the MLLM’s understanding and generation branches collaboratively construct the homologous preference data from the input pair.\n\nBased on this preference data, we propose a **Pair-DPO training strategy** to optimize the backbone model. Meanwhile, we design an **iterative self-play** mechanism (illustrated by the red dashed arrows in the figure) to continuously update the preference dataset in a dynamic fashion—gradually aligning and enhancing both the model’s understanding and generation capabilities. \n\nThank you again for your valuable feedback. We will redraw Figure 3 to make it clearer and easier to understand. If you still have any questions, please feel free to let us know.\n\n> **Q4: Results are largely missing in Table 1.**\n\n **A4:** This was an oversight on our part. Table 1 references results from the VILA-U, Janus, and Show-o. We have now completed most experiments for Table 1 and removed redundant baseline models. Moreover, to validate **the generalization ability of HermesFlow across different architectures and model scales, we reproduced HermesFlow on both Janus (1.3B) and VILA-U (7B)**. The results are shown below:  \n\n**Table 2: Evaluation on Multimodal Understanding Benchmarks.**\n\n| | #Params|POPE| MME| Flickr30k | VQAv2| GQA| MMMU|\n|-|-|-|-|-|-|-|-|\n| Emu|13B|80.9|1289.2|77.4| 57.2| 57.7| 25.7|\n| NExT-GPT|13B|82.3 |1222.1| 84.5| 66.7 | 60.2| 25.5|\n| SEED-X |13B|84.2|1435.7| 52.3| -| 47.9| 35.6 |\n| Chameleon| 34B|78.9|-| 74.7| 66.0| 54.4 | 29.5|\n| Show-o | 1.3B| 80.0|1232.9| 67.6| 74.7| 61.0 | 27.4|\n| **HermesFlow(Show-o)** |1.3B|**81.4**|**1249.7**| **69.2** |**75.3** | **61.7** | **28.3** |\n| Janus| 1.3B| 87.0| 1338.0 | 68.8| 77.3 | 59.1| 30.5|\n| **HermesFlow(Janus)**  | 1.3B | **87.4** | **1377.3**|**70.2** |**78.4**|**60.0** | **30.9** |\n| VILA-U | 7B | 85.8| 1401.8 | 69.9| 79.4| 60.8 | 30.1|\n| **HermesFlow(VILA-U)** | 7B| **86.9** |**1427.1**|**70.8** | **79.7**|**61.4** | **31.8** |\n\n**Table 3: Evaluation on Visual Generation Benchmark: GenEval.**\n\n| | #Params | Single Obj. | Two Obj. | Counting | Colors | Position | Color Attri. | Overall  |\n| -| -| - | -| - | - | - | - | -|\n| Show-o | 1.3B| 0.98|0.77|0.58|0.81|0.23|0.44|0.64|\n| **HermesFlow(Show-o)** | 1.3B | **0.98**| **0.84**|**0.66**|**0.82**|**0.32**|**0.52**|**0.69**|\n| Janus|1.3B|0.97|0.68|0.30|0.84|0.46|0.42|0.61|\n| **HermesFlow(Janus)** | 1.3B | **0.98**| **0.89** | **0.48** | **0.87** | **0.56** | **0.55** | **0.72**|\n| VILA-U  | 7B | 0.95| 0.62 | 0.48| 0.84 | 0.16 | 0.36| 0.57|\n| **HermesFlow(VILA-U)** | 7B  | **0.98**  | **0.88** | **0.65** | **0.85** | **0.39** | **0.53** |**0.71**|\n\nHermesFlow still shows significant improvements on backbones of larger sizes and different architectures. This indicates that our architecture has **strong generalization ability and is a general unified model post-training framework**. We will incorporate the relevant experimental updates in the revised manuscript. We appreciate your valuable feedback.\n\n> **Q5: The paper lacks evaluation on the quality of the grouped winning and losing samples, and how they differ.** \n\n**A5:** We thank the reviewer for raising this important point. We provide a comparison between the win and lose samples in terms of both understanding and generation aspects:  \n\n**Table 4: Comparison between Win Samples and Lose Samples.**\n\n|   | Count | Mean Caption Length | Und: BERT Similarity | Und: Model Judge | Gen: CLIP-Score | Gen: VQA-Score |\n| - | - | - | -| -| -| -|\n| Win samples  | 5K| ~26.3 words | 0.673| 79.2% | 31.11| 0.482 |\n| Lose samples | 5K| ~18.8 words| 0.488| 20.8%  | 29.98 | 0.392 |\n\n**Regarding the understanding data of image captions**, we use BERT Similarity and the win rate of the Model Judge (using Gemini-2.5-Pro) to evaluate the caption quality of win samples and lose samples. It is clear that captions of win samples have more accurate image descriptions. Moreover, our analysis of mean caption length reveals that captions in win samples tend to be longer, and therefore provide more detailed descriptions.  \n\n**Regarding the generation data of generated images**, we use CLIP-Score and VQA-Score (using Gemini-2.5-Pro) for evaluation. This indicates that win samples exhibit higher quality in image-text alignment. It also suggests that leveraging the unified model’s understanding capabilities during generation enables self-filtering of higher-quality outputs. We appreciate your suggestion and will update the paper to include this important analysis. \n\n> **Q6: What are the designing philosophies of the generation score and understanding score? Are there any alternative approaches to implement them?** \n\n**A6:** Since understanding and generation are two entirely distinct tasks, existing benchmarks evaluate these two aspects separately, employing different evaluation criteria and assessment methods. Consequently, **it is difficult to quantitatively verify the relative strengths between understanding and generation.** To address this gap where no unified standard exists to measure both understanding and generation, we propose the method shown in Figure 2(a) of our paper and provide a detailed explanation in the paragraph at line 239.\n\nIt is a highly promising and meaningful direction to explore more robust methods to uniformly measure understanding and generation. Here we propose a novel evaluation approach: \n\nWe consider the conflict between understanding and generation. For an input prompt $p$, we define the generated image as $\\phi^{Gen}(p)$. Using the understanding part of MLLM to judge whether this generated image satisfies the prompt, we define the **contradiction score** as:\n$$\nS_c = \\frac{1}{|\\mathcal{P}|}\\sum_{p\\in\\mathcal{P}}\\mathbb{I}[\\phi^{Und}(\\phi^{Gen}(p), query)=0] \n$$\n$$\nJudge_\\phi=\\phi^{Und}(\\phi^{Gen}(p), query)\n$$\nWhere $\\mathcal{P}$ represents the prompt dataset, and $query$ denotes the judge prompt for understanding. Using Gemini-2.5-Pro as the supervised model, the understanding score and generation score can be respectively defined as follows:\n$$\nS_{Gen} = 1-\\frac{1}{|\\mathcal{P}|}\\sum_{p\\in\\mathcal{P}}\\mathbb{I}( Judge_\\phi = 0 \\land Judge_{Gemini} = 0 )\n$$\n$$\nS_{Und} = 1-\\frac{1}{|\\mathcal{P}|}\\sum_{p\\in\\mathcal{P}}\\mathbb{I}( Judge_\\phi = 0 \\land Judge_{Gemini} = 1 )\n$$\nUsing this approach, we conducted new experiments and obtained the following results:\n\n**Table5: Comparison between Method1 (used in the paper) and Method2 (the newly proposed).**\n\n| Method     | # Params | Generation Score(method1) | Understanding Score(method1) | Generation Score(method2) | Understanding Score(method2) |\n| - | -| - | -| -| -|\n| VILA-U | 7B | 0.473| 0.646| 0.622| 0.901|\n| Janus | 1.3B | 0.410| 0.599| 0.607| 0.927 |\n| Show-o | 1.3B | 0.438| 0.520| 0.644| 0.893  |\n| HermesFlow | 1.3B| 0.495| 0.533| 0.770| 0.912 |\n\nThis table demonstrates that both the new method and the original method in the paper can effectively measure the gap between understanding and generation in the unified model. Notably, the new method provides more pronounced and amplified measurements of this gap. We sincerely appreciate your insightful suggestion and will incorporate it into the updated version of our paper."}, {"review_id": "8cJRjXhaZj", "reviewer": "Reviewer_A63A", "summary": "This paper studies an interesting topic: bridging the gap between visual understanding and generation of a unified MLLM through reinforcement learning. One observation is that current unified models usually perform better on image understanding than image generation tasks. Therefore, this paper proposes to use Pair-DPO and self-play iterative opetimization to close the gap between the two tasks. The evaluation on understanding and generation tasks verify its effectiveness.", "questions": "see weaknesses.", "limitations": "yes", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 2, "significance": 2, "originality": 3, "strengths_and_weaknesses": "**Strengths:**\n- The motivation is clear. Current unified models do good on understanding but are limited to their visual generation ability. Therefore, a method to mitigating the gap between two tasks is important for future research.\n- The proposed method is interesting. HermesFlow constructs preference pairs by comparing the generated text/image with training image-caption pair. Then Pair-DPO is performed on both image understanding and generation tasks. The policy MLLM is optimized to generate text/image that aligned better with the training image-caption pairs.\n\n**Weaknesses:**\n- Limited improvement on benchmarks, such as Tab.1 multimodal understanding, Tab.2 image generation.\n- As for the comparisons with show-o on Geneval. I am confused which version of show-o is used, 256px or 512px? In Tab.2, the baseline show-o is 0.53 on Geneval, but I guess HermesFlow use 512px show-o from the source code. So the comparison is not fair in Tab.2 and Tab.5. The real gain is 0.68 -> 0.69, not 0.53 -> 0.69.", "was_revised": true, "final_justification": "I increase my score to 4. But I still find it very strange that the authors did not test the results of their own baseline method. This has also shifted my rating from initially positive to negative.", "rebuttal": "*We sincerely thank you for your time and efforts in reviewing our paper and for your valuable feedback. We are glad to see that the motivation of our paper is clear, the topic is interesting and important, the proposed method is effective. Please see below for our responses to your comments.*\n\n> **Q1: Limited improvement on benchmarks, such as Tab.1 multimodal understanding, Tab.2 image generation.** \n\n**A1:** Sorry for the confusion. This paper investigates whether **self-improvement and self-alignment without any external supervision data** can be realized in unified multimodal LLMs, particularly addressing the existing gap between understanding and generation capabilities. While our current work presents an initial approach, we aim to explore more refined optimization methods in future research to further advance performance.\n\nMoreover, to validate **the generalization ability of HermesFlow across different architectures and model scales, we reproduced HermesFlow on both Janus (1.3B) and VILA-U (7B)**. The results are shown below:  \n\n**Table 1: Evaluation on Multimodal Understanding Benchmarks.**\n\n|                        | #Params | POPE     | MME        | Flickr30k | VQAv2    | GQA      | MMMU     |\n| ---------------------- | ------- | -------- | ---------- | --------- | -------- | -------- | -------- |\n| Show-o                 | 1.3B    | 80.0     | 1232.9     | 67.6      | 74.7     | 61.0     | 27.4     |\n| **HermesFlow(Show-o)** | 1.3B    | **81.4** | **1249.7** | **69.2**  | **75.3** | **61.7** | **28.3** |\n| Janus                  | 1.3B    | 87.0     | 1338.0     | 68.8      | 77.3     | 59.1     | 30.5     |\n| **HermesFlow(Janus)**  | 1.3B    | **87.4** | **1377.3** | **70.2**  | **78.4** | **60.0** | **30.9** |\n| VILA-U                 | 7B      | 85.8     | 1401.8     | 69.9      | 79.4     | 60.8     | 30.1     |\n| **HermesFlow(VILA-U)** | 7B      | **86.9** | **1427.1** | **70.8**  | **79.7** | **61.4** | **31.8** |\n\n**Table 2: Evaluation on Visual Generation Benchmark: GenEval.**\n\n|                        | #Params | Single Obj. | Two Obj. | Counting | Colors   | Position | Color Attri. | Overall  |\n| ---------------------- | ------- | ----------- | -------- | -------- | -------- | -------- | ------------ | -------- |\n| Show-o                 | 1.3B    | 0.98        | 0.77     | 0.58     | 0.81     | 0.23     | 0.44         | 0.64     |\n| **HermesFlow(Show-o)** | 1.3B    | **0.98**    | **0.84** | **0.66** | **0.82** | **0.32** | **0.52**     | **0.69** |\n| Janus                  | 1.3B    | 0.97        | 0.68     | 0.30     | 0.84     | 0.46     | 0.42         | 0.61     |\n| **HermesFlow(Janus)**  | 1.3B    | **0.98**    | **0.89** | **0.48** | **0.87** | **0.56** | **0.55**     | **0.72** |\n| VILA-U                 | 7B      | 0.95        | 0.62     | 0.48     | 0.84     | 0.16     | 0.36         | 0.57     |\n| **HermesFlow(VILA-U)** | 7B      | **0.98**    | **0.88** | **0.65** | **0.85** | **0.39** | **0.53**     | **0.71** |\n\nHermesFlow still shows significant improvements on backbones of larger sizes and different architectures. In addition, it can be observed that HermesFlow demonstrates remarkable enhancements in Two Obj., Counting, Position, and Color Attribute on both the 1.3B and 7B backbones. This is because of the excellent understanding ability of MLLM regarding image attributes, spatial position relationships, etc., which makes the preference data contain a large number of features in this regard. \n\nMoving beyond the framework limitations of model self-improvement without any external supervision, we use **Gemini-2.5-Pro as a judge** to collect preference datasets from multiple generated data samples:\n\n**Table 3: Use Gemini-2.5-Pro as the Judge to Create Preference Data**\n\n|                               | #Params | POPE | MME    | MMMU | GenEval(Overall) | DPG-Bench (Average) |\n| ----------------------------- | ------- | ---- | ------ | ---- | ---------------- | ------------------- |\n| Show-o                        | 1.3B    | 80.0 | 1232.9 | 27.4 | 0.64             | 67.48               |\n| HermesFlow (Self-improvemrnt) | 1.3B    | 81.4 | 1249.7 | 28.3 | 0.69             | 70.22               |\n| HermesFlow (MLLM-as-a-judge)  | 1.3B    | 81.7 | 1258.4 | 28.7 | 0.73             | 70.99               |\n\nFrom the table, employing a more powerful external model as a data collector yields better results in both understanding and generation. This is because Gemini-2.5-Pro is capable of thoroughly mining and selecting the optimal win and lose samples, which also highlights the potential of the HermesFlow architecture. \n\nWe will incorporate the relevant experimental updates in the revised manuscript. We appreciate your valuable feedback on this matter. \n\n\n\n\n\n> **Q2: As for the comparisons with show-o on Geneval. I am confused which version of show-o is used, 256px or 512px? In Tab.2, the baseline show-o is 0.53 on Geneval, but I guess HermesFlow use 512px show-o from the source code. So the comparison is not fair in Tab.2 and Tab.5. The real gain is 0.68 -> 0.69, not 0.53 -> 0.69.**\n\n\n\n**A2:** We sincerely appreciate your careful observation and bringing this important point to our attention. We referred to papers Janus and Emu3, and followed their reported value of 0.53 without carefully verifying the resolution setting. HermesFlow used the 512px version. We sincerely apologize for this oversight, as we did not rigorously check this value.\n\nHere, we re-evaluate the performance on Show-o using the following configuration: **guidance scale = 5**, **generation timesteps = 50**, and **resolution = 512px**. The updated results are as follows:\n\n**Table 4: Comparison between Show-o and HermesFlow on GenEval.**\n|            | #Params | Single Obj. | Two Obj.    | Counting    | Colors      | Position    | Color Attri. | Overall     |\n| ---------- | ------- | ----------- | ----------- | ----------- | ----------- | ----------- | ------------ | ----------- |\n| Show-o     | 1.3B    | 0.98        | 0.77        | 0.58        | 0.81        | 0.23        | 0.44         | 0.64        |\n| HermesFlow | 1.3B    | 0.98(+0.00)  | 0.84(+0.07) | 0.66(+0.08) | 0.82(+0.01) | 0.32(+0.09) | 0.52(+0.08)  | 0.69(+0.05) |\n\nHermesFlow achieved significant improvements, particularly on Two Obj., Counting, Position, and Color Attribute. This is because the **understanding of MLLMs exhibits a clear advantage, especially in handling multi-object and spatial relationship scenarios, where it can easily identify and select high-quality preference data**. This further validates the existence of a generation-understanding gap in MLLMs, indicating clear room for improvement.\n\nWe sincerely appreciate your careful observation of this issue. We will revise our paper accordingly and thoroughly verify the correctness of other reported metrics."}, {"review_id": "j5XecvLmWS", "reviewer": "Reviewer_WWD6", "summary": "This article studies the performance gap between understanding and generation abilities in MLLMs. The authors observe that unified MLLMs tend to have stronger understanding capabilities than generation, which has not been widely addressed before. To tackle this issue, they introduce HermesFlow, aiming to align these two capabilities. The method uses homologous data (data that shares meaning across modalities) to create preference pairs for both understanding and generation. Then, it applies Pair-DPO and self-play training to improve alignment between the two abilities. Experimental results show that HermesFlow improves both understanding and generation, especially in closing the performance gap.", "questions": "As stated in the first or third weakness, why not choosing a SOTA unified MLLM, and using your method to improve its capability to achieve both best and more balanced performance.", "limitations": "yes", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 4, "significance": 2, "originality": 3, "strengths_and_weaknesses": "**Strengths**:\n1. The paper designs a framework to synthesize preference data for both understanding and generation with the target MLLM model and make use of pair-DPO to improve both capabilities iteratively.\n2. The paper is clearly written and easy to follow.\n\n**Weaknesses**:\n1. Filling the gap between generation and understanding might be a less important problem than improving both of the capabilities. For instance, the improved show-o (the proposed model) still lags behind VILA-U for a large distance in terms of understanding (Table 4), and the performance on generation is maiginally higher, then it maybe still a better choice to use VILA-U. \n2. The improvements from self-playiterative method seems limited and marginal.\n3. The method might be more useful when applying to less powerful model which is less robust, adding experiment results on larger model like at least 7B to show its generalization and scalability might be a more rogorous setting.", "was_revised": true, "final_justification": "I have read the rebuttal from the authors, and raised my score accordingly.", "rebuttal": "*We sincerely thank you for your time and efforts in reviewing our paper and for your valuable feedback. We are glad to see that the paper is well written and easy to follow, and the proposed method is effective. Please see below for our responses to your comments.*\n\n>**Q1: Filling the gap between generation and understanding might be a less important problem than improving both of the capabilities. For instance, the improved show-o (the proposed model) still lags behind VILA-U for a large distance in terms of understanding (Table 4), and the performance on generation is maiginally higher, then it maybe still a better choice to use VILA-U.** \n\n**A1:** I completely agree with your point. However, HermesFlow is the first to empirically demonstrate the **significant gap between understanding and generation**, and it proposes a **general self-improvement post-training method. Without relying on any external supervision data**, it leverages the existing gap to seamlessly align understanding and generation capabilities, while also enabling joint improvement. This makes it a highly generalizable post-training approach that can be applied to **any pretrained unified model** for aligning and enhancing both understanding and generation.\n\nIn addition, we argue that a truly unified MLLM should excel in both understanding and generation, **striking a balance rather than favoring one over the other**. However, current unified MLLMs struggle to achieve this equilibrium. To address this, we propose a self-improvement paradigm that effectively bridges the gap, yielding an MLLM performs strongly in both aspects. This balance is crucial in practice: RPG enhances prompt understanding using more powerful MLLM, ultimately improving generation quality; MetaMorph incorporates more generative data, leading to better understanding. This interplay demonstrates that understanding and generation are mutually reinforcing, bridging the gap between them is essential for developing a truly unified MLLM. \n\nMoreover, to validate **the generalization ability of HermesFlow across different architectures and model scales, we reproduced HermesFlow on both Janus (1.3B) and VILA-U (7B)**. The results are shown below:  \n\n**Table 1: Evaluation on Multimodal Understanding Benchmarks.**\n\n|                        | #Params | POPE     | MME        | Flickr30k | VQAv2    | GQA      | MMMU     |\n| ---------------------- | ------- | -------- | ---------- | --------- | -------- | -------- | -------- |\n| Show-o                 | 1.3B    | 80.0     | 1232.9     | 67.6      | 74.7     | 61.0     | 27.4     |\n| **HermesFlow(Show-o)** | 1.3B    | **81.4** | **1249.7** | **69.2**  | **75.3** | **61.7** | **28.3** |\n| Janus                  | 1.3B    | 87.0     | 1338.0     | 68.8      | 77.3     | 59.1     | 30.5     |\n| **HermesFlow(Janus)**  | 1.3B    | **87.4** | **1377.3** | **70.2**  | **78.4** | **60.0** | **30.9** |\n| VILA-U                 | 7B      | 85.8     | 1401.8     | 69.9      | 79.4     | 60.8     | 30.1     |\n| **HermesFlow(VILA-U)** | 7B      | **86.9** | **1427.1** | **70.8**  | **79.7** | **61.4** | **31.8** |\n\n**Table 2: Evaluation on Visual Generation Benchmark: GenEval.**\n\n|                        | #Params | Single Obj. | Two Obj. | Counting | Colors   | Position | Color Attri. | Overall  |\n| ---------------------- | ------- | ----------- | -------- | -------- | -------- | -------- | ------------ | -------- |\n| Show-o                 | 1.3B    | 0.98        | 0.77     | 0.58     | 0.81     | 0.23     | 0.44         | 0.64     |\n| **HermesFlow(Show-o)** | 1.3B    | **0.98**    | **0.84** | **0.66** | **0.82** | **0.32** | **0.52**     | **0.69** |\n| Janus                  | 1.3B    | 0.97        | 0.68     | 0.30     | 0.84     | 0.46     | 0.42         | 0.61     |\n| **HermesFlow(Janus)**  | 1.3B    | **0.98**    | **0.89** | **0.48** | **0.87** | **0.56** | **0.55**     | **0.72** |\n| VILA-U                 | 7B      | 0.95        | 0.62     | 0.48     | 0.84     | 0.16     | 0.36         | 0.57     |\n| **HermesFlow(VILA-U)** | 7B      | **0.98**    | **0.88** | **0.65** | **0.85** | **0.39** | **0.53**     | **0.71** |\n\nHermesFlow still shows significant improvements on backbones of larger sizes and different architectures. It can be observed from the table that HermesFlow demonstrates remarkable enhancements in Two Obj., Counting, Position, and Color Attribute on both the 1.3B and 7B backbones. This is because of the excellent understanding ability of MLLM regarding image attributes, spatial position relationships, etc., which makes the preference data contain a large number of features in this regard. This indicates that our architecture has **strong generalization ability and is a general unified model post-training framework**. We will incorporate the relevant experimental updates in the revised manuscript. We appreciate your valuable feedback.\n\n\n\n> **Q2: The improvements from the self-play iterative method seems limited and marginal.** \n\n**A2:** We sincerely apologize for the confusion. Iterative optimization represents a novel paradigm for unified models. HermesFlow shows the most significant performance improvement during the first iteration, which confirms the effectiveness of mutual refinement between understanding and generation, therefore, to continuously narrow the gap between these two aspects, we adopt an iterative training method. Moreover, we observe that the effectiveness of the iterative training method is **highly dependent on the initial gap between understanding and generation** in the backbone model, **the larger the initial gap, the more significant the performance gain from iterative training**. We present the performance metrics of Janus under iterative self-play training: \n\n**Table 3: Effect of Pair-DPO Iterations on Janus**\n\n|                    | #Params | POPE | MME    | MMMU | GenEval(Overall) | DPG-Bench (Average) |\n| ------------------ | ------- | ---- | ------ | ---- | ---------------- | ------------------- |\n| Janus              | 1.3B    | 87.0 | 1338.0 | 30.5 | 0.61             | 73.20               |\n| Pair-DPO (Iter. 1) | 1.3B    | 87.2 | 1365.5 | 30.7 | 0.67             | 75.97               |\n| Pair-DPO (Iter. 2) | 1.3B    | 87.4 | 1363.1 | 30.9 | 0.70             | 76.31               |\n| Pair-DPO (Iter. 3) | 1.3B    | 87.4 | 1377.3 | 30.9 | 0.72             | 77.21               |\n\nAs shown in the Table, HermesFlow (Janus) demonstrates more substantial improvements in the second and third iterations. We argue that bridging the gap between understanding and generation is a challenging optimization problem, and a single round of DPO training may not suffice. Therefore, **for models with a larger initial gap, we recommend applying iterative training**. We appreciate your insightful question and will incorporate this important analysis into the revised version of the paper.  \n\n> **Q3: The method might be more useful when applying to less powerful model which is less robust, adding experiment results on larger models like at least 7B to show its generalization and scalability might be a more rogorous setting.** \n\n**A3:** Thank you for your suggestion! **In Table 1 and Table 2 in A1**, we present the experimental results of HermesFlow on two models with different architectures and scales: Janus (1.3B) and VILA-U (7B). We will incorporate the relevant experimental updates in the revised manuscript. We appreciate your valuable feedback on this matter. \n\n> **Q4: As stated in the first or third weakness, why not choose a SOTA unified MLLM, and use your method to improve its capability to achieve both best and more balanced performance.** \n\n **A4:** HermesFlow is an initial attempt, and we have already validated its effectiveness on Show-o, Janus, and VILA-U  **In Table 1 and Table 2 in A1**.  We are committed to continuing experiments of HermesFlow on more backbone models (e.g., Bagel and Show-o2) to achieve stronger unified performance."}, {"review_id": "eVaAqEM7BD", "reviewer": "Reviewer_zMCR", "summary": "This paper introduces HermesFlow, a framework intended to bridge the notable gap between multimodal understanding and generation capabilities in unified Multimodal Large Language Models (MLLMs). They propose a methodology called Pair-DPO, leveraging paired preference data for joint optimization, which is iteratively refined via a self-play strategy. The authors demonstrate that their approach, built on the Show-o model, narrows the capability gap while improving performance in both domains.", "questions": "1. Can you do more experiments on other backbone models, like Janus?\n2. Given the dependency on homologous data quality, could you provide more details about the methods or heuristics you used to ensure diversity and quality in the homologous preference dataset?\n3. What are the framework's failure modes if a base model is too weak to generate high-quality candidate samples for the preference curation process?\n4. Janus and VILA-U are used as baselines for generation in Figure 4 and for gap analysis in Table 4. Why are their quantitative results on the standard understanding benchmarks in Table 1 not included?", "limitations": "Authors have discussed their limitations.", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths:\n1. The paper precisely identifies a significant issue within unified MLLMs, where the capability between understanding and generation is imbalanced.\n2. HermesFlow leverages Pair-DPO, a creative approach employing homologous preference data to align multimodal understanding and generation effectively. The self-play iterative optimization scheme is particularly interesting, allowing the model to improve autonomously without external data supervision.\n3. Extensive experiments and comparisons with other baselines demonstrate HermesFlow's superiority.\n\nWeaknesses:\n1. Limited Generalization Validation. The claim that HermesFlow is a \"general alignment framework\"  is not sufficiently supported by the experiments. The entire validation is performed on a single backbone model, Show-o. While the results are promising, it is impossible to know if they generalize to other MLLMs with different architectures (e.g., Emu3, Janus, VILA-U).\n2. Homologous Data Dependency. The proposed method’s efficacy strongly relies on generating high-quality homologous data pairs. However, the quality and diversity of these pairs greatly influence performance, as briefly mentioned in ablation studies.\n3. Computational Efficiency and Scalability Concerns. The iterative self-play optimization might impose significant computational costs. The paper does not sufficiently discuss potential efficiency issues or computational complexity.", "was_revised": true, "final_justification": "The authors provide experiments about generalization and data quality. Therefore, I keep my positive scores.", "rebuttal": "*We sincerely thank you for your time and efforts in reviewing our paper and for your valuable feedback. We are glad to see that the topic of our paper is significant and important, the proposed method is interesting, and the experiments are extensive. Please see below for our responses to your comments.*\n\n>**Q1: Limited Generalization Validation. While the results are promising, it is impossible to know if they generalize to other MLLMs with different architectures.** \n\n**A1:** We apologize for the oversight. Here, **we reproduced HermesFlow on both Janus (1.3B) and VILA-U (7B)**. The results are shown below:  \n\n**Table 1: Evaluation on Multimodal Understanding Benchmarks.**\n\n|                        | #Params | POPE     | MME        | Flickr30k | VQAv2    | GQA      | MMMU     |\n| ---------------------- | ------- | -------- | ---------- | --------- | -------- | -------- | -------- |\n| Show-o                 | 1.3B    | 80.0     | 1232.9     | 67.6      | 74.7     | 61.0     | 27.4     |\n| **HermesFlow(Show-o)** | 1.3B    | **81.4** | **1249.7** | **69.2**  | **75.3** | **61.7** | **28.3** |\n| Janus                  | 1.3B    | 87.0     | 1338.0     | 68.8      | 77.3     | 59.1     | 30.5     |\n| **HermesFlow(Janus)**  | 1.3B    | **87.4** | **1377.3** | **70.2**  | **78.4** | **60.0** | **30.9** |\n| VILA-U                 | 7B      | 85.8     | 1401.8     | 69.9      | 79.4     | 60.8     | 30.1     |\n| **HermesFlow(VILA-U)** | 7B      | **86.9** | **1427.1** | **70.8**  | **79.7** | **61.4** | **31.8** |\n\n**Table 2: Evaluation on Visual Generation Benchmark: GenEval.**\n\n|                        | #Params | Single Obj. | Two Obj. | Counting | Colors   | Position | Color Attri. | Overall  |\n| ---------------------- | ------- | ----------- | -------- | -------- | -------- | -------- | ------------ | -------- |\n| Show-o                 | 1.3B    | 0.98        | 0.77     | 0.58     | 0.81     | 0.23     | 0.44         | 0.64     |\n| **HermesFlow(Show-o)** | 1.3B    | **0.98**    | **0.84** | **0.66** | **0.82** | **0.32** | **0.52**     | **0.69** |\n| Janus                  | 1.3B    | 0.97        | 0.68     | 0.30     | 0.84     | 0.46     | 0.42         | 0.61     |\n| **HermesFlow(Janus)**  | 1.3B    | **0.98**    | **0.89** | **0.48** | **0.87** | **0.56** | **0.55**     | **0.72** |\n| VILA-U                 | 7B      | 0.95        | 0.62     | 0.48     | 0.84     | 0.16     | 0.36         | 0.57     |\n| **HermesFlow(VILA-U)** | 7B      | **0.98**    | **0.88** | **0.65** | **0.85** | **0.39** | **0.53**     | **0.71** |\n\nHermesFlow still shows significant improvements on backbones of larger sizes and different architectures. It can be observed from the table that HermesFlow demonstrates remarkable enhancements in Two Obj., Counting, Position, and Color Attribute on both the 1.3B and 7B backbones. This is because of the excellent understanding ability of MLLM regarding image attributes, spatial position relationships, etc., which makes the preference data contain a large number of features in this regard. This indicates that our architecture has **strong generalization ability and is a general unified model post-training framework**. We will incorporate the relevant experimental updates in the revised manuscript. We appreciate your valuable feedback.\n\n\n\n>**Q2: Homologous Data Dependency. The proposed method’s efficacy strongly relies on generating high-quality homologous data pairs. However, the quality and diversity of these pairs greatly influence performance, as briefly mentioned in ablation studies.**\n\n **A2:** We thank the reviewer for raising this important point. The quality of homologous data pairs has a significant impact on the final training results. What we explore here is a **fully self-improvement and self-alignment paradigm that requires no external supervision**. We find that simply leveraging the gap between the MLLM’s own understanding and generation capabilities is sufficient to effectively align the two and achieve joint improvement. \n\nHowever, constructing higher-quality preference data is indeed a highly valuable and promising direction. Moving beyond the framework limitations of model self-improvement or self-critique, we use **Gemini-2.5-Pro as a judge** to collect preference datasets from multiple generated data samples.:\n\n**Table 3: Use Gemini-2.5-Pro as the Judge to Create Preference Data**\n\n|                               | #Params | POPE | MME    | MMMU | GenEval (Overall) | DPG-Bench (Average) |\n| ----------------------------- | ------- | ---- | ------ | ---- | ---------------- | ------------------- |\n| Show-o                        | 1.3B    | 80.0 | 1232.9 | 27.4 | 0.64             | 67.48               |\n| HermesFlow (Self-improvement) | 1.3B    | 81.4 | 1249.7 | 28.3 | 0.69             | 70.22               |\n| HermesFlow (MLLM-as-a-judge)  | 1.3B    | 81.7 | 1258.4 | 28.7 | 0.73             | 70.99               |\n\nFrom the table, employing a more powerful external model as a data collector yields better results in both understanding and generation. This is because Gemini-2.5-Pro is capable of thoroughly mining and selecting the optimal win and lose samples, which also highlights the potential of the HermesFlow architecture. We will investigate better data construction approaches to further enhance MLLM performance and close existing gaps.\n\n>**Q3: Computational Efficiency and Scalability Concerns. The iterative self-play optimization might impose significant computational costs. The paper does not sufficiently discuss potential efficiency issues or computational complexity.** \n\n**A3:** Thank you for the constructive feedback. Pair-DPO maintains comparable computational requirements to standard DPO. Although it introduces homologous preference data construction and iterative optimization, the small dataset size (5k image-text pairs) and low overhead per iteration (typically 2-3 rounds) prevent significant resource increases. In practice, training can be efficiently performed using just 4×A100 GPUs on Show-o. \n\nHere is an approximate listing of the training time for each stage (all our experiments were conducted on 4*NVIDIA A100-80G GPUs):\n\n**Table 4: The Training Time for Each Stage.**\n| Phrase      | Training Time |\n| :---------- | :------------ |\n| Iteration 1 | ~4h 12min     |\n| Iteration2  | ~1h 46min     |\n| Iteration3  | ~1h 46min     |\n| Total       | ~7h 44min     |\n\nTo mitigate overfitting, we used fewer training samples and fewer training steps starting from iteration 2. As a result, with the continuous refinement of iterative feedback learning, the training time of the model decreases progressively.\n\n>**Q4: Can you do more experiments on other backbone models, like Janus?** \n\n**A4:** Thank you for your suggestion. We provide the results on Janus and VILA-U **in Tables 1 and 2 in A1.** \n\n>**Q5: Given the dependency on homologous data quality, could you provide more details about the methods or heuristics you used to ensure diversity and quality in the homologous preference dataset?** \n\n**A5:** We thank the reviewer for raising this important point. We sample the homologous data from JourneyDB, a comprehensive and high-quality dataset. During the filtering process, we apply a **strict threshold of 0.8 using BERT similarity score** to control the diversity of the prompts in the homologous data. Additionally, we use the **CLIP score to filter prompt-image pairs, with a threshold set at 0.3** to ensure the quality of text-image pairs. \n\n>**Q6: What are the framework's failure modes if a base model is too weak to generate high-quality candidate samples for the preference curation process?**  \n\n**A6:** Thank you for this insightful question. HermesFlow is a highly flexible architecture that enables parameter tuning to improve the quality of homologous preference data. For weaker backbones, we **increase the number of samples per preference pair from 10 to 16**, which leads to a notable improvement in the quality of the collected preferences. Additionally, an auxiliary judge model can be introduced to further support preference data collection under weaker backbone settings. Our experimental results on **Chameleon-7B** are presented below:\n\n**Table 5: Results of Hermesflow Using a Weaker Backbone.**\n|                        | #Params | POPE | MME   | MMMU | GenEval(Overall) | DPG-Bench (Average) |\n| ---------------------- | ------- | ---- | ----- | ---- | ---------------- | ------------------- |\n| Chameleon              | 7B      | 72.1 | 947.7 | 26.2 | 0.39             | 66.10               |\n| HermesFlow (Chameleon) | 7B      | 72.2 | 968.1 | 26.4 | 0.44             | 68.94               |\n\nIncreasing the sampling number leads to a substantial improvement in data quality. Furthermore, leveraging a stronger judge model can effectively compensate for deficiencies in the understanding component, as **shown in Table 3 in A2.**\n\n>**Q7:  Quantitative results on the standard understanding benchmarks of Janus and VILA-U.** \n\n**A7:** Thank you for pointing this out. We provide the experimental results on Janus and VILA-U **in Tables 1 and 2 in A1.**"}]

Author rebuttal:

Meta-review: The reviewers agree that the proposed HermesFlow addresses a critical issue of bridging the gap between understanding and generation capabilities for a unified multimodal LLM. The proposed Pair-DPO and self-play iterative optimization techniques are novel and effective, as evidenced by the experimental results. The authors are also actively engaged in the rebuttal phase and have addressed the majority of concerns raised by the reviewers. We recommend that the authors incorporate the discussions into the next version of the work and expand the evaluation to encompass a broader range of benchmarks.
Decision: Accept (poster)