Title: FUDOKI: Discrete Flow-based Unified Understanding and Generation via Kinetic-Optimal Velocities
Venue: NeurIPS 2025
Accepted: yes
Keywords: Multimodal Large Language Models (MLLMs); Discrete Flow Matching
Abstract: The rapid progress of large language models (LLMs) has catalyzed the emergence of multimodal large language models (MLLMs) that unify visual understanding and image generation within a single framework. However, most existing MLLMs rely on autoregressive (AR) architectures, which impose inherent limitations on future development, such as the raster-scan order in image generation and restricted reasoning abilities in causal context modeling. In this work, we challenge the dominance of AR-based approaches by introducing FUDOKI, a unified multimodal model purely based on discrete flow matching, as an alternative to conventional AR paradigms. By leveraging metric-induced probability paths with kinetic optimal velocities, our framework goes beyond the previous masking-based corruption process, enabling iterative refinement with self-correction capability and richer bidirectional context integration during generation. To mitigate the high cost of training from scratch, we initialize FUDOKI from pre-trained AR-based MLLMs and adaptively transition to the discrete flow matching paradigm. Experimental results show that FUDOKI achieves performance comparable to state-of-the-art AR-based MLLMs across both visual understanding and image generation tasks, highlighting its potential as a foundation for next-generation unified multimodal models. Furthermore, we show that applying test-time scaling techniques to FUDOKI yields significant performance gains, further underscoring its promise for future enhancement through reinforcement learning.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper introduces FUDOKI, the first unified vision-language model that replaces the standard autoregressive (AR) decoder with Discrete Flow Matching (DFM) decoding. Tokens are noised along a metric-induced probability path in embedding space, and a kinetic-optimal velocity field is learned so the model iteratively transports the corrupted sequence toward the target, allowing bidirectional attention and step-wise self-correction. Trained on Jauns-1B, FUDOKI achieves performance comparable to SOTA AR-based MLLMs across both visual understanding and image generation tasks,\n\nSTRENGTHS AND WEAKNESSES:\n**Strengths**\n1. This paper presents the first unified multimodal generation model based on Discrete Flow Matching, introducing DFM techniques into this domain.\n2. The figures in this paper are clear and effectively illustrate the differences between the proposed method and existing approaches.\n\n**Weaknesses**\n1. The novelty of this paper is somewhat limited. The self-correction capability is an inherent property of DFM models, and while the paper applies it to a unified model, it does not demonstrate any unique advantages in this setting. In my view, this reflects a lack of sufficient motivation.\n2. The results presented in this paper are also not particularly strong. Despite initializing from Janus-1B weights, the model undergoes an additional 43,000 GPU hours of training—roughly equivalent to training a 7B autoregressive MLLM from scratch. However, the final performance only matches that of existing AR-based models, and on many benchmarks, the results are not especially competitive.\n3. The main motivation of this paper lies in the self-correction capability of DFM. However, this is only demonstrated through a few qualitative visualizations. While these are intuitive, they lack statistical rigor by academic review standards. The paper does not provide quantitative experiments to convincingly show that self-correction is truly effective and important for MMU tasks.\n4. The Related Work section of this paper lacks citations to some relevant papers, such as Muse-vl, LlamaFusion and OmniMamba. Including references to these papers would help make this section more comprehensive.\n\nQUESTIONS:\nThe questions are the same as the weaknesses we discussed above. My main concern is whether the motivation for introducing DFM is sufficiently justified. While the self-correction capability is certainly appealing, the paper lacks quantitative experiments to demonstrate the effectiveness of this capability within the unified model. Addressing this issue would  improve my rating of the paper.\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-06-27"}

comments: SUMMARY:
This paper presents FUDOKI, the first unified multimodal generative and understanding model based entirely on discrete flow matching (DFM). The model supports both image and text modalities, featuring a mask-free, bidirectional, and dynamically revisable generation process. Experiments demonstrate that FUDOKI matches or surpasses state-of-the-art autoregressive MLLMs in visual understanding and image generation benchmarks.

STRENGTHS AND WEAKNESSES:
**Strengths**
- Technically sound and well-motivated methodology.
- Unified framework for multimodal generation and understanding.
- Supports dynamic token revision during generation, improving flexibility.
- Strong empirical results on standard benchmarks.
- Novel application of discrete flow matching with kinetic-optimal velocity.

**Weaknesses**
- The paper does not provide experimental results or analysis for more complex multimodal scenarios, such as multi-image conditioning or compositional tasks.
- No comparison is made with recent strong unified AR+DiT models like Bagel, leaving it unclear how FUDOKI performs against state-of-the-art baselines on unified generation and understanding.

QUESTIONS:
The paper does not compare FUDOKI with recent strong unified models like Bagel [1]; could the authors briefly discuss FUDOKI's relative strengths or weaknesses compared to it?
[1] Unified Model for Multimodal Understanding and Generation

LIMITATIONS:
The paper primarily evaluates standard single-image tasks. Broader validation on diverse, practical, or multi-image scenarios, and deeper discussion of societal impacts, would strengthen the work.
TITLE: Official Review
CLARITY: 4
IMPACT: 4
ORIGINALITY: 4
DATE: 2025-06-28

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper introduces FUDOKI, a unified multimodal model for both visual understanding and image generation, built on a discrete flow matching paradigm rather than traditional autoregressive (AR) methods. This shift addresses key AR limitations, such as rigid generation order and weaker reasoning or self-correction.\n\nFUDOKI refines outputs via metric-based probability paths and kinetically optimal velocity fields, allowing for richer bidirectional context and improved generation quality. To ease training, it starts from a pre-trained AR MLLM and transitions gradually to the flow-based framework.\n\nExperiments show that FUDOKI matches or exceeds state-of-the-art AR models on tasks like visual question answering and text-to-image generation. Inference-time strategies (e.g., sampling multiple outputs) further enhance performance, highlighting the model’s flexibility and potential for continued improvement.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n\n1.It is the first work to build a general-purpose vision–language model entirely on discrete flow matching, which advance beyond prior diffusion models relying on basic mask-based corruption. It addresses key limitations of autoregressive multimodal models.\n\n2.The paper is clearly written, with a well-structured presentation. Section 2 provides a solid overview of discrete flow matching, while Section 3 clearly explains the proposed extensions—metric-induced paths and kinetically optimal velocity fields. The distinctions from prior methods, such as mask-based diffusion versus their continuous refinement approach, are effectively illustrated.\n\n3.In terms of experiments, the paper provides thorough validation. FUDOKI is evaluated on a wide range of well-known tasks and benchmarks. \n\n\n\nWeaknesses:\n\n1.Inference speed remains unclear. While the model benefits from iterative refinement and self-correction and show quality-speed Trade-off, it’s not obvious how this translates to actual inference efficiency. There is no direct comparison with optimized AR models in terms of speed or number of steps required to reach similar output quality.\n\n2.Lack of ablations. While the paper introduces several novel components—such as metric-induced probability paths, kinetically optimal velocity fields, and test-time inference scaling—there is no systematic ablation to isolate their individual impact. For example, it would be informative to see how performance changes if a simpler mask-based path replaces the metric-induced one.\n\n3.Limited Evaluation Scope. While FUDOKI shows promising generalization ability and robustness across several visual generation and understanding tasks, its effectiveness on more challenging reasoning tasks or open-ended question answering scenarios remains underexplored. Further evaluation is needed to confirm whether its strengths extend to these more complex settings.\n\nQUESTIONS:\n1.Could the authors clarify the inference speed and step count required by FUDOKI versus an AR model? For instance, how many refinement iterations are typically needed for generation.\n\n2.Do the authors think *any* strong AR language model could be converted to a flow-based model via their adaptation strategy? It would be insightful to discuss if the success depends on specific properties of the chosen base model (Janus-1.5B), or if one could take a larger model or different architecture and apply the same paradigm.\n\n3.The paper introduces multiple novel components. It would strengthen the work to see more ablations.\n\nLIMITATIONS:\nYes.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 4, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-01"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper introduce a multimodal model based on kinetic flow matching. The author leverage pretrained AR model and the formulation from [1] to generate both text and image. For efficient finetuning from AR models, they also adopt strategies from [2] i.e., shifting, and discard the time embedding layers. The paper demonstrate the performance of their models on image understanding and generation are competitive with AR-based approaches. \n\n\n\n\n\n[1] Flow Matching with General Discrete Paths: A Kinetic-Optimal Perspective (ICLR 2025)\n\n[2] Scaling Diffusion Language Models via Adaptation from Autoregressive Models (ICLR 2025)\n\nSTRENGTHS AND WEAKNESSES:\n### Strengths:\n- The model performs competitive with AR models of similar scale on various benchmarks.\n- The paper is well-written and the experiment is well-designed\n\n### Weaknesses:\n- The algorithm itself is not novel:  the kinetic flow matching formulation is introduced in [1], the idea of adopting pre-trained AR for discrete diffusion is proposed in [2] (the author use technique from that paper as well), discarding time embedding layers is proposed and investigated in [3] and previous absorbing diffusion. The novelty mostly lie in the fact that the author extend that to visual domain. In my point of view, this makes their contribution borderline.\n- Similar to previous language diffusion models, the proposed approach operates under a fixed compute budget, which is limited for multimodal generation. For instance, a $1024\\times 1024$ pixel image translates to roughly $4096$ tokens, allocate a \"reasonable\" number of generated tokens during inference phase is not straightforward e.g., $4096$ tokens are generally not efficient for language generation task.\n- The model use two type of image encoders (SigLIP and VAE), thus, training pipeline could be rather cumbersome. For instance, multi-turn chat with multiple images, which is not straightforward to separate the understanding and generation task. In addition, it would be difficult to extend image generation tasks beyond text-to-image e.g., (multiple) image to image generation.  \n\n\n[3] Is Noise Conditioning Necessary for Denoising Generative Models? (ICML 2025)\n\nQUESTIONS:\n- Including a GIF that visualizes the text‐generation process would add valuable insight. From my experience, absorbing‐diffusion models like LLaDA and Dream often behave largely autoregressively, so a clear demonstration of their bidirectional generation capability would be especially informative.\n- It would be helpful if the author add the performance of model with different number of denoising steps. \n- Are experiments run on H100s? Why don't the author fine-tune existing (larger) language diffusion baselines such as LLaDA or Dream (as I believe 43k gpu hours is quite substantial and enough for fine-tuning 7B models)? how straightforward it is to adapt an absorbing diffusion model to the kinetic diffusion?\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-07-01"}

comments: SUMMARY:
This paper introduces a unified model based on discrete flow matching for both generation and understanding tasks by leveraging metric-induced probability paths with kinetic optimal velocities. The model achieves promising results across multiple benchmark datasets.

STRENGTHS AND WEAKNESSES:
Strengths:

This work is the first to use discrete flow matching to train unified models and it achieves promising results on multiple benchmarks.

The approach leverages metric-induced probability paths with kinetic optimal velocities for self-correction.

Weaknesses:

Discrete flow matching requires limiting the output length during inference compared with autoregressive methods, which makes it somewhat inconvenient for understanding tasks.

QUESTIONS:
Questions:

Although the paper applies flow matching to unified models, it uses the AR-based Janus as the base model for training. Could the authors explain why they did not choose MaskGIT or another diffusion model instead?

Beyond benchmark tests, it would strengthen the paper to include other applications. Could the authors provide results on interleaved image generation or on image editing tasks?

LIMITATIONS:
It can be seen in weeknesses.
TITLE: Official Review
CLARITY: 3
IMPACT: 3
ORIGINALITY: 2
DATE: 2025-07-02