Transition Matching: Scalable and Flexible Generative Modeling
Authors: Neta Shaul
Uriel Singer
Itai Gat
Yaron Lipman
Venue: NeurIPS 2025 poster ()
Keywords: generative models
flow matching
diffusion
large scale
multimodal
Primary area: deep_learning
TL;DR: Transition matching (TM) is a discrete-time continuous-state generative modeling that advances both flow/diffusion and autoregressive models. TM variants achieve state-of-the-art text-to-image generation.

Abstract: Diffusion and flow matching models have significantly advanced media generation, yet their design space is well-explored, somewhat limiting further improvements. Concurrently, autoregressive (AR) models, particularly those generating continuous tokens, have emerged as a promising direction for unifying text and media generation, showing improved performance at scale. This paper introduces Transition Matching (TM), a novel discrete-time, continuous-state generative paradigm that unifies and advances both diffusion/flow models and continuous AR generation. TM decomposes complex generation tasks into simpler Markov transitions, allowing for expressive non-deterministic probability transition kernels and arbitrary non-continuous supervision processes, thereby unlocking new flexible design avenues. We explore these choices through three TM variants: (i) Difference Transition Matching (DTM), which generalizes flow matching to discrete-time  by directly learning transition probabilities, yielding state-of-the-art image quality and text adherence. (ii) Autoregressive Transition Matching (ARTM) and (iii) Full History Transition Matching (FHTM) are partially and fully causal models, respectively, that generalize continuous AR methods. They achieve continuous causal AR generation quality comparable to non-causal approaches and potentially enable seamless integration with existing AR text generation techniques. Notably, FHTM is the first fully causal model to match or surpass the performance of flow-based methods on text-to-image task in continuous domains. 
  We demonstrate these contributions through a rigorous large-scale comparison of TM variants and relevant baselines, maintaining a fixed architecture, training data, and hyperparameters.

However, the design space Transition Matching: Scalable and Flexible Generative Modeling
Neta Shf
Uriel Singer* .2
Itai Gat?
Yaron Lipman2
Weizmann Institute of Science, 2FAIR at Meta, Work done during internship at Meta FAIR, "Joint first author
Abstract
Diffusion and flow matching models have significantly advanced media generation, yet their design space is well-explored, somewhat limiting further improvements. Concurrently, autoregressive (AR) models, particularly those generating contin- uous tokens, have emerged as promising direction for unifying text and media generation. This paper introduces Transition Matching (TM). novel discrete-tima continuous-state generative paradigm that unifies and advances both diffusion/floo models and continuous AR generation. TM decomposes complex generation tasks into simpler Markov transitions, allowing for expressive non-deterministic probabil- ity transition kernels and arbitrary non-continuous supervision processes, thereby unlocking new flexible design avenues. We explore these choices through three TM variants: (i) Difference Transition Matching (DTM), which generalizes flow matching to discrete-time by directly learning transition probabilities, yielding state-of-the-an image quality and text adherence as well as improved sampling efficiency. (ii) Autoregressive Transition Matching (ARTM) and (iii) Full History Transition Matching (FHTM) are partially and fully causal models, respectively, that generalize continuous AR methods. They achieve continuous causal AR gener- ation quality comparable to non-causal approaches and potentially enable seamless integration with existing AR text generation techniques. Notably, FHTM is the first fully causal model to match or surpass the performance of flow-based methods text-to-image image task in continuous domains. We demonstrate these contributions through a rigorous large-scale comparison of TM variants and relevant baselines, maintaining a fixed architecture, training data, and hyperparameters.
Introduction
Recent progress in diffusion models and flow matching has significantly advanced media generation
(images, video, audio), achieving state-of-the-art results [31
24
34
4
of these methods has been extensively investigated [43
20
30
40
7 potentially limiting further
significant improvements with current modeling approaches. An alternative direction focuses on autoregressive (AR) models to unify text and media generation. Earlier approaches generated media sequenees of discrete tokens either in raster order [37 54 6); or in random order [3]. Further advancement was shown by switching to continuous token generation 25 47], while also improving
performance scale [10]
This paper introduces Transition Matching (TM), general discrete-time continuous-stan generation paradigm that unifies diffusion/flow models and continuous AR generation. TM aims advance both paradigms and create new state-of-the-art generative models. Similar to diffusion/flow models, TM breaks down complex generation tasks into a series of simpler Markov transitions. However, unlike diffusion/flow. TM allows for expressive non-deterministic probability transition kernels and arbitrary non-continuous supervision processes, offering and flexible design choices.
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

MAR
FHITM (Ours)
DTM (Ours)
MAA
"A portrait of metal statue of pharaok wearing steampunk glasses and leather jacket over white t-shirt that has drawing of a space shuttle on it."
"A solitury figure shrouded in mists peers up from the cobble stone street at the imposing and dark gothic buildings surrounding it. an old-fashioned lamp shines nearby. oil puinting."
Figure 1: Transition Matching methods (FHTM and DTM) compared to baselines (FM and MAR) under fixed architecture, dataset and training hyper-papnee parameters.
We explore these design choices and present three TM variants:
(i) Difference Transition Matching (DTM); A generalization of flow matching to discrete time, DTM directly learns the transition probabilities of consecutive states in the linear (Cond-OT) process instead of just its expectation. This straightforward approach yields state-of-the totte--f-the-rt generation model with improved image quality and text adherence, as well as significantly faster sampling.
(ii) Autoregressive Transition Matching (ARTM) and (iii) Full History Transition Matching (FHTM): These partially and fully causal models (respectively) generalize continuous AR models by incor- porating a multi-step generation process guided by discontinuous supervising processes. ARTM and FHTM achieve continuous causal AR generation quality comparable non-causal methods. Importantly, their causal nature allows for seamless integration with existing AR text generation methods. FHTM the first fully causal model match or surpass the performance of flow-based methods in continuous domains.
In summary, contributions are:
1. Formulating transition matching: simplified and generalized discrete- time generative models based on matching transition kernels.
2. Identifying and exploring key design choices, specifically the supervision process, kernel parameterization, and modeling paradigm.
3. Introducing DTM, which improves upon state-of oor flow matching in image quality, prompt alignment, and sampling speed.
4. Introducing ARTM and FHTM: partially and fully causal AR models (resp.) that match non-AR generation quality and state-of-the-art prompt alignment.
5. Presenting a fair, scage-scale comparison of the different TM variants and relevant baselines using fixed architecture, data, and training

---

2
Transition Matching
We start by describing the framework of Transition Matching (TM), which can be seen as a simplified and general discrete time formulation for diffusion/flow models. Then, we focus on several, unex- plored TM design choices and instantiations that goes beyond diffusion/flow models. In particular: we consider more powerful transition kernels and/or discontinuous noise-to-data processes. In the experiments section we show these choices lead to state-of-the of--te--rttt-tttto-e image generation methods.
2.1 General framework
Notation We use capital letters X, X,Y, Y, Z, A, B denote random variables (RVs) and lower-case letter T, y: 2, a, b to denote their particular states. One exception is time where we abuse notation a and use denote both particular times and RV. All our variables and states reside in euclidean spaces I E Rd. The probability density function (PDF) of a random variable Y is denoted py (x). For RVs X, (and only for them) use the simpler PDF notation pr (2+). We use the standard notations for joints Px, (z. 3) and conditional densities Px y (zly) densities. We denote [T] = {0. IT).
Problem definition Given training set of ii.d. samples from unknown target distribution pr. and some easy to sample source distribution Po- Our goal is to learn a Markov Process, defined by probability transition kernel P ( [2:4, where & [T taking from Xo po to Xr ~ pr. That is, we define series of random variables (Xt)tez) such that Xo ~ po and
X ~ (+Ix,) for all E [T - 1] then Xy ~ pr-
Supervising process Training such a Markov process is done with the help of supervising process, which is a stochastic process (Xo, XI,..., XT) defined given data samples Xr using a condi- Figure 2: Supervising process. tional process To.....T-III1 11T i.e.,
X Xa.....T-1I Xr-1
4o to.... (To..... IT) = -o....T-I|| 1 ( (xT),
FT
and 9o..... denotes the joint probability of the supervising process (X,) The only constraint on the conditional process is that its marginal at time / 0 is the easy sample distribution po, i.e.,
Uo
Po-
Note that this definition is very general and allows, for example, arbitrary non-continuous processes, and indeed utilize such a process below. Transition matching engages with the supervising process
(Xt)ee by sampling pairs of consecutive states Xx+1) ~ (1,+1, t E [T -
Loss The model Pft1le is trained to transition between consecutive states X, - Xi+1 in the sense of equation by regressing 4r+ 1|e defined from the supervising process q. This motivates the loss utilizing distance/divergeene D between distributions
(()) E,, (
where / is sampled uniformly from [ 1]. However, this loss requires evaluating 4++1 which is usually hard to compute. Therefore, to make the training tractable we require that the distance D has an empirical form, i.e., can be expressed as an expectation of an empirical one-sample loss D over target samples. We define the loss
D empirical form
C(0)= = -,,x, Ex.., D(X p P (1xx)))) Ex,x,. D(Stxx,,,,((Aeis... D(Xe+1. (1
(5)
where (X,, X(+1) ) are sampled from the joint q1,41 with the help of equation namely, first sample data XT ~ pr and then (X,, X ~ (t. (-) Notably, equation can be used to learn arbitrary transition kernels, in contrast e.g., Gaussian kernels used in discrete time diffusion models or deterministic kernels used in flow matching. The particular choice of the cost D depends on the modeling paradigm chosen for the transition kernel Pi and discussed later.
3

---

Algorithm 1 Transition Matching Training
Algorithm 2 Transition Matching Sampling
Require: PT
b Data
Require: Po
P Source distribution
Require: 9.Y(T
P. Process
Require: Pyle
Trained model
Require: T
P. Number of TM steps
Require: 1++1leY
Parametrization
1: while not converged do
Require: T
b Number of TM steps
2: Sample t ~ 1[[ Xy Noppp Pr
1: Sample Xo ~ Po
3: Sample (X,,Y) ~ (.x)
2: for = 0 to T - 10-10 do
4:
[(0) - p (e))(-(x))
3: Sample Y P3y (+
5:
0 0+0 --0-7Vo b Optimization step
4: Sample X2+1 ~ Q+t1l, (-|X,, Y)
6: end while
5: end for
7: return 0
6: return X7
Kernel parameterizations The first and natural option to parameterize p P It is to regress qc+ directly as is done in equation 5 This turns out to be a good modeling choice in certain cases. However, in some cases one can use other parameterizations that turn out to be beneficial, as is also done for flow and diffusion models. To do that in the general case, we use the law of total probability applicd to the conditional probabilities 4++1/t with some latent RV Y:
ge+ g (ma) = ff
=
- (e+ p (vlz))dy
(6)
where gy is the posterior distribution of Y given X; 2z and g is easy to sample (often deterministic function of X, and Y). Then the posterior of Y is set as the new target of the learning
process instead of the transition kernel. That is, instead of the loss in equation 5 we consider
[(0)- = E,,x,y D(Y, (-)x)))))))
(7)
Similarly, during training, sampling from the joint (X,, Y) ~ ot,,, is accomplished by first sampling data Xy and then (X,, Y) ~ (- Once the posterior pry is trained, sampling from the transition Pl+ during inference is done with the help of equation 6 To summarize, cases we want to use non-trivial kernel parameterization, i.e., f X1+1, we sample from [++1jt.Y y (in sampling) and 4r,1I (in training). See Algorithms [
and 2 the training and sampling pseudocodes.
Kernel modeling Once desirable Y identified. the remaining part is to choose a generative model for the kernel PLy1 Importantly, one of the key advantages in TM comes from choosing expressive kernels that result in more elaborate transition kernels than used previously. A kernel modeling is set by a choice of a probubility model for pr and a loss to learn We denote the probability model choice by BIA, where A denotes the condition and B the target. For example, Y|X, will denote model that predicts a sample of Y given sample of X₁. We will also use more elaborate probability models, such as autoregressive models. To this end, consider the state Y reshaped into individual tokens Y (Y',. ....., Y"), and then Y (Y<i, Xt) means that our model samples the token Y given previous tokens of Y, y<i (Y1 .y.,,.., yi-1), and X₂-
All our models are learned with flow matching (FM) loss. For completeness, we provide the key components of flow matching formulated generically to learn to sample from B A. Individual states of A and B are denoted and b, respectively. Flow matching models P/ via velocity field un (bla) that is used to sample from (+1a) by solving the Ordinary Differential Equation (ODE)
dB. ds
=B.laa
(8)
initializing with a sample Bo ~ N/0, I) (the standard normal distribution) and solving until In turn, B, is the desired sample, i.e., B ~ P The loss D, used train FM, has an empirical form and minimizes the difference between 4BtP and P/
D ((B.Pbia((a)) E. v. 11 (B.la) (B- (B-B))l²
(9)
where is sampled uniformly in [0. 1]. Bo ~ N/(0 1), B (B)a(-1a), and B, (1 Bs + sB.

---

We summarize the key design choices in Transition Matching:
Supervising process
Parametrization
Modeling
TM design:
B
2.2 Transition Matching made practical
The key contribution of this paper is identifying previously unexplored design choices in the TM framework that results in effective generative models. We focus on two TM variants: Difference Transition Matching (DTM), and Autoregressive Transition Matching (ARTM/FHTM).
Difference Transition Matching
Our first instance of TM makes the following choices:
DTM: Supervising process
Parametrization
Modeling
X, linear
XT Xo
B A
As the supervising process we use the standard linear process (a.k.a,, Conditional Optimal Trans- port), defined by
X, (1 (1- Xo + XT.
t [T],
(10)
where X, Po (O, 7) and XT Pr- This is the same process used in [27 28 For the kernel parameterization Y we will use the difference latent (see Figure 3 left),
Xx X
(11)
During training, sampling yr (+Ix) (i.e., given Xr) is done by sampling Xo. and using and compute X,,Y Y. Using the definition in 10 and rearranging gives
X X; + 7
(12)
and this equation can be used to sample from 4e+1jerr (-)X,, (-IXX, Y) during inference. See Figure forr an illustration of a sampled path from this supervising process. We learn to sample from the posterior P qy|e using flow matching with = (t, X.) and B Y. This means we learn a velocity field u. 2x) and train it with
X
Figure Difference prediction given X, (left) and flow matching velocity u, (X,) (right).
Algorithm [ and the CFM loss in equation 9
Note that in this case one can also learn a continuous time E sampling.
[0, T] which allows more flexible
The last remaining component is choosing the architecture for u. Let x²)) be a reshaped state to n tokens. For example, each x² can represent patch in an image æ. Next, note that in each transition step we need to sample Y ~ (- by approximating the solution of the ODE in equation 8 Therefore, to keep the sampling process efficient, we follow 25 and use a small head ga that generates all tokens in a batch and is fed with latents from a large backbone f". Our velocity model is defined as
X
== (x (x"",...,"""
Figure 4: DTM path sampled with eq.
(yltt, xT)) = (v',, ,h/). (t)........... ( (4))
(13)
where hf is the i-th output token of the backbone. i.e., [n] hg ... _,, = fx(x) See Figure (DTM) for an illustration of this architecture. One limitation this architecture worth mentioning is that in each transition step, each token y is generated independently. which limits the power of this kernel. We discuss this in the experiments section but nevertheless demonstrate that DTM with this architecture still leads to state-of-the-art image generation model.
28 is deterministic process while DTM samples from stochastic transition kernel in each step, connection between the two is revealed by noting that the expectation of a DTM step coincides with Flow Matching Euler step, i.c.
Connection to flow matching Although flow matching 27
E [ x] = E [Xr Xo æ] uz(2),
(14)
5

---

backbone
backbene
head
backouua
head
DTM
ARTM
FHTM
Figure 5: Architectures of the methods suggested in the paper. Backbone (orange) is the main network (transformer); head (green) is small network (2% backbone parameters); blue tokens use full attention, gray tokens are causal; u² is the output velocity.
which is exactly the marginal velocity in flow matching, see Figure 3 In fact, as T + oc (or equivalently, steps are getting smaller), DTM is becoming more and more deterministic, converging to FM with Euler step, providing a novel and unexpected elementary proof (i.e., without the continuity equation) for FM marginal velocity. In Appendix C
we prove
Theorem 1. (informal) As the number of steps increases, T - 0o, DTM converges to Euler step FM,
Xeth R In 7 E(XT - Xo = =z],
as k/T O, where X Ve L defined by Algorithm 2 with a optimally trained ph
We attribute the empirical success of DTM over flow matching to its more elaborate kernel.
Autoregressive Transition Matching Our second instance of TM geared towards incorporating state-of-the oh-the-rrtttttooo art media generation in autoregressive models, and utilizes the following choices;
ARTM: Supervising process
Parametrization
Modeling
X, independent linear
Y = =X++1
B B-Y (t, X, X
In this case we use a novel supervising process we call independent linear process, defined by
T -
t Xo. +
t T Xr, t [T),
X;
1
(15)
where Xo.t (), T are all i.i.d. samples. Sampling [e.t11p((| (-Tx) is done by sampling Xo.: and Xo,++1 and using [5 Although the independent linear process has the same marginals qe as the linear process in equation| 10 it enjoys better regularity of the conditional ge+1je (+(xx)) see Figure o for an illustration, and as demonstrated later in experiments key for building state-of-the-aa Autoregressive image generation models.
For the transition kernel we use an Autoregressive (AR) model with the choice of Y = X As before, we a state written as series of tokens (x=1, "." using the probability chain rule (as usual in AR modeling),
Xa
X, X
Xo,
X X
Xa.
r²) and write the target kernel 4++1/
XT
XTT
Figure 6: Linear process (left) and independent linear process (right) showing possible X given a sample X,. The independent process has much wider support for X given Xt.
1641je (X
II 1=1
X X
where X the empty state. We will learn
sample from o'41 using FM with = (t, X1, and Xi+1 That is, we learn a velocity field (y'l (t, Te, a trained with the CFM loss in equation 9 This method builds upon the initial idea (25] that uses such AR modeling to map in a single transition step from Xo to XT using diffusion, and in that sense ARTM is generalization of that method. Lastly the architecture for uy is based on a similar construction to DTM with a few, rather minor changes. Using the same notation for the head g" and backbone fd models we define
( 21, 211) (v' hi41),
(16)
with h441 f2 x111 Figure (ARTM) shows illustration of this architecture.

---

Full-History ARTM We consider a variant of ARTM that allows full "teacher- "teacher-forcing forcing" training and consequently provides a good candidate to be incorporated into multimodal AR model.
Supervising process
Parametrization
Modeling
FHTM:
Xs independent linear
Y =
B B-Y-AA (Xo.....
................................ Xx, x
The idea is to use the full history of states, namely considering the kernel
ge+ 1
X,) = II i=1
1 (X Xoo Xt, Xx11)
(17)
and train an FM sampler from o ijo....... with the choices A (Xo...., Xx, Xx,1) (no need to add time due to the full state sequence) and B = -Xi+1- The architecture of the velocity u, is defined by
"g(y' Fo ................................ Ft. x (o ('', M(+1),
(18)
with h1+1 = f (2o........ It. 2 a1t)) and we take f to be fully causal. See Figure (HTMM.
Related work
Diffusion and flows We draw the connection to previous works from the perspective of transition matching. Diffusion models [41 16 42 21 can be seen as an instance TM by choosing D the
loss (5) to be the KL divergence, derived diffusion literature as the variational lower bound [22]. The popular e-prediction [16] in transition matching formulation is achieved by the design choices
Supervising process
Parametrization
Modeling
e-prediction:
X, No + a,Xr
Y = Xo
YJX, ~ (Y ( (X),)
where (ar, a;) is the scheduler, and non-zero wy reproduces the sampling algorithm in [16], while taking the limit w: - yields the sampling of [42). Similarly, x-prediction [21] is achieved by the parametrization Y = XT. In contrast these work, our TM instantiations use more expressive kernel modeling. Relation to flow matching 27 28 1 is discussed in Section 2.2 Generator matching [17 generalizes diffusion and flow models to general continuous time Markov process modeled with arbitrary generators, while we focus on discrete time Markov processes. Another line works adopted supervision processes that transition between different resolutions; [19 used flow matching with a particular coupling between different resolution as the kernel modeling; (56) implemented a similar scheme but allowed the FM be dependent the previous states (frames) in AR manner. Denoising Diffusion GANs (50) uses st-preiccioon parametrization Y = Xy and utilize a GAN [12) model as the transition kernel. In concurrent work, (57] proposes similar parameterization DTM (Y = XT - Xn) however uses a backbone-only architecture consequently making transition sampling computationally very expensive and sub-par in generation quality compared to the backbone-head architecture.
Autoregressive image generation Early progress in text-to-image generation achieved using autoregressive models over discrete latent spaces [37 g [44, with recent advances [46] claiming to surpass flow-based approaches. A complementary line of work explores autoregressive modeling directly continuous space 255 47 demonstrating some advantages over discrete methods. In 110 this direction scaled further, achieving SOTA results. In experiments
compare these models in controlled setting and show that our autoregressive transition matching variants improves upon these models and achieves SOTA text-to-image performance with a fully causal architecture. Lastly, DART-AR [ uses a supervising process similar to the independent linear process [5] with an autoregressive backbone however utilizing a Gaussian transition kernel per patch in contrast to an FM head used in our case.

---

PAINT
LeT's POINT
rich paaknn of paay cader d.vand ANEN juble EORE ENCITONN AANE
DEEP LEARNING
VEEREY DEET LEARINIIGG
Veriy dep Lerning
Rewe Beop
sppirrmer
Figure 7: Samples comparison of our DTM, FHTM vs. FM, and MAR; Images generated on similar DiT models trained for 1M iterations.
Experiments
TM (DTM), with 7 32 TM steps, Autoregressive TM (ARTM-2,3) with T 2.3 (resp.), and Full History TM (FHTM-2.3) with T = 2, 3 (resp.) - on the text-to-image generation task. In Appendix B we provide training and sampling pseudocodes of the three variants in Algorithms 3 and python code
We evaluate the performance of our Transition Matching (TM)
for training in Figures 26 and 27 Our baselines include flow matching (FM) [9, continuous-
token autoregressive (AR) and masked AR (MAR) (25] and discrete-token AR [54] and MAR [3]
For continuous-token MAR we include two baselines: the original truncated Gaussian scheduler
version 25 and the cosine scheduler used by Fluid (MAR-Fluid) 10].
Datasets and metrics Training dataset is a collection of 350M licensed Shutterstock image-caption pairs. Images are of 256 x 256 x 3 resolution and captions span 1-128 tokens embedded with the CLIP tokenizer (35] Consistent with prior work [38] for continuous state space, the images are embedded using the SDXL-VAE
3 into a 32 x 32 latent space, and subsequently all
model training are done within this latent space. For discrete state space, images are tokenized with Chameleon- VQVAE 22. Evaluation datasets are PartiPrompts [54] and AS-COOC (26] text/image benchmarks. And the reported metrics are: CLIPScore [15]. that emphasize prompt alignment; Aesthetics 39 and DeQA Score
5X that focus on image quality; PickScore 23), ImageReward [51],
and Undr [49 which are human preference-haie based and consider both image quality and text
adherence. Lastly, we report results on the GenEval [1] and 121-CompBench [18] benchmarks.
Architecture and optimization All experiments are performed with the same 1.7B parameters DiT backbone (f³) [32], excluding single case in which we compare to standard LLM architecture 48 29 Methods that require a small flow head (g²), replace the final linear layer with a 40M parameters
MLP 25 Text conditioning is embedded through Flan-UL2 encoder (45) and injected via cross attention layers, or as prefix in the single case of the LLM architecture. Finally, the models are trained for 500K iterations with 2048 batch size. Precise details are Appendix AA.I We aim to facilitate fair and useful comparison between methods in large scale by fixing the training data, using the same size architectures with identical backbone (excluding the L.M architecture that use standard transformer backbone), and same optimization param-parameters... To this end, we restrict our comparison to baselines which we re-implemented.
4.1 Main results: Text-to-image generation
Our main evaluation results are reported in Tables and X (in Appendix) on the DiT architecture. We find that DTM outperforms all baselines, and yields the best results across metries except the CLIPScore, where on the PartiPrompts benchmark it is runner-up to MAR and our ARTM-3 and FHTM-3. On the MS-COCO benchmark, the discrete-state space models achieve the highest CLIPScore but lag behind on all other metrics, well as the GenEval benchmark. DTM

---

shows considerable gain in text adherence over the baseline FM and sets new SOTA on the text-to-image task. Next, our AR kernels with TM steps: ARTM-3 and FHTM-3, demonstrate significant improvement compared to the AR baseline, see comparison of samples in Figure 13 in the Appendix. When compared to MAR, ARTM-3 and FHTM-3 have comparable CLIPScore, but improve considerably on all other image quality metrics, where this is also noticeable qualitatively in Figure 7 and Figures
[1 in the Appendix. GenEval and 22-Commbenchh results are reported in Tables 2 and 9 (resp.) showing that overall DTM is leading with PHTM-3/ARTMV 3 and MAR closely follows. To our knowledge, FHTM the first fully causal model to match FM performance on text-to-image task in continuous domain, with improved text alignment.
Table I: Evaluation of TM vs. baselines PartiPrompts. t Inference with activation caching. NFE"
counts only backbone model evaluation (f"), LLM and DiT have comparable number of parameters.
Atlention
Kernel
Arch
NHE
CLIPSoore
PickScore 1 ImageRoward T
UniliedRoward
Aesthecic
DeQA Soore"
MAR- -iscceeee
DIT
256
26.8
20.7
0.14
4.31
5.15
2.48
Buseline
MAR
256
0.33
4.95
Full
MAR-Fliid
DIT
256
26.0
20.5
0.07
3.82
4.74
2.36
FM
DIT
250
26.0
21.0
0.23
4.78
5.29
2.55
TM
DTM
DIT
32
26.8
21.2
0.53
5.12
5.42
2.65
Easeline
AR-disenatel
DIT
230
26.7
20.4
-0.01
3.71
4.81
2.38
ART
256
ARTM 2
DiT
2 256
26.8
20.8
0.29
4.49
5.03
2.37
Causal
FHTM-22 22
DIT
2 256
26.8
20.8
0.30
4.59
5.13
2.44
TM
ARTM- 33
DIT
3 258
27.0
20.9
0.38
4.77
5.21
2.53
31
3xx
0.31
FHIM
I.M
256
21.0
0.43
5.02
5.30
2.54
Table 2: Evaluation of TM versus baselines on GenEval; same settings as Table
Kernel
Arch
Overall 1 Single-object
Two-abjects
Counting
Colors
Position
Attribute
MiR-diieeete
INT
256
0.43
0.66
0.13
0:291
Baseline
MAR
DXT
256
0.52
0.98
0.56
0.43
0.73
0.11
0.38
MAR Fluid
D&T
256
0.44
0.90
0.33
0.37
0.76
0.12
0.28
DST
256
0.52
0.71
0.34
TM
DIM
INT
32
0.54
0.93
0.58
0.35
0.79
0:20
0.46
AR-discretel
D&T
256
0.41
0.06
0.40
0.33
0.60
0.07
0.19
Baseline
DRT
256
0.34
0.26
0.15
0.63
0.06
0.15
Cansal
ARTM
INT
2566
0.40
0.51
0.11
0.27
FHTM 2
DXT
2 256
0.48
0.96
0.48
0.25
0.78
0.09
0.37
TM
ARTM-31
D&T
256
0.51
0.95
0.54
0.41
0.79
0.16
0.28
FHTM-93 3/
2266
0.32
0.54
0.74
0.16
FHTM 3²
L.M
3 256
0.49
0.94
0.55
0.37
0.00
0.17
0.29
Image generation with causal model Beyond improving prompt alignment and image quality in sot-tto-imaee task, a central goal recent re- search [58 [55 is to develop multimodal models also capable of reasoning about images. This direction aligns naturally with our approach, the fully causal FHTM variant enables seamless integration with large-language models (LLM) standard architecture, training, and inference al- gorithms. As a first step toward this goal, we demonstrate in Table [ and & that FHTM, im- plemented with an LLM architecture replacing 2D with ID positional encoding and input the text condition only at the first layer, can match and even surpass the performance of approximately the same size DiT architecture. Furthermore,
Table 3: Global ranking of and baselines on the benchmarks PartiPrompts, MS-COCO, GenEval,
andT2J-CompBench: same settings as Table !
Attention
Kerne
Arch
NHR:
Glabal rank
MAAR-discette
DiT
256
200
MAR
DiT
256
Raseline
Full
MAR-Flal Fluid
DiT
256
220
FM
DIT
256
179
TM
DIM
DiT
32
Baseline
AR-discrere
DIT
256
245
AR
DIT
256
321
Causal
ARTM
DIT
2 236
184
TM
FHTM 2
DIT
2 256
185
FHTM-3"
DIT
3 256
130
FHTM-2
LLM
matches or improve upon all baselines across all metries. Further implementation details are in Appendix A.

---

Global ranking As part of our main effort to empirically validate the Transition Matching framework, we trained 6 variants: DTM, ARTM-2/3, FHTM-2/3, and FHTM-3 (LLM) and 6 baselines: FM. MAR, MAR-Fluid, MAR-discrete AR, AR-discrete. All models were evaluated on four benchmarks: PartiPrompts, GenEval, MS-COCO, and 121-CompBench (Tables ( In total, we considered 12 models and 27 metrics per model. To derive single measure of overall performance, we assigned each model a rank from I (best) to 12 (worst) per metric and summed them across all benchmarks.
Table 3 reports the global rank, where DTM substantially outperforms all other TM variants and baselines, followed by FHTM, ARTM, and MAR.
4.2 Evaluations
Sampling efficiency One important benefit in the DTM variant its sampling efficiency compared to flow matching. In Table 10 we report CLIPScore and PickScore for DTM and FM for different numbers of backbone and head steps while in Table we log the corresponding forward times. Notably, the number of backbone forwards in DTM sampling can be reduced con-
Table 4: FM and DTM sampling
times.
Kereel
time (sec)
CLIPScore
PickScore
FM
10.8
26.0
21.0
DTM
1.6
26.8
21.1
siderably without sacrificing generation quality. Table A the superior sampling efficiency of DTM over FM: DTM achieves state-of off-the-an art results with only 16 backbone forwards, leading to an almost 7-fold speedup compared to FM, which requires 128 backbone forwards for optimal quality in this case. In contrast to DTM, ARTM/FHTM do not offer any speed-up. in fact they require backbone forwards equal to the number of transition steps times the number of image tokens, specified in Tables 22 demonstrates that this number be reduced up to 4 with some limited reduction in performance for ARTM/FHTM sampling.
CLIPScore and PickScore for different number of head forwards which
Dependent vs. independent linear process To highlight the impact of the supervising process, we compare the linear process (10), where Xg is sampled once for all / E [T]. with the independent linear process (15), where Xo.t is sampled for each t [T] independently,
autoregressive kernels: ARTM-3 and FHTM-3. The models are trained for 100K iterations and CLIPScore and PickScore evaluated every 10K iterations. As shown in Figure 9 the independent linear process is far superior to the linear process on these kernels, see further discussion in Appendix A..4
DTM Kernel expressiveness The DTM kernel (see equation 13 generates each token yf of dimension 2 x 2 4, corresponding to an image patch, independently in each transition step. This architecture choice is done mainly for performance reasons to allow efficient transitions. In Figure I0] we compare performance using a higher dimension y", corresponding to a x 8 x patches. As can be seen in these graphs, performance improves for this larger patch kernel for low number of transition steps (1-4 steps) and surprisingly stays almost constant for very low number of head step, up to even a single step. The fact that performance does not improve for larger number of transition steps can be partially explained with Theorem that shows that larger number of steps result in a simpler transition kernel (which in the limit coincides with flow matching).
5 Conclusions
We introduce Transition Matching (TM), a novel generative paradigm that unifies and generalizes diffusion, flow and continuous autoregressive models. We investigate three instances of TM: DTM, which surpasses state-of-tt thettrrt flow matching in image quality and text alignment; and the causal ARTM and fully causal FHTM that achieve generation quality comparable to non-causal methods. The improved performance of ARTM/FHTM comes at the price of a higher sampling cost, i.c., NFE counts are proportional to the number of transition steps. e.g., in Table I DTM, in contrast, requires less backbone forwards and leads to significant speed-up over flow matching sampling, see e.g., Table Future research directions include improving the training and/or sampling via different time schedulers and distillation, as well as incorporating FHTM in a multimodal system. Our work does not introduce additional societal risks beyond those related to existing image generative models.
10

---

References
[1] Michael S Albergo and Eric Vanden-Eijnden. Building normalizing flows with stochastic interpolants. arXiv preprint arXiv:2209. 2022.
[2] Chamelcon-Team Chameleon: Mixed-modal early-fusion foundation models, 2025,
[3] Huiwen Chang, Han Zhang, Lu Jiang, Ce Liu, and gedthnoddinrhccilllliiiiiiiii T. Freeman. Maskgit: Masked generative image transformer, 2022.
[4] Yushen Chen, Zhikang Niu, Ziyang Ma, Keqi Deng. Chunhui Wang, Jian Zhao, Kai Yu, and Xie Chen. F5-tts: fairytaler that fakes fluent and faithful speech with flow matching. arXiv preprint arXiv:2410.0688) 2024.
[5] Mostafa Dehghani, Basil Mustafa, Josip Djolonga, Jonathan Heek. Matthias Minderer, Mathilde Caron, Andreas Steiner, Joan Puigcerver, Robert Geirhos, Ibrahim Alabdulmohsin, Avital Oliver, Piotr Padlewski, Alexey Gritsenko, Mario Lučić, and Neil Houlsby. Patch n' pack: Navit, vision transformer for any aspect ratio and resolution, 2023.
[6] Prafulla Dhariwal, Heewoo Jun, Christine Payne, Jong Wook Kim, Alec Radford, and Πly Sutskever. Jukebox: A generative model for music, 2020.
[7] Prafulla Dhariwal and Alex Nichol. Diffusion models beat gans on image synthesis, 2021.
[8] Ming Ding, Zhuoyi Yang, Wenyi Hong, Wendi Zheng, Chang Zhou, Da Yin, Junyang Lin, Xu Zou, Zhou Shao, Hongxia Yang, and Jie Tang. Cogview: Mastering text-to-image generation via transformers, 2021.
[9] Patrick Esser, Sumith Kulal, Andreas Blattmann, Rahim Entezari, Jonas Müller, Harry Saini, Yam Levi, Dominik Lorenz, Axel Sauer, Frederic Boesel, Dustin Podell, Tim Dockhorn, Zion English, Kyle Lacey, Alex Goodwin, Yannik Marek, and Robin Rombach. Scaling rectified flow transformers for high-resolution image synthesis, 2024.
[10] Lijie Fan, Tianhong Li, Siyang Qin, Yuanzhen Li, Chen Sun, Michael Rubinstein, Deqing Sun, Kaiming Hc, and Yonglong Tian. Fluid: Scaling autoregressive text-to-image generative models with continuous tokens, 2024.
[11] Dhruba Ghosh, Hanna Hajishirzi, and Ludwig Schmidt. Geneval: An fojuct-focused framework for evaluating text-to-image alignment, 2023.
[12] Ian Goodfellow, Jean Pouget-Abadic Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Generative adversarial networks, 2014.
[13] Jiatao Gu, Yuyang Wang, Yizhe Zhang, Qihang Zhang, Dinghuai Zhang, Navdeep Jaitly, Josh Susskind, and Shuangfei Zhai. Dart: Denoising autoregressive transformer for scalable text-to-image generation, 2025.
[14] Jian Han, Jinlai Liu, Yi Jiang, Bin Yan, Yuqi Zhang, Zehuan Yuan, Bingyue Peng, and Xiaobing Liu. Infinity: Scaling bitwise autoregressive modeling for high-resolution image synthesis, 2024.
[15] Jack Hessel, Ari Holtzman, Maxwell Forbes, Ronan Le Bras, and Yejin Choi. Clipscore: A reference-free evaluation metric for image captioning, 2022.
[16] Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models, 2020.
[17] Peter Holderrieth, Marton Havasi, Jason Yim, Neta Shaul, Itai Gat, Tommi Jaakkola, Brian Karrer, Ricky T. Q. Chen, and Yaron Lipman. Generator matching: Generative modeling with arbitrary markov processes, 2025.
[18] Kaiyi Huang, Chengqi Duan, Kaiyue Sun, Enze Xie, Zhenguo Li, and Xihui Liu. T2i- compbench++: An enhanced and comprehensive benchmark for compositional text-to-image generation, 2025.
11

---

[19] Yang Jin, Zhicheng Sun, Ningyuan Li, Kun Xu, Kun Xu, Hao Jiang, Nan Zhuang, Quzhe Huang, Yang Song, Yadong Mu, and Zhouchen Lin. Pyramidal flow matching for efficient video generative modeling, 2025.
[20] Tero Karras, Miika Aittala, Timo Aila, and Samuli Laine. Elucidating the design space of diffusson-based generative models. Advances in neural information processing systems, 35:26565-265777 2022.
[21] Diederik Kingma, Tim Salimans, Ben Poole, and Jonathan Ho. Variational diffusion models, 2023.
[22] Diederik Kingma, Max Welling, al. Auto-encoding variational bayes, 2013.
[23] Yuval Kirstain, Adam Polyak, Uriel Singer, Shahbuland Matiana, Joe Penna, and Omer Levy. Pick-a-pic: An open dataset user preferencees for text-to-image generation, 2023.
[24] Black Forest Labs. Flux. http:://ggthhh.occo////oooreertt wwwssoorrrttttttlllleeeee 2024.
[25] Tianhong Li, Yonglong Tian, He Li, Mingyang Deng, and Kaiming He. Autoregressive image generation without vector quantization. Advances in Neural Information Processing Systems, 37:56424-56445, 2024.
[26] Tsung-Yi Lin, Michael Maire, Serge Belongie, Lubomir Bourdev, Ross Girshick, James Hays, Pietro Perona, Deva Ramanan, C. Lawrence Zitnick, and Piotr Dollár. Microsoft coco: Common objects in context, 2015.
[27] Yaron Lipman, Ricky TQ Chen, Heli Ben-Hamu, Maximilian Nickel, and Matt Le. Flow matching for generative modeling. arXiv preprint arXiv:2210.027447 2022.
[28] Xingchao Liu, Chengyue Gong, and Qiang Liu. Flow straight and fast: Learning to generate and transfer data with rectified flow. arXiv preprint arXiv:2209.03003, 2022.
[29] Llama 3 Team Meta. The Ilama 3 herd of models, 2024.
[30] Alex Nichol and Prafulla Dhariwal. Improved denoising diffusion probabilistic models, 2021.
[31] Patrick, Robin Rombach, and Björn Ommer. Taming transformers for high-resolution image synthesis, 2021.
[32] William Peebles and Saining Xie. Scalable diffuuson models with transformers, 2023.
[33] Dustin Podell, Zion English, Kyle Lacey, Andreas Blattmann, Tim Dockhorn, Jonas Müller, Joe Penna, and Robin Rombach. Sdxl: Improving latent diffusion models for high-resolution image synthesis, 2023.
[34] A Polyak, A Zohar, A Brown, Tjandra, A Sinha, A Lee, A Vyas, B Shi, CY Ma. CY Chuang, et al. Movie gen: A cast of media foundation models, 2025. URL htips://arxiv. 1377, page 51.
[35] Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agar- wal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and Ilya Sutskever. Learning transferable visual models from natural language supervision, 2021.
[36] Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J. Liu. Exploring the limits of transfer learning with unified text-to-text transformer, 2023.
[37] Aditya Ramesh, Mikhail Pavlov, Gabriel Goh, Scott Gray, Chelsea Voss, Alec Radford, Mark Chen, and Ilya Suiskever. Zero-shot text-to-image generation, 2021.
[38] Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bjöm Ommer. High-resolution image synthesis with latent diffusion models, 2022.
12

---

[39] Christoph Schuhmann, Romain Beaumont, Richard Vencu, Cade Gordon, Ross Wightman, Mchdi Cherti, Theo Coombes, Aarush Katta, Clayton Mullis, Mitchell Wortsman, Patrick Schramowski, Srivatsa Kundurthy, Katherine Crowson, Ludwig Schmidt, Robert Kaczmarczyk, and Jenia Jitsev. Laion-5b: An open large-scale dataset for training next generation image-text models, 2022.
[40] Neta Shaul, Ricky T. Q. Chen, Maximilian Nickel, Matt Le, and Yaron Lipman. On kinetic optimal probability paths for generative models, 2023.
[41] Jascha Sohl-Dickstein, Eric A. Weiss, Niru Maheswaranathan, and Surya Ganguli. Deep unsupervised learning using nonequilibrium thermodynamics, 2015.
[42] Jiaming Song, Chenlin Meng, and Stefano Ermon. Denoising diffusion implicit models, 2022.
[43] Yang Song, Jascha Sohl-Dickstein, Diederik P. Kingma, Abhishek Kumar, Stefano Ermon, and Ben Poole. Score-based generative modeling through stochastic differential equations, 2021.
[44] Peize Sun, Yi Jiang, Shoufa Chen, Shilong Zhang, Bingyue Peng, Ping Luo, and Zehuan Yuan. Autoregressive model beats diffusion: Llama for scalable image generation, 2024.
[45] Yi Tay, Mostafa Dehghani, Vinh Q Tran, Xavier Garcia, Jason Wei, Xuezhi Wang, Hyung Won Chung. Siamak Shakeri, Dara Bahri, Tal Schuster, et al. U12: Unifying language learning paradigms. arXiv preprint arXiv:2205.0513/ 2022.
[46] Keyu Tian, Yi Jiang, Zehuan Yuan, Bingyue Peng, and Liwei Wang. Visual autoregressive modeling: Scalable image generation via next-scale prediction, 2024.
[47] Michael Tschannen, Cian Eastwood, and Fabian Mentzer. Givt: Generative infinite-vocabue vocabulary transformers, 2024.
[48] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need, 2023.
[49] Yibin Wang, Yuhang Zang, Hao Li, Cheng Jin, and Jiaqi Wang. Unified reward model for multimodal understanding and generation, 2025.
[50] Zhisheng Xiao, Karsten Kreis, and Arash Vahdat. Tackling the generative learning trilemma with denoising diffusion gans, 2022.
[51] Jiazheng Xu, Xiao Liu, Yuchen Wu, Yuxuan Tong, Qinkai Li, Ming Ding, Jie Tang, and Yuxiao Dong. Imagereward: Learning and evaluating human preferences for text-to-image generation, 2023.
[52] Yilun Xu, Mingyang Deng, Xiang Cheng, Yonglong Tian, Ziming Liu, and Tommi Jaakkola. Restart sampling for improving generative processes. Advances in Neural Information Process- ing Systems, 36:76806-7633 76838, 2023.
[53] Zhiyuan You, Xin Cai. Jinjin Gu, Tianfan Xue, and Chao Dong. Teaching large language models to regress accurate image quality scores using score distribution, 2025.
[54] Jiahui Yu, Yuanzhong Xu, Jing Yu Koh, Thang Luong, Gunjan Baid, Zirui Wang. Vijay Vasudevan, Alexander Ku, Yinfei Yang. Bureu Karagol Ayan, Ben Hutchinson, Wei Han, Zarana Parekh, Xin Li, Han Zhang, Jason Baldridge, and Yonghui Wu. Scaling autoregressive models for content-rich text-to-image generation, 2022.
[55] Lili Yu, Bowen Shi, Ramakanth Pasunuru, Benjamin Muller, Olga Golovneva, Tianlu Wang, Arun Babu, Binh Tang, Brian Karrer, Shelly Sheynin, et al. Scaling autoregressive multi-modal models:
Pretraining and instruction tuning, arXiv preprint #xXiv:2309.0259, 2023.
[56] Zhihang Yuan, Yuzhang Shang, Hanling Zhang, Tongcheng Fang, Rui Xie, Bingxin Xu, Yan Yan, Shengen Yan, Guohao Dai, and Yu Wang. E-car: Efficient continuous autoregressive image generation via multistage modeling, 2024.
[57] Yichi Zhang, Yici Yan, Alex Schwing, and Zhizhen Zhao. Towards hierarchical rectified flow, 2025.
13

---

[58] Chunting Zhou, Lili Yu, Arun Babu, Kushal Tirumala, Michihiro Yasunaga, Leonid Shamis, Jacob Kahn, Xuezhe Ma. Luke Zettlemoyer. and Omer Levy. Transfusion: Predict the next token and diffuse images with one multi-modal model, 2024.
14

---

NeurIPS Paper Checklist
Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
2. Limitations
Question:
Does the paper discuss the limitations the work performed by the authors?
Answer: [Yes]
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?
Answer: [Yes]
4. Experimental result reproducibility
Question: Does the paper fully disclose all the information needed reproduce the main ex- perimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Answer: [Yes]
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [No]
Justification: We use licensed data, code will be potentially released at a later date, all implementation details are provided in the main paper and appendix.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
7. Experiment statistical significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [No]
Justification: We follow the standard practice for the benchmarks and evaluations include in the paper.
8. Experiments compute resources
Question: For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Answer: [Yes]
9. Code ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurlPS Code of Ethics
Answer: [Yes]
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative socictal impacts of the work performed?
15

---

Answer: [No]
Justification:
Foundational research and not tied to particular applications, let alone deploy-
ments.
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [No]
Justification: Foundational research and not tied to particular applications, let alone deploy- ments.
12. Licenses for existing assets
Question: Are the creators or original owners of assets (e.g.. code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
13. New assets
Question: Are assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
14. Crowdsoureing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based the requirements of your country institution) were obtained?
Answer: [NA]
16. Declaration of LLM usage
Question: Does the paper describe the usage of LLMs if it is important, original, or non-standard component of the core methods this research? Note that the LLM is used only for writing, editing, formatting purposes and does not impact the core methodology, scientific rigorousness, or originality the research, declaration is not required.
Answer: [NA]
16

---

A Experiments
A.1 Implementation details
DiT architecture The DiT architecture [32 uses 24 blocks of self-attention layer followed by cross attention layer with the text embedding 36], with a 2048 hidden dimension, 16 attention heads, and utilize 3D positional embedding [5]. Embedded image [33 size is 32 x 32 4 and input the DiT trough a patchify layer with patch size of x 2 x 4. The total number of parameters is 1.7B. Since ARTM gets as imput both X, and X which results in longer sequence length, for fair comparison across TM variations and baselines, for all other models we pad the input sequence to double its length.
LLM architecture The LLM architecture 29 is similar to the DiT with the following differences: (i) time injection is removed, (ii) cross attention layer is removed and text embedding is input as prefix (iii) it uses a simple ID instead of 3D positional embedding. To compensate for reduction number of parameters, we increase the number of self-attention layers to 34, reaching 1.7B total number of parameters (comparable to the DīT). FHTM-DiT gets Xo as input but does not take a loss on it, while for FHTM-LLM we remove the Xo all together and instead use a single boi (begin of image) token to save sequence length.
Flow head architecture Following 25 we use an MLP with 6 layers and a hidden dimension of 1024. convert from the backbone hidden dimension (2048) to the MLP hidden dimension (1024) we use simple linear layer. Finally, we replace the time input with AdaLN 32 time injection.
Optimization The models are trained for 500K iterations, with a 2048 total batch size, * e- constant learning rate and 2K iterations warmup.
Classifier free guidance To support classifier free guidance (CFG), during training, with probability of 0.15, we drop the text prompt and replace it with empty prompt. Following (25), during sampling, we apply CFG to the velocity of the flow head (g²) with a guidance scale of 6.5.
A.2 Main results: Text-to-image generation
Additional Kernels and Baselines Similar to the extension of the AR kernel ARTM, We extend the MAR- kernel to 2 and 3 transition steps, resulting with the MARTM-2 and MARTM-3 kernels. Furthermore, we investigate the performance the Restart sampling algorithm (52) on the FM kernel, were noise is added during the sampling process. We follow the authors' suggestion and perform restart from = 0.6 to t = 0.4, restarts from 0.8 to t = 0.6, and an additional restarts from = 1 to 0.8. The sampling is performed on base of 1000 steps, resulting with total of 2400 NFE. As an additional baseline, we sample the FM kernel with 2400 NFE. Results can be found in Tables 5
Table 5: Evaluation of MARTM and the Restart sampling algorithm baselines on PartiPrompts.
Attention Kemel
Arch
NFE
Cl.PSoore
PickSenre
ImageReward
UnifedReward 1 Aesthetic DeQASSore
256
26.0
2.36
MARTM-2
26.7
2.12
MARTM
D:T
256
26.4
21.9
0.25
4.48
5.11
2.49
Daseline
FM
DIT
256
26.0
21.0
0.23
1.78
5.29
2.55
FM
DJT
2400
26.0
21.1
0.24
4.81
5.29
2.55
FM- MM-Res
D:T
2400
26.1
21.1
0.34
4.83
5.31
2,53
Flow head NFE We ablate the number of NFE required by the flow head (g¹) to reach best performance for each model. As shown in Figure g we observe the models reach saturation with relatively low NFE, and decide to report results on Tables and with 64 NFE for the flow head.
TMM steps vs Flow head NFE for DTM We test the performance of the DTM variant as function of TM steps and Flow head NFE, As shown in Table 10 our DTM model achieve reach saturation about
17

---

Table 6: Evaluation of MARTM and the Restart sampling algorithm baselines on GenEval.
Attention
Single-object
Two-objects
Counting
Position
MAR-Fluid
DXT
256
0.44
0.90
0.33
0.37
0.76
0.12
0.28
MARTM-2
INT
256
0.51
0.34
0.50
0.35
0.77
0:21
0.32
Baseline
Full
MARTM-3
DST
206
0.52
0.91
0.58
0.41
0.77
0.14
0.38
FM
DNT
256
0.47
0.81
0.52
0.27
0.71
0.12
0.34
FM- Restar
DST
2:00
0.49
0.89
0.59
0.29
0.73
0.13
0.38
Table 7: Evaluation of MARTM and the Restart sampling algorithm baselines MS-COCO.
Attention Kemel
Arch
NFE
CLIPScore
PickScore
ImageReward
UnifeedRewand
Aesthetic
DeQAScure
MAR-Fulid
DIT
256
25.5
21.3
-0.11
3.91
4.86
2.38
MARTM-
DIT
256
25.9
21.0
0.17
4.93
5.33
2.41
Baseline
Full
MARTM-3
DIT
256
25.7
21.9
0.01
1.7
5.21
2.15
FM
DIT
256
25.8
21.1
0.09
5.00
5.45
2.47
FM
DIT
2100
25.8
5.00
5.45
2.17
16 TM steps and Flow head steps, according to CLIPScore and PickScore. Generation time for single image on single H100 GPU is provided in Table [1]
Table 8: Evaluation TM versus baselines on MS- us-COCO. Inference is done with activation caching. NFE* counts only backbone model evaluation (f LLM and DiT have comparable number of parameters.
Attention
Kernel
Arch
NFE
C1.IPScoree
PickScare
ImageRewand
UniftedReward
Aeshetic 1 DeQAScore
MAR-disencte
DXT
256
26
20.6
0.01
4.14
5.27
2.11
256
Buseline
Pull
MAR-Fulid
D:T
256
25.5
20.5
-0.11
3.94
4.56
2.38
FM
D&T
256
25.8
21.1
0.00
5.00
5.45
2.47
TM
DTM
DET
32
26.2
21.2
0.22
5.58
5.55
2.58
Baseline
AR AR-discrete
DET
256
26.7
20.3
-0.06
3.83
4.93
2.31
ARTM
1XT
256
20.8
4.70
2.41
Causal
FITM-22 2
D:T
2 256
25.9
20.8
0.07
4.78
5.27
2.45
ARTM 37
DXT
256
26.1
20.9
0.11
4.99
5.35
2.46
256
21.0
5:23
2.41
FHTM-3
256
18

---

Table Evaluation of TM versus baselines on T2l-CompBench: same settings as Table 8
Arch
NFE
Shupe
2D-Sptial
3D-Spuril
Numerccy
Non-Sputiial
Complex
MAR-diserete
DIT
256
0.6666
0.4535
0.5316
0,1474
0,2693
0.4388
0.3090
0.3196
Baseline
MAR
DIT
256
0.7378
0.5174
0.6888
0.1638
0,3002
0.4962
0.3082
0.3392
Full
MAR-Flald
DIT
256
0,997
0.4768
D.6499
0.1454
0,2038
04681
0.3057
0.3289
FM
DiT
206
0.6855
0.4511
0.5615
0,1372
0,2706
0.4526
0.3026
0.3138
DIT
32
0.7316
0.4865
0.1839
0.3075
Baseline
AR-discrete!
DIT
256
0.6068
0.4757
0.5958
0.2535
0.423
0.3098
0.3097
AR
DIT
206
0,5062
0,3669
0.5061
0.1041
0.2441
0,4200
0.2989
0.2963
Consal
0.3074
FHTM 22
DiT
2 256
0.6318
0,4318
0.5730
0,1403
0.2818
0-4830
0.3058
0.3229
ARTM
DIT
3 266
0.6355
0.4738
0.5842
0.1459
0,2832
0.4855
0.3062
0.3227
FHTM-3'
DiT
0.6604
0.3066
FHTM 3
L.M
256
0.6166
0.4618
0.5945
0.1688
0,308
0.5010
0.3079
0,3310
A.3
Sampling efficiency
Table 10:
Performance of FM (c-d) and DTM (a-b) for different combinations of Head NFE and TM steps, computed on subset of the PartiPrompts dataset (1024 out of 1632). Color intensity increases with higher performance.
(a) DTM CLIPScore
(b) DTM PickScore
TM
steps
TM
steps
1
2
4
8
16
32
64
128
2
4
8
16
32
64
128
15.8
17.0
20.4
22.8
23.2
23.2
23.0
22.8
17.6
17.8
18.6
19.4
19.6
19.7
19.6
19.6
16.1
18.6
24.2
26.2
26.4
26.4
26.2
26.2
2
17.7
18.3
19.7
20.6
20.9
21.0
21.0
21.0
Head NFE
17.9
21.1
25.4
26.7
26.8
26.7
26.5
26.5
Head NFE
18.1
18.8
20.0
20.8
21.1
21.1
21.1
21.1
18.8
21.2
25.5
26.6
26.8
26.6
26.5
26.5
18.3
18.8
20.0
20.8
21.1
21.1
21.1
21.2
18.9
21.3
25.5
26.7
26.8
26.6
26.5
26.4
18.3
18.8
20.0
20.9
21.1
21.1
21.1
21.1
32
19.0
21.2
25.5
26.7
26.7
26.7
26.6
26.5
32
18.3
18.8
20.0
20.9
21.1
21.1
21.1
21.1
64
19.0
21.3
25.4
26.7
26.8
26.6
26.4
26.5
64
18.3
18.8
20.0
20.9
21.1
21.1
21.1
21.1
128
18.9
21.3
25.4
26.7
26.9
26.7
26.5
26.4
128
18.3
18.8
20.0
20.8
21.1
21.1
21.1
21.1
(c) FM CLIPScore
(d) FM PickScore
Euler steps
Euler
steps
4
128
8
16
32
128
0
15.8
16.6
19.7
23.8
25.6
25.9
25.9
0
17.9
18.7
20.0
20.8
21.0
21.0
21.0
27
21.0
2055
25
CLIPScorn
BclScrre
20.0
13
14.5
ARTM 3-steps
19.0
ARTM 3-steps (DIT)
FHTM 3-steps (DI))
FHTM 3 stops (DIT)
FHTM
185
FHTM steps (LLM)
1.
22
64
123
10
32
126
NFE
Figure 8: Comparison of flow head NFE vs. CLIPScore (left), and PickScore (right) computed on the PartiPrompts dataset.
19

---

Table 11: DTM inference time (in seconds) for different combinations of Head NFE and TM steps on a single H100 GPU. Color intensity increases with runtime. Note that O head steps refers to FMM. TM steps (84 ms/step)
2
4
8
16
32
64
128
Head NFE (3.5 ms/step)
0
0.1
0.2
0.3
0.7
1.3
2.7
5.4
10.8
0.1
0.2
0.4
0.7
1.4
2.8
5.6
11.2
2
0.1
0.2
0.4
0.7
1.5
2.9
5.8
11.6
0.1
0.2
0.4
0.8
1.6
3.1
6.3
12.5
8
0.1
0.2
0.4
0.9
1.8
3.6
7.2
14.3
16
0.1
0.3
0.6
1.1
2.2
4.5
9.0
17.9
32
0.2
0.4
0.8
1.6
3.1
6.3
12.5
25.1
64
0.3
0.6
1.2
2.5
4.9
9.9
19.7
39.4
128
0.5
1.1
2.1
4.3
8.5
17.0
34.0
68.1
A.4 Dependent vs. independent linear process
Further analysis of the generated images reveals that the AR kernels are unable to learn the linear process, resulting in low quality image generation. We hypothesize that the AR kemnlls exploit the linear relationship between X, and X,+1 during training, which leads the model to learn a degenerate function and causes it fail in inference.
MS-COCO
PartiPrompts
24
24-
22-
20
CUPPOO
20
1.0
30
ART Dependent ARTTMinndeenndee Indegendent
16
ARTM:
ARTM: dependent
12
FITT Depeedeent
14
FHTM gpenndent
FHTM:in degendennt
Indepenent
104
12
23500
4c0n0
80000
2000
40000
c0nco
and5a
Trainling iteration
Training iteration
205
20
20.0
Picksrre
1.
185
ARTML Dapandent
18.0
ARTM. ent
Indeppeent
ARTMin dependent
Deppendet
FITT ependent
FITW:indeeeeddet
FHTM indepednenn
20000
42000
60000
$2000
22000
40030 60000
80030
Training Iberation
Training iteration
vs. Independent linear process 15 on the AR kernels: ARTM-3 and FHTMM33. The models are evaluated on the MS-COCO (left) PartiPrompts (right) with CLIPScore and PickScore every 10K training iterations across 100K iterations. Observe that on the AR kernels trained with the independent linear process are far superior the ones trained with the dependent linear process.
Figure 9: Dependent linear process 10
20

---

A.5
DTM Kernel expressiveness
26.5
26.0
CLiPScore
23.5
25.0
22
20
Fow Head 2x2 Head 2x8
Flow Head 2x2x4
Flow
19
1.0
12
64
128
10
22
d4
128
TM steps
Head NFE
21.2
21.0
20.5
20.0
20.0
20.8
20.4
20.0
nlw Head 2x2x4
floow llaed 2x2x8 Flow Head 2 2
Flow Head 2xBad
19.
128
19.8
16
32
128
steps
NFE
Impact of flow head patch size: x 4 vs. 2 x 8 x 4. on the DTM performance, evaluated across varying numbers of TM steps (Left, with 32 Head NFE) and variying number of Head NFE (Right, with 32 TM steps). The metrics are CLIPScore (Top) and PickScore (Bottom) computed on the PartiPrompts dataset. On low number TM steps, the larger flow head patch size shows advantage in both metrics. On high number of TMM steps, both patch sizes yield comparable results. This aligns with Theorem which predicts that for infinitesimal steps size, the entries of Y E Rd defined in equation 11 become independent.
Figure 10:
A.6 Scheduler ablation for independent linear process
We have experimented with two transition scheduler options: uniform (as described in T5 and "exponential", i.e., & {0, 0.5, 0.75, 1}. The results for ARTM and FHTM are reported in Table I2 and show almost the same performance with slight benefit towards exponential in DiT architecture and these are used in our main implementations.
Table 12: Comparison of uniform and exponential transition steps.
MS-COCO
PartiPrompts
Kernel
Arch
TM Steps
Scheduler
CLIPScore f
PickScore t CLIPScore t
PickScore t
Uniform
26.0
20.8
26.8
20.8
ARTM DiT
3
Exponential
26.1
20.9
27.0
20.9
Uniform
25.9
21.0
26.9
20.9
FHTM DIT
3
Exponential
26.1
21.0
27.0
20.9
Uniform
26.1
21.0
27.0
21.0
FHTM LLM
3
Exponential
26.1
21.1
27.0
21.0
21

---

A.7 Additional generated images comparison
FM
MAR
FHTM
DTM
"a coffee mug floating in the sky"
"The Alamo with bright white clouds above
"a tornado possing over corn field"
AI
"A wearing formal clothes, wearing tophat and holding The holding garbage bag.
Oil painting in the style of Rembrands."
"a harp with carved eagle fogure the top"
Figure 11: Additional generated samples of FM, MAR, FHTM, and DTM with models that trained for 1M iterations.
22

---

FM
MAR
FHTM
DTM
"A close-up photo of wombar wearing red backpack and raising both arms in the air. Mount Rushmore is in the background."
"a blue airplane taxiing on runway with the sun behind it"
"The Statue of Liberty with the face owl"
"A photograph of bird made of wheat bread and an egg."
"a living room with large Egyptian statue in the corner"
Figure 12: Additional generated samples FM, MAR, FHTM, and DTM with models that are trained for 1M iterations.
23

---

FM
MAR
FHTM
DTM
"a blue wooden pyramid on top of red plastic box"
bowl of soup that looks like monster knitted of woo" "
OO
"Portrait of a gecko wearing a train conductor's 's hat and holding a flag that has yin-yang symbol on it. Child's crayon drawing."
"a dolphin in an astronau suit"
"a moose standing over fax"
Figure 13: Additional generated samples FM, MAR, FHTM, and DTM with models that are trained for 1M iterations.
24

---

FM
MAR
FHTM
DTM
"a portrait of statue of a pharaoh wearing steampunk glasses, white I-shirt and leather jacket. dslr photograph. "M
"panda mad scientist"
"a futuristic city synthwave style"
Figure 14: Additional generated samples of FM, MAR, FHTM, and DTM with models that are trained for 1M iterations.
25

---

AR
ARTM-2
ARTM-3
"a half-peeled banana"
"beer"
"a blue wall with large largeframed watercolor painting of mountain"
S T START START
"the word 'START"
Figure 15: Samples comparison of AR (left) vs. ARTM-2 (middle) vs. ARTM-3 (right) on models trained for 500K iteration with the DIT architecture.
26

---

A.8 Generation process visualization
"a comic about an owl family in the forest"
"A photo of maple leaf made of water."
tornado made of sharks crashing into skyscraper. painting the style of watercolor"
"A single beam of light enter the room from the ceiling. The beam of light is illuminating an easel. On the easel there is a Rembrandt painting of a raccoon"
STARSSTARSTTRTTS STARSSTRISTARRS STARTSTART
SEARESTAATTS START STARTTSTARY STARTTSTART STARTTSTART STAKTTSTART STARTSSTAKT
SCAAS START CANK STAISS
"the word 'TAART' written in chalk on a sidewalk"
Figure 16: Generation process of FM (first row), DTM (second row), and FHTM (third row) with models that are trained for IM iterations. FM and DTM are visualized using a denoising estimation. FHTM-3 is visualized with 4 intermediates per transition step.
27

---

"a photograph of an ostrich wearing fedora and singing soulfully into a microphone"
"An oil painting of two rabbits in the style of American Gothic, wearing the same clothes in the original."
dt
d
dt
"a cloud in the shape teacup"
"A map of the United States with pin on San Francisco" "
"A giant cobra snake made from sushi"
Figure 17: Generation process of FM (first row), DTM (second row), and FHTM (third row) with models that are trained for IM iterations. FM and DTM are visualized using a denoising estimation. FHTM-3 is visualized with 4 intermediates per transition step.
28

---

"a stained glass window of a panda eating hamboo"
"a cross-section view of walnut"
"A photo of lotus flower made of water."
"A map of the United States made out sushi. It table next a glass of wine."
"a green pepper cut in half on a plate"
Figure 18: Generation process of FM (first row), DTM (second row), and FHTM (third row) with models that are trained for IM iterations. FM and DTM are visualized using a denoising estimation. FHTM-3 is visualized with 4 intermediates per transition step.
29

---

SP
"a cloud in the shape of a elephant"
"A photo of teddy bear made of water."
"a robot painted graffiti brick wall. sidewalk is in from of the wall, and grass is growing of cracks in the concrete.
"a capybora"
"A television made of water that displays on image of a cityscape at night."
Figure 19: Generation process FM (first row), DTM (second row), and FHTM (third row) with models that are trained for IM iterations. FM and DTM are visualized using a denoising estimation. FHTM-3 is visualized with 4 intermediates per transition step.
30

---

A.9
Classifier free guidance sensitivity
21.25
11.00
20.75
CLIPScare
FHTM 3-step (LUM)
DTM
20.50
FM (DiT)
20.29
AR (DIT)
24
AR-d iscrette (DIT)
20.00
MAR (DTT
MAR-nuid (DIT)
19.75
NAR-nudd-discete (DI)
22
10
12
24
Guidance scale
Guidance scale
Figure 20: CLIPScore CFG guidance scale (left) and PickScore vs. CFG guidance scale (right) of DTM and FHTM variants, and the baselines: FM, AR, AR-Discrete, MAR, MAR-Fluid, MAR-Fluid Discrete the PartiPrompts dataset.
31

---

S
7
10
12
13
wearing formal clotkes, wearing a tophat holding The holding garbuge bag. Oil painting in the style of Hokarsii""
OTOP
STOP
STOP
STOP
STOP
STOP
STOP
STOP
STOP
Sy
SWP
STOP
S OP
SP
STOP
STOP
STOP
STOP
STOP
STOP
"a stop sign knocked over on sidewalk
beack with cruise ship passing by"
"a spaceship hovering over The Alamo"
8
8
8
88
8
"five balls table"
Figure 21: Classifier free guidance sensitivity for (first row), DTM (second row), and FHTM (third row) with models that are trained for 500k iterations.
32

---

B Training and sampling algorithms
Algorithms I [ndd deseribe and training and sampling (resp.) of transition matching for general
supervision process, kernel parametrization, and kernel modeling. In this section, we provide training and sampling algorithms tailored the specific desgin choices of our three variants: (i) DTM
described in Figure 22 (ii) ARTM is described in Figure 23] and (iii) FHTM is described in Figure 24
Additionally, we provide Python code of training step for each variant: (i) DTM Figure 25
ARTM in Figure 26 and (iii) FHTM in Figure? 27
33

---

Algorithm 3 DTM Training
Require:
D Data Number of TM steps
Require: T
I: while not converged do
Sample XT PT
3: Sample ~ -(['-11)
4: Sample Xo ~ (N(0,I))
}
5:
X, (1 {) Xo + XT
Sample (X,, (-
6:
Y XT x
h4 + (X,,
8:
parallel for = 1, 12 do
Sample Y N (0, Ia/n)
10:
Sample [d[1 1])
11:
Y: + (1 -8) Yo + s
((0) D(X,)&)((1xx.)))
(0) (10 h() (Y* 1|
12:
13:
end for
14:
1 ((0) v E, (0)
15:
0- 0->VV Optimization step
16: end while
17: return /
Algorithm 4 DTM Sampling
Require:
D Trained model
Require: T
Number of TM steps
1: Sample Xo ~ ~O Ta)
2: for / 0 to T do
hu + (X),t)
4: parallel for i = 1,.... do
Sample Yjj N (0, La/m)
Sample Y (11,)
Y ode solve (Yo k())
7: end for
8: X + X1 + #
Sample X (-
9: end for
10: return Xy
Figure 22: n is the effective sequence length after patchify layer. The parallel operations run simultaneously across the "sequence length" dimension of the tensor; ode_solve is any generic ODE solver for solving equation g
34

---

Algorithm 5 ARTM Training
Require: Pr
Data
Require: T
D Number of TM steps
I: while not converged do
2: Sample XT Pr
3: Sample ~ 11)
4: Sample Xo.t NO0, Ta)
5: X, (1- 4) Xa. {X7
Sample (), ~ (),x11(111))
6: Sample X0,1+1 ~ NN(0, Ia)
7:
XL+1 (1 -11-) X0.11 4 Xr
8:
parallel for i = 1,... do
9:
h141 - 53 (Xt, X
10:
Sample Y0 ~ N(0. 14/n)
11:
Sample U[10, 1])
[()) D(Y,1fic((1x,))
12:
Y; 4 (1 s) Yo +
13:
(0) + |192, (Yg, h(+1) - (X 1/)
14:
end for
15:
((0) 1/2 E; (0)
16:
0- 0-0-7Vol2 Optimization step
17:
end while
18: return /
Algorithm 6 ARTM Sampling
Require:
Trained model
Require: T
b Number of TM steps
1: Sample Xg NO, Ia)
2: for f = 0 to T do
for I..... n do
4:
h1 + ff (Xe, X
5: Sample Yú ((,1)))))
Sample Xe+1 ~ (1
6: Xi+1 ode solve (Yo (114))))
7: end for
8: end for
9:
return XT
Figure 23: n the effective sequence length after patchify layer. The parallel for operations run simultaneously across the "sequence length" dimension of the tensor; ode_solve is any generic ODE solver for solving equation g
35

---

Algorithm 7 FHTM Training
Require: pr
DDaa b Number of TM steps
Require: T
I: while not converged do
2: Sample XT ~
3: parallel for = 0,... do
Sample Xo, (O, Id) X; (1 +) X XXT
Sample (X, ( Xx)
6: end for
7: parallel for / = 0,... T 1,i
m do
h1+1 -- - (Xo.... X,, Xx+1)
9:
Sample Y0 ((,, Id/n)
10:
Sample ~ ~(( 1))
[()) < D(X,p(1((1xx))
11:
Y: +- (1 - s) Yo + +8X1+1 1+1
12:
()) + (V M(+1) -
13:
end for
14:
((0) - l .f Σi, (0)
15:
0 o Optimization step
16: end while
17: return 0
Algorithm 8 FHTM Sampling
Require:
Trained model
Require: T
b Number of TM steps
1: Sample Xo NN Ta)
2: for f = 0 to T do
3:
for i ..... do
4:
h141 + f (Xo,... X1, <i
1+1
5: Sample Y² ~ NO0,
Sample N141 ~ (1
6:
Xi+1 f- ode_solve (Yi.g (1 (s1b42))
7: end for
8: end for
9: return X²
Figure 24: is the effective sequence length after patchify layer. The parallel for operations run simultancously across the "sequence length" dimension of the tensor; ode_ solle is any generic ODE solver for solving equation g
36

---

import torch from torch import nn, Tensor from einops import rearrange
def dtm_train_ (rarpinainste(((
backbone: mn. Module, Denoted "f"ltheta" head:nn. Module,
Denoted "g"ltheta"
X_T:Tensor,
Image from training set "_T"p_T"
T:int
Number of IM steps Patch size
patch_size:int int
Tensor:
# Convert amage to sequence using patchify
X_T rearrange(
X_T,
"b (h dh) (u dw) > b (h w) (dh du c)", dh-patch_size,
dw-patch_size,
18
bsz, seq_len X_T shape[[]
# Sample time step "t-U[T-]]' torch. randint т, (baz,))
22
23
24
# Sample pasr (X_ (x (IT)) x_0 torch. randn Like(X_I)
26
x_t (1-t/T), viex(-1,1,1) X_0 (t/T). viee((1,11) X_T
27
X_T X_0
28
# Backbone forward h_t backbone (X_t, t)
31
32
# Reshape sequence for head
33
b_t t h_t. (Lv.:vieebbszeeqe.ll -1)
s.viee((bs==ee,_een. -1)
i.repat.interllavcssq_....
# Flow matching loss with the head as velocity and as target 37
Y_0 = torch. randn randn_like(V like
torch. rarchh.rn(((s22=eee_e111
40
Y_s (1-s) viix((1,1) Y_O s view(-11,1
41
42
# Head forward
head(h_t, Y_8, s)
loss torch. functional mee_ luss(u, Y Y_0)
45
46
return loss
Figure 25: Python code for DTM training
37

---

import torch from torch import nn, Tensor
def artm_train_sted
backbone: nn. Module, Denoted as "f"ltheta"
neaduen.Module,
Denoted as "g"ltheta"
X_T:Tensor,
Image from training set 'x_ ppp"" Number of steps Patch size
T:int
patch_size:int
-> Tensor:
# Convert smage to sequence using patchify
X_T rearrange(
X_T,
"b (h dh) (u du) -> b (h w) (dh du c)", dh-patch_size,
dx-patch_size,
18
bsz, seq_len X_T. shTpehape
# Sample time step "t-U[T-]] torch. randint (0, т, (baz,))
22
23
# Sample pair
24
x_o_t torch. Landn_like(X___ (X_T)
X_t (1-t/T), viex(-1,1,1) X_0_t + vi1),,i)ww-11,11)) X_T
26
x_0_tp1 torch randn_ randn_like(X__]
(1-(t+1)/T) vieu(-1,1,1) x_O_tp1 + ((t+1)/T).veer vier(-1,1,1) X_T
28
# Backbone forward output backbone (torch, ([X_t, Y], dim=1), t)
31
h_tp1 output[:, seq_len-1--1]
32
33
# Reshape sequence for head h_tp1 h_tpl. viez (esz*seq_len, -1)
(bsie t..epeat areppat_inerreeeeeee...11
# Flow matching loss with the head as velocity and as target
Y_0 borch.randı lYke(Y)
40
(orzh.rand
41
Y_s (1-s) (-1,( Y_0 s viex((1111)
42
# Head forward head(h_tp) (h_tp1, t, Y_s, loss torch. mn functional mse lu, Y Y_0)
45
46
return loss
Figure 26: Python code for ARTM training
38

---

import torch from torch amport nn, Tensor
def fhtm_ aatm_train_stel
backbone: nn. Module, Denoted as "f"ltheta"
nead:en.Module,
Denoted as "g"ltheta"
X_T:Tensor,
Image from training set 'x_ XpT'" Number of steps Patch size
T:int
patch_size:int
-> Tensor:
# Convert smage to sequence using patchify X_T rearrange(
X_T,
"b c (h dh) (u du) -> b (h u) (dh du c)", dh-patch_size,
dx-patch_size,
18
bsz, len, d = X_T shape
# Sample pair tX_ 1) g- {t, (. (X__T)
22
boi torch. zeros (baz, 1, # begin of image token
23
X_FH [boi]
24
for t in range(1,T+)) I+1):
x_0_t torch randn like (X_T)
26
X_FH append
27
(1-t/T) I_0_t + t/T X_T
28
X_FH torch. cat cat (H,F, dim=1)
x_t X_FH[; :-1] I_FH[: 1:]
31
32
33
# forward for teacher forcing h_tp1 - backbone(X_t)
Reshape sequence for head
h tp1 h_tp1. hepl.vvewwbbarr (bazrbeeeqqleee -1)
Y Y. (baz
-1)
40
Flow matching with the head as velocity and as target Y_0 torch. randn_liked
41
42
torch. (baz
Y_s (1-s) v-ew(-1,1) Y_0 + s. viex(11),)))
45
Head forward u head (h_tp,, Y_3, 8)
46
loss torch mn functional mae_ sieeloss(( (u, Y Y_0)
return
Figure 27: Python code for FHTM training
39

---

C Convergence of DTM to flow matching
Here we want to prove the following fact: Assume we have a sequence of Markov chains {Xo, Xp: X....... X1), with an initial state Xy z, where T and T cc. For conve- nience note that we index the Markov states with fractions th, & E [T]. and we denote the RV
Xeth Xt
Y
(19)
Assume the Markov chains satisfy:
1. The function f((x)) = X x is Lipshcitz continuous. By Lipschitz we mean that llf((v) f((=) CL (|s t + -ll).
2. For l [k:] the quadratic variation satisfies, IE [ILYOal | Xx 1/] s clx).
Let Ah)h) E N be an integer-valued function of l such that + ao and 2 kh - 0 as - 0. We will prove that the random variable
XAA - Xo
(20)
kh
converges in mean to fo(x). That is, we want to show
Theorem 2. Considering a sequence of Markov processes (Xo, Xh. -.h...., X1} satisfying the assumptions above, then
Xkh Xx
2
lim E h00
So(Xo)
No 0.
(21)
kh
Proof. First,
X Xo kh
fo(Xo)
2
Xo
E
k-1
Yen - fo(Xo)
2
Xo = x
(22)
and if we open the squared norm we get three terms:
[(116((X))1? Xo = 2) = =11f(())11
(23)
E
Yon fo(Xo)
Xo
fo(x)
E
Yeh
Xo
(24)
k t=
K
k- k-1
h
Σ
Yen Ynn
No
|| 21
1No = =x]
t= m=0
t=
k-1
Yen Ymh
Xo
(=1 m=
We will later show that IF [YA, Xo xx] = =o(r) + (kh) and for [Ye You Xo = =1.ooo + O(kh). Plugging these we get that equation 22 equals
* m we have
k
[(f(())) 2 [l))(())
[[f(())] + O(kh + k- + 0,
(25)
40

---

as h + where we used assumption 2 above to bound E || 12 |xo = 2)
S d(x). Now to
conclude we show
|E [Ye x| = = |E [E [Yen (xo x] - fo(x))l
(26)
||F [Jen (Xen)) X0 = fo(2))
(27)
|IE [Jen (Xen) - (o
x|]
(28)
E fen (Xen) -o(()) [Xo =
(29)
S Cz
[th + +leee - Xoll [x = x]
(30)
S cr,
th + Σ
X (m+11) - Xmall
Xo
(31)
m=
O(kh) + ca.
Σ
E
|Ymalll
Xo x
(32)
< O(kh) + vczlk/e(xx)
(33)
O(hk).
(34)
Now for < l we have
[E [Yen- Ymn Xo x| - fo(r) - E [Y Xo x]
(35)
JE [Y (Yeh- fo((oo) Xo z]|
(36)
JE [mmn E(en= - [o(X))lXen) Xx= = x||
(37)
JE [Ymn- (Jen(Xen) fo(Xo)) | = x]
(38)
S E ms (fen(Xen) fo(Xo)) |oo x]
(39)
E [ (Xea) So(Xo)
| = x]
(40)
E
[Ymall (kh +
|o01> - X Xo
(41)
E
|Ymbll (:hh + h
_
())al)) |o =
(42)
1=0
< O(kh) + ELAE
Σ
|m ||gall Xo
(43)
S Okkh) + cah
L
| 11 + +1|2,n111
Xo = x
(44)
2
O(kh).
(45)
Therefore.
IE Yen Yin Xo x] - -f((x)- fo(z)]
(46)
< E [Yen Yun X, = x] ff((x) E [Y (X, =x x||
(47)
+ lfo(z) E [[mm Xo- x] - fo(z) fo(x))
(48)
< (kh),
(49)
where we used equation 34 and equation 45 and the proof done since kh ++ 0 as h + 0.
41

---

C.1 The DTM case
We note show that the DTM process satisfies the two assumptions above. We recall that the DTM process is defined by Y ~ (-(x)- where Y = X1 - Xo-
First we check the Lipchitz property.
fx( = Xr x]
(50)
=[1x- - Xo = x]]
(51)
= (z)
(52)
= 1 1 x
1 - (111(01(z)dzz
(53)
/ 11 x t [an (<1 ddr; dr
Pa (x) (21)
(54)
which is Lipschitz for t < 1 as long as Pin (2](²) > 0 for all æ, and is continuously differentiable in t and x, both hold for the Gaussian kernel Ptp (x|[2) = (atttt, (1 t).).
Let us check the second property. For this end we make the realistic assumption that our data is
bounded, i.e. [|X S r for some constant r > 0. Then, consider some RV x; - -X = Y, ~
py)x(-(XX,) Then by definition we have that X1+h = X2 + h(x) X²) and X, = (1 - t) + tXp. Therefore,
||
|X + h(x; x1)II h)X, + h(1 + t)x
(55)
(56)
(1-t) t)
(1 (t + h)) |Xell h 1 t (1 t)
(57)
1-t"" t
We apply this to t-tt + h th where f [k] and therefore
1 + kh
| S X(e- 13th || h
(58)
kh
1 + kh
< ||olll + kh
(59)
1 kh
3r
< |
+
c(x)
(60)
2
where we used kh S 2. Finally,
F
| Noo = a]
= F
\
2 |x,
1Xo 2x
(61)
=
| X61l |x.]
1.80 = x
(62)
X1 X, ix 2
E
1x,
1xu σ
(63)
(1 - 1)
X, (1 + t1x)
1/2
E
E
|xX
1Xo = σ
(64)
(1 (1-t) t)
S 2E
1
|x
(1 1)2
|Xo
(65)
t)z
(1- 1)2
where we used again X,- = (1 - x)x + 1.1. Lastly, applying this to = eh < kh S | and using
2
equation 60 we get
|Yell [oo = s (1- 1)2
2
0(x)² + (1 t1+
t)2 (1 f)2
1 c(x).
(66)
42


Reviews:
[{"review_id": "ZmRpXZro7r", "reviewer": "Reviewer_ag7i", "summary": "This paper introduces Transition Matching (TM), a general framework for generating discrete-time continuous-state sequences. TM unifies both flow-based methods and continuous autoregressive generation. The framework explores three novel variants: Difference Transition Matching, Autoregressive Transition Matching, and Full History Transition Matching. Experiments demonstrate that FHTM outperforms flow-based methods in the text-to-image task for continuous domains.", "questions": "Please refer to the weaknesses.", "limitations": "I’m unable to find the discussion about the limitation except line 157.", "rating": 4, "confidence": 1, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "**Strengths**\n\n1. ARTM and FHTM generate continuous causal AR with quality comparable to non-causal approaches.\n\n2. The proposed methods achieve state-of-the-art image quality and text adherence in text-to-image generation. They significantly improve text adherence compared to Flow matching and set a new state-of-the-art on this task.\n\n**Weaknesses**\n\nExperiments indicate that the Non-Formal Error (NFE) for ARTM/FHTM is high. Further analysis or experiments specifically focusing on the trade-off between quality and NFE for these causal models would be beneficial.", "was_revised": false, "final_justification": "", "rebuttal": "> Experiments indicate that the Non-Formal Error (NFE) for ARTM/FHTM is high. Further analysis or experiments specifically focusing on the trade-off between quality and NFE for these causal models would be beneficial.\n\nWe assume the reviewer means Number of Function Evaluations (NFE). \nDTM is in fact **considerably faster** than FM. We have conducted several experiments and added several relevant Tables below. \n\nMainly, Table 1 compares the wall-clock time of FM and DTM, and in particular DTM (16 backbone NFE and 4 head NFE) can **achieve superior performance** to FM (128 backbone NFE) with a **7x speedup**. \nIn more detail, Tables 2(a–b) present the dependence of DTM’s CLIPScore and PickScore on the number of function evaluations (NFE) in the flow head and backbone. Table 4 reports the corresponding inference times. Tables 3(a–b) present flow matching’s (FM) CLIPScore and PickScore as a function of backbone NFE.\n\n\n**Table 1.**: DTM vs FM sampling time.\n| Kernel | Time (sec) | CLIPScore | PickScore |\n| ------ | ---------- | --------- | --------- |\n| FM     | 10.8       | 26.0      | 21.0      |\n| DTM    | **1.6**    | **26.8**  | **21.1**  |\n\n\n\n**Table 2.a**: DTM CLIPScore,  Head NFE (Rows) vs. TM steps (Columns). \n|HeadNFE/TMsteps|1|2|4|8|16|32|64|128|\n|--------|----|----|----|----|----|----|----|----|\n|1|15.8|17.0|20.4|22.8|23.2|23.2|23.0|22.8|\n|2|16.1|18.6|24.2|26.2|26.4|26.4|26.2|26.2|\n|4|17.9|21.1|25.4|26.7|26.8|26.7|26.5|26.5|\n|8|18.8|21.2|25.5|26.6|26.8|26.6|26.5|26.5|\n|16|18.9|21.3|25.5|26.7|26.8|26.6|26.5|26.4|\n|32|19.0|21.2|25.5|26.7|26.7|26.7|26.6|26.5|\n|64|19.0|21.3|25.4|26.7|26.8|26.6|26.4|26.5|\n|128|18.9|21.3|25.4|26.7|26.9|26.7|26.5|26.4|\n\n\n\n\n\n**Table 2.b**: DTM PickScore,  Head NFE (Rows) vs. TM steps (Columns). \n|HeadNFE/TMsteps|1|2|4|8|16|32|64|128|\n|--------|----|----|----|----|----|----|----|----|\n|1|17.6|17.8|18.6|19.4|19.6|19.7|19.6|19.6|\n|2|17.7|18.3|19.7|20.6|20.9|21.0|21.0|21.0|\n|4|18.1|18.8|20.0|20.8|21.1|21.1|21.1|21.1|\n|8|18.3|18.8|20.0|20.8|21.1|21.1|21.1|21.2|\n|16|18.3|18.8|20.0|20.9|21.1|21.1|21.1|21.1|\n|32|18.3|18.8|20.0|20.9|21.1|21.1|21.1|21.1|\n|64|18.3|18.8|20.0|20.9|21.1|21.1|21.1|21.1|\n|128|18.3|18.8|20.0|20.8|21.1|21.1|21.1|21.1|\n\n\n\n**Table 3.a**: FM CLIPScore vs. Euler steps (Columns).\n|EulerSteps|1|2|4|8|16|32|64|128|\n|-----------|----|----|----|----|----|----|----|----|\n|0|15.8|16.6|19.7|23.8|25.6|25.9|25.9|26.0|\n\n\n\n**Table 3.b**: FM PickScore vs. Euler steps (Columns).\n|EulerSteps|1|2|4|8|16|32|64|128|\n|-----------|----|----|----|----|----|----|----|----|\n|0|17.9|18.0|18.7|20.0|20.8|21.0|21.0|21.0|\n\n\n\n\n**Table 4.**:DTM inference time (in seconds) for different combinations of Head NFE and TM steps on a single H100 GPU. Note that 0 head steps refers to FM. Head NFE (Rows) vs. TM steps (Columns). \nHead NFE / TM Steps | 1     | 2     | 4     | 8     | 16    | 32    | 64    | 128   |\n|------------|-------|-------|-------|-------|-------|-------|-------|--------|\n| 0          | 0.1   | 0.2   | 0.3   | 0.7   | 1.3   | 2.7   | 5.4   | 10.8   |\n| 1          | 0.1   | 0.2   | 0.4   | 0.7   | 1.4   | 2.8   | 5.6   | 11.2   |\n| 2          | 0.1   | 0.2   | 0.4   | 0.7   | 1.5   | 2.9   | 5.8   | 11.6   |\n| 4          | 0.1   | 0.2   | 0.4   | 0.8   | 1.6   | 3.1   | 6.3   | 12.5   |\n| 8          | 0.1   | 0.2   | 0.4   | 0.9   | 1.8   | 3.6   | 7.2   | 14.3   |\n| 16         | 0.1   | 0.3   | 0.6   | 1.1   | 2.2   | 4.5   | 9.0   | 17.9   |\n| 32         | 0.2   | 0.4   | 0.8   | 1.6   | 3.1   | 6.3   | 12.5  | 25.1   |\n| 64         | 0.3   | 0.6   | 1.2   | 2.5   | 4.9   | 9.9   | 19.7  | 39.4   |\n| 128        | 0.5   | 1.1   | 2.1   | 4.3   | 8.5   | 17.0  | 34.0  | 68.1   |\n\n> I’m unable to find the discussion about the limitation except line 157.\n\nMore limitations appear in the conclusions section in lines (286-287). We will modify/elaborate the limitations to include “The improved performance of ARTM/FHTM comes at the price of a higher sampling cost, i.e., NFE counts are proportional to the number of transition steps, see e.g., Table 1 in the paper.”"}, {"review_id": "72QtRbC26K", "reviewer": "Reviewer_QC47", "summary": "This paper presents Transition Matching (TM), a new class of generative models that merges the strengths of diffusion and flow-based approaches. TM broadens the generative design space by introducing non-deterministic probabilistic transitions and flexible, non-consecutive supervision. The authors develop three variants: Difference (DTM), Autoregressive (ARTM), and Full History (FHTM) Transition Matching. DTM enhances image quality and text consistency while speeding up sampling. As partially and fully causal models, ARTM and FHTM achieve generation quality on par with or exceeding non-causal methods and integrate easily with existing text generation technologies. In particular, FHTM is the first fully causal model in the continuous domain to outperform flow-based methods, excelling in text-to-image synthesis. The paper validates the advantages of the TM framework through extensive comparisons and outlines future work on improving efficiency and integrating FHTM into larger multimodal systems.", "questions": "1. Has this method been validated on large-scale generative models?", "limitations": "yes.", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "### **Strengths:**\n\n1. The paper presents a novel generative model called Transition Matching (TM) that combines the advantages of diffusion and flow models. The authors introduce non-deterministic probability transfer kernels and arbitrary non-continuous supervised processes, significantly expanding the design space. They propose three different TM variants - Difference Transition Matching (DTM), Autoregressive Transition Matching (ARTM), and Full History Transition Matching (FHTM) - each achieving performance optimization in specific aspects.\n\n2. The proposed TM framework offers a powerful tool for generating high-quality samples while maintaining computational efficiency. It addresses limitations of existing approaches and provides a new direction for future research in the field of generative modeling. The authors highlight the significance of their work in terms of its potential impact on various applications such as image generation, text-to-image synthesis, and multi-modal systems.\n\n### **Weaknesses:**\n\n1. As authors stated, the improved performance of DTM/ARTM/FHTM comes at the price of a higher sampling cost.\n\n2. The analysis of the model's performance is simplistic.", "was_revised": true, "final_justification": "Thank you for your response. The authors have addressed some of my concerns, and I appreciate the thorough exploratory experiments conducted in the paper. I have decided to raise my score to 5.", "rebuttal": "> As authors stated, the improved performance of DTM/ARTM/FHTM comes at the price of a higher sampling cost.\n\nDTM is in fact **considerably faster** than FM. We have conducted several experiments and added several relevant Tables in **Reviewer’s ag7i response**. \nMainly, Table 1 in Reviewer’s ag7i response compares the wall-clock time of FM and DTM, and in particular DTM (16 backbone NFE and 4 head NFE) can **achieve superior performance** to FM (128 backbone NFE) with a **7x speedup**. \nIn more detail, Tables 2(a–b) in Reviewer’s ag7i response present the dependence of DTM’s CLIPScore and PickScore on the number of function evaluations (NFE) in the flow head and backbone. Table 4 Reviewer’s ag7i response reports the corresponding inference times. Tables 3(a–b) in Reviewer’s ag7i response present flow matching’s (FM) CLIPScore and PickScore as a function of backbone NFE.\n\n\n> The analysis of the model's performance is simplistic.\n\nWe kindly ask the reviewer to be more specific. We have made an honest effort to collect all popular benchmarks and auto-metric that are open-source and open-license: The GenEval benchmark and additional 6 auto-metrics: CLIPScore, PickScore, ImageReward, UnifiedReward, Aesthetic, and DeQA Score, where the metrics are evaluated on two benchmarks: MS-COCO and PartiPrompts. \nThe GenEval benchmark is widely accepted metric for evaluating text-to-image and has been used by many acclaimed previous works: SD3 [1], Fluid [2], Transfusion [3], EMU3 [4], Janus [5], Flow-GRPO [6], and so does the metrics we report. For example, CLIPScore has been used by SD3 [1], Transfusion [3], EMU3 [4],  DALLE-3 [7], DPO [8], SDXL [9], Imagen [10], and PickScore by Flow-GRPO [6], DPO [8].\n\nWe also added during the rebuttal the benchmark suggested by Reviewer jNpW: T2I-CompBench. See Table 7  in **Reviewer’s jNpW response**.\n\nIf the reviewer feels we have missed other well known, open-source and open-license auto-evals we would be happy to incorporate them as-well.  \n\nLastly, we also added two fully discrete baselines: discrete-AR and discrete-MAR using the tokenizer of Chameleon [11]. The results are summarized below in Tables 5.(a-b) and 6, and are consistent with the submitted paper’s claims. To summarize all the evaluations we add Table 8 **in Reviewer’s jNpW response** showing the global ranking of each method (defined as the sum of its ranking across all benchmarks: GenEval, T2I-Compbench, PartiPrompts, and MS-COCO ).\n\nFor tables 5-6 below: Best score is denoted by **Bold** with ★ and second best is denoted by **Bold** only.  NFE\\* is the number of function evaluations in the backbone. † Indicates inference with activation caching.\n\n**Table 5.a**: Main Table. PartiPrompts\n||Attention|Kernel|Arch|NFE*|CLIPScore↑|PickScore↑|ImageReward↑|UnifiedReward↑|Aesthetic↑|DeQAScore↑|\n|---|---|---|---|---|---|---|---|---|---|---|\n|Baseline|Full|MAR-discrete|DiT|256|26.80|20.70|0.14|4.31|5.15|2.48|\n|Baseline|Full|MAR|DiT|256|**27.00★**|20.70|0.33|4.26|4.95|2.36|\n|Baseline|Full|MAR-Fluid|DiT|256|26.00|20.50|0.07|3.82|4.74|2.36|\n|Baseline|Full|FM|DiT|256|26.00|21.00|0.23|4.78|5.29|**2.55**|\n|Baseline|Full|FM|DiT|2400|26.00|**21.10**|0.24|4.81|5.29|**2.55**|\n|Baseline|Full|FM-Restart|DiT|2400|26.10|**21.10**|0.34|4.83|**5.31**|2.53|\n|TM|Full|DTM|DiT|32|26.80|**21.20★**|**0.53★**|**5.12★**|**5.42★**|**2.65★**|\n|Baseline|Causal|AR-discrete†|DiT|256|26.70|20.40|-0.01|3.74|4.81|2.38|\n|Baseline|Causal|AR†|DiT|256|24.90|20.10|-0.43|3.41|4.50|2.27|\n|TM|Causal|ARTM–2†|DiT|2×256|26.80|20.80|0.29|4.49|5.03|2.37|\n|TM|Causal|FHTM–2†|DiT|2×256|26.80|20.80|0.30|4.59|5.13|2.44|\n|TM|Causal|ARTM–3†|DiT|3×256|**27.00**|20.90|0.38|4.77|5.21|2.53|\n|TM|Causal|FHTM–3†|DiT|3×256|**27.00**|20.90|0.31|4.77|5.15|2.44|\n|TM|Causal|FHTM–3†|LLM|3×256|**27.00**|21.00|**0.43**|**5.02**|5.30|2.54|\n\n\n**Table 5.b**: Main Table. MS-COCO\n|Model|Attention|Kernel|Arch|NFE*|CLIPScore↑|PickScore↑|ImageReward↑|UnifiedReward↑|Aesthetic↑|DeQAScore↑|\n|---|---|---|---|---|---|---|---|---|---|---|\n|Baseline|Full|MAR-discrete|DiT|256|**26.57**|20.63|0.01|4.14|5.27|2.41|\n|Baseline|Full|MAR|DiT|256|26.12|20.66|0.17|4.62|5.06|2.34|\n|Baseline|Full|MAR-Fulid|DiT|256|25.46|20.45|-0.11|3.94|4.86|2.38|\n|Baseline|Full|FM|DiT|256|25.78|**21.11**|0.09|5.00|5.45|2.47|\n|Baseline|Full|FM|DiT|2400|25.78|**21.11**|0.09|5.00|5.45|2.47|\n|Baseline|Full|FM-Restart|DiT|2400|25.78|**21.11**|0.15|5.11|5.48|2.44|\n|TM|Full|DTM|DiT|32|26.16|**21.19★**|**0.22**|**5.38**|**5.55★**|**2.58★**|\n|Baseline|Causal|AR-discrete†|DiT|256|**26.69★**|20.31|-0.06|3.83|4.93|2.34|\n|Baseline|Causal|AR†|DiT|256|24.83|20.11|-0.48|3.60|4.76|2.34|\n|TM|Causal|ARTM-2†|DiT|2×256|25.90|20.75|0.07|4.70|5.19|2.41|\n|TM|Causal|FHTM-2†|DiT|2×256|25.91|20.79|0.07|4.78|5.27|2.44|\n|TM|Causal|ARTM-3†|DiT|3×256|26.07|20.92|0.11|4.99|5.35|2.46|\n|TM|Causal|FHTM-3†|DiT|3×256|26.14|20.98|0.15|5.23|5.38|2.41|\n|TM|Causal|FHTM-3†|LLM|3×256|26.14|21.08|**0.24★**|**5.51★**|**5.53**|**2.51**|\n\n\n**Table 6.**: Main Table. GenEval\n||Attention|Kernel|Arch|NFE*|Overall↑|Single-object↑|Two-objects↑|Counting↑|Colors↑|Position↑|ColorAttribute↑|\n|---|---|---|---|---|---|---|---|---|---|---|---|\n|Baseline|Full|MAR-discrete|DiT|256|0.44|0.86|0.43|0.37|0.66|0.13|0.29|\n|Baseline|Full|MAR|DiT|256|**0.52**|**0.98★**|0.56|**0.43**|0.73|0.11|**0.38**|\n|Baseline|Full|MAR-Fluid|DiT|256|0.44|0.90|0.33|0.37|0.76|0.12|0.28|\n|Baseline|Full|FM|DiT|256|0.47|0.91|0.52|0.27|0.71|0.12|0.34|\n|Baseline|Full|FM|DiT|2400|0.47|0.91|0.51|0.25|0.72|0.14|0.36|\n|Baseline|Full|FM-Restart|DiT|2400|0.49|0.89|**0.59★**|0.29|0.73|0.13|**0.38**|\n|TM|Full|DTM|DiT|32|**0.54★**|0.93|**0.58**|0.35|**0.79★**|**0.20★**|**0.46★**|\n|Baseline|Causal|AR-discrete†|DiT|256|0.41|**0.96**|0.40|0.33|0.60|0.07|0.19|\n|Baseline|Causal|AR†|DiT|256|0.34|0.86|0.26|0.15|0.63|0.06|0.15|\n|TM|Causal|ARTM-2†|DiT|2×256|0.49|0.95|0.51|0.39|**0.79★**|0.11|0.27|\n|TM|Causal|FHTM-2†|DiT|2×256|0.48|**0.96**|0.48|0.25|**0.78**|0.09|0.37|\n|TM|Causal|ARTM-3†|DiT|3×256|0.51|0.95|0.54|0.41|**0.79★**|0.16|0.28|\n|TM|Causal|FHTM-3†|DiT|3×256|**0.52**|**0.98★**|0.54|**0.44★**|0.74|0.16|0.34|\n|TM|Causal|FHTM-3†|LLM|3×256|0.49|0.94|0.55|0.37|0.69|**0.17**|0.29|\n\n\n> Has this method been validated on large-scale generative models?\n\nWe would like to bring to the reviewer’s attention that the results reported in the paper include 10 models of 1.7B parameters, trained from scratch on a large dataset of 350M image-caption pairs for 500k training iterations. We believe this is **among the largest, most comprehensive and fair large-scale evaluation openly available**. For comparison, SD3 [1], which we consider the largest available fair comparison of diffusion and flow models, made most of their evaluation efforts on models with 2B parameters or less, and trained only a single model with 8B parameters. \n\n[1] Esser, Patrick, et al. \"Scaling rectified flow transformers for high-resolution image synthesis.\" Forty-first international conference on machine learning. 2024.\n\n[2] Fan, Lijie, et al. \"Fluid: Scaling autoregressive text-to-image generative models with continuous tokens.\" arXiv preprint arXiv:2410.13863 (2024).\n\n[3] Zhou, Chunting, et al. \"Transfusion: Predict the next token and diffuse images with one multi-modal model.\" arXiv preprint arXiv:2408.11039 (2024).\n\n[4] Wang, Xinlong, et al. \"Emu3: Next-token prediction is all you need.\" arXiv preprint arXiv:2409.18869 (2024).\n\n[5] Chen, Xiaokang, et al. \"Janus-pro: Unified multimodal understanding and generation with data and model scaling.\" arXiv preprint arXiv:2501.17811 (2025).\n\n[6] Liu, Jie, et al. \"Flow-grpo: Training flow matching models via online rl.\" arXiv preprint arXiv:2505.05470 (2025).\n\n[7] Betker, James, et al. \"Improving image generation with better captions.\" Computer Science. openai. com/papers/dall-e-3. pdf 2.3 (2023):8.\n\n[8] Wallace, Bram, et al. \"Diffusion model alignment using direct preference optimization.\" Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2024.\n\n[9] Podell, Dustin, et al. \"Sdxl: Improving latent diffusion models for high-resolution image synthesis.\" arXiv preprint arXiv:2307.01952 (2023).\n\n[10] Saharia, Chitwan, et al. \"Photorealistic text-to-image diffusion models with deep language understanding.\" Advances in neural information processing systems 35 (2022): 36479-36494.\n\n[11]  Team, Chameleon. \"Chameleon: Mixed-modal early-fusion foundation models.\" arXiv preprint arXiv:2405.09818 (2024)."}, {"review_id": "nwVtRo0iqI", "reviewer": "Reviewer_jNpW", "summary": "The paper proposes a novel auto-regressive generative model based on \"Transition Matching\". Here, the idea is to form a Markov model that takes some easily sampled data to the target distribution gradually based on a fixed process called \"supervising process\". The idea is then to use various transition modeling strategies and train them to generate data at the test time. This results in a generic and rich set of generative models, for which diffusion models and flow matching could be considered as its special cases. Because of the generality of the model, one would expect a greater flexibility (e.g. could be made auto-regressive as a result of the modeling choices) and potentially improved generative capabilities compared to previous models whose design choices are saturated for the most part. Three variants of the proposed model was trained on massive datasets \"350M licensed Shutterstock\" and tested based on PartiPrompts and MS-COCO benchmarks, based on various metrics such as CLIPScore, PickScore, etc.", "questions": "Please see above.", "limitations": "N/A", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "The paper overall looks promising and clear. I have few concerns:\n\n- The problem of text-to-image generation is multi-facet problem and may previous work focused on improving specific aspects, e.g. compositionality, quality, bias, style transfer, etc. It is not clear which of these aspects is expected to be improved by the proposed method. With regard to compositionaliity, one would expect benchmarks such as T2I-CompBench and other related ones to be tested against. Furthermore, many of the proposed metrics lack alignment with human perception, making results less reliable. \n- The fact that DTM (Difference Transition Matching) gave better results compared to more complex ARTM and FHTM sounds puzzling and was not discussed in details in the paper. My question to the authors is why DTM outperforms such models. \n- As the authors mentioned in line 286 \"The improved performance of DTM/ARTM/FHTM comes at the price of a higher sampling cost\". But we know certain test-time scaling methods, such as noise optimization, that improve the image generation quality by large margins when given higher test-time compute budget (e.g. see ReNO). The authors should elaborate how their method compare to such schemes.", "was_revised": true, "final_justification": "All my concerns are addressed.", "rebuttal": "> With regard to compositionality, one would expect benchmarks such as T2I-CompBench and other related ones to be tested against. \n\nWe have followed the reviewer’s suggestion on the benchmark they recommended and evaluated our models and baseline on the T2I-CompBench. Results are provided below in Table 7 and the global ranking of each method (defined as the sum of its ranking across all benchmarks: GenEval, T2I-Compbench, PartiPrompts, and MS-COCO) is shown in Table 8 Note that the new benchmark does not change the fact that DTM is best among non-AR and FHTM is best among AR methods. \n\nFor tables 7-8 below: Best score is denoted by **Bold** with ★ and second best is denoted by **Bold** only.  NFE\\* is the number of function evaluations in the backbone. † Indicates inference with activation caching.\n\n**Table 7.** T2I-CompBench\n|Kernel|Color|Shape|Texture|2D-Spatial|3D-Spatial|Numeracy|Non-Spatial|Complex|\n|---|---|---|---|---|---|---|---|---|\n|MAR-discrete|0.6666|0.4535|0.5316|0.1474|0.2693|0.4538|**0.3090**|0.3096|\n|MAR|**0.7378★**|**0.5174★**|**0.6588**|0.1638|0.3002|0.4962|0.3082|**0.3392★**|\n|MAR-Fluid|0.6997|0.4768|0.6149|0.1454|0.2938|0.4681|0.3037|0.3289|\n|FM|0.6855|0.4511|0.5615|0.1372|0.2706|0.4526|0.3026|0.3138|\n|DTM|**0.7316**|**0.4865**|**0.6597★**|**0.1839★**|**0.3113★**|**0.5043★**|0.3075|**0.3382**|\n|AR-discrete|0.6068|0.4757|0.5958|0.1095|0.2535|0.4423|**0.3098★**|0.3097|\n|AR|0.5062|0.3669|0.5061|0.1041|0.2441|0.4210|0.2989|0.2983|\n|ARTM-2|0.6520|0.4430|0.5870|0.1475|0.2748|0.4800|0.3074|0.3267|\n|ARTM-3|0.6555|0.4738|0.5842|0.1459|0.2832|0.4855|0.3062|0.3227|\n|FHTM-2|0.6318|0.4318|0.5730|0.1403|0.2818|0.4830|0.3058|0.3229|\n|FHTM-3|0.6604|0.4640|0.5839|0.1394|0.2755|0.4810|0.3066|0.3223|\n|FHTM-3LLM|0.6166|0.4618|0.5945|**0.1688**|**0.3081**|**0.5010**|0.3079|0.3310|\n\n**Table 8.** Global ranking: For metric in GenEval, T2I-Compbench, PartiPrompts, and MS-COCO  we rank all models from 1 to 12 (where 1 is best model) and we sum the ranks from all metrics. Lower is better.\n|Kernel|Rank↓|\n|---|---|\n|MAR-discrete|200|\n|MAR|127|\n|MAR-Fluid|220|\n|FM|179|\n|DTM|**58★**|\n|AR-discrete|245|\n|AR|321|\n|ARTM-2|184|\n|FHTM-2|185|\n|ARTM-3|130|\n|FHTM-3|130|\n|FHTM-3 LLM|**99**|\n\n\n> The problem of text-to-image generation is multi-facet problem and many previous work focused on improving specific aspects, e.g. compositionality, quality, bias, style transfer, etc. It is not clear which of these aspects is expected to be improved by the proposed method ;  … Furthermore, many of the proposed metrics lack alignment with human perception, making results less reliable.\n\nWe have collected standard and widely used metrics found in similar papers including for example: \n\nGenEval eval in: SD3 [1], Fluid [2], Transfusion [3], EMU3 [4], Janus [5], Flow-GRPO [6].\n\nCLIPScore eval in: SD3 [1], Transfusion [3], EMU3 [4],  DALLE-3 [7], DPO [8], SDXL [9], Imagen [10]\n\nPIckScore eval in: Flow-GRPO [6], DPO [8]. \n\nWe now added T2I compBench. \nIf the reviewer feels we have missed other well accepted, open-source and open-license auto-evals we would be happy to incorporate them as-well.  \n\n\n> The fact that DTM (Difference Transition Matching) gave better results compared to more complex ARTM and FHTM sounds puzzling and was not discussed in detail in the paper. My question to the authors is why DTM outperforms such models.\n\nFirst, we would not characterize DTM as “simpler” or “less performant” than ARTM/FHTM. If anything, its bidirectional attention makes it more expressive and in our empirical  apples-to-apples comparison it was favorable. \n\n\n> As the authors mentioned in line 286 \"The improved performance of DTM/ARTM/FHTM comes at the price of a higher sampling cost\". But we know certain test-time scaling methods, such as noise optimization, that improve the image generation quality by large margins when given higher test-time compute budget (e.g. see ReNO). The authors should elaborate how their method compares to such schemes.\n\n**Regarding time-performance of DTM:**\nDTM is in fact **considerably faster** than FM. We have conducted several experiments and added several relevant Tables **in Reviewer’s ag7i response**. \nMainly, Table 1 in Reviewer’s ag7i response compares the wall-clock time of FM and DTM, and in particular DTM (16 backbone NFE and 4 head NFE) can **achieve superior performance** to FM (128 backbone NFE) with a **7x speedup**. \nIn more detail, Tables 2(a–b) in Reviewer’s ag7i response present the dependence of DTM’s CLIPScore and PickScore on the number of function evaluations (NFE) in the flow head and backbone. Table 4 in Reviewer’s ag7i response reports the corresponding inference times. Tables 3(a–b) present flow matching’s (FM) CLIPScore and PickScore as a function of backbone NFE.\n\n\n**Regarding test-time scaling methods:**\nNevertheless, we did compare to one rather popular test-time scaling method Restart [11], see Tables 5.(a-b) and 6 **in Reviewer’s QC47 response**. Note that although Restart (with up to 2400 NFE) shows some improvement over FM it is still *considerably sub-par* to DTM.\nReNO [12] utilizes an additional reward loss and therefore cannot be considered a strictly apples-to-apples comparison. Incorporating reward based optimization to TM is, however, a very interesting future research avenue!\n\n**Regarding ARTM/FHTM test-time scaling:**\nWe are not aware of any useful test-time scaling methods for AR image generation. The ARTM/FHTM variants do not offer any improvement in sampling efficiency, however compared to the AR baseline they achieve far superior performance. And we do not claim these variants outperform the FM baseline.\n\n[1] Esser, Patrick, et al. \"Scaling rectified flow transformers for high-resolution image synthesis.\" Forty-first international conference on machine learning. 2024.\n\n[2] Fan, Lijie, et al. \"Fluid: Scaling autoregressive text-to-image generative models with continuous tokens.\" arXiv preprint arXiv:2410.13863 (2024).\n\n[3] Zhou, Chunting, et al. \"Transfusion: Predict the next token and diffuse images with one multi-modal model.\" arXiv preprint arXiv:2408.11039 (2024).\n\n[4] Wang, Xinlong, et al. \"Emu3: Next-token prediction is all you need.\" arXiv preprint arXiv:2409.18869 (2024).\n\n[5] Chen, Xiaokang, et al. \"Janus-pro: Unified multimodal understanding and generation with data and model scaling.\" arXiv preprint arXiv:2501.17811 (2025).\n\n[6] Liu, Jie, et al. \"Flow-grpo: Training flow matching models via online rl.\" arXiv preprint arXiv:2505.05470 (2025).\n\n[7] Betker, James, et al. \"Improving image generation with better captions.\" Computer Science. openai. com/papers/dall-e-3. pdf 2.3 (2023):8.\n\n[8] Wallace, Bram, et al. \"Diffusion model alignment using direct preference optimization.\" Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2024.\n\n[9] Podell, Dustin, et al. \"Sdxl: Improving latent diffusion models for high-resolution image synthesis.\" arXiv preprint arXiv:2307.01952 (2023).\n\n[10] Saharia, Chitwan, et al. \"Photorealistic text-to-image diffusion models with deep language understanding.\" Advances in neural information processing systems 35 (2022): 36479-36494.\n\n[11] Xu, Yilun, et al. \"Restart sampling for improving generative processes.\" Advances in Neural Information Processing Systems 36 (2023): 76806-76838.\n\n[12] Eyring, Luca, et al. \"Reno: Enhancing one-step text-to-image models through reward-based noise optimization.\" Advances in Neural Information Processing Systems 37 (2024): 125487-125519."}, {"review_id": "IyJRi33KQA", "reviewer": "Reviewer_kPcj", "summary": "This paper introduces a general framework called Transition Matching (TM), which unifies continuous-state autoregressive models. The framework is built upon three core design choices: the supervision process, kernel parameterization, and the modeling paradigm. By making specific selections within these dimensions, the authors propose three powerful new variants: DTM, ARTM, and FHTM. The empirical results show that the proposed method FHTM can match the quality of full-sequence flow model in image generation.", "questions": "Please address the weaknesses above.", "limitations": "Yes.", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 1, "significance": 2, "originality": 1, "strengths_and_weaknesses": "**[Distinctions to previous works]**\n\nThe paper proposes a general framework of continuous state autoregressive models by defining the three major design spaces. \n\nHowever, the overall writing was not easy to follow with unconventional notations and missing justifications and explanations. \nFirst, design space analysis has been widely explored in previous works [1] (diffusion), [2-3] (diffusion and flow), [4] (diffusion, flow, and \nCTMC). For me Sec. 2.1 seems as a preliminary, not a core component of the paper, and the author's claims of unifying flow, diffusion under TM is too bold. \n\nIn Sec. 2.2, the paper specifically focuses on continuous state autoregressive models, which I could not spot the differences of DTM from MAR [5]. While the authors propose modeling the latent difference, rather than the conventional $\\epsilon$-prediction in diffusion models or velocity prediction in flow models, I am not convinced why this is needed and such a design choice should give better results. Similarly, the use of independent linear process is not justified. What does it exactly mean by a better regularity of the conditional reverse kernel? \n\n\n**[DTM results]**\n\nThe authors show that DTM achieves the best performances to full sequence denoising such as FM. \nHowever, I find the results contradictory since the DTM also outperforms AR-based methods that adopt causal attention. \nAdditionally, how does DTM compete against FM in inference time or training stability? \n\n\nMinor typo: well-explored (L2), mixed use of discrete, continuous time (L125-126). \n\n\n\n[1] Denoising Diffusion Implicit Models\n\n[2] Flow Matching for Generative Modeling\n\n[3] Stochastic Interpolants: A Unifying Framework for Flows and Diffusions\n\n[4] Generator Matching: Generative modeling with arbitrary Markov processes\n\n[5] Autoregressive Image Generation without Vector Quantization", "was_revised": true, "final_justification": "The paper presents a new parameterization that benefits from larger mode coverage. \nAdditionally, the comprehensive results show promising results for AR-diffusion models and provide apple-to-apple comparison results which can benefit the future researches. I recommend this paper to be accepted in NeurIPS 2025.", "rebuttal": "> The overall writing was not easy to follow with unconventional notations and missing justifications and explanations. \n\nWe thank the reviewer for the feedback and are sorry to hear that parts of the paper were difficult to follow. We made an honest effort to find a notation to describe the design space of TM. If the reviewer can provide more specifics on what is unclear we will try and clarify further. \n\n\n> Design space analysis has been widely explored in previous works [1] (diffusion), [2-3] (diffusion and flow), [4] (diffusion, flow, and CTMC). For me Sec. 2.1 seems as a preliminary, not a core component of the paper.\n\nThe reviewer correctly notes that previous works have explored learning transition kernels of Markov chains with a simulation free loss similar to equations 4 and 5, particularly in the context of diffusion and flow-based models. \nHowever, we identify several key differences that make Section 2.1 more than just a preliminary: (1) [1-4] focus on factorized transition kernels (e.g., Gaussian/deterministic kernels), with the exception of MAR [5] which is patch-wise factorized, and we further elaborate on DTM vs. MAR to an answer below. In contrast, we formulate a general, simulation-free learning methodology of **arbitrary** transition kernels. (2) we provide a formulation that allows arbitrary supervision processes and a systematic way to consider different kernel parameterizations; and (3) we feel the level of generality we chose allowed us to provide a full exposition to TM, including all design choices, within (hopefully clear) 2 pages. We expect this to allow future researchers to rather quickly get familiar with the field in a rather general manner, capturing the main design choices of existing methods. \n\n\n> The author's claims of unifying flow, diffusion under TM is too bold.\n\nTM includes diffusion/flow matching and AR, however we do not claim it to be the first framework to include both diffusion and flow matching (or AR to that matter). If the reviewer can point to where such a claim is made in the paper we are happy to clarify it in the text.\n\n> In Sec. 2.2, the paper specifically focuses on continuous state autoregressive models, which I could not spot the differences of DTM from MAR [5]. \n\nDTM and MAR indeed both use the same _Modeling_, i.e.,  DiT backbone with a small flow/diffusion head but they are distinct in their _Supervising Process_ and _Parametrization_ which lead to different training and generation processes and notably lead to very different performance where MAR is **considerably sub-par** to DTM .\n\nIn detail:\nSupervising Process:\n* DTM: $X_t = \\left( 1-\\frac{t}{T} \\right)X_0 + \\frac{t}{T}X_T$, where $X_0\\sim\\mathcal{N}(0, I)$. \n\n* MAR: $X_t = \\left(1-B_t \\right) \\circ M + B_t \\circ X_T$, where $M$ is a masked image, $B_t^{i}, i=1,…,d$ are i.i.d Bernouli$\\left(\\frac{t}{T}\\right)$ random variables, and $\\circ$ is the Hadamard product.\n\nParametization:\n* DTM: $Y=X_T-X_0$.\n* MAR: $Y=X_T$.\n\nHence, (i) the DTM model’s input ($X_t$) is a noisy image while the MAR model’s input ($X_t$) is an image where some patches are clean and some are masked. (ii) DTM model’s output ($Y$) is a direction $X_T-X_0$ intersecting $X_t$ while MAR model’s output ($Y$) is an estimation of the clean image. (iii) On generation, DTM is **non-AR** and predicts a direction and takes a small step in that direction, changing **all patches at each step**. In contrast, MAR is **arbitrary-order-AR** that predicts a clean image and masks back some of the patches, thus changing only a **“set of tokens” at each step**. A set of tokens of size one leads to an autoregressive model in the order of patch generation.\n\n\n> While the authors propose modeling the latent difference, rather than the conventional $\\eps$-prediction in diffusion models or velocity prediction in flow models, I am not convinced why this is needed and such a design choice should give better results. \n\nFirst we would like to emphasize that in practice, as shown in the paper, DTM is **considerably favorable** to the flow matching (FM) method. \n\nSecond, as the reviewer is probably aware, all diffusion-type methods reproduce the training set at a global minimum loss, therefore the hope of theoretically distinguishing generalization abilities among them is probably futile. However, we can offer an intuition: Given a current state $X_t=x_t$, FM learns to approximate the **expected transition** $Y_t = \\mathbb{E}[X_T-X_0|X_t=x_t]$ while DTM learns to **sample the underlying distributions of transitions** $Y_t \\sim p_{Y|t}(\\cdot|X_t=x_t)$. We hypothesize that in the large scale settings (as in our case) where model capacity is not a constraint, the more elaborate supervision of DTM is beneficial for the model training.\n\n> Similarly, the use of independent linear process is not justified. What does it exactly mean by a better regularity of the conditional reverse kernel? \n\nWe respectfully disagree that the independent process is unjustified.  \n\nFirst we refer the reviewer to Figure 10 in the Appendix that shows a significant advantage for the independent linear process, therefore using this process is **justified empirically**. \n\nSecond, regarding regularity, given a current state $X_t=x_t$, Figure 6 in the paper illustrates that the support of the transition kernel of the independent linear process (equation 15) is significantly larger than the support of the linear process (equation 10). In practice, using the linear process with TM steps$>1$ for training an autoregressive kernel results in trivial overfitting (since a linear interpolant is fully determined by any two points on the trajectory) and leads to poor results. Thus, the use of the independent linear process which has a wider support results in a better regularized training objective and leads to state-of-the-art results.  \n\n> The authors show that DTM achieves the best performances to full sequence denoising such as FM. However, I find the results contradictory since the DTM also outperforms AR-based methods that adopt causal attention. \n\nWe are not sure we understand the reviewer’s comment, why “DTM outperforms AR-based” methods is contradictory? As we found in our experiments, which were performed in a controlled and fixed settings, and is also shown in [5] that the reviewer mentions, we found AR (causal) methods to be sub-par in general to non-AR (bidirectional) methods in generated image quality. \n\n\n> Additionally, how does DTM compete against FM in inference time or training stability?\n\nTo show inference time of DTM versus FM we have conducted several experiments and added several relevant Tables **in Reviewer’s ag7i response**. \n\nOverall, Table 1 in Reviewer’s ag7i response compares the wall-clock time of FM and DTM, and in particular DTM (16 backbone NFE and 4 head NFE) can **achieve superior performance** to FM (128 backbone NFE) with a **7x speedup**. \n\nIn more detail, Tables 2(a–b) in Reviewer’s ag7i response present the dependence of DTM’s CLIPScore and PickScore on the number of function evaluations (NFE) in the flow head and backbone. Table 4 reports the corresponding inference times. Tables 3(a–b) in Reviewer’s ag7i response present flow matching’s (FM) CLIPScore and PickScore as a function of backbone NFE.\n\nRegarding training stability, the reported DTM and FM model are trained with the exact same hyper-parameters and training steps, and we didn’t observe any stability issues. \n\n> I could not find limitations nor societal impact of the work in the paper.\n\nThe limitations and societal impact statement appears in the conclusions section in lines (286-287) and lines (289-290). We will modify/elaborate the limitations to include “The improved performance of ARTM/FHTM comes at the price of a higher sampling cost, i.e., NFE counts are proportional to the number of transition steps, see e.g., Table 1 in the paper.”"}]

Author rebuttal:

Meta-review: The paper introduces Transition Matching (TM), a novel discrete-time generative modeling framework that unifies diffusion/flow models and autoregressive generation through three variants: DTM, ARTM, and FHTM. Key contributions include DTM achieving 7× speedup over flow matching while maintaining superior performance, and FHTM representing the first fully causal continuous-domain model to match non-causal methods. The comprehensive evaluation on 1.7B parameter models across multiple benchmarks with controlled experimental conditions provides strong empirical validation.

The rebuttal period was highly productive, with authors successfully addressing all major reviewer concerns through additional experiments and clarifications, leading three reviewers (QC47, jNpW, kPcj) to raise their ratings. While presentation clarity could be improved, the core technical contributions are substantial and theoretically sound.

Overall, the work advances state-of-the-art in generative modeling, opens new research directions for multimodal systems, and demonstrates significant practical improvements in both efficiency and generation quality. Hence, the AC recommends acceptance.
Decision: Accept (poster)