Exploring the Design Space of Transition Matching
Authors: Uriel Singer
Yaron Lipman
Venue: ICLR 2026 Poster (2026)
Keywords: flow matching
transition matching
generative models
Primary area: generative models

Abstract: Transition Matching (TM) is an emerging paradigm for generative modeling that generalizes diffusion and flow-matching models as well as continuous-state autoregressive models. TM, similar to previous paradigms, gradually transforms noise samples to data samples, however it uses a second ``internal'' generative model to implement the transition steps, making the transitions more expressive compared to diffusion and flow models. To make this paradigm tractable, TM employs a large backbone network and a smaller "head" module to efficiently execute the generative transition step.
In this work, we present a large-scale, systematic investigation into the design, training and sampling of the head in TM frameworks, focusing on its time-continuous bidirectional variant. Through comprehensive ablations and experimentation involving training 56 different 1.7B text-to-image models (resulting in 549 unique evaluations) we evaluate the affect of the head module architecture and modeling during training as-well as a useful family of stochastic TM samplers. We analyze the impact on generation quality, training, and inference efficiency. We find that TM with an MLP head, trained with a particular time weighting and sampled with high frequency sampler provides best ranking across all metrics reaching state-of-the-art among all tested baselines, while Transformer head with sequence scaling and low frequency sampling is a runner up excelling at image aesthetics.
Lastly, we believe the experiments presented highlight the design aspects that are likely to provide most quality and efficiency gains, while at the same time indicate what design choices are not likely to provide further gains.

Published as a conference paper at ICLR 2026
EXPLORING THE DESIGN SPACE OF TRANSITION MATCHING
Uriel Singer, Yaron Lipman FAIR at Meta
ABSTRACT
Transition Matching (TM) is an emerging paradigm for generative modeling that generalizes diffusion and flow-macching models as well as continuous-state autore- gressive models. TM, similar previous paradigms, gradually transforms noise samples to data samples. however it uses second "internal" generative model to implement the transition steps, making the transitions more expressive compared to diffusion and flow models. To make this paradigm tractable, 'TM employs large backbone network and a smaller "head" module to efficiently exccute the generative transition step. In this work, present large-scale, systematic investigation into the design, training and sampling the head in TM frameworks, focusing its time-continuous bidirectional variant. Through comprehensive ablations and experimentation involving training 56 different 1.7B text-to-image models (resulting in 549 unique evaluations) architecture and modeling during training as-well as a useful family of stochastic TM samplers. We analyze the impact on generation quality, training, and inference efficiency. We find that TM with an MLP head, trained with particular time weighting and sampled with high frequency sampler provides best ranking across all metrics reaching state-of-the- head with sequence scaling and low frequency sampling is runner up excelling at image aesthetics. Lastly, we believe the experiments presented highlight the design aspects that are likely to provide most quality and efficiency gains, while at the same time indicate what design choices not likely to provide further gains.
evaluate the affect of the head module
among all tested baselines, while Transformer
INTRODUCTION
Transition Matching (TM) Shaul et al. (2025) is recent generalization of several media generative paradigms including diffusion models Sohl- Dick-Dicsttenn et al. (2015): Ho al. (2020); [Song al.)
(2020), flow matching models Lipman et al. (2022) Liu et al. (2022) Albergo & Vanden-Eijnden (2022). , and continuous-state autoregressive image generation Li et al. (2024); Team et al, (2025) that offers new design choices that go beyond the scope of these former paradigms and already shown to yield improved image quality and/or more efficient sampling at inference time.
In this work we focus TM's continuous time bidirectional variant, which, similarly to previous paradigms, learns a transition function (kernel) that gradually transfers a source (noise) sample Xo to target (data) sample X, by iteratively producing future samples X₃ from previous samples Xt- 0 < < < 1. Differently from previous work, TM models the transition kernel with second "internal" generative model, offering more expressive transition kernels than, e.g., diffusion models that utilize a factorized (i.e., independent in each coordinate) multivariate Gaussian as kernels. To keep things tractable, TM adopts backbone head paradigm, which: The backbone (typically large transformer) encodes current state X, as well as conditioning information, producing a rich latent representation per input token. The head (typically much smaller than the backbone) serves as a learnable module tasked with translating backbone latent representations into concrete transition outputs, producing the next state Xe with '' > While backbone architecture for diffusion models have been, and still are, thoroughly investigated (e.g., Peebles & Xie (022 ), systematic exploration of head architecture and hyperparameters is lacking in the current literature. Most existing works treat the head fixed, minimal component- often single MLP lightweight mapping- hriphuggwtthhutt -wihhut investigating how variations in design might impact model behavior and efficiency (Li et al.] 2024]
Fan al. Team al. 2025 Shaul et al. 2025). In fact, due its particular role in the generative

---

Published as a conference paper at ICLR 2026
DTM+ (ours)
DIM+ (ours)
DTM
FM-lognorm
AR-dis
MAR-dis
MAR
fox
stone bust next to an and un eggplant
a tiger wearing a tuxedo
Greek comforting The has big head. The looks angry.
A high resolution photo of large bowl of ramen. There are several origami hoats in the ramen of different colors.
Figure 1: Samples comparing of our Best D-TM MLP (DTM++) and Transformer (DTM+) Models with DTM baseline, FM-lognormal, AR, MAR, AR-discrete, MAR-discrete as basclines. All models share similar architecture and training recipe.
process and its small relative size, the head design holds much potential for improving the model performance by exploring head-specific architectures and different scaling laws. this paper we take on this opportunity and explore different design choices for the head both in training and inference stages, with the goal of improving one or more of the three main properties highly performant generative model: generation quality, training efficiency, and inference efficiency. We offer the following contributions:
(i) Comprehensive exploration: We perform a large-scale (i.c., 56 different 1.7B unique model trains resulting in 549 unique evaluations), systematic ablation study of TM design space including exploration of head model architecture and size, sequence scaling laws, batch size scaling laws, time weighting, model parameterizations and inference algorithms. For fair comparison across models and baselines we keep backbone, training dataset, and most training hyperparameters strietly fixed, while evaluating all models on comprehensive setup of datasets with 25 individual metrics summed up to a single performance rank.
(ii) TM sampling: We design novel stochastic sampling algorithm for Transition Matching that is shown to considerably improve generative quality while keeping the computational cost the same.
(iii) Actionable Guidelines: Our ablations illuminate tradeoff's and best practices for continuous-time bidirectional TM-based generative models. In nutshell: TM with MLP head trained with particular backbone-head time weighting and sampled with high frequency stochastic sampler leads the best ranking model (DTM++), where Transformer head with sequence scaling and low frequency sampling is the runner-up (DTM+) that excels in image aesthetics, see fig.
2

---

Published as a conference paper at ICLR 2026
2 BACKGROUND: CONTINUOUS-TING TRANSITION MATCHING
Goal and motivation of Transition Matching In this paper we focus on continuous-timg fully- bidirectional variant of Transition Matching (TM) (Shaul al. 2025), which we found to lead best results in our text-to-image experimental setup. An image is encoded as a sequence of continuous tokens in dimension that variables. Similar to diffusion and flow models, TM learns a transition kernel,
(x' ,...,, .²") E Raxd Capital letters are used to denote random
Xy ~ (x))) 0 s<t<t<
gradually transforming a source (e.g., noise) sample Xo ~ Po to a target (e.g., data) sample X1 Pi where / denotes the learnable parameters. In diffusion models IT transitions p.y have the simplified form of a factorized Gaussian kernel.
(r])(-12)= = = -0)-N((1
where in flow matching this kernel is even simpler, i.e., a delta function. Transition Matching (TM) is generalization of diffusion and flow models that utilizes more expressive transition kernels ply which are modeled by an "internal" generative flow model that learns to sample X² given state X₁, TM learns P by matching it to the transitions of some user defined supervision process denoted 2e defined next.
Supervising process and kernel parameterization The TM model plr is learned by regressing
a supervision process 4. A supervision process is any random process (X,) (e()) with probability density q such that its marginals, denoted by q: (xx), at time 0 and / = coincides with the desired source distribution Po and target distributions Pl. respectively. That is, do Po and @1 Pi- The choice of supervision process q is a design freedom of TM, where in this paper we follow the standard choice of the linear (a.k.a. conditional optimal transport) path (Lipman et al. 2022 Liu
et al. 2022 Shaul et al. 2025):
X, = (1 - 1) Xo + LX,
Linear supervising process
where E [0, 1], Xo 1), is noise sample, and X1 ~ P1 is data sample.
The conditional distribution (xter (axz) is the main object we want to regress in TM. That is, given a current state X₃ learn to sample X2 so is distributed according to gr (-| However, is often useful to learn predict different random quantity Y given X, instead of directly X The reason is two-fold: First, avoid dealing with the extra time variable t' during training; and second, improve inductive bias and performance by e.g., removing the dependence make the Y parameterization useful in practice, one needs to make sure that predicting Xy given samples and X, rather easy and computationally cheap. The parameterization process justified mathematically by the law of total (conditional) probability.
& in the target quantity. To
(el((velz)) = qeree (re (a (
(4)
That is, TM learns to sample Y pr1e(-121)) After a sample of Y produced, the next state is sampled according to
X J (,X,,,
which is guaranteed to have the distribution genel (X)) by the law in eq. (4). The choice of Y is another degree of freedom of TM. (Shaul al. (2025) made the choice of noise-data difference,
Y= X1 Xo
D-TM
(6)
motivated by the relation
X = X, x (X -o)
(7)
that holds for the linear process (eq. Bh: this variant is called Difference Transition Matching (D-TM). An equivalent parameterization is also explored in Zhang et al. (2025).
We use the forward-time convention similar flow matching, while standard diffusion models use backward- time convention moving from state X, to X,-1-
3

---

Published as a conference paper at ICLR 2026
Baciboe
MIP
MIP
MIP
Conotation
Turaforna
Figure 2: Head architectures explored (in green): MLP, Convolution, and Transformer. The backbone
model (in orange) is kept fixed.
Modeling The goal in TM is to learn model to sample Y PY]e(- Xx) for each / E [0. 1) and X₁ ~ qt. This entails an "interior" model that is resampled for each t. To make this tractable (and in fact, improve performance), TM uses backbone-heal architecture. That is, given current state X a backbone network computes a latent vector representation.
he = (X.).
(8)
Next, this latent vector is used by a head network u e (1)xg) ) to sample ~ (+1,) The head
will be used to sample Y via flow matching (Lipman et al., 2022; Liu et al. 2022 Albergo & Vanden-Fijnden, 2022). That is, is learned by minimizing the flow matching loss
(Im( ,, || (Y,(k,) (Yi -Yi -o))l
(9)
where t,s ~ (O,,1) 1) uniformly and independently, X, (1 t)X, + tX, gc. Yo N/O. I).
Yi ~ (x,, and Y, = (1 1--s] 3) + sY,, Once training is completed, sampling Y ~ (-1)(- done by solving the ordinary differential equation (ODE),
Y, (Y,
(10)
ds
starting with Yo .N(0,) I) and solving until 1, with h2 f (1:2). The total learnable parameters of the TM model are = (4p,o) ,); for brevity, we sometimes omit the parameters superscript. The sampling pseudocode is provided in algorithm
EXPLORING THE DESIGN SPACE
The main goal of this paper is explore the design space of continuous-ting Transition Matching for maximizing performance (i.e., quality and text adherence of generated images) and efficiency (i.e., inference and training speed). First we consider the training phase, focusing on head modeling and architecture. The head Uzjt is responsible for sampling Y given the current state X, encoded via latent h, computed by the backbone ft. The head introduces a useful leverage point as it can be chosen (as we will see) to be significantly smaller and faster than the backbone model and therefore exhibits scaling laws that can improve the overall performance and efficiency without significant increase to the overall computational and memory costs. Second, we explore different inference options including Efficiency-qualiin tradeoff's and a novel TM stochastic sampling algorithm. We start by describing experimental setup (text-to-image generation), and then move to discuss the different design choices explored and the relevant experiments.
3.1 EXPERIMENTAL SETUP
We start with explaining the experimental setup that is fixed throughout the experiments.
Backbone model and Data We fix our backbone model f( a DiT transformer model (Peebles Xie 2022) with 24 layers (including self and cross attention) and hidden dimension of d, 2048; this gives total of 1.7B parameters for the backbone. Our data consists of 350M text-image pairs. Each image is of dimension 256 x 256 x 3; we move it to a latent representation as follows: we first embed it into a latent space using SDXL-VAE (Podell et al. 2023) to dimension 32 x 32 x 4 and then 2 2 patched to get a latent representation of 16 x 16 x 16 and therefore our data and noisy vectors X.Y E Rnxd where n = k² with & = 16 and d = 16. The standard hyperparameters are following Shaul al. (2025) including optimizer, number of training iterations (500k), where the only difference is that we use learning rate decay. We use standard Classifier Free Guidance (CFG)
(Ho & Salimans] 2022) training and sampling for flow matching, see appendix F
4

---

Published as a conference paper at ICLR 2026
EIN ACA
14
(a) Performance
(b) Inference Speed
(c) Training Speed
Figure 3: (a) Performance as a function of additional parameters. Increasing the head size does not
improve performance. (b) Inference speed (seconds). (c) Training speed (iteration, seconds).
Metrics We report text-to-image metrics over four different prompt datasets: MS-COCO (Lin al. 2014), PartiPrompts (Yu al. 2022). GenEval (Ghosh al. 2023), and T2ICompRench
(Huang et al. 2023). Over MS-COCO and PartiPrompts
calculate standard image quality metries:
CLIPScore (Hessel et al. 202T). PickScore (Kirstain et al. 2023), ImageReward (Xu et al. 2023a),
UnifiedReward ( (Wang al. 2025), Aesthetic (aes 2022), and DeQA (You et al. 2025). On GenEval
and T2ICompBench we calculate their internal metrics and the corresponding overall scores. As there is large amount of datasets (4) and metrics (25 different metrics), aggregate all metrics into a single rank calculated for each evaluated model as follows. We rank each model to be tested (we have total of 549 such models from 56 unique trains) according each metric (scores from 1 to 549, where higher is better). We then average these ranks across all metrics and divide by the total number of models to get final rank score in [0, 1] for each model.
3.2 TRAINING AND HEAD MODELING
In this section we discuss the TM training and explore different design choices for the head model: head architecture type and size, sequence scaling, model parameterization Y, head batch size, and time weighting. In each of the following experiments we start from a base model and ablate on the specific design choice. Our base model is: a medium size head, with sequence scaling of 1, difference paramterization Y = X Xo. head batch size 4, uniform backbone and head time weighting, and sampled with 32 32 backbone-head steps; FM uses 256 steps.
Head architectural type One natural design choice is the architecture type of the head model Malt- We considered three options, see also fig. 2 (i) MLP- This is the most basic choice, used in (Shaul et aaall|200 2025), where an MLP acting independently on each image token y given the current step's latent he, and producing a prediction token dimension Rd, i.e. u (y" [4j) E RRd, 1,..., n.
(ii) Convolution we use 2D convolution layers across the image tokens y' with kernels of size 3 x 3. (iii) Transformer- incorporating attention layers across the image tokens y². Both Convolution and Transformer head architectures take in the entire sequence of tokens y E Raxd and produce a prediction in the same dimension, i.e., Ugje(1) (1) Raxd For the transformer head further apply 16 x 16 X 16 + 8 x 8 x 64 reshape layer to allow for equivalently efficient head.
Head model size To check the influence of both head type and size, we have experimented with a variety of head model sizes for each type: x-small, small, medium, large, and x-large. In particular these correspond to head models with hidden dimensions of d, E {768, 1024, 1280, 1536, 2048} (remember that our backbone hidden dim is dn 2048) and {6, 6, 12, 16) layers, respectively. Figure [ [asuure]shooss shows, for each architecture type, the model rank as a function of the relative additional parameters to the backbone. For D-TM this corresponds flow matching (FM) we also show what happens when we add the same number of parameters to the backbone as our medium size head. Dense shows the affect of using the backbone for both backbone and head similar to Zhang et al. (2025). Note that while having head considerably improves the model's s rank, the size of the head does not show strong correlation with performance, even in the limit case with head almost the size of the backbone (>l relative size). Figure 3 (b) and (c) show the effect of different heads on inference and training iteration time. As expected both inference time and training time increase with the head size and is particularly costly for dense inference. To summarize, smaller head sizes already provide good performance, improved inference time compared to FM and roughly equivalent training time.
the relative size of the head, while for
5

---

Published as a conference paper at ICLR 2026
Sequence scaling We tested the affect of scaling the number of tokens inserted into the head. To that end we trained three learnable linear layers (see fig. 10 (a)): Lin.y : R + R!xd Lin Rdh + R R!xd, and Lout.y RR(dd - RR, where Lin.y maps each token y' into / tokens of the same dimensions; Lin maps backbone latents he into I tokens of the same dimensions; and Lout,y maps back I output tokens into a single latent token yy" We experimented with different scalings E {1,22, 3,, 42, 42,52, 6²) where vt is applied to each dimension (of size k) of the latent image IR xd or he RRxxd xd. The sequence scaling law is incorporated in the head as follows
Lount Mal c/ (fin.yy/ he)-
(11)
Figure 4 shows that for the transformer head, scaling the head sequence improves the model ranking significantly. In contrast, for the MLP head, scaling up the sequence does not impact performance consistently. One possible explanation is that MLP is applied on each scaled token independently. compared to the transformer that share information across all tokens via the attention layers. In fig. (b) we report the inference speed as a function of sequence scaling, and in (c) we show the affect on training speed. Notably, while the affect of sequence scaling is limited in inference speed it is rather significant during training.
2415
(a) Performance
(b) Inference Speed
(c) Training speed
Figure 4: (a): Performance function of the sequence scaling factor. Scaling the input sequence to the head improves Transformer heads while not affecting MI. heads.
Model parameterization Y Another design choice of the TM head the choice of posterior Y learned by the head and used to sample the next state Xp. For the linear process (eq. (3) formulating the equations for X₂ and X, results in three unknowns, Xo, X1, Xv, and single known quantity, X,. Therefore predicting one more quantity such as Xo or X, or any independent relation of those two, would allow us compute Xp. Alternatively, we can also predict Xy directly but as mentioned above that would force us to introduce a second time parameter L'. Therefore, here we opt to experiment with the following options:
Figure 5: Performance of different Y choices.
E (X1 Xo: X1, Xo) Y-TM parameterizations
(12)
Calculating Xy given X, and Y gives instantiation of the sampling relation in eq. 5 and given for completeness in appendix H In fig. 12 we log the effect of different Y parameterizations on the ranking of Y-TM models, where the difference parameterization Y Xi X is better than
denoiser = X2 which is much better than noise prediction Y == Xp- In appendix G we show the ablation of the flow matching target used in the FM loss, where the difference parameterization (used
in eq. (9) is also favorable.
Head batch size Another simple and mod- erately effective scaling law can be achieved by increasing the batch size used by the head (see fig. [0 (b)). In practice for each time t and state sample X,, we consider head batch sizes kn E {1,4, 4, 16, 64), see more details in appendix C For the MLP architecture, since each token is processed independently (see fig. [eeonnd from left), we use different i..dd s samples for each token; name this pe---eer- (TPT). In table (a) we report experiments
(a) Performance
(b) Training Speed
Table 1: (a) Performance as a function of the head
batch size. (b) Training iteration time (seconds).

---

Published as a conference paper at ICLR 2026
comparing different head batch sizes for D-TM; performance, while reaching plateau after 16 for the Transformer head. The impact of head batch size is smaller for the MLP head compared to Transformer, which can potentially be explained by the fact that MIP attends to each token as an independent sample, compared to a transformer where all patches of the same image treated as a single sample. In (T) report training speed, which shows that going beyond K; = 16 might lead to significantly slower training times.
that using a larger head batch size boosts
Time weighting A useful training design choice explored (Esser et 2024) for flow matching is time weighting during training. In our case, instead of uniform time sampling t,8 ~ (0, 1) in the TM loss (eq. (9), we con- sider non-uniform time sampling that will concentrate certain times for the backbone (t) and head (s). Esser et al. (2024) noticed that for flow matching time it is beneficial to use a centered distribution favoring mid-
(a) weighting
(b) 8 weighting
dle times in the interval (0, 1). In particular, the log-normal distributions Tin (p, where (p, = (0, 1) was favorable. In our case, we tested time weighting for both the backbone t and head s time parameters for D-TM. We tested two t-weighting distributions similar to flow matching, (0, 1) and mi (0, 1), while for s-weighting we tested (10 1), Min (0, 1), mm mm (-0.5, 1) and (eta( 1.3), Beta(0.5, 2.0). and B 2.4), where Beta denotes the beta distributions. We chose the values to cover different s s-time profiles, see fig. [T for an illustration. We report results for the ablations of the backbone time weighting in table (a) and for the head time (with ma (0, 1) for backbone) in (b). In general, backbone training enjoys standard log-normal time weighting Tia (0, 1), although the transformer head is equally good with uniform time weight. For the head time
Table 2: Performance as function of the backbone and head time weighting distribution. (a) Backbone time weighting (l), with Uniform time weighting on head. (b) Head time weighting (8), with lognormal time weighting backbone.
weighting, both Beta and standard log- log-normal works well with the exception of Beta(0.1, 1.3) for the Transformer head.
3.3 INFERENCE
We move to discuss design choices in sampling of the models inference time, i.e., once training is done. We focus on efficiency-quality tradeoffs and
Dasking
investigate the affect of novel stochastic sampling method that, under a particular design and hyper- parameter regime, offers significant quality boost at no extra sampling computational cost to D-TM.
DCM Tommseeent
14/26
Efficiency-qualle tradeoff When sampling a D- TM model we have the degree of freedom of choosing the t and time discretizations both affecting the total (wall-clock) generation time. Throughout this experiment we keep and t equidistant, i.e., f i/T and s = 5/S where T, T are number of steps we ablate over. Figure i scatter plot showing inference wall-clock speed (in seconds) versus model rank for a collection of T, S, namely we consider T E [1, 2, 4, 8, 16, 32, 64, 128, 256} and S e 21, 4, 8, 16, 32). We compare flow matching (FM), and D-TM with the three head architectures (MLP, Convolution, and Transformer), as well as the Dense architecture. As can be seen in the figure, D-TM (MIP, Transformer, and Convolution) sampling can made both faster and of higher quality. For example, FM peak performance is with 32 midpoint samples (64 NFEs, corresponding to the 4sec red dot), where D-TM-MILP can achieve higher ranking with 0.8 second providing ~ 5x wall-clock speedup.
Figure 6: Performance as a function of Infer- ence speed. Each color represents a different model, and each dot represents a different setup of transition steps and head NFE. We show Pareto optimal dots.
7

---

Published as a conference paper at ICLR 2026
Stochastic samplers In this last section we show that a certain family of stochastic sam- plers provides significant improvements to D- TMM sample quality at no additional computa- tional cost over standard TM sampling. The basic observation is that we can sample our trained model using any given supervision pro- cess q as long as: (i) it has the same marginal distributions as the one we trained our model with q; and (ii) (- Y) is known and can sampled efficiently. Inspired by (Xu et at 2023b) we develop a family of stochastic samplers for D-TM for the case of Gaussian source noise, i.c., Po = =N(00, 1). For Gaussian noise the conditional probability path takes the form (2] N' (zrltzı, (1 - =)) Now, let 0 < < t" I 0 be three consecutive times, then
a) DD-TMMMLI (b) D-TM (Transformer)
Table 3:
Stochastic sampling performance (i.e., model rank) for various T for the D-TM with MLP head in (a), D-TM with Transformer head in (b). Red colors indicate low ranking, while blue correspond to high ranking.
given a sample Xx" ~ ge" (-)X1) we can use it to sample from the marginal qe by (see appendix B for details)
1 (t'Xgrr + Z), with Z ~ N (t" - t)((' + - 2t
X = t"
(13)
Note that the joint probability of (Xx(X") no longer the same as the supervision pro- cess q but shares its marginals at times t', t,,";;
Algorithm 1 Stochastic D-TM sampling.
Require: pry
D Trained model
where in the extreme case of (" = I we get that X₁ 1 X (independent). This allows introducing more stochasticity into the D-TM sampling process where explore two hy- perparameters: (i) scale e E [0. used to set t" t' + c(1 t'); and (ii) frequency r T (1,....,,T) T) setting how often should we add a stochastic step; see algorithm [ for the pseudocode of the stochastic D-TM sampling.
Require: T
Backbone steps
Require: C, T
v Scale and frequency
1: Sample Xo ~ I0,
2: for I. = 0, T: :.....
+ do 3: Sample ~ (-1,))
4:
t' t+ 7
5:
it (mod [mood///T]>= [ then
6:
t" + -tt's +
Intuitively, the algorithm uses the D-TM pre-
7:
Compute X₃" Compute X,-
eq. 7
diction of Y = X1 Xo to move to future 8:
b eq.
state X,"' at time t", and then adds indepen- 9:
else
dent noise at the right amount so to achieve 10:
Compute X
D eq.
sample X₂ at the earlier time f'. In this exper-
11: end if
iment we tested 32 x 32 backbone-head sam-
12: end for
pling, which gave near optimal performance
13: return Xyr
in table 33 and the following hyper-parameters
(c. r) E TO, 0.02, 0.04. ........ 0.1, of stochastic sampling on D-TM MIP and Transformer head (resp.). Interestingly, MLP head enjoys the stochastic sampling more that Transformer head leading to a 0.66 rank (+0.15 from standard sampling), which is the highest in our experiments, and is consistently achieved across high frequency sampling. Transformer head reaches its peak performance of 0.58 rank (+0.06) with low frequency we discuss applying algorithm T to flow matching.
1} x f1,2, 8, 16, 32}. Table 3 (a) and (b) show the affect
sampling. In appendix M
3.4 SUMMARY RESULTS FOR SELECTED MODELS
Each of the design space exploration above tested ablations from base model (detailed in section 3.2) Among all these ablations we picked two design choices balancing performance and train/inference speed: (i) D-TM with MLP head, head batch size kn, = 16, lognormal and time weights, with and without high frequency stochastic sampling т = 32, c = 0.2; and (ii) D-TM with Transformer head, sequence scale 4. head batch size kn 16, lognormal and time weights, with and without low frequency stochastic sampling 7 = 1, c = 0.8. In table 4 and figs. [ and 7 to we present most performing D-TM variants discovered via the previous experiments compared with relevant baselines implemented. trained and evaluated under the exact same setting. The baselines include: the D-TM variant in (Shaul et al. 2025) and the Dense version in (Zhang al. 2025): Flow
8

---

Published as a conference paper at ICLR 2026
Matching (FM) with its optimal time-weighting variant (Esser et al.
2024); Autoregressive image
generation with a flow matching head generating continuous tokens (AR/MAR) Team et al. (2025); Li al. (0224) Fan et al.] discrete tokens autoregressive (AR) Yu et al., 2022): and discrete tokens masked autoregressive (MAR) (Chang et al. 2022). Due to lack of space we only present subset of the metrics where all the data, including all evaluations are presented in appendix L As can be seen in the table, design choice (i) with MLP head and high frequency sampling (denoted DTM++) reaches the top performance with 0.66 rank score; and design choice (ii) includes the Transformer head and excels at image quality (see e.g., Aesthetic and PickScore) but do not benefit as much from the stochastic sampling and ends up being second to the MLP head (denoted DTM+). As sequence scaling is training-coslie we limited it to I = 4, note that for I = 36 (see Sequence scaling) the Transformer head model becomes competitive with our best model.
Table 4: Main Results comparing most performant D-TM variants and relevant baselines.
Heat
NS-0CC
Pott Peoepe
Gorthil
12Dxeplosh
Suesiig
CLISSoN:1
CLES
Drank.
Ovcult
Roa
HLP
16
Irce
264
21.4
568
E4
27.
11.3
1.47
0A4
1:444)
8.51
DTM6+
MLP
16
Pu to -00.3) <- -277-1
363
21.4
576
144
21.
21A
4.57
0.8
0.55
14424
0166
DTM
Creviiiie
A1C.3y 0MW.11
Iree
361
31.4
576
0.33
26.
1.50
0.51
1.4MM
64
Taasfornc
16
(0.3)
Boae
362
583
1.4
266
139
042
[.451
0.5
16.
71.6
198
9:38
306
714
001
6.441
DTM
Doeic
t D1W.31
Iroe
359
21.3
588
8.1
26.1
5.44
0.5
14181
1.25
PM
367
11.5
67
11.
5.44
0.8
1.017
1.3
A4
199 13
511
0.3
301
0.1
36
108
266
8.1
354
535
1354
819
4 RELATED WORK
Iterative generative models, which gradually transform noise to data, were pioneered with diffusion
models (Sohl-Dickstein et al. 2015
[o et al. 2020) and further generalized and improved with
2022). Transition further recent generalization that replaces the simplified transition kernel in diffusion and flow models with a more expressive internal generative model. A related method that used the same difference modeling but with a single backbone architecture (called Dense in this paper) was developed in /Zhang et al. (2025). TM uses head- baddd-bcbonee construction to inject relevant inductive bias and maintain efficiency; similar head MLP constructions was introduced and incorporated in continuous autoregressive image generation methods (Li et al. 2024 Fan et a
flow matching (Lipman et al. 2022
Liu et al. 2022] Albergo & Vanden-
Matching (TM) (Shaul et al. 2025)
Team et al.] 2025). ) however different head models and architectures were not systematically explored previously. Similar to work, systematic design space exploration was done for diffusion models in Karras et al. (2022). Stochastic sampling has shown some benefit in diffusion (Song et al.. 2020) and flow (Ma al.] 2024) models while usually based on adding Langevin dynamics to existing SDE/ODE formulation. Our TMM stochastic sampler is inspired by Xu et al. (2023b) that re-noises the variance exploding denoiser in Karras et al. (2022). ) see more details in the Stochastic samplers section.
5 CONCLUSIONS
We conducted a large-scale ablation of design choices in continuous-time Transition Matching, focusing on the D-TM variant. Our experiments highlight that head design size, sequence scaling, head batch size, and time scheduling - can significantly improve generative quality and efficiency, even with a fixed backbone and dataset. Some design choices, such as large sequence scaling and head batch size improve performance but at the cost of increase training times and/or memory footprint. Stochastic TM sampling offers a further significant improvement in generation quality at not extra computational cost. It is not clear to the authors why token-wis Wise head (i.e., MLP) leads to improved text-adherence scores compared to the more expressive Transformer head, which opens an interesting future research question. Furthermore, we emphasize that the conclusions of this paper are based on a focused study of image generation at 256 resolution, where higher resolution images and/or other media modalities such as video and audio could potentially lead to different conclusions and provide valuable future work direction. LLMs were used in this paper to aid/polish writing in introduction and abstract sections.
9

---

Published as a conference paper at ICLR 2026
REFERENCES
Improved aesthetic predictor.
Limprovei-asstteee----reeede httpes//gttabbbcco////hhshhhhhht
2022.
Michael S Albergo and Eric Sanden-Eijnden, Building normalizing flows with stochastic interpolants. arXiv preprint arXiv:2209. 15571, 2022.
Huiwen Chang, Han Zhang, Lu Jiang, Ce Liu, and William T Freeman. Maskgit: Masked generative image transformer. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 11315-11255 2022.
Patrick Esser, Sumith Kulal, Andreas Blatmaann, Rahim Entezari, Jonas Müller, Harry Saini, Yam Levi, Dominik Lorenz, Axel Sauer, Frederic Boesel, et al. Scaling rectified flow transformers for high-resolution image synthesis. In Forty-first international conference on machine learning, 2024.
Lijie Fan, Tianhong Li, Siyang Qin, Yuanzhen Li, Chen Sun, Michael Rubinstein, Deqing Sun, Kaiming He, and Yonglong Tian. Fluid: Scaling autoregressive text-to-image generative models with continuous tokens. In The Thirteenth International Conference on Learning Representations.
Dhruba Ghosh, Hannaneh Hajishirzi, and Ludwig Schmidt. Geneval: An object-focused framework for evaluating text-to-image alignment. Advances in Neural Information Processing Systems, 36: 52132 521522, 2023.
Jack Hessel, Ari Holtzman, Maxwell Forbes, Ronan Le Bras, and Yejin Choi. Clipscore: A reference- free evaluation metric for image captioning. arXiv preprint arXiv:2104.08718, 2021.
Jonathan Ho and Tim Salimans. Classifier-free diffusion guidance. arXiv preprint arXiv:2207.125w0 2022.
Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. Advances in neural information processing systems, 33:6840-6851, 2020.
Kaiyi Huang, Kaiyue Sun, Enze Xie, Zhenguo Li, and Xihui Liu. T2icooppeenhh: A compre- hensive benchmark for open-world compositional text-to-image generation. Advances in Neural Information Processing Systems, 36:78723-78747 2023.
Tero Karras, Miika Aittala, Timo Aila, and Samuli Laine. Flucidating the design space of diffusion- based generative models. Advances in neural information processing systems, 35:26565-26577, 2022.
Yuval Kirstain, Adam Polyak, Uriel Singer, Shahbuland Matiana, Joe Penna, and Omer Levy. Pick- a-pic: An open dataset of user preferences for text-to-image generation. Advances neural information processing systems, 36:36652-36661 2023.
Tianhong Li, Yonglong Tian, He Li, Mingyang Deng, and Kaiming He. Autoregressive image generation without vector quantization. Advances in Neural Information Processing Systems, 37: 56424-56445, 2024.
Tsung-Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and C Lawrence Zitnick. Microsoft coco: Common objects in context. In European conference on computer vision, pp. 740-755. Springer, 2014.
Yaron Lipman, Ricky TQ Chen, Heli Ben-Hamu, Maximilian Nickel, and Matt Le. Flow matching for generative modeling. arXiv preprint 1rX1v:2210.02747, 2022.
Yaron Lipman, Marton Havasi, Peter Holderrieth, Neta Shaul, Matt Le, Brian Karrer, Ricky TQ Chen, David Paz, Heli Ben-Hame and Itai Gat. Flow matching guide and code. arXiv preprint arXiv:2412.06264, 2024.
Xingchao Liu, Chengyue Gong, and Qiang Liu. Flow straight and fast: Learning to generate and transfer data with rectified flow. arXiv preprint arXiv:2209.03003 2022.
10

---

Published as a conference paper at ICLR 2026
Nanye Ma, Mark Goldstein, Michael S Albergo, Nicholas M Boffi, Eric Vanden-Eijnden and Saining Xie. Sit: Exploring flow and diffusion-basee generative models with scalable interpolant transformers. In European Conference on Computer Vision, pp. 23 40. Springer, 2024.
William S Peebles and Saining Xie. Scalable diffusion models with transformers. 2023 ieee. In CVF International Conference on Computer Vision (ICCV), volume 4172, 2022.
Dustin Podell, Zion English, Kyle Lacey, Andreas Blattmann, Tim Dockhorn, Jonas Müller, Joe Penna, and Robin Rombach. Sdxl: Improving latent diffusion models for high-resolution image synthesis, 2023.
otg/ abs/ 231.
Tim Salimans and Jonathan Ho. Progressive distillation for fast sampling of diffusion models. arXiv preprint 22X0022202..0051,, 2022.
Neta Shaul, Uriel Singer, Itai Gat, and Yaron Lipman. Transition matching: Scalable and flexible generative modeling. arXiv preprint arXiv:2506. 23589, 2025.
Jascha Sohl-Dickstein, Eric Weiss, Niru Maheswaranathan, and Surya Ganguli. Deep unsupervised learning using nonequilibrium thermodynamics. In International conference machine learning, pp. 2256-2265. pmlr, 2015.
Yang Song, Jascha Sohl-Dickstein, Diederik P Kingma, Abhishek Kumar, Stefano Ermon, and Ben Poole. Score-based generative modeling through stochastic differential equations. arXiv preprint arXiv:2011 1. 13456, 2020.
NextStep Team, Chunrui Han, Guopeng Li, Jingwei Wu, Quan Sun, Yan Cai, Yuang Peng, Zheng Ge. Deyu Zhou, Haomiao Tang. et al. Nextstep-1: 1: Toward autoregressive image generation with continuous tokens at scale. arXiv preprint arXiv:2508. 10711, 2025.
Yibin Wang, Yuhang Zang, Hao Li, Cheng Jin, and Jiaqi Wang. Unified reward model for multimodal understanding and generation. arXiv preprint arXiv:2503.0522 2025.
Jiazheng Xu, Xiao Liu, Yuchen Wu, Yuxuan Tong, Qinkai Li, Ming Ding, Jie Tang, and Yuxiao Dong. Imagereward: Learning and evaluating human preferences for text-to-image generation. Advances in Neural Information Processing Systems, 16:15903-55935, 2023a.
Yilun Xu, Mingyang Deng, Xiang Cheng, Yonglong Tian, Ziming Liu, and Tommi Jaakkola. Restart sampling for improving generative processes. Advances in Neural Information Processing Systems, (6.76806-768388 2023b.
Zhiyuan You, Xin Cai, Jinjin Gu, Tianfan Xue, and Chao Dong. Teaching large language models to regress accurate image quality scores using score distribution. In IEEE Conference on Computer Vision and Pattern Recognition, 2025.
Jiahui Yu, Yuanzhong Xu, Jing Yu Koh, Thang Luong, Gunjan Baid, Zirui Wang, Vijay Vasudevan, Alexander Ku, Yinfei Yang, Burcu Karagol Ayan, et al. Scaling autoregressive models for content- rich text-to-image generation. arXiv preprint arXiv:2206.1079,, 2(3):5, 2022.
Yichi Zhang, Yici Yan, Alex Schwing, and Zhizhen Zhao. Towards hierarchical rectified flow. arXiv preprint 2rXi 17436, 2025.
Qinqing Zheng, Matt Le, Neta Shaul, Yaron Lipman, Aditya Grover, and Ricky TQ Chen. Guided flows for generative modeling and decision making. arXiv preprint axXiv:2311.13443, 2023.

---

Published as a conference paper at ICLR 2026
A
QUALITATIVE RESULTS
DTM++ (ours)
DTM+ (ours)
DIM
FM-lognorm
AR-dis
MAR-dis
MAR
OO
Oil painting of giani robot made of sushri, holding chopsticks.
giant snake made salad
Portrail gecko wearing a train conductor's hat and holding Bag that has yang symbal on it. Chikd's crayon drawing.
Liger standing flowers
MIR LITTY
International Space
sunken submarine at the buttom of the ocean
belicopter National Park
giant uobra snake made from pancakes
A penguin wearing avintor goggles and Ilying confidently bemused eagle.
bloody mary
Figure 7: (Cont.) Samples comparing of our Best D-TM MLP (DTM++) and Transformer (DTM+) Models with DTM baseline, FM-lognormal, AR, MAR, AR-discrete, MAR-discrete as baselines. All models share similar architecture and training recipe.
12

---

Published as a conference paper at ICLR 2026
DTM++ (ours)
DTM+ tours)
DTM
FM-lognorm
AR-dis
MAR-dis
AR
MAR
drawing of space shuttle in the da Vinci
MA ALA
a horse reading newspaper
watercolor painting of snowy owl standing in grassy field
peeled
AHH
black hi-top sncakers
A tediyheaa skutehoard Times Square, doing tricks
cardboard hox ramp.
shape of a castle
baby daikon radish in tutu
paramoid android freaking jumping
hy egus
A photograph of the inside of subway train. There are frogs sitting on the seats.
One reading a newspaper.
background.
Figure 8: (Cont.) Samples comparing of our Best D-TM MLP (DTM++) and Transformer (DTM+) Models with DTM baseline, FM-lognormal, AR, MAR, AR-discrete, MAR-discrete as baselines. All models share similar architecture and training recipe.
13

---

Published as a conference paper at ICLR 2026
DTM++ (ours)
DTM+
DTM
FM-lognorm
AR-dis
MAR-dis
MAR
bloudy mary cocktail
whale breaching
whale
JOLA
BANF
700
BUW MAYA
redi
A photo a frog reading the newspuper namedd "Tunday" written on it. There is a frog printed the newspuper too.
photo of Japanese Porcelain Imari the ground below wooden chair.
an actopus
group of not than five meeriats standing with the sun setting behind them
looking window
coo est
a large open book showing text and an illustration of cal
Figure 9: (Cont.) Samples comparing of our Best D-TM MLP (DTM++) and Transformer (DTM+) Models with DTM baseline, FM-lognormal, AR, MAR, AR AR-discrete, MAR-discrete as baselines. All models share similar architecture and training recipe.
14

---

Published as a conference paper at ICLR 2026
B STOCHASTIC SAMPLING
We derive eq.
13 used in the D-TM stochastic sampling in algorithm consider arbitrary times 0 S a < b in the [0, 1] time interval, and a constant data sample. Let X be a sample from the conditional probability path qo time b, i.c.
X, (: bz (1 - b))
(14)
Now, we want to transform X a sample Xa from the conditional probability path at time a, i.e.,
Na ~ NN..Naaaa (1 - aa)).))
(15)
To that end note that
(: a21, [a(1 (a(1-b)
b) 2 1)
b Xo N
(16)
Now let 7 ~ N..100,07 be an independent Gaussian sample with >> 0 as degree of freedom. Therefore
(1 azı, [(at1))) + 02 (1.
(-az))
π Xb 2 gN
(17)
Lastly, we solve for such that
[a(1 - b) (a(1-b)
+ + (1 - aa²
(18)
leading to
(b (b- a)(a b 2ab)
(19)
b²
This coincides with eq.
13 we set t" and t'.
C
HEAD BATCH SIZE
The loss corresponding for head batch size kn (1, 4, 16, 64) takes the form
An
((() Ex,X Yo.
(Y,15)(())) -( -(G-Yaa))??
(20)
s/r
where 8; U(0 1) are random i.i.d,, Yo, ~ N(O, I), and Y.; =(1-s: V for 1,...,k h
15

---

Published as a conference paper at ICLR 2026
D ILLUSTRATIONS
Luata
Head
(b)
Figure 10: (a) Scaling the sequence length entering the head. (b) Scaling the head batch size.
16

---

Published as a conference paper at ICLR 2026
E
TIME WEIGHTING DISTRIBUTIONS
5
(etad 1.33
5,00
1.24)
nu/0, 3)
nuj-0.5.1
Protebility Densty
1
0.0
0.2
0.8
1.0
Figure 11: Visualization of the Beta and lognormal distributions used for time weighting.
F CLASSIFIER FREE GUIDANCE
We use Classifier Free Guidance (CFG) (Ho Salimans 2022). defined by a weight w and the corresponding transformation to the velocity (Zheng et al. 2023
T.je (-),e)) (1 uu) (4,c) ww (-1/2,0),
(21)
where the standard choice of w 6.5 and latent representation is learned with a condition C' = e.g., text prompts, that can also be empty, denoted by C' = 0,
1 fe. (X,, C)-
(22)
In the training loss (eq. 9 random prompt-image pair from the dataset, (C, X1), where with probability 0.15 we set
G FLOW MATCHING HEAD PARAMETERIZATION
Another design choice for the TM head is the target predicted by the head model used to sam- ple Y (i.e., the target in the FM loss eq. (9)) As known in flow matching and diffusion literature (Lipman al. 2024) can choose different targets to sample Y Y Prje(-IX, (and change the sampling in eq. (IO) accordingly). Flow matching lears the difference Yi Yo: the denoiser prediction learns Yi and noise pre- diction Yo. Figure 12 (b) shows comparison of these parameterizations for where in essence: the difference target Y1 Yo slightly better than denoiser Yi which is con- siderably better than noise prediction Yo. We set the difference Yi - Yo as in eq. (9) as our default choice.
OEEP:
3.0
0.40
0.33
X1 Xo
3.0
Figure 12: Performance of different flow matching targets.
17

---

I APPLYING TM STOCHASTIC SAMPLING TO FM Published as a conference paper at ICLR 2026
H Y SAMPLING RELATIONS
Calculating X given X, and Y can be done by writing the equations of the linear process (in eq. (3) for t and E' and solving for XŞ functions of Y and X; leading to instantiation of the sampling relation in eq. (5) for the Y choices in eq. [2)
Xy = x, -ty
with Y = =X1 -o (Difference)
(23)
X (1- x)x,, + (t' t)Y
with Y = X1
(Denoiser)
(24)
1-1
Xy I'X, + (1-1") -t)y
with Y = =Xo
(Noise)
(25)
As side contribution, found that applying algorithm where sampling with eq. (7) replaced with standard ODE solve in FM also leads to considerable gains in FM sampling, specifically for medium frequency 8 16 (for T 32 NFEs) and scale e 0.1, fig. [3 Note that this algorithm is similar to the restart algorithm in Xu et al. (2023b) with several key differences: we use if for flow matching rather than variance exploding denoiser EDM, we parameterize with two simple parameters, and we avoid the EDM discretization scheme. Lastly, note that for FM sampling this algorithm increases computational cost as it requires ODE solving during the sampling algorithm, this is in contrast to using it in TM which does not increase computational cost.
0:36
2:41
0.53
0.47
3:41
0:37
0:30
2.11
0.67
0:46
5:40
0:46
0.36
3.00
0.52
0.66
0.66
0.36
0.85
6.40
0.43
0.68
9.45
0.60
2.30
0.45
5.38
0:47
0.20
5:38
0.91
0.61
8:41
0.39
0:33
0:38
0.41
0:43
6,0 GU
5.37
0.40
0.30
5:33
0.31
0.20
0:35
0.33
Figure 13: Applying version of the TM stochastic sampling algorithm (algorithm I flow matching, where 7 is replaced with ODE sampling (and consequently requires more NFE), seems also improve FM generation ranking. Red colors indicate low ranking, while blue colors correspond to high ranking; colors are on consistent scale with table
18

---

Published as a conference paper at ICLR 2026
J RELATION OF Y PARAMETERIZATTON TO DIFFUSION AND FLOW MATCHING
Predicting Y X1 - Xy can seen as the TM version of flow matching (Shaul et al. 2025, which instead predicts the deterministic function [[[] Xol X; Øz}: Y X1 is the TM versions of a denoiser (Salimans & Ho. 2022) which predicts the function e = zzJ. while Y = Xo is the TM version woise-predictionn Ho al. (2020) which predicts the function E[Xp] X, = xz]. In contrast to flow matching or diffusion, TM learns to sample from the posterior X1 Xu, X1 or Xo directly rather than estimating their mean as done in flow matching and diffusion models. Note that similarly to the situation with denoiser and noise-prediction (see e.g.. (Lipman et al. (024)), also for Y-TM, the former has singularity near / and the latter near /= 0. The singularity near t = 0 becomes issue in sampling numerical instability is introduced at the beginning of the sampling rather at the end, providing potential explanation to the lower performance of the Xo parameterization.
19

---

Published as a conference paper at ICLR 2026
K D-TM SAMPLING PSEUDOCODE
For completeness we provide the standard D-TM sampling pseudocode for continuous time in algorithm 2
Algorithm 2 DTM Sampling adapted from Shaul al. (2025) for continuous time.
Require: Trained model (u, fP)
Require: Time grids to
tr and so 81 < 85 1.
I: Sample Xn ~ Ia)
2: for 0 T do
3: h; + ft. (Xi)
4: Sample Yn N(O, 1)
5: Y + sode_solveYYnn u le, (-1b,), (so.....ss)
6:
X,+1 + Y
7: end for
8: return XT
20

---

Published as a conference paper at ICLR 2026
L FULL RESULTS
Table 5: All ablations and evaluated models used in the paper.
21

---

Published as a conference paper at ICLR 2026
Table 6: (Cont.) All ablations and evaluated models used in the paper.
22


Reviews:
[{"review_id": "EUmhWBH6Is", "reviewer": "Reviewer_4riu", "summary": "This paper systematically investigates the design space of Transition Matching (TM) models for generative modeling, focusing on the time-continuous bidirectional variant and the architectural and algorithmic choices of the \"head\" module. The authors present a thorough empirical study spanning 56 different 1.7B parameter text-to-image models and 549 total evaluations, exploring factors such as head type (MLP, Convolution, Transformer), sequence scaling, batch size, time weighting, model parameterizations, and sampling strategies. The main findings are that TM with an MLP head, specific time weighting, and a high-frequency stochastic sampler gives top performance across broad metrics, while a Transformer head with sequence scaling rivals in aesthetic quality. The paper provides actionable recommendations for practitioners and positions TM as a state-of-the-art competitive approach among contemporary generative models.", "questions": "Please refer to Weaknesses", "limitations": "", "rating": 8, "confidence": 5, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "- Comprehensiveness: The paper offers an admirably thorough experimental exploration, running significant computational experiments (56 trained large-scale models, 549 evaluations) that are rarely matched in scope in generative modeling work.\n- Systematic Ablation: The design/practice space is cut along multiple axes—head type, head size, sequence scaling, batch, time weighting, parameterization, and samplers—granting nuanced insights into what factors matter for TM performance.\n- Solid Empirical Support: Results are benchmarked across four diverse prompt datasets and 25 evaluation metrics, aggregated and analyzed to prevent cherry-picking, which is evident in the detailed presentation in Table 4 and the ablation tables/figures.\n- Clear Positioning and Contributions: The roles of MLP vs. Transformer heads are well interrogated, and findings (such as sequence scaling’s effect in Transformer heads) are substantiated in both plots (see Figure 4) and tables, enabling actionable and evidence-based recommendations.\n- Efficiency-Quality Tradeoffs: Generated data (see Figures 1, 7, 8, 9) visually and quantitatively demonstrates not only state-of-the-art quality but also improved generation cost and inference speed, which is particularly valuable to practitioners.\n- Mathematical Clarity: Mathematical formulations (see Equations 3–13, especially the ODE-based head sampling) are overall sound, tying TM methodology to both flow/diffusion formalisms and exposing the degrees of freedom unique to TM (e.g., supervisory process, $Y$ parameterization).", "weaknesses": "- While the empirical exploration is outstanding, the theoretical explanation for why certain design changes—such as the specific benefit of token-wise MLP heads for text alignment, or the stochastic sampler’s effectiveness—remain largely empirical. There is limited grounding in theory or analysis for these effects, and at points, the paper admits uncertainty (\"It is not clear to the authors...\"), which limits the generalizability and explanatory power of reported findings.\n- While the paper competently implements and evaluates strong baselines, the connection to recent alternative methods—including energy-based, equilibrium, and posterior mean matching approaches—is absent; no experimental or discussion-based comparison is made, which hinders the completeness of the empirical evaluation and weakens claims to \"SOTA\".", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "hDwWJ5k4sI", "reviewer": "Reviewer_fGYX", "summary": "The paper considers the question of optimal design of a Transition Matching (TM) model. In particular, the paper focuses on the design of the head module in TM. The authors perform a thorough exploration of the various choices pertaining to the head such as the architecture, model size, sequence scaling etc., The paper also presents \"ideal\" choices for the hyperparameters/choices considered wrt to the head module. In addition, the authors show, through some theoretical and experimental justification, a family of stochastic samplers that provide improved performance to D-TM models.", "questions": "- Given that you have fixed the other parts of the TM (not related to the head module) fixed, would your findings change if any of those other components are chosen differently?", "limitations": "", "rating": 8, "confidence": 3, "soundness": 4, "presentation": 4, "contribution": 4, "strengths": "- The degree of thoroughness with which the authors designed the experiments and studied the various aspects of the head module is impressive and is presented clearly.\n- Since TM is a relatively new and under-explored direction, such a study shedding some light on designing performant TM models is significant.", "weaknesses": "- There is not much discussion on the reasons for the behaviors observed in the experiments. While, I understand that is probably not the focus of the paper, without some justified rationale, it is difficult to translate these findings when any of the assumptions or the control variables change.\n\n**Minor formatting/Grammar errors:**\n\nPlease note that following items did not affect my score. I understand errors tend to naturally creep up when preparing a manuscript and I am pointing that out merely to improve the quality of the draft.\n\n- \"TM\" is incorrectly included in the citation in Section 2 (just before equation 6).\n- Extraneous \"'s\" at the end of the first line of Section 3.2.\n- Minor Typo in Section 3.2: \"Figure 3 (b) and (c) show the **e**ffect of different heads ...\". Same spelling error is present in other places in the manuscript.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "aHYEXiAzlK", "reviewer": "Reviewer_gvxM", "summary": "Transition Matching (TM) is an emerging paradigm for generative modeling that generalizes diffusion, flow-matching, and continuous-state autoregressive models. TM gradually transforms noise into data samples, but it utilizes a second \"internal\" generative model to execute the transition steps, making these steps more expressive than in diffusion or flow models. The method employs a large backbone network and a smaller \"head\" module for efficiency. This paper presents a large-scale, systematic exploration of the TM design space, investigating critical components like network architecture, training objectives, and parameter configurations. The goal is to provide the community with in-depth insights and empirical guidance on how to efficiently design and deploy TM models.", "questions": "None", "limitations": "", "rating": 4, "confidence": 3, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "1. Extensive and Systematic Experimental Exploration: The authors conducted a large-scale, systematic study, thoroughly investigating numerous design choices (architecture, loss functions, hyperparameters) within the Transition Matching framework, offering valuable empirical knowledge for this new domain.\n\n2. Excellent Clarity and Writing: The paper is very well-written and clearly structured, articulating technical concepts and experimental findings effectively, which makes the core components and design trade-offs of this complex model paradigm easily digestible.", "weaknesses": "1. Limited Technical Novelty: The contribution leans more toward an empirical study that explores and summarizes existing design choices rather than introducing fundamental technical or algorithmic innovations.\n\n2. Restricted Evaluation Scope: The training and testing are exclusively conducted on low-resolution images (256x256), which significantly undermines the reliability and generalizability of the findings for real-world applications where high-fidelity, high-resolution image generation is currently a major focus.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "AK3kY0rk3g", "reviewer": "Reviewer_fSgD", "summary": "This paper presents a comprehensive study on design and training strategies for text-to-image generation models, focusing on how different head architectures, parameterizations, and scaling choices affect performance. The authors evaluate over 500 models across multiple datasets and 25 metrics, aggregating results into a unified ranking system. They find that simpler head architectures, such as MLP-based ones, can surprisingly outperform more complex transformer heads in text adherence, raising questions for future research. The study provides extensive ablation results and benchmarking insights relevant to improving model quality and efficiency in large-scale generative modeling.", "questions": "A significant drawback of the Transition Matching paradigm, as implemented and studied here, is the extraordinarily high inference cost. The paper's architecture requires the large 1.7B-parameter Backbone network to be called $T$ times (e.g., 32 times), and crucially, the Head network to be called $T \\times S$ times (e.g., $32 \\times 32 = 1024$ times) per generation. This places TM models at a major computational disadvantage compared to conventional diffusion or flow models, which only require $T$ evaluations.\n\nThe inference speed reported in the paper (e.g., **21.3 seconds per sample** for Model, DTM MLP) is excessively slow for practical application, making the framework currently intractable for real-time or high-throughput generation. While the paper provides an exhaustive empirical analysis of the design space, it is regrettable that the authors did not dedicate effort to mitigating this core issue. The lack of proposed architectural or algorithmic solutions (e.g., knowledge distillation, progressive sampling optimization, or alternative ODE solvers) to tackle this major computational bottleneck is a key limitation of this work. We believe addressing the high latency of the $T \\times S$ Head evaluations is the most pressing challenge for TM models.", "limitations": "", "rating": 4, "confidence": 5, "soundness": 2, "presentation": 3, "contribution": 1, "strengths": "- The primary contribution of this work is a large-scale, systematic exploration of the head module's design space within the Transition Matching (TM) framework. To my knowledge, this is the first paper to conduct such a comprehensive ablation study, systematically investigating the impact of head architecture (e.g., MLP, Transformer) , model size , sequence scaling , and batch size.\n\n- While the paper's strength is its comprehensive empirical analysis (training 56 unique 1.7B models) , it also introduces a novel stochastic sampling algorithm for TM, which is shown to significantly improve generative quality at no extra computational cost.\n\n- A key novel insight is the paper's clear and distinct analysis of the dual time-sampling parameters. It is the first to systematically decouple and study the effect of time weighting for both the backbone's transition time $t$ and the head's internal generative time $s$ during training. This is further complemented by an analysis of the corresponding $T$ and $S$ discretization steps at inference time.\n\n- The extensive experiments provide actionable guidelines and valuable insights for the community regarding which design choices are most likely to yield improvements in quality and efficiency for this promising class of models", "weaknesses": "The paper's main contribution is a large-scale, systematic empirical study of the TM head's design space. While this comprehensive ablation is valuable and provides actionable insights, the work is light on fundamental algorithmic innovation. The paper does not introduce a new generative paradigm but rather exhaustively explores the design choices of an existing one. Although a novel stochastic sampler is presented, the paper's core feels more like an extensive experimental report than a proposal of a new, innovative method.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "lBMFcUentd", "reviewer": "Reviewer_yyoT", "summary": "This paper presents a large-scale and systematic investigation into the design, training, and sampling of the head module in the continuous-time bidirectional variant of Transition Matching (TM) for generative modeling, with a focus on text-to-image generation. The work addresses a notable gap in existing literature— the lack of in-depth exploration of the head module’s architecture and hyperparameters— despite its crucial role in translating backbone latent representations into concrete transition outputs. By training 56 different 1.7B text-to-image models and conducting 549 unique evaluations, the authors provide comprehensive insights into how head design choices impact generation quality, training efficiency, and inference efficiency. The paper’s contributions, including the novel stochastic sampling algorithm and actionable guidelines for TM-based models, are valuable for advancing generative modeling research.", "questions": "Since the work focuses exclusively on text-to-image generation, how do you think your findings might generalize to other modalities like text-to-video or audio generation? Could you discuss potential adaptations for other modalities or highlight limitations specific to text-to-image?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 4, "presentation": 4, "contribution": 4, "strengths": "Rigorous Experimental Design\nThe authors ensure fair comparison across models and baselines by keeping the backbone architecture (a 1.7B parameter DiT transformer), training dataset (350M text-image pairs), and most training hyperparameters fixed. Evaluations are conducted on four datasets (MS-COCO, PartiPrompts, GenEval, T2ICompBench) using 25 metrics, which are aggregated into a single rank score— this comprehensive setup enhances the reliability and generalizability of the results.\n Comprehensive Exploration of Head Design Space\nThe paper systematically explores key design choices for the head module, including architecture type (MLP, Convolution, Transformer), size (x-small to x-large), sequence scaling, batch size, time weighting, and model parameterization (Y). This exploration reveals non-trivial findings, such as the observation that smaller head sizes can already deliver good performance without excessive computational cost, and that sequence scaling benefits Transformer heads (via attention-driven information sharing) but not MLP heads (which process tokens independently).", "weaknesses": "Lack of Analysis on Why MLP Heads Outperform Transformer Heads in Text Adherence\nThe paper notes that MLP heads (token-wise processing) achieve better text-adherence scores than more expressive Transformer heads, but does not provide a detailed analysis of the underlying reasons. Further investigation— such as analyzing token-level alignment between text prompts and generated images, or exploring how attention in Transformer heads might distract from text-image alignment— would strengthen the paper’s insights and open avenues for future research.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)