Scaling Laws for Diffusion Transformers
Authors: Zhengyang Liang
Hao He
Ceyuan Yang
Bo Dai
Venue: ICLR 2026 Poster (2026)
Keywords: Scaling laws
diffusion models
transformers
Primary area: foundation or frontier models, including LLMs
TL;DR: We demonstrate the existence of scaling laws for diffusion transformers.

Abstract: Diffusion transformers (DiT) have already achieved appealing synthesis and scaling properties in content recreation, \emph{e.g.,} image and video generation.

However, scaling laws of DiT are less explored, which usually offer precise predictions regarding optimal model size and data requirements given a specific compute budget.

Therefore, experiments across a broad range of compute budgets, from 1e17 to 6e18 FLOPs are conducted to confirm the existence of scaling laws in DiT \emph{for the first time}. Concretely, the loss of pretraining DiT also follows a power-law relationship with the involved compute.

Based on the scaling law, we can not only determine the optimal model size and required data but also accurately predict the text-to-image generation loss given a model with 1B parameters and a compute budget of 1.5e21 FLOPs.

Additionally, we also demonstrate that the trend of pretraining loss matches the generation performances (e.g., FID), even across various datasets, which complements the mapping from compute to synthesis quality and thus provides a predictable benchmark that assesses model performance and data quality at a reduced cost.

Published as a conference paper at ICLR 2026
SCALING LAWS FOR DIFFUSION TRANSFORMERS
Zhengyang Liangh. Hao He Ceyuan Yang
Bo Daj
l University Toronto
Vector Institute
Thee Chinese University of Hong Kong
"The University of Hong Kong
ABSTRACT
Diffusion transformers (DiT) have already achieved appealing synthesis and scal- ing properties in content recreation, e.g. image and video generation. However, scaling laws of DiT are less explored, which usually offer precise predictions re- garding optimal model size and data requirements given specific compute bud- get. Therefore, experiments across a broad range of compute budgets, from 1e17 to 6e18 8 FLOPs are conducted to confirm the existence of scaling laws in DiT for the first time. Concretely, the loss of pretraining DiT also follows power-law relationship with the involved compute. Based on the scaling law, determine the optimal model size and required data but also accurately predict the text-to-image generation loss given model with 1B parameters and compute budget of 1. 5e22 FLOPs. Additionally, we also demonstrate that the trend pretraining loss matches the generation performances (e.g., FID), even across var- ious datasets, which complements the mapping from compute to synthesis quality and thus provides predictable benchmark that assesses model performance and data quality at reduced cost.
not only
INTRODUCTION
Scaling laws in large language models (LLMs) (Kaplan et al. 2020 Hestness et al. [017 Henighan al. 2020 Hoffmann et al., 2022) have been widely observed and validated, suggesting that pre- training performance tollows power-law relationship with the compute C. The actual compute could be roughly calculated as 6ND,, where N the model size and D is the data quantity. Therefore, determining the scaling law helps make informed decisions about resource allocation to maximize computational efficiency, namely, figure out the optimal balance between model size and training data (i.e., the optimal model and data scale) given a fixed compute budget. However, scaling laws in diffusion models remain less explored.
The scalability has already been demonstrated in diffuuson models, especially for diffusion trans- formers (DiT). Specifically, several prior works (Mei et al.) 2024 Li et al. 2024) reveal that larger models always result in better visual quality and improved text- alignment. However, the scal- ing property of diffusion transformers is clearly observed but not accurately predicted. Besides, the absence of explicit scaling laws also hinders comprehensive understanding of how training budget relate to model size, data quantity, and loss. As result, we cannot determine accordingly the opti- mal model and data sizes for given compute budget and accurately predict training loss. Instead, heuristic configuration searches of models and data are required, which are costly and challenging to ensure optimal balance.
In this work, we characterize the scaling behavior of diffusion models for text-to-iman imaoe synthesis, resulting in the explicit scaling laws of DiT for the first time. To investigate the explicit relation- ship between pretraining loss and compute, wide range of compute budgets from 1e17 to 6e1 8 FLOPs are used. Models ranging from 1M to IB are pretrained under given compute budgets. As shown in Fig. a), for each compute budget, we can fit parabola and extract an optimal point that corresponds to the optimal model size and consumed data under that specific compute constraint. Using these optimal configurations, we derive scaling laws by fitting a power-law relationship be- tween compute budgets, model size, consumed data, and training loss. To evaluate the derived
"Equal Supervision. Corresponding Author.

---

Published as a conference paper at ICLR 2026
Loss vs Parameter
Optim Parameter vs Budget
Opti Tokens ve Budget
1.0e117
30c+17
5 62 Te
2:599e111
500+18
Optin Paramelee
Optinn Tokena
10"
Eds+11
Optin Parmn
Opan
10 T0
1.500e+21
1.800++21
10"
10"
10"
10"
10
10"
10" 10"
10"
10"
10"
Paramater (a)
Budget (FLOPs)
Budget (FLOPs)
(b)
(c)
Figure 1: IsoFLOP and Model/Data Scaling Curves. For each training budget, we train multiple models of varying sizes. A parabola is fitted to the loss for each training budget, and the minimum point on each parabola (represented by the purple dots) corresponds to the optimal allocation of model size and data for that specific budget. By identifying the model and data sizes at these optimal points, we can plot the scaling trends of model parameters, tokens, and training budgets. The power- law curves shown allow us to predict the optimal configurations for larger compute budgets, such as 1. 5e21 FLOPs.
scaling laws, we extrapolate the compute budget to 1. 5e21 FLOPs that results in the compute- optimal model size (approximately IB parameters) and the corresponding data size. Therefore, 1B- 1B-parameter model is trained under this budget and the final loss matches our prediction, demon- strating the effectiveness and accuracy of our scaling laws.
To make the best use of the scaling laws, we demonstrate that the generation performances (e.g., FID (Fréchet Inception Distance)) also match the trend of pretraining loss. Namely, the synthesis quality also follows the power-law relationship with the compute budget, making it predictable. More importantly, this observation is transferable across various datasets. We conduct additional experiments on the COCO validation set (Lin et al.) 2014), and the same scaling patterns hold, even when tested on out-of-domain data. Accordingly, scaling laws could serve as a predictable benchmark where computational cost, enabling efficient evaluation and optimization of the model training process.
assess the quality of both models and datasets at significantly reduced
To summarize, We at first confirm the presence of scaling laws in diffusion transformers during training, revealing a clear power-law relationship between compute budget and training losses. Next, establish connection between pretraining loss and synthesis evaluation metries. Finally, We conducted preliminary experiments that demonstrate the potential of using scaling laws to evaluate both data and model performance. By conducting scaling experiments relatively low cost, we can assess and validate the effectiveness of different configurations based on the fitted power-law coefficients.
2 RELATED WORK
Scaling Laws Scaling laws (Hestness al. 2017) have been fundamental in understanding the performance of neural networks as they scale in size and data. This concept has been validated
several large pretraining models (Dubey al. 2024 Bi et al. 2024 Achiam et al. 2023).
Kaplan et al. (2020): Henighan et al.] (2020) were the first to formalize scaling laws in language models and extend them to autoregressive generative models, demonstrating that model performance scales predictably with increases in model size and dataset quantity. [Hoffmnn et al. (2022) further highlighted the importance of balancing model size and dataset size achieve optimal performance.
In the context of diffusion models, prior works (Mei et al.) [024] Li et al.] 2024) have empirically demonstrated their scaling properties, showing that larger compute budgets generally result in bet- ter models. These studies also compared the scaling behavior of various model architectures and explored sampling efficiency. Hu et al.] explains the properties of diffusion transformers from a statistical perspective, where the approximation and generalization theory respectively support the scalability of diffusion transformers in terms of model and data. However, no previous works pro- vide explicit formulation of scaling laws for diffusion transformers capture the relationship
2

---

Published as a conference paper at ICLR 2026
Validation Loss vs Budget
Offset VLB vs Budget
Offset Likelhood vs Budget
Treining Loss Budget
0.84
1.55
0.5
0.84
Valection Loss
0.82
1.50
Libelhoood
0.4
Trakiing LOGS
0.02
0.3
0.80
0.78
0.76
0.1
0.78
0.74
1.35
0.0
1.30
10
20 Budget (FLOPs)
10"
10 Budget (FLOPs)
10
10th Budget (FLOPs)
10
10
Budget (FLOPS) (4)
(a)
(b)
(c)
Figure 2: Scaling Curves for Different Metries. We present the scaling curves for training and validation loss, offset VLB, and offset likelihood. Validation metries are evaluated on the COCO 2014 validation set. The metrics display consistent trends and similar shapes, all adhering to power-law relationship. This demonstrates that each of these metries can be used to observe scaling laws effectively. For simplicity, we primarily focus training loss in subsequent analyses.
between compute budget, model size, data, and loss. In this paper, we aim to address this gap by systematically investigating the scaling behavior of diffusion transformers (DiTs), offering a more comprehensive understanding of their scaling properties.
3 METHOD
3.1 BASIC SETTINGS
Unless otherwise stated, all experiments in this paper adhere to the same basic settings. While the training techniques and strategies employed may be optimal, they primarily affect the scaling coefficients rather than the scaling trends. In this section, we outline the critical settings used in our experiments. Additional details are provided in Appendix
Diffusion Formulation All experiments are conducted using the Rectified Flow (RF) formulation (Liu et al. 2022 2022 Lipman et al. [022 Albergo al. 2023) with v-prediction. For timestep sampling, we adopt the Logit-Normal (LN) Sampling scheduler Tin (4,m, s), as proposed in Esser et al. (2024). A detailed ablation study of this choice can be found in Appendix H.1
Models & Dataset As observed in Kaplan et al. (2020), model design has limited influence on scaling behavior unless the architecture is extremely shallow or narrow. We adopt vanilla trans- former architecture (Vaswani 2017) with minimal modifications. Input lakens-comprisingg text, concatenated following in-context conditioning (Peebles & Xie, 2023). Our dataset consists of 108 million image-text pairs randomly sampled from Laion- Aesthetic (Schuhmann et al. 2022), and re-captioned using LLAVA 1.5 (Liu et al.. 2024). A val- idation set of million samples is drawn from the same subset. Further details are provided in Appendix E. and E.2 In most our experiments, each data point is seen only once during train- ing, which is consistent with the data-infinite setting commonly adopted in the industry. However, scaling laws do not rely on this assumption. To demonstrate this, we also evaluate scaling behav- ior under a data-constrained setting using ImageNet (Deng et al., 2009), with results presented in Appendix
image, and timestep embeddings-
3.2 SCALING METRICS
A natural question arises when investigating scaling laws during training: What metrics should be selected to observe scaling behavior? the context of Large Language Models (LLMs), the standard approach is autoregressive training (Radford) 2018 Radford et al. 2019), where the model trained to predict the next token in a sequence, directly optimizing the likelihood of the training data. This has proven to be reliable method for measuring model performance the compute budget scales up. Inspired by this approach, we extend the concept of scaling laws to diffusion models, using loss and likelihood as our key metrics.
3

---

Published as a conference paper at ICLR 2026
3.2.1 LOSS
Loss is the primary metric chosen observe scaling behavior during training. Unlike autoregressive models, diffusion models do not directly optimize likelihood. Instead, the objective is to match tine-conditioned velocity field (Ma al.) 2024b] Liu et al.) [022] Lipman et al. 2022
Albergo
et al. 2023 Specifically, the velocity v 1) at timestep is defined as:
v t) = x a/2o + 8/e.
(1)
where Zi represents the original data and denotes the noise. Here, the prime symbol indicatess the derivative with respect time l. In the rectified flow framework, the coefficients og and 3, are defined as at - t, Rr = t. Thus, the velocity v can be further simplified as:
v(z,, t) = -0 + c.
(2)
The corresponding loss function is expressed terms of the expected value:
(())= = Exp PPam (=((((().....)) ((..)
[[o(xe, 1) + xo =c(
(3)
[ x E: =il1?
(4)
N
The training loss is estimated using Monte Carlo method, which involves timesteps and noise sampling. In Eq. 4 denotes the number of training samples the mini-batch, t; and e; are the timestep and noise for constructing the i-th sample. The stochasticity inherent in this process can cause significant fluctuations, which are mitigated by employing larger batch size of 1024 and applying Exponential Moving Average (EMA) smoothing on the loss value. In experiments, set OEMA = 0.9, which is found to produce stable results. detailed ablation study on the choice of loss EMA coefficients is provided in Appendix H.2 This smoothing procedure helps reduce the variance loss value and provides clearer insights into training dynamics.
In addition to the training loss, validation loss is also computed the COCO 2014 dataset (Lin et al. (014). To ensure consistency with the training loss, timesteps are sampled using the LN timestep sampler Tin (I; m, 8), and evaluation is performed 10,000 data points, with 1,000 timesteps sampled per point.
3.2.2 LIKELIHOOD
Likelihood is our secondary metric. The likelihood over the dataset distribution Pp given model parameters represented as Eg- [po(r)), which can be challenging compute directly. In this paper, we measure likelihood using two different methods. The first method is based on the Variational Autoencoder (VAE) framework (Kingma et al., 2021 Song al. 2021 Vahdat al., 2021), which approximates the lower bound of likeliodd using the Variational Lower Bound (VLB). Since the VAE component in our experiments fixed to Stable Diffusion 1.5, terms related to the VAE remain constant and are ignored in our computation, they do not affect the scaling behavior. The second method uses Neural Ordinary Differential Equations (ODEs) (Chen et al,, 2018 Grathwohl et al., 2018), enabling the computation of exact likelihood through reverse-time sampling.
Variational Lower Bound (VLB) We estimate the VLB following the approach in Kingma et al. (2021), where it is treated as a reweighted version of the validation loss. The process starts by con- verting the estimated velocity into a corresponding estimate of Tg. after which the loss is computed based the difference between the estimated and the original sample. To obtain the VLB, this loss is further reweighted with the weighting coefficient being the derivative signal-to-noise ratio of noisy samples with respect to time t, i.e., SNR (t). More details can be found in Appendix F All models are evaluated on the COCO 2014 validation set using 10,000 data pairs. For each data point, 1,000 timesteps are sampled to ensure accurate estimations of the VLB.
Exact Likelihood The exact likelihood is computed using reverse- time sampling, where a clean sample is transformed into Gaussian noise. The accumulated likelihood transition is calculated using
4

---

Published as a conference paper at ICLR 2026
the instantaneous change of variables theorem (Chen al. 2018):
/'" So( 1) dl,
log (x) log (o(T))
(5)
where fo represents the model's output, and log po (xT) is the log density of the final Gaussian noise. The reverse process evolves from O (data points) to T (noise). The reverse sampling is performed over 500 steps using the Euler method.
Following model training, we evaluate models on the validation set to assess their compute-optima performance. As illustrated in Fig. 2 all four metrics (training loss, validation loss, offset VI.B, and offiset exact likelihood exhibit similar trends and shapes the training budget increases, showing their utility in observing scaling laws. All metrics for all models are computed using the evaluation
protocol described in Sec. 3.3 These consistent patterns suggest that any of these metrics can be effectively used to monitor scaling behavior. However, to simplify our experimental workflow, we prioritize training loss as the primary metric. Training loss be observed directly during the training process, without the need for additional evaluation steps, making it a more practical choice for tracking scaling laws in real-time.
3.3 SCALING LAWS IN TRAINING DIFFUSION TRANSFORMERS
Scaling Laws In this section, we investigate the scaling laws governing diffusion transformers, which describe the relationships between several key quantities: the objective function, model pa- rameters, tokens, and compute budget. The objective measures discrepancy between the data and model's predictions. The number of parameters N reflects model's capacity, while tokens D denote the total amount of data (in tokens) processed during training. Compute budget C' (we also refer to C as compute), typically measured in Floating Point Operations (FLOPs), quantifies the total computational resources consumed. In our experiments, the relationship between compute, tokens, and model size is formalized as C = GND, directly linking the number of parameters and tokens the overall compute budget. We estimate C' by explicitly counting the floating-poing operations required by the transformer blocks of our diffusion transformer. Focusing on these blocks, which dominate the total cost, and accounting for both forward and backward passes, we find that, on av- erage, processing single token requires approximately 6N FL.OPs. Hence, training a model with N parameters on D tokens consumes
C = 6ND,
and in our setting, the compute budget, number of parameters, and tokens approximately satisfy this relationship. More details about FL.OPs computation can be found Appendix G and See. 2.1 in Kaplan et al. (2020).
For each compute budget C, we extract the corresponding compute-optimal parameter count (Cp(CC) and token count Dop (C) from configuration at the minimum point (purple marker) of each isoFLOP parabola in Fig. [aa. When we plot these optimal points on log-log axes, both log NNp and log Dept vary approximately linearly with log C. indicating relationship between compute and the optimal allocation of parameters and data. Building on this, hypothesize that power law equations can effectively capture the scaling relationships between these quantities. Specifically, we represent the optimal model size and token count as functions of the compute budget as follows:
underlying power-law
Nop X C and Deps x Co.
where Nopt and Dop denote the optimal number of parameters and tokens for a given compute budget C, with a and b as scaling exponents that describe how these quantities grow with compute.
To empirically verify these scaling relationships, following Approach 2 in Hoffmann et al. 2002) we plot the isoFLOP figure to explore scaling laws. We select compute budgets [1e17, 7, 3e1 7, 6e17, 1e18, 3e18, 6e18]. We change the In-context Transformers from 2 layers to 15 layers. For all experiments, we use AdamW (Kingma) 2014; Loshchilov 2017) as the default optimizer with constant learning rate of 1c-4. Following several common practice from large-scale models Wortsman et al. (2023): Team (2024); Molybog et al.] (2023), we use a weight decay of 0.01, an epsilon value of 1e-15 5, and betas [0.9, 0.95]. For all experiments, we employ batch size of
5

---

Published as a conference paper at ICLR 2026
FID vs Budget
Loss vs Budget
Fitee
Filted curve
0.9
Model Performance
10a
0.8
Loss
0.7
10a
10t8
10th
1020
10a:
10 10"
10"
10
10"
10"
10"
10"
Budget (FLOPs)
Budget (FLOPs)
Figure 3: Performance Scaling Curves. The plot shows the relationship between training loss, FID, and compute budget, with both axes displayed on a logarithmic scale. The orange lines represent the fitted power-law curves for both metrics across various budgets, while the purple dots mark the performance of compute-optimal models at smaller budgets. both cases, the blue pentagons indicate the predicted performance at budget of 1. 1.5e21 FLOPs, demonstrating highly accurate predictions for large-scale models.
1024 and apply gradient clipping with a threshold of 1.0. To enable classifier-free guidance (Ho & Salimans 2022), we randomly drop the label with a probability of 0.1. During training, we use mixed precision with the BF16 data format. As is standard in diffusion models, we also maintain an EMA version of the model with decay coefficient of 0.99, which is saved for evaluation. The scaling curves of the EMA model are presented in Appendix H.4
For each budget, we fit parabola to the resulting performance curve, illustrated in the Fig. aa), to identify the optimal loss, model size, and data allocation (highlighted by the purple dots). By collecting the optimal values from different budgets and fitting them to power law, we establish relationships between the optimal loss, model size, data, and compute budgets.
As shown in the Fig. [a), except for the 1e17 budget, the parabolic fits align closely with the empirical results. This analysis confirms that the optimal number of parameters and tokens scale with the compute budget according to the following expressions:
Nops 0.0009 50.5681
(8) In this way, we establish the relationship between compute budget and model/data size. And from the fitted scaling curves, we observe that the ratio between the scaling exponent for data and the scal- ing exponent for model size 0.4319/ 0.4319/0.5661.. 00.5681. This indicates that, under our specific settings. both the model and data sizes need to increase in tandem as the training budget increases. However, the model size grows at a slightly faster rate compared the data size, as reflected by the proportional relationship between the two exponents.
Dapt 186.8535 0..339
Additionally, in Fig. 3b), we fit the relationship between the compute budget and loss, which follows the equation:
L - 2.3943 C C-000273
To validate the accuracy of these fitted curves, we calculate the optimal model size (958.3M pa- rameters) and token count for a compute budget of 1. 5e21. A model then trained with these specifications to compare its training loss with the predicted value. As demonstrated in Fig. bb), this model's training loss closely matches the predicted loss, further confirming the validity of our scaling laws.
3.4 PREDICTING GENERATION PERFORMANCE
Evaluating generative models has long been a challenge, as many commonly used metrics have been criticized for their limitations. However, recent work (Esser et al. 2024 Fan al. 2025) large-

---

Published as a conference paper at ICLR 2026
scale foundation generative models has shown that metrics such as FID, GenEval, and loss, despite their imperfections, correlate well with human perception and preference widely considered the gold standard in image generation evaluation. Notably, EEseer et al. (2024) reports that the diffusion transformer's loss serves as strong predictor overall model quality. In previous sections, we have analyzed the scaling behavior of loss. Here, extend our analysis to additional evaluation metrics. Specifically, we present scaling curves for FID, GenEval, and human preferences. Due to space constraints, the GenEval and Human preferences results are included in Appendix H0.10 and H.I1 These results confirm that generation quality also scales predictably with compute, following a power-law trend. We also provide some visual samples generated by compute-optimal models in Appendix H.8
We evaluate generation quality using multiple complementary metrics: FID, GenEval, and human preference reward models (HPSv2.1 (Wu et al. 2023) and ImageReward (Xu
Evaluation Metries
ct al. 2023)). FID quantifies the distance between the distributions of generated and real images in feature space, with lower values indicating higher fidelity. Following Sauer et al. (02T)).
compute FID using CLIP features (ViT-L/14 Dosovitskiy et al.] (2020)) instead of the traditional Inception features. GenEval captures the capability of instance-level compositionality by employing object detectors to provide fine-grained analyses, such as color binding and object counting. Finally, human preference reward models (HPSv2.1 and ImageReward) approximate human judgments by scoring generations according to preference signals.
Scaling laws for FID Predictions In this section, we provide analysis for FID. We reveal that the relationship between FID and the training compute budget follows a clear power-law trend, as shown in Fig. 3b). The relationship is captured by the following equation:
FID = 2.2566 x 10 CDC-C-0.20 C-0.234
(10)
where C' is the training compute budget. The purple dots in the figure represent the FID scores of compute-optimal models at various budgets, and the orange line represents the fitted power-law curve. Notably, the prediction for FID at a large budget of 1.5e21 FLOPs (blue pentagon) is highly accurate, confirming the reliability of scaling laws in forecasting model performance even at larger scales. As the compute budget increases, FID values decrease consistently, demonstrating that scaling laws can effectively model and predict the quality of generated images as resources grow.
3.5 SCALING LAWS FOR OUT-OF- DATA
Scaling laws remain valid even when models are tested on out-of-domain datasets. To demonstrate this, we conduct validation experiments the COCO 2014 validation set (Lin et al., 2014), using models that were trained on the Laion5B subset. In these experiments, we evaluate four key metries: validation loss, Variational Lower Bound (VLB), exact likelihood, and Fréchet Inception Distance (FID). Each metric is tested on 10,000 data points to examine the transferability of the scaling laws across datasets. Additional experiments on other datasets (Flickr30k (Plummer et al. 2016 JourneDBB (Sun et al. 2024)) are presented in Appendix H.7 H.7
The results, shown in Fig. 4 reveal two key observations:
Consistent Trends: Across all four metrics (validation loss, VLB, exact likelihood, and FID), the trends are consistent between the Laion5B subset and the COCO validation dataset. As the training budget increases, model performance improves in both cases, indicating that scaling laws generalize effectively across datasets, regardless of domain differences.
Vertical Offsset: There is a clear vertical offset between the metrics for the Laion5B subset and the COCO validation dataset, with consistently higher metric values observed on the COCO dataset. This suggests that while scaling laws hold, the absolute performance is influenced by dataset-specific characteristics, such as complexity or distribution. For met- ries like validation loss, VLB, and exact likelihood, this offset remains relatively constant across different training budgets. The gap between the FID values for the LaionSB subset and the COCO validation set widens as the training budget increases. However, the lationship between FID and budget on the COCO validation set still follows a plwer-law trend. This suggests that even out-of-domain data, the FID-budget relationship can be
7

---

Published as a conference paper at ICLR 2026
Loss vs Budget
VLB vs Budget
0.900
lbionSb subset
laionSb subset
0.875
CoCO vol
COCO vol
1.60
Validetion Los
0.850
Offect VLB
1.55
0.825
1.50
0.800
1.45
0.775
0.750
1.35
0.725
1.30
10ar
10
10a
1010
(e) Budget
(b) Budget
Exact NLL vs Budget
F10 vs Budget
0.8
laion subset
laion5b subset
200
coco val
CoCo val
Offset Exect NLL
0.6
180
160
0.4
140
0.2
120
0.0
100
10a7
10t4 (c) Budget
10
1039
(c) Sudget
Figure 4: Scaling laws for OOD data. The evaluation of loss, VLB, and NLL are conducted on an out-of-domain dataset COCO 2014 validation set. All metrics decrease monotonically as the budget increases and they share a similar shape. However, notable shift can be observed. Models have worse performance on the COCO validation set since they are trained on a different data distribution. The shift reflects the differences between datasets.
reliably modeled using a power law, allowing us to predict the model's FID for a given budget.
In summary, these results demonstrate that scaling laws are robust and can be applied effectively to out-of-domain datasets, maintaining consistent trends while accounting for dataset-specific perfor- mance differences. Despite the vertical offset in absolute performance, particularly in metrics like FID, the power-law relationships remain intact, allowing for reliable predictions of model perfor- mance under varying budgets. These findings highlight the potential of scaling laws as a versatile tool for understanding model behavior across datasets. The ability to project model efficiency and performance onto new data domains underscores the broader applicability of scaling laws in real- world scenarios.
4 SCALING LAWS AS A PREDICTABLE SCALABILITY BENCHMARK
Scaling laws offer a robust framework for evaluating the design quality of both models and datasets, particularly with respect to their scalability. Previous work such as Dubey et al. (2024); Bi et al. (2024) have all explored the scaling laws in data mix and quality. By modifying either the model architecture or the data pipeline, isoFLOP curves can be generated at smaller compute budgets to assess the impact of these changes. Specifically, after making adjustments to the model dataset, experiments can be conducted across a range of smaller compute budgets, and the relationship be- tween compute and metrics such as loss, parameter count, or token count can be fitted. The effec- tiveness of these modifications can then be evaluated by analyzing the exponents derived from the power-law fits. Scaling laws provide predictive framework for estimating model's performance when scaled in both data and model size, offering a robust benchmark for evaluating the scalability of data and model designs.
Our evaluation follows three key principles:
Model improvements: With a fixed dataset, a more efficient model will exhibit a lower model scaling exponent and a higher data scaling exponent. This suggests that the model
8

---

Published as a conference paper at ICLR 2026
Vanilla In-Context vs Cross-Attention
1
Vanilla In-Context Transformer
0.9
Cross-Attention Transformer
0.8
0.7
Loss
0.6
0.5
0.4
101s
1016
10
10
1019
1020
1021
Budget (FLOPs)
Figure 5: Scaling curves for In-Context and Cross-Attention Transformers. The plot compares the scaling behavior of In-Context Transformers and Pross-AAtention Transformers with respect to compute budget (FLOPs). In-context transformers, which concatenate image, text, and time tokens, show a more gradual decline in loss compared to Cross-Attention Transformers, which inject time and text tokens via AdaLN and cross-attention blocks. The steeper slope Cross-Attention Trans- formers indicates more efficient performance improvement with increasing compute, meaning they achieve lower loss with the same budget. These results highlight the efficiency Cross-Attentio Transformers and illustrate the potential of scaling laws in predicting performance trends across dif- ferent model architectures.
can more effectively utilize the available data, allowing for greater focus increasing the dataset size with additional compute resources.
Data improvements: When the model is fixed, higher-quality dataset will result in lower data scaling exponent and a higher model scaling exponent. This implies that better dataset enables the model to scale more efficiently, yielding superior results with fewer resources.
Loss/FID improvements: Across both model and data modifications, an improved train- ing pipeline reflected in a smaller loss/FII scaling exponent relative to compute. This indicates that the model achieves better performance with less compute, signaling overall gains in training efficiency.
To illustrate the utility of scaling laws as a predictive benchmark for scalability, we compare two spe- eific transformer designs: (1) Vanilla In-Context Transformers, which concatenate image and visual tokens as input and utilize only standard self-attention blocks, and (2) Cross-Attention Transform- ers, where conditioning is incorporated through cross-attention mechanisms. We train and evaluate both models with broad range of compute budgets, spanning lc17, 3c17, 6c17, 1e18, 3e18, 6c18, and lc19.
The scaling trends for both models are clearly illustrated in Fig. The loss scaling curves demon- strate that as the compute budget increases, the performance of both models improves, but differ- ent rates. The Aooss-Attention Transformer shows steeper decline in loss compared to the Vanilla In- nn-Context Transformer, indicating that when scaled up. achieves better performance with the same amount of compute.
The fitted scaling curves, as summarized in Tab. 1 support this observation. The Cross-Attention Transformer exhibits smaller model exponent, meaning that as compute budgets increase, more resources should be allocated toward scaling the dataset. Additionally, the smaller loss exponent of the Cross-Attention Transformer suggests more rapid decline in loss, indicating that this model achieves superior performance compared to the Vanilla In-Context Transformer. These findings align with the conclusions Peebles & Xie (2023). However, this observation should not be gener- alized to conclude that the In- In-Context mechanism inherently superior to works, such Flux and the MMDiT architecture in SD3 Esser et (2024), have demonstrated that
Recent

---

Published as a conference paper at ICLR 2026
models utilizzig In-Context Conditioning outperform those relying Cross-Attention. Our analysis does not aim to compare these mechanisms. Instead, the scaling laws benchmark serves a tool for assessing model scalability within given architecture. The method evaluates how efficiently a model benefits from increased data and computational resources as it scales. For com- pleteness, we also provide data quality comparison in Appendix H.6
Table 1: Exponents of model, data, and loss for different model architectures.
Model
Model Exposent
Data Expeneet
Loss Exponent
Vanilla In conteett
0.56
0.43
-0.0273
Cross-Attenton
-0.0385
This example illustrates how scaling laws serve reliable and predictable scalability bench- mark for evaluating the effectiveness of both model architectures and datasets. By analyzing the scaling exponents,
draw meaningful conclusions about the potential and efficiency of differ- ent design choices in model and data pipelines.
5 CONCLUSION
In this work, we explored the scaling laws of Diffusion Transformers (DiT) across a broad range of compute budgets, from 1e17 to 6e18 FLOPs,, and confirmed the existence of power-law relationship between pretraining loss and compute. This relationship enables precise predictions of optimal model size, data requirements, and model performance, even for large-scale budgets such as 1e21 FLOPs. Furthermore, we demonstrated the robustness of these scaling laws across different datasets, illustrating their generalizability beyond specific data distributions. In terms of generation performance, we showed that training budgets can be used to predict the visual quality of generated images, as measured by metrics like FID. Additionally, by testing both In-context Transformers and Cross-Attention Transformers, benchmark for evaluating and optimizing both model and data design, providing valuable guidance for future developments in text-to- text-to-image generation using DiT.
validated the potential of scaling laws to serve predictable
ACKNOWLEDGMENTS
We would love thank Tao Lu, Tianyi Lu for constructive feedback and help during the project. The project is supported in part by HKU Startup Fund and Institute of Data Science.
REFERENCES
Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Ale- man, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, al. Gpt-4 technical report. arXiv preprint arXiv:2303. 08774, 2023.
Michael S Albergo, Nicholas M Boffi, and Eric Vanden-Eijnden Stochastic interpolants: A unifying framework for flows and diffusions. arXiv preprint arXiv:2303.0879) 2023.
Fan Bao, Shen Nic, Kaiwen Xue. Yue Cao, Chongxuan Li, Hang Su, and Jun Zhu. All are worth words: A vit backbone for diffusion models. In Proceedings of the IEEE/CVE conference on computer vision and pattern recognition, pp. 28669, 2023.
Xiao Bi, Deli Chen, Guanting Chen, Shanhuang Chen, Damai Dai, Chenggi Deng, Honghui Ding, Kai Dong, Qiushi Du, Zhe Fu, et al. Deepseek llm: Scaling open-source language models with longtermism. arXiv preprint 2rXhv:2401.02954 2024.
Junsong Chen, Jincheng Yu, Chongjian Ge. Lewei Yao, Enze Xie, Yue Wu, Zhongdao Wang, James Kwok, Ping Luo, Huchuan Lu, et al. Pixart-a: Fast training of diffusion transformer for photore- alistic text-to-image synthesis. arXiv preprint arXiv:2310.0044 2023.
Ricky TQ Chen, Yulia Rubanova, Jesse Bettencourt, and David K Duvenaud. Neural ordinary differential equations. Advances in neural information processing systems, 31, 2018.
10

---

Published as a conference paper at ICLR 2026
Mostafa Dehghani, Josip Djolonga, Basil Mustafa, Piotr Padlewski, Jonathan Heek, Justin Gilmer, Andreas Peter Steiner. Mathilde Caron, Robert Geirhos, Ibrahim Alabdulmohsin, et al. Scaling vision transformers to 22 billion parameters. In International Conference on Machine Learning, pp. 7480-7512. PMLR, 2023.
Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale scale hi- erarchical image database. In 2009 IEEE conference on computer vision and pattern recognition, pp. 248-255. leee, 2009.
Jacob Devlin. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv: (at::8110000000 2018.
Prafulla Dhariwal and Alexander Nichol. Diffusion models beat gans image synthesis. Advances in neural information processing systems, 34:8780-877 0094,,994222 2021.
Laurent Dinh, Jascha Sohl-Dickstein, and Samy Bengio. Density estimation using real nvp. arXiv 2016.
preprint arXiv:
Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, Jakob Uszko- reit, and Neil Houlsby. An image worth 16x words: Transformers for image recognition at scale. arXiv preprint arXiv:2010. 1929, 2020.
Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, et al. The llama 3 herd of models. arXiv preprint arXiv:2407.21780 2024.
Patrick Esser, Sumith Kulal, Andreas Blattmann, Rahim Entezari, Jonas Müller, Harry Saini, Yam Levi, Dominik Lorenz, Axel Sauer, Frederic Boesel, et al. Scaling rectified flow transformers for high-resolution image synthesis. In Forty-first International Conference on Machine Learning, 2024.
Lijie Fan, Tianhong Li, Siyang Qin, Yuanzhen Li, Chen Sun, Michael Rubinstein, Deqing Sun, Kaiming He, and Yonglong Tian. Fluid: Scaling autoregressive text-to-image generative models with continuous tokens. In The Thirteenth International Conference on Learning Representations, 2025. htt.ltts::///cm http:/:://///eeeeeee.ee.e net/ www.:eeee//oommmoemmo
1VAV
Dhruba Ghosh, Hannaneh Hajishirzi, and Ludwig Schmidt. Geneval: An object-focused framework for evaluating -ext-to-image alignment. In Thirtt-seventh Conference on Neural Information Pro- cessing Systems Datasets and Benchmarks Track, 2023. URL httttst:t//ppeeeeeeeee
/
Eorum?id= 5v3331
Will Grathwohl, Ricky TQ Chen, Jesse Bettencourt, Ilya Sutskever, and David Duvenaud. Ffjord: Free-form continuous dynamics for scalable reversible generative models. arXiv preprint arXiv:1810.01367. 2018.
Kaiming He, Xinlei Chen, Saining Xie, Yanghao Li, Piotr Dollár, and Ross Girshick. Masked au- toencoders are scalable vision learners. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 16000-16009, 2022.
Tom Henighan, Jared Kaplan, Mor Katz, Mark Chen, Christopher Hesse, Jacob Jackson, Heewoo Jun, Tom B Brown, Prafulla Dhariwal, Scott Gray, et al. Scaling laws for autoregressive generative modeling. arXiv preprint arXiv:2010. /4701, 2020.
Joel Hestness, Sharan Narang, Newsha Ardalani, Gregory Diamos, Heewoo Jun, Hassan Kianinejad, Md Mostofa Ali Patwary, Yang Yang, and Yanqi Zhou. Deep learning scaling is predictable, empirically, arXiv preprint arXiv: 1712. 20409, 2017.
Jonathan Ho and Tim Salimans. arXiv:2207. 12598, 2022.
Classifier-free diffusion guidance.
arXiv preprint
Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. Advances in neural information processing systems, 33:6840-6851, 2020.

---

Published as a conference paper at ICLR 2026
Jordan Hoffmann, Sebastian Borgeaud, Arthur Mensch, Elena Buchatskaya, Trevor Cai, Eliza Rutherford, Diego de Las Casas, Lisa Anne Hendricks, Johannes Welbl, Aidan Clark, et al. Train- ing compute-optimal large language models. arXiv preprint arXiv:2203.15556, 2022.
Jerry Yao-Chich Hu, Weimin Wu, Zhuoru Li, Sophia Pi, Zhao Song, and Han Liu. On statistical rates and provably efficient criteria of latent diffusion transformers (dits). In The Thirty-eighth Anmual Conference on Neural Information Processing Systems.
Jared Kaplan, Sam McCandlish, Tom Henighan, Tom B Brown, Benjamin Chess, Rewon Child, Scott Gray, Alec Radford, Jeffrey Wu, and Dario Amodei. Scaling laws for neural language models. arXiv preprint arXiv:2001.0836/ 2020.
Diederik Kingma, Tim Salimans, Ben Poole, and Jonathan Ho. Variational diffusion models. Ad- vances in neural information processing systems, 34:21696-21707 2021.
Diederik Kingma. Auto-encoding variational bayes. arXiv preprint arXiv: 2013.
Diederik Kingma. Adam: A method for stochastic optimization. arXiv preprint arXiv: 1412.6980, 2014.
Durk P Kingma and Prafulla Dhariwal. Glow:
Generative flow with invertible 1x1 convolutions. Advances in neural information processing systems, 31, 2018
Hao Li, Yang Zou, Ying Wang, Orchid Majumder, Yusheng Xie, R Manmatha, Ashwin Swami- nathan, Zhuowen Tu, Stefano Ermon, and Stefano Soatto. On the scalability of diffusion-based text-to-image generation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 9400-9409, 2024.
Tsung-Y Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and C Lawrence Zitnick. Microsoft coco: Common objects in context. In Computer Vision-ECCV 2014: 13th European Conference, Zurich, Switzerland, September 6-12, 2014, Proceedings, Part V 13, pp. 740-755. Springer, 2014.
Yaron Lipman, Ricky TQ Chen, Heli Ben-Hame Maximilian Nickel, and Matt Le. Flow matching for generative modeling. arXiv preprint arXiv:2210.02747, 2022.
Haotian Liu, Chunyuan Li, Yuheng Li, and Yong Jae Lee. Improved baselines with visual instruction tuning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recogni- tion, pp. 26296-2666 26306, 2024.
Xingchao Liu, Chengyue Gong, and Qiang Liu. Flow straight and fast: Learning to generate and transfer data with rectified flow. arXiv preprint arXiv:2209. 00000, 2022.
I Loshchilov. Decoupled weight decay regularization. arXiv preprint 171.0500,,
Zeyu Lu, Zidong Wang, Di Huang, Chengyue Wu, Xihui Liu, Wanli Ouyang, and Lei Bai. Fit: Flexible vision transformer for diffusion model. arXiv preprint arXiv:2402. 240. 2024.
Nanye Ma, Mark Goldstein, Michael Albergo, Nicholas M. Boffi, Eric Vanden -ijnden, and Saining Xie. Sit: Exploring flow andnbsp.diifusiion-aaas generative models withnbsp:scalable interpolant transformers. In Computer Vision - ECCV 2024: 18th European Conference, Milan, Italy, September 19-October 4, 2024, Proceedings, Part LXXVII, pp. 23-40, Berlin, Heidelberg, 2024a. Verlag, ISBN 978-3-031-72979-9, doi: 10.10077978--3-31-772888 htttps://oi... org/ 3tt.://001000778---0011111
URL
Nanye Ma, Mark Goldstein, Michael S Albergo, Nicholas M Boffi, Eric Vanden- Vanden-Eijnden, and Sain- ing Xie. Sit: Exploring flow and tiffusion-based generative models with scalable interpolant transformers. arXiv preprint arXiv:2401.087400 2024b.
Kangfu Mei, Zhengzhong Tu, Mauricio Delbracio, Hossein Talebi, Vishal M Patel, and Peyman Milanfar. Bigger is not always better: Scaling properties of latent diffusion models. arXiv preprint arXiv:2404.013366 2024.
12

---

Published as a conference paper at ICLR 2026
Igor Molybog, Peter Albert, Moya Chen, Zachary Detito, David Esiobu, Naman Goyal, Punit Singh Koura, Sharan Narang, Andrew Poulton, Ruan Silva, et al. A theory on adam instability in large- scale machine learning. arXiv preprint arXiv:2304.098777 2023.
William Peebles and Saining Xie. Scalable diffusion models with transformers. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 4195-4205, 2023.
Ethan Perez, Florian Strub, Harm De Vries, Vincent Dumoulin, and Aaron Courville. Film: Visual reasoning with general conditioning layer. In Proceedings of the AAAI conference on artificial intelligence, volume 32, 2018.
Bryan A. Plummer, Liwei Wang, Chris M. Cervantes, Juan C. Caicedo, Julia Hockenmaier, and Svetlana Lazebnik. Flickr30k entities: Collecting region-to-phrase correspondences for richer image-to-sentence models. International Journal of Computer Vision, 123(1):74 October 2016. ISSN 1573-1405. doi: 10.1007//11223--16--00
htLptttp:/// //dx: doi. dog.oog/1c..
Alec Radford. Improving language understanding by generative pre-training. 2018.
Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, llya Sutskever, et al. Language models are unsupervised multitask learners. OpenAl blog. 1(8):9, 2019.
Aditya Ramesh, Mikhail Pavlov, Gabriel Goh, Scott Gray, Chelsea Voss, Alec Radford, Mark Chen, and Ilya Sutskever. Zero-shot text-to-image generation. International conference on machine learning, pp. 8821-8831. Pmlr, 2021.
Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bjöm Ommer. High- resolution image synthesis with latent diffusion models. In Proceedings of the IEEF/CVF confer- ence on computer vision and pattern recognition, pp. 10684-10695, 2022.
Axel Sauer, Kashyap Chitta, Jens Müller, and Andreas Geiger. Projected gans converge faster. Advances in Neural Information Processing Systems, 34: 14410000-1799, 2021.
Christoph Schuhmann, Romain Beaumont, Richard Vencu, Cade Gordon, Ross Wightman, Mehdi Cherti, Theo Coombes, Aarush Katta, Clayton Mullis, Mitchell Wortsman, al. Laion-5b: An open large-scale dataset for training next generation image-text models. Advances in Neural Information Processing Systems, 35:25278-25294, 2022.
Jascha Sohl-Dickstein, Eric Weiss, Niru Maheswaranathan, and Surya Ganguli. Deep unsupervised learning using nonequilibrium thermodynamics. In International conference on machine learn- ing. pp. 2256-2265. PMLR, 2015.
Yang Song and Stefano Ermon. Generative modeling by estimating gradients of the data distribution. Advances in neural information processing systems, 32, 2019.
Yang Song, Jascha Sohl-Dickstein, Diederik Kingma, Abhishek Kumar, Stefano Ermon, and Ben Poole. Score-based generative modeling through stochastic differential equations. arXiv preprint arXiv:2011.13456. 2020.
Yang Song, Conor Durkan, lain Murray, and Stefano Ermon. Maximum likelihood training of score-based diffusion models. Advances in neural information processing systems, 34:1415- 1428, 2021.
Keqiang Sun, Junting Pan, Yuying Ge, Hao Li, Haodong Duan, Xiaoshi Wu, Renrui Zhang, Aojun Zhou, Zipeng Qin, Yi Wang, al. Journeydb: A benchmark for generative image understanding. Advances in Neural Information Processing Systems, 36, 2024.
Chameleon Team. Chameleon: Mixed-modal early-fusion foundation models. arXiv preprint
arXiv:2405.09818, 2024.
Arash Vahdat, Karsten Kreis, and Jan Kautz. Score-based generative modeling in latent space. Advances in neural information processing systems, 14:11287-1130,, 2021.
A Vaswani. Attention is all you need. Advances in Neural Information Processing Systems, 2017.
13

---

Published as a conference paper at ICLR 2026
Mitchell Wortsman, Peter J Liu, Lechao Xiao, Katie Everett, Alex Alemi, Ben Adlam, John D Co- Reyes, Izzeddin Gur, Abhishek Kumar, Roman Novak, et al. Scall-scale proxies for large-seale transformer training instabilities. arXiv preprint 2...v:22309.1322.. 2023.
Xiaoshi Wu, Yiming Hao, Kegiang Sun, Yixiong Chen, Feng Zhu, Rui Zhao, and Hongsheng Li. Human preferenace score v2: A solid benchmark for evaluating human preferences of text-to- image synthesis, 2023. URL https:
org/
09341
Jiazheng Xu, Xiao Liu, Yuchen Wu, Yuxuan Tong, Qinkai Li, Ming Ding, Jie Tang, and Yuxiao Dong. Imagereward: learning and evaluating human preferences for text-to-image generation. In Proceedings of the 37th International Conference on Neural Information Processing Systems, pp- 15990--1995,, 2023.
Biao Zhang and Rico Sennrich. Root mean square layer normalization. Advances in Neural Infor- mation Processing Systems, 32, 2019.
Hongkai Zheng, Weili Nie, Arash Vahdat, and Anima Anandkumar, Fast training of diffusion models with masked transformers. arXiv preprint arXiv:2306.099 09305, 2023.
14

---

Published as a conference paper at ICLR 2026
A
APPENDIX
APPENDIX
This appendix organized follows:
In Section
B
we list overview of the notation used in the paper.
Section
D we provide extended related work.
In Section
we provide experimental details.
In Section
provide the derivation of the likelihood used in this paper.
In Section
we provide the details of scaling FLOPs counting hased on our models.
In Section provide extra experiments.
In Section we list the limitations and future works.
In Section J
we discuss the in-context condition and cross attention mechanism in detail.
15

---

Published as a conference paper at ICLR 2026
B
NOTATION
Tab. B
provides an overview of the notation used this paper.
Table 2: Summary of the notation and abbreviations used in this paper.
Symbol
Meaning
RF
Rectified Flow (Lipman et al. 2022 Liu et al. 2022 Albergo et al.
2023
LN
Logit-Normal timestep sampler Ttn (1; m, 8) EEssr al. 2024
SNR
OggaalNNose-Reenl
X
DDPM
Denoising Diffusion Probabilistic Models (Ho et al.
2020
LDM
Latent Diffusion Models (Rombach et al. 2022
VP
Variance Preserving formulation (Song et al. 2020
VLB
Variational Lower Bound
VAE
Variational Autoencoder
KL
KL Divergence K (p(xx) ((x)) = En [In
p(z)
9(2)
PD
Dataset distribution
dutta
Dimension the attention output
fs
Dimension of the Feedforward layer
dmodel
Dimension of the residual stream
Mayer
Depth of the Transformer Context length of input tokens
leta
limg
Context length of image tokens
lteat
Context length of text tokens
himm
Context length of time tokens
Nooc
Size of Vocabulary list
Whedd
Number of heads in Multi-Head Attention
N
Number of parameters
D
Size training data (token number)
C
Compute budget, C 6ND
Nopt
Optimal number of parameters for the given budget Optimal training tokens for the given budget.
D Dopt
Gaussian Noise N (0, /)
Xa
sample created at timestep &
t
Timestep ranging from [0, 1]
Velocity X1 X
a₁ represents the scaling factor during noise sample creation.
B
3, represents the diffusion factor during noise sample creation. Noise Level defined for each timestep
fo(x)
The network we use to learn the transition kernel. So : RNxM + R
17
Learning rate Logit-Normal Timestep sampling schedule, m is the location parameter, s the scale parameter Loss given model parameters, data points, and timesteps. Fixed step size for ODE/SDE samplers Parameters for diffusion models Parameters VAE encoder
Tin
(t;
L(O, x,t)
0
Φb
2b
Parameters for VAE decoder EMA coefficient
OEMA
16

---

Published as a conference paper at ICLR 2026
C LLM USAGE
In this work, large language models (LLMs) were only employed to polish the wording and improve the readability of the manuscript. No part of the research design, data analysis, result interpreta- tion involved the use of LLMs. All scientific content and conclusions were produced entirely by the authors.
D EXTENDED RELATED WORK
Diffusion Transformers Transformers have become the de facto model architecture in language modeling (Radford 2018: Radford et al.) 2019; Devlin 2018). and they have also achieved sig- al. 2020; He et al. 2022). Recently,
nificant success computer vision tasks (Dosovitskiy Transformers have been introduced into diffusion models (Peebles & Xie 2023)) where images are divided into patches (tokens), and the diffusion process is learned on these tokens. Additional con- ditions, such as timestep and text, are incorporated into the network via cross-attention (Chen et al.) 2023), Adaptive Normalization (Perez et al. 2018), or by concatenating them with image tokens [Bao et al. 2023) Zheng et al. (2023) proposed masked transformers to reduce training costs, while Lu et (2024) introduced techniques for unrestricted resolution generation. Diffusion Transformers (DiTs) inherit the scalability, efficiency, and high capacity of Transformer architectures, positioning them as a promising backbone for diffusion models. Motivated by this scalability, we investigate the scaling laws governing these models in this work. To ensure robust and clear conclusions, adopt a vanilla Transformer design (Vaswani 2017), using a concatenation of image, text, and time tokens input the models.
Diffusion Models Diffusion models have gained significant attention due to their effectiveness in generative modeling, starting from discrete-time models (Sohl- Sickstein et al.) 2015] Ho al.] 2020 [Song& & Ermon (019) to more recent continuous-time extensions (Song et al.) (020). The core idea of diffusion models to learn a sequence of noise-adding and denoising steps. In the forward process, noise is gradually added to the data, pushing it toward a Gaussian distribution, and in the reverse process, the model learns to iteratively denoise, recovering samples from the noise. Continuous- variants (Song et al. 2020) further generalize this framework using stochas- tic differential equations (SDEs), allowing for smoother control over the diffusion process. These methods leverage neural network architectures to model the score function and offer flexibility and better convergence properties compared to discrete versions. Diffusion models have shown remark- able success in various applications. For instance, ADM (Dhariwal & Nichol 2021) outperforms GAN on ImageNet. Following this success, diffusion models have been extended to more com- plex tasks such as text-to-image generation. Notably, models like Stable Diffusion (Rombach al.) 2022) and DALLE (Ramesh et al.) 1021) have demonstrated the ability to generate highly realistic and creative images from textual descriptions, representing a significant leap in the capabilities of generative models across various domains.
Normalizing Flows Normalizing flows has been a popular generative modeling approach due to their ability to compute exact likelihoods while providing flexible and invertible transformations. Early works like GLOW (Kingma & Dhariwal 2018) and RealNVP (Dinh et al., 2016) introduced powerful architectures that allowed for efficient sampling and likelihood estimation. However, these models were limited by the necessity of designing specific bijective transformations, which con- strained their expressiveness. To address these limitations, Neural ODE (Chen al.) 2018) and FFJORD (Grathwohl et al., 2018) extended normalizing flows to the continuous domain using dif- ferential equations. These continuous normalizing flows (CNFs) allowed for more flexible trans- formations by parameterizing them through neural networks and solving ODEs. By modeling the evolution of the probability density continuously, these methods achieved higher level of expres- siveness and adaptability compared to their discrete counterparts. Recent work has begun to bridge the gap between continuous normalizing flows and diffusion models. For instance, ScoreSDE (Song) al., 2020) demonstrated how the connection between diffusion models and neural ODEs can be leveraged, allowing both exact likelihood computation and flexible generative processes. More re- cent models like Flow Matching (Lipman et al. 2022 and Rectified Flow (Liu al.) 2022 further combined the strengths of diffusion and flowebased models, enabling efficient training via diffusion processes while maintaining the ability to compute exact likelihoods for generated samples. In this
17

---

Published as a conference paper at ICLR 2026
paper, we build upon the formulation introduced by rectified flow and Flow Matching. By leverag- ing the training approach of diffusion models, we benefit from their generative performance, while retaining the capability to compute likelihoods.
Likelihood Estimation Likelihood estimation in diffusion models can be approached from two primary perspectives: treating diffusion models as variational autoencoders (VAEs) or as normal- izing flows. From the VAE perspective, diffusion models can be interpreted as models where we aim to optimize variational lower bound (VLB) on the data likelihood (Kingma 2013). The vari- ational lower bound decomposes the data likelihood into a reconstruction term and a regularization term, where the latter measures the divergence between the approximate posterior and the prior. In diffusion models, this framework allows us to approximate the true posterior using series of gradually noised latent variables. Recent works (Ho et al., 2020; Kingma et al.) 202 1 Song et al.) 2021) have derived tighter bounds for diffusion models, enabling more accurate likelihood estima- tion by optimizing this variational objective. Alternatively, diffusion models can be viewed form of normalizing flows, particularly in the context of continuous- time formulations. Using neu- ral ODEs (Chen et 2018)) diffusion models can be trained to learn exact likelihoods by modeling the continuous reverse process as an ODE. By solving this reverse-time differential equation, one directly compute the change in log-likelihood through the flow of probability densities (Grath- wohl ct al., (2018). This approach provides a method for exact likelihood computation, bridging the gap between diffusion models and normalizing flows, and offering more precise estimate of the likelihood for generative modeling.
E EXPERIMENTAL DETAILS
E.1
DATA
We primarily utilized three datasets in our work. Several ablation studies on formulation and model design were conducted using JourneyDB (Sun et al. 2024). Additionally, we curated a subset of 108 million image-text pairs from the Laion-Aesthetic dataset, applying threshold of for the aesthetic score. After collecting the data, we re-captioned all images using LLAVA 1.5 (Liu et al., 20024), specifically employing the
model for this task. We then split the data into training and validation sets with ratio of 100:1. Our third dataset is COCO (Lin al., 2014), where we used the 2014 validation set to test scaling laws on an out-of-dommin dataset.
E.2 MODEL DESIGN
In this paper, we evaluate two distinct model architectures. For the PixArt model, we follow the original design presented in Chen et al. (202).. The In-Context Transformers are based on the In-Context block described in Peebles & Xiel (2023). To facilitate large-scale model training, we employ QK-Norm (Dehghani et al. (202)) and RMSNorm (Zhang & Sennrich] 2019). The patch size set to 2. Although previous work (Kaplan et al. 2020) suggests that the aspect ratio (width/depth) of Transformers does not significantly impact scaling laws, it is crucial to maintain consistent ratio when fitting models to scaling laws. To demonstrate this, we train a series of models under a fixed computational budget, selecting models of various sizes and aspect ratios (32 and 64). We then plot the relationship between the number parameters and loss. As illustrated in Fig. 6 mixing models with aspect ratios of 64 and 32 obscure the overall trend. To address this issue, maintain the aspect ratio at 64 throughout.
F DERIVATION OF THE LIKELIHOOD
In this section, we provide derivation of the likelihood estimation in paper. In this paper, use two ways to compute the likelihood. The first method is estimating the VLB (variational lower bound). Following Kingma et al. (021); Vahdat al. 20221, derive VLB in latent space. However, we cannot compute the entropy terms in the VAE. So our surrogate metric differs from the true VLBB up constant factor.
18

---

Published as a conference paper at ICLR 2026
Loss vs Parameters for All Models
Loss Parameters for AR=64 Models
0.94
0.94
0.92
0.92
0.90
Loss
0.88
Loss
0.88
0.86
0.86
0.84
0.84
0.82
0.82
0.80
0.80
10a
10a
10a
10a
Parameters (log scale)
Parameters (log scale) (b)
Figure 6: The effect of AR.
VAE The latent diffusion model (Vahdat al. 2021 Rombach al. 2022) consists of com- ponents: a variational autoencoder (VAE) that encodes images into a latent space and decodes latents back into images, and a continuous diffusion model that operates in the latent space. To train the latent diffusion model, we optimize the variational encoder 9á. the decoder P*t and the diffusion model po- Following Ho et al. (2020), the models are trained by minimizing the variational upper bound on the negative log-likelihood log P(z):
Le (2r) = Eg (x) Pw (x) (z) + K (zol(z
(11)
= Eg (x) log Po
+ (z)
(=( [x)] + + (s Po (12)
In our implementation, we directly adopt the VAE from Stable Diffusion 1.5 and keep it fixed during training. As a result, the reconstruction term (first term) and the negative encoder entropy term (second term) remain constant across different models. In fact, the VAE in Stable Diffusion is trained following the VQGAN approach, which uses both L1 loss and an additional discriminator for training. Therefore, distribution not tractable. To simplify further, we omit the VAE encoding process altogether. Specifically, we skip both encoding and decoding through the VAE and treat the latents produced by the VAE as the dataset samples. Under this assumption, we estimate the offset VLB directly in the latent space.
cannot effectively estimate the reconstruction term since the decoder
In the latent space, we model the distribution of latent variables that can be decoded into images using the VAE decoder. We denote the samples in latent space as x, and the noisy latent timestep t as The variational lower bound (VLB) in the latent space is given by Kingma et al. (202T):
- logp((>) < VLB(x) - Dd (o + Eg(a0 [ log (og2(())))))))))))) + (x)), (13)
where the first two terms depend only on the noise schedule, and we treat these terms as irreducible losses since the noise schedule is fixed across all models. The third term the KL divergence between each pair of the reverse process p(2)(24+1) and the forward process g(2z |2, 2++1):
Cr(x)
Σ E [e
(14)
Since we assume that the forward and reverse processes share the same variance and both and Gaussian distributions, the KI. terms reduce to weighted 7.2 distances:
19

---

Published as a conference paper at ICLR 2026
1 Cr(x)
2 Benerronn
Σ (SNN())) -NR()))E= xn (x:00::000 (=)
(15)
where -- 1. In the limit as T oo, the loss becomes:
1 Cr(x) -- / SNR rr (=1
(16)
In our case, we utilize the velocity to predict the clean sample and compute the VLB.
Normalizing Flows Another method to compute the likelihood in our diffusion model is by view- ing the diffusion process type of normalizing flow. Specifically, leverage the theoretical results from Neural ODEs, which allow us to connect continuous normalizing flows with the evolu- tion of probability density over time. In Neural ODEs, the transformation of data through the flow can be described by the following differential equation for the state variable 2; as function of time:
dæ, dt
fe( t),
(17)
where fo( t) represents the network that predicts the time-dependent vector field To compute the change in log-probabiligg of the transformed data, the log-likelihood of the input data under the flow is given by:
d plgg) TTr dt
(af(z,))) (T((pteen))) Oz
(18)
where Tr represents the trace of the Jacobian matrix of fe( t). This equation describes how the log-density evolves as the data is pushed forward through the flow. To compute the likelihood, we integrate the following expression over the trajectory from the initial state to to the terminal state f
/" A1
(ola(ae:)) () dt.
logp(z,,) = logg(1t.))=
Tr
(19)
Oxz
Here, log p(r.)) represents the log-likelihood of the initial state (often modeled as Gaussian), and the integral accounts for the change probability density over time as the data evolves through the ODE. In our formulation, the network predicts the velocity vo t) = I = a, + 8,c, which corresponds to the derivative of z, with respect to time. Thus, start with clean samples, estimate the velocity, perform an iterative reverse-time sampling, and convert the samples into Gaussian noise. We can then compute the prior likelihood of the noise easily and add it to the probability shift accumulated during reverse sampling. In our experiments, we set the steps of reverse sampling to 500 obtain a rather accurate estimation.
G SCALING FLOPS COUNTING
In this section, provide detailed explanation of our FLOPs scaling calculations. Several prior works have employed different methods for counting FLOPs. In Kaplan et al. (2020), the authors exclude embedding matrices, bias terms, and sub-leading terms. Moreover, under their framework, the model dimension dmodel is much larger than the context length lete. allowing them to disregard context-dependent terms. Consequently, the FLOPs count N for their model given by:
M 12 dmodel X Mayger x (2dattn ddf);
(20)
20

---

Published as a conference paper at ICLR 2026
where dmodel represents the model dimension, Magyer denotes the depth of the model, dattn refers to the attention dimension, and d represents the feed-forward layer dimension.
In contrast, Hoffmann al. (2022) includes all training FLOPs, accounting for embedding matrices, attention mechanisms, dense blocks, logits, and context-dependen terms. Specifically, their FLOP computation includes:
Embedding: x lot x Nove x dmodel
Attention:
QKV Mapping: 2 3 x lets dnnodel dmnddl
QK: 2 letæ let X dmodel
Softmax: Theead X letx lete
Mask: 2 let letz X amodel
Projection: leta X dmodel dmodel
Dense: 2 lotz (dmodel dyf x 2)
Logits: 2 X Let dmoddl Nooc
Further details can be found in the Appendix F Hoffmann et al. (2022).
In Bi al. (2024), the authors omit the embedding computation but retain the context-dependent terms, which aligns with our approach. The parameter scaling calculation for vanilla Transformers in this paper follows the same format as theirs. We now present detailed scaling FLOPs calculations for the In-Context Transformers and Cross-Attn Transformers used in our experiments.
Attention blocks are the primary components responsible for scaling in Transformer architectures. In line with previous studies, we only consider the attention blocks, excluding embedding matrices and sub-leading terms. Unlike large language models (LLMs), model dimension is comparable to the context length, and therefore, we include context-dependent terms. In this section, we present FLOPs per sample rather than parameters, different tokens participate in different parts of the cross-attention computation. Additionally, since our input length is fixed, the FLOPs per sample are straightforward compute.
In-Context Transformers In-Context Transformers process joint embedding consisting of text, image, and time tokens, all of which undergo attention computation. Tab. B details the FLOPs calculations for single attention layer.
Table 3: Scaling FLOPs Calculation in In-Context Transformers
Operation
FLOPs per Sample
Self-Attn: QKV Projection
3 x 2 Mayyr Letw X d'modee X 3 dattn
Self-Attn: QK
3 x 2 Wayer let letx X dattn
Self-Attn: Mask
3 x 2 Mayer X leta X leta X dattn
Self-Attn: Projection
x 2 Mayer let dinodte X dattn
Self-Aur Attn: FFN
3 x 2 2 X Mayer X lax X 4 X deoodet
In our experiments, we set dmodel dattn and lata = 377, where latz = limg (256) + lteat (120) + 4isme (1). Thus, the simplified FLOPs cae--pe--saa sample scaling equation M is:
M 72 x lete Mayer & + 12 Blayer 12 dmode!
(21)
Cross-Attn Transformers In Cross-Attn Transformers, each attention block consists of self- attention and cross-attention mechanisms to integrate text information. The cross-attention uses image embeddings as the query and text embeddings as the key and value. The attention mask re- flects the cross-modal similarity between image patches and text segments. As a result, the FLOPs calculation differs from that of models using joint text-image embeddings. Tab, lists the FLOPs costs for each operation.
21

---

Published as a conference paper at ICLR 2026
Table 4: Scaling FLOPS Calculation in Cross-Attn Transformers
Operation
FLOPs per Sample
Self-Attn: QKV Projection
3 x Mlayer X limg X dmodel X dattn
Self-Attn: QK
3 x 2 Mayer X limg X limg X dattn
Self-Attm: Mask
3 x 2 Wayer limg X limg x datta
Self-Attn: Projection
3 x 2 X Maaye X Vamg X dmodel X dattn
Cross-Attn QKV
3 X Mayer (limg + 22 leeut) X dmodel X
dattn
Cross-Attn QK
3 x 2 Mayer ltezt limg X dattn
Cross-Attn Mask
3 x 2 x Wayer X ltezt X limg X dattn
Cross-Attn Projection
3 x 2 Mayer limg dmodel X dattn
FFN
3 x 2 x Blayer X limg x x d &model
Based on the experimental settings, we can simplify the FLOPs calculation as follows:
M = 84 X Maayer x Limg X &model + 12 x Blayer X Pimg X dattn
+12 X Riager beeut X dmodee + 12 X Mlayer X lezt X Limg X &moddl
(22)
Context-Dependent Terms From the equations above, it is evident that some context-dependent terms, such as 12x X Rlager 12 dmmodet, cannot be omitted. In our experiments, the aspect ratio of Transformers (width/depth=644) is maintained across all model sizes. The context length lets is 377 (image: 256, text: 120, time: 1), and dmodel = Riayer X 64. Since let and dmode! are comparable, the context-dependent terms must be retained.
H ABLATIONS
H.1 DIFFUSION FORMULATION
In diffusion models, various formulations for noise schedules, timestep schedules, and prediction objectives have been proposed. These three components are interdependent and require specific tuning to achieve optimal performance. this paper, we explore several common formulations and conduct ablation studies identify the best combination in setting.
Below, we list the candidate formulations used our ablation study.
Noise Schedule
Discrete Diffusion Models
DDPM Denoising Diffusion Probabilistic Models (DDPM) (Ho al. 2020 is discrete-time diffusion model that generates noisy samples via the following formula:
It og + B.e
where e is Gaussian noise, and and B, satisfy af + 83 = 1. Given a sequence of o,, the scaling factor can be defined as:
04
(1 σ,)
(23)
In DDPM, o, follows:
σ, = 0o T ( o0)
(24)
22

---

Published as a conference paper at ICLR 2026
LDM Latent Diffusion Models (LDM) (Rombach et al. 2022), used in Stable Diffusion, is a variant of the DDPM schedule. is also a variance- variance-preserving formulation, sharing the same structure DDPM but employing different noise schedule:
ot = -VVo-
T [ voo)
Continuous Diffusion Models
VP Variance Preserving (VP) diffusion (Song ct al. 2020 is the continuous counterpart of DDPM, where the noise added while preserving variance across timesteps. The sampling process given by:
21 e So a,ds xo + V e- aode
(25)
where / E [0, l].
Rectified Flow Rectified Flow (RF) (iuu et al. 2022 Lipman et al. 2022 Albergo et al., 2023)
another continuous time formulation, where a straight- line interpolation is defined between the initial sample To and the Gaussian noise e. The process is described by:
(1 t)zo + te
(26)
Prediction Type
Noise Prediction (e) The network predicts the Gaussian noise _N(0, 1) added to the samples during the diffusion process.
Velocity Prediction (v) In this formulation, the network predicts the velocity v t), which is defined as the derivative of the noisy sample x, with respect to time. If the noisy sample x, is defined by:
It atTo + Bic
(27)
The velocity given by:
vix,, =1 B/e
(28)
where af and S are the derivatives of and Bg with respect to timestep t.
Score Prediction (s) The network predicts the score function s(x, 1) log P(x, (), which the gradient of the log-probability density function. The score can be derived as:
s(x, t)
(29)
B
Timestep Sampling Schedule
Uniform Timestep Schedule In this schedule, the timestep / is uniformly sampled. For discrete- time diffusion models:
t ~~ 2... ..999
(30)
For continuous- time diffusion models:
1(0,))
(31)
23

---

Published as a conference paper at ICLR 2026
Logit-Normal (LN) Timestep Schedule The Logit-Normal (LN) timestep schedule Tin (t: m,
proposed in Esser et al. 2024). generates timesteps according to the following distribution:
SV2 2m ((1-t) exp (- (loggit(t) -m)²
The (t, s)
(32)
2s²
where logit(t) = log (12.). The LN schedule has two parameters: and s. It defines an unimodal distribution, where m controls the center of the distribution in logit space, shifting the emphasis of training samples towards noisier or cleaner regions. The parameter adjusts the spread of the distribution, determining its width. As suggested in Esser et al. (2024), to obtain a timestep, we first sample (m, s), and then transform it using the logistic function. For discrete-ting time diffusion, after obtaining ~> Tin (f: m, s). we scale t by = round x 999) to obtain a discrete timestep. Following Esser al. (2024),
utilized the parameters m and 8. More details and visualizations can be found Esser et al. 2024) Appendix B.4.
0.0, 1.00 and didn't 't sweep over
We conducted series of experiments using different combinations of formulations. Selective com- indicates that the combination is either not comparable with
binations are listed Tab. [ other formulations or that training diverges. We assume that the choice of formulation will be significantly affected by specific model designs or datasets. All experiments were conducted using Pixart (Chen et al., 2023), a popular text-to-image diffusion transformer architecture, Specifically, we used a small model with 12 layers and hidden size of 384, setting the patch size to 2. The models were trained on JB (Sun et al. 2024). medium-sized text-to-image dataset con- sisting of synthetic images collected from Midjourney. All models were trained for 400k steps using Adam W as the optimizer. As shown Tab. [S] the optimal combination [RF, LN, v], achieving an FID of 36.336 and Clip Score of 0.26684. This combination achieved the best performance both metrics and therefore, we used this setting in the remaining experiments.
Table 5: Ablation on diffusion formulations.
Noise Schedule
Timestep Schedule Prediction Type
FID
CLIP Score
DDPM
Uniform
40.469
0.26123
DDPM
Uniform
74.049
0.23136
DDPM
LN
6 40.100
0.26283
LDM
Uniform
39.001
0.26423
LDM
Uniform
V
LDM
LN
C
38.196
0.26624
VP
Uniform
C
44.343
0.26148
VP
Uniform
V
41.808
0.26320
VP
Uniform
S
45.808
0.25970
VP
LN
E
42.872
0.26354
VP
LN
44.107
0.26380
RF
Uniform
V
44.840
0.25682
RF
Uniform
RF
Uniform
C
RF
LN
36.336
0.26684
H.2 EFFECT OF EMA ON LOSS
The Exponential Moving Average (EMA) coefficient plays crucial role in shaping the loss curve and affects final outcomes. In EMA, the loss updated via = (1 - OEMA ) CEEM where is the latest loss value. This smoothing mechanism reduces fluctuations in cumulative loss. However, during the early stages training, EMA can overestimate the loss. As shown in Fig. 7 higher EMA coefficients lead to elevated loss values compared actual loss, potentially introducing bias in curve fitting and causing inefficient use of compute. From Fig. 7 we observe that αςMА 0.9 strikes a good balance, effectively smoothing the loss while only mildly inflating values early in training.
24

---

Published as a conference paper at ICLR 2026
EM i
D0 9 EiiL-O
oritor to
Enla-r so
atlone
Farationa
Aarationa
taratons
Eitt de
iitt--3 piet- do pit--
Enis- o
Orginal Loaa
pie dota pate
EMA-- do
2 de
BM4e6.322
Earabion
Faruttona
Eerations
Barations
oniiral Loss EMMiiii EMA les
Origiaal iaet-i taet
Groanal
Lags
Ohigrel Lise EMA-t0
3-
BRG-E
Rasatorn
Oriinall Loss Eilendia
Ofi nail Lass DMM--o p40-- 0979
Ong Loss EMAnti
Original isss
Crigina Los
Origina Loss
EMAm
EM 569
nail E4
Loer
Criginal Erti-1 o
D4
Original Loss
DRA $69
EMA-22 2
B4A-097
Sitke 59
Ellee 5p
359
dine
Farelies
lerel ena
teatoon
Orig rail
Origin Low
Original Dnekid 59 pituo 590
Criginal
Origin
Origiall Lone
pi0--097 p44-0090
EMAM6.00
EW Ellw- spo
DM4--0 519
Figure 7: Effect of EMA on loss values.
25

---

Published as a conference paper at ICLR 2026
FID vs CFG (steps=25)
Clip Score CFG (steps=25)
200
0.225
175
0.200
Clip Score
(a)
2
150
0.175
(b)
125
0.150
100
0.125
1.0
2.0
2.5
5.0
7.5
10.0
15.0 20.0
1.0
2.0
2.5
5.0
7.5
10.0
15.0
20.0
CFG
CFG
FID vs CFG (steps=50)
Clip Score vs CFG (steps=50)
200
0.225
175
Clip Score
0.200
(c)
150
0.175
(d)
125
0.150
100
0.125
1.0
2.0
2.5
5.0
7.5
10.0
15.0
20.0
1.0
2.0
2.5
5.0
7.5
10.0
15.0 20.0
CFG
Figure 8: Ablation sampling steps.
H.3 CLASSIFIER-FRES GUIDANCE & SAMPLING STEPS
We conduct ablation studies on CFG scales and sampling steps using compute-optimal models trained with budget of 10 In Fig. [2] we evaluate various CFG scales (2.5, 5.0, 7.5, 10.0) across different step counts and compute the FID. We find that 25 steps suffice for good perfor- mance. Next, fixing the number of steps to 25, we assess different CFG scales. As shown in Fig. CFG scale of 10.0 yields the best FID and is thus chosen as the default.
Although CFG influences generated results, the linearity of the FID scaling curves remains largely invariant to the CFG scale. To demonstrate this, we plot FID scaling curves under various CFG settings Fig. 10 The results confirm that scaling behavior remains linear across all tested CFG values.
H.4 EFFECT OF EMA ON MODEL
Maintaining an EMA copy of the model is common practice in diffusion models, as it often leads to improved sample quality. To evaluate its impact, we maintain model copy with EMA coefficient of 0.99 during training and measure the FID of the EMA versions across different compute budgets. As shown in Fig. I1 a). the EMA models also exhibit power-law scaling trend.
H.5 SCALING LAWS FOR MORE ARCHITECTURES DATA
We extend our scaling law analysis to additional architectures, including PixArt (Chen et al.) 2023) and Flux. As shown in Fig. b) and Fig. b), both models exhibit clear linear behavior in their loss eurves. These results suggest that scaling laws consistent across different diffusion transformer architectures.
We further test the applicability of scaling laws higher-resolution data. Specifically, train models on 512 512 images using budgets ranging from 1e17 to 5e18. The results, presented in Fig. 14 b). show clear linear trends, indicating that scaling laws hold regardless of data resolution.
26

---

Published as a conference paper at ICLR 2026
FID vs Steps
Clip Score vs Steps
180
cfg=2.5
0.23
cfg=5.0
160
cfg=7.5
0.22
cfg=10.0
Clip Score
0.21
FID
140
0.20
120
0.19
cfg=2.5
cfg=5.0
0.18
100
cf5=7.5
0.17
=fg=10.0
80
20
80
100
20
40
60
80
100
Steps
Steps
(a)
(b)
Figure 9: Ablation on CFG scale.
dg
cg
107
(a)
(b)
(c)
Figure 10: Scaling curves for different CFG scales.
EMA FID Compute Budget
FID
Fitted Line
Compute Budget
Figure 11: Scaling curves for EMA models.
27

---

Published as a conference paper at ICLR 2026
Loss vs Parameter Count
Loss vs Budget
1udget=10e+117
6x
Riz: y=3.7308 000000463
Budget=3.0e+17
Budget-6.0e+11
Budget= 0e+18
Dudget=3.0ee4e
Budget==6.0+11
10 1
Loss
Los
10-1
10
10a
10
10th
Parameter Count
Budgat
(a)
(b)
Figure Scaling laws for Flux.
Loss Parameter Count
Loss Budget
Budget-1.0e+16
y-3.7410
Du0gst=3.0e+16
9x
Budget=6.00e
Budget-110e+17
Budget-3.0ee 1.7 0c+17
bodget=6.0e+11
10a
Budget=10e+118
Loss
Losss
10
80-1
10a
10th
10t7
1000
Parameter
Budget (b)
Figure 13: Scaling laws for PixArt.
28

---

Published as a conference paper at ICLR 2026
Loss vs Parameter Count
Loss vs Budget
10a
Oudget=1.0e+11 0e+17
Fit: y-5.7882 00000099
Dudget=3.0e+4 0e+117
Budget=6.00 0e+17
Budget=1 0c+18
Budget=3 0e+18
8 80-1
Budget=6.0e+4e 08
Loss
8x 10
7x
10-1
10a
20
1018
Parametar Count
Budget
(a)
(b)
Figure 14: Scaling laws for 512x512 resolution.
FID vs Budget
Original Data
y=1059994.438xx-02216
10?
FID
102
10
10 Budget (FLOPs)
Figure 15: Scaling curves for FID on sparse-captioned dataset.
H.6 DATA ASSESSMENT
To demonstrate how scaling laws can assess data quality, we apply them to a dataset with the same images as our main experiment but using sparse tag captions instead of dense descriptions. As shown in Fig. [5| the FID scaling exponent -0.216, whereas the main dataset achieves -0.234. This indicates that dense captions lead to faster FID improvement, suggesting superior data quality compared to sparse tags.
H.7 TRANSFER RESULTS ON MORE DATASETS
To further validate the generalizability of our scaling laws, we perform transfer experiments on Flickr30k (Plummer al., 2016) and JourneyDB (Sun et al. 2024), following the same setup as
29

---

Published as a conference paper at ICLR 2026
Loss (Flickr) vs Compute Budget
FID (Flickr) vs Compute Budget
Dota
Data
Fitted Line
Ritted Line
Loos
360
10a
10
109
10at
120
Compute Budget
Computo Budget
(a)
(b)
Figure 16: Transfer results of loss and FID on Flickr30k.
Loss (JourneyDB) vs Compute Budget
FID (ourneyDB) vs Compute Budget
5x30
Data
2x39
Date
Fitted Line
Fitted Line
FDD
162
10t0 Compute Budget
Compute Budget
(a)
(b)
Figure 17: Transfer results of loss and FID on JourneyD..
with COCO. Models pretrained on our dataset are evaluated on these new domains. As illustrated in Fig. 16 and Fig. 17 both loss and FID maintain clear linear relationships, confirming that our findings transfer well across datasets.
H.8 GENERATED SAMPLES
We provide qualitative results by showcasing generated samples from models trained with increasing compute. Sampling is performed using fixed prompts and seeds. As shown in Fig. [8] sample quality improves with more compute. However, our goal is not to produce state-of-the-ar results, but rather understand scaling behavior in diffusion transformers. Hence, we did not curate high-quality datasets or optimize for SOTA image generation.
H.9 SCALING LAWS ON IMAGENET
While our primary experiments assume data-infinite settings, we also test scaling behavior under constranndt conditions using ImageNet (Deng et al. 2009). For this, we adopt the model from SiT (a et al. 2024a) and vary compute from 2 x -0 to 6 x 18 As shown in Fig. 19)c) the loss curves still follow clear plwer-law trends, indicating that scaling laws also emerge in finite-data
regimes.
30

---

Published as a conference paper at ICLR 2026
1e17
8e17
3e18
6e18
1.5e21
Figure 18: Generated samples from models with increasing compute.
Parameters
FID
$1x307
13.300
2/
12"
Compute fel
Figure 19: Scaling curves on ImageNet.
31

---

Published as a conference paper at ICLR 2026
Gen Eval Score vs Compute Budget
50
Filted line
Gen Score
19P
101a
1014
10P
10%
Compute Budget
Figure 20: Scaling laws on GenEval benchmark.
Compute Budget
ImageReward Compute Budget
Date Line
Data
NP002
10.22
Compute Budget
Compute Budget
(0)
Figure 21: Scaling laws Human Preference Rewards.
H.10 GENEVAL RESULTS
In addition to traditional metrics like FID and loss, we evaluate models using the GenEval (Ghosh al. 2023) benchmark, which measures text-image alignment across aspects such object count, color, and position. For each compute budget, we evaluate the corresponding compute-optimal model. Results are presented Fig. 20 further supporting the presence of consistent scaling behav- ior across diverse evaluation metrics.
H.11 HUMAN PREFERENCE RESULTS
We also employ Human--refferene ImageReward (Xu al. 2023), evaluate models. These benchmarks capture alignment with human preference. Across different compute budgets, the performance measured by these human preference rewards also follows power-law scaling trend. The results are plotted in Fig
reward models, including HPSv2.1 (Wu et al. 2023) and
H. SAMPLING STEPS
To study the effect of the sampling schedule on scaling behavior, we additionally vary the number of diffusion sampling steps while fixing the classifier-free guidance scale to 10.0. Specifically, we compare models evaluated with 35 and 50 sampling steps. As shown in Figure 22 changing the number of sampling steps shifts the overall performance, but the resulting points still lie on clear straight line in log-log space. This indicates that the power-law relationship between compute and generation quality remains intact, and that modifying the sampling steps mainly affects the offset of the scaling curve rather than the existence of the scaling law itself.
32

---

Published as a conference paper at ICLR 2026
FID, Sampling Steps 35
FID, Sampling Steps 50
10
10
e
10
10
10t
10th
10
1027
10
10
10a
10
Buxgett (FLDPs)
Budget (FLOPs)
(a)
(b)
Figure 22: Scaling laws of FID with differnet sampling steps.
Vanilla In-Context vs Cross-Attention
102
Vanilla
x 10
2 10
FID
10a
10t6
10
10
10tH
Budget (FLOPs)
Figure 23: FID results of cross attention vs vanilla in-context.
H.13 FID RESULTS OF CROSS-ATTN VS VANILLA IN-CONTEXT
We provide the FID results of cross attention architecture and vanilla in-context architecture. The result shown in Fig. 23 the cross attention architecture has more steep slope and shows greater scalability, which is consistent with the loss curve.
I LIMITATIONS AND FUTURE DIRECTIONS
Limited modalities and downstream tasks. Our empirical study focuses exclusively the text- to-image setting. We do not experiment with additional modalities such as text-to-video or other multi-modal generative tasks. Moreover, evaluation is restricted to the core generative objective and a small set of aggregate metrics; we do not consider richer downstream applications such as image editing, inpainting, other task-specific evaluations. As result, the extent to which the observed scaling laws transfer to these broader modalities and downstream tasks remains an open question.
Compute-limited regime only. All experiments are conducted in compute-limited regime, where we assume access to sufficiently large datasets and primarily vary the compute budget via model size and training steps. In practice, large-scale training often becomes data-constrained when scaling up compute, and the effective scaling behavior in such regimes may deviate from the laws observed here. In particular, data-constrained settings typically require corrections to stan- dard compute-optimal scaling laws to account for finite-data effects. We leave a systematic study of scaling behavior and its correction under lama-li regimes to future work.
33

---

Published as a conference paper at ICLR 2026
More accurate hyperparameters. Our results are obtained using single, well-tuned set of train- ing hyperparameters for each model family, which are reused across different compute budgets. While this choice reflects realistic training protocol, it also means that our scaling laws may not reflect fully optimized performance at each operating point. Better or more carefully adapted hy- perparameters (e.g., learning rate schedules, regularization strength, or optimization settings) could further reduce the variance and systematic bias in the observed scaling curves, thereby improving the accuracy of compute-optimal predictions. However, with our current setting, can already reveal the scaling dynamics of diffusion models, and we leave the more accurate hyperparameter as an important direction for future research.
DISCUSSION: IN-CONTEXT VS. CROSS-ATTENTION TRANSFORMERS
Our main experiments in Sec. 4 indicate that, under our controlled setting, cross-attention transform- ers exhibit more favorable scaling trend with compute compared to the simple in-context baseline we study. At the same time, recent text-to-image systems, such as MMDiT-style architectures and Flux-like models, report very strong absolute performance with in-context conditioning. This may appear to be in tension with our findings.
We emphasize that this apparent discrepancy can arise from many factors beyond the condition- ing mechanism itself. Publicly available systems differ not only in whether they use in-context or cross- cross-attention conditioning, but also in total compute, data mixture and quality, and overall training recipe, all of which strongly affect both performance and observed scaling behavior. In practice, models like Flux are among the strongest systems largely because of their full recipe (data com- pute + architecture), rather than solely because in-context conditioning is intrinsically superior to cross-attention. Under the same data and compute budget, a carefully tuned cross-attention architec- ture could plausibly achieve similar or even better performance. However, such a direct, controlled comparison is currently available in the literature.
Moreover, modern in-context architectures typically incorporate several important improvements relative the simple in-context baseline considered in this work: (i) Stronger conditioning: they often use multiple CLIP encoders plus large language model (c.g.. T5-XXL) and inject text fea- tures into every block via MIPs that modulate attention and FFN layers, leading to much stronger conditioning; (ii) Better timestep conditioning: instead of encoding timesteps as in-context tokens, they employ adaptive LayerNorm-style conditioning, where a timestep embedding is mapped to scale/shifi/gate parameters that modulate each block, which is empirically more efficient; (iii) More efficient modality interaction: as in SD3-style designs, text and image tokens may maintain sepa- rate parameter sets, with attention used only for cross-modal interaction and text injected primarily in earlier blocks, reducing cross- modal compute; (iv) More advanced architectural and training choices: improved positional encodings, noise/timestep schedules, and other refinements further enhance scaling efficiency.
Taken together, these design choices, combined with different data and compute budgets, can explain why state-of-the-art in-context models achieve very strong performance in practice, without imply- ing a direct, controlled advantage of in-context conditioning over cross-altention in terms of intrinsic scalability. Our results should therefore be interpreted as characterizing scaling behavior within simplified and carefully matched setting. Extending this analysis to more advanced in context and cross-attention architectures is an important direction for future work.
34


Reviews:
[{"review_id": "6TcyqmjROW", "reviewer": "Reviewer_knY9", "summary": "This paper is an empirical study on how pretraining loss and downstream generation quality of DiTs scale with compute, model size and data. It also shows that the pretraining loss follows a power-law relationship with compute. It proposes rules to to pick optimal model / data and to predict downstream metrics from pretraining loss. Additionally authors also identify the relationship between pretraining loss and generation performance.", "questions": "Same as weeknesses", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "Paper is very well-written and easy to follow. The method section has been particularly well-described.\n1. Authors explore a very timely topic in diffusion based transformer models. \n2. Authors conduct experiments with DiT across various compute budgets and provide empirical proofs.\n3. The paper also reports a correlation between pretraining loss and downstream FID, GenEval and human preference  metric which can be potentially beneficial for practitioners.", "weaknesses": "1. This paper fixes  a particular training dataset derived from Laion-5B. \nHowever, with generative models, we are observing that a careful data curation pipeline can affect the training and model scaling options. Authors can consider also studying how noisy / clean data can affect scaling properties.\n\n2. The power-law relationship between training budget and generation performance provides a sign that the scaling law can predict generation performance. However, it is a bit unknown if this law will continue to hold under different hyper-parameter / diffusion sampling settings including varying classifier guidance strength for example.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "sdkd6P4dRc", "reviewer": "Reviewer_KekD", "summary": "This paper investigates the scaling laws of diffusion models for text-to-image generation. The authors conduct experiments across a wide range of compute budgets, spanning from 1e17 to 6e18 FLOPs. The results reveal empirical relationships between training loss, evaluation metrics, and compute expenditure. Based on these relationships, the paper claims that the performance of larger diffusion models can be accurately predicted.", "questions": "Please refer to the weakness session.", "limitations": "", "rating": 4, "confidence": 3, "soundness": 2, "presentation": 3, "contribution": 3, "strengths": "- This paper addresses an important problem: establishing scaling laws for text-to-image generation with diffusion models. The findings have the potential to offer valuable insights to the community.\n- The authors conduct extensive experiments and dedicate substantial effort to derive and validate the proposed scaling laws.", "weaknesses": "- In Figure 1, the assumption underlying the use of parabolic fitting for the performance curve is not clearly stated. If the curve is assumed to be unimodal, then a ternary search strategy could directly identify the optimal loss without requiring curve fitting.\n- In Figure 20 (GenEval results), only the value \"10\" appears on the y-axis, making it difficult to determine the specific GenEval scores associated with each data point. Providing a complete y-axis scale would significantly improve readability.\n- The scaling analyses use log scale for FID (Figure 3) and GenEval (Figure 20), but a linear scale for Human Preference Rewards (Figure 21). It would strengthen the consistency and interpretation of the results to explain why different scales are used and why each metric is expected to exhibit a linear or log-linear relationship with model performance.\n- The analysis would be further strengthened by comparing the derived scaling laws to existing text-to-image diffusion models. Including these models in figures or table would provide a more comprehensive empirical grounding and reinforce the conclusions about scaling behavior.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "2tCJGJswzU", "reviewer": "Reviewer_y2Z9", "summary": "The paper investigates the scaling laws of text-to-image diffusion transformers (DiTs). It shows that these models follow a scaling behavior similar to that of large language models: the training loss and other performance metrics exhibit a power-law relationship with compute when the model size and token count are optimally balanced. Furthermore, the paper demonstrates that this trend generalizes across out-of-domain datasets and different model architectures.", "questions": "Overall, I enjoyed reading this paper. While the techniques are not new and the results are not particularly surprising given the established scaling laws of large language models, confirming that similar laws hold for DiTs is nevertheless a valuable and solid contribution. I have a few questions listed below and hope the authors can address them in the rebuttal.\n\n* I would like to confirm how Figure 2 is derived. Is it obtained using the same procedure as Figure 1? That is, for each metric, (1) train models of varying sizes under different compute budgets, (2) fit a parabola between the metric and model size for each compute budget, (3) identify the optimal model size for each compute budget, and (4) plot the metric versus compute budget using these optimal model sizes?\n\n* Eq. 4: This equation is somewhat confusing. First, N is not defined. Second, the summation index i does not appear in the summand, making it unclear what is being summed over.\n\n* In Section 3.1, \"Variational Lower Bound\" and \"Exact Likelihood\" are subpoints under \"Likelihood.\" However, in the current formatting, \"Likelihood,\" \"Variational Lower Bound,\" and \"Exact Likelihood\" appear at the same heading level, which may mislead readers into thinking they are parallel sections, causing potential confusion.\n\n* Figure 5: How many data points are included in each line?\n\n* Section 4 states that the experiments show cross-attention transformers exhibit a superior scaling trend compared to in-context transformers. However, as discussed in Section 4, recent works suggest that in-context conditioning performs better. Could the authors clarify where this discrepancy might come from?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "* The paper is very well written.\n\n* The findings enable practitioners to tune the hyperparameters of DiTs more efficiently.\n\n* The paper demonstrates scaling laws not only with respect to the loss, but also for other useful metrics such as FID and human preference reward.", "weaknesses": "Beyond text-to-image generation, diffusion models have also been applied to tasks such as class-conditioned image generation and text-to-video generation. However, the paper only experiments on text-to-image generation, so it remains unclear whether the same scaling laws extend to these other tasks.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "HtkWVC0qGA", "reviewer": "Reviewer_fsKy", "summary": "This paper presents the first systematic investigation of the scaling laws of Diffusion Transformers (DiT) for text-to-image synthesis. The authors conduct extensive expriements accross compute budgets $C$ from $1e17$ to $6e18$, and predict the performance for a larger 1B-model with compute budgets $1.5e21$.\n\nThe key contributions are:\n1. **First Establishment of Scaling Laws of DiT**: This paper is the first to investigate and confirm the power-law relationship with the compute budget.\n1. **Fit the Power Law Equations**: This papers fits the power-law equations for model/data size: $N_{opt}=0.0009\\cdot C^{0.5681}$ and $N_{opt}=186.8535\\cdot C^{0.4319}$, as well as for the loss $L=3.3943\\cdot C^{-0.0273}$.\n1. **Validation on Larger Compute Budget**: The derived laws are validated by extrapolating to a significantly larger compute budget ( $1.5e21$), a 1B parameter model, and demonstrating that its loss and performance metrics match the predictions.\n1. **Evalution Metrics Justification**: The authors also verify generative quality metrics like FID also follows the scaling laws ($FID = 2.2566 \\times 10^6 \\cdot C^{-0.234}$), as well as other evaluation metrics like VLB, exact likelihood, GenEval, and human preference scores. This also justifys that these evaluation metrics are suitable for text-to-image synthesis evaluation.\n1. **Generative Robustness**: The paper demonstrates the robustness of these laws by showing they hold for out-of-domain (OOD) data (e.g., COCO) and across different model architectures.\n1. **Guidance on Model/Data design**: This paper shows difference model architecture (In-Context *v.s.* Cross-Attention) has different coefficients on the power-law equations, which helps evalutate future model designs using small compute budgets.", "questions": "1. As shown in Figure 3, as the $C$ increases, FID and Loss decrases, and the fitted curve is strage line (as y-axis is log-scale). However we cannot increase compute budget forever to obtain a minus loss, there should be saturation or even overfit. How the scaling curves show staturation and overfit?\n1. For a fixed compute budge $C$, the relationship between it and $N$, $C$ is defined as $C=6ND$, but how about $N_{opt}$ and $D_{opt}$?  Say we have a designed a model, and fit the coefficents of $N_{opt}$ and $D_{opt}$, and we have limited compute budget, how can we balance the between $N_{opt}$ and $D_{opt}$?\n1. The second and the third sub-figures of Figure 1, the y-axis should be *optimal* parameter $N_{opt}$ and *optimal* token $D_{opt}$?\n1. Any experimental FID result of Vanilla In-Context vs Cross-Attention?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 4, "presentation": 3, "contribution": 4, "strengths": "1. This paper for the first time provide a systematic investigation and verification of predictive, quantitative scaling laws of Diffusion Transformer (DiT).\n1. The expriments conducted to investigate the scaling laws of DiT is reasonable and thorough, which include large-scale validation, evaluation of various metrics and coefficents comparision of different models.\n1. The proposal to use scaling exponents as a predictable benchmark is practically usable and provides the community with a powerful and low-cost tool for architectural and data-quality comparisons.", "weaknesses": "The paper is generally good, but there's some mirror \"weaknesses\" or improvement sugesstion. I am happy to increase the score if they are addressed.\n1. **Contradiction in Section 4**: There is a small contradiction between the text and Table 1 when comparing model architectures. The text states, \"The Cross-Attention Transformer exhibits a *larger* model exponent\". However, Table 1 shows the Cross-Attention model exponent (0.54) is smaller than the Vanilla In-Context model's exponent (0.56). The text's conclusion—that \"more resources should be allocated toward scaling the dataset\" —is correct and does align with the table (data exponent 0.46 > 0.43).\n1. **Some writtings are confusing**:\n    1. Line 241, $C$ is the symbol of \"Compute\", but later it's referenced as compute budget in line 252. First-time reader might not know \"Compute\" and \"Compute budget\" are the same.\n    1. There are three sub-figures in Figure 1, but they are refered as a whole, e.g., in Line 265, it should refer to the first sub-figure of Figure 1.\n    1. The logic from Line 243 to 251 is not clear to me. It says $C=6ND$, a linear relationship between $C$ and $N$ or $D$, but later it ypothesizes $N_{opt} \\propto C^{a}$ and $D_{opt} \\propto C^{b}$. It's unclear to me why, untill I review Figure 1 that the x-axis is in log-scale and the fitting curve is a strage line. But still the y-axis of Figure 1 does not say they are $N_{opt}$ and $D_{opt}$ so it's still unclear to me how the hypothesis comes from.\n    1. Figure 3 shows legend the second and the third sub-fogures of Figure 1 do not. And the second sub-figure of Figure 3's legend is *fitter* curve.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)