STARFlow: Scaling Latent Normalizing Flows for High-resolution Image Synthesis
Authors: Jiatao Gu
Tianrong Chen
David Berthelot
Huangjie Zheng
Yuyang Wang
Ruixiang ZHANG
Laurent Dinh
Miguel Ángel Bautista
Joshua M. Susskind
Shuangfei Zhai
Venue: NeurIPS 2025 spotlight ()
Keywords: Normalizing Flows
Autoregressive Models
Latent Autoregressive Flows
Scalable Image Generation
Primary area: deep_learning
TL;DR: We propose STARFlow, a scalable autoregressive flow model that enables high-quality image generation

Abstract: We present STARFlow, a scalable generative model based on normalizing flows that achieves strong performance on high-resolution image synthesis. 
STARFlow's main building block is Transformer Autoregressive Flow (TARFlow), which combines normalizing flows with Autoregressive Transformer architectures and has recently achieved impressive results in image modeling. In this work, we first establish the theoretical universality of TARFlow for modeling continuous distributions. Building on this foundation, we introduce a set of architectural and algorithmic innovations that significantly enhance the scalability: (1) a deep-shallow design where a deep Transformer block captures most of the model’s capacity, followed by a few shallow Transformer blocks that are computationally cheap yet contribute non-negligibly, (2) learning in the latent space of pretrained autoencoders, which proves far more effective than modeling pixels directly, and (3) a novel guidance algorithm that substantially improves sample quality. Crucially, our model remains a single, end-to-end normalizing flow, allowing exact maximum likelihood training in continuous space without discretization. STARFlow achieves competitive results in both class- and text-conditional image generation, with sample quality approaching that of state-of-the-art diffusion models. To our knowledge, this is the **first** successful demonstration of normalizing flows at this scale and resolution. Code and weights available at https://github.com/apple/ml-starflow.

STARFLOW: Scaling Latent Normalizing Flows for High-resolution Image Synthesis
Jiatao Gu, Tianrong Chen, David Berthelot, Huangjie Zheng, Yuyang Wang, Ruixiang Zhang, Laurent Dinh, Miguel Angel Bautista, Josh Susskind, Shuangfei Zhai
Apple
jgg32,, szhaii Capple. com
Figure 1: Text conditioned high-resolution samples of variable aspect ratios generated from our 3.8B STARFlow
model. Resolutions are adjusted for the ease of visualization.
Abstract
We present STARFlow, scalable generative model based normalizing flows that achieves strong performance on high-resolution image synthesis. STARFlow's 's main building block is Transformer Autoregressive Flow (TARFlow), which com- bines normalizing flows with Autoregressive Transformer architectures and has recently achieved impressive results in image modeling. In this work, first estab- lish the theoretical universality of TARFlow for modeling continuous distributions. Building on this foundation, we introduce a of architectural and algorithmic innovations that significantly enhance the scalability: (1) deep-shallow design where deep Transformer block captures most of the model's capacity, followed by few shallow Transformer blocks that are computationally cheap yet contribute non-negligi,,,, (2) learning in the latent space of pretrained autoencoders, which proves far more effective than modeling pixels directly, and (3) novel guidance algorithm that substantially improves sample quality. Crucially, our model remains a single, end-to-end normalizing flow, allowing exact maximum likelihood train- ing in continuous space without discretization. STARFlow achieves competitive results in both class- and text-conditional image generation, with sample quality approaching that state-of-the-art diffusion models. To our knowledge, this the first successful demonstration of normalizing flows at this scale and resolution. Code and weights available at https: / github. com/ homm/appl///ie ml starflox.
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

Introduction
Recent years have witnessed remarkable progress in high-resolution text-lo-image generative model- ing, with state-of-the-ar approaches predominantly falling into two distinct categories. On one hand, diffusion models (Ho et al., 2020; Rombach al., 2022; Peebles & Xie, 2023; Esser et al., 2024) operating in continuous space have set new benchmarks in image quality. However, their reliance on iterative denoising processes renders both training and inference computationally intensive. On the other hand, autoregressive image generation methods (Yu et al., 2022; Sun et al., 2024; Tian et al., 2024) inspired by the success of large language models (LLMs, Brown et al., 2020; Dubey et al., 2024) avoid such inefficiencies by modeling images in discrete space via quantization; yet, this quantization can impose stringent limitations and adversely affect fidelity. More recently, a promising trend has emerged to explore hybrid models (Li et al., 2024; Gu et al., 2024b; Fan et al., 2024) that apply autoregressive techniques directly in continuous space. However, the inherently distinet characteristics of these two paradigms introduce additional complexity in effective unification.
In this paper, turn our eyes on the yet another modeling approach Normalizing Flows (NFs, Rezende & Mohamed, 2015; Dinh et al., 2016), a family of likelihood based models that have received relatively little attention in the recent wave of Generative AI. We start from inspecting TARFlow (Zhai et al., 2024), a recently proposed model that combines a powerful Transformer architecture with autoregressive flows (AFs, Kingma et al., 2016; Papamakarios al., 2017). While TARFlow demonstrates promising results on the potential of NFs as a modeling principle, it remains unclear whether it can perform as a scalable method, in comparison to other approaches such as diffusion and discrete autoregressive models. To this end, we propose STARFlow, a family of generative models that shows for the first-time that NF models can successfully generalize to high- resolution and large-scale image modeling. We first provide a thcoretical insight on why AFs can be capable generative models by showing the universality of multi-block AFs in modeling continuous distributions. On top of this, we propose a novel deep-shallow architecture. We found that the architecture configuration, e.g., the number of flows as well as the depth and width of the Transformer for each flow, plays pivotal role to the model's performance. While TARFlow (Zhai et al., 2024) proposes to uniformly allocate model depth among all flows, we found that it beneficial to have a skewed architecture design, where we allocate most of the model parameters to the first AF block (i.e,, the one closest to the prior), which is followed by a few shallow but negligiie blocks. Importantly, our model still yields stand-alone normalizing-flow framework that supports end-to- end maxiimu--likelhhoo training in continuous space, thereby sidestepping the quantization limits inherent to discrete models. Rather than operating directly in data space, we instead learn AFs in the latent space of pretrained autoencoders. Crucially, we demonstrate that NFs align naturally with compressed latents intuitive yet vital obseevvttonneenallii resolution inputs, as verified in our experiments, compared with training directly on pixels. Similar to TARFlow, noise injection proves essential: by fine-tuning the decoder, we train the model on noisy latents and at the same time simplify the original sampling pipeline. Moreover, we revisit the classifier-fi free guidance (CFG) algorithm for AFs from more principled way and propose novel guidance algorithm, which substantially improves image quality, especially at high guidance weights in text-to-image generation tasks.
far superior modeling of high-
Together, these innovations represent the first demonstration of NF models applied to large-scale, high- resolution image generation. Our approach offers scalable and efficient alternative to conventional diffusion- baseid and autoregressive approaches, achieving competitive performance on benchmarks for both class-conditioned image and large- text-to-image synthesis. Moreover, our framework is highly flexible, and we demonstrate that it easily enables interesting settings such as image inpainting and instruction based image editing by finetuning.
2 Preliminaries
2.1 Normalizing Flows
In this paper, we consider Normalizing Flows (NFs, Rezende Mohamed, 2015; Dinh et al., 2014, 2016) as the class of likelihood method that follows the change of variable formula. Given continuous inputs x ~ Pdata x E TRP,, NF learns an invertible transformation so RD - RD (with being the parameters) which maps data into the noise space fo(æ), and can be trained with maximum

---

likelihood estimation (MLE):
max Bx-paass log PNp 0) = log log)o(fe(z)) + log (atet
(Ofo(z)) dx
(L)
where the first term rewards sending data to high-density regions of the prior Po- while the Jacobian term penalizes excessive local volume shrinkage, ensuring the transformation remains bijective and does not collapse nearby points onto a lower-dimensional set. One automatically obtains generative model by inverting fe, with sampling procedure z Po(z). a = fo (z).
2.2 Autoregressive Flows and TARFlow
An interesting variant of NFs is autoregressive flows (AFs, Kingma et al., 2016; Papamakarios et al.,
2017). In the simplest affine form, an AF constructs z = fo(x) = {1g, de (x) as a standalone invertible model with the forward (æ - 2) and sampling (z + x) process:
2d aa ca)) / (x)a),
Fd pa x ca) + a (d) Za, Ad E [I,
(2)
where Ty is constant <sos>. This can be and training with Eq. (I) where the Jacobian term becomes extremely simple as Ed log de (a)a). The extension to multi multi-channel inputs x E R DxC (e.g., C 3 for RGB image) immediate as channels at each step can be treated as conditionally independent. We omit the channel dim henceforth.
"next-token prediction" with affine transformation,
Recently, Zhai al. (2024) introduced TARFlow, compelling framework for building performant NFs for image data. Specifically, TARFlows can be viewed as special form of AFs by pairing Transo-Trassformer blocks with extension of classical AF formulation stacking multiple AF layers whose autoregressive ordering alternates from one layer to the next. To be concrete, with T' flows, we have z f0 own ordering 2a = (C Lap (a permutation of (x)... xD))) enabling the stack to capture dependencies in both directions of the data sequence. Training is still performed end-to-end:
fz(),. (z), where each block f2(.) processes the input in its
T D
max Em-paats logPAF(<; =
- 1| 2
1
ΣΣ
log (a
where xx' = (x'-11) defines the forward propagation (Eq. (2)); we denote the data x a: and the final output xT is modeled with standard Gaussian. Additionally, Zhai al. (2024) also proposed several techniques to improve the modeling capability, including noise augmented training, score-based denoising and incorporating guidance (Ho & Salimans, 2021).
STARFlow
In this section, we propose Scalable Transformer Autoregressive Flow (STARFlow), method that pushes the frontier of NF based high-resolution image generation. We first establish- on theoretical grounds-AFi expressivity as general modeling method in $ 3.1, based on which we propose our core approaches by improving TARFlow several key aspects: (1) better architecture configuration ($ 3.2). (2) working recipe of learning in the latent space ($ 3.3) and (3) novel guidance algorithm
(8 3.4). An illustration of the learning and inference pipeline presented Fig.
3.1 Why TARFlows are Capable Generative Models?
While empirical results confirm that TARFlow is highly competitive (Zhai et al., 2024), we ask-from a modeling perspective-wheteee whether they are expressive enough warrant scaling, Here, we claim:
Stacked autoregressive flows with T blocks of D autoregressive steps and alternating orderings are highly expressive for modeling continuous densities on R.
Proposition 1.
Sketch of Proof. Let's consider T' 2. Without loss of generality, we model fo fā få where fō and 13 employ reversed orderings (forward and backward) for data R::
va = (x)))) oo((za)) yd. ya (3( + oo za, za (O. de [1. D). (4)
3

---

We assume 2 ~ (0, under the base distribution. This yields the autoregressive factorization p(x) [T==1 p( x<dd) follows:
p(as x<aa)= f N(xa po(a U>d). 4od)4) ply x a<d)dysa,
(5)
where Pig = [ + (3
0g
defined in Eq. (4).
For every d < D, we have y>d # 2, so Eq. (5) is latent-variable marginalization of Gaussians (mixture- like) and can represent complex continuous condition- als. For the final coordinate d= we have y Ø. Eq. (5) reduces to single Gaussian and the expres- sivity is restricted. This restriction can be lifted by extending additional augmented variables. Moreover, the above derivation only uses the base assumption Zd (0, 1) in the generative direction. In general, conditioning on observed coordinates induces a non- Gaussian latent distribution (i.e., (= æ<<a) is not necessarily Gaussian). Consequently, the resulting con- ditional go ( <<D) can be even more complex than single Gaussian. Additional derivation details appear in the Appendix A.
21
Zd
Zd+1
Za
Y1
Ya
Ya+1
Yo
X1
Xa
Xd+1
Xo
Figure 2: An illustration of 2-block AFs.
The preceding proposition clarifies why we can safely scale-up AFs on large data. Even in the minimal setting
where full universality is not attained, the resulting limitation negligible in high-dimensional domains such natural images.
3.2 Proposed Architecture
The derivation in & 3.1 motivates a redesign of scalable AF architectures within real- istic computational budgets, emphasizing that we need not greatly expand the num- ber of flow blocks -nndeed (even T' 2 often suffices). However, the remark leaves unresolved how best to allocate com- pute across those blocks. We first inspect the proposed architecture configuration in TARFlow, which suggests allocate equal sized Transformer layers for each flow, In- terestingly, our reproduced TARFlow re- sults, we see that most effective compute (measured through the lens of guidance) concentrates in just the top few AF blocks (see motivating examples Fig. 3). We conjecture that end-to-end training drives the network to exploit layers closest to the noise, behavior that contrasts that of diffusion models.
Figure 3: Top to bottom, guiding the first 0, 3, 8 flow blocks
with TARFlow model with 8 flow blocks. We see that guidance is only effective up to the top blocks.
be intuitively considered standard autoregressive language models (e.g., LLaMA (Dubey et al., 2024)) with a general deep- shallow design. At inference time, deep AF block first autoregressively generates x! from noise followed by sequence of shallow AF blocks that iteratively refine it to xV", all while keeping the total number of blocks T small. Given total depth budget L, instantiate the model as I(T): one deep l-layer block and T-1 shallow 2-layer blocks, satisfying L = / + -1). This asymmetric design turns the deep block into a Gaussian language model, while the shallow stack plays the role of learned image tokenizer.
Deep-shallow Architecture Our architecture
extension of
Conditional STARFlow This design naturally extends to conditional generation by simply prepend- ing the control signal (e.g., class label, caption) to the input of the flow. Interestingly, our preliminary experiments show that conditioning only the deep blck-while leaving the shallow blocks to focus solely on local image refinement-incurs no loss performance. This not only simplifies the overall architecture, but also enables seamless initialization of the deep block with any pre- pre-trained language

---

xf
Pg
Shallow Tranatormer (eo
Shall one Transtorme Block (e)
Right
<t>>
Synthesized Imaga
Image
Corg
Dog
Pg
Corpl
Doa
(O I)
Log Prob(.)
Sampling(2-X)
x no (NG)e)
Training (x-z)
Σ
log of (xtee)
An illustration of the autoregressive inference (left) and parallel training (right) process of proposed model text-to-image generation. The upward (green) and downward (purple) arrows refers to inverse and forward AF step as shown in Eq. (2).
Figure 4:
model, without major modifications. As result, our image generator can be directly integrated into any LLM's semantic space, eliminating the need for separate text encoder.
3.3 Moving to Latent Space
Analogy Stable Diffusion (SD), Rombach et al., 2022) w.r.t standard diffusion models, STARFlow directly models the latent space of a pretrained autoencoders æ z D c {(x), enabling high- resolution image generation. For instance, when using SD-1.4 autoencoder', one can reduce input shape from 256 x 256 to 32 x 32. As noted by Zhai et al. (2024), injecting proper amount of Gaussian noise, instead of small dequantization noise Dinh al. (2016); Ho al. (2019), is crucial for stable training and high quality sampling. This then makes it necessary to perform an additional score-based denoising step to clean up the noise components in the samples Zhai al. (2024).
In the context of latent normalizing flows, however, the added noise becomes an integral component of the latent representation. Specifically, encode each sample as i Jone N (E(x); of We perform preliminary search for the noise scale (aTz) to based on the choice of autoencoders. For ø 0.3 throughout the paper.
example,
Learning Learning in the latent space leaves additional flexibility that the flow model can focus on high-level semantics and leave the low-level local details with the pixel decoder. In this way, acts learnable prior for the latents. Following VAEs (Kingma & Welling, 2013), we optimize the entire model by maximizing the evidence lower-bound (ELBO) where the entropy term is constant:
max Eag Rene arreiita [log 0) + log Paec (z ã σ) log gene (x|2)), 0,6
(6)
where @ are the parameters of decoder Paec which transforms the noisy latents back to the pixel space. Here, jointly train the AF prior and pixel decoder, freezing the encoder distribution -as in SD>, which stabilizes training and decouples their optimization. Relaxing the encoder Jeuc and training with the full ELBO loss including entropy regularization are left for future work.
Pixel Decoder As shown in Eq. (6), the prertaiend decoder has to be adapted in order to decode from the noisy latents. Different from Zhai al. (2024) which relies on gradient-base denoising, modeling in the latent allows simpler solution by directly fine-tuning the decoder over noisy latents:
min L (D(E(< oe);
where following Esser et al. (2021), L Cirrs + 3LAAN. We empirically observe consistently better performance than score-based denoising technique proposed (Zhai et al., 2024), with FID decreasing from 2.96 to 2.40 on ImageNet-256. See Appendix C for more discussions.
'attps://huggiingaaca http://g......a..ooooaaaaaaaaa//
5

---

w=8
WF2 W
w=4.5 5
W=B
(a)
(b)
Figure 5: (a) Guidance from TARFlow (Zhai al., 2024) (b) Proposed guidance on ImageNet 256 256.
3.4 Revisiting Classifier-Free Guidance for Autoregressive Flows
Classifier-free guidance (CFG), originally introduced for diffusion models (Ho & Salimans, 2021). has become a cornerstone in modern generative modeling, proving broadly effective across various architectures, including AR models (Yu et al., 2022). At high level, CFG amplifies the difference between conditional and unconditional predictions, encouraging more mode-secking behavior.
In the context of AFs, Zhai et al. (2024) made the first attempt to apply CFG by linearly extrapolating the mean and variance at each step (Eq. (2)): Pe Me + w(fe - Pa) and o oe + w(a, oa)²,
where w > 0 denotes the guidance weight. While effective to some extent, this naïve formulation lacks principled justification, leaving unclear how μ and should be jointly modulated under guidance. Furthermore, as shown in Fig. 5, this approach becomes unstable at high guidance weights-precisse the regime required for visually compelling results in text-to-image generation.
We propose to revisit CFG from the perspective of score function, the original intuition of Ho & Salimans (2021). In short, we want to sample from guided distribution ğ which score satisfies:
V log (a()) V lo + w (V (o)) V (x))
generally non- non-trivial to determine P. for every flow block. Fortunately, under the design of our proposed model, guidance is only required in the deep block, which functions as Gaussian Language Model ($ 3.2). Therefore, Eq. (8) can be easily simplified into the following:
Proposition 2. also Gaussian Pe = (pcc I.. and satisfies:
Given Pa. N( I),, and pe - NNess I),,, the guided distribution p. is
ws
1
ce Me +
(Ne ua), δe Vl+ ws
1 w - ws
where 8 =
=00// 02/0 and w > 0.
proof: A detailed derivation is provided in the Appendix A.
Notably, when a. Gu. Eq. (9) reduces the standard CFG used in diffusion models. However, directly applying Eq. (9) can lead to severe numerical instability, as the denominator 1 + w ws may approach zero or even become negative. To address this, we propose clipping via 8 CLIP(s. (s. 0, 1). motivated by the intuition that the guided distribution should be more mode-seeking than the original, implying that + w
ws 2 for any w, therefore S
3.5 Applications
STARFlow versatile generative model that not only produces diverse, high-quality images under various conditions but also extends naturally to downstream applications. We showcase two examples: image inpainting and editing.
We use and denote the conditional and unconditional predictions, respectively.

---

Training-Fres Inpainting We first map the masked image the latent space, replacing masked regions with Gaussian noise, Reverse sampling then performed, restoring unmasked pixels with ground truth. We perform generation iteratively until the final inpainted output.
A corgi dog
wearing red hat
STARF1oW
nasked Input
caption
(a)
Inpainted output
Deep Conditional Autoregressive Flow
Interactive Generation and Edit- ing We finetune STARFlow on an image editing dataset (Fig. 6b), abling joint modeling of generation and editing with single conditional AF model. Its invertibility also allows direct image encoding, making suit- able for interactive use.
TS-XL
Shallow AFS
TS-XL
Shallow AFs
teddy bear sat cable
Change the bear's color to red
(b)
Figure 6: (a) image inpainting (b) interactive editing.
Experiments
4.1 Experimental Settings
Dataset We experiment with STARFlow on both class-conditioned and text-to-image generation tasks. For the former, conduct experiments on ImageNet-IK (Deng al., 2009) including 256 x 256 and 512 x 512 resolutions. For text-to-image, we show two settings: a constrained setting CC12M (Changpinyo al., 2021), where each image accompanied by a synthetic caption following (Gu et al., 2024a). We also demonstrated a scaled setting where our models trained an in-house dataset with CC12M, in total 700M text-image pairs.
Evaluation In line with prior works, we report Fréchet Inception Distance (FID) (Heusel et al., 2017) to quantify the the realism and diversity generated images. For text-to-image generation, we use MSCOCO 2017 (Lin et al., 2014) validation set to assess the zero-shot capabilities of these models. We also report additional evaluation (e.g., GenEval (Ghosh al., 2023)) in Appendix C.
Model and Training Details We implement all models following the setup of Dubey et al. (2024), using RoPE (Su et al., 2024) for positional encoding. By default, we set the architecture to d(NN 18(6) with model dimension of 2048 (XL) and 24(6) with dimension of 3096 (XXL) for class- conditioned and text-to-image models, respectively ($ 3.2), resulting in 1 1.4B and 3,8B parameters. Since STARFlow operates in compressed latent space, are able to train all models with patch size of 1. For text-to-image models, we use XL (Raffel et al., 2020) as the text encoder. To showcase the generality of our approach, from a pretrained LLM (Gemma2 (Team et al., 2024) in this case), without additional text encoder.
also train variant where the deep block is initialized
All models are pre-trained at 256 x 256 resolution on 400M images with global batch size of 512. High-resolution finetuning is done by increasing input length. For text-to-image models, variable- length inputs are supported via mixed-resolution training: images are pre-classified into 9 shape buckets and flattened into sequences for unified processing. See Appendix B for detailed settings.
4.2 Results
Comparison with Baselines We benchmark our approach on class-conditioned ImageNet-256. comparing against diffusion and autoregressive models across both discrete and continuous domains (Table 1). For fair comparison, we train a TARFlow model Zhai al. (2024) in pixel space with similar parameter count and original architecture (8 flows, 8 layers each, width 1 1280). We also train a variant with our shallow design, identical to STARFlow except for using pixel inputs with linearly scaled patch sizes. Among NF models, the deep-shallow architecture consistently outperforms the standard design, and switching to latent-space inputs yields further gains. Our method achieves competitive results compared to other baselines (Tables 1 and 2). Note the FID on ImageNet 256 x 256 is near saturated to the upper- bound of the finetuned decoder (see additional details in Appendix B). Zero-shot evaluations on COCO (Table 3) show strong performance
7

---

Figure 7: Random samples of STARFlow on ImageNet 256 256 and 512 x 512 (w 3.0).
Table 1: Class-cond ImageNet 256x256 (FID-50K)
Table 2: Class-cond ImageNet 512x512 (FID-50K)
Model
FID Param.
Model
FID # Param.
Diffusion Models
ADM-U (Dhariwal & Nichol, 2021)
3.85
731M
ADM (Dhariwal Nichol, 2021)
10,94
554M
DiT-XL/2 (Peebles & Xie, 2023)
3.04
674M
CDM (Ho et al., 2022b)
4.88
LEGO (Zheng al., 2024b)
3.74
681M
LDM (Rombach al., 2022)
3.60
400M
MaskDiT-G (Zheng et al., 2024a)
2.50
730M
RIN (Jahri et al., 2022)
3.76
410M
EDM2-XXI. et al., 2024)
1.25
1.5B
DIT (Peebles & Xie, 2023)
2.27
675M
STARFlow (Ours)
3.00
1.4B
(Ma et al., 2024)
2,06
675M
Autoreg. (discrete)
Table 3: Zero-shot on COCO (FID-30K)
VQGAN (Esser et al., 2021)
15.78
1.4B
Method
FID # Param.
RQTran (Lee et al., 2022)
3.80
3,8B
LlamaGen-3B (Sun et al., 2024)
2.18
3.1B
DALL-E (Ramesh al., 2021)
27.5
12B
VAR (Tian et al., 2024)
1.73
2.0B
CogView2 (Ding et al., 2021)
24.0
6B
Make-A-Scene (Gafni et al., 2022)
11.8
Autoreg. (continuous)
DART (Gu et al., 2024b)
11.1
800M
Jetformer (Tschannen et al., 2024b)
6.64
2.75B
DALL-E (Ramesh et al., 2022)
10.4
5.5B
MAR-AR (Li et al., 2024)
4.69
479M
GigaGAN (Kang et al., 2023)
9.1
1B
MAR (Li et al., 2024)
1.55
943M
Muse (Chang et al., 2023)
7.9
DART (Gu et al., 2024b)
3.82
820M
Imagen (Ho ct al., 2022a)
7.3
3B
GIVT (Tschannen al., 2024a)
2.59
Parti-20B (Yu al., 2022)
7.2
20B
Normalizing Flow
eDiff-I (Balaji et al., 2022)
7.0
9B
TARFlow (Zhai al., 2024)*
5.56
1.3B
STARFlow-CCH!
10.3
3.8B
TARFlow deep-shallow
4.69
1.4B
TTARFlow-CCI2M-OO -22mmaa
11.4
2.4B
STARFloW (Ours)
2.40
1.4B
STuRFlow-FullData
9.1
3.8B
'Implemented using their official codebase.
text-conditioned generation, demonstrating that NFs can also serve as a scalable and competitive generative modeling framework.
Qualitative Results Fig. 7 and Appendix Fig. 10 present representative class- and text-conditioned generations, respectively. Our method delivers high-resolution images over a wide range of aspect ratios, with perceptual quality comparable to state-of- of-te-of-the-art diffusion and autoregressive approaches. Fig. also highlights our model's support for image editing, Further qualitative and interactive editing results appear in Appendix G, underscoring the breadth and fidelity of our outputs.
Comparison with Diffusion and Autoregressive Models We further compare STARFlow with diffusion and autoregressive (AR) models to analyze training dynamics. Fig. 8a shows FID trajectories

---

2AOOHM,, 1-10
OT DROMA 2:30
Start low (20/2940 1/D1
35
00 De stens
37
Row
160
180m
Inai ng images
Eairing images
satch Siees
(a) Generation Quality (4096 and 50K samples) and Speed Comparison with DiT and AR
uniiiaa Songhh Ae- saainn
4422
12.00
Ouiconns Scolt (w))
Torrng Images
Traekng Brages
(b) Original vs. Proposed CFG
(c) NLL Model size
(d) FID vs. Model size
Hode Corfo
14.5
Nobi Confa
Mode
1921
6111
1992
18:41
20-41
1290
L1.8
1891
2801
1155
20 4000
001
J0M 330m 100 MMM U6B M LS0 203
28 4016 C2M 30Y 10G 120 M 180m 2300
20 4E 66
JCM 1307 1209 280M 280 280R 2009
(e) Impact of varying the number of layers within deep blocks.
(1) Ablation of parameter allocation.
Figure& 8: Experimental results comprehensive ablation study
Figure 9: Example of Image editing using STARFlow. Given an input image and simple description, our model can seamlessly edit the contents based on various instruction using with the learned model prior.
using nearly identical architectures. While the FID gap between STARFlow and the baselines is smaller when computed over 4,096 samples. STARFlow consistently achieves the lowest FID every training checkpoint when evaluated with 50,000 samples. This suggests that STARFlow produces more diverse outputs, which may not be fully captured with smaller evaluation sets.
Fig. 8a also compares inference throughput on a single H100 GPU for diffusion, AR, and STARFlow models. Diffusion's 's wall-clock time grows linearly with its number of refinement steps- 250 steps at best FID-so i's the slowest. By contrast, each step in AR and STARFlow is only a lightweight forward pass whose per-token cost is low, allowing throughput rise as batch size increases. Beyond a batch size of 32, STARFlow outperforms the AR baseline by restricting guidance to the deep block and removing the per-token multinomial sampling loop, yielding superior inference-time scalability.
Comparison of CFG Strategies As shown the Fig. 8b, the original strategy used in Zhai et al. (2024) exhibits sharp "dip-and-spike" behavior: it achieves its best FID at similar guidance weight as the newly proposed CFG, but then degrades quickly as you move away from that optimum. Even when using the "annealing trick" (Zhai et al., 2024), performance still suffers dramatically both

---

scales. By contrast, our proposed CFG not only improves on the original's best point without additional tricks-but-aa more importanly-maintee nearly the same quality over a much wider range of guidance weights, which gives more flexibility in tuning text-conditioned generation tasks.
Scalability Analysis To assess the scalability, we perform a study by varying the depth of the deep block and tracking performance Fig. 8d shows FID with 4096 samples across iterations. Both metrics indicate that deeper models converge faster and achieve better final performance, demonstrating the increased capacity.
training. Fig. 8c reports negative log-likelihood (NLL) and
Ablation on Model Design To validate the theoretical insights from Prop. 1, study how model expressivity varies with the number of layers T in the deep block. Performance drops sharply when with Prop. 1. We also ablate the
T < while models with T 2 perform similarly-consiseer number and depth of deep blocks in Figs. 8e and 8f, finding that block depth is more critical than quantity, providing practical guidance for architectural design.
Related Work
Continuous Normalizing Flows, Flow Matching, and Diffusion Models Normalizing Flows (NFs) can be extended to continuous-time via Continuous Normalizing Flows (CNFs) (Chen et al., 2018), which model transformations as ODEs. This relaxes the need for explicit invertible mappings and simplifies Jacobian computation to a trace (Grathwohl et al., 2018), though it requires noisy stochastic estimators (Hutchinson, 1989). Flow Matching (Lipman et al., 2023), inspired by CNFs, learns sample-wise interpolations between prior and data using vector fields grounded in Tweedie's Lemma (Efron, 2011), 1). While CNFs and NFs optimize exact likelihoods through invertible mappings, Flow Matching aligns more closely with diffusion models, sharing variational training objectives.
Autoregressive Models Discrete autoregressive models, especially large language models (Brown et al., 2020; Dubey et al., 2024; Guo et al., 2025), dominate modern generative AI by scaling next-token prediction. Scaling laws (Kaplan et al., 2020) show predictable gains with more data and parameters. These models now power leading multimodal systems for both understanding and generation (Liang al., 2024; Sun et al., 2024; Tian et al., 2024; Li al., 2025).
To overcome information loss from quantization, recent work extends AR modeling to continuous spaces, using mixture-of- -ixture-of-Gaussians (Tschannen et al., 2024a,b) or diffusion decoding (Li et al., 2024; Gu et al., 2024b; Fan et al., 2024). Hybrid approaches also emerge, unifying AR and diffusion paradigms (Gu et al., 2024a; Zhou et al., 2024; OpenAl, 2024).
6 Conclusion and Limitation
We have presented STARFlow, the first latent based normalizing flow model that scales to high resolution images and large scale text to image modeling. Our results demonstrate that normalizing flows are scalable generative modeling method, and is capable of achieving comparable results to strong diffusion and autoregressive baselines.
There are also limitations to our work. For example, we have exclusively relied on pretrained autoencoders for simplicity, but it leaves the question of a potential joint latent-No NF model design unexplored. Moreover, this work we have primarily focused on training high-quality models, which comes at the cost of un-optimized inference speed. Additionally, our evaluation has been restricted to class- and text-conditional image generation on standard benchmarks; how well the approach generalizes to other modalities (e.g., video, 3D scenes) more diverse, real-world data distributions remains to be seen.
Acknowledgements
We thank Ying Shen, Yizhe Zhang, Navdeep Jaitly, Alaa El-Nouby and Preetum Nakkiran for helpful discussions. We also thank Samy Bengio for leadership support that made this work possible.
10

---

References
Balaji, Y., Nah, S., Huang, X., Vahdat, A., Song, J., Kreis, K., Aittala, M., Aila, T., Laine, S., Catanzaro, B., et al. ediffi: Text-to-mmaee diffusion models with ensemble of expert denoisers. arXiv preprint arXiv:2211.01322 2022.
Betker, J., Goh, G., Jing, L., Brooks, T., Wang. J., Li, L.. Ouyang, L.. Zhuang, J., Lee, Guo, Y., et al. Improving image generation with better captions. Computer Science. https:///dn. openai. com/papershlall- e-33. pdf. 2(3):8, 2023.
Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam,
P. Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler,
M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., and Amodei, D. Language models are few-shot learners. Advances neural information processing systems, 2020.
Chang, H., Zhang, H., Barber, Maschinot, A., Lezama, J., Jiang, L., Yang, M.-H., Murphy, K. P., Freeman, W. T., Rubinstein, M., Li, Y., and Krishnan, D. Muse: Text-to-image generation via masked generative transformers. In Krause, A., Brunskill, E., Cho, K., Engelhardt, B., Sabato, S., and Scarlett, J. (eds.), Proceedings of the 40th International Conference on Machine Learning, volume 202 of Proceedings of Machine Learning Research, pp. 4055-4077 PMLR. 23-29 Jul 2023. URL https: // //tppeeeeeeengseee mmr.............mmmm.m.......... press/ /asess/222//// chang2bh 2atmnn1 html.
Changpinyo, S., Sharma, P., Ding, N., and Soricut, R. Conceptual 12M: Pushing web-scale image-text pre-training to recognize long-tail visual concepts. In CVPR, 2021.
Chen, J., Yu, J., Ge, C., Yao. L., Xie, E., Wu, Y., Wang, Z., Kwok, J., Luo, P., Lu, H., et al. Pixart- alpha: Fast training of diffusion transformer for photorealistic text-to-image synthesis. arXiv preprint arXiv:2310.00426, 2023.
Chen, R. T., Rubanova, Y., Bettencourt, J., and Duvenaud, D. K. Neural ordinary differential equations. Advances neural information processing systems, 31, 2018.
Chung, H. W., Hou, L., Longpre, S., Zoph, B., Tay, Y., Fedus, W., Li, E., Wang. X., Dehghani, M., Brahma, S., Webson, A., Gu, S. S., Dai, Z., Suzgun, M., Chen, X., Chowdhery, A., Narang, S.,
Mishra, G., Yu, A., Zhao, V., Huang. Y., Dai, A., Yu, H., Petrov, S., Chi, E. H., Dean, J., Devlin, J..
Roberts, A., Zhou, D., Le, Q. V., and Wei, J. Scaling instruction- finetuned- language models, 2022.
URL https:tt... testts://arrzn httpes//aaxxvvvvveee wwweiiii.rr//baaa httg//b//200 /2100. 11416.
Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., and Fei-Fei, L. ImageNet: Large-scale Hierarchical Image Database. IEEE Conference on Computer Vision and Pattern Recognition, pp. 248-255, 2009.
Dhariwal, P. and Nichol, A. Diffusion models beat gans image synthesis. Advances in Neural Information Processing Systems, 14:8780-8794, 2021.
Ding, M., Yang, Z., Hong, W., Zheng, W., Zhou, C., Yin, D., Lin, J., Zou, X., Shao, Z., Yang, H., et al. Cogview: Mastering text-to-image generation via transformers. Advances in Neural Information Processing Systems, 34: 198829992998, 2021.
Dinh, L., Krueger, D., and Bengio, Nice: Non-linear independent components estimation, arXiv preprint 4rXiv:1410.8516 2014.
Dinh, L., Sohl-Dickstein. J., and Bengio, S. Density estimation using real nvp. arXiv preprint 1rXiv:1605.08800 2016.
Dubey, A., Jauhri, A., Pandey, A., Kadian, A., Al-Dahle, A., Letman, A., Mathur, A., Schelten, A., Yang, A., Fan, A., et al. The llama herd of models. arXiv preprint arXiv:2407.21788. 2024.
Efron, B. Tweedie's formula and selection bias. Journal of the American Statistical Association, 106 (496):1602-1))) 1614, 2011.
Esser, P., Rombach, R., and Ommer, B. Taming transformers for high-resolution image synthesis. In Proceedings of the IEFF/CVE Conference on Computer Vision and Puttern Recognition, pp-
12873-12883, 2021.
11

---

Esser, P.. Kulal, S., Blattmann, A., Entezari, R., Müller, J., Saini, H., Levi, Y., Lorenz, D., Sauer, A., Boesel, F., et al. Scaling rectified flow transformers for high-resolution image synthesis. In Forty-first International Conference on Machine Learning, 2024.
Fan, L., Li, T., Qin, S., Li, Y., Sun, C., Rubinstein, M., Sun, D., He, K., and Tian, Y. Fluid: Scaling autoregressive iext-to-image generative models with continuous tokens. arXiv preprint 11Xiv:2410.13863, 2024.
Gafni, O., Polyak, A.. Ashual, O., Sheynin, S., Parikh, D., and Taigman, Y. Make-a-scene: Scene- based text-to-image generation with human priors. 2022. doi: 1044850AARXY::20011111.. URL
https: /tttps://axiv...o org/ /bss/220. 13131.
Ghosh, D., Hajishirzi, H., and Schmidt, I.. Geneval: An object-foeused framework for evaluating text-to-image alignment. Advances in Neural Information Processing Systems, 36:52132-521522 2023.
Goodfellow, I., Bengio, Y., and Courville, A. Deep Learning. MIT Press, 2016. http://www. deeplearningbook.c org.
Grathwohl, W. Chen, R. T., Bettencourt, J., Sutskever, I,, and Duvenaud, D. Ffjord: Free-form continuous dynamics for scalable reversible generative models. arXiv preprint 180.0:1110.01367, 2018.
Gu, J., Shen, Y., Zhai, S., Zhang, Y.. Jaitly, N., and Susskind, J. M. Kaleido difffusio:: Improving conditional diffusion models with autoregressive latent modeling. arXiv preprint arX1v22405.21048, 2024a.
Gu, J., Wang, Y., Zhang, Y., Zhang, Q.. Zhang, D., Jaitly, N., Susskind, J., and Zhai, S. Dart: Denoising autoregressive transformer for scalable text-to-image generation. arXiv preprint arXiv:2410.08159, 2024b.
Guo, D., Yang, D., Zhang, H., Song. J., Zhang, R., Xu, R., Zhu, Q. Ma, S., Wang, P., Bi, X., et al. Deepseek-rl: Incentivizing reasoning capability Ilms via reinforcement learning. arXiv preprint arXiv:2501.12948, 2025.
Heusel, M., Ramsauer, H., Unterthiner, T., Nessler, B., and Hochreiter, S. Gans trained by a two time-scale update rule converge to local nash equilibrium. Advances in neural information processing systems, 30, 2017.
Ho, J. and Salimans, T. Classifier-free diffusion guidance. In NeurIPS 2021 Workshop on Deep Generative Models and Downstream Applications, 2021.
Ho, J., Chen, X., Srinivas, A., Duan, Y., and Abbeel, P. Flow++: Improving flow-based generative models with variational dequantization and architecture design. In International conference on machine learning. pp. 2722-22222222 2730. PMLR, 2019.
Ho, J., Jain, A., and Abbeel, Denoising diffusion probabilistic models. Advances in Neural Information Processing Systems, 33:6840-6851, 2020.
Ho, J., Chan, W., Saharia, C., Whang, J., Gao, R., Gritsenko, A., Kingma, D. P., Poole, B., Norouzi, M., Fleet, D. J., et al. Imagen video: High definition video generation with diffusion models. arXiv preprint N.XXi:22200023000 2022a.
Ho, J., Saharia, C., Chan, W., Fleet, D. J., Norouzi, M., and Salimans, T. Cascaded diffusion models for high fidelity image generation. J. Mach. Learn. Res., 23:47-1, 2022b.
Hutchinson, M. F. A stochastic estimator of the trace of the influence matrix for laplacian smoothing and Computation, 1859) 1989.
splines. Communications
Jabri, A., Fleet, D., and Chen, T. Scalable adaptive computation for iterative generation. arXiv preprint wtXi::2212.119722 2022.
Kang, M., Zhu, J.-Y., Zhang, R., Park, J., Shechtman, E., Paris, S., and Park, T. Scaling up gans for text-to-image synthesis. In Proceedings of the IEEF/CVF Conference on Computer Vision and Pattern Recognition, pp. 10124-10134, 2023.
12

---

Kaplan, J., McCandlish, S., Henighan, T., Brown, T. B., Chess, B., Child, R., Gray, S., Radford, A.. Wu, J., and Amodei, D. Scaling laws for neural language models. arXiv preprint XXix:2001000866.. 2020.
Karras, T., Aittala, M., Kynkäänniemi, T., Lehtinen, J., Aila, T., and Laine, S. Guiding a diffusion model with bad version of itself. Advances in Neural Information Processing Systems, 37: 52996-53001 2024.
Kingma, D. P. and Welling, M. Auto-encoding variational bayes. arXiv preprint arXiv: 1312.611 2013.
Kingma, D. P., Salimans, T., Jozefowicz, R., Chen, X., Sutskever, I., and Welling, M. Improved variational inference with inverse autoregressive flow. Advances neural information processing systems, 29, 2016.
Lee, D., Kim, C., Kim, S., Cho, M., and Han, W.-S. Autoregressive image generation using residual quantization. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 11523-11532 2022.
Li, T., Tian, Y., Li, H., Deng, M., and He, K. Autoregressive image generation without vector quantization. arXiv preprint arXiv:2406.1 11838, 2024.
Li, T., Sun, Q., Fan, L., and He, K. Fractal generative models. arXiv preprint arXiv:2502.14453 2025.
Liang Z., Xu, Y., Hong, Y., Shang, P., Wang. Q., Fu, Q., and Liu, K. survey of multimodel large language models. In Proceedings of the 3rd International Conference Computer, Artificial Intelligence and Control Engineering, pp. 405-409, 2024.
Lin, T.-Y., Maire, M., Belongie, S., Hays, J., Perona, P., Ramanan, D., Dollár, P., and Zitnick, C. L. Microsoft COCO: Common Objects in Context. European Conference on Computer Vision, pp. 740-755, 2014.
Lipman, Y., Chen, R. T. Q., Ben-Hamu, H., Nickel, M., and Le, M. Flow matching for generative modeling. In The Eleventh International Conference on Learning Representations, 2023. URL https:// https://openreeiest net wtt.///rtn
Ma, N.,
Goldstein, M., Albergo, M. S., Boffi, N. M., Vijnen, E., and Xie, S. Sit: Exploring flow and diffusion based generative models with scalable interpolant transformers. arXiv preprint arXiv:2401. 08740,, 2024.
OpenAl. Gpt-4o system card. https://openai: cww http://aw...ooiiiiaee/////oo-/-s Accessed: April 12, 2025.
2024.
Papamakarios. G., Murray, and Pavlakou, T. Masked autoregressive flow for density estimation. In Guyon, I., von Luxburg, U., Bengio, S., Wallach, H. M., Fergus, R., Vishwanathan, S. V. N., and Garnett, R. (eds.), Advances in Neural Information Processing Systems 30: Annual Confer- ence on Neural Information Processing Systems 2017, December 4-9, 2017, Long Beach, CA, USA, pp. 2338-2347, 2017. URL https: httttss:/ppocceeiinnggeoooo 0c118866822266882223333377888860 Abstteeeeaaaaaaaaatttaeaacaaaace
httpippp..ccpppp////1111 www..neeiiiiccmmm cc/ ppppp//pp1/11//// //ash// Abstract html.
Peebles, W. and Xie. S. Scalable diffusion models with transformers. In Proceedings of IEEE/CVF International Conference on Computer Vision, pp. 4195-4200, 2023.
Podell, D., English, Z., Lacey, K., Blattmann, A., Dockhorn, T., Müller, J., Penna, J., and Rombach,
R. Sdxl: improving latent diffusion models for high-resolution image synthesis. arXiv preprint arXiv:2307.01952 2023.
Raffel, C., Shazeer, N., Roberts, A., Lee, K.. Narang, S., Matena, M., Zhou, Y., Li, W. and Liu, P. J. Exploring the limits of transfer learning with a unified text-to-text transformer. Journal of machine learning research, 21(140):1-67 2020.
Ramesh, A., Pavlov, M., Goh, G., Gray, S., Voss, C.,, Radford, A., Chen, M., and Sutskever, I. Zero- shot text-to-image generation. International Conference on Machine Learning, pp. 8821-8831. PMLR, 2021.
13

---

Ramesh, A., Dhariwal, P., Nichol, A., Chu, C., and Chen, M. Hierarchical text-conditional image generation with clip latents. arXiv preprint arXiv:2204.06125, 2022.
Rezende, D. and Mohamed. S. Variational inference with normalizing flows. In Bach, F. and Blei,
D. (eds.), Proceedings of the 32nd International Conference Machine Learning, volume 37 of Proceedings of Machine Learning Research, pp. 1530-1553 1538,, Lille, France, 07-09 Jul 2015. PMLR. URL https: / https::/proceeeeeeeeeee /ttp//pprceeen.....////// mlr, press v37 v37/rezende15 html.
Rombach, R., Blattmann, A., Lorenz, D., Esser, P., and Ommer, B. High-resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF conference on computer vision and paltern recognition, pp. 10684-10695, 2022.
Su, J., Ahmed, M., Lu, Y., Pan, S., Bo, W., and Liu, Roformer: Enhanced transformer with rotary position embedding. Neurocomputingg
2024.
Sun, P., Jiang, Y., Chen, S., Zhang, S., Peng. B., Luo, P., and Yuan, Z. Autoregressive model beats diffusion: Llama for scalable image generation. arXiv preprint (it.::2000006222. 2024.
Team, C. Chameleon: Mixed-modal carly-fusion foundation models, 2024.
Team, G., Riviere, M., Pathak, S., Sessa, P. G., Hardin, C., Bhupatirajj, S., Hussenot, L., Mesnard,
T., Shahriari, B., Ramé, A., et al. Gemma 2: Improving open language models at practical size. arXiv preprint arXiv:2408.000 000 18, 2024.
Tian, K., Jiang, Y., Yuan, Z., Peng, B., and Wang, L. Visual autoregressive modeling: Scalable image generation via next-scale prediction, arXiv preprint arXiv:2404.0290. 2024.
Tschannen, M., Eastwood, C., and Mentzer, Givt: Generative dufnite-vocabulayy transformers. In European Conference on Computer Vision, pp. 292-309. Springer, 2024a.
Tschamnen, M., Pinto, A. S., and Kolesnikov, A. Jetformer: An autoregressive generative model of raw images and text. arXiv preprint arXi::2411.1972) 2024b.
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., and Polosukhin, I. Attention Is All You Need. Advances in Neural Information Processing Systems, pp. 6998-6008, 2017.
Wang, X., Zhang, X., Luo, Z., Sun, Q., Cui, Y., Wang, J., Zhang, F., Wang. Y., Li, Z., Yu, Q., et al. Emu3: Next-token prediction is all you need. arXiv preprint arXiv:2409. 18869, 2024.
Xie, J., Mao, W., Bai, Z., Zhang, D. J., Wang, W., Lin, K. Q., Gu, Y., Chen, Z., Yang. Z., and Shou, M. Z. Show-o: One single transformer to unify multimodal understanding and generation. arXiv preprint arXiv:2408. /2528, 2024.
Yu, J., Xu, Y., Koh, Y., Luong, T., Baid, G., Wang, Z., Vasudevan, V., Ku, A., Yang, Y., Ayan, B. K., et al. Scaling autoregressive models for content-rich text-to-image generation. Transactions on Machine Learning Research, 2022.
Zhai, S., Zhang, R., Nakkiran, P., Berthelot, D., Gu, J., Zheng, H., Chen, T., Bautista, M. A., Jaitly, N., and Susskind, J. Normalizing flows are capable generative models. arXiv preprint arXiv:2412.06322 2024.
Zheng, H., Nie, W., Vahdat, A., and Anandkumar, A. Fast training of diffusion models with masked transformers. In Transactions on Machine Learning Research (TMLR), 2024a.
Zheng, H., Wang, Z., Yuan, J., Ning, G., He, P., You, Q. Yang, H., and Zhou, M. Learning stackable and skippable LEGO bricks for efficient, reconfigurable, and variable-reser reeiable-rsoolutionn diffusion modeling. The Twelfth International Conference on Learning Representations, 2024b. 20244b. URL https: htttp:://ppnnrevee
Zhou, C., Yu, L., Babu, A., Tirumala, K., Yasunaga, M., Shamis, L., Kahn, J., Ma, X., Zettlemoyer, L., and Levy, O. Transfusion: Predict the next token and diffuse images with one multi-modal model, arXiv preprint or 4r0iv:2408.1 1039, 2024.
14

---

Appendix
This Fine-
flnor (580:284); rofnforest ecosyster, complete with tiny vaterfalls, esotic plants, taall antwals 1ike fross and bitterfltes, the glass veflecting Light froe morty winsow. (8805384); british shorthair sitton playing with yorn in rone batched in sunlight (1241154)) seiling car aartonn zog zits tabie, coffee nag nars,
origant
middle
with
mintature
goss riams. 11820110291:
is
birsilt. (386x6a0).
apple foles
sinielist bsokground, craftsmwtsni,, high-reselation scutll lighting. (416x524): detalled seinting of Atlantis, festuring ietricate detailing vibrant colora (25asa50); 9. A vagorsus coral eutrosolis embedsed in frazen time vivid katercoior blase (2840480); 10, A natarestor painting of otbrant flnser field spring, with rainow slessees. 12645764): Lighthouse weiteing raintos teans into coastst fog, vateronlor illustration, bashed golden light (339x784); ornate golden (S125512);
cartoon (35a12sa):
pastel pzinting
engineer biniurdnessen: pool.
(286-791).
Figure 10: Selected samples of various aspect ratios from STARFlow for text- text-to-image generation (w 4.0).
Image resolutions are adjusted proportionally the case of visualization.
15

---

A Derivations
A.1 Extended Discussion of Prop. 1
Why a Single Block (7 = Cannot Be Universal.
With only one autoregressive-flow block, Dd = (o((2a)) + (o( Zd. 2d ~ 1), d = 1,..... D,
each conditional p(a.a ætdd) is necessarily a single Gaussian. Because no latent variable influ- ences the affine parameters beyond the current coordinate, the model cannot represent multimodal densities or heavy tails. Consequently, T flows are not dense in L (IR) and fail the universal- approximation criterion.
Why T = Is Sufficient. For = 2 blocks with opposite orderings, all coordinates except the last (d < D) are expressed as infinite Gaussian mixtures (Eq. (5)) and hence enjoy the universal- approximation property via the density of Gaussian mixtures (Goodfellow et al., 2016). The principal reason why zp fails to possess the universal approximation property lies in the structure:
æD = ())== +0((D .y.
+
(10)
where yp is defined
yD = (K (g DD = + zD.
(11)
(12)
is evident that yo follows a unimodal Gaussian, since dD is sampled from a unimodal Gaussian prior and the functions 1á and og receive random variable input, regardless of their nonlinearity. Consequently, zp also becomes a unimodal Gaussian, inheriting this limitation from yo- Moreover, the above derivation only uses the base assumption Za (N, 1) in the generative direction. In general, conditioning on observed coordinates induces a non-Gaussian latent distribution (i.e., qo (= æ<aa) not necessarily Gaussian). Consequently, the resulting conditional ( #<D) can be even more complex than single Gaussian.
In summary, T = flows are fundamentally limited to unimodal Gaussians; T = 2 flows with alternating orderings achieve universality on D coordinates but leave the final one unimodal; and T> 3 flows overcome this last obstacle, granting full universal approximation power.
A.2 Proof of Prop.
Proof. For an isotropic Gaussian p(æ) NN,, T² the score is
V x logg p(x)
2c
Hence
V Po)
He
a Pr
Pbu
Step 1: Guided score. Insert these into Eq. (8) (CFG):
Me
α Mu
Va (o)) = (1 + wi)
+ w
o?
How
Me
ने Mm
(13)
Step 2: Match to Gaussian form. Any Gaussian N (pe, o?. has score -(x po)//3. Equating with Eq. (13) gives, for all x,
1
1
(14)
02
+
w
w
Me
Pa.
(15)
7
16

---

Step 3: Solve for o.. Let == = 0). Rewrite Eq. (14):
1
+ w)
W8
0²
s
0
s
(1 + w) - ws
1 1+ w W8
so that
o. + w ws
Step 4: Solve for jog. Multiplying Eq. (15) by oe and substituting the expression above yields
(1 + Will wB fuu
WS
ie
pe
1 ws (Ne Mu).
1 WS
Additional Discussion.
Consistency with standard CFG. When the two Gaussians share the same variance (a. F.
$ == 1), Eq. (9) reduces to o, 0r and tıe = Pe + w(pe pn)), exactly matching conventional CFG used in diffusion models (Ho & Salimans, 2021).
Numerical stability. The denominator 1 + 8 is large, causing a² to blow up or change sign. Intuitively, guidance should sharpen Pe, which entails _² oe i.e. + ws 2 We therefore clip the variance ratio to
approach 0 even become negative when
CLIP(s, 1),
guaranteeing + : ws for any w > 0 and ensuring both numerical stability and genuinely mode-seeking guided distribution.
B Implementation Details
B.1 Architecture Design
Overall Structure. We implement STARFlow with a decoder-only Transformer (Vaswani et al., 2017). The shorthand I(N) (see & 3.2) denotes a single deep AF block of layers followed by N-1 shallow blocks (two layers each) with hidden width d. Our class-conditioned baseline uses 18(6) - 2048 (11.44 B parameters), while the text-conditioned model uses 24(6) 3072 (=3388 B parameters). Layer-allocation sweeps Fig. 8(b-e) probe scalability and convergence. Unlike Zhai et al. (2024), we apply a final layer norm at the predictions of each Transformer block.
Conditioning Mechanism. For both conditioning modes, the context is prepended as prefix to the deep block, and we omit AdaLN (Peebles & Xie, 2023)-a choice that simplifies the network and marginally improves quality. ImageNet classes provided as one-hot vectors. Text captions (T2I) are encoded by frozen FLAN-T'S T5-XL XL encoder (Raffel et al., 2020), truncated to 128 tokens.
VAE Latent Space. Images are first mapped to continuous latent tokens via the DIT VAE (Peebles Xie, 2023), which compresses spatial dimensions by 48x. Because performance is highly sensitive to patch size, we keep p = 1 for all resolutions, yielding sequences of 1024, 4096, and 16384 tokens for 256 x 256, 512 x 512, and 1024 1024 images, respectively. We also applied our proposed deep-shallow architecture in pixels (see Table 3). To match similar computation, we adopted a patch size of p for learning 256 x 256 images.
Positional Embeddings. All variants employ rotary positional embeddings (RoPE) (Su et al., 2024); adopt 3D-RoPE, giving each token (I, y, (), where (I,,) encodes its spatial grid location ((0, 0) for text tokens) and I its caption index (0 for image tokens). During fine-tuning from 256 x 256 higher resolutions, we align positions by setting (x'.8', (z/a, y/a, t), where the up- sampling ratio.
This is equivalent to clip the unconditional variance ou when it is smaller than dc-
17

---

(8)
[b)
(c)
(d)
Figure 11: (a) Direct generation results using the model's 's latent samples without decoder fine-tuning score-based denoising. (b) Results after applying decoder fine-tuning, effectively reducing latent-space noise.
(c) (d) provide comparisons of classifier- free guidance (CFG) strategies for text-to- to-image generation: (c) demonstrates degraded outputs at guidance weight w 5 using the approach of Zhai al. (2024), whereas (d) shows stable results with our proposed CFG method, confirming its improved effectiveness and suitability for lexi-conditioed applications.
Default Configuration. Below is the default configurations of STARFlows:
model config for (model()-11N)-4
patch_size=1
hidden_size=d
num num_layers=[1] [1] [2] (N-1)
wum_channeel__pr_heeee -ee__e66r_
use__siiglu_ffn-Faaeee
use_rope=True use
B.2
Training Details
In all the experiments, we share the following training configuration for our proposed STARFlow.
Models are trained on 32 (for 1. 1.4B model) 64 (for 3.8B model) H100 GPUs for around weeks.
training config:
batch_sise 5icch_si
optimizer= AdamW'
adam_beta1=0.9
adam_beta2=0.cm
adam_eps=1e-8
Learning_rate-1e-4
nib__eaamiingrrtee-e--6
weight_decay=ler
max www.traininn_mmageseepeee
Stability of Eq. (3). The maximization term - log o in Eq. (3) is unbounded: the model can drive some values arbitrarily close to zero whenever this hardly influences echoing classic pathology of normalizing-o flow training. We mitigate it with three safeguards:
Soft clipping. Each raw Transformer output is mapped through f(x) a tanh(x/a) softly limiting its magnitude to ta.
Positive scale parameterization. The scale is enforced positive via softplus(6) (6), where the network's variance output.
3. Latent norm penalty. We add a small norm penalty over the intermediate latents x' to avoid extremely large values. Typically weight of le-4 is enough to keep the magnitude stable without hurting the performance.
18

---

Table 4: GenEval comparison across different methods.
Method
Overall
Single Ohj.
Ohj.
Counting
Colors
Position
Color Attri.
Diffusion Models
SDv1.5 (Rombach 2022)
0.43
0.97
0.38
0.76
0.04
PixArt-a (Chen et al., 2023)
0.48
0.98
0.50
0.44
0.81
0.08
0.07
SDv2.1 (Rombach al., 2022)
0.50
0,98
0.51
0.44
0.85
0.07
0.17
DALL-E (Ramesh et al., 2022)
0.52
0.94
0.66
0.49
0.77
0.10
0.19
SDXL (Podell et al., 2023)
0.55
0.98
0.74
0.85
0.15
DALL-E3 (Betker ct al., 2023)
0.67
0,96
0.87
0.47
0.83
0.43
0.45
SD3 (Esser et al., 2024)
0.74
0.99
0.94
0.72
0.89
0.33
0.60
Auforegressive Models
LlamaGen (Sun et ul.. 2024)
0.32
0.71
0.34
0.21
0.58
0.07
0.04
Chameleon (Team, 2024)
0.39
Show-o (Xic al., 2024)
0.53
0.95
0.52
0.49
0.82
0.11
0.28
Emu3 (Wang et al., 2024)
0.54
0.98
0.71
0.34
0.81
0.17
0.21
Normalizing Flows
STARFlow (Ours)
0.56
0.97
0.58
0.47
0.77
0.20
0.34
Reeellttso Training. During the high-resolution phase, STARFlow supports mixed resolu- tions, preserving each image's native aspect ratio. Because the backbone is a Transformer, variable sequence lengths are handled naturally, so no aggressive cropping is required; this better retains scene content and improves caption-image alignment. We bucket images into nine aspect-ratio bins: 21:9,
16:9, 3:2, 5:4, 1:1, 4:5, 2:3, 9:16, and 9:21 with the ratio appened in the caption:
Selgiinal_captinn An in (aspect_ratio, aspect ratio.
Image is senter-cropped and resized so that its token count roughly matches that of a square reference. For a 512 x 512 target, enforce H x W 5122. This procedure stabilizes optimization, maximizes GPU utilization, and is used in conjunction with the 3D-RoPE alignment described above.
B.3 Decoder Fintuning Details
We perform decoder fine-tuning by freezing the encoder and introducing controlled noise into the latent representations. The decoder is then trained using standard autoencoder loss comprising L2, perceptual, and GAN losses. Training is conducted on ImageNet images at resolution of 256x256 for 200K updates with batch size of 64, utilizing single node with 8 GPUs. To monitor performance, we compute an rFID by randomly sampling 50K real images, adding Gaussian noise with a standard deviation of 0.3, and directly decoding these perturbed images to compare with real images. Our resulting rFID is approximately 2.73, which exceeds the best achievable gFID from STARFlow 2.40. This suggests current STARFlow with this finetuned decoder might have reached performance ceiling under the specified noise conditions, highlighting an avenue for future exploration. Notably, although trained only on ImageNet 256 256 resolution, the fine-tuned decoder can seamlessly generalize to arbitrary resolutions, aspect ratios, and text-to-image domains. See visual comparison Fig. 11 (a) and (b).
B.4 Inference Details
Notation.
Let [fa] denote the autoregressive flow blocks ordered from deep (b=1) to shallow (b=B). Deep blocks text-conditioned; shallow blocks unconditional. Each block predicts Gaussian head for the next token. The learnable start token is denoted s R ([sOs]). pretrained VAE decoder D maps the final latent to image space.
Sampling procedure.
Closed-form classifier-free guidance for Gaussian heads (Prop. 2). Let (..) N (Juns En) and Pa(x) = -N Σ.) be the unconditional and conditional heads. For scale 7
E - > pr >E,1 Mc] Σ = (1- > Pg
(16)
E., then Bg = ph + 7(1, - nw) and E, Σ,, practice, Eq. equation 16 applied only in deep blocks; shallow blocks remain unconditional.
19

---

Algorithm Sampling from STARFlow Models
Require: Prompt y, guidance scale 2 0, blocks (fo), block types 7a E (deep, shallow), per-block
length T
Draw top latent seed (BB+1)~~
2: for = 1 to B do
deep block - shallow blocks
3: Initialize per-head KV caches (K&s, Vo)
4: if T6 deep then
5: Prefill (K,, Vo) with a single forward pass over text embeddings e(y)
6: else
7:
Zero-initialize (Ka, Vo)
8: end if
9:
I1 5-S for
inject [SOS] at the first position in every block
10:
f = 1 to I do
11:
(Mu: 2.) +- fo Kc, Vo, COND = Ø) if nh deep and CFG on then
12:
13:
(/c: (e. f-f((11 Ko, Vo. COND = y) (Mg, Egg GAUSSIANCFG(IU Eus Me: Dc: 7)
14:
15:
else
16:
(pg. Mng) (nuu En)
17:
end
T++1 N (Pg. E append keys/vals of rz+1 into (Ko, V))
19:
end for
u (B-b+1) REVERSE(Z1.T)
hand off the next block reverse order
21: end for
return D(RESHAPHTO2D(u"))
Implementation details. (i [SOS] used as the first input in every AR block at train and test time. (ii) KV caching follows standard LLM practice and is used only at inference in our code. (iii) Deep blocks maintain longer caches due to prompt prefill; shallow caches grow only with generated tokens.
Optional parallel refinement. For very long sequences, Jacobi iteration style K-sweep variant can replace the inner loop by repeatedly updating all positions from stale context and refreshing caches between sweeps. While faster wall-clock on some regimes, standard left-to-right sampling was most stable in our experiments.
B.5 Baseline Details
Diffusion Model Baseline We deploy the official implementation of DiT and report the perfor- mance. To make the architecture comparable STARFlow, set the number of layers 28 and hidden dimension to 2048 while keeping the number attention heads to 16, resulting in a model size of 2.1B parameters. We kept all of the other official repository settings the same. Notably the pretrained VAE of the official repository matches the one used in STARFlow. The baseline DiT is trained for 200M samples with batch size 256 using the official implementation settings: AdamW optimizer with learning rate 0.0001 and no weight decay 0.0. In inference, we set the number of sampling steps 250 and classifier-free guidance scale to 1.5 following the best reported setting in the original paper.
Autoregressive Model Baseline We deploy the official implementation of LlamaGen' (Sun et al., 2024) and report the performance. In particular, make the architecture comparable to STARFlow, we set the number of layers as 28, hidden dimension 2048, and number of atten- tion heads 32, which leads to the total model size of 1.4B parameters. We also adopt the VQ- "Q-VAE from the official repository with downsample factor 8 which matches the downsample factor used in STARFlow. The baseline LlamaGen is trained for 200M samples with batch size 512 using AdamW optimizer with learning rate 0.0001, weight decay 0.05 and betas (0.9, 0.95). In inference, we set the
20

---

top-k the same the vocabulary size 16384 and temperature 1.0. We also implement classifier- free guidance with scale 1.75 following the best reported setting in the original paper.
C
Additional Experiments
C.1 Additional Evaluation Text-to-Image Generation
Table 4 summarizes our GenEval (Ghosh et al., 2023) performance against representative diffusion and autoregressive (AR) baselines. STARFlow attains an Overall score of 0.56 0.56-slightly above SDXL (0.55) and well ahead of carlier Stable Diffusion checkpoints- wwile simultancously surpassing the several recent AR models for text-to-image generation, including Emu-3 (0.54), Chameleon (0.39), and LlamaGen (0.32). Improvements are most pronounced on the more compositional sub-tests. Crucially, these gains achieved without any reward-base based alignment, target-dataset finetuning, or caption
-STARFlow trained once, end-to-end, and evaluated exactly as generated. Because GenEval isolates visual grounding, purposefully restrict comparison to image-only generators; nonetheless, STARFlow 's single-pass inference already delivers substantial latency advantages over diffusion models that require to hundreds of denoising steps. The availability of exact neelikklisooods further opens avenues for principled preference learning, sequential planning, cascaded generation- cearattionccaaaiiiiiiie.e capabilities that likelihood-free baselines lack. An exciting next step is to couple STARFlow with large pretrained language- vision- language models, forming unified system that reasons jointly over text and images while retaining the speed, stability, and strong grounding demonstrated here.
C.2 Inference Speed
Because STARFlow autoregressive, tokens must be generated sequentially through every AF block, which makes inference latency the dominant bottleneck. Our deep-shallow redesign partially mitigates this issue: by concentrating parameters in the first few "deep" blocks and leaving the remaining ones lightweight with no condition or guidance, the incremental cost of later blocks becomes minimal. In practice, while the sampling speed is still relatively slow, this layout also outperforms the equal-sized architecture of Zhai et al. (2024) (see Table 5), and its overall runtime approaches that of standard LLM leaving additional head-room for techniques such as distillation or speculative decoding.
C.3 Latent Denoising
second limitation is that STARFlow cannot be trained directly clean latents; adding Gaussian noise is required to keep the flow learning stable (Ho et al., 2019; Zhai et al., 2024), but this both complicates optimization and necessitates an explicit denoising stage inference time. In this work, we investigated three strategies of denoising:
(1) Single-step
denoising. Use the flow itself score estimator and apply denoising step. Works only for mild noise; at o = 0.3 outputs are noticeably blurry.
(2) Multi-step diffusion denoising. Start from the noisy latent and run unconditional DDIM steps with a pretrained DIT. Quality improves, but latency and model complexity increase substantially.
(3) Decoder finetuning (ours). Finetune the VAE decoder it can reconstruct directly from noisy latents. Training can be done very efficiently on unconditional images, and the GAN objective effectively handles the uncertainty. This option is the simplest deploy.
Future work will aim for principled solution that trains directly on clean data, eliminating the denoising stage entirely.
21

---

Table 5: Per-block inference time (s) with a fixed batch size the 1.4B sized model for generating 256 x 256 images. Sampling speed measured with CFG. The proposed deep-shallow uses 6 blocks: an 18-layer Transformer followed by 5 blocks of 2-layer Transformer. The hidden dimension is 2048. The equal-sized (Zhai al., 2024) baseline uses 8 blocks where cach block has 8 layer of Transformers. To match the overall parameters, we reduce the hidden dimensions to 1280.
Block ID
0
2
3
4
5
6
7
Total (s)
Equal-sized (Zhai et al., 2024) Deep-shallow (ours)
9
9
9
9
9
9
9
9
72
18
2
2
2
2
2
35
Table 6: Comparison of latent-denoising strategies øz = 0.3 on ImageNet 256
256.
Method
Extra Model
Extra Steps
FID 50K
Remarks
Single-step score
1
2.96
Blurry Slowest
Multi-step DiT' (from 0.3)
DiT-XL
30
2.53
Decoder finetune
Finetuned Decoder
0
2.40
Best, simplest
D Application Details
D.1 Training-fres Image Inpainting with STARFlow
Let M {0, 1) 1) be binary mask that selects the pixels be filled and let ae E Hx be the ground-truth image (available only at evaluation time for measuring fidelity). We split the image into the observed part #o = (1 .M) ④ zet and the missing part a = M @ Fegt The pretrained flow fo + induces a tractable density (a =N()(()) 0, 7) |det fo To sample from the conditional po (mr mo) without retraining, we construct a Seroopois-HHstinge (MH) chain in latent space:
1. Init. Replace the missing region by Gaussian noise, (0)) xo + M 6, (~((0,021), and map to latent space z(0) fo(
2. Proposal. Draw fresh noise in the same masked region of latent space
z' = + M MOT,
7 ~
and obtain the candidate image x' = fo (z'). We then restore the context pixels, E' 20 M @ x', ensuring every proposal satisfies the observed evidence.
Acceptance. Because the forward and reverse proposals are symmetric, the MH acceptance probability reduces the ratio of conditional probabilities:
= min 1,
Po x00=
min{1 1, po(') log po (1]
Po (1) [
Accept with probability o; otherwise keep the current state.
4. Iteration. Set f- reported as the inpainted image.
t 1 and repeat steps (ii)-(iii) until convergence; the final sample z (T) is
Intuitively, each step perturbs only the masked latents, letting the powerful flow prior propose content that is globally coherent with the context while the MH test enforces exact consistency with the joint density. The chain ergodic Gaussian noise gives non-zero probability to every latent configuration and its stationary distribution is precisely pe(xx xo)). In practice we set both = 1 and r 1. Since our STARFlow well-trained on large-scale text-to-image data with sufficient capacity, it yields high acceptance rates and we set the total iterations to 20. No additional training, guidance network, or data-specific tuning is required. effective plug-in for image inpainting with pretrained autoregressive flows.
22

---

Figure 12: Demonstration of generation trajectories of inpainting output. D.2 Interactive Image Editing with STARFlow
STARFlow can be naturally extended to multi-round tasks such as interactive image editing. We start from a pretrained tex--o-imaae checkpoint and finetune on the ANYEDIT corpus". For simplicity, we use only the subset that provides text instructions. Each training quadruple (æ*re, teap, giust x'st) contains a source image, its caption, free-form instruction, and the edited target (see Fig. 6b).
We serialize every sample into the sequence
(T(()))), AF AF<(VE((=*0)) T5 VAE
where image segments are tokenized by our VAE (p 1) and text segments are embedded by frozen FLAN-T5-XL (Raffel al., 2020; Chung al., 2022). Image latents first pass through the shallow-AF blocks independently, after which all tokens are processed by the shared deep-AF Transformer. Because the deep block is strictly causal, the edited image and all later tokens can attend to the entire prefix-iiiii including the source image image-without without any special masking. During inference the prefix is written once into the KV cache; sampling the edited tokens simply reads from this cache, mirroring the behavior of language-only LLMs.
Joint Training Objective. Instead of optimizing a single conditional likelihood, we maximize the joint log-likelihood of both images:
max Ljoint Eggar teap ginst antet)
logg pe (x=
((a)) + loo pe 28t
egt ginst "re tman)]
where each term is evaluated via the change-of variables formula (Eq. (3)). This objective maintains maximum- -ikellhood training, allows gradients to propagate across all modalities, and enables the same network to generate from scratch (empty image prefix) or perform edits (given image prefix).
charaelofvvrrialless
Unlike diffusion-based MILMs that first generate pixels and then re-encode them with a separate vision backbone, our autoregressive flow invertible; a single forward pass encodes the user image, and single reverse pass decodes the edited result. Encoding and decoding share parameters, introduce no information loss, and integrate seamlessly with the Transformer's KV cache. This single-pass round-trip property sharply reduces latency and highlights autoregressive flows compelling choice for tightly coupled vision language applications. We show interactive image generation and editing examples in Fig. 13 where given caption and editing instruction, model predicts two images one after another.
E Related Topic Discussion
E.1 Autoregressive Model v.s. Autoregressive Flow
Connections between the two families emerge in masked autoregressive flows (MAF, (Papamakar- ios et al., 2017)), which impose invertibility on an autoregressive factorisation, yet fundamental differences remain. Autoregressive models dispense with any latent prior; each conditional distri- bution is learned directly in the data domain, which is typically discrete-tokent integer pixels, or quantized audio samples- and generation proceeds strictly one element at a time. Normalizing flows. by contrast, begin from explicit Gaussian prior in continuous latent space and learn an invertible transformation that warps this prior into the target distribution. This design delivers exact ooglliklihoood, parallel one-shot sampling, and bidirectional latent inference, but at the cost of enforcing invertibility and differentiability in every layer. While MAF narrows the gap by marrying an autoregressive factorisation with invertibility, the reliance on Gaussian base and continuous formulation remains the defining hallmark of normalizing flows, whereas the absence of a prior and the natural alignment with discrete data continue characterise pure autoregressive models.
"tttpe://dc--nnydd:: anyeditt.ttittt.. ceatt...ittub...
23

---

E.2 Flow Matching v.s. Autoregressive Flow
Normalizing flows (NF) and Flow Matching (FM) both map a simple latent prior the data dis- tribution, but they differ fundamentally in what they optimise and how they realise the map. A normalizing flow learns a time-independent bijection whose parameters are updated by directly minimising the data's s negative log-likelihood (NIL); the thange-of-varialee formula provides an exact, unbiased gradient, so every parameter update moves the model toward the true maximum- likelihood solution. Flow Matching instead specifies time-dependent vector field that transports probability mass along a chosen path and trains this field with a velocity velocity-matching loss. In short, Flow Matching reduces per-iteration cost by relaxing the objective, but Normalizing Flows retain the rigorous maximum-likelle likelihooo foundation, and exact densities.
E.3
Relation to JetFormer (Tschannen et al., 2024b)
CeeeffomerrReealNVe flow ("Jet") in pixel space and employs a GMM prior; training optimizes an ELBO due to the latent prior. In contrast, STARFlow applies uniform affine autoregressive flows throughout, is invertible end-to-end, and operates in the latent space of a fixed pretrained auto-encoder, so the flow itself is trained by exact likelihood while the overall objective can be viewed fixed-encoder ELBO.
Architectural differences. JetFormer constructs
Expressivity and depth. Prop. establishes that a small number of autoregressive flow blocks already achieves universality our construction (2-3 blocks suffice), which keeps STARFlow compact and scalable. In our experience, the Jet mapping typically requires many blocks for comparable expressivity.
Priors and guidance. Gaussian prior in STARFlow enables the closed-form guidance in Eq. equa- tion 16. While a GMM prior could be substituted, we found unnecessary. Conversely, replacing JetFormer's GMM with standard Gaussian weakens its modeling power under the Jet architecture (consistent with its reported ablations) and removes a straightforward path to closed-form guidance.
Factor-out. JetFormer adopts multi-scale factor-out compress activations for its GMM prior. STARFlow does not factor-out by default; adopting similar device is natural extension but not required for our results.
Table 7: Concise comparison with JetFormer.
Aspect
JetFormer
STARFlow
Domain
Pixels
Pretrained VAE latent
Transform
TransformerRealNee (partial invertibility)
Affine AR flows (fully invertible network)
Objective
ELBO (GMM prior)
Exact flow NLL (fixed-encoder ELBO overall)
Depth
Many Jet blocks often required
2-3 AR blocks suffice (Prop. 1)
Prior
GMM
N(0, I) (enables Eq. 16)
Factor-out
Multi-scale factor-out
Not used by default
Guidance
No dosed-form CFG
Closed-form Gaussian CFG
F
Broader Impacts
Positive societal impacts. STARFlow shows-for the first time-that normalizing flow- models can scale to the same resolutions and sample quality previously dominated by diffusion and discrete autoregressive methods. The invertibility of normalizing flows enables interactive image editing (See Fig. 9b for examples) making STARFlow suitable for assistive technologies (c.g., real- time diagram manipulation for education or accessibility) and for professional design workflows that demand faithful, iterative refinement.
Potential risks and negative impacts. Higher-quality image generation lowers the barrier to fabricating realistic- but false-visee evidence. Interactive editing magnifies this risk by enabling

---

rapid revision cycles. We advocate the concurrent development of reliable flow-specific watermarking and provenance tools.
G Additional Samples
We show more generated samples from STARFlow in Figs. 14 to 17.
25

---

siaaping sunny windowtll [odic] and smill putled twotus beside the
rosbont floating orystal [edit] change the Line day to golder wunset
sural brick wall
landscape under clear
[edit] the
[edit] add drarecis clouds rolling
pienic blanket spread green [udit] wicker basket filled with fruit
glass of red wine wooden herrel [edit] switch the color white
steazing bowl of pho wooden table [edit]] add frrsh chili slices
sunflower facing the [edit] rotate the Clowe
A slate board [edit]
forest rootprints leading distance
Figure 13: Interactive editing with STARFlow. Starting from initial caption, STARFlow generates base image. Given subsequent user-provided editing instruction, model then modifies image accord- ingly-withh without requiring re-encoding. Each example illustrates generic instruction applied to generated image. All images are synthesized at resolution of 512 512.
26

---

Figure 14: Additional class conditioned generation from STARFlows trained on 256 256 and 512 512, respectively. The classes are sulphur-crest cresstd cockatoo, Kakatoe galerita, Cacatua galerita, loggerhead. loggerhead turtle, Caretta caretta, and Siberian husky.
27

---

QUANTUM
1. Scheratic cutasty of clockork heart pumpzing luninous liquid, technccal drawing style: 2. Gress section illustration of layered cake that resembles planetary strata: Surreal desert with floating sandstone norolichs casting long shadsws at golden hour, ultra-udde less; cubist still life of fruit and musical instrunents, vivid complanantary colors, top-dow shot of labyrinth garden crimmed into Escher like ixpossible consetry. 6. aatercolar portrait of ar abstract huranoid with translucent skin revnal ing gelaxies; rilr-shift phato of feativel Tantern perade through narine cthblastone streats a. all ill--nncaves sanscape brush strokes
where
hourglass where forms airisture sombtain lue-roly 300 model nautilus shell cross-section revealing logarithasc Antarctic
Javenlle
haddling
snoufall:
shot concentric
festuring 12, Quokka
hind
swirling parchrent: Photoreallstic close-up righiresolution chalkboard typography skatch spelling 'Quantun' cherry snawfall, drasatic lighting, inttroallsstio
background; tropical rainforest
surrounded silently
1. Tak-on Tok-or-parcheent concept art Floating pagads tethered by chains aountain peaks; 2. Blowing jyllyfish drifting through nisty püne forest at dawn, photoreal conposite: Tirber-frase hobbit-style cottage under glant zunelowers, golden afternoom: 4. Steampunk clockwork version of an 1815 surrounced by geonetric patterns: 6, Ceramic glazed statue of a koala against black veivet backdrog; 6. Dreaniike smage of an ise creas cone neiting into desert landscape, surrealion.
Figure 15: Additional text-conditioned generation from STARFlows trained on 256 x 256 and 512 x 512, respectively.
28

---

VELOCITY
N
groffiti spelling "velocity" ulcrawide rainforest
dramatic Phocorealistie looking
close-up hedgehog basking trees and Lianss; bareque sil painting featuring jallyfish under monlli sky, Design antiqus cospass floating above storey aeas vector flat design style: Retro g-bst pixel art soone of cosy lakeside cabin under meteor shower
luth river
Figure 16: Additional text-conditioned generation from STARFlows trained on 1024 x 1024.
29

---

Evenung silhouette of boobab trees reflected in seasonal "loocpiain under vielet
ancient psinting of hedgehuy floating costiu vold
Tousetric floating garden spilling
hypercetatled photerealistic
celescial drifting
Panoranic ridge steaspunk Glockwork version kangaruu on vintage parchrent
Silhoue afternoon
Whinsical pencil sketch cen party with levicacing porcelain, seft shadingg
Depact Library glowing hyperresl
Scheratic cutaway of a clockwork heart pumpiing Luxinoes liquid, technical drawing style
Photerealittic clase up of hippopotaru, bathingg playfully the starlit ocean surface
vibrent bool of soup Ehst looks 15k6 nonster knitted out of wool
Figure 17: Additional text-conditioned samples from STARFlows trained various aspect ratios.
30

---

NeurIPS Paper Checklist
The checklist is designed to encourage best practices for responsible machine learning research, addressing issues of reproducibility, transparency, research ethics, and societal impact. Do not remove the checklist: The papers not including the checklist will be desk rejected. The checklist should follow the references and follow the (optional) supplemental material. The checklist does NOT count towards the page limit.
Please read the checklist guidelines carefully for information on how to answer these questions. For each question in the checklist:
You should answer [Yes], [No], or [NA].
[NA] means either that the question is Not Applicable for that particular paper or the relevant information is Not Available.
Please provide short (1-2 sentence) justificcton right after your answer (even for NA).
The checklist answers are an integral part of your paper submission. They are visible to the reviewers, area chairs, senior area chairs, and ethics reviewers. You will be asked to also include (after eventual revisions) with the final version of your paper, and its final version will be published with the paper.
The reviewers of your paper will be asked to use the checklist as one of the factors in their evaluation.
While "[Yes] " is generally preferable to "[No] ", it is perfectly acceptable to answer "[No] provided proper justification is given (e.g., "error bars are not reported because it would be too computationally expensive" or "we were unable to find the license for the dataset we used"). In general, answering "[No] or "[NA] is not grounds for rejection. While the questions are phrased in binary way, we acknowledge that the true answer often more nuanced, so please just use your best judgment and write justification to elaborate. All supporting evidence can appear either in the main paper or the supplemental material, provided in appendix. If you answer [Yes] to a question, in the justification please point to the section(s) where related material for the question can be found.
IMPORTANT, please:
Delete this instruction block, but keep the section heading "NeurIPS Paper Checklist",
Keep the checklist subsection headings, questions/answers and guidelines below.
Do not modify the questions and only use the provided macros for your answers.
1. Claims
Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Question:
Answer: [Yes]
Justification: we prove theocratically and empirically it is scalable method for generative model in the following sections.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made the paper and important assumptions and limitations. A No or
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
It is fine to include aspirational goals as motivation as long as is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors? Answer: [Yes]
31

---

Justification:
We have a limitation section in the end.
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification. asymptotic approxinations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g. if the approach was only tested on a few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low images are taken in low lighting. Or speech-to-text system might be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and proof'
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [Yes]
Justification: We have rigorous proof in the appendix, and the statement of the proof is accurate.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged to provide short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix supplemental material.
Theorems and Lemmas that the proof relies upon should properly referenced.
4. Experimental result reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Answer: [Yes]
Justification:
We will release the code later.
Guidelines:
32

---

The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is a dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, other means that appropriate to the research performed.
also be provided via detailed
While NeurlPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) the contribution is primarily a new algorithm, the paper should make clear how to reproduce that algorithm. (b) If the contribution is primarily new model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution is a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome deseribe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [Yes]
Justification: We do not provide code in the paper, but we denote the open access link for the data.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nips.co cc/
for more details.
While we encourage the release of code and data, we understand that this might not be possible, "No" an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https:
/ tnips.cc/publle http:/publcc///udeessseeeess http///ieee//eee/ http:////ggiieee//ddeuttiiiiiiii htt////ooeeuessiiieeieee s8ionPolicy) for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
33

---

At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code is permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification:
We discussed the experiment detail and setup in the paper.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, or as supplemental material.
7. Experiment statistical significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [No]
Justification: No because the experiments are large scale and we cannot afford repeat experiments.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall
run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula,
call a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
should be clear whether the error bar is the standard deviation or the standard error of the mean.
It is OK to report 1-sigma error bars, but should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative error rates).
If error bars are reported in tables or plots, The authors should explain the text how they were calculated and reference the corresponding figures tables in the text.
8. Experiments compute resources
Question:
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
Answer: [Yes]
Justification:
We will disclose the computation resource in the paper.
Guidelines:
The answer NA means that the paper does not include experiments.
34

---

The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount compute required for each of the individual experimental runs well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make into the paper).
9. Code of ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurlPS Code of Ethics
Answer: [Yes]
Justification:
We do.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g.. there a special consid- eration due to laws regulations in their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification: This is a generative model method paper. While there no direct societal impact of the method, the improvements it brings might have effects applications such as deepfake. The products it enables might also positively help with the society's 's productivity.
Guidelines:
The answer NA means that there no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional unintentional) misuse of the technology.
there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Question: Does the paper describe safeguards that have been put in place for responsible release of data models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
35

---

Answer:
[No]
Justification: This is the initial exploration of a brand new method. It has not reached the level of safeguards, so we did not describe
Guidelines:
The answer NA means that the paper poses such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Question: Are the creators or original owners of assets (e.g.. code, data, models), used in the paper. properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification: We took care the liscence and terms of use.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset is used and, if possible, include URI
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
assets are released, the license, copyright information, and terms use in the package should be provided. For popular datasets, paperswithcon thcode. com com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-pane re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
If this information is not available online, the authors encouraged to reach out the asset's 's creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification: We did not release a new assets. The assets will be documented once we release it.
Guidelines:
The answer NA means that the paper does release assets.
Researchers should communicate the details the dataset/code/moodll as part their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset used.
At submission time, remember to anonymize your assets (if applicable). You can either create anonymized URL or include an anonymized zip file.
36

---

14. Crowdsourcing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given participants and screenshots, if applicabll, as well as details about compensation (if any)?
Answer: [No]
Justification:
we not have crowdsourcing experiments.
Guidelines:
The answer NA means that the paper does not involve crowdsoureing nor research with human subjects.
Including this information the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then much detail as possible should be included the main paper.
According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/teview based on the requirements of your country institution) were obtained?
Answer: [No]
Justification: we do not have human subjects in this work.
Guidelines:
The answer NA means that the paper does not involve crowdsoureing nor research with human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors adhere the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
16. Declaration of LLM usage
Question: Does the paper describe the usage of LLMss if it is important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, formatting purposes and does not impact the core methodology, scientific rigorousness, or originality of the research, declaration is not required.
Answer: [No]
Justification: We did not use LLM for important, original component of the core methods in the paper.
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, non-standard components.
Please refer to L.LM policy (https: bectps:/neeuris. http://uuurpppeeeeeeeeeeeeeeeeee wwt.e///eoeeeeeee/////O/./ for what should or should not be described.
37


Reviews:
[{"review_id": "uyVGZRqcXE", "reviewer": "Reviewer_Fiwy", "summary": "STARFLOW is an thorough extension/update from previous great work TARFLOW. In detail STARFLOW adds some practical training skills:\n1. TARFLOW performs at pixel space of ImageNet-64/128, while STARFLOW is built in the re-designed VAE latent space.\n2. TARFLOW can be seen as the part of the  shallow AF blocks in STARFLOW (the continuous visual “tokenizer”), while its deep block can leverage the pre-trained LLM.\n3. The added noise (to pixel) skill in TARFLOW has been replaced as the 0.3 std noise added on VAE latents, and thus the extra denoising langevin dynamic step can be remved during inference by finetuning the decoder.\n4. The deep-shallow block introduce the conditional flow by which  the conditions (like |sos| ) are only needed in  deep LLM block. Therefore the deep/shallow blocks are seperated for simplicity. Notice that the conditon in TARFLOW are added on the input to maintain the characteristics of Normalizing Flow.\n5. New CFG are proposed.\n\nExperiment parts:\n1. ImageNet256/512 and text2img zero-shot COCO exps are provided, which shows the effectiveness of the proposed STARFLOW.\n2. The editing applications and the visualization are impressive.", "questions": "See weakness:\n1. The inference part should be described in detaill. I have not found how STARFLOW performs the inference with text and |sos| tokens except figure4. Please added the inference and sampling part in detail, especially how you deal with the |sos| token, and how you leverage the KV cache in inference.\n\n2. The figure4 is hard to understand with various arrows. I know that    it is difficult to understand each steps in each coupling block of TARFLOW/STARFLOW. I consider that you can seperate the figure4 into four figures, the training/ inference of Deep blocks (multi-modal AR) and shallow blocks (visual encoder).\n\n\nQuestions:\n\n3. The deep-shallow blocks are very similar with the design of Jetformer, even the   learning objective in final multi-modal AR is quite different. Could please claim the similarities and differences with  Jetformer?  Could the GMM in Jetformer replaced as the single N(0,1) in STARFLOW? or Could the  N(0,1) in STARFLOW be replaced with the  GMM in Jetformer?  \n\n4. In jetformer, when the input of txt2img pairs are replaced with the img2txt pairs, the model can be trained as a MLLM understanding model. Could STARFLOW be trained in the same way? I think STARFLOW could be improved as a unified model containing both generation and understanding with good potential. (Please don't show me the editing model in your paper \n\nI am wiliing to raise my ratings if your solve all my questions.", "limitations": "yes", "rating": 6, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 4, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths:\n1. Novelty:\nEven though STARFLOW is an extension of TARFLOW, every improvement STARFLOW added to TARFLOW is highly practical, directly addressing the  bottlenecks of TARFLOW, and demonstrates good novelty. The design of deep-shallow blocks not only leverage the pre-trained LLM model, but also utilize the conditional flow to seperate the visual encoder (shallow AF blocks) and the multimodal \"AR\" part. \nIf we send text tokens into the shallow AF blocks (it contains about 64 layers transformer attn-ffn layers), the computation burden is very very heavy  and the train efficiency could be very very low. Besides, the text NF is very hard to train [2].\n\n2. As for the rationality of deep-shallow blocks, STARFLOW provides sufficient proof.\n\n3. The added VAE latent space not only reduce the dimension of pixel space, but also remove the reliance of pixel-added noise in TARFLOW which further requires an extra step (with lots of computation) to denoise. The added noise is transferring to the VAE latents, like the skills in Jetformer[1] (who added the latents noise (std=0.3 same with STARFLOW) on Normalizing Flow latents).\n\n\nNeutral comment:\n1. Performance:\nEven though the performance is a bit lower than the mainstream diffusion or discrete AR methods,  in my consideration STARFLOW brings new light in current CV community and with the continuous improvement on Flow-Transformer-based models (other works like Jetformer[1]), the performance can catch up with DIT models  shortly afterwards.  \n\n\n[1] JETFORMER: AN AUTOREGRESSIVE GENERATIVE MODEL OF RAW IMAGES AND TEXT. ICLR 2025\n[2] Flexible Language Modeling in Continuous Space with Transformer-based Autoregressive Flows. Arxiv 2025.07\n\nWeakness:\n\n1. The inference part should be described in detaill. I have not found how STARFLOW performs the inference with text and |sos| tokens except figure4. Please added the inference and sampling part in detail, especially how you deal with the |sos| token, and how you leverage the KV cache in inference.\n\n2. The figure4 is hard to understand with various arrows. I know that    it is difficult to understand each steps in each coupling block of TARFLOW/STARFLOW. I consider that you can seperate the figure4 into four figures, the training/ inference of Deep blocks (multi-modal AR) and shallow blocks (visual encoder).", "was_revised": true, "final_justification": "The authors have solved most of my questions, beside the part of TARFLOWLM, which in my opinion is also a great work for NF modeling, even though the inference is unsatisfactory (which is the advance of JET NF network).\nAnother question is that can you provide some ablation insight for STARFLOW without VAE?\n\n\nConsidering about the novelty, completeness, and the current lack of diversity in CV generative  community, \nI have raise my ratings from 4 to 6, and I strongly recomment AC to accept STARFLOW as an ORAL paper.\n\nAnd the authors should  add the details in the final version, as they have promised, including:\n1. Code\n\n2. Add sampling and testing details.\n\n3. Add Tarflow details with Ablation Insights.\n\n4. Polish the figure.\n\n5. Decalre the inference speed, espscially with the sequence of 32*32 =1024 (in Tarflow 256 * 256 case, for STARFLOW  case I don't know how you set your  VAE patch size)\n\n6. Other reviewer's suggestions.", "rebuttal": "We sincerely thank the reviewer for their thoughtful and constructive feedback. Below, we address each point in detail and outline the corresponding revisions we will incorporate in our next submission.\n\n---\n\n# **Major Clarifications & Planned Revisions**\n\n## **1. Inference with Text and `[SOS]` Tokens**\n\nWe acknowledge that the inference pipeline could be explained more clearly. The inference (sampling) process corresponds to the **right plot** in Figure 4. Below is a step-by-step explanation of the sampling procedure:\n\n* Randomly sample $z_1, \\ldots, z_N \\sim \\mathcal{N}(0, I^2)$.\n* For each block $t$ from $1$ to $T$:\n    * Initialize the KV cache for all attention layers.\n        * If $f_t$ is a deep block, we **prefill** the KV cache by performing a forward pass on $f_t$ with the T5 embeddings of the prompt (e.g., \"A Corgi Dog\") or one-hot vector embeddings of class labels.\n        * If $f_t$ is a shallow block, the KV cache is initialized with zeros.\n    * Initialize $x_0$ with the `[SOS]` token (a learnable continuous vector with the same dimension as the input channel size).\n    * For each step $n$ from $1$ to $N$:\n        * Run a forward pass $\\mu_n, \\sigma_n = f_t(x_{n-1}, \\text{KV})$, updating the KV cache during the pass.\n        * If CFG is enabled and $f_t$ is a deep block, compute the updated $\\mu_n, \\sigma_n$ similarly to diffusion models but based on Proposition 2.\n        * Compute the next token $x_n = \\mu_n + \\sigma_n \\cdot z_n$.\n    * The output sequence is reversed for the next block: $z_1, \\ldots, z_N \\leftarrow x_N, \\ldots, x_1$.\n* Finally, reshape the sequence $x_1, \\ldots, x_N$ to 2D and pass it to the VAE decoder to generate the final image.\n\n**Additional Clarifications:**\n\n* The equation in Figure 4 (right) computes the conditional mean and variance, which can be combined with the Classifier-Free Guidance (CFG) formulation from Proposition 2.\n* As an autoregressive model, we inject the `[SOS]` token (denoted `
` in Figure 4) as the first input during both training and inference for every autoregressive block, regardless deep or shallow blocks.\n* The KV cache works just like in standard LLMs, as each flow block is strictly autoregressive. In our implementation, the KV cache is used only at inference time. Each attention head has its own KV cache, which is appended with the current \"key\" and \"value\" at each step to compute attention based on the history.\n* Since conditioning is only performed in the deep blocks, their KV caches are longer than those of shallow blocks. CFG is also applied only in the deep blocks.\n\n*In the revision, we will add a detailed explanation to explicitly elaborate on this inference procedure.*\n\n---\n\n## **2. Complexity of Figure 4**\n\nWe appreciate the reviewer’s suggestion regarding Figure 4. We agree that breaking it into four modular sub-figures would enhance clarity.\n\n*We will update the figure accordingly in the revised manuscript.*\n\n---\n\n## **3. Clarification and Relationship to JetFormer**\n\nWe will revise the text to clearly differentiate STARFlow from JetFormer and clarify our design choices in greater detail. The major differences are as follows:\n\n* **JetFormer's Architecture:** JetFormer uses a flow model called Jet (a Transformer-based RealNVP) to map pixels to a latent space. It employs a GMM-based latent prior and is not fully invertible. Therefore, JetFormer is technically a VAE that optimizes the ELBO. In our experiments, we found that the Jet flow requires many blocks to be sufficiently expressive.\n* **STARFlow's Architecture:** In STARFlow, all layers are affine autoregressive flows and are implemented uniformly. The only difference is that deep blocks have more layers and include text conditioning. This means our flow model directly optimizes the likelihood and is fully end-to-end trainable, and the whole network is invertible. As our Prop.1 shows, only 2-3 autoregressive flow blocks are needed to capture an arbitrary distribution, which makes the model highly scalable. This is a key difference from JetFormer.\n* **Operating Space:** JetFormer operates directly on pixels, whereas our work operates in the latent space of a pretrained auto-encoder, which is a more scalable approach. In this sense, the overall objective of STARFlow is also optimizing an ELBO, but with a fixed encoder. So there is no ``moving target``.\n* **Factoring-Out:** JetFormer uses a multi-scale factor-out technique to compress the space for its GMM prior. While STARFlow does not currently use this, we consider it a promising future extension.\n* **Replace N(0,I) with GMM:** STARFlow could replace the deep block with a GMM prior, but this is not necessary as the model is already powerful enough. **A key drawback of the GMM prior is the lack of an easy analytic solution for guidance**, requiring approximation via rejection sampling. In STARFlow, the Gaussian prior allows us to use a closed-form guidance (Proposition 2), similar to diffusion models, which is a significant advantage. * \n* **Replace GMM with N(0,I):** JetFormer **cannot** simply replace its GMM with a Gaussian, as this would result in insufficient modeling power from the Jet flow alone (by checking results from Jet's paper).\n\n---\n\n## **4. Potential Use as a Unified Model**\n\n* **Short Answer: Yes**\n* STARFlow can be extended for joint modeling with a language modeling loss (cross-entropy) to perform both understanding and generation tasks. By incorporating a joint modeling objective, STARFlow could, *in theory*, work as well as other unified models like Transfusion, which can perform text-to-image generation, editing, and, in the reverse direction, image understanding.\n* **The Challenges:** However, to train a state-of-the-art unified model, we would need to consider how to incorporate information from a ViT encoder for understanding tasks, which is non-trivial. The critical question is whether STARFlow itself should be trained to \"encode\".\n    * **If not:** We could train an MLLM version of STARFlow. As supporting evidence, in our paper, we successfully finetuned a pretrained Gemma for text-to-image generation. In this setup, a separate text encoder is not needed, and it achieves image generation quality nearly as good as the default version. We could further explore approaches where the MLLM (with a ViT encoder) is frozen, fixing the understanding ability, though this may not be a \"truly unified\" model.\n    * **If yes:** The challenge would be training STARFlow to match the performance of a dedicated ViT. Recent work like Show-o2 has made attempts in this direction.\n* In summary, STARFlow can be extended for unified models, and our preliminary results are suggestive of this potential. However, significant research and exploration would be needed to make it performant.\n\n---\n\n# **Summary of Revisions**\n\nTo summarize, we will:\n\n1.  **Clarify the inference process,** including the role of `[SOS]` tokens and KV caching.\n2.  **Redesign Figure 4** to enhance modular understanding and readability.\n3.  **Differentiate STARFlow from JetFormer** with clear explanations and a comparative summary.\n4.  **Refine the manuscript’s presentation,** improving clarity, labeling, and overall readability.\n\nWe hope these changes will address the reviewer’s concerns and significantly enhance the clarity and impact of our paper."}, {"review_id": "2tqHT4PEOD", "reviewer": "Reviewer_M6cW", "summary": "The paper presents STARFlow, a scalable generative model based on normalizing flows that achieves strong performance on high-resolution image synthesis. The core contributions include: 1) Deep-Shallow Architecture: A novel design where a deep Transformer block captures most of the model's capacity, followed by shallow blocks that are computationally cheap yet contribute non-negligibly. 2) Latent Space Learning: Training in the latent space of pretrained autoencoders, which proves far more effective than modeling pixels directly. 3) Novel Guidance Algorithm: A principled approach to classifier-free guidance that substantially improves sample quality, especially at high guidance weights. The model remains a single, end-to-end normalizing flow, allowing exact maximum likelihood training in continuous space without discretization. STARFlow achieves competitive results in both class- and text-conditional image generation, with sample quality approaching that of state-of-the-art diffusion models.", "questions": "Besides the weakness, I have following questions:\n1. Joint latent-NF modeling: the authors mentioned the limitation of joint latent-NF model design. To my knowledge, existing latent-space based generative models leverage retrained auto encoders for compressing visual signals into latent codes, I wonder if there are better optimization solutions for joint latent-generative modeling?\n2. Video modeling: any further plans to explore such model for video modeling? What's the main challenges for temporal modeling with your auto-regressive flow? Besides, finetuning the decoder for noised video decoding might leads to sub-optimal performance, especially for the motion modeling.", "limitations": "Overall, this paper successfully brings normalizing flows back to the generative modeling community and demonstrates its potential for scaling visual generation under high-resolution settings. This is a nice work, my major concerns and questions are shown in the weakness and questions.", "rating": 5, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 4, "originality": 3, "strengths_and_weaknesses": "**Strength:**\n1. Theoretical Foundation: The paper establishes the theoretical universality of stacked autoregressive flows (AFs) for modeling continuous distributions, providing a solid basis for scaling up the normalizing flow models with autoregressive transformer architectures.\n2. Improved Guidance: The proposed guidance algorithm offers a more principled approach to classifier-free guidance for AFs, leading to stable and high-quality image generation across a wide range of guidance weights.\n3. Competitive Performance: STARFlow achieves results comparable to state-of-the-art diffusion and autoregressive models on both class-conditioned and text-to-image generation tasks, demonstrating the scalability and effectiveness of normalizing flows.\n4. Well-presented and easy-to-follow: the technical insight of this paper is solid, and the overall presentation is well-structured and easy-to-follow.\n5. Various applications: the proposed method could be further extended to downstream applications including image pinpointing and editing.\n\n\n** Weaknesses:**\n1.  Inference speed: Despite the deep-shallow architecture, the autoregressive nature of STARFlow might still results in relatively slow inference, especially for high-resolution images. It would be better to include some comparisons about the inference time and NFE.\n2. Controllable generation: More controllable generation approaches such as ControlNet, T2I-Adapter enables user to produce images from various conditions, I wonder how the proposed method would perform when integrated more controllable signals to the system?\n3. About decoder fine-tuning: finetuning the VAE decoder to handle noisy latents is an additional step that may not generalize well to other datasets or domains without further adaptation.", "was_revised": true, "final_justification": "Most of my concerns are addressed during rebuttal, l keep my score as accept.", "rebuttal": "We sincerely thank the reviewer for the encouraging evaluation and thoughtful suggestions. We are pleased that you found the theoretical foundation, guidance algorithm, competitive performance, and presentation of **STARFlow** to be strong. Below, we address each of your concerns and questions in detail.\n\n---\n\n## 1. Inference Speed and Comparison\n\nWe appreciate your comments regarding inference speed. While STARFlow employs a deep-shallow separation to enhance efficiency, we agree that the autoregressive nature of the model may still lead to slower inference, especially at high resolutions. In response, We will provide a detailed comparison of inference time and FLOPs across baselines and STARFlow variants.\nBesides, there are also promising techniques which might be useful to improve the sampling speed such as Jacobi iteration. We will include these discussion in the updated version as well.\n\n---\n\n## 2. Controllable Generation\n\nThank you for emphasizing the importance of controllability. We have included additional interactive editing results in Appendix Fig 10, and agree that control inputs like pose, depth, and sketch can be considered special cases of the broader editing framework we present. Our framework should be able to handle these editing cases as well with paired training set. More detailed experiments should be conducted to demonstrate. Please let us know if there are specific types of control signals we may have overlooked and will include them in the revised version.\n\n---\n\n## 3. Decoder Fine-tuning and Generalization\n\nWe acknowledge the concern regarding decoder fine-tuning. In STARFlow, the VAE decoder is fine-tuned to better reconstruct from noisy latent codes, thereby improving sample quality. As noted, this approach may not generalize across datasets or domains. However, this limitation also points to an exciting research opportunity. In practice, we find that the fine-tuning process is **lightweight**, and only needs to be performed once per VAE, making it a practical and acceptable interim solution. In the future, we will also explore more end-to-end approaches without relying on finetuning the VAEs. \n\n---\n\n## 4. Joint Latent–NF Modeling\n\nAs mentioned in our limitations, the joint design of latent space modeling and normalizing flows remains an open question. While we currently rely on decoder fine-tuning to bridge this gap, we agree that a principled joint training approach is a promising direction (especially for cases VAEs are hard to finetune like videos) for future work. We thank the reviewer for highlighting this important opportunity!\n\n---\n\n## 5. Temporal Extension for Video Modeling\n\nThank you for the insightful suggestion on extending STARFlow to video modeling. We fully agree and it can be ``THE NEXT BIG STEP`` to show. This direction introduces both **challenges** and **opportunities**:\n\nThe temporal coherence and causal nature of video data are naturally well-suited for modeling with autoregressive flows. However, this direction presents distinct challenges and opportunities:\n\n* **Challenge: Temporal Coherence vs. Latent Noise.** As you have rightly pointed out, introducing noise into the latent space can disrupt frame-to-frame coherence. Unlike in static images where visual fidelity is the primary goal, video generation demands strict temporal consistency. Adding noise can degrade this essential structure, leading to suboptimal performance and creating significant challenges for fine-tuning the underlying VAE.\n* **Opportunity: Causal and Interactive Prediction.** A further challenge, which we also view as a significant opportunity, is extending STARFlow to causal and interactive video prediction. Diffusion models are often suboptimal for such tasks due to their non-causal sampling process. STARFlow's autoregressive foundation may provides a more direct and potentially advantageous framework for handling causal generation.\n\nThis tradeoff between modeling flexibility and temporal preservation highlights both the risks and promises of this research direction, which we are excited to explore in future extensions of our work.\n\n---"}, {"review_id": "iZsg0onxH5", "reviewer": "Reviewer_382B", "summary": "The authors show for the first time that autoregressive flow models can be scaled to deliver compelling high-resolution text-to-image generations approaching the level of quality of diffusion-based models. This result is achieved through a thorough theoretical and technical analysis of autoregressive flow models and underlying architecture designs. This analysis results in an improved deep-shallow design for the model backbone, a principled CFG guidance mechanism and the possibility to train the model in the latent space of an VAE. Experimental results show extensive ablations for all the proposed components and comparison on ImageNet 256, 512 and text-conditioned image generation, producing promising results. Finally, image inpainting and editing results are shown. The results presented in the work pave the way for a renewed exploration of normalizing flows as alternative generation frameworks to the current state-of-the-art.", "questions": "Main questions affecting my rating:\n- Will the code be open sourced? While the authors answer [Yes] to the \"Open access to data and code\" section of the checklist, the justification suggests that code will not be released. The absence of an open source implementation might hamper further development of this research direction, reducing significance.\n- A missing aspect in the paper is an analysis of training and inference costs with respect to baseline methods in Tab. 1, 2 and 3. This is not a critical issue, and the aspect is briefly touched upon in the limitations section. I suggest a thorough inclusion of training and inference flops and times as part of the experimental section to further strengthen the work.\n\nMinor doubts not affecting my rating:\n- The method requires application of noise to the latent space with a high sigma value of 0.3 (LL151 and LL153). This is fairly discussed as a limitation in the manuscript. The employed SD auto encoder is already trained as a VAE, making it possible to sample Z from the posterior distribution, resulting in a similar application of noise. Could the authors discuss why posterior sampling alone would not be a sufficient source of noise?\n- Eq. (7) shows noise being applied in the pixel space, before the Encoder, while LL151 discusses application of the noise to the latent space. Could the authors clarify this aspect?\n- LL208 why is the T5-XL encoder preferred over the XXL variant?", "limitations": "- Limitations are discussed in detail in the supplementary material", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 4, "originality": 3, "strengths_and_weaknesses": "QUALITY\n- The paper takes a holistic view at autoregressive normalizing flows and thoroughly analyzes their design both theoretically and empirically. The analysis results in a model approaching performance of state-of-the-art diffusion-based generators.\n- A missing aspect in the paper is an analysis of training and inference costs with respect to baseline methods in Tab. 1, 2 and 3. While inference times are informally discussed as a limitation in the supplementary material, a thorough benchmarking of training flops, inference flops and inference times for the proposed method and baselines reported in Tab. 1, 2 and 3 would further strengthen the experimental section.\n\nCLARITY\n- The paper is clearly explained and written. Figures summarize the framework intuitively, theoretical results are clearly presented and accompanied by proofs. The supplementary material discussed implementation details and limitations extensively.\n\nSIGNIFICANCE\n- The paper shows for the first time that normalizing flows can approach the quality of state-of-the-art methods in text-to-image image generation. While a quality gap still exists, the work is highly significant as it provides strong theoretical and empirical results that can pave the way for further development of normalizing flows as alternative generation frameworks to the current state-of-the-art.\n- A series of theoretical results are proven that characterize the expressive power of normalizing flows and derive a principled CFG mechanism for them\n- Extensive ablations are shown into model design that will be informative to guide future work in this research direction\n- The feasibility of performing downstream applications such as image inpainting and editing is shown\n\nORIGINALITY\n- The work is the first to scale autoregressive normalizing flows, reaching a level of quality approaching large-scale text-conditioned image generators.\n- The authors refine autoregressive normalizing flows through their analysis. Novel contributions such as a the deep-shallow design and a principled CFG mechanism are introduced", "was_revised": true, "final_justification": "The author's provided answer on inclusion of performance metric and confirmation that the code will be open sourced make me keep my positive rating and I continue to argue for acceptance.", "rebuttal": "We sincerely thank the reviewer for their positive evaluation and constructive feedback. We are encouraged by your recognition of our contributions in advancing autoregressive normalizing flows, and we appreciate your comments on the theoretical, empirical, and architectural aspects of our work. Below, we address your specific comments in detail.\n\n---\n\n\n## 1. Request for Inference Time and Training FLOPs\n\nWe agree that a more thorough quantitative analysis of training and inference efficiency would strengthen the paper. While we provided a brief discussion in the limitations section, we acknowledge the importance of presenting more concrete numbers. In the revision:\n\n- We will expand the supplementary material to include:\n  - Inference time per image for STARFlow and all baselines\n  - Training and sampling FLOPs per iteration\n  - A runtime vs. sample quality trade-off comparison across methods\n\nWe believe these additions will provide a clearer understanding of STARFlow’s efficiency profile.\n\n---\n\n## 2. Code Availability\n\n**We will release the code on public**, and the checkpoint release is subject to legal related consideration.\n\n---\n\n## 3. Why Not Use Posterior Sampling Instead of Injecting Noise?\n\nThank you for raising this insightful question. In principle, posterior sampling from the VAE is a reasonable approach. However, in practice, we observe that the standard deviation of the VAE posterior tends to be extremely small, often around  ``1e-4`` level, due to the training dynamics and the ``tiny`` weighting of the KL divergence term. After all, pretrained AEs are mainly for reconstruction. So having a high entropy latent is not useful in their objectives.\nAs a result, the injected noise is insufficient to effectively train the normalizing flow component of STARFlow. Therefore, we opt to introduce noise explicitly at a calibrated level to enable robust training.\nHowever, in the future, one can also explore options of training a tokenizer from scratch with native entropy regularization in the latent space for STARFlow.\n\n---\n\n## 4. T5-XL vs. T5-XXL Preference\n\nWe used T5-XL as a representative encoder. In our preliminary experiments, we find T5XL has almost the same performance as T5XXL on text-conditioned generation while with significant small memory usage.\nAlso, our goal was to demonstrate STARFlow's generality, and we anticipate similar outcomes with larger or alternative encoder backbones. We are open to further exploration in future work.\n\n---"}, {"review_id": "lWBXy7ioVd", "reviewer": "Reviewer_Zv2v", "summary": "The authors introduce STARFlow, a scalable generative model grounded in autoregressive flows (AFs). They first prove the theoretical ability of AFs to model continuous distributions. Then, they propose several innovations to boost the scalability of normalizing flows: a deep-shallow architecture balancing capacity and efficiency, latent-space learning via pretrained autoencoders instead of direct pixel modeling, and a new guidance algorithm for better sample quality. Notably, STARFlow remains a single, end-to-end normalizing flow enabling exact continuous-space maximum likelihood training. The model shows competitive performance in class-to-image and text-to-image generation, with sample quality close to leading diffusion models. This work represents the first successful application of normalizing flows at such a large scale and high resolution.", "questions": "1. Could you elaborate on the inference process details with Classifier-Free Guidance (CFG)? Specifically, does Starflow apply CFG **only to each deep block**, or **to intra-layers within each block**?\n2. STARFlow's SFT performance on GenEval and DPG benchmark. As Blip-3o open-sourced a high-quality SFT dataset, this would be a good SFT data choice[1].\n3. Could you provide more visualizations or online demos?\n4. Can the authors discuss the advantages compared to latent diffusion models?\n\n[1] https://huggingface.co/datasets/BLIP3o/BLIP3o-60k", "limitations": "see weakness", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths\n1. The writing is clear and easy to understand.\n2. The proposed deep-shallow structure sounds reasonable.\n2. The pixel decoder seems natural and makes sense. Traditional Langevin sampling and diffusion methods involve heavy computational costs, which restrict their applications.\n3. The performance improvement is significant.\n\nWeaknesses\n1. The performance of T2i on the GenEval benchmark appears inferior. I conjecture that a proper supervised fine-tuning (SFT) stage could significantly boost its performance.\n2. The inference resources, including memory usage, time consumption, and GFlops, should be reported.\n3. The scaling property of STARFlow versus parameters or Flops.", "was_revised": true, "final_justification": "After carefully reviewing the author’s response, I have decided to maintain the current score (Accept).", "rebuttal": "We sincerely thank the reviewer for their constructive feedback and thoughtful evaluation. We are pleased that you found the writing clear, the deep–shallow design reasonable, and the performance improvements meaningful. Below, we address your comments in detail.\n\n---\n\n# Weaknesses\n\n## 1. GenEval Performance and Supervised Fine-Tuning (SFT)\n\nWe thank the reviewer for this valuable suggestion regarding supervised fine-tuning (SFT). We would like to clarify that all current numerical results were obtained without any SFT or Reinforcement Learning (RL) steps, which relies solely on the pre-training data.\n\nWe fully agree that applying SFT—particularly with powerful instruction datasets like those from recent work such as BLIP-3o [1]—would significantly improve performance. Thank you for highlighting this important direction. We will add a discussion to our paper and plan to include results from SFT and RL fine-tuning in future work to further improve our benchmark scores.\n\n\n---\n\n## 2. Reporting of Inference Resources\n\nWe agree that a more detailed analysis of training and inference efficiency would strengthen the work. While we briefly discussed this in the limitations section, we will enhance the supplementary material with:\n\n- Inference time per image across STARFlow variants and all the baselines with a table comparison.  \n- Training and sampling FLOPs per iteration  \n- A runtime vs. sample quality trade-off analysis\n\n\n---\n\n## 3. Scaling Efficiency of STARFlow\n\nWe acknowledge that understanding STARFlow’s performance scaling with model size and computational cost is important. \nWe beliefly showed the scaling curves againt the model sizes in Fig 8.\nWe have also conducted several ablation studies in this direction and will include these results in the revised version. Thank you for pointing this out!\n\n---\n\n## 4. Application of CFG Across STARFlow Blocks\n\nThe classifier-free guidance (CFG) mechanism proposed in Proposition 2 is applied exclusively at the output of the deep blocks. Here is a more detailed breakdown:\n\n* **Conditioning and Purpose:** In our implementation, only the deep blocks accept conditioning inputs. The purpose of CFG is to integrate the **conditional prediction** (using the text prompt) and the **unconditional prediction** (using a null token) to produce an output that more strongly reflects the guidance.\n* **Mechanism:** At each autoregressive step within a deep block, the final layer predicts two sets of parameters: $(\\mu_c, \\sigma_c)$ for the conditional case and $(\\mu_u, \\sigma_u)$ for the unconditional case. The CFG function then recalculates these into a single, guided output: $\\tilde{\\mu}, \\tilde{\\sigma} = f_{CFG}(\\mu_c, \\sigma_c, \\mu_u, \\sigma_u)$.\n* **Derivation:** The specific function $f_{CFG}$ used in our paper, as shown in Proposition 2, is mathematically derived from the score function perspective of the deep block. We note that alternative implementations may also be possible.\n* **Application:** Analogous to guidance in diffusion models, the intermediate layers of a block operate independently of the CFG calculation. Guidance is applied only to the final output of a step. This guided output then determines the input for the subsequent autoregressive step via the equation $x = \\tilde{\\mu} + \\tilde{\\sigma} \\cdot z$.\n\n---\n\n## 5. Additional Visualizations and Demos\n\nWe will include more qualitative examples in the new appendix, including samples for GenEval-specific queries. Additionally, we are planning to release an online demo to better showcase STARFlow’s capabilities.\n\n---\n\n## 6. Comparison with Latent Diffusion Models\n\nBelow, we summarize STARFlow’s advantages over latent diffusion models:\n\n- **Stronger likelihood guarantee** through direct maximum likelihood training in continuous latent space.\n- **Faster convergence** in terms of FID, using the same backbone (see Fig. 8a).\n- **End-to-end trainability**, which opens new avenues for research in image generation.\n- **Higher sampling throughput** due to autoregressive flow structure.\n- **Easier to combine with LLMs** due to the shared autoregressive structure in deep blocks.\n\n---\n\n# Reference\n\n[1] [BLIP-3o Dataset on Hugging Face](https://huggingface.co/datasets/BLIP3o/BLIP3o-60k)\n\n---\n\nWe thank the reviewer again for the thoughtful suggestions. We believe the planned revisions and additional results will help address the concerns and further strengthen the paper."}]

Author rebuttal:

Meta-review: The paper presents STARFlow, an autoregressive flow (a type of normalizing flow) for high-resolution image synthesis.
It builds on TARFlow and introduces key architectural and algorithmic changes to improve scalability, including a deep–shallow design and feature-space learning.

Reviewers unanimously praised the work for its novelty, competitive performance, and extensive experiments, and recommended acceptance as of broad interest to the computer vision community.

For the camera-ready, as requested by the reviewers and acknowledged by the authors, please report detailed training and inference efficiency/cost metrics and release the source code.
Decision: Accept (spotlight)