DiCo: Revitalizing ConvNets for Scalable and Efficient Diffusion Modeling
Authors: Yuang Ai
Qihang Fan
Xuefeng Hu
Zhenheng Yang
Ran He
Huaibo Huang
Venue: NeurIPS 2025 spotlight ()
Keywords: Image Generation
Diffusion Models
ConvNets
Primary area: deep_learning
TL;DR: A simple yet powerful Diffusion ConvNet

Abstract: Diffusion Transformer (DiT), a promising diffusion model for visual generation, demonstrates impressive performance but incurs significant computational overhead. Intriguingly, analysis of pre-trained DiT models reveals that global self-attention is often redundant, predominantly capturing local patterns—highlighting the potential for more efficient alternatives. In this paper, we revisit convolution as an alternative building block for constructing efficient and expressive diffusion models. However, naively replacing self-attention with convolution typically results in degraded performance. Our investigations attribute this performance gap to the higher channel redundancy in ConvNets compared to Transformers. To resolve this, we introduce a compact channel attention mechanism that promotes the activation of more diverse channels, thereby enhancing feature diversity. This leads to Diffusion ConvNet (DiCo), a family of diffusion models built entirely from standard ConvNet modules, offering strong generative performance with significant efficiency gains. On class-conditional ImageNet generation benchmarks, DiCo-XL achieves an FID of 2.05 at 256$\times$256 resolution and 2.53 at 512$\times$512, with a **2.7$\times$** and **3.1$\times$** speedup over DiT-XL/2, respectively. Furthermore, experimental results on MS-COCO demonstrate that the purely convolutional DiCo exhibits strong potential for text-to-image generation.

DiCo: Revitalizing ConvNets for Scalable and Efficient Diffusion Modeling
Yuang Ai!
Qihang Fan 1.2
Xuefeng Hu
Zhenheng Yang*
Ran Hel
Huaībo Huang 1.2
CASIA UCAS
*ByteDance
Code and models:
com. http://www.....oaaaaaaaaaaaaaaae
ORI
Figure Difffusion ConvNet achieves superior image quality with high efficiency. We show samples from two of our DiCo-XL models trained on ImageNet at 512x512 and 2 resolution.
Abstract
Diffusion Transformer (DiT), promising diffusion model for visual generation, demonstrates impressive performance but incurs significant computational over- head. Intriguingly, analysis of pre-trained DiT models reveals that global self- attention is often redundant, predominantly capturing local patterns-hight hatterns--tihhlighting the potential for efficient alternatives. In this paper, revisit convolution as an alternative building block for constructing efficient and expressive diffusion models. However, naively replacing self-attention with convolution typically re- sults in degraded performance. Our investigations attribute this performance gap the higher channel redundancy in ConvNets compared Transformers, To resolve this, we introduce a compact channel attention mechanism that promotes the activation more diverse channels, thereby enhancing feature diversity. This leads to Diffusion ConvNet (DiCo), family of diffusion models built entirely from
"Corresponding author: Huaibo Huang chuaibo. chuaibo.huang ia.....c....cccm...
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

standard ConvNet modules, offering strong generative performance with significant efficiency gains. On class-conditional ImageNet generation benchmarks, DiCo-XL achieves an FID of 2.05 at x resolution and 2.53 at 512x512, with 2.7x and 3.1 x speedup over DiT-X1/2, respectively. Furthermore, experimental results on MS-COCO demonstrate that the purely convolutional DiCo exhibits strong potential for text-to-image generation.
1 Introduction
Diffusion models [73. 75, 33, 74, 76] have sparked a transformative advancement in generative learning, demonstrating remarkable capabilittes in synthesizing highly photorealistic visual content. Their versatility and effectiveness have led to widespread adoption across a broad spectrum of real- world applications, including text-to-image generation [66, 69, 67], image editing [59, 46, 10], image restoration [45, 3, 4], video generation [36, 88, 7]. and 3D content creation [64, 87,84) 84).
Early diffusion models (e.g., ADM [14] and Stable Diffusion [67]) primarily employed hybrid U-Net [68] architectures that integrate convolutional layers with self-attention. More recently, Transformers [83] have emerged a more powerful and scalable backbone [62. 6]. prompting a shift toward fully Transformer-based designs. As a result, Diffusion Transformers (DiTs) are gradually supplanting traditional U-Nets, as seen in leading diffusion models such as Stable Diffusion 3 [20], FLUX [49], and Sora [9]. However, the quadratic computational complexity of self- self-attention presents substantial challenges, especially for high-resolution image synthesis. Recent efforts [100, 79, 25, 63, 91, 2] have explored more efficient alternatives, focusing on linear-complexity RNN- like architectures, such as Mamba [26] and Gated Linear Attention [92]. While these models improve efficiency, their causal design inherently conflicts with the bidirectional nature of visual generation [30, 55], limiting their effectiveness. Furthermore, as illustrated in Fig. even with highly optimized CUDA implementations, their runtime advantage over conventional DiTs remains modest in high-resolution settings. This leads us to a key question: Is possible to design a hardwure-effiieet diffusion buckbone thut also preserves strong generative capabilities like DiTs?
To approach this question, we begin by examining the characteristics that underlic the generative power of DiTs. In visual recognition tasks, the success of Vision Transformers [18] is often credited to the self-attention's 's ability to capture long-range dependencies [42, 23, 22]. However, in generative tasks, we observe a different dynamic. As depicted in Fig. 4, for both preitedined class-conditional (DiT-XL/2 [62]) and text-to- -o-tto--mmaee (PixArt-a [12] and FLUX [49]) DiT models, when queried with an anchor token, attention predominantly concentrates on nearby spatial tokens, largely disregarding distant ones. This finding suggests that computing global attention may be redundant for generation, underscoring the significance of local spatial modeling. Unlike recognition tasks, where long-range interactions are critical for global semantic reasoning, generative tasks appear to emphasize fine- grained texture and local structural fidelity. These observations reveal the inherently localized nature of attention in DiTs and motivate the pursuit of more efficient architectures.
In this work, we revisit convolutional neural networks (ConvNets) and propose Diffusion ConvNet (DiCo), simple yet highly efficient convolutional backbone tailored for diffusion models. Compared to self-attention, convolutional operations are more hardware-friendly offering significant advantages for lcrge-scale and essource-constrainee deployment. While substituting self-attention with convolu- tion substantially improves efficiency, it typically results in degraded performance. As illustrated in Fig. 5, this naive replacement introduces pronounced channel redundancy, with many channels remaining inactive during generation. We hypothesize that this stems from the inherently stronger representational capacity of self-attention compared to convolution. To address this, we introduce compact channel attention (CCA) mechanism, which dynamically activates informative channels with lightweight linear projections. As a channel-wise global modeling approach, CCA enhances the model's s representational capacity and feature diversity while maintaining low computational over- head. Unlike modern recognition ConvNets that rely on large, costly kernels [15, 28]. DiCo adopts streamlined design based entirely on efficient 1 1x pointwise convolutions and 3x3 depthwise convolutions. Despite its architectural simplicity, DiCo delivers strong generative performance.
As shown in Fig. 2 and Fig. 3, DiCo models outperform recent diffusion models on both the ImageNet 256x256 and 512x512 benchmarks. Notably, our DiCo-XI. models achieve impressive FID scores of 2.05 and 2.53 at 256 2256 and 512x512 resolution, respectively. In addition to performance gains,
2

---

(a) FID vs. FLOPs w/ ing__ize size - 256
(b) FID vs. Runtime w/ ing sin_size 256
DIT (ICCV23)
DiT (ICCV23)
FID-50K (400K iter.)
65
(CVPR25)
FID-50K (400K iter.)
65
DIT-FlashAttn
DIG (CVPR25)
DIG (CVPR25)
45
DICo (ours)
45
DiCo (ours)
2,9x faster, 1.6x gain
5
5
0
25
50
75
100
125
3
6
12
15
FLOPs (G)
Runtime (ms)
Comparison of performance and efficiency with recent diffusion models (DiT [62], DIC [81], and DiG [100]) on ImageNet 256x256. Our proposed DiCo achieves the best performance while maintaining high efficiency. Compared DiG-XI/22 with CUDAA-ptimized Flash Linear Attention [92], DiCo-XI. runs 2.9x faster and achieves a 1.6x improvement in FID.
Figure 2:
(a) Runtime Comparison w/ img size 512
(b) FID vs. Runtime w/ - 512
DIT (ICCV23)
U-VIT-L/4
90
DIS (arXiv24)
4.4
Runtime (ms)
DIG (CVPR25)
6.8x faster
FID -0-50K (w/ cfg)
U-VIT-H/4
DIM-H
60
DiCo (ours)
3.6
7 7.8x faster, 1.5x gain
DIT-XL/2
30
2.8
DIS-H/2
DiCo-XL (ours)
2
Small
Base
Large
XLarge
35
70
105
140
Model Variants
Runtime (ms)
Figure 3: (a) Runtime comparison between DiT [62], DiS [25] (with Mamba [26]), DiG [100] (with Gated Linear Attention [92]). and our DiCo 5 512 resolution. DiCo is 3.3x faster than DiS at the small model scale and 6.8 faster at the XL scale. (b) FID vs. runtime of various methods on ImageNet 512x512. DiCo-XL XL achieves FID of 2.53 while maintaining high efficiency.
DiCo models exhibit considerable efficiency advantages over attention [83], Mamba- [26], and linear attention- based [44] diffusion models. Specifically, at x56x resolution, DiCo-XL achieves 26.4% reduction in Gflops and is 2.7 x faster than DiT-XI/2 [62]. At 512x512 resolution, DiCo-XI. operates 7.8x and 6.7x faster than the Mamba-based DiM-H [79] and DiS-H/2 [25] models, respectively. Our largest model, DiCo-H with billion parameters, further reduces the FID) on ImageNet 256x256 to 1.90. In addition, we validate the applicability DiCo for text-to-image generation on the MS-COCO dataset. These results collectively highlight the strong potential of DiCo diffusion-based generative modeling.
Overall, the main contributions of this work can be summarized as follows:
We analyze pre-trained DiT models and reveal significant redundancy and locality within their global attention mechanisms. These findings may inspire researchers to develop more efficient strategies for constructing high-performing diffusion models.
We propose DiCo, simple, efficient, and powerful ConyNet backbone for diffusion models. By incorporating compact channel attention, DiCo significantly improves representational capacity and feature diversity without sacrificing efficiency.
We conduct extensive experiments class-conditional ImageNet benchmarks. DiCo outperforms recent diffusion models in both generation quality and speed. Furthermore, the purely convolutional DiCo demonstrates strong potential in text-to-imase generation.
3

---

(a) Attention maps from DiT blocks of DiT-XL/2-512 [62].
(b) Attention maps from DiT blacks of PixArt-a-512 [12].
Hello DiCo
(c) Attention maps from MM-DiT [20] blocks of FLUX.1-dew [49].
Figure 4: Visualization of attention maps from well-known DiT models. The intensity of the blue color indicates the magnitude of attention scores. For self-attention across different layers in these models, only a few neighboring tokens contribute significantly to the attention distribution of a given anchor token (red box), resulting in highly redundant and localized representations.
432
10
(a) DiT [62]
(b) DiCo w/o channel activation
(c) DiCo (ours)
Figure 5: Comparison of channel activation scores across different diffusion models. Channel
activation scores are computed using ReLU followed by global average pooling on the final layer's self-attention convolution outputs [102]. Directly replacing self-attention in DiT with convolution introduces significant channel redundancy, as most channel activation scores remain at low levels.
2 Related Work
Architecture of Diffusion Models. Early diffusion models commonly employ U-Net [68] as the foundational architecture [14, 34, 67]. More recently, a growing body of research has explored Vision Transformers (ViTs) [18] as alternative backbones for diffusion models, yielding remarkable results [62, 6, 58, 96, 65, 54]. Notably, DiT [62] has demonstrated the excellent performance of transformer-bes based architectures, achieving SOTA performance on ImageNet generation. However, the quadratic computational complexity inherent in VIT' presents significant challenges in terms of efficiency long sequence modeling. mitigate this, recent studies have explored the use of RNN-like architectures with linear complexity, such as Mamba [26] and linear attention [44], as backbones for diffusion models [25, 100, 79, 91, 63]. DiS [25] and DiM [79] employ Mamba to reduce computational overhead, while DiG [100] leverages Gated Lincar Attention [92] achieve competitive performance with improved efficiency.
this work, we revisit ConvNets as backbones
for difffusion models. We show that, with proper design, pure convolutional architectures can achieve superior generative performance, providing an efficient and powerful alternative to DiTs.
ConvNet Designs. Over the past decade, convolutional neural networks (ConvNets) have achieved remarkable success computer vision [31, 41, 89, 5, 19]. Numerous lightweight ConvNets have been developed for real-world deployment [39, 71, 38, 16]. Although Transformers have gradually become the dominant architecture across wide range of tasks, their substantial computational overhead remains significant challenge. Many modern ConvNet designs achieve competitive performance

---

while maintaining high efficiency. ConvNeXt [57] explores the modernization of standard ConvNets and achieves superior results compared to transformer-basee based models. RepLKNet [15] investigates the use of large-kernel convolutions, expanding kernel sizes up to 31 x31. MeiRRepLKNNe [17] further generalizes large-kernel ConvNets to domains such as audio, point clouds, and time-series forecasting. In this work, we explore the potential of pure ConvNets for diffusion- show that simple, efficient ConvNet designs can also deliver excellent performance.
image generation, and
3 Method
3.1 Preliminaries
Diffusion formulation. We first revisit essential concepts underpinning diffusion models [33, 76]. Diffusion models are characterized by forward noising procedure that progressively injects noise into a data sample øo. Specifically, this forward process can be expressed as:
T
q( (m)) = II (1)))==)))) = ( (1 (1--aj) a,))), 1=1
(I)
where og are predefined hyperparameters. The objective of a diffusion model is to learn the reverse
process: (0 (x2) (z)) [x(x))) where neural networks parameterize the mean and
covariance of the process. The training involves optimizing a variational lower bound on the log- likelihood of zo, which simplifies to:
(()) = p((ola1) + DDLLl 2o)lo
(2)
To simplify training, the model's predicted mean Pa can be reparameterized as noise predictor eg. The objective then reduces straightforward mean-squared error between the predicted noise and the true noise c₁: Leimpte (f) = ||co ce11. Following DiT [62], we train the noise predictor co using the simplified loss Lsimple. while the covariance Eg is optimized using the full loss L.
Classifier-free guidance. Classifier-free guidance (CFG) [35] is an effective method to enhance sample quality in conditional diffusion models. It achieves such enhancement by guiding the sampling process toward outputs strongly associated with a given condition c. Specifically, modifies the predicted noise to obtain high p(x|cc as: (g( c) (2 0) + s V log p(xle) x (I 0) + (ea (z+, c) - eg (Zt, 0)). where 2 controls the guidance strength, and eø 0) is an unconditional prediction obtained by randomly omitting the conditioning information during training. Following prior works [62, 100], we adopt this technique to enhance the quality of generated samples.
3.2 Network Architecture
Currently, diffusion models are primarily categorized into three architectural types: (1) Isotropic architectures without any downsampling layers, as seen DiT [62]; (2) Isotropic architectures with long skip connections, exemplified by U-ViT [6]; and (3) U-shaped architectures, such as U-DiT [82]. Motivated by the crucial role of multi-scale features in image denoising [97, 1]. we adopt U-shaped design to construct hierarchical model. We also conduct extensive ablation study to systematically compare the performance of these different architectural choices in Table 4.
As illustrated in Fig. 6 (a), DiCo employs three-stage U-shaped architecture composed of stacked DiCo blocks. The model takes the spatial representation generated by the VAE encoder as input. For an image of size 256 x 256 3, the corresponding = has dimensions 32 x 32 x 4. To process this input, DiCo applies a 3 x convolution that transforms & into an initial feature map 20 with D channels. For conditional information-sspciffcallll specifically, the timestep t and class label y-we employ lulti-layer perceptron (MLP) and an embedding layer, serving as the timestep and label embedders, respectively. At each block ( within DiCo, the feature map 2 1 is passed through the (-th DiCo block to produce the output 21.
Within each stage, skip connections between the encoder and decoder facilitate efficient information flow across intermediate features. After concatenation, a x convolution applied to reduce the channel dimensionality. To enable multi-scale processing across stages, we utilize pixel-unshuffle operations for downsampling and pixel-shuffle operations for upsampling. Finally, the output feature 21 is normalized and passed through 3 3 convolutional head to predict both noise and covariance.
5

---

(a) Latent Diffusion ConvNet
DiCo Block
(c) Conv Module
Noieed
Timestep
Noise
Σ
1x1 Gonv
Labely
U-Shaped Architecture
Scale
Og
Compact
Conv
Embed
Layer Norm Conv
Feedforward
02
GELU
Scale Shift
DiCo Blocks
x
Slage
DiCo Blocks
3x3 DCanv
Concal & Linear
Norm
1xt Conv
2x Downssmple
2x Upsample
Scale
(d)
DiCo Blocks
x
Slage
DiCo Blocks
xL1
Conv Module
Concat Linear
Scale Shift
Sigmoid
2x Downsample Stage 3
2x Upsample
Layer Norm
MLP
1x1 Conv
DiCo Blocks
Globai AugPool
Inpul
Figure 6: Architecture of DiCo, which consists of (b) DiCo Block, (c) Conv Module, and (d)
Compact Channel Attention (CCA). DConv denotes depthwise convolution.
3.3 DiCo Block
Motivation. As shown Fig. 4, the self-attention computation in DiT models-whethee for condiicon or text-to-image generation enterattoo-exhibiits distinctly local structure and significant redundancy. This observation motivates us to replace the global self-attention in DiT with more hardware-efficient operations. A natural alternative is convolution, which is well-known for its ability to efficiently model local patterns. We first attempt to substitute self-attention with combination of 1 x pointwise convolutions and 3 depthwise convolutions.
However, the direct replacement leads to a degradation in generation performance. As shown in Fig. 5, compared to DIT, many channels in the modified model remain inactive, indicating substantial channel redundancy. We hypothesize that this performance drop stems from the fact that self-attention, being dynamic and content-dependert which relies on static weights. To address this limitation, we introduce a compact channel attention mechanism to dynamically activate informative channels. We describe the full design in detail below.
provides greater representational power than convolution,
Block designs. The core design of DiCo is centered around the Conv Module, as shown Fig. (c). We first apply x convolution to aggregate pixel-wise cross-channel information, followed by a x depthwise convolution to capture channel-wise spatial context. GELU activation employed for non-linear transformation. To further address channel redundancy, introduce compact channel attention (CCA) mechanism to activate more informative channels. As illustrated in Fig. 6 (d), CCA first aggregates features via global average pooling (GAP) across the spatial dimensions, then applies learnable x 1 convolution followed by sigmoid activation to generate channel-wist attention weights. Generally, the whole process of Conv Module can be described as:
Y Wr, CCA(GELL(W..,,, X)), CCA(X) X Sigmoid(W,GAP(XX).
where WT ) is the 1 x point-wise conv, Wa is the depthwise conv, and @ denotes the channel-wise multiplication. As shown in Fig. 5 (c), this simple and efficient design effectively reduces feature redundancy and enhances the representational capacity of the model. To incorporate conditional information from the timestep and label, we follow DiT by adding the input timestep embedding and label embedding y, and using them to predict the scale parameters a, and the shift parameter B.
Modification for text-to-image. We investigate two different approaches for incorporating textual features into DiCo. The first uses the widely adopted cross- attentttt [12] mechanism, integrated into DiCo architecture to fuse text and visual features. The second transforms CLIP text embeddings into dynamic depthwise convolution (DWC) kernels. We pad the text embeddings to length of 81, feed them through a learnable MI.,, and reshape the output into a 9 x 9 kernel. This kernel dynamically modulates DiCo's 's features via depthwise convolution. In this way, can construct

---

Algorithm PyTorch code of text conditional depthwise convolution
import torch
import torch.nn. fu..tiiiinnn
def text, coxdiiinili diconv (x, context):
(8, C, H, W) input feature maps
context: (B, 77, c) CLIP text eabeddings after an NLP
H, output B, C, H, x x.shape context_pad corch.cat (ccottrr,, contextl:, -1:] expand(-1, 4, -1)], din-1) (B, 0) kernela context_pad res.--paaaaerrra...... 9, 9, O-permta(O, a, 1, reshhape C, 1, 9, 9)
outpat:
depthwise convolution
x_flat x.view(1, B C, B, W)
output F.conv2d(x_ffas. kernele, padding=4, groups-B Viviie C, H, return output
fully convolutional exx--to-image DiCo model without relying on any self-attention or cross-attention operations. We provide its detailed PyTorch implementation in Algorithm Both feature fusion modules are inserted after the Conv Module within each DiCo block.
3.4 Architecture Variants
We establish four model Pariants-DiCo-- DiCo-B, DiCo-L, and DiCo-XL- whose parameter counts are aligned with those of DiT-S/2, D/T-B/2, DiT-L/2, and DiT-XL/2, respectively. Compared their DiT counterparts, our DiCo models achieve a significant reduction in computational cost, with Gflops ranging from only 70.1% to 74.6% of those of DīT. Furthermore, to explore the potential of our design, we scale up DiCo to billion parameters, resulting in DiCo-H. The architectural configurations of these models are detailed in Appendix Table 6.
4 Experiments
4.1 Experimental Setup
Following previous works [62, 100, 81], we conduct experiments on class- conditional ImageNet-1K 1K [13] generation benchmark at 256x256 and 512x512 resolutions. We use the Fréchet Inception Distance (FID) [32] as the primary metric to evaluate model performance. In addition, we report the Inception Score (IS) [70], Precision, and Recall [48] as secondary metrics. All these metrics are computed using OpenAl's TensorFlow evaluation toolkit [14].
Datasets and Metrics.
Implementation Details. For DiCo-S/B/L/XL, we adopt exactly the same experimental settings as used for DiT. Specifically, we employ a constant learning rate of 10-4, no weight decay, and batch size of 256. The only data augmentation applied is random horizontal flipping. We maintain an exponential moving average (EMA) of the DiCo weights during training, with a decay rate of 0.9999. The pre-trained VAE [67] is used to extract latent features. For our largest model, DiCo-H, we follow the training settings of U- VIT [6], increasing the learning rate to 10-4 and scaling the batch size to 1024 to accelerate training. Additional details are provided in Appendix Sec. B.
4.2 Main Results
Comparison under the DiT Setting. In addition to DiT [62], we also select recent diffusion models, DiG [100] and DiC [81], as baselines, since they similarly follow the experimental setup of DiT.Table presents the comparison results on ImageNet 256x Across different model scales trained for 400K iterations, our DiCo consistently achieves the best or second- best performance across all metrics. Furthermore, when using classifier-free guidance (CFG), our DiCo-XL achieves an FID of 2.05 and an IS of 282.17. Beyond performance improvements, DiCo also demonstrates significant efficiency gains compared to both the baselines and Mamba-base models.
Table 2 presents the results on ImageNet 512x512. At higher resolutions, our model demonstrates greater improvements in both performance and efficiency. Specifically, DiCo-XL achieves an FID of 2.53 and an IS of 275.74, while reducing Gflops by 33.3% and achieving 3.1xx speedup compared DIT-XI1/2. These results highlight that our convolutional architecture remains highly efficient and effective for high- resouutootttion image generation.

---

Table 1: Comparison under the DiT setting on ImageNet 256x256. The performance at 400K
training steps is reported without CFG for early-stage comparison. We mark the best results for each model scale bold. Throughput (image/s) measured on A100 with batch size at fp32 precision. DiT and DiG are optimized with FlashAttention-2 and Flash Linear Attention, respectively.
Model
Token Mixing Type
Gflops
Throughput
FID
1St
Pre.t
Rec.t
ADM-U [14]
Conv + Attn
742
3.94
215.84
0.83
0.53
LDM-4 [67]
Conv Attn
3.95
178.22
0.81
0.55
U-V [6]
Attn
133.25
73.45
2.29
263.88
0.82
0.57
Mamba-based diffusion models.
DIM-H [79]
Conv SSM
210
23,06
2.21
DiS-H/2 [25]
Conv SSM
33.95
2.10
271.32
0.82
0.58
DiffuSSM-XL 1911
SSM
280.3
2.28
259.13
0.86
0.56
DiMSUM-L/2 [63]
Attn + SSM
84.49
59.13
0.59
Baselines and Ours (w/ the hyperparameters).
DIT-S/2 (400K) [62]
Attn
6.06
1234.01
68.40
DIC-S (400K) [81]
Conv
5.9
58.68
25.82
DiG-S/2 (400K) [100]
Conv + Attn
4.30
961.24
62.06
22.81
0.39
0.56
DiCo-S (400K)
Conv
4.25
1695.73
49.97
31.38
0.48
0.58
DřT-B/2 (400K)
Attn
23.02
380.11
43.47
DiC-B (400K)
Conv
32.33
48.72
DiG-B/2 (400K)
Conv + Atta
17.07
345.89
39.50
37.21
0.51
0.63
DiCo-B (400K)
Conv
16.88
822.97
27.20
56,52
0.60
0.61
DiT-L/2 (400K)
Attn
80.73
114.63
23.33
DiG-L/2 (400K)
Conv + Attn
61.66
109.01
22.90
59.87
0.60
0.64
DiCo-L (400K)
Conv
60.24
288.32
13.66
91.37
0.69
0.61
DIT-XL/2 (400K)
Attn
118.66
76.90
19.47
DiC-XI (400K)
Conv
116.1
13.11
100.15
DiG-XI./ (400K)
Conv Attn
89.40
71.74
18.53
68.53
0.63
0.64
DiCo-XL (400K)
Conv
87.30
208.47
11.67
100.42
0.71
0.61
DiT-XL/2 (w/ CFG)
Attn
118.66
76.90
2.27
278.24
0.83
0.57
DIG-XL/2 (w/ CFG)
Conv + Atta
89.40
71.74
2.07
278.95
0.82
0.60
DiCo-XL (w/ CFG)
Conv
87.30
208.47
2.05
282.17
0.83
0.59
DiC-H (w/ CFG)
Conv
204.4
2.25
DiCo-H (w/ CFG)
Conv
194.15
117.57
1.90
284.31
0.83
0.61
Scaling the model up. To further explore the potential of our model, scale it up to 1 billion parameters. As shown in Table 1, compared to DiCo-XL, the larger DiCo-H achieves further improvements in FID (1.90 2.05), demonstrating the great scalability of our architecture. More comparison results and generated images can be found in the Appendix Sec. and Sec. E.
Text-to-Image Generation. We follow [6] to conduct small-scale text-to-image generation experi- ments. Specifically, we adopt the same experimental setup as [96]: training and evaluating models from scratch on MS-COCO [53], using CLIP as the text encoder with token length of 77.
As shown Table 3, our DiCo achieves superior generation quality for to-iage generation. Notably, using text conditional DWC in place of cross-attention further improves throughput while maintaining competitive performance. This suggests that the fully convolutional DiCo has potential to serve as the backbone for large-scale text-to-image diffusion models.
4.3 Ablation Study
For the ablation study, we use the small-scale model and evaluate performance on the ImageNet 256x256 benchmark to enable fast training speed. All models are trained for 400K iterations and evaluated without CFG. Notably, this section, self-attention in DiT not accelerated us- ing FlashAttention- 2 to ensure fair speed comparison with other efficient attention mechanisms.

---

Table 2: Comparison under the DiT setting on ImageNet 512x We mark the best performance with CFG in bold. The performance at 1.3M/3M training steps is reported without using CFG.
Model
Token Mixing Type
Gflops
Throughput
FID4
ISt
Pre.1
Rec.1
ADM-U [14]
Conv Attn
2813
3.85
221.72
0.84
0.53
U-ViT-L/4
Attn
76.52
128,49
4.67
213.28
0.87
0.45
U-ViT-H/4 [6]
Attn
133.27
73.42
4.05
263.79
0.84
0.48
Mamba-based diffussion models.
DIM-H [79]
Conv + SSM
708
7.39
3.78
DiS-H/2 [25]
Conv + SSM
8.59
2.88
272.33
0.84
0.56
DiffuSSM-XL [91]
Atin SSM
1066.2
3.41
255.06
0.85
0.49
Baselines and Ours (w/ the same hyperparameters).
DiT-XL/2 (1.3M) [62]
Attn
524.70
18.58
13.78
DiCo-XL (1.3M)
Conv
349.78
57.45
8.10
132.85
0.78
0.62
DIT-XL/2 (3M)
Attn
524.70
12.03
105.25
0.75
0.64
DiCo-XL (3M)
Conv
349.78
57.45
7.48
146.35
0.78
0.63
DIT-XL/2 (w/ CFG)
Attn
524.70
18.58
3.04
240.82
0.84
0.54
DiCo-XL CFG)
Conv
349.78
57.45
2.53
275.74
0.83
0.56
We analyze both the overall architecture and the contributions of individual components within DiCo to better understand their impact on model performance.
Table 3: Comparison for text-to-image genera- tion on MS-COCO. We follow the setup in [96].
Model
Type
FID
Architecture Ablation. We evaluate the perfor- mance of DiCo under various architectural de-
AtteGAN [90]
GAN
35.49
DM-GAN [101]
GAN
32.64
signs and conduct fair comparison with DIT. As shown Table DiCo consistently outperforms DiT across all structures while also delivering sig- nificant efficiency gains. These results highlight the potential of DiCo strong and efficient alternative to DIT.
VQ-Diffusion [27]
Discrete Diffusion
19.75
DF-GAN [78]
GAN
19.32
XMC-GAN [98]
GAN
9.33
Frido [24]
Diffusion
8.97
LAFITE [99]
GAN
8.12
U-Net [6]
Diffusion
7.32
Component-wiss wise Ablation. We conduct a component-wise analysis of DiCo, examining the effects of the activation function, convolutional kernel size, compact channel attention (CCA), and the conv module (CM). The overall ablation results are summarized Table Increasing the convolutional kernel size leads to further perfor-
U-ViT-S/2 [6]
Diffusion
5.95
U-VIT-S/2 (Deep) [6]
Diffusion
5.48
MMDIT [96]
Diffusion
5.30
MMDiT+REPA [96]
Diffusion
4.14
AiCo-CrossAttn
Diffusion
4.87
DiCo-DWC
Diffusion
4.93
Figure 7: CCA effectively reduces channel redundancy and enhances feature diversity. Left: Features from the first stage of DiCo without CCA. Right: Features from the first stage of DiCo.

---

Table 4: Ablation study on architectural design. We compare different architectural variants, including isotropic, isotropic with skip connections, and U-shape. For all these architectural designs, our DiCo consistently outperforms DIT.
Model
Skip
Hierarchy
#Params
Gfoops
Throughput
FID
1St
Pre.
Rec.1
DłT (iso.) [62]
x
32.9M
6.06
1086.81
67.16
20.41
0.37
0.57
DiCo (iso.)
X
33.7M
5.67
1901.39
60.58
25.44
0.44
0.55
DIT (iso, &skip)
/
x
34.3M
6,44
1037.55
62.63
22.08
0.39
0,56
DiCo (iso.&skip))
/
x
35. 1M
6.04
1807.22
56.95
26.84
0.46
0.56
DiT (U-shape)
/
/
33.0M
4.23
1140.93
54.00
28.52
0.42
0.59
DiCo (U-shape)
33. IM
4,25
1695.73
49.97
31.38
0.48
0,58
Table 5: Ablation study on the components of DiCo. $ indicates that the model depth and width are adjusted for fair comparison. We analyze the effects of the activation function, DWC kernel size, compact channel attention (CCA), and the conv module (CM). For the CM, we compare it against several advanced efficient attention mechanisms validate its effectiveness and efficiency.
Model
#Params
Gflops
Throughput
FID
IS1
Pre.f
Rec.
DiCo-S
33. IM
4.25
1695.73
49.97
31.38
0.48
0,58
GELU-ReLU
33.1M
4.25
1695.68
51.26
30.23
0.47
0.57
3x DWC
33.2M
4.29
1628.59
48.03
32.51
0.49
0.58
3x3+7x7 DWC
33.4M
4.34
1469.45
47.49
32.93
0.49
0.59
Compact Channel Attn (CCA)
33.1M
4.25
1695.73
49.97
31.38
0.48
0.58
wfo CCA
33.0M
4.24
1731.13
54.78
28.40
0.48
0.57
CCA-SE module $ [40]
32.9M
4.25
1657.10
50.89
30.49
0.48
0.57
CCA-ehannel Self-Attn $ [97]
33.2M
4.26
1569.51
50.24
30.85
0.45
0.59
Cony Module (CM)
33.1M
4.25
1695.73
49.97
31.38
0.48
0.58
CM CM--eeffAtttn- $ [83]
33.0M
4.23
1140.93
54.00
28.52
0.42
0.59
CM Wiiddw Attn $ [56]
33.0M
4.34
1165.22
53.23
28.33
0.43
0.59
CM Focused Linear Attn [29]
32.9M
4.33
971.49
50.60
30.85
0.46
0.60
CM Agent Atta $ [65]
33.3M
4.24
1160.89
52.07
28.82
0.43
0.60
mance gains but at the expense of reduced efficiency, highlighting a trade-off between performance and computational cost.
The introduction of CCA results in a 4.81 4.81-point improvement in FID. As illustrated Fig. 7, CCA significantly enhances feature diversity, demonstrating its effectiveness in improving the model's representational capacity. We also compare CCA with SE module [40] and Channel-wise Self- Attention [97]; despite its simplicity, CCA achieves superior performance and higher efficiency.
For the Conv Module, we benchmark against several advanced efficient attention mechanisms (Window Attention [56], Focused Linear Attention [29], Agent Attention [65]). The results show that our CM offers both better performance and computational efficiency.
Conclusion
We propose a new backbone for diffusion models, Diffusion ConvNet (DiCo), a compelling alternative to the Diffusion Transformer (DiT). DiCo replaces self-attention with a combination of 1 x pointwise convolutions and 3 x depthwise convolutions, and incorporates compact channel attention mechanism to reduce channel redundancy and enhance feature diversity. As fully convolutional network, DiCo surpasses recent diffusion models on the ImageNet 256x256 and 512x512 benchmarks, while achieving significant efficiency gains. Furthermore, the purely convolutional DiCo demonstrates strong potential in text-to-image generation. We look forward to further scaling up DiCo and extending to broader generative tasks.
10

---

Acknowledgements
This research is partially funded by National Natural Science Foundation of China (Grant No. 62576342), Beijing Natural Science Foundation (4252054), Youth Innovation Promotion Association CAS(Grant No.2022132), Beijing Nova Program)(2023044227))
References
[1] Yuang Ai, Huaibo Huang, and Ran He. Lora-ir: laming low-rank experts for efficient all-in-one image restoration, arXiv preprint arXiv:2410.15530
2024.
[2] Yuang Ai, Huaibo Huang, Tao Wu, Qihang Fan, and Ran He. Breaking complexity barriers: High- resolution image restoration with rank enhanced linear attention. arXiv preprint (rXX::2505.16157, 2025.
2
[3] Yuang Ai, Huaibo Huang, Xiaoqiang Zhou, Jiexiang Wang, and Ran He. Multimodal prompt perceiver:
Empower adaptiveness generalizability and fidelity for all-in-one image restoration. In CVPR, 2024. 2
[4] Yuang Ai, Xiaogiang Zhou, Huaibo Huang, Xiaotian Han, Zhengyu Chen, Quanzeng You, and Hongxia Yang. Dreamclear: High-capacity real-world image restoration with privacy-safe dataset curation. In NeurIPS, 2024. 2
[5] Yuang Ai, Xiaoqiang Zhou, Huaibo Huang, Lei Zhang, and Ran He. Uncertainty aware source-free
adaptive image super-resolution with wavelet augmentation transformer.
CVPR, 2024,
[6] Fan Bao, Shen Nic, Kaiwen Xue, Yue Cao, Chongxuan Li, Hang Su, and Jun Zhu. All are worth words:
A vit backbone for diffusion models. In CVPR, 2023. 4, S, 7. 8, 9. 22, 25. 26
[7] Andreas Blattmann, Tim Dockhom, Sumith Kulal, Daniel Mendelevitch, Maciej Kilian, Dominik Lorenz,
Yam Levi, Zion English, Vikram Voleti, Adam Letts, et al. Stable video diffusion: Scaling latent video
diffusion models to large datasets. arXiv preprint ar ar3iv:2311. 15127, 2023. 2
[8] Andrew Brock, Jeff Donahue, and Karen Simonyan. Large scale gan training for high fidelity natural
image synthesis, arXiv preprint arXiv:1809 11096, 2018 25.26
[9] Tim Brooks, Bill Peebles, Connor Holmes, Will DePue, Yufci Guo, Li Jing, David Schnurr, Joe Taylor, Troy Luhman, Eric Luhman, Clarence Ng. Ricky Wang, and Aditya Ramesh. Video generation models as world simulators, 2024. 2
[10] Mingdeng Cao, Xintao Wang. Zhongang Qi. Ying Shan, Xiaohu Qie, and Yingiang Zheng. Masactrl: Tuming-free free mutual self- aelf-attention control for consistent image synthesis and editing. In ICCV, 2023. 2 [11] Huiwen Chang, Han Zhang, I.u Jiang, Ce Liu, and William Freeman. Maskgit: Masked generative image transformer.
CVPR, 2022. 25,
[12] Junsong Chen, Jincheng Yu, Chongjian Ge, Lewei Yao, Enze Xie, Zhongdao Wang, James T Kwok, Ping Luo, Huchuan Lu, and Zhenguo Li. Pixart-o: Fast training of diffusion transformer for photorealistic
ext-to-image synthesis. In ICLR, 2024. 2, 4, 6
[13] Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei.
Imagenet: A large-scale
hierarchical image database. In CVPR, 2009. 7
[14] Prafulla Dhariwal and Alexander Nichol. Diffusion models beat gans on image synthesis. In NeurlPS,
2021. 2, 4, 7, 8, 9, 25, 26
[15] Xiaohan Ding, Xiangyu Zhang, Jungong Han, and Guiguang Ding. Scaling up your kernels to 31x31:
Revisiting large kernel design in cnns. In CVPR, 2022. 2.5
[16] Xiaohan Ding, Xiangyu Zhang, Ningning Ma, Jungong Han, Guiguang Ding, and Jian Sun. Repvgg:
Making vgg-style convnets great again. In CVPR, 2021. 4
[17] Xiaohan Ding. Yiyuan Zhang. Yixiao Ge, Sijie Zhao, Lin Song. Xiangyu Yue, and Ying Shan. Unireplknet:
A universal perception large-kernel convnet for audio video point cloud time-series and image recognition. CVPR, 2024.
[18] Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, et al. An image is Transformers for image recognition at scale. arXiv preprint arXiv:2010. 1999, 2020.
worth 16x16 words:
2. 4
[19] Junxian Duan, Yuang Ai, Jipeng Liu, Shenyuan Huang, Huaibo Huang, Jie Cao, and Ran He. Test-time forgery detection with spatial-frequence prompt learning. IJCV, 2025.
[20] Patrick Esser, Sumith Kulal, Andreas Blattmann, Rahim Entezari, Jonas Müller, Harry Saini, Yam Levi, Dominik Lorenz, Axel Sauer, Frederic Boesel, al. Scaling rectified flow transformers for high-resolution image synthesis. In /CML, 2024, 2, 4
[21] Patrick Esser, Robin Rombach, and Bjorn Ommer. Taming transformers for high-resolution image synthesis. In CVPR, 2021. 25, 26
[22] Oihang Fan, Huaibo Huang. Yuang Ai, and Ran He. Rectifying magnitude neglect in linear attention. In
ICCV, 2025. 2
[23] Qihang Fan, Huaibo Huang, and Ran He. Breaking the low-rank dilemma of linear attention. In CVPR,
2025. 2
[24] Wan- Man-Cyuan Fan, Yen-Chur Chen, DongDong Chen, Yu Cheng, Lu Yuan, and Yu- Chiang Frank Wang. Frido: Feature pyramid diffusion for complex scene image synthesis. In AAAI, 2023. 9
11

---

[25] Zhengcong Fei, Mingyuan Fan, Changqian Yu, and Junshi Huang. Scalable diffusion models with state
space backbone. arXiv preprint arXiv:2402.0555 00000050 2024. 2. 4. 8. 9, 25, 26
[26]
Albert Gu and Tri Dao. Mamba: Lincar-time sequence modeling with selective state spaces. arXiv
preprint arXiv:2312.000 00752, 2023. 2, 3. 4
[27] Shuyang Gu, Dong Chen, Jianmin Bao, Fang Wen, Bo Zhang. Dongdong Chen, Lu Yuan, and Baining
Guo. Vector quantized diffusion model for text-to-image synthesis. CVPR, 2022.
[28] Meng-Har Han Guo, Cheng-Ze Ze Lu, Qibin Hou, Zhengning Liu, Ming-Ming Cheng. and Shi-Min Hu. Segnext:
Rethinking convolutional attention design for semantic segmentation. In NeurlPS, 2022. 2
[29] Dongehen Han, Xuran Pan, Yizeng Han, Shiji Song. and Gao Huang. Flatten transformer: Vision
transformer using focused linear attention. In ICCV, 2023, 10
[30] Dongchen Han, Ziyi Wang, Zhuofan Xia, Yizeng Han, Yifan Pu, Chunjiang Ge, Jun Song. Shiji Song, Bo Zheng, and Gao Huang. Demystify mamha in vision: A linear attention perspective. In NewrlPS, 2024. 2 Kaiming He, Xiangyu Zhang. Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition.
[31]
In CVPR, 2016. 4
[32]
Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter. Gans
trained by two time-scale update rule converge to a local nash equilibrium. In NeurlPS, 2017.
[33] Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. In NewrIPS, 2020.
5
[34] Jonathan Ho, Chitwan Saharia, William Chan, David J Fleet, Mohammad Norouzi, and Tim Salimans.
Cascaded diffusion models for high fidelity image generation. JMLR, 2022. 4
[35]
Jonathan Ho and Tim Salimans. Classifier- free diffusion guidance. arXiv preprint arXiv:2207.12596
2022.
[36] Jonathan Ho, Tim Salimans, Alexey Gritsenko, William Chan, Mohammad Norouzi, and David J Fleet.
Video diffuusion models. In NeurlPS, 2022. 2
[37]
Emiel Hoogeboom, Jonathan Heek, and Tim Salimans. simple diffusion: End-to-end diffusion for high
resolution images. In /CML, 2023. 25, 26
[38] Andrew Howard, Mark Sandler, Grace Chu, Liang-Chieh Chen, Bo Chen, Mingxing Tan, Weijun Wang. Yukun Zhu. Ruoming Pang. Vijay Vasudevan, et al. Searching for mobilenetv3. In ICCV, 2019. 4
[39] Andrew G Howard, Menglong Zhu, Bo Chen, Dmitry Kalenichenko, Weijun Wang. Tobias Weyand, Marco Andrectto, and Hartwig Adam. Mobilencts: Efficient convolutional neural networks for mobile vision applications. arXiv preprint arXiv: 1704.0461,, 2017. 4
[40]
Jie Hu, Li Shen, and Gang Sun. gqueeze-and-excitatle
networks. In CVPR, 2018. 10
[41]
Gao Huang, Zhuang Liu, Laurens Van Der Maaten, and Kilian Q Weinberger. Densely connected convolutional networks. In CVPR, 2017. 4
[42] Huaibo Huang. Xiaoqiang Zhou, Jie Cao, Ran He, and Tieniu Tan. Vision transformer with super token sampling. In CVPR, 2023. 2
[43] Minguk Kang, Jun-Yan Zhu, Richard Zhang, Jacsik Park, Eli Shechtman, Sylvain Paris, and Tacsung
Park. Scaling up gans for texl-lo-image synthesis. In CVPR, 2023. 25
[44] Angelos Katharopoulos, Apporv Vyas, Nikolaos Pappas, and François Fleuret. Transformers mns:
Fast autoregressive transformers with linear attention. In ICML, 2020. 3.4
[45] Bahjat Kawar, Michael Flad, Stefano Ermon, and Jiaming Song. Denoising diffusion restoration models.
NeurIPS, 2022. 2
[46] Bahjat Kawar, Shiran Zada, Oran Lang, Omer Tov, Huiwen Chang, Tali Dekel, Inbar Mosseri, and Michal
Irani. Imagic: Text-based real image editing with diffusion models. In CVPR, 2023.
[47] Diederik Kingma and Ruiqi Gao. Understanding diffusion objectives as the elbo with simple data
augmentation. In NeurIPS, 2023. 25, 26
[48] Tuomas Kynkäänniemi, Tero Karras, Samuli Laine, Jaakko Lehtinen, and Timo Aila. Improved precision
and recall metric for assessing generative models. In NeurlPS, 2019. 7
[49] Black Forest Labs. Flux: Official inference repository for flux.1 models, 2024. Accessed: 2024-11-12. 2,
4
[50] Doyup Lee, Chiheon Kim. Sachoon Kim. Minsu Cho, and Wook-Shin Han. Autoregressive image
generation using residual quantization. In CVPR, 2022. 25
[51] Tianhong Li. Dina Katabi, and Kaiming He. Return of unconditional generation: A self self-supervised
representation generation method. In NeurlPS, 2024. 25
[52] Tianhong Li, Yonglong Tian, He Li, Mingyang Deng, and Kaiming He. Autoregressive image generation
without vector quantization. In NeurlPS, 2024. 25
[53] Tsung- Yi Lin. Michael Maire. Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, Common objects in context. In ECCV, 2014. [54] Qihao Liu, Zhanpeng Zeng, Ju He, Qihang Yu, Xiaohui Shen, and Liang-Chich Chen. Alleviating distor- tion in image generation via multi-rcsolution diffusion models and time-dependent layer normalization. In NeurlPS, 2024. 4
and C Lawrence Zitnick. Microsoft coco:
[55] Songhua Liu, Weihao Yu, Zhenxiong Tan, and Xinchao Wang. Linfusion: gpu, 1 minute, 16k image.
arXiv preprint orXiv:2409.02099 2024. 2
[56] 7e Liu, Yutong Lin, Yue Cao, Han Hu, Yixuan Wei, Zheng Zhang, Stephen I Lin, and Baining Guo. Swin transformer: Hierarchical vision transformer using shifted windows. In ICCV, 2021. 10
[57] Zhuang Liu, Hanzi Mao, Chao-Yus Wu, Christoph Feichtenhofer, Trevor Darrell, and Saining Xie. A convnet for the 2020s. In CVPR, 2022. 5
12

---

[58] Nanye Ma, Mark Goldstein, Michael S Albergo, Nicholas M Boffi, Eric Vanden-Eijndert and Saining Xie. Sit: Exploring flow and diffusion-based generative models with scalable interpolant transformers. In ECCV, 2024. 4, 25, 26, 27
[59] Chenlin Meng. Yutong He, Yang Song, Jiaming Song. Jiajun Wu, Yan Zhu, and Stefano Ermon.
Sdedit: Guided image synthesis and editing with stochastic differential equations. arXiv preprint
arXiv:2/08.01073, 2021. 2
[60]
Alexander Quinn Nichol and Prafulla Dhariwal. Improved denoising diffusion probabilistic models. In
ICML,, 2021. 2021. 23
[61]
Ziqi Pang. Tianyuan Zhang, Fujun Luan, Yunze Man, Hao Tan, Kai Zhang, William T Freeman, and Yu-Xiong Wang. Randar: Decoder-only autoregressive visual generation in random orders. In CVPR,
2025. 25
[62] William Peebles and Saining Xie. Scalable diffusion models with transformers. In ICCV, 2023. 2, 3, 5,
7,8, 8, 9. 10, 25, 26
[63] Hao Phung. Quan Dao, Trung Tuan Dao, Hoang Phan, Dimitris N. Metaxas, and Anh Tuan Tran. DiMSUM: Diffusion mamba- a scalable and unified spatial-frequency method for image generation. In
NeurlPS, 2024. 2, 4, 8, 25
[64] Ben Poole, Ajay Jain, Jonathan Barron, and Ben Mildenhall. Dreamfusion: Text-to-3d using 2d diffusion, arXiv preprint arXiv:2209.44900
2022. 2
[65] Yifan Pu, Zhuofan Xia. Jiayi Guo, Dongchen Han, Qixiu Li, Duo Li, Yuhui Yuan, Ji Li, Yizeng Han, Shiji
Song, et al. Efficient diffusion transformer with wiep dynamic attention mediators. In ECCV, 2024.
4, 10
[66]
Aditya Ramesh, Prafulla Dhariwal, Alex Nichol, Casey Chu, and Mark Chen. Hierarchical text-conditional image generation with clip latents. arXiv preprint arXiv:2204.061
2022.
[67]
Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bjöm Ommer. High-resolution
image synthesis with latent diffusion models. In CVPR, 2022. 2, 7,8
[68]
Olaf Ronneberger, Philipp Fischer. and Thomas Brox. U-net: Convolutional networks for biomedical
image segmentation. MICCAL, 2015. 2.4
[69]
Chitwan Saharia, William Chan, Saurabh Saxena, Lala Li, Jay Whang, Emily L Denton, Kamyar Ghasemipour, Raphael Gontijo Lopes, Burcu Karagol Ayan, Tim Salimans, et al. Photorealistic text-to- image diffussin models with deep language understanding. In NeurlPS, 2022. 2
[70] Tim Salimans, Ian Goodfellow, Wojeiech Zaremba, Vicki Cheung, Alec Radford, and Xi Chen. Improved
techniques for training gans. In NewrIPS, 2016. 7
[71] Mark Sandler, Andrew Howard, Menglong Zhu, Andrey Zhmoginov, and Liang-Chich Chen. Mobilenety2:
Inverted residuals and linear bottlenecks. In CVPR, 2018. 4
[72] Axel Sauer, Katja Schwarz, and Andreas Geiger. Stylegan-xl: Scaling stylegan to large diverse datasets.
In SIGGRAPH, 2022, 25, 26
[73] Jascha Sohl-Dickstein Fiic Weiss, Niru Maheswaranathan, and Surya Ganguli. Deep unsupervised
learning using nonequilihrium thermodynamies. In ICMI., 2015. 2
[74] Jiaming Song. Chenlin Meng. and Stefano Ermon. Denoising diffusion implicit models. In ICLR, 2021. 2 Yang Song and Stefano Ermon. Generative modeling by estimating gradients of the data distribution. In
[75]
NeurIPS, 2019. 2
[76] Yang Song, Jascha Sohl-Dickster Dickstein, Diederik P Kingma, Abhishek Kumar, Stefano Ermon, and Ben Poole.
Score-based generative modeling through stochastic differential equations. In ICLR, 2021. 2,
[77] Peize Sun, Yi Jiang, Shoufa Chen, Shilong Zhang, Ringyue Peng, Ping Luo, and Zehuan Yuan. Autore- gressive model beats diffusion: Llama for scalable image generation. arXiv preprint arXiv:2406.06522
2024. 25
[78] Ming Tao, Hao Tang, Fci Wu, Xiao Yuan Jing, Bing-Kun Kun Bao, and Changsheng Xu. Df-gan: A simple and effective baseline for text ext--to-ima -mmatoo-mmae synthesis. In CVPR, 2022. 9
[79] Yao Teng. Yue Wu, Han Shi. Xuefei Ning. Guohao Dai, Yu Wang, Zhenguo Li, and Xihui Liu. Dim: Diffusion mamba for efficient high regouuroolttion image synthesis. arXiv preprint arXiv:2405. 14224, 2024.
2. 3. 4, 8. 9. 25, 26
[80] Keyu Tian, Yi Jiang, Zehuan Yuan, Bingyuc Peng, and Liwei Wang. Visual autoregressive modeling:
Scalable image generation via next-scale prediction. In NeurlPS, 2024. 25, 26
[81] Yuchuan Tian, Jing Han, Chengeheng Wang. Yuchen Liang, Chao Xu, and Hanting Chen. Dic: Rethinking
conv3x3 designs in diffusion models. In CVPR. 2025. 3. 7, 8, 25
[82] Yuchuan Tian, Zhijun Tu, Hanting Chen, Jie Hu, Chao Xu, and Yunhe Wang. U-dits: Downsample tokens
u-shaped diffusion transformers. In NewrlPS, 2024. 5
[83] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need. In NewrlPS, 2017. 2, 3, 10
[84] Hongbo Wang. Cao, Jin Liu, Xiaoqiang Zhou, Huaibo Huang, and Ran He. Hallo3d: Multi-modal hallucination detection and mitigation for consistent 3d content generation. In NewrIPS, 2024. 2
[85] Shuai Wang. Zexian Li. Tianhui Song. Xubin Li. Tiezheng Ge, Bo Zheng, and Limin Wang. Flowden: Exploring den-like architectures for fast image generation with arbitrary resolution. In NewrlPS, 2024.
25, 27
[86] Yuging Wang. Shuhuai Ren, Zhijie Lin, Yujin Han, Haoyuan Guo, Zhenheng Yang, Difan Zou, Jiashi Feng, and Xihui Liu. Parallelized autoregressive visual generation. In CVPR, 2025. 25
13

---

Yufan Zhou, Ruiyi Zhang. Changyou Chen, Chunyuan Li, Chris Tensmeyer, Tong Yu, Jiuxiang Gu, Jinhui Xu, and Tong Sun. Towards language-free training for texl-to-image generation. CVPR, 2022. [87] Zhengyi Wang, Cheng Lu, Yikai Wang, Fan Bao, Chongxuan Li, Hang Su, and Jun Zhu. Prolificdreamer: High-fidelitg and diverse text-to-3d generation with variational score distillation. In NeurlPS, 2023. 2 [88] Jay Zhangjie Wu, Yixiao Ge, Xintao Wang, Stan Weixian Lei, Yuchao Gu, Yufci Shi, Wynne Hsu, Ying Shan, Xiaohu Qie, and Mike Zheng Shou. Tune-a-video: One-shot tuning of image diffusion models for text-to-video generation. In ICCV, 2023.
[89]
Saining Xie. Ross Girshick, Piotr Dollár, Zhuowen Tu, and Kaiming He. Aggregated residual transforma-
tions for deep neural networks. In CVPR, 2017. 4
[90] Tao Xu, Pengchuan Zhang, Qiuyuan Huang, Han Zhang, Zhe Gan, Xiaolci Huang, and Xiaodong He.
Altngan: Fine-grained text to image generation with attentional generative adversarial networks. In CVPR,
2018. 9
[91]
Jing Nathan Yan, Jiatao Gu, and Alexander M Rush. Diffusion models without attention. In CVPR, 2024.
2. 4, 8. 9, 25,
[92] Songlin Yang. Bailin Wang, Yikang Shen. Rameswar Panda, and Yoon Kim. Gated linear attention
transformers with hardware-efficient training. In ICML, 2024. 2. 3,4
[93] Jingfeng Yao, Bin Yang. and Xinggang Wang. Reconstruction
generation: Taming optimization
dilemma latent diffusion models, In CVPR, 2025. 25, 27
[94] Jiahui Yu, Xin Li, Jing Yu Koh, Han Zhang, Ruoming Pang, James Qin, Alexander Ku, Yuanzhong
Xu. Jason Baldridge. and Yonghui Wu. Vector-quantized image modeling with improved vggan. arXiv preprint 1:Xiv:2110.04633
2021. 25
[95] Qihang Yu, Mark Weber, Xueqing Deng, Xiaohui Shen, Daniel Cremers, and Liang-Chie Chen, An image is worth 32 tokens for reconstruction and generation. In NeurlPS, 2024, 25
[96] Sihyun Yu, Sangkyung Kwak, Huiwon Jang, Jongheon Jeong, Jonathan Huang. Jinwoo Shin, and Saining
Xie. Representation alignment for generation: Training diffusion transformers is casier than you think. In
ICLR, 2025. 4, 8, 9, 27
[97] Syed Waqus Zamir, Aditya Arora, Salman Khan, Munawar Hayat, Fahad Shahbaz Khan, and Ming-Hsuan Yang. Restormer: Efficient transformer for high-resolution image restoration. In CVPR, 2022. 10
[98] Han Zhang, Jing Yu Koh, Jason Baldridge, Honglak Lee, and Yinfei Yang. Cross-modal contrastive learning for text-to-image generation. In CVPR, 2021. 9
[99]
[100] I. Lianghui Zhu, Zilong Huang, Hencheng Liao, Jun Hao Liew, Hanshu Yan, Jiashi Feng, and Xinggang Wang. Dig: Scalable and efficient diffusion models with gated linear attention. In CVPR, 2025. 2, 3, 4, 5,
7, 25
[101] Minfeng Zhu, Pingbo Pan, Wei Chen, and Yi Yang. Dm-gan: Dynamic memory generative adversarial
networks for text-to-image synthesis. In CVPR, 2019. 9
[102] Zhuangwei Zhuang, Mingkui Tan, Bohan Zhuang, Jing Liu, Yong Guo, Qingyao Wu, Junzhou Huang, and Jinhui Zhu, Discrimination-awen aware channel pruning for deep neural networks, In NeurlPS, 2018. 4
14

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
The abstract and introduction clearly articulate the scope and main contribu- tions of the paper. A concise summary of our contributions is also provided at the conclusion of Sec. 1.
Justification:
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
is fine to include aspirational goals as motivation as long as is clear that these goals not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification:
We discuss the limitations of the work in Sec.
Guidelines:
The answer NA means that the paper has limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (c.g.. independence assumptions, noiseless settings, model sell-speciiiatiinn asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low or images are taken in low lighting. Or speech-to-tex, system might not be used reliably to provide closed captions for online lectures because it fails handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and faimesss.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community, Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
15

---

Answer:
[NA]
Justification:
This work does not include theoretical derivations.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proof's can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided not)?
Question:
Answer: [Yes]
Justification: We provide clear description of the proposed method in Sec.3, and detail the experimental setup and implementation in Sec.4.1.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, descríbing the architecture fully might suffice, or if the contribution is specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) the contribution is primarily a new algorithm, the paper should make clear how reproduce that algorithm. (b) If the contribution is primarily new model architecture, the paper should describe the architecture clearly and fully.
(c) the contribution is new model (e-g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should possible for other researchers to have some path reproducing verifying the results. 5. Open access to data and code
16

---

Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, as described in supplemental material?
Question:
Answer: [Yes]
Justification: We the publicly available ImageNet dataset, and provide code and usage instructions in the Supplementary Material.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://nips.cr
irriiiissoooooiiii onPolicy) icy for more details.
While encourage the release of code and data, understand that this might not be possible, so "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https:
/ /nips. wctpublic//aaccccccccccccccc ggwii/gs///////eeeeee /Cae//ooeubbiiic catsSuubmision.. ssionPolicy) for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended the paper) is recommended, but including URLs to data and code is permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification:
Detailed experimental settings are provided Sec, 4.1.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to a level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, as supplemental material.
7. Experiment statistical significance
Does the report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [No]
Justification: Our model is trained on large-scale dataset, and the inference process of diffusion models is relatively slow. Due to limited computational resources, we were unable to conduct extensive statistical significance testing.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by emor bars, confi- dence intervals, or statistical significance tests, least for the experiments that support the main claims of the paper.
17

---

The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
It should be clear whether the error bar is the standard deviation or the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CL, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that out of range (e.g. negative error rates).
If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification:
We provide the information in Table 7.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary failed experiments that didn't make into the paper).
9. Code of ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurlPS Code of Ethics https: htttpa:/nneurpps cc/ cttpppiicccc///cccppppcpcpccccct c/Ethi seGuuidelines""
Answer:
[Yes]
Justification: All the authors have reviewed the code of ethics and obey the code.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No. they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., there a special consid- eration due to laws regulations in their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification:
We discuss the impacts in Sec. G.
Guidelines:
The answer NA means that there no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact why the paper does not address societal impact.
18

---

Examples of negative societal impacts include potential malicious unintended uses (e.g., disinformation, generating fake profiles, surveillance), faireess considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms monitor how system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper deseribe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification: As purely academic study, this work poses no such risks.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification:
We have cited related original papers in our paper.
Guidelines:
The answer NA means that the paper does use existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset used and, possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website). the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in package should be provided. For popular datasets, paperswi wwwode.hcoe...c. com/ datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
19

---

For existing datasets that are re-packaged, both the original license and the license of the derived asset (if has changed) should be provided.
If this information is not available online, the authors are encouraged to reach out to the asset's s creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification: Our paper does not release new assets.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/moddl as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset used.
At submission time, remember to anonymize your assets (if applicable). You can either create anonymized URL or include anonymized zip file.
14. Crowdsourcing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?
Answer: [NA]
Justification: Our paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According the NeurIPS Code of Bthies, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: Our paper does not involve crowdsoureing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurlPS Code of Ethics and the guidelines for their institution.
For initial submissions, not include any information that would break anonymity (if applicable), such the institution conducting the review.
20

---

16. Declaration of LLM usage
Question: Does the paper describe the usage of LLMs if it is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, or formatting purposes and does not impact the core methodology. scientific rigorousness, originality of the research, declaration is not required.
Answer: [NA]
Justification: Our core method development does not involve LLMs.
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, non-standard components.
Please refer to L.LM policy (https: h ht////urrpsscccc Satrrrips.ccc/ommeeeennee/////// /LLM) for what should or should not be described.
21

---

Appendix
We provide the following supplementary information in the Appendix:
Sec. A. detailed configurations of DiCo model variants.
Sec. B. additional implementation details of DiCo.
Sec. C. scalability analysis of DiCo models.
Sec. D. additional comparison results with generative model family.
Sec, additional samples generated by DiCo-XL, models.
Sec. F. discussion of limitations of this work.
Sec. G. discussion of broader impacts of this work.
A DiCo Model Variants
We introduce several variants of DiCo, each scaled to different model sizes. Specifically, we present five variants: Small (33M parameters), Base (130M), Large (460M), XLarge (700M), and Huge (IB). These variants are created by adjusting the hidden size D, the number layers (I,, L2, Lg, L1, Lg),, and the FFN ratio. They span wide range of parameter counts and FLOPs. from 33M to IB parameters and from 4.25 Gflops to 194.15 Gflops, offering a comprehensive foundation for evaluating scalability and efficiency. Notably, when compared their corresponding DiT counterparts, our DiCo models require only 70.1 % to 74.6% of the FLOPs, demonstrating the computational efficiency of our design. The detailed configurations for these variants are provided in Table
Table 6: Architecture variants of DiCo. Gflops are measured with an input size of 32 x 32 Compared to DiT, our DiCo models more computationally efficient.
Model
#Params (M)
Gflops
Gllupppio Gáapiper
Hidden Size
#Layers
FFN Ratio
DiCo-S
33.1
4.25
70.1 %
128
I5, 4, 4, 4, 4]
2
DiCo-B
130,0
16.88
73.3%
256
[5.
2
DiCo-L
463.9
60.24
74.6%
352
19, 8, 9, 8, 91
2
DiCo-XL
701.2
87.30
73.6%
416
9,,9, 10, 9,
2
DiCo-H
1037.4
194.15
416
[14.
12,
10,
12,
14]
4
Table 7: Implementation details of DiCo variants. For DiCo-S/B/L/XL, we follow the same experi- mental settings DiT [6]. For the largest variant DiCo-H, we adopt the training hyperparameters from U-ViT [6], increasing the batch size and learning rate to accelerate training.
Model
DiCo-S
DiCo-B 236 x 256
DiCo-L 256 x 256
DiCo-XL 256 x 256
DiCo-XL 512 512
DiCo-H 256 x 256
Resolution
256 256
Optimizer
AdamiW (0.9, 0.9.9.
AdamW
AdamW (0.9. 0.999)
AdamW (0.9, 0.999)
Adamm 0.9.0.
Adamw (0.99.0.90 0.99)
Betas
Weight decay
0
Peak learning rute
1 x 10-4
10-1
x 10-4
1 x 10-1
1 x 10-4
2 10
Learning rate schedule
constant
constant
constant
constant
constant
constant
Warmup steps
0
0
0
5K
Global batch size
256
256
256
1024
Numerical precision
fp32
fp32
fp32
fp32
fp32
fp32
Training steps
400K
400K
3750K
3000K
1000K
Computational resources
A100
A.000
16 A100
32 A100
64 A100
64 A100
Training Time
11 hours
hours
29 hours
266 hours
256 hours
113 hours
Data Augmentation
random flip
random tlip
random tlip
random tlip
random flip
random flip
sd-ftema iDDPM 250
sd-ft-ema iDDPM 250
sd-ft-ema iDDPM 250
sd-ft-ema iDDPM 250
sd-ft-ema iDDPM 250
sd-ft-ema
Sampler
iDDPM
Sampling steps
250
22

---

DiCo-S
190
DKoo5
100
BiCo-B
DKC-8
DiCO-L
DiCo-L
Inception Score
80
80
FID-50K
60
20
200K
300K
400K
100K
200K
300K
Training Steps
Training Steps
Figure 8: Scaling the DiCo models consistently improves performance throughout training. We report FID-50K and Inception Score across training iterations for four DiCo variants.
0.21
0.19
Training Loss
0.17
0.15
0.13
X30K
<.K
500K
Training Iterations
0.21
Dicao (5328532)
6.14
Training Loss
0.17
0.15
0.3
0.254
0.50M
0.7MM
LOON
123M
3.50M
1.75M
3.00
22M
25:3
2.75M
3.00M
3.25M
350M
3.70M
Training
Ite ations
Figure 9: Training loss curves for DiCo models. We also highlight early training dynamics during the first 100K iterations. Larger DiCo variants exhibit lower training losses, reflecting improved optimization with scale.
B Additional Implementation Details
For DiCo-S/B/L/XL models, we adopt the same experimental settings as those used for DiT. For DiCo-H, the largest variant, we use the hyperparameters from U-VIT, increasing the batch size and learning rate to expedite training. All experiments are conducted on NVIDIA A100 (80G) GPUs. During inference, models use the iDDPM [60] sampler with 250 sampling steps. The whole implementation details are summarized in Table 7.
C
Scalability Analysis
Impact scaling on metrics. Table Fig. illustrate the impact of DiCo model scaling across various evaluation metrics. Our results show that scaling DiCo consistently enhances performance across all metrics throughout training, highlighting its potential as a strong candidate for large-scale foundational diffusion model.
23

---

Table 8: Performance of DiCo models without CFG at different training steps on ImageNet
256 x 256.. Scaling the ConvNet backbone consistently leads to improved generative performance.
Model
Gilops
Training Steps
FID
sFID
IS
Precision
Recall
50K
109.50
18.28
12.00
0.283
0.322
100K
79.79
15.60
16.39
0.382
0.440
150K
68.93
13.98
19.93
0.411
0,498
200K
62.61
13.18
22.87
0.433
0.531
DiCo-S
4.25
250K
57.95
12.47
25.58
0.450
0.546
300K
54.58
11.99
27.74
0.467
0.565
350K
51.77
11.69
29.68
0.479
0.576
400K
49.97
11.41
31.38
0.481
0.582
50K
84.78
14.24
14.85
0.392
0.423
100K
53.66
9.13
25.01
0.494
0.544
150K
43.09
8.21
33.02
0.536
0.571
200K
37.24
8.03
39.10
0.561
0.596
DiCo-B
16.88
250K
33.31
7.72
44.89
0.577
0.598
300K
30.68
7.59
49.34
0.584
0.597
350K
28.72
7.55
52.95
0.594
0.599
400K
27.20
7.43
56.52
0.603
0.617
50K
68.11
11.50
18.62
0.480
0.465
100K
35.42
6.52
38.12
0.607
0,561
150K
25.69
6.04
53.91
0.643
0.579
200K
20.81
5.80
65.78
0.665
0.589
DiCo-L
60.24
250K
18.19
5.70
74.77
0.676
0.588
300K
16.11
5.60
81.47
0.685
0.594
350K
14.92
5.58
86.01
0.689
0,602
400K
13.66
5.50
91.37
0.694
0.604
50K
66.53
11.35
19.41
0.501
0.472
100K
31.78
6.39
42.81
0.637
0.563
150K
22.61
5.94
60.49
0.672
0.572
200K
17.95
5.67
73.21
0.693
0,582
DiCo-XL
87.30
250K
15.60
5.58
82.96
0.697
0.591
300K
13.70
5.52
89.44
0.707
0.599
350K
12.59
5.48
95.49
0.710
0.605
400K
11.67
5.38
100.42
0.711
0.606
Impact of scaling on training loss. We further analyze the effect of model scale training loss. As shown in Fig. 9, larger DiCo models consistently achieve lower training losses, indicating more effective optimization with increased scale.
D Additional Comparison Results
ImageNet 256x256. Table presents a comparison across generative model family on ImageNet 256x256. Among diffusion models, our DiCo-XL achieves strong FID of 2.05 with only 87.3 Gilops. Our largest variant, DiCo-H, attains an FID of 1.90. When compared to other generative model types, DiCo also demonstrates competitive performance. Notably, DiCo-H, with just IB parameters, outperforms VAR-d30 which has 2B parameters in terms FID.
ImageNet 512x Table 10 presents the results on ImageNet 512x512. Among diffusion models, our DiCo-XL. achieves an FID 2.53 with only 349.8 GFLOPs. Compared to other generative models, DiCo continues to demonstrate strong performance. Specifically, DiCo-XI, with only 701M parameters, outperforms VAR-d36-s, which has 2.3B parameters, achieving superior FID performance with significantly fewer parameters.

---

Table 9: Comparison with generative model family on ImageNet 256x256. We report the performance of state-of th- generative models across different paradigms, including GAN- DAs-based, masked prediction (Mask.)-based) autoregressive (AR), visual-autoregressive (VAR), and diffusion (Diff.)- models.
Type
Model
#Params
Gflops
FID 4
IS f
Precision f
Recall
GAN BigGAN-deepp [8]
160M
6.95
171.4
0.87
0.28
GAN
GigaGAN [43]
569M
3.45
225.5
0.84
0.61
GAN
StyleGAN-XL [72)
166M
2.30
265.1
0.78
0.53
Mask.
MaskGIT [11]
227M
6.18
182.1
0.80
0.51
Mask.
RCG [51]
502M
3.49
215.5
Mask.
TYTok-S- S-28 [95]
287M
1.97
281.8
AR
VQ-GAN-re [21]
1.4B
5.20
280.3
AR
ViTVQ-re [94]
1.7B
3.04
227.4
AR
RQTran.-re [50]
3.8B
3.80
323.7
AR
LLamaGen-3B [77]
3.1B
2.18
263.3
0.81
0.58
AR
MAR-H [52]
943M
1.55
303.7
0.81
0.62
AR
PAR-3B [86]
3.1B
2.29
255.5
0.82
0.58
AR
RandAR-XXI [61]
1.4B
2.15
322.0
0.79
0.62
VAR
VAR-d16 [80]
310M
3.30
274.4
0.84
0.51
VAR
VAR-d20
600M
2.57
302.6
0.83
0.56
VAR
VAR-d24
1.0B
2.09
312.9
0.82
0.59
VAR
VAR-d30
2.0B
1.92
323.1
0.82
0.59
Diff.
ADM-U [14]
608M
742
3.94
215.8
0.83
0.53
Diff.
U-ViT-L/2 [6]
287M
77
3.40
219.9
0.83
0.52
Diff.
U-VIT-H/2 [6]
501M
133.3
2.29
263.9
0.82
0.57
Diff.
Simple Diffusion [37]
2.0B
2.77
211.8
Diff.
VDM++ [47]
2.0B
2.12
267.7
Diff.
DiT-XL/2 [62]
675M
118.7
2.27
278.2
0.83
0.57
Diff.
SIT-XL [58]
675M
118.7
2.06
277.5
0.83
0.59
Diff.
DiM-H [79]
860M
210
2.21
Diff.
DiS-H/2 [25]
901M
2.10
271.3
0.82
0.58
Diff.
DiffuSSM-XL [91]
673M
280.3
2.28
259.1
0.86
0.56
Diff.
DiMSUM-L/2 [63]
460M
84.49
2.11
0.59
Diff.
DiG-XL/2 [100]
676M
89.40
2.07
279.0
0.82
0.60
Diff.
DiC-H [81]
1.0B
204.4
2.25
Diff.
DiCo-XL
701M
87.30
2.05
282.2
0.83
0.59
Diff.
DiCo-H
1.0B
194.15
1.90
284.3
0.83
0.61
ImageNet 1024x1024. We conduct experiments ImageNet 1024x 1024. Since the original DiT paper does not report results at this resolution, we train both DiT and DiCo from scratch under the same settings for a fair comparison. We small-scale models and train them for 400K iterations. Table 11 clearly shows that DiCo scales much more efficiently to high essoutions--aciivinng a 5x speedup with better generation quality.
Comparison with FlowDCN Deformable convolution a strong and widely used convolutional variant, and FlowDCN [85] represents important bascline in this space. To fairly compare with FlowDCN, we retain its original isotropic architecture and training setup, and replace its MultiScale DCN with our proposed Cony Module. Table 12 shows that our Conv Module achieves better generative performance and 2.1 x higher throughput compared to MultiScale DCN, despite being structurally simpler and easier implement.
Comparison with Modern DiTs. Given the rapid advancements in modern DiTs, it important to compare our method with recent DiTs such as LightningDiT [93]. We find that after aligning with their experimental settings, our method achieves competitive performance along with clear efficiency advantage. Table 13 shows that our model achieves FID score of 1.33 and offers 2.8
25

---

Table 10: Comparison with generative model family on ImageNet 512x512. We report the performance of state-of of art generative models across different paradigms, including GAN-base DAN-based, masked prediction (Mask. -based, autoregressive (AR), visual-autoregressive (VAR), and diffusion (Diff.)-based models.
Type
Model
#Params
Gflops
FID 4
IS 1
Precision f Recall
GAN BigGAN-deep [8]
160M
8.43
177.9
0.88
0.29
GAN
StyleGAN-XI [72]
166M
2.41
267.8
0.77
0.52
Mask.
MaskGIT [11]
227M
7.32
156.0
0.78
0.50
AR
VQ-GAN [21]
227M
26.52
66.8
0.73
0.31
VAR
VAR-d36-s [80]
2.3B
2.63
303.2
Diff.
ADM-U [14]
731M
2813
3.85
221.7
0.84
0.53
Diff.
U-ViT-L/4 [6]
287M
76.5
4.67
213.3
0.87
0,45
Diff.
U-ViT-H/4 [6]
501M
133.3
4.05
263.8
0.84
0.48
Diff.
Simple Diffusion [37]
2.0B
4.53
205.3
Diff.
VDM++ [47]
2.0B
2.65
278.1
Diff.
DiT-XL/2 [62]
675M
524.7
3.04
240.8
0.84
0.54
Diff.
SIT-XL [58]
675M
524.7
2.62
252.2
0.84
0.57
Diff.
DiM-H [79]
860M
708
3.78
Diff.
DiS-H/2 [25]
901M
2.88
272.3
0.84
0.56
Diff.
DiffuSSM-XL [91]
673M
1066.2
3.41
255.1
0.85
0.49
Diff.
DiCo-XL
701M
349.8
2.53
275.7
0.83
0.56
Table Comparison with DiT on ImageNet 1024x1024 iterations without CFG.
The performance reported at 400K
Model
#Params
FLOPs
Throughput f
FID |
IS 1
Precision f
Recall f
DiT-S/2
33M
241.8G
28 it/s
113.06
12.60
0.25
0.37
DiCo-S
33M
67.8G
143 it/s
102.60
16.45
0.33
0.38
throughput gain over modern DiTs. This demonstrates that our convolutional backbone can match or exceed the generative quality of modern DiTs while being significantly more efficient.
E
Model Samples
We present samples generated by the DiCo-XL models at resolutions of 256x256 and 512x512. Fig. 10 through 29 display uncurated samples across various classifier-free guidance scales and input class labels. As illustrated, our DiCo models demonstrate the ability to generate high-quality, detail-rich images.
Limitations
While this work demonstrates the strong performance and scalability of DiCo through extensive experiments, there are some limitations to note. Due to limited computational resources, our model is scaled to IB parameters, while some advanced generative models have been scaled even larger sizes. We aim to investigate the broader generative potential of DiCo in future research.
G Broader Impacts
Our work the generative model DiCo contributes to advances in controllable image generation. This has potential positive applications in data augmentation, scientifie visualization, and accessibility technologies. However, such models may also be misused to generate misleading harmful content,
26

---

Table 12:
Comparison with FlowDCN on ImageNet 256x256. The performance is reported at
400K iterations without CFG.
Model
#Params
FLOPs
Throughput
FID
IS t
SiT-S/2 [58]
32.9M
6.1G
1234 it/s
57.6
24.8
FlowDCN-S/2 [85]
30.3M
4.4G
1194 it/s
54.6
26.4
Ours-S/2
31.3M
4.6G
2489 it/s
52.1
29.2
Table 13: Comparison with modern DiTs ImageNet 256x256. We follow the setup in [93].
Model
Epochs
#Params
FLOPs
Throughput t
FID /
IS f
REPA [96]
800
675M
118.7G
77 it/s
1.42
305.7
LightningDiT [93]
800
675M
118.8G
73 it/s
1.35
295.3
Ours
800
679M
118.3G
201 it/s
1.33
300.2
especially contexts involving deepfakes or biased representations of specific classes. To mitigate these risks, encourage responsible usage aligned with ethical AI guidelines and emphasize the importance of transparency when deploying generative models in roal-world applications.
27

---

Figure 10: Uncurated 512x512 X.Co-XX..ample Classifier-free guidance scale = 4.0 Class label "arctic wolf" (270)
Figure 11: Uncurated 512x512 DiCo-XI. samples Classifier-free guidance scale 4.0 Class label "volcano" (980)
28

---

Figure 12: Uncurated 512x512 DiCo-XI. DiCo-XI..amples Classifier-free guidance scale = 4,0 Class lahel "husky" (250)
Figure 13: Uncurated 512 XiCoo-Xessmples Classifier-free guidance scale 4.0 Class label "ulphur-crested cockatoo" (89)
29

---

Figure 14: Uncurated 512x512 DiColel. Classifier-free guidance scale = 4,0 Class lahel "cliff drop-off" (972)
Figure 15: Uncurated 512x512 DiCo-XI. samples. Classifier-free guidance scale 4.0 Class label "balloon" (417)
30

---

Figure 16: Uncurated 512x512 oo Classifier-free guidance scale = 4,0 Class label "lion" (291)
Figure 17: Uncurated 512x512 xxXppllesammpes Classifier-free guidance scale 4.0 Class label "otter" (360)
31

---

Figure 18: Uncurated 512x512 DICo-XI. Classifier-free guidance scale = 2.0 Class lahel "red panda" (387)
Figure 19: Uncurated 512 DiCo-XI. samples
Classifier-free guidance scale 2.0 Class label "panda" (388)
32

---

Figure 20: Uncurated 512 DiCo-XI. Classifier-free guidance scale = 1.5 Class lahel "coral reef" (973)
Figure 21: Uncurated 512x512 samples. Classifier-free guidance scale 1.5 Class label "macaw" (88)
33

---

Figure 22: Uncurated 256 256 DiCo-XI. Classifier-free guidance scale = 4.0 Class lahel "macaw" (88)
Figure 23: Uncurated x DiCo-XI. DiCo-XI. samples. Classifier-free guidance scale 4.0 Class label "dog sled" (537)
34

---

Figure 24: Uncurated 256 256 DiCo-XI. samples. Classifier-free guidance scale = 4.0 Class lahel "arctic fox" (279)
Figure 25: Uncurated 256 x 256 DiCo-XI. saColes Classifier-free guidance scale 4.0 Class label "loggerhead turtle" (33)
35

---

Figure 26: Uncurated 256 256 DiCo-XI. samples. Classifier-free guidance scale = 2.0 Class lahel "golden retriever" (207)
Figure 27: Uncurated 256x251 DiCo-le..amppes Classifier-free guidance scale 2.0 Class label "lake shore" (975)
36

---

Figure 28: Uncurated 256 256 DiCo-XI. samples. samples. Classifier-free guidance scale = 1.5 Class lahel "space shuttle" (812)
Figure 29: Uncurated DiCo-XI samples. Classifier-free guidance scale 1.5 Class label "ice cream" (928)
37


Reviews:
[{"review_id": "eFysnk1KDj", "reviewer": "Reviewer_3NNh", "summary": "This paper replace self-attention layer in DiT with a novel CNN block which is enhanced by proposed compact channel attention (CAA) mechanism. By this means, the proposed model DiCo outperforms DiT on higher performance and efficiency at class-conditional image generation task.", "questions": "1. Insufficient innovation in the model architecture. The proposed DiCo seems to be the fusion of Stable Diffusion U-shape architecture and CNN-version DiT block. The highlighed CCA mechanism is quite similar to SE block. The biggest difference is removing the channel compression FC layer? Although combining different methods to achieve new functions can be considered as an innovation, it is undoubtedly that the degree of this mode of innovation is not sufficient enough for NeurIPS. \n2. Lacking comparison with U-net based diffusion models.  Since DiCo owns the U-shape architecture and the DiCo blocks are actually based on convolution module, why not conduct comparison with early U-net based diffusion, such as classic Stable Diffusion v1.5 and xl? Since authors have compared DiCo with DiT, why not compare to DiT based model such as Stable Diffusion 3 and FLUX? Though these models are original for text-to-image generation, I think simply replacing text prompt encoder with class embedder of DiT can enable the fair comparison. \n3. Inappropriate abbreviation. Authors abbreviated convolutional neural networks as ConvNets, which easily lead readers to mistake it for ConvNeXt. I believe it is a commen sense in the deep learning community that the abbreviation of convolutional neural networks is \"CNN\". If the authors insist on using the abbreviation \"ConvNets\", please provide a convincing reason for it.", "limitations": "yes", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 2, "originality": 2, "strengths_and_weaknesses": "Strengths:\n1. Considerable implemention details and experimental setting;\n2. Reader-friendly good  and clear writing;\n3. Visual insights, especially finding of localized representations in Figure4.\n\nWeakness:\n1. Insufficient innovation in the model architecture. \n2. Lacking comparison with U-net based diffusion models.  \n3. Inappropriate abbreviation.", "was_revised": true, "final_justification": "4: Borderline accept", "rebuttal": "Thank you for your feedback. We appreciate your recognition of the value of visual insights and the adequacy of experiments. Here are our responses to your comments.\n\n---\n\n# Main Comments\n\n> **Q1: Insufficient innovation in the model architecture.**\n\n**A1:** Thank you for your valuable comments. From a broader perspective, our work **challenges a popular assumption** in today’s community: that Vision Transformers (e.g., DiT) are inherently superior to convolutional architectures for generative modeling. Our proposed DiCo demonstrates that a **purely convolutional design** can not only match but **outperform ViT-based diffusion models** like DiT in terms of both quality and efficiency. This offers a **fresh and timely perspective** to the research community, which in itself constitutes an important form of innovation.\n\nMoreover, we should know that **the technical innovation should be consistent with the research background and motivation.** In terms of model design, our innovations are reflected in both the **overall architecture** and **key components:**\n\n1. **Overall architecture:** DiT is a pioneering diffusion model that applies the ViT as its backbone. Building on this line of research, DiCo introduces a **fully convolutional backbone** for diffusion models, inspired by the **localized nature of attention** in DiT. This design remains largely unexplored in the diffusion-based generative modeling literature. Furthermore, we extend DiCo to the text-to-image generation setting (refer to **A2**), resulting in **a fully convolutional text-to-image model.** This is a significant departure from existing text-to-image architectures, which predominantly rely on different attention mechanisms.  \n2. **Component-level design:** We identify a specific challenge in convolutional designs—**channel redundancy**—and propose a simple yet effective solution: the CCA module. While CCA shares some similarities with SE modules, it removes the FC layers and RELU non-linearities, resulting in a lighter and more efficient design. As shown in Table 4, CCA improves both generation quality and throughput compared to SE, validating its suitability in generative settings.\n\n\nOverall, our model offers fresh insights to the research community and introduces a novel design perspective, through which it showcases meaningful technical innovation.\n\n---\n\n> **Q2: Lacking comparison with Stable Diffusion's UNet and FLUX's MM-DiT.**\n\n**A2:** Thank you for raising this important point. We agree that comparing DiCo with Stable Diffusion's UNet and FLUX's MM-DiT is essential for evaluating its effectiveness.\n\nFirst, regarding FLUX's MM-DiT: its architecture is a variant of DiT, with the primary difference lying in how textual features are handled. When replacing the text prompt encoder with a class embedding module (as in class-conditional ImageNet generation), the architecture effectively reduces to DiT. Therefore, **we consider MM-DiT a meaningful representative of DiT-style backbones in the text-to-image setting.**\n\nSecond, we acknowledge the importance of comparing against strong baselines such as Stable Diffusion's UNet and FLUX's MM-DiT. Given the limited time during the rebuttal phase, we follow prior works (U-ViT [1], REPA [2]) and **conduct a fair small-scale text-to-image evaluation using Stable Diffusion’s U-Net and FLUX's MM-DiT as baselines.** Specifically, we adopt the same experimental setup as [1,2]: training and evaluating models from scratch on the MS-COCO dataset, using CLIP as the text encoder with a token length of 77.\n\nWe investigate two different approaches for **incorporating textual features into DiCo.**  \n1. The first uses the widely adopted **cross-attention** mechanism, integrated into the DiCo architecture to fuse text and visual features.  \n2. The second **transforms CLIP text embeddings into dynamic depthwise convolution (DWC) kernels.** We pad the text embeddings to a length of 81, feed them through a learnable MLP, and reshape the output into a 9×9 kernel. This kernel dynamically modulates DiCo's features via depthwise convolution. In this way, we can construct a **fully convolutional** text-to-image DiCo model **without relying on any self-attention or cross-attention operations.** We provide its PyTorch implementation below:\n\n***Algorithm:*** *PyTorch code of text conditional depthwise convolution*\n```python\nimport torch\nimport torch.nn.functional as F\n\ndef text_conditional_dwconv(x, context):\n    # x: (B, C, H, W) input feature maps\n    # context: (B, 77, C) CLIP text embeddings after an MLP\n    # output: (B, C, H, W) output after depthwise convolution\n    B, C, H, W = x.shape\n    context_pad = torch.cat([context, context[:, -1:].expand(-1, 4, -1)], dim=1)  # (B, 81, C)\n    kernels = context_pad.reshape(B, 9, 9, C).permute(0, 3, 1, 2).reshape(B * C, 1, 9, 9)\n    x_flat = x.view(1, B * C, H, W)\n    output = F.conv2d(x_flat, kernels, padding=4, groups=B * C).view(B, C, H, W)\n    return output\n```\n\nBoth fusion modules are inserted after the Conv Module in each DiCo block. Following U-ViT, we match model size and use a CFG scale of 2.0. The FID result for Stable Diffusion’s U-Net is taken from U-ViT [1], and that for FLUX's MM-DiT is from REPA [2].\n\n***Table:*** *Comparison on MS-COCO for text-to-image generation*\n| Method|Token Mixing Type | Params | FLOPs | Throughput↑ | FID w/ CFG↓|\n|-|:-:|:-:|:-:|:-:|:-:|\n| U-ViT-S/2 (Deep) |  Full Self-Attn |58M   |   21.3G       | 377 it/s | 5.48 |\n| Stable Diffusion's UNet |  Conv + Self-Attn + CrossAttn |53M   |   20.9G       | 441 it/s | 7.32 |\n| FLUX's MM-DiT | Full Self-Attn| 512M  |62.4G |135 it/s | 5.30 |\n| DiCo-CrossAttn (ours)|Conv + CrossAttn | 67M   |  14.9G    | 594 it/s | **4.87** | \n| DiCo-DWC (ours) |Full Conv| 69M   |  **12.9G**    | **659 it/s** | 4.93 | \n\nDiCo **outperforms both Stable Diffusion's UNet and FLUX's MM-DiT** in terms of efficiency and generation quality. Notably, using text conditional DWC in place of cross-attention further improves throughput while maintaining competitive performance. This suggests that the **fully convolutional DiCo** has the potential to serve as the backbone for large-scale **text-to-image diffusion models**. We will include these results in the final version of our paper.\n\n---\n\n> **Q3: Inappropriate abbreviation.**\n\n**A3:** Thank you for pointing this out. We understand the concern that the abbreviation \"ConvNet\" could be confused with \"ConvNeXt,\" and we agree that \"CNN\" is the more widely recognized and unambiguous term in the deep learning community. While “ConvNet” has been used in several prior works (e.g., ConvNeXt [3], RepVGG [4]), we acknowledge that using \"CNN\" would provide greater clarity for readers. We will revise the manuscript to replace \"ConvNet\" with \"CNN\" throughout, in order to avoid confusion and maintain consistency with common terminology.\n\n---\nThank you again for helping us strengthen our work. We hope that our response has addressed your concerns. We look forward to further communication with you!\n\n[1] All Are Worth Words: A ViT Backbone for Diffusion Models. In CVPR, 2023.  \n\n[2] Representation Alignment for Generation: Training Diffusion Transformers Is Easier Than You Think. In ICLR, 2025.\n\n[3] A ConvNet for the 2020s. In CVPR, 2022.\n\n[4] RepVGG: Making VGG-style ConvNets Great Again. In CVPR, 2021."}, {"review_id": "kk9PRdv4tr", "reviewer": "Reviewer_yq4n", "summary": "In this paper,, the authors observe that self-attention in diffusion transformers seems to mostly capture local patterns, motivating going back to a convolutional architecture that could be more efficient. However, naively replacing the self attention layers with convolutional layers degrades performance, and the authors observe many channels remaining inactive during generation. Therefore, the authors introduce a compact channel attention (CCA) mechanism, which dynamically activates informative channels with lightweight linear projection. This enables this convolutional architecture achieve SOTA results on variety of benchmarks.", "questions": "My understanding is all the experiments shown here are for latent diffusion. do the observations about local attention patterns and improvements with CCA also hold true for pixel-level diffusion?\n\nare the baselines in table 1 with CFG or not?\n\na big advantage of using a convolutional arch is that the scaling of computation with resolution will not be quadratic like with self-attention. Could a comparison at a larger scale like 1024x1024 be done to demonstrate this?", "limitations": "yes", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 4, "clarity": 3, "significance": 4, "originality": 4, "strengths_and_weaknesses": "Strengths:\n* well-structured and well-motivated\n* compelling results on benchmarks\n* unique architectural modifications\n\nWeaknesses:\n* no text-to-image results, even small-scale\n* only 256x256 and 512x512 results on ImageNet", "was_revised": true, "final_justification": "addressed my suggestions", "rebuttal": "Thank you for your feedback. We appreciate your recognition of our method's innovation and effectiveness. Here are our responses to your comments.\n\n---\n\n# Main Comments\n\n> **Q1: No text-to-image generation results, even small-scale.**\n\n**A1:** Thank you for your insightful suggestions. Given the limited time during the rebuttal phase, **we follow U-ViT [1] and REPA [2] to conduct small-scale text-to-image generation experiments.** Specifically, we adopt the same experimental setup as [1,2]: training and evaluating models from scratch on the MS-COCO dataset, using CLIP as the text encoder with a token length of 77.\n\nWe investigate two different approaches for **incorporating textual features into DiCo.**  \n1. The first uses the widely adopted **cross-attention** mechanism, integrated into the DiCo architecture to fuse text and visual features.  \n2. The second **transforms CLIP text embeddings into dynamic depthwise convolution (DWC) kernels.** We pad the text embeddings to a length of 81, feed them through a learnable MLP, and reshape the output into a 9×9 kernel. This kernel dynamically modulates DiCo's features via depthwise convolution. In this way, we can construct a **fully convolutional** text-to-image DiCo model **without relying on any self-attention or cross-attention operations.** We provide its PyTorch implementation below:\n\n\n***Algorithm:*** *PyTorch code of text conditional depthwise convolution*\n```python\nimport torch\nimport torch.nn.functional as F\n\ndef text_conditional_dwconv(x, context):\n    # x: (B, C, H, W) input feature maps\n    # context: (B, 77, C) CLIP text embeddings after an MLP\n    # output: (B, C, H, W) output after depthwise convolution\n    B, C, H, W = x.shape\n    context_pad = torch.cat([context, context[:, -1:].expand(-1, 4, -1)], dim=1)  # (B, 81, C)\n    kernels = context_pad.reshape(B, 9, 9, C).permute(0, 3, 1, 2).reshape(B * C, 1, 9, 9)\n    x_flat = x.view(1, B * C, H, W)\n    output = F.conv2d(x_flat, kernels, padding=4, groups=B * C).view(B, C, H, W)\n    return output\n```\n\nBoth feature fusion modules are inserted after the Conv Module within each DiCo block. Following U-ViT, we match the model size and use a CFG scale of 2.0. All FID results  of other methods reported below are taken from REPA [2].\n\n***Table:*** *Comparison on MS-COCO for text-to-image generation*\n| Method|Token Mixing Type | Params | FLOPs | Throughput↑ | FID w/ CFG↓|\n|-|:-:|:-:|:-:|:-:|:-:|\n| U-ViT-S/2 (Deep)|  Full Self-Attn |58M   |   21.3G       | 377 it/s | 5.48 |\n| MM-DiT | Full Self-Attn| 512M  |62.4G |135 it/s | 5.30 |\n| DiCo-CrossAttn (ours)|Conv + CrossAttn | 67M   |  14.9G    | 594 it/s | **4.87** | \n| DiCo-DWC (ours) |Full Conv| 69M   |  **12.9G**    | **659 it/s** | 4.93 | \n\nOur DiCo **outperforms prior methods in both efficiency and generation quality.** Notably, using text conditional DWC in place of cross-attention further improves throughput while maintaining competitive performance. This suggests that the **fully convolutional DiCo** has the potential to serve as the backbone for large-scale **text-to-image diffusion models**. We will include these results in the final version of our paper.\n\n---\n\n> **Q2: Applicability to pixel-level diffusion.**\n\n**A2:** Thank you for your thoughtful question. Yes, the effectiveness of our method also holds in the pixel-level diffusion setting.\n\nFollowing U-ViT’s setup, we conduct experiments on the ImageNet 64×64 dataset using pixel-level diffusion. We adjust DiCo’s width and depth to match the overall model size of U-ViT-L/4 for a fair comparison. As shown below, DiCo achieves significantly better performance and efficiency:\n\n***Table:*** *Pixel-level diffusion on ImageNet 64×64*\n| Method      | Params | FLOPs | Throughput↑ | FID w/o CFG↓  | IS↑  |\n|------------|:--------:|:--------:|:------------:|:------:|:------:|\n| U-ViT-L/4  | 287M    | 76.5G   | 117 it/s | 4.26 |  40.66 |\n| DiCo  | 281M    | **75.1G**   | **201 it/s** | **3.07**  | **48.32** |\n\nThese results confirm that DiCo generalizes well to pixel-level diffusion. The benefits of convolutional locality and lightweight channel-wise modulation (via CCA) remain consistent in this setting. We will include these results in the final version of the paper.\n\n---\n\n> **Q3: Are the baselines in Table 1 with CFG or not?**\n\n**A3:** Thank you for the careful observation. In Table 1, **all models except those trained for 400K iterations use CFG during evaluation.** The **400K-iteration models are intended for early-stage comparison and are reported without CFG**, consistent with prior works (e.g., DiT, DiG).\n\nAll baseline metrics are taken directly from their respective published papers to ensure fairness. We will clarify the use of CFG in Table 1 and the main text in the final version of the paper.\n\n---\n\n> **Q4: Comparison with DiT at a larger resolution like 1024x1024.**\n\n**A4:** Thank you for the valuable suggestion. We conduct experiments on ImageNet 1024×1024. Since the original DiT paper does not report results at this resolution, we train both DiT and DiCo from scratch under the same settings for a fair comparison. Due to DiT’s significantly slower training speed at high resolutions, we use small-scale models and train them for 400K iterations. The results are shown below:\n\n***Table:*** *Comparison on ImageNet 1024x1024 (400K iterations, Small models)*\n| Method      | Params | FLOPs | Throughput↑ | FID w/o CFG↓  | IS↑  |\n|------------|:--------:|:--------:|:------------:|:------:|:------:|\n| DiT-S/2 | 33M    | 241.8G   | 28 it/s | 105.82 |  12.59 |\n| DiCo-S  | 33M    | **67.8G**   | **143 it/s** | **95.64**  | **16.44** |\n\nThese results clearly show that DiCo scales much more efficiently to high resolutions—achieving a **5× speedup** with better generation quality. We will include these findings in the final version of the paper.\n\n---\n\nThank you again for helping us strengthen our work. We hope that our response has addressed your concerns. We look forward to further communication with you!\n\n[1] All Are Worth Words: A ViT Backbone for Diffusion Models. In CVPR, 2023.  \n\n[2] Representation Alignment for Generation: Training Diffusion Transformers Is Easier Than You Think. In ICLR, 2025."}, {"review_id": "iPSHyoOgFQ", "reviewer": "Reviewer_LJ6t", "summary": "This paper proposes a new backbone for diffusion models, Diffusion ConvNet (DiCo), as an alternative to DiT. It replaces self-attention with convolutions and proposes the CCA module to reduce channel redundancy. On the ImageNet benchmarks, DiCo surpasses state-of-the-art diffusion models while achieving significant efficiency gains.", "questions": "1. Regarding the CCA Module: Given the functional overlap between CCA and existing channel attention mechanisms (e.g., channel self-attention) in modulating channel features and the seemingly minor performance differences in the ablation study, it would be beneficial to elucidate the specific innovations within CCA that are critical for its effectiveness.\n\n2. Addressing Long-Range Dependencies and Generalizability. Given ConvNets' inherent locality, it is unclear how DiCo effectively captures the crucial long-range dependencies essential for generating high-quality images, especially in complex scenarios such as high-resolution image synthesis, multi-object scenes, or intricate image editing tasks. It would benefit from a discussion on how DiCo addresses this challenge.\n\n3. Questions about the ablation experiment. Table 4 consistently shows FID values below 54 across various configurations, whereas baseline models (DiT-S/2, DiC-S, DiG-S/2) in Table 1 report FID scores exceeding 58. This substantial performance improvement observed in DiCo warrants further explanation.", "limitations": "YES", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 4, "significance": 3, "originality": 3, "strengths_and_weaknesses": "- Strengths:\n1. DiCo models demonstrate substantial computational efficiency gains compared with DiTs, concurrently achieving SOTA performance on ImageNet benchmarks. \n2. This work identifies the channel redundancy issue when introducing convolutions to DiTs and addresses it by introducing CCA.  The proposed solution appears to be reasonable.\n3. Comprehensive ablation studies are provided, detailing both architectural design and the contribution of individual components.\n\n- Weaknesses:\nThe experimental evaluation is primarily restricted to class-conditional image generation. While the authors acknowledge limitations for not extending evaluations to other prominent generative tasks (e.g., text-to-image synthesis), this circumscribes a comprehensive assessment of DiCo's broader applicability and generalizability across the diverse landscape of diffusion model applications.", "was_revised": true, "final_justification": "The authors have addressed my major concerns through experiments. Consequently, I recommend acceptance of this manuscript.\nWhile the COCO experiments (Q1) demonstrate promising results, the integration of DiCo with more complex prompts and image editing tasks warrants further exploration in future work.\nI encourage the authors to release the code and models publicly to accelerate progress in this area.", "rebuttal": "Thank you for your feedback. We appreciate your recognition of the motivation and effectiveness of our method. Here are our responses to your comments.\n\n---\n\n# Main Comments\n\n> **Q1: Lack of experiments for text-to-image generation.**\n\n**A1:** Thank you for your insightful suggestions. Given the limited time during the rebuttal phase, **we follow U-ViT [1] and REPA [2] to conduct small-scale text-to-image generation experiments.** Specifically, we adopt the same experimental setup as [1,2]: training and evaluating models from scratch on the MS-COCO dataset, using CLIP as the text encoder with a token length of 77.\n\n\nWe investigate two different approaches for **incorporating textual features into DiCo.**  \n1. The first uses the widely adopted **cross-attention** mechanism, integrated into the DiCo architecture to fuse text and visual features.  \n2. The second **transforms CLIP text embeddings into dynamic depthwise convolution (DWC) kernels.** We pad the text embeddings to a length of 81, feed them through a learnable MLP, and reshape the output into a 9×9 kernel. This kernel dynamically modulates DiCo's features via depthwise convolution. In this way, we can construct a **fully convolutional** text-to-image DiCo model **without relying on any self-attention or cross-attention operations.** We provide its PyTorch implementation below:\n\n***Algorithm:*** *PyTorch code of text conditional depthwise convolution*\n```python\nimport torch\nimport torch.nn.functional as F\n\ndef text_conditional_dwconv(x, context):\n    # x: (B, C, H, W) input feature maps\n    # context: (B, 77, C) CLIP text embeddings after an MLP\n    # output: (B, C, H, W) output after depthwise convolution\n    B, C, H, W = x.shape\n    context_pad = torch.cat([context, context[:, -1:].expand(-1, 4, -1)], dim=1)  # (B, 81, C)\n    kernels = context_pad.reshape(B, 9, 9, C).permute(0, 3, 1, 2).reshape(B * C, 1, 9, 9)\n    x_flat = x.view(1, B * C, H, W)\n    output = F.conv2d(x_flat, kernels, padding=4, groups=B * C).view(B, C, H, W)\n    return output\n```\n\nBoth feature fusion modules are inserted after the Conv Module within each DiCo block. Following U-ViT, we match the model size and use a CFG scale of 2.0. All FID results  of other methods reported below are taken from REPA [2].\n\n***Table:*** *Comparison on MS-COCO for text-to-image generation*\n| Method|Token Mixing Type | Params | FLOPs | Throughput↑ | FID w/ CFG↓|\n|-|:-:|:-:|:-:|:-:|:-:|\n| U-ViT-S/2 (Deep) |  Full Self-Attn |58M   |   21.3G       | 377 it/s | 5.48 |\n| MM-DiT | Full Self-Attn| 512M  |62.4G |135 it/s | 5.30 |\n| DiCo-CrossAttn (ours)|Conv + CrossAttn | 67M   |  14.9G    | 594 it/s | **4.87** | \n| DiCo-DWC (ours) |Full Conv| 69M   |  **12.9G**    | **659 it/s** | 4.93 | \n\nOur DiCo **outperforms prior methods in both efficiency and generation quality.** Notably, using text conditional DWC in place of cross-attention further improves throughput while maintaining competitive performance. This suggests that the **fully convolutional DiCo** has the potential to serve as the backbone for large-scale **text-to-image diffusion models**. We will include these results in the final version of our paper.\n\n---\n\n> **Q2: Functional overlap between CCA and existing channel attention mechanisms (e.g., channel self-attention).**\n\n**A2:** Thank you for raising this insightful point. We acknowledge that CCA and channel self-attention both aim to modulate channel-wise feature importance. However, our proposed **CCA is specifically designed to be more efficient,** particularly at large model scales, while retaining comparable generative performance.\n\nUnlike standard channel self-attention, which introduces quadratic complexity with respect to channel numbers, **CCA adopts a simplified, structured attention mechanism that avoids heavy pairwise computations.** This design choice enables substantial computational savings without sacrificing effectiveness. The comparison between CCA and channel self-attention at the XL model scale is shown below:\n\n***Table:*** *CCA vs. channel self-attention on ImageNet 256x256 (400K iterations, XL models)*\n| Method | Params | FLOPs | Throughput↑ | FID w/o CFG↓  | IS↑  |\n|-|:-:|:-:|:-:|:-:|:-:|\n| Channel self-attention | 701M | 109.5G   | 150 it/s |11.71|98.76|\n| CCA  | 701M | **87.3G** | **208 it/s** | **11.67**| **100.42**|\n\nCCA achieves a **~1.4× speedup with slightly better FID and IS**, demonstrating its advantage as a lightweight and scalable alternative to conventional channel attention. We will include this analysis and additional clarification on CCA’s design motivation in the final version of the paper.\n\n---\n\n> **Q3: Discussion of addressing long-range dependencies and generalizability.**\n\n**A3:**  Thank you for highlighting this important point. We acknowledge that the local receptive field of convolutional networks may limit their ability to capture long-range dependencies, especially in high-resolution settings.\n\nTo address this, we explore two complementary solutions in DiCo:  \n1. **Enlarging convolutional kernel sizes:** A straightforward yet effective way to expand the receptive field is to use larger convolution kernels. As shown in Table 4 of our paper, increasing the kernel size leads to consistent improvements in generation quality, albeit with a trade-off in computational efficiency. This approach enables DiCo to capture broader context while maintaining the simplicity of convolution.  \n2. **Employing high-compression autoencoders:** An alternative solution is to leverage modern latent diffusion frameworks with **highly compressed latent spaces,** which reduces the burden on the backbone to model long-range spatial dependencies directly. Recent works such as VA-VAE [3] and DC-AE [4] propose autoencoders with downsampling ratios of up to 16× or 32×, which effectively shrink the spatial dimensions.\n\nWe conduct experiments with **VA-VAE on ImageNet 256×256** and **DC-AE on ImageNet 512×512** following their experimental setups. The results are shown below:\n\n***Table:*** *Comparison on ImageNet 256x256 using different Autoencoders*\n| Method    | Autoencoder | Downsample Ratio |Latent Size  | FID w/ CFG↓  | IS↑  |\n|-|:-:|:-:|:-:|:-:|:-:|\n| DiT-XL/2 | SD-VAE  |8x | 4x32x32   | 2.27   |  278.2 |\n| DiCo-XL (ours)| SD-VAE|8x | 4x32x32  |  2.05  | 282.2 |\n| DiCo-XL (ours)| VA-VAE|16x| 32x16x16|  **1.33**  | **300.2** |\n\n***Table:*** *Comparison on ImageNet 512x512 using different Autoencoders*\n|Method| Autoencoder | Downsample Ratio |Latent Size|FID w/ CFG↓| IS↑  |\n|-|:-:|:-:|:-:|:-:|:-:|\n| DiT-XL/2 | SD-VAE  |8x  | 4x64x64 | 3.04 | 240.8|\n| DiCo-XL (ours)| SD-VAE|8x | 4x64x64  |2.53 | 275.7 |\n| DiCo-XL (ours)| DC-AE |32x |32x16x16|**1.78**| **280.1**|\n\nThe use of high-compression autoencoders mitigates the locality limitation of convolution and enables DiCo to model long-range dependencies more effectively. Notably, DiCo achieves better **FID scores of 1.33 on ImageNet 256×256 and 1.78 on ImageNet 512×512** with these autoencoders. We hope to further explore the application of DiCo in more complex scenarios in future work. We will include the discussion in the final version of our paper.\n\n---\n\n> **Q4: Table 4 consistently shows FID values below 54 across various configurations, whereas baseline models (DiT-S/2, DiC-S, DiG-S/2) in Table 1 report FID scores exceeding 58.**\n\n**A4:** Thank you for your careful review. The difference in FID values between Table 1 and Table 4 primarily stems from **architecture settings.** In Table 1, both DiCo and DiC are based on the U-Shape architecture, while DiT and DiG adopt isotropic architectures. In the ablation studies presented in Table 4, we keep the U-Shape architecture fixed while ablating different modules within DiCo. Moreover, in Table 3, we show that DiCo consistently achieves superior performance across different architectural settings. To make this distinction more explicit, **we summarize the fair comparisons below by isolating the effect of architecture:**\n\n***Table:*** *Comparison on ImageNet 256x256 (Isotropic, 400K iterations, Small models)*\n| Method   |  Arch  | Params | FLOPs | Throughput↑ |FID w/o CFG↓| IS↑|\n|-|:-:|:-:|:-:|:-:|:-:|:-:|\n| DiT (Self-Attn) | Isotropic |32.9M    | 6.1G  |  1087 it/s | 67.2 |  20.4 |\n| DiG (Gated Linear Attn) |  Isotropic|33.0M | 4.3G | 961 it/s| 62.1 |  22.8 |\n| DiCo (Conv Module)|Isotropic| 33.7M| 5.7G| **1901 it/s** | **60.6**  | **25.4** |\n\n***Table:*** *Comparison on ImageNet 256x256 (U-Shape, 400K iterations, Small models)*\n| Method|Arch|Params|FLOPs|Throughput↑|FID w/o CFG↓|IS↑|\n|-|:-:|:-:|:-:|:-:|:-:|:-:|\n| DiT (Self-Attn) | U-Shape |33.0M    | 4.2G  |  1141 it/s | 54.0 |28.5|\n| DiG (Gated Linear Attn) | U-Shape |33.1M    | 4.2G  |937 it/s| 53.2| 28.9  |\n| Window Attn | U-Shape |33.1M| 4.2G|1165 it/s| 53.2|28.3|\n| Focused Linear Attn | U-Shape |32.9M| 4.3G  |971 it/s| 50.6|30.8|\n|Agent Attn| U-Shape | 33.3M| 4.3G| 1161 it/s | 52.1 |28.8|\n| DiCo (Conv Module, w/o CCA)|U-Shape|33.0M|4.2G|**1731 it/s**|54.8|28.4|\n| DiCo (Conv Module)|U-Shape|33.0M|4.2G|1696 it/s|**50.0**|**31.4**|\n\n**Under a fair comparison with identical architectures, our DiCo consistently outperforms the baseline methods.** This indicates that the **performance and efficiency gains of DiCo stem from our proposed Conv Module and CCA.** We will clarify this point more explicitly in the final version of the paper.\n\n---\n\nThank you again for helping us strengthen our work. We hope that our response has addressed your concerns. We look forward to further communication with you!\n\n[1] All Are Worth Words: A ViT Backbone for Diffusion Models. In CVPR, 2023.  \n\n[2] Representation Alignment for Generation: Training Diffusion Transformers Is Easier Than You Think. In ICLR, 2025.\n\n[3] Reconstruction vs. Generation: Taming Optimization Dilemma in Latent Diffusion Models. In CVPR, 2025.\n\n[4] Deep Compression Autoencoder for Efficient High-Resolution Diffusion Models. In ICLR, 2025."}, {"review_id": "kqNyk9AaZ9", "reviewer": "Reviewer_c9AS", "summary": "Diffusion Transformer (DiT), a promising diffusion model for visual generation, demonstrates impressive performance but incurs significant computational overhead. Intriguingly, analysis of pre-trained DiT models reveals that global self-attention is often redundant, predominantly capturing local patterns. This paper proposes a pure convolution-based diffusion model, deemed as DiCo, to explore the cnn-like diffusion variant.", "questions": "see weakness", "limitations": "see weakness", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 2, "significance": 4, "originality": 1, "strengths_and_weaknesses": "Strengths\n1. The comparison is fair and reasonable.\n2. The writing is clear and easy to understand.\n3. The acceleration of DiCo is significant and obvious.\n\nWeaknesses\n1. Lack of text-to-image results. As convolution is naturally incompatible with attention, I am doubtful about the multimodal performance, like text-to-image tasks.\n2. The necessity of the long residuals. UNet often converges faster with an inferior final performance.  As pointed out by DiT[1] and SD3[2], ***the U-Net inductive bias is not crucial(see the introduction section in DiT), and the U-shape limits further scaling (see Fig.4 in SD3)***.  Thus, DiCo looks completely like a technological regression.\n3. The claim of being state-of-the-art (SoTA) is unjustified, as DiCo's performance significantly lags behind modern diffusion transformers such as lightning-dit[3].\n4. There is no comparison with FlowDCN[4]. Deformable convolution is one of the most powerful convolution variants. It is appropriate to provide some discussions and comparisons.\n\n[1] Peebles W, Xie S. Scalable diffusion models with transformers[C]//Proceedings of the IEEE/CVF international conference on computer vision. 2023: 4195-4205.\n\n[2] Esser P, Kulal S, Blattmann A, et al. Scaling rectified flow transformers for high-resolution image synthesis[C]//Forty-first international conference on machine learning. 2024.\n\n[3]Yao J, Yang B, Wang X. Reconstruction vs. generation: Taming optimization dilemma in latent diffusion models[C]//Proceedings of the Computer Vision and Pattern Recognition Conference. 2025: 15703-15712.\n\n[4]Wang S, Li Z, Song T, et al. FlowDCN: Exploring DCN-like Architectures for Fast Image Generation with Arbitrary Resolution[J]. arXiv preprint arXiv:2410.22655, 2024.", "was_revised": true, "final_justification": "The author's response has addressed all my concerns, and I agree to raise the final score to accept.", "rebuttal": "Thank you for your feedback. We appreciate your recognition of the effectiveness of our method and the writing of our paper. Here are our responses to your comments.\n\n---\n\n# Main Comments\n\n> **Q1: Lack of text-to-image results.**\n\n**A1:** Thank you for your insightful suggestions. Given the limited time during the rebuttal phase, **we follow U-ViT [1] and REPA [2] to conduct small-scale text-to-image generation experiments.** Specifically, we adopt the same experimental setup as [1,2]: training and evaluating models from scratch on the MS-COCO dataset, using CLIP as the text encoder with a token length of 77.\n\nWe investigate two different approaches for **incorporating textual features into DiCo.**  \n1. The first uses the widely adopted **cross-attention** mechanism, integrated into the DiCo architecture to fuse text and visual features.  \n2. The second **transforms CLIP text embeddings into dynamic depthwise convolution (DWC) kernels.** We pad the text embeddings to a length of 81, feed them through a learnable MLP, and reshape the output into a 9×9 kernel. This kernel dynamically modulates DiCo's features via depthwise convolution. In this way, we can construct a **fully convolutional** text-to-image DiCo model **without relying on any self-attention or cross-attention operations.** We provide its PyTorch implementation below:\n\n***Algorithm A:*** *PyTorch code of text conditional depthwise convolution*\n```python\nimport torch\nimport torch.nn.functional as F\n\ndef text_conditional_dwconv(x, context):\n    # x: (B, C, H, W) input feature maps\n    # context: (B, 77, C) CLIP text embeddings after an MLP\n    # output: (B, C, H, W) output after depthwise convolution\n    B, C, H, W = x.shape\n    context_pad = torch.cat([context, context[:, -1:].expand(-1, 4, -1)], dim=1)  # (B, 81, C)\n    kernels = context_pad.reshape(B, 9, 9, C).permute(0, 3, 1, 2).reshape(B * C, 1, 9, 9)\n    x_flat = x.view(1, B * C, H, W)\n    output = F.conv2d(x_flat, kernels, padding=4, groups=B * C).view(B, C, H, W)\n    return output\n```\n\nBoth feature fusion modules are inserted after the Conv Module within each DiCo block. Following U-ViT, we match the model size and use a CFG scale of 2.0. All FID results of other methods reported below are taken from REPA [2].\n\n***Table:*** *Comparison on MS-COCO for text-to-image generation*\n| Method|Token Mixing Type|Params| FLOPs |Throughput↑|FID w/ CFG↓\n|-|:-:|:-:|:-:|:-:|:-:\n|U-ViT-S/2 (Deep)|Full Self-Attn|58M|21.3G |377 it/s| 5.48\n|MM-DiT| Full Self-Attn|512M|62.4G |135 it/s|5.30\n|DiCo-CrossAttn (ours)|Conv + CrossAttn |67M|14.9G |594 it/s|**4.87**\n|DiCo-DWC (ours)|Full Conv|69M|**12.9G**|**659 it/s**|4.93\n\nOur DiCo **outperforms prior methods in both efficiency and generation quality.** Notably, using text conditional DWC in place of cross-attention further improves throughput while maintaining competitive performance. This suggests that the **fully convolutional DiCo** has the potential to serve as the backbone for large-scale **text-to-image diffusion models**. We will include these results in the final version of our paper.\n\n---\n\n> **Q2: The necessity of the long residuals. UNet often converges faster with an inferior final performance. The U-Net inductive bias is not crucial (see introduction in DiT), and the U-shape limits further scaling (see Fig.4 in SD3).**\n\n**A2:** Thank you for this insightful observation. We would like to clarify the following points:  \n1. While the inductive bias of U-Net may become less crucial when ample training resources are available, it remains valuable for **reducing training costs in resource-constrained scenarios.**  \n2. Our proposed method is **not limited to the U-Net structure**. It also achieves **strong performance within isotropic architectures.**  \n\nBelow, we provide a more detailed explanation of these points.\n\n***For point 1:*** We acknowledge that recent works such as DiT and SD3 have argued that the U-Net inductive bias is less critical when models are trained at large scales with ample computational resources. However, **in many practical settings where compute is constrained,** the U-Net-style long residual connections remain valuable for **accelerating convergence and improving training efficiency.**\n\nAs shown in the table below, DiCo with a U-Shape architecture **achieves better performance than DiT** while requiring **significantly fewer training iterations.** This highlights that the U-Shape structure enables **faster convergence and lower training cost,** which is especially important in resource-constrained settings.\n\n***Table:*** *Comparison on ImageNet 256x256 for different iterations*\n|Method|Arch|Iterations|FID w/ CFG↓|IS↑\n|-|:-:|:-:|:-:|:-:\n|DiT-XL/2|Isotropic|7000K|2.27|278.2\n|DiCo-XL|U-Shape|1500K|2.25|270.4\n|DiCo-XL|U-Shape|3750K|2.05|282.2\n\n\nFurthermore, we respectfully clarify that Fig. 4 in SD3 does not claim U-Net inherently limits scalability (see the FID and CLIP score curves). In fact, the SD3 paper explicitly notes that \"Vanilla DiT underperforms UViT\" (see Sec. 5.2.3, page 9). A similar observation is made in DreamOmni [3] (see Fig. 3 and Sec. 4, page 6). In practice, several large-scale diffusion models continue to use long residuals successfully. For example, DreamOmni [3] (U-Shape) and Vidu [4] (Isotropic with skip connections) both adopt architectures with long residuals at scale.\n\n***For point 2:*** Equally important, we emphasize that **the U-Shape design is not our core architectural contribution.** As shown in Table 3 of the paper, **DiCo consistently outperforms DiT** in both performance and efficiency **across various architectural settings** (isotropic, isotropic with skip connections, and U-shape). Moreover, we demonstrate that our method can be effectively applied to build **isotropic diffusion models**, achieving superior performance (FID of 1.33 on ImageNet 256x256) compared to modern DiTs (refer to **A3**). This provides strong evidence that our method **generalizes well beyond U-Net designs and is not inherently tied to long residuals.**\n\nWe will include the above discussion in the final version of the paper.\n\n---\n\n> **Q3: The claim of being SOTA is unjustified, as DiCo's performance lags behind modern DiTs such as LightningDiT.**\n\n**A3:** We appreciate your feedback regarding our SOTA claim. Given the rapid advancements in modern DiTs, we agree that the SOTA statement in our paper should be revised. We also recognize the importance of comparing our method with recent DiTs such as LightningDiT. We find that after aligning with their experimental settings, our method achieves **competitive performance along with a clear efficiency advantage.** The details are shown as follows.\n\nIn our paper, we adopt **the original DiT experimental setup** (e.g., using SD-VAE, standard DDPM training, fixed hyperparameters) to ensure a fair and controlled comparison with DiT, DiG, and DiC. However, **LightningDiT differs from DiT's setup in several aspects**—for example, it replaces the SD-VAE with VA-VAE, adopts the Rectified Flow training objective, and modifies training hyperparameters, etc.\n\nTo fairly assess DiCo’s competitiveness under this enhanced setup, we construct **LightningDiCo,** which **replaces LightningDiT’s self-attention with our Conv Module,** while keeping all other settings identical. This version uses an **isotropic backbone** (no U-Net structure), aligning directly with LightningDiT.\n\n***Table:*** *Comparison with modern DiTs on ImageNet 256x256*\n| Method |Epochs|Params|FLOPs|Throughput↑|FID w/ CFG↓|IS↑\n|-|:-:|:-:|:-:|:-:|:-:|:-:\n|REPA |800|675M|118.7G|77 it/s|1.42|**305.7**\n|LightningDiT|800|675M|118.8G|73 it/s|1.35|295.3\n|LightningDiCo (ours)|800|679M|118.3G|**201 it/s**|**1.33**|300.2\n\nLightningDiCo achieves **an FID score of 1.33** and offers a **~2.8× throughput** gain over modern DiTs. This demonstrates that our convolutional backbone can **match or exceed the generative quality of modern DiTs while being significantly more efficient.** We will include this comparison in the final version of the paper and revise the SOTA claim.\n\n---\n\n> **Q4: Missing comparison with FlowDCN.**\n\n**A4:** Thank you for pointing this out. We agree that deformable convolution is a strong and widely used convolutional variant, and FlowDCN represents an important baseline in this space.\n\nTo fairly compare with FlowDCN, we **retain its original isotropic architecture and training setup,** and replace its **MultiScale DCN**  with our proposed **Conv Module.** The results are shown below:\n\n***Table:*** *Comparison on ImageNet 256x256 (400K iterations, Small models)*\n|Method|Params|FLOPs|Throughput↑|FID w/o CFG↓|IS↑\n|-|:-:|:-:|:-:|:-:|:-:\n|SiT-S/2|32.9M|6.1G|1234 it/s|57.6|24.8\n|FlowDCN-S/2|30.3M|4.4G|1194 it/s|54.6|26.4\n|Ours-S/2|31.3M|4.6G|**2489 it/s**|**52.1**|**29.2**\n\nOur Conv Module achieves **better generative performance and ~2.1× higher throughput** compared to MultiScale DCN, despite being structurally simpler and easier to implement. We will include this comparison in the final version of the paper and add a discussion of FlowDCN in the related work.\n\n---\nThank you again for helping us strengthen our work. We hope that our response has addressed your concerns. We look forward to further communication with you!\n\n[1] All Are Worth Words: A ViT Backbone for Diffusion Models. In CVPR, 2023.  \n\n[2] Representation Alignment for Generation: Training Diffusion Transformers Is Easier Than You Think. In ICLR, 2025.\n\n[3] DreamOmni: Unified Image Generation and Editing. In CVPR, 2025.\n\n[4] Vidu: a highly consistent, dynamic and skilled text-to-video generator with diffusion models. arXiv preprint arXiv:2405.04233."}]

Author rebuttal:

Meta-review: ***(a) Scientific Claims and Findings***

This paper introduces DiCo, a family of convolutional diffusion models enhanced with a lightweight Compact Channel Attention (CCA) mechanism to address channel redundancy. The work challenges the prevailing assumption that Vision Transformers (ViTs) are inherently superior for diffusion modeling, showing that a fully convolutional design can match or surpass DiT-based models in both quality and efficiency. Empirical results demonstrate strong improvements in throughput and FID across ImageNet (256, 512, 1024 resolutions), text-to-image generation on MS-COCO, and pixel-level diffusion.

***(b) Strengths and Weaknesses***

The submission is well-written, clearly motivated, and supported by extensive ablations and comparisons. Strengths include the novel perspective on convolutional backbones, convincing efficiency advantages (especially at high resolutions), and thorough rebuttal clarifications. Weaknesses raised by reviewers included questions of novelty, lack of multimodal results, generalizability beyond ImageNet, and overstated SOTA claims.

***(c) Discussion and Rebuttal Summary***
+ Reviewer 3NNh: Concerned about novelty, lack of UNet comparisons, and terminology. Authors clarified innovation, added UNet/MM-DiT comparisons, and revised terminology. The reviewer was satisfied and supported acceptance.
+ Reviewer yq4n: Requested text-to-image and higher-resolution results. Authors added MS-COCO, pixel-level ImageNet 64×64, and ImageNet 1024×1024 experiments, addressing concerns. The reviewer raised the score to strong accept.
+ Reviewer LJ6t: Questioned CCA’s originality and DiCo’s generalizability. Authors showed efficiency advantages, discussed strategies for long-range dependencies, and clarified ablation settings. The reviewer raised the score to accept.
+ Reviewer c9AS: Flagged missing multimodal evaluation, U-Net reliance, SOTA claims, and FlowDCN comparisons. Authors added text-to-image experiments, clarified independence from U-Nets, compared against LightningDiT and FlowDCN, and softened SOTA claims. The reviewer raised the score to accept.

After the rebuttal, all reviewers increased their scores, resulting in 1 borderline accept, 2 accepts, and 1 strong accept.

***(d) Reasons to Final Recommendation: Accept (Spotlight)***

This work stands out for (1) challenging a dominant paradigm by reviving ConvNets for diffusion models, (2) providing strong empirical validation across diverse tasks, (3) demonstrating scalability to billion-parameter and multimodal settings, and (4) showing compelling efficiency gains at high resolutions (e.g., 5× faster at 1024×1024). These qualities make it a timely and influential contribution likely to inspire further research.
Decision: Accept (spotlight)