Taming Diffusion Transformer for Efficient Mobile Video Generation in Seconds

## Abstract

Abstract Diffusion Transformers (DiT) have shown strong performance in video generation tasks, but their high computational cost makes them impractical for resource-constrained devices like smartphones, and practical on-device generation is even more challenging.
In this work, we propose a series of novel optimizations to significantly accelerate video generation and enable practical deployment on mobile platforms.
First, we employ a highly compressed variational autoencoder (VAE) to reduce the dimensionality of the input data without sacrificing visual quality.
Second, we introduce a KD-guided, sensitivity-aware tri-level pruning strategy to shrink the model size to suit mobile platforms while preserving critical performance characteristics.
Third, we develop an adversarial step distillation technique tailored for DiT, which allows us to reduce the number of inference steps to four.
Combined, these optimizations enable our model to achieve approximately 15 frames per second (FPS) generation speed on an iPhone 16 Pro Max, demonstrating the feasibility of efficient, high-quality video generation on mobile devices.

## 1 Introduction

The rapid advancement of generative models (Liu et al., 2022; Esser et al., 2024; Rombach et al., 2022) has led to significant breakthroughs in video generation (Blattmann et al., 2023; OpenAI, 2023; Zheng et al., 2024; Wan et al., 2025; Yang et al., 2024b; Team, 2024; Kong et al., 2024), with Diffusion Transformers emerging as one of the most effective architectures for producing temporally coherent and visually compelling video content.
These models leverage the strengths of diffusion processes (Rombach et al., 2022; Ho et al., 2020; Liu et al., 2022) for stepwise refinement and transformer-based attention for capturing long-range dependencies across frames, making them particularly suitable for generating complex, high-fidelity video sequences.
As such, they have become a cornerstone in state-of-the-art video synthesis pipelines.

Despite their impressive generative capabilities, Diffusion Transformers suffer from substantial computational overhead, especially when applied to high-resolution video generation.
While this brings significant quality improvements, the computation and memory consumption of the 3D full attention (Yang et al., 2024b; Kong et al., 2024) scale quadratically with respect to total tokens ($t\times H\times W$).
This limitation poses a critical challenge for deploying these models in interactive settings, particularly on mobile devices with limited processing power and energy budgets.
Existing efforts to optimize diffusion-based models, such as step reduction (Zhang et al., 2024; Li et al., 2023; Yin et al., 2024a), efficient backbone (Wu et al., 2025b; Yahia et al., 2024; Zhao et al., 2024), are mainly focused on UNet-based denoisers, which are naturally less expressive. Very few work (HaCohen et al., 2024b) investigates the efficiency of DiT, and often suffers from perceptual quality or temporal consistency loss.
Furthermore, most current acceleration methods are designed for desktop (HaCohen et al., 2024b; Wan et al., 2025), and do not translate well to edge devices.

**Table 1: Our model is the first DiT-based mobile video generator. Generation speed is reported as FPS. See [Sec.˜5](https://arxiv.org/html/2507.13343v2#S5) for details.**
| Model | Params (B) | VBench | A100 | iPhone |
| --- | --- | --- | --- | --- |
| Wan2.1 | 1.3 | 83.33 | 0.2 | ✗ |
| LTX | 1.8 | 80.00 | 6.1 | ✗ |
| Ours-Server | 2.0 | 83.09 | 6.4 | ✗ |
| Ours-Mobile | 0.9 | 81.45 | 151.3 | $\sim$15 |

In this work, we present a comprehensive optimization pipeline tailored specifically to accelerate video diffusion transformers for mobile deployment.
Our approach combines four key strategies.

(A) High-Compression Video Variational Autoencoder (VAE).
First, we investigate the compression rate of the VAE.
High compression video VAE can significantly reduce the number of tokens in the latent representation, thus speeding up DiT inference.
However, VAEs with an aggressive compression ratio often suffer from a loss of reconstruction quality, which likely leads to a loss in diffusion model generation quality.
The trade-off between compression ratio and diffusion quality remains underexplored for on-device video generation.
In this work, we create a series of video VAEs with different compression ratios and compare the speed gain versus generation quality loss.
We have several findings:
(i) The reconstruction and diffusion generation quality corresponds well with the compression ratio.
(ii) The speedup from the higher VAE compression ratio is significant.
(iii) Though slightly degraded, we can still find a sweet point that balances speed and quality.

(B) Efficient Mobile DiT.
Second, we find that directly training a lightweight DiT designated for mobile is challenging.
Instead, we start from a larger pre-trained supernet and propose a sensitivity-aware tri-level pruning with a KD-Guided framework that selectively removes less critical components of the model based on their contribution to both runtime and output quality.
This pruning reduces the number of DiT blocks, feed-forward features, and attention heads.
The final architecture has $915$M parameters and can be easily deployed on a modern device such as an iPhone 16 Pro Max.
Further, we improve the pruned model performance by aligning features of the pruned network and the supernet through knowledge distillation.

(C) Adversarial Step Distillation.
Third, we design a new discriminator head tailored for adversarial step-distillation on DiTs that achieves full-step quality with only a few sampling steps.
Prior adversarial distillation methods for video diffusion models mainly focus on UNet backbones (Yahia et al., 2024; Zhang et al., 2024) and less challenging image-to-video tasks (Blattmann et al., 2023) and do not transfer directly to DiTs.
Our discriminator design inherits the first $K$ frozen pretrained generator blocks as a time-conditioned feature parser, and adds learnable with self-attention and cross-attention to fully capture conditions.
This design enables *four-step* generation without classifier-free guidance (CFG), yielding $20\times$ faster inference than a typical 40-step CFG recipe.

(D) Operator Optimization for Efficient Inference.
Finally, we identify a memory bottleneck in the linear layer of DiTs (*i.e*. feed-forward network), where the limited bandwidth prevents operators from approaching their theoretical speed on device.
To address this, we introduce a tiled GEMM strategy that alleviates the memory bottleneck without requiring kernel-level modifications.
The design achieves over 50% speedup on targeted linear layers and $\sim$10% acceleration for DiT inference.

With these optimizations, our model can generate high-quality video at over $15$ frames per second (FPS) on an iPhone 16 Pro Max using only four denoising steps.
Extensive experiments demonstrate that our method maintains strong visual fidelity and temporal consistency, closely matching the outputs of full-resolution, unpruned models.
For the first time, our work advances the state-of-the-art for on-device efficient video generation by making practical diffusion-based video synthesis feasible on consumer-grade mobile hardware. Our contributions can be summarized as follows,

- •
We are the first to systematically investigate the trade-off between latent compression ratio, generation quality, and speed for on-device video generation. We find that for diffusion transformer, a $8\times 32\times 32$ VAE achieves a good trade-off between generation speed and quality.
- •
To obtain an efficient DiT backbone, training a smaller network from scratch gives inferior results. Instead, we start from a large pre-trained super network and apply distillation-guided, sensitivity-aware pruning, yielding a compact network with optimized depth and width.
- •
For adversarial step-distillation, we propose a new discriminator design tailored for DiTs which outperforms prior methods by a large margin. We achieve $4$-step inference without CFG.
- •
We identify the memory bottleneck in the feed-forward layers of DiTs on device performance and introduce a tiled GEMM strategy that alleviates this issue, enabling more efficient and hardware friendly on-device inference.

## 2 Related Work

Video Diffusion Models.
Recent years have seen rapid progress in video generation models (Wan et al., 2025; OpenAI, 2023; Lin et al., 2024; Yang et al., 2024b; Kong et al., 2024; Team, 2024; Kuaishou, 2024; Ma et al., 2025).
Most advances focus on large diffusion models that iteratively denoise Gaussian noise into realistic videos, conditioned on text or images.
These approaches include pixel-space models (Menapace et al., 2024; Ho et al., 2022) and latent-space models (Wan et al., 2025; Yang et al., 2024b).
While such systems (OpenAI, 2023; Zheng et al., 2024; Menapace et al., 2024; Polyak et al., 2024; HaCohen et al., 2024b; Yang et al., 2024b) generate high-quality videos, their resource demands make them unsuitable for on-device use.

On-Device Models.
In contrast, only limited work targets on-device video generation (Wu et al., 2025b; Kim et al., 2025).
The Wan2.1 family (Wan et al., 2025) includes a $1.3$B T2V model, but its low VAE compression yields too many latent tokens for deployment.
LTX-Video (HaCohen et al., 2024a) applies a high-compression VAE and runs in real time on GPUs, yet its $1.9$B parameters remain prohibitive for mobile devices.
SnapGen-V (Wu et al., 2025b) adopts a lightweight UNet but sacrifices visual fidelity.
Mobile Video Diffusion (Yahia et al., 2024) reduces Stable Video Diffusion (Blattmann et al., 2023) by pruning channels and blocks.
On-device Sora (Kim et al., 2025) achieves low-resolution video generation on iPhones via temporal token merging and concurrent block loading to handle memory limits.

Step Distillation.
Diffusion models (Esser et al., 2024; Podell et al., 2023; Hoogeboom et al., 2023) require many denoising steps, each involving a full network pass, which creates latency.
Reducing the number of steps directly improves efficiency.
Numerous methods address this in text-to-image tasks (Yin et al., 2024b; a; Yang et al., 2024a; Wang et al., 2024b; Kim et al., 2024; Mei et al., 2024; Dao et al., 2025), representative work including progressive distillation (Salimans & Ho, 2022; Li et al., 2023), consistency models (Song et al., 2023; Song & Dhariwal, 2023), adversarial training (Xu et al., 2024; Sauer et al., 2023b; 2024), shortcut models (Frans et al., 2024), and mean flow (Geng et al., 2025).
For video, Zhang et al. (2024); Wu et al. (2025b) achieves few-step generation Blattmann et al. (2023) with adversarial training specially-designed spatio-temporal discriminator.

## 3 Preliminaries

Following popular practices of latent diffusion (Zheng et al., 2024), we employ a video autoencoder to encode video data $\mathbf{X}\in\mathbb{R}^{3\times T\times H\times W}$ into a compressed latent space $\mathbf{x}\in\mathbb{R}^{c\times t\times h\times w}$, where $T$ is the number of temporal frames, $H$ and $W$ are the spatial resolutions, and $c$ is the latent channels.
The VAE compression ratio is thus $\frac{T}{t}\times\frac{H}{h}\times\frac{W}{w}$, e.g., $4\times 8\times 8$ (Yang et al., 2024b; Kong et al., 2024; Wan et al., 2025) and $8\times 16\times 16$ (Agarwal et al., 2025) VAEs.
The objective of the DiT generator is to generate $\mathbf{x}$ under certain guidance (i.e., text prompt).

We employ Rectified Flow (Wang et al., 2024b) to train our latent DiT model.
According to the flow-matching-based diffusion process, given a clean video latent $\mathbf{x}_{0}=\mathbf{x}$, the intermediate noisy state $\mathbf{x}_{t}$ at a timestep $t$ is:

$$ $\mathbf{x}_{t}=\left(1-t\right)\mathbf{x}_{0}+t\epsilon,\text{where}~\epsilon\sim\mathcal{N}\left(0,\mathit{I}\right),$ (1) $$

which is a linear interpolation between the data distribution and a standard normal distribution.
The model aims to learn a vector field $v_{\theta}\left(t,\mathbf{x}_{t}\right)$ using the Conditional Flow Matching objective, *i.e*.,

$$ $\mathcal{L}_{\text{fm}}=\mathbb{E}_{t,\epsilon,\mathbf{x}_{0}}\left\|v_{\theta}\left(t,\mathbf{x}_{t}\right)-(\epsilon-\mathbf{x}_{0})\right\|_{2}^{2}.$ (2) $$

Figure: Figure 2: Overview of proposed KD-Guided Tri-Level Pruning and new discriminator head. The tri-level pruning scheme operates across three levels of granularity, the block, attention-head, and feed-forward network dimension, ranging from coarse to fine. This design enables flexible, efficient, and stable model compression. Additionally, the proposed discriminator adopts standard DiT blocks with a MLP classifier head, improved condition alignment for adversarial training.
Refer to caption: x2.png

## 4 Method

We optimize Diffusion Transformer (DiTs) for efficient on-device video generation from four perspectives:
(a) High compress VAE:we employ a high-compression autoencoder, as the computational complexity of transformer scales quadratically with token length.
Reducing the number of token decreases computation while enabling video generation at higher resolution and longer duration.
(b) Efficient DiT architecture:we design an efficient DiT using a KD-Guided Tri-Level pruning method.
The method balances model fidelity to the baseline with the hardware constraint of the target device.
(c) Step-distillation:we adopt adversarial fine-tuning for step distillation with a new discriminator head design, reducing the number of sampling steps and achieving up to $20\times$ acceleration during inference.
(d) Operator Optimization:we identify the memory bottlenecks in DiT feed-forward layers and introduce a tiled GEMM strategies, alleviating bandwidth limitations and enabling efficient on-device inference without requiring kernel/compiler modification.

### 4.1 Scaling Latent Compression Ratio

DiT demonstrates superior generation capabilities when attending on full token length ($thw$), however, it is also notorious for its quadratic computational cost.
The key idea of the latent diffusion model is to construct a compressed latent space and reduce the generation cost.
As a result, a straightforward idea to accelerate DiT is to further increase the VAE compression ratio.
State-of-the-art models (CogVideoX (Yang et al., 2024b),Hunyuan(Kong et al., 2024),Wan (Wan et al., 2025)) employ a $4\times 8\times 8$ VAE combined with a $1\times 2\times 2$ patchify module, which comprises a $4\times 16\times 16$ total compression rate, while the recent OpenSora-2 (Zheng et al., 2024) adopts a $4\times 32\times 32$ VAE, and LTX (HaCohen et al., 2024b) adopts an $8\times 32\times 32$ VAE to reduce the dimensionality of the latent features input to the DiT and results in faster generation speed.
However, there has been limited research on how the VAE compression ratio affects the quality and speed of video generation.
Upon aggressive compression, it becomes more challenging for the VAE decoder to fully reconstruct the details, which may result in quality loss.
In this work, we perform a comprehensive study on the scaling of the VAE compression ratio.
We follow HaCohen et al. (2024b); Wu et al. (2025a) and construct video VAEs with various compression ratios from $4\times 16\times 16$ to $8\times 64\times 64$.
We build the VAE with 3D convolutions to better handle video modality, and use a fixed latent channel number for all variants.
We train the same DiT network under each latent space and benchmark the generation speed and quality.
Results and discussions are in [Secs.˜5.3](https://arxiv.org/html/2507.13343v2#S5.SS3) and [G](https://arxiv.org/html/2507.13343v2#A7).

Figure: Figure 3: Sensitivity Analysis of DiT Components. The sensitivity analysis is conducted by progressively pruning DiT blocks, attention-heads and feed-forward network (FFN) dimension. For each setting, we benchmark FLOPs, memory usage, inference speed, and VBench score to assess the impact of each component on model efficiency and performance.
Refer to caption: x3.png

### 4.2 Efficient DiT Architecture via KD-Guided Tri-Level Pruning

Despite operating in a highly compressed latent space, the size of the Diffusion Transformer (DiT) remains a critical factor in edge generation scenarios (Wu et al., 2025b), where mobile devices are constrained by limited memory, power, and computational resources. Training a compact DiT that still achieves high-quality generation is a non-trivial challenge.
First, DiT models generally exhibit strong generation capabilities only when scaled to a sufficiently large capacity. Moreover, designing an effective small-scale DiT is difficult due to the high-dimensional design space—including network depth (number of transformer blocks), width (channel size), and attention head count.

A promising alternative is to begin with a well-trained large model and prune it to meet resource constraints. Prior work such as TinyFusion (Fang et al., 2024) explored this approach via block-wise pruning using a learnable layer mask, effectively constructing a shallower DiT. However, as the properties demonstrated in [Figure˜3](https://arxiv.org/html/2507.13343v2#S4.F3), it is still of great value to design a fine-grained pruning method that offers deeper insight into which parameters are critical or redundant, thereby enabling a better trade-off between efficiency and generation quality.

To address these, we propose a tri-level pruning scheme combined with knowledge alignment, enabling us to derive an efficient DiT architecture from a larger teacher model. Our approach maintains competitive performance while meeting the requirements for edge deployment.

#### 4.2.1 Tri-level Pruning

As exhibited in [Figure˜3](https://arxiv.org/html/2507.13343v2#S4.F3), the transformer block pruning is a simple yet coarse approach, we consider it a low-granularity pruning. To enable finer granularity and better address redundancies, we propose a tri-level pruning scheme that incorporates block pruning and further introduces fine-grained pruning techniques, including head pruning for the multi-head attention mechanism and channel pruning for the linear layer.
Notably, the pruned model can be converted into a dense and compact form, enabling execution on mobile devices without requiring additional compilation or specialized hardware support.

We employ a set of learnable binary masks to implement the tri-level pruning scheme.
Each binary mask encodes the importance of its corresponding granularity, *i.e*. *block*, *attention-head*, or *linear dimension*.
A mask value of 0 indicates that the corresponding unit should be pruned, while a value of 1 denotes that it should be preserved.
Specifically, block pruning can be formulated as shown in [Eq.˜3](https://arxiv.org/html/2507.13343v2#S4.E3), where $y_{b_{i}},x_{b_{i}}$ indicate the input and output features of the $b_{i}^{\text{th}}$ DiT block, $m_{b_{i}}\in\{0,1\}$ is the binary mask associated with that block, and $\mathcal{M}_{b}=[m_{b_{1}},\dots,m_{b_{N}}]\in\{0,1\}^{N}$ denotes the set of binary masks for all DiT blocks.
When $m_{b_{i}}=1$, the block is active; otherwise, its output is bypassed through a residual connection.

$$ $y_{b_{i}}=\text{Block}_{b_{i}}(x_{b_{i}})\odot m_{b_{i}}+x_{b_{i}}\odot(1-m_{b_{i}}),\quad m_{b_{i}}\in\{0,1\},\mathcal{M}_{b}=[m_{b_{1}},\dots,m_{b_{N}}]\in\{0,1\}^{N}$ (3) $$

The other two pruning schemes can be expressed using a unified formulation, since pruning *attention heads* is equivalent to removing specific output features before the multi-head attention operation for each token.
By integrating the pruning mechanism into the linear layer, the operation can be formulated as shown in [Eq.˜4](https://arxiv.org/html/2507.13343v2#S4.E4):

$$ $y_{l_{i}}=\text{Linear}_{i}(x_{l_{i}},W_{l_{i}},b_{l_{i}})\odot\mathcal{M}_{l_{i}},\quad m^{d}_{l_{i}}\in\{0,1\},\mathcal{M}_{l_{i}}=[m^{1}_{l_{i}},\cdots,m^{D}_{l_{i}}]\in\{0,1\}^{D}$ (4) $$

where $y_{l_{i}},x_{l_{i}}$ denote the output and input features of the $l_{i}^{\text{th}}$ linear layer, and $\mathcal{M}_{l_{i}}\in\{0,1\}^{D}$ is a binary mask with $D$-dimension corresponding to the output channels.
For each $m^{d}_{l_{i}}\in{M}_{l_{i}}$, a value of $m^{d}_{l_{i}}=0$ zeros out the corresponding output channel at dimension $d$ for layer $l^{\text{th}}_{i}$; otherwise the channel remains active.

The proposed tri-level pruning scheme begins by generating a candidate mask set $\mathbb{M}$ for each pruning target, as illustrated in [Figure˜2](https://arxiv.org/html/2507.13343v2#S3.F2).
These candidate masks are selected based on the desired number of active components, which are constrained by the memory limitation of the target device.
Notably, exhaustively exploring all pruning combinations results in an extremely large search space, making the optimization problem intractable (*e.g*., pruning 6 out of 32 attention heads results in 906,192 possible configurations).
To mitigate this, we adopt a group-wise masking mechanism that partitions overall search space into smaller subspaces, allowing pruning to be performed efficiently within each subspace.
Once the candidate masks are generated, we further optimize them to identify the optimal configuration that minimizes the information loss caused by pruning.

#### 4.2.2 Knowledge Distillation via Feature Alignment

Knowledge distillation (KD) is a widely adopted technique for transferring knowledge from a teacher model to a student model.
Therefore, it is an effective strategy for preserving the performance of the pruned model.
However, due to varying pruning schemes, the pruned student model may have different feature widths compared to the teacher model, which poses challenges for traditional distillation.
Inspired by Yu et al. (2025), we employ a trainable affine transformation to align the features between the teacher and the student model.
Thus, distillation is then performed using the aligned features.
This process is formally defined in [Eq.˜5](https://arxiv.org/html/2507.13343v2#S4.E5), where $y_{\text{t}_{i}}$ and $y_{\text{s}_{i}}$ represent the output features of $i^{\text{th}}$ DiT block group for the teacher model and the student model respectively:

$$ $\mathcal{L}_{distill}=\frac{1}{N}\sum^{N}_{i=1}\text{sim}(y_{\text{t}_{i}},\mathcal{F}_{i}(y_{\text{s}_{i}};\Theta_{i}));$ (5) $$

Here, $N$ denotes the number of DiT block groups, and $\text{sim}(\cdot,\cdot)$ is a similarity alignment function used to match the feature distributions between teacher and student.
The function $\mathcal{F}_{i}(\cdot;\Theta_{i})$ is an affine transformation parameterized by $\Theta_{i}$, introduced to align the dimensionality of the student features with that of the teacher.
The overall training loss is formulated as [Eq.˜6](https://arxiv.org/html/2507.13343v2#S4.E6), where $\mathcal{L}_{\text{flow-matching}}$ is the conditional flow-matching objective from [Eq.˜2](https://arxiv.org/html/2507.13343v2#S3.E2), and $\alpha$ is a hyper-parameter to adjust the weight of distillation.
$\alpha$ is set to $0.01$ in our experiments.

$$ $\mathcal{L}=\mathcal{L}_{\text{fm}}+\alpha\mathcal{L}_{\text{distill}}$ (6) $$

#### 4.2.3 Integration with Hardware-aware Objective

Here, we specify the details of our tri-level pruning scheme for constructing an efficient Diffusion Transformer architecture tailored to the iPhone 16 Pro Max.
Due to the memory limitation of the device, the total number of parameters must remain under 1 billion.
Based on the sensitivity analysis in [Figure˜3](https://arxiv.org/html/2507.13343v2#S4.F3), which indicates that FFN contributes more significantly to performance than attention heads, we prioritize pruning attention heads more aggressively.
Starting from a 2B parameter base model with 28 DiT blocks, 32 attention-heads, and FFN dimension of 8192, our final efficient model archives 915M parameters by pruning 8 blocks, 12 attention heads, and reducing the FFN dimension by 25% following [Algorithm˜1](https://arxiv.org/html/2507.13343v2#alg1).
More details can be found in [Appendix˜C](https://arxiv.org/html/2507.13343v2#A3).

### 4.3 Adversarial Fine-tuning for Step Distillation

We adopt adversarial fine-tuning of step-distillation,
following  Wu et al. (2025b), with a generator generator $\mathcal{G}_{\theta}(t,\mathbf{x}_{t})$ and a discriminator $\mathcal{D}\phi(t,\mathbf{x}_{t})$.
The generator $\mathcal{G}_{\theta}$ is initialized with the pretrained DiT denoiser weights.
The discriminator $\mathcal{D}\phi$ inherits the DiT backbone, whose first $K$ blocks are initialized from $\mathcal{G}_{\theta}$ and frozen to serve as a timestep-conditioned feature parser, while the subsequent DiT block is learnable and include 3D self-attention and cross-attention to enhance spatio-temporal capacity.
An MLP head with SiLU activation is appended to produce real/fake logits as shown in [Figure˜2](https://arxiv.org/html/2507.13343v2#S3.F2).
The generator $\mathcal{G}_{\theta}$ learns to generate clean samples in a few steps (*i.e*. 4-step), while the discriminator $\mathcal{D}\phi$ distinguishes real/generated samples, see [Appendix˜D](https://arxiv.org/html/2507.13343v2#A4) for details.
The adversarial fine-tuning reduces diffusion sampling budget by up to $20\times$ comparing to full-step diffusion.

### 4.4 Tiled GEMM for Efficient FFN Inference

In Transformers, the Feed-Forward Network (FFN) is a token-wise two layer MLP applied after the attention mechanism.
It expands the channel dimension from $d$ to $Nd$ (typically $N{\in}[2,4]$) and projects back to $d$ via a nonlinearity activation layer (*e.g*. SiLU).
The design increases the expressive capacity of the token-wise mapping, raising the effective rank and refining token-level representation, while attention primarily mixes information across tokens.

While the expansion in FFN ($d\rightarrow Nd\rightarrow d$) improves quality, the large General Matrix to Matrix Multiplications (GEMMs) become a memory bottleneck on mobile devices.
Although our KD-guided pruning ([Sec.˜4.2](https://arxiv.org/html/2507.13343v2#S4.SS2)) and adversarial step distillation ([Sec.˜4.3](https://arxiv.org/html/2507.13343v2#S4.SS3)) reduce overall complexity, GEMMs in FFN remain limited by the device’s memory bandwidth.
Addressing this bottleneck with conventional kernel-level optimization is infeasible as the deployment compiler, Apple’s CoreML, is a closed-source tool.
Therefore, we introduce an operator-level tiled GEMM strategy for the $d\rightarrow Nd\rightarrow d$ projections as illustrated in [Figure˜5](https://arxiv.org/html/2507.13343v2#A5.F5).
This method partitions the weight matrix along the expansion dimension $Nd$ into smaller, cache-friendly tiles, and activation are fused within each tile to reduce extra reads/writes.
This design mitigates I/O bottleneck, improving cache usage and alleviating bandwidth pressure, particularly for large feature dimensions such as $d{=}8192$.

We benchmark the latency of a fixed number of input tokens ($L{=}2048$) and $N{=}4$ while varying $d$, comparing a naïve implementation to the tiled GEMM as shown in [Figure˜6](https://arxiv.org/html/2507.13343v2#A5.F6).
For reference we also plot the theoretical scaling estimated from FLOPs, $4LNd^{2}$.
As $d$ increases, the naïve GEMM’s latency grows faster than the theoretical baseline, indicating a memory-bound issue, whereas the tiled GEMM remains close to the theoretical baseline, demonstrating reduced memory traffic.

The tiled-GEMM in FFN yields an $\sim$10% end-to-end DiT forward speedup on-device, without fine-tuning and complementary to the KD-guided pruning and step-distillation benefit.

## 5 Experiments

Training. We train on both curated real-world image/video data and synthetic data.
We use $128$ NVIDIA A100 80GB GPUs for DiT training, using AdamW optimizer with $5e-5$ learning rate and betas values as $\left[0.9,0.999\right]$.
We build our Diffusion Transformer following public models (Yang et al., 2024b; HaCohen et al., 2024b), and incorporate QK normalizations and Rotary Positional Embeddings (RoPE) (Su et al., 2024).
The T5 text encoder (Raffel et al., 2020) is employed to capture textual information.
The training is conducted using low resolution image and video for pretraining and then finetuning with high resolution data.
More training details in [Appendix˜E](https://arxiv.org/html/2507.13343v2#A5).

Adversarial Fine-tuning is conducted for $20K$ iterations on $64$ A100 GPUs, using the AdamW optimizer with a learning rate of $1e-6$ for the generator (*i.e*., DiT) and $1e-4$ for the discriminator heads.
We set the betas as $\left(0.9,0.999\right)$ for both the generator and the discriminator optimizers. We set the EMA rate as $0.95$ following Zhang et al. (2024). Additional details are reported in [Appendix˜E](https://arxiv.org/html/2507.13343v2#A5).

Evaluation and Deployment. Our models are evaluated following the standard Vbench (Huang et al., 2024) setting, that is, we generate 5 videos for each prompt, and test the scores over the 1K prompt set.
Both server and mobile-deployed models are step-distilled and evaluated with 4-step generation.
The server model generates $121$-frame horizontal videos at a resolution of $576\times 1024$, without classifier-free guidance.
The generated video is saved at 5 seconds 24 FPS for score testing and qualitative visualization.
We use different seeds and find the $\Delta\text{VBench score}$ is lower than $\pm 0.2$.
For mobile demo, we generate $49\times 384\times 512$ videos on iPhone 16 Pro Max using CoremlTools (Apple Inc., 2024) under half precision.
We employ the CLIP text encoder for on-device text encoding efficiency, while the T5 encoder is utilized for the server-side model.
Additional details are in [Appendix˜K](https://arxiv.org/html/2507.13343v2#A11).
We measure the latency by 50 runs and take the median.

### 5.1 Qualitative Results

We visualize our generated videos in [Figure˜4](https://arxiv.org/html/2507.13343v2#S5.F4). Our model consistently produces high-quality video frames and smooth object movements.
To demonstrate the generic text-to-video generation ability, we show various generation examples, including human, animal, photorealistic, and art-styled scenes.
We include more video visualizations and the efficient model demo in *supplementary material*.

Figure: Figure 4: Video generated by our efficient diffusion transformer.
Refer to caption: x4.png

**Table 2: VBench (Huang et al., 2024) comparison with popular open-source Diffusion Transformer video generation models. Scores for open-source models are collected from the [VBench Leaderboard](https://huggingface.co/spaces/Vchitect/VBench_Leaderboard).**
| Model | Params (B) | Total | Quality | Semantic | Flickering | Aesthetics | Imaging | Obj. Class | Scene | Consistency |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| Wan2.1 | 14 | 84.70 | 85.64 | 80.95 | 99.53 | 61.53 | 67.28 | 94.24 | 53.67 | 27.44 |
| Wan2.1 | 1.3 | 83.31 | 85.23 | 75.65 | 99.55 | 65.46 | 67.01 | 88.81 | 41.96 | 25.50 |
| Open-Sora-2.0 | 11 | 84.34 | 85.40 | 80.12 | 99.40 | 64.39 | 65.66 | 94.50 | 52.71 | 27.50 |
| Open-Sora-1.2 | 1.2 | 79.76 | 81.35 | 73.39 | 99.53 | 56.85 | 63.34 | 82.22 | 42.44 | 26.85 |
| Hunyuan | 13 | 83.24 | 85.09 | 75.82 | 99.44 | 60.36 | 67.56 | 86.10 | 53.88 | 26.44 |
| CogVideoX1.5 | 5 | 82.01 | 82.72 | 79.17 | 98.53 | 62.07 | 65.34 | 83.42 | 53.28 | 27.42 |
| CogVideoX | 5 | 81.91 | 83.05 | 77.33 | 78.97 | 61.88 | 63.33 | 85.07 | 51.96 | 27.65 |
| CogVideoX | 2 | 81.55 | 82.48 | 77.81 | 98.85 | 61.07 | 62.37 | 86.48 | 50.04 | 27.33 |
| Step-Video | 30 | 81.83 | 84.46 | 71.28 | 99.40 | 61.23 | 70.63 | 80.56 | 24.38 | 27.12 |
| Mochi-1 | 10 | 80.13 | 82.64 | 70.08 | 99.40 | 56.94 | 60.64 | 86.51 | 36.99 | 25.15 |
| LTX-Video | 1.8 | 80.00 | 82.30 | 70.79 | 99.34 | 59.81 | 60.28 | 83.45 | 51.07 | 25.19 |
| Ours-Server | 2.0 | 83.09 | 84.65 | 76.86 | 98.74 | 64.72 | 65.85 | 90.57 | 52.76 | 27.28 |
| Ours-Mobile | 0.9 | 81.45 | 83.12 | 74.76 | 98.11 | 64.16 | 63.41 | 92.26 | 51.06 | 25.51 |

### 5.2 Quantitative Benchmark

We evaluate our method on VBench (Huang et al., 2024) and compare it against recent state-of-the-art DiT-based video generation models, as shown in [Tab.˜2](https://arxiv.org/html/2507.13343v2#S5.T2).
Although our model is compact and optimized for fast inference on mobile devices,
it achieves a higher total score than several strong baselines, including the OpenSora-1.2, CogVideoX-2B (Yang et al., 2024b), LTX-Video (HaCohen et al., 2024b).
Compared to current open-source SOTA, Wan2.1-1.3B (Wan et al., 2025), our server variant achieves comparable quality while delivering faster inference speed per sampling step.
Importantly, the mobile deployment  (0.9B parameters) maintains competitive scores relative to larger models while running efficiently on the iPhone 16 Pro Max.
These results highlight the effectiveness of our DiT pruning and distillation method.
Human evaluation studies further demonstrate the perceptual quality of our models as reported in [Appendix˜A](https://arxiv.org/html/2507.13343v2#A1).
Comparison between our mobile variant and other mobile efficient methods are shown in [Appendix˜B](https://arxiv.org/html/2507.13343v2#A2).

### 5.3 Ablation Study

Scaling VAE Compression Ratio.
In [Tab.˜3](https://arxiv.org/html/2507.13343v2#S5.T3), we scale the VAE compression ratio and compare DiT generation speed and video quality.
For a fair comparison, we train a 2B-parameter DiT with each VAE and measure per-step generation speed by testing one denoising step on the Nvidia A100 GPU at $121\times 576\times 1024$ resolution.
We observe that though lower compression VAEs ($4\times 16\times 16$) can achieve better reconstruction PSNR, the generation speed is slower by magnitudes.
On the other hand, aggressive compression ($8\times 64\times 64$) results in poor reconstruction, and will negatively impact generation quality (*i.e*. VBench scores).
We find that ($8\times 32\times 32$) hits a balance between speed and quality, and employ this configuration for our Diffusion Transformer.
The training details and more experiment results for the video VAE can be found in [Appendix˜G](https://arxiv.org/html/2507.13343v2#A7) and [H](https://arxiv.org/html/2507.13343v2#A8).

**Table 3: Scaling VAE compression ratio. VAE PSNR is measured on DAVIS (Perazzi et al., 2016) with $33\times 512\times 512$ resolution. Latencies are for one denoising step. VBench scores are provided.**
| VAE | Diffusion Transformer |  |  |  |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- | --- |
| Compression Factor | PSNR | Latency (ms) | Total | Quality | Semantic | Aesthetic | Consistency | Flickering |
| $4\times 16\times 16$ | 33.1 | 7900 | 80.35 | 82.05 | 73.54 | 64.45 | 26.80 | 98.59 |
| $4\times 32\times 32$ | 30.9 | 920 | 79.95 | 82.99 | 67.83 | 61.52 | 27.07 | 97.46 |
| $8\times 32\times 32$ | 30.6 | 380 | 79.80 | 82.59 | 68.66 | 61.80 | 27.17 | 97.70 |
| $8\times 64\times 64$ | 28.2 | 90 | 78.40 | 81.79 | 64.86 | 55.29 | 26.11 | 97.52 |

Impact of KD-Guided Training.
We evaluate the effect of the proposed KD-guided training via ablation with and without distillation.
As shown in [Sec.˜5.3](https://arxiv.org/html/2507.13343v2#S5.SS3), the tri-level trained with the proposed distillation objective consistently outperforms the counterpart without distillation, indicating that it helps recover capacity lost due to pruning.

Impact of Tri-Level Pruning.
We compare the proposed tri-level pruning against random masking and block-only pruning (*shallow*) following Fang et al. (2024); Xie et al..
All variants are initialized from the same pretrained 2B DiT.
As shown in [Sec.˜5.3](https://arxiv.org/html/2507.13343v2#S5.SS3), the tri-level pruned model consistently outperforms both random-masked and *shallow* baseline in Quality and Total scores.
These results indicate the proposed tri-level pruning can effectively remove redundant components while preserving capacity, thereby achieving pruning with minimum performance degradation.

Comparison with Compact Model Trained from Scratch.
To separate the effect of tri-level pruning and KD-guided training from model size, we train a compact DiT the same model configuration from scratch.
As reported in [Sec.˜5.3](https://arxiv.org/html/2507.13343v2#S5.SS3), this compact baseline underperforms the KD-guided tri-level pruned model by large margin across Quality, Semantic, and Total scores.
These results indicate that training a small DiT from scratch is suboptimal, while starting from a larger teacher and applying tri-level pruning with knowledge distillation is crucial for preserving overall quality under mobile constraints.

Table: Table 4: Ablation study on tri-level pruning schemes and fine-tuning using proposed knowledge distillation.

| Method | KD | Params(M) | Quality | Semantic | Total |
| --- | --- | --- | --- | --- | --- |
| tri-level | ✓ | 915 | 83.12 | 74.76 | 81.45 |
| tri-level | ✗ | 915 | 82.19 | 66.23 | 79.00 |
| random | ✗ | 915 | 82.01 | 65.01 | 78.68 |
| *shallow* | ✗ | 932 | 81.63 | 67.15 | 78.73 |
| compact^1 | ✗ | 915 | 79.23 | 63.94 | 76.17 |
| ^1 Train from scratch |  |  |  |  |  |

Table: Table 5: Ablation study on different discriminator head design. The evaluation is conducted with a 4-step generation without classifier-free guidance.

| Head | #Steps | Quality | Semantic | Total |
| --- | --- | --- | --- | --- |
| DiT block + MLP | 4 | 83.81 | 72.89 | 81.63 |
| ResBlock-2D + Temporal-Attn | 4 | 83.24 | 67.78 | 80.14 |
| Lightweight ResBlock | 4 | 80.05 | 66.01 | 77.24 |

Full Guidance Adversarial Distillation.
We validate the proposed discriminator tailored for the DiT denoiser through an ablation study on the prediction head.
The experiments are conducted using our pre-trained 2B parameter DiT model with various discriminator head designs.
We compare our design with spatial-temporal heads introduced in Wu et al. (2025b) and the lightweight ResBlock head proposed in Wang et al. (2024a).
Since the discriminator head in Wang et al. (2024a) was originally designed for the text-to-image model, we extend it to a Conv3D variant in the ablation.
We show that the proposed prediction head (a transformer block followed by an MLP classifier) achieves best 4-step generation performance, with notable gains in semantic scores.
We attribute this to improved alignment between the text condition and the hidden states.

## 6 Conclusion

In this work, we present an efficient video generation framework that significantly accelerates Diffusion Transformers, making efficient synthesis feasible on mobile devices.
By combining a high-compression VAE, latency- and sensitivity-aware pruning, and adversarial step distillation, we successfully deploy DiT video generator to iPhone and reduce inference to just four steps while maintaining high visual quality.
Our pipeline achieves over $15$ FPS generation speed (generate 49-frame within 4 seconds) on an iPhone 16 Pro Max, demonstrating the practical viability of DiT-based video generation on edge devices.
We discuss limitations and broader impact in [Appendix˜N](https://arxiv.org/html/2507.13343v2#A14).

## References

- Agarwal et al. (2025)
Niket Agarwal, Arslan Ali, Maciej Bala, Yogesh Balaji, Erik Barker, Tiffany Cai, Prithvijit Chattopadhyay, Yongxin Chen, Yin Cui, Yifan Ding, et al.
Cosmos world foundation model platform for physical ai.
*arXiv preprint arXiv:2501.03575*, 2025.
- Apple Inc. (2024)
Apple Inc.
Core ML Tools.
[https://coremltools.readme.io/](https://coremltools.readme.io/), 2024.
Version 8.0, accessed on September 7, 2025.
- Blattmann et al. (2023)
Andreas Blattmann, Tim Dockhorn, Sumith Kulal, Daniel Mendelevitch, Maciej Kilian, Dominik Lorenz, Yam Levi, Zion English, Vikram Voleti, Adam Letts, et al.
Stable video diffusion: Scaling latent video diffusion models to large datasets.
*arXiv preprint arXiv:2311.15127*, 2023.
- Chen et al. (2024)
Tsai-Shien Chen, Aliaksandr Siarohin, Willi Menapace, Ekaterina Deyneka, Hsiang-wei Chao, Byung Eun Jeon, Yuwei Fang, Hsin-Ying Lee, Jian Ren, Ming-Hsuan Yang, and Sergey Tulyakov.
Panda-70m: Captioning 70m videos with multiple cross-modality teachers.
In *Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition*, 2024.
- Dao et al. (2025)
Trung Dao, Thuan Hoang Nguyen, Thanh Le, Duc Vu, Khoi Nguyen, Cuong Pham, and Anh Tran.
Swiftbrush v2: Make your one-step diffusion model better than its teacher.
In *European Conference on Computer Vision*, pp. 176–192. Springer, 2025.
- Esser et al. (2024)
Patrick Esser, Sumith Kulal, Andreas Blattmann, Rahim Entezari, Jonas Müller, Harry Saini, Yam Levi, Dominik Lorenz, Axel Sauer, Frederic Boesel, et al.
Scaling rectified flow transformers for high-resolution image synthesis.
In *Forty-first International Conference on Machine Learning*, 2024.
- Fang et al. (2024)
Gongfan Fang, Kunjun Li, Xinyin Ma, and Xinchao Wang.
Tinyfusion: Diffusion transformers learned shallow.
*arXiv preprint arXiv:2412.01199*, 2024.
- Frans et al. (2024)
Kevin Frans, Danijar Hafner, Sergey Levine, and Pieter Abbeel.
One step diffusion via shortcut models.
2024.
URL [https://arxiv.org/abs/2410.12557](https://arxiv.org/abs/2410.12557).
- Geng et al. (2025)
Zhengyang Geng, Mingyang Deng, Xingjian Bai, J Zico Kolter, and Kaiming He.
Mean flows for one-step generative modeling.
*arXiv preprint arXiv:2505.13447*, 2025.
- HaCohen et al. (2024a)
Yoav HaCohen, Nisan Chiprut, Benny Brazowski, Daniel Shalem, Dudu Moshe, Eitan Richardson, Eran Levin, Guy Shiran, Nir Zabari, Ori Gordon, Poriya Panet, Sapir Weissbuch, Victor Kulikov, Yaki Bitterman, Zeev Melumian, and Ofir Bibi.
Ltx-video: Realtime video latent diffusion.
2024a.
URL [https://arxiv.org/abs/2501.00103](https://arxiv.org/abs/2501.00103).
- HaCohen et al. (2024b)
Yoav HaCohen, Nisan Chiprut, Benny Brazowski, Daniel Shalem, Dudu Moshe, Eitan Richardson, Eran Levin, Guy Shiran, Nir Zabari, Ori Gordon, et al.
Ltx-video: Realtime video latent diffusion.
*arXiv preprint arXiv:2501.00103*, 2024b.
- Ho et al. (2020)
Jonathan Ho, Ajay Jain, and Pieter Abbeel.
Denoising diffusion probabilistic models.
In *Advances in Neural Information Processing Systems (NeurIPS)*, 2020.
- Ho et al. (2022)
Jonathan Ho, William Chan, Chitwan Saharia, Jay Whang, Ruiqi Gao, Alexey Gritsenko, Diederik P Kingma, Ben Poole, Mohammad Norouzi, David J Fleet, et al.
Imagen video: High definition video generation with diffusion models.
*arXiv preprint arXiv:2210.02303*, 2022.
- Hoogeboom et al. (2023)
Emiel Hoogeboom, Jonathan Heek, and Tim Salimans.
simple diffusion: End-to-end diffusion for high resolution images.
*arXiv preprint arXiv:2301.11093*, 2023.
- Hu et al. (2024)
Dongting Hu, Jierun Chen, Xijie Huang, Huseyin Coskun, Arpit Sahni, Aarush Gupta, Anujraaj Goyal, Dishani Lahiri, Rajesh Singh, Yerlan Idelbayev, Junli Cao, Yanyu Li, Kwang-Ting Cheng, S.-H. Chan, Mingming Gong, Sergey Tulyakov, Anil Kag, Yanwu Xu, and Jian Ren.
Snapgen: Taming high-resolution text-to-image models for mobile devices with efficient architectures and training.
*arXiv:2412.09619 [cs.CV]*, 2024.
- Huang et al. (2024)
Ziqi Huang, Yinan He, Jiashuo Yu, Fan Zhang, Chenyang Si, Yuming Jiang, Yuanhan Zhang, Tianxing Wu, Qingyang Jin, Nattapol Chanpaisit, Yaohui Wang, Xinyuan Chen, Limin Wang, Dahua Lin, Yu Qiao, and Ziwei Liu.
VBench: Comprehensive benchmark suite for video generative models.
In *Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition*, 2024.
- Kim et al. (2024)
Beomsu Kim, Yu-Guan Hsieh, Michal Klein, Marco Cuturi, Jong Chul Ye, Bahjat Kawar, and James Thornton.
Simple reflow: Improved techniques for fast flow models.
*ArXiv preprint*, abs/2410.07815, 2024.
URL [https://arxiv.org/abs/2410.07815](https://arxiv.org/abs/2410.07815).
- Kim et al. (2025)
Bosung Kim, Kyuhwan Lee, Isu Jeong, Jungmin Cheon, Yeojin Lee, and Seulki Lee.
On-device sora: Enabling training-free diffusion-based text-to-video generation for mobile devices.
2025.
URL [https://arxiv.org/abs/2502.04363](https://arxiv.org/abs/2502.04363).
- Kong et al. (2024)
Weijie Kong, Qi Tian, Zijian Zhang, Rox Min, Zuozhuo Dai, Jin Zhou, Jiangfeng Xiong, Xin Li, Bo Wu, Jianwei Zhang, et al.
Hunyuanvideo: A systematic framework for large video generative models.
*arXiv preprint arXiv:2412.03603*, 2024.
- Kuaishou (2024)
Kuaishou.
Kling.
[https://kling.kuaishou.com/en](https://kling.kuaishou.com/en), 2024.
- Li et al. (2023)
Yanyu Li, Huan Wang, Qing Jin, Ju Hu, Pavlo Chemerys, Yun Fu, Yanzhi Wang, Sergey Tulyakov, and Jian Ren.
Snapfusion: Text-to-image diffusion model on mobile devices within two seconds.
In *Advances in Neural Information Processing Systems 36: Annual Conference on Neural Information Processing Systems 2023, NeurIPS 2023, New Orleans, LA, USA, December 10 - 16, 2023*, 2023.
URL [http://papers.nips.cc/paper_files/paper/2023/hash/41bcc9d3bddd9c90e1f44b29e26d97ff-Abstract-Conference.html](http://papers.nips.cc/paper_files/paper/2023/hash/41bcc9d3bddd9c90e1f44b29e26d97ff-Abstract-Conference.html).
- Lim & Ye (2017)
Jae Hyun Lim and Jong Chul Ye.
Geometric gan.
*ArXiv preprint*, abs/1705.02894, 2017.
URL [https://arxiv.org/abs/1705.02894](https://arxiv.org/abs/1705.02894).
- Lin et al. (2024)
Bin Lin, Yunyang Ge, Xinhua Cheng, Zongjian Li, Bin Zhu, Shaodong Wang, Xianyi He, Yang Ye, Shenghai Yuan, Liuhan Chen, et al.
Open-sora plan: Open-source large video generation model.
*arXiv preprint arXiv:2412.00131*, 2024.
- Liu et al. (2022)
Xingchao Liu, Chengyue Gong, and Qiang Liu.
Flow straight and fast: Learning to generate and transfer data with rectified flow.
*arXiv preprint arXiv:2209.03003*, 2022.
- Ma et al. (2025)
Guoqing Ma, Haoyang Huang, Kun Yan, Liangyu Chen, Nan Duan, Shengming Yin, Changyi Wan, Ranchen Ming, Xiaoniu Song, Xing Chen, et al.
Step-video-t2v technical report: The practice, challenges, and future of video foundation model.
*arXiv preprint arXiv:2502.10248*, 2025.
- Mei et al. (2024)
Kangfu Mei, Mauricio Delbracio, Hossein Talebi, Zhengzhong Tu, Vishal M Patel, and Peyman Milanfar.
Codi: Conditional diffusion distillation for higher-fidelity and faster image generation.
In *Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition*, pp. 9048–9058, 2024.
- Menapace et al. (2024)
Willi Menapace, Aliaksandr Siarohin, Ivan Skorokhodov, Ekaterina Deyneka, Tsai-Shien Chen, Anil Kag, Yuwei Fang, Aleksei Stoliar, Elisa Ricci, Jian Ren, et al.
Snap video: Scaled spatiotemporal transformers for text-to-video synthesis.
*arXiv preprint arXiv:2402.14797*, 2024.
- OpenAI (2023)
OpenAI.
Video generation models as world simulators.
[https://openai.com/index/video-generation-models-as-world-simulators/](https://openai.com/index/video-generation-models-as-world-simulators/), 2023.
- Perazzi et al. (2016)
F. Perazzi, J. Pont-Tuset, B. McWilliams, L. Van Gool, M. Gross, and A. Sorkine-Hornung.
A benchmark dataset and evaluation methodology for video object segmentation.
In *2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)*, pp. 724–732, 2016.
doi: 10.1109/CVPR.2016.85.
- Podell et al. (2023)
Dustin Podell, Zion English, Kyle Lacey, Andreas Blattmann, Tim Dockhorn, Jonas Müller, Joe Penna, and Robin Rombach.
Sdxl: Improving latent diffusion models for high-resolution image synthesis.
*arXiv preprint arXiv:2307.01952*, 2023.
- Polyak et al. (2024)
Adam Polyak, Amit Zohar, Andrew Brown, Andros Tjandra, Animesh Sinha, Ann Lee, Apoorv Vyas, Bowen Shi, Chih-Yao Ma, Ching-Yao Chuang, et al.
Movie gen: A cast of media foundation models.
*arXiv preprint arXiv:2410.13720*, 2024.
- Raffel et al. (2020)
Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J Liu.
Exploring the limits of transfer learning with a unified text-to-text transformer.
*The Journal of Machine Learning Research*, 21(1):5485–5551, 2020.
- Rombach et al. (2022)
Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer.
High-resolution image synthesis with latent diffusion models.
In *Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition*, pp. 10684–10695, 2022.
- Salimans & Ho (2022)
Tim Salimans and Jonathan Ho.
Progressive distillation for fast sampling of diffusion models.
In *The Tenth International Conference on Learning Representations, ICLR 2022, Virtual Event, April 25-29, 2022*. OpenReview.net, 2022.
URL [https://openreview.net/forum?id=TIdIXIpzhoI](https://openreview.net/forum?id=TIdIXIpzhoI).
- Sauer et al. (2021)
Axel Sauer, Kashyap Chitta, Jens Müller, and Andreas Geiger.
Projected gans converge faster.
In Marc’Aurelio Ranzato, Alina Beygelzimer, Yann N. Dauphin, Percy Liang, and Jennifer Wortman Vaughan (eds.), *Advances in Neural Information Processing Systems 34: Annual Conference on Neural Information Processing Systems 2021, NeurIPS 2021, December 6-14, 2021, virtual*, pp. 17480–17492, 2021.
URL [https://proceedings.neurips.cc/paper/2021/hash/9219adc5c42107c4911e249155320648-Abstract.html](https://proceedings.neurips.cc/paper/2021/hash/9219adc5c42107c4911e249155320648-Abstract.html).
- Sauer et al. (2023a)
Axel Sauer, Tero Karras, Samuli Laine, Andreas Geiger, and Timo Aila.
Stylegan-t: Unlocking the power of gans for fast large-scale text-to-image synthesis.
In Andreas Krause, Emma Brunskill, Kyunghyun Cho, Barbara Engelhardt, Sivan Sabato, and Jonathan Scarlett (eds.), *International Conference on Machine Learning, ICML 2023, 23-29 July 2023, Honolulu, Hawaii, USA*, volume 202 of *Proceedings of Machine Learning Research*, pp. 30105–30118. PMLR, 2023a.
URL [https://proceedings.mlr.press/v202/sauer23a.html](https://proceedings.mlr.press/v202/sauer23a.html).
- Sauer et al. (2023b)
Axel Sauer, Dominik Lorenz, Andreas Blattmann, and Robin Rombach.
Adversarial diffusion distillation.
*ArXiv preprint*, abs/2311.17042, 2023b.
URL [https://arxiv.org/abs/2311.17042](https://arxiv.org/abs/2311.17042).
- Sauer et al. (2024)
Axel Sauer, Frederic Boesel, Tim Dockhorn, Andreas Blattmann, Patrick Esser, and Robin Rombach.
Fast high-resolution image synthesis with latent adversarial diffusion distillation.
*ArXiv preprint*, abs/2403.12015, 2024.
URL [https://arxiv.org/abs/2403.12015](https://arxiv.org/abs/2403.12015).
- Song & Dhariwal (2023)
Yang Song and Prafulla Dhariwal.
Improved techniques for training consistency models.
*ArXiv preprint*, abs/2310.14189, 2023.
URL [https://arxiv.org/abs/2310.14189](https://arxiv.org/abs/2310.14189).
- Song et al. (2023)
Yang Song, Prafulla Dhariwal, Mark Chen, and Ilya Sutskever.
Consistency models.
In Andreas Krause, Emma Brunskill, Kyunghyun Cho, Barbara Engelhardt, Sivan Sabato, and Jonathan Scarlett (eds.), *International Conference on Machine Learning, ICML 2023, 23-29 July 2023, Honolulu, Hawaii, USA*, volume 202 of *Proceedings of Machine Learning Research*, pp. 32211–32252. PMLR, 2023.
URL [https://proceedings.mlr.press/v202/song23a.html](https://proceedings.mlr.press/v202/song23a.html).
- Su et al. (2024)
Jianlin Su, Murtadha Ahmed, Yu Lu, Shengfeng Pan, Wen Bo, and Yunfeng Liu.
Roformer: Enhanced transformer with rotary position embedding.
*Neurocomputing*, 568:127063, 2024.
- Team (2024)
Genmo Team.
Mochi, 2024.
- Wan et al. (2025)
Team Wan, Ang Wang, Baole Ai, Bin Wen, Chaojie Mao, Chen-Wei Xie, Di Chen, Feiwu Yu, Haiming Zhao, Jianxiao Yang, Jianyuan Zeng, Jiayu Wang, Jingfeng Zhang, Jingren Zhou, Jinkai Wang, Jixuan Chen, Kai Zhu, Kang Zhao, Keyu Yan, Lianghua Huang, Mengyang Feng, Ningyi Zhang, Pandeng Li, Pingyu Wu, Ruihang Chu, Ruili Feng, Shiwei Zhang, Siyang Sun, Tao Fang, Tianxing Wang, Tianyi Gui, Tingyu Weng, Tong Shen, Wei Lin, Wei Wang, Wei Wang, Wenmeng Zhou, Wente Wang, Wenting Shen, Wenyuan Yu, Xianzhong Shi, Xiaoming Huang, Xin Xu, Yan Kou, Yangyu Lv, Yifei Li, Yijing Liu, Yiming Wang, Yingya Zhang, Yitong Huang, Yong Li, You Wu, Yu Liu, Yulin Pan, Yun Zheng, Yuntao Hong, Yupeng Shi, Yutong Feng, Zeyinzi Jiang, Zhen Han, Zhi-Fan Wu, and Ziyu Liu.
Wan: Open and advanced large-scale video generative models.
2025.
URL [https://arxiv.org/abs/2503.20314](https://arxiv.org/abs/2503.20314).
- Wang et al. (2024a)
Fu-Yun Wang, Zhaoyang Huang, Alexander William Bergman, Dazhong Shen, Peng Gao, Michael Lingelbach, Keqiang Sun, Weikang Bian, Guanglu Song, Yu Liu, et al.
Phased consistency model.
*arXiv preprint arXiv:2405.18407*, 2024a.
- Wang et al. (2024b)
Fu-Yun Wang, Ling Yang, Zhaoyang Huang, Mengdi Wang, and Hongsheng Li.
Rectified diffusion: Straightness is not your need in rectified flow.
*ArXiv preprint*, abs/2410.07303, 2024b.
URL [https://arxiv.org/abs/2410.07303](https://arxiv.org/abs/2410.07303).
- Wu et al. (2025a)
Yushu Wu, Yanyu Li, Ivan Skorokhodov, Anil Kag, Willi Menapace, Sharath Girish, Aliaksandr Siarohin, Yanzhi Wang, and Sergey Tulyakov.
H3ae: High compression, high speed, and high quality autoencoder for video diffusion models.
*arXiv preprint arXiv:2504.10567*, 2025a.
- Wu et al. (2025b)
Yushu Wu, Zhixing Zhang, Yanyu Li, Yanwu Xu, Anil Kag, Yang Sui, Huseyin Coskun, Ke Ma, Aleksei Lebedev, Ju Hu, et al.
Snapgen-v: Generating a five-second video within five seconds on a mobile device.
In *Proceedings of the Computer Vision and Pattern Recognition Conference*, pp. 2479–2490, 2025b.
- (48)
Enze Xie, Junsong Chen, Yuyang Zhao, Jincheng YU, Ligeng Zhu, Yujun Lin, Zhekai Zhang, Muyang Li, Junyu Chen, Han Cai, et al.
Sana 1.5: Efficient scaling of training-time and inference-time compute in linear diffusion transformer.
In *Forty-second International Conference on Machine Learning*.
- Xu et al. (2024)
Yanwu Xu, Yang Zhao, Zhisheng Xiao, and Tingbo Hou.
Ufogen: You forward once large scale text-to-image generation via diffusion gans.
In *Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition*, pp. 8196–8206, 2024.
- Yahia et al. (2024)
Haitam Ben Yahia, Denis Korzhenkov, Ioannis Lelekas, Amir Ghodrati, and Amirhossein Habibian.
Mobile video diffusion.
2024.
URL [https://arxiv.org/abs/2412.07583](https://arxiv.org/abs/2412.07583).
- Yang et al. (2024a)
Ling Yang, Zixiang Zhang, Zhilong Zhang, Xingchao Liu, Minkai Xu, Wentao Zhang, Chenlin Meng, Stefano Ermon, and Bin Cui.
Consistency flow matching: Defining straight flows with velocity consistency.
*ArXiv preprint*, abs/2407.02398, 2024a.
URL [https://arxiv.org/abs/2407.02398](https://arxiv.org/abs/2407.02398).
- Yang et al. (2024b)
Zhuoyi Yang, Jiayan Teng, Wendi Zheng, Ming Ding, Shiyu Huang, Jiazheng Xu, Yuanming Yang, Wenyi Hong, Xiaohan Zhang, Guanyu Feng, et al.
Cogvideox: Text-to-video diffusion models with an expert transformer.
*arXiv preprint arXiv:2408.06072*, 2024b.
- Yin et al. (2024a)
Tianwei Yin, Michaël Gharbi, Taesung Park, Richard Zhang, Eli Shechtman, Fredo Durand, and William T Freeman.
Improved distribution matching distillation for fast image synthesis.
*ArXiv preprint*, abs/2405.14867, 2024a.
URL [https://arxiv.org/abs/2405.14867](https://arxiv.org/abs/2405.14867).
- Yin et al. (2024b)
Tianwei Yin, Michaël Gharbi, Richard Zhang, Eli Shechtman, Fredo Durand, William T Freeman, and Taesung Park.
One-step diffusion with distribution matching distillation.
In *Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition*, pp. 6613–6623, 2024b.
- Yu et al. (2025)
Sihyun Yu, Sangkyung Kwak, Huiwon Jang, Jongheon Jeong, Jonathan Huang, Jinwoo Shin, and Saining Xie.
Representation alignment for generation: Training diffusion transformers is easier than you think.
In *International Conference on Learning Representations*, 2025.
- Zhang et al. (2024)
Zhixing Zhang, Yanyu Li, Yushu Wu, yanwu xu, Anil Kag, Ivan Skorokhodov, Willi Menapace, Aliaksandr Siarohin, Junli Cao, Dimitris N. Metaxas, Sergey Tulyakov, and Jian Ren.
SF-v: Single forward video generation model.
In *The Thirty-eighth Annual Conference on Neural Information Processing Systems*, 2024.
URL [https://openreview.net/forum?id=PVgAeMm3MW](https://openreview.net/forum?id=PVgAeMm3MW).
- Zhao et al. (2024)
Yang Zhao, Yanwu Xu, Zhisheng Xiao, Haolin Jia, and Tingbo Hou.
Mobilediffusion: Instant text-to-image generation on mobile devices.
2024.
URL [https://arxiv.org/abs/2311.16567](https://arxiv.org/abs/2311.16567).
- Zheng et al. (2024)
Zangwei Zheng, Xiangyu Peng, Tianji Yang, Chenhui Shen, Shenggui Li, Hongxin Liu, Yukun Zhou, Tianyi Li, and Yang You.
Open-sora: Democratizing efficient video production for all, 2024.
URL [https://github.com/hpcaitech/Open-Sora](https://github.com/hpcaitech/Open-Sora).

## Appendix A User Study

To evaluate the human preference across different video generation models, we conduct human evaluations comparing our model against baselines such as CogVideoX-2B, LTX-Video in [Appendix˜A](https://arxiv.org/html/2507.13343v2#A1).
We generate video clips using prompts from VBench and MovieGenBench (Polyak et al., 2024) and
ask human labelers to select the best results across prompt alignment, aesthetics, and motion quality.
The results indicate that our model significantly outperforms the baselines.

To further demonstrate the effectiveness of our proposed method, we also evaluate the human preferences across our server-side model and mobile-deployed model in [Appendix˜A](https://arxiv.org/html/2507.13343v2#A1).
We generate video samples in $49\times 512\times 384$ for both models.
The result illustrates the trade-off in visual quality in terms of efficiency.
Notably, our mobile-deployed model uses CLIP as text-encoder for efficiency while the server-side model uses T5-Encoder.

**Table 6: Human preference across different video generation models.**
| Model | Prompt Alignment | Aesthetics | Motion Quality |
| --- | --- | --- | --- |
| LTX-Video^* | 16.7% | 10.0% | 6.7% |
| CogVideoX-2B | 40.0% | 33.3% | 43.3% |
| Ours | 43.3% | 56.7% | 50.0% |
| ^*We notice the performance of LTX-Video highly depends on the prompts enhancement. |  |  |  |

## Appendix B Comparison with Other Mobile Video Generation Methods

We compare the VBench score of our model against SnapGen-V (Wu et al., 2025b) and on-device Sora (Kim et al., 2025), using benchmark metrics reported in their paper to show the performance of our mobile-deployed model as in [Tab.˜8](https://arxiv.org/html/2507.13343v2#A2.T8).
Notably, SnapGen-V is based on UNet architecture for efficient video generation and on-device Sora is a training-free method that enables open-sora (Zheng et al., 2024) on the mobile device.
Due to on-device Sora’s published precision and scale differ from the standard VBench values, we have converted them to a common scale and also include its baseline, OpenSora v1.2 as reference.
Note that our mobile model outperformed SnapGen-V by a notable margin in both Semantic Score and Total Score, and it also achieves better performance than the baseline of on-device Sora.
These results demonstrate that our approach provides competitive or superior performance compared to existing mobile video generation methods.

**Table 8: VBench (Huang et al., 2024) comparison with popular open-source Diffusion Transformer video generation models. Scores for open-source models are collected from the [VBench Leaderboard](https://huggingface.co/spaces/Vchitect/VBench_Leaderboard).**
| Model | Total | Quality | Semantic | Subject | Background | Temporal | Motion | Dynamic | Aesthetic | Imaging |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| Score | Score | Score | Consistency | Consistency | Flickering | Smoothness | Degree | Quality | Quality |  |
| Ours-Mobile | 81.45 | 83.12 | 74.76 | 95.73 | 96.64 | 98.11 | 99.23 | 58.33 | 64.16 | 63.41 |
| SnapGen-V | 81.14 | 83.47 | 71.84 | – | – | 99.37 | – | 51.11 | 62.19 | – |
| Open-Sora V1.2^* | 79.76 | 81.35 | 73.39 | 96.75 | 97.61 | 99.53 | 98.50 | 42.39 | 56.85 | 63.34 |
| On-device Sora^† | – | – | – | 96.00 | 97.00 | 99.00 | 99.00 | 27.00 | 47.00 | 53.00 |
| ^*We report the result of open-sora for reference since on-device Sora reported result precision and scale differ from the standard VBench values. |  |  |  |  |  |  |  |  |  |  |
| ^†We converted reported on-device Sora’s VBench values to common scale for comparison. |  |  |  |  |  |  |  |  |  |  |

## Appendix C Search for Optimal Pruning Configuration

We describe the detail to determine the optimal pruning configuration for on-device model as [Algorithm˜1](https://arxiv.org/html/2507.13343v2#alg1).
Starting from a pretrained DiT, our objective is to satisfy the device memory budget while minimizing quality degradation.

Search space and constraints.
We enumerate candidate configurations over three granularities:
$\text{Attn-Heads}\in\{20,24,28,32\},\text{Blocks}\in\{16,20,24,28\},\text{FFN dim}\in\{5120,6144,7168,8192\}$.
Here, values denote the *kept* attention heads per block, transformer blocks, and FFN dimension, respectively.
To reduce the search space, we ignore extreme cases (*e.g*. pruning $>50\%$ of blocks).

Sensitivity-guided pruning order.
Given by the sensitivity analysis in [Figure˜3](https://arxiv.org/html/2507.13343v2#S4.F3), we rank pruning sensitivity as:
$\text{FFN dim}>\text{Blocks}>\text{Attn-Heads}$.
Accordingly, we prefer to prune more aggressively on attention heads and conservatively on FFN dimension, with blocks in between.

For each candidate $c$, we initialize all possible binary masks and optimize them with the KD-guided tri-level pruning objectives [Eqs.˜3](https://arxiv.org/html/2507.13343v2#S4.E3), [4](https://arxiv.org/html/2507.13343v2#S4.E4) and [6](https://arxiv.org/html/2507.13343v2#S4.E6).
We employ Gumbel-Softmax sampling for differentiable mask selection.
Candidates are evaluated on a fixed validation set using the average flow-matching objective [Eq.˜2](https://arxiv.org/html/2507.13343v2#S3.E2) over multiple timesteps (*i.e*. 25-step), subject to device budget constraints.

Figure: Algorithm 1 Search for Optimal Pruning Configuration

## Appendix D Adversarial Finetuning for Step-Distillation

To obtain a $k-$step distillation procedure, we predefine the intermediate diffusion timesteps as $\mathcal{T}=\{T_{1},T_{2},\dots,T_{k}\}$ with the following ordering $T_{1}=1>T_{2}>\dots>T_{k}>0$. Typically, $k$ is set to $4$ to achieve a $4-$step diffusion model. Given a real data sample $\mathbf{X}_{0}$, we can obtain the latent $\mathbf{x}_{0}$ using the VAE. We sample two timesteps $t$ and $t^{\prime}$ uniformly at random from the set $\mathcal{T}$ such that $t^{\prime}<t$. We can construct the fake and real samples using the diffusion forward [Eq.˜1](https://arxiv.org/html/2507.13343v2#S3.E1) and velocity from the generator $\mathcal{G}_{\theta}(t,\mathbf{x}_{t})$ as follows:

$$ $\text{Fake : }\hat{\mathbf{x}}_{t^{\prime}}=\mathbf{x}_{t}+\left(t^{\prime}-t\right)\cdot\mathcal{G}_{\theta}\left(t,\mathbf{x}_{t}\right);\,\,\,\,\text{Real : }\mathbf{x}_{t^{\prime}}=\left(1-t^{\prime}\right)\mathbf{x}_{0}+t^{\prime}\epsilon;\epsilon\sim\mathcal{N}(0,I)$ (7) $$

Using the above real and fake samples, we can define the discriminator and generator losses. Below, we employ the widely used (Sauer et al., 2023b; a; 2021; Zhang et al., 2024) hinge loss (Lim & Ye, 2017) as the adversarial training objective.
The discriminator’s goal is to differentiate between real and fake samples by minimizing:

$$ $\begin{split}\mathcal{L}_{\text{adv}}^{\mathcal{D}}=&\mathbb{E}_{t^{\prime},\mathbf{x}_{0}}\left[\text{ReLU}(1+\mathcal{D}_{\phi}\left(\mathbf{x}_{t^{\prime}},t^{\prime}\right))\right]+\mathbb{E}_{t,t^{\prime},\mathbf{x}_{0}}\left[\text{ReLU}\left(1-\mathcal{D}_{\phi}\left(\hat{\mathbf{x}}_{t^{\prime}},t^{\prime}\right)\right)\right],\end{split}$ (8) $$

The adversarial objective for the generator $\mathcal{L}_{\text{adv}}^{\mathcal{G}}$ and the reconstruction objective $\mathcal{L}_{\text{recon}}$ are defined as:

$$ $\mathcal{L}_{\text{adv}}^{\mathcal{G}}=\mathbb{E}_{t,t^{\prime},x_{0}}[\mathcal{D}_{\phi}\left(\hat{\mathbf{x}}_{t^{\prime}},t^{\prime}\right)];\,\,\,\,\,\,\,\mathcal{L}_{\text{recon}}=\sqrt{\left\|\hat{\mathbf{x}}_{0}-\mathbf{x}_{0}\right\|_{2}^{2}+c^{2}}-c.$ (9) $$

where $\hat{\mathbf{x}}_{0}=\mathbf{x}_{t}-t\cdot\mathcal{G}_{\theta}\left(t,\mathbf{x}_{t}\right)$, and $c>0$ is an adjustable constant. Following Zhang et al. (2024); Hu et al. (2024), we also incorporate a reconstruction objective to enhance training stability.

## Appendix E Training Details for DiT

##### Pre-training.

The DiT training is trained on the internally collected dataset containing high-quality images and videos, which are similar to public large-scale datasets such as Chen et al. (2024).
Training is performed in two stages: (i) pretraining on low-resolution (288p) images and videos for $150K$ iterations, followed by (ii) fine-tuning for an additional $50K$ iterations on a mixed-resolution setting (288p and 576p).
The KD-guided tri-level pruning is only conducted to obtain the mobile variant after the pretraining stage.
The fine-tuning stage for the mobile variant takes $50K$ iterations.
Video clips the mobile variant uses is 49-frame clips.
All video clips are resampled to 24 fps and cropped to 5-second segments.
In the first stage, we only adopt T5 encoder as the text-encoder, leveraging its stronger capacity for modeling long caption and capturing richer text information.
During fine-tuning, we additionally incorporate the CLIP text-encoder alongside with T5, since CLIP is the text encoder deployed on-device.
Each encoder output is first projected into the DiT latent space, then the projected embeddings are concatenated and fed into the DiT as conditioning.
To improve robustness, we randomly mask either the T5 embeddings or the CLIP embeddings during training, enabling better model generalization capacity under both text-encoders.

##### KD-guided Tri-level Pruning.

The tri-level pruning procedure is initialized from the fine-tuned DiT model.
To determine the pruning ratios at each granularity (blocks, attention heads, and FFN dimensions), we consider both the hardware constrain of the iPhone 16 Pro Max  (parameter budget $<1\text{B}$) and the sensitivity analysis in [Figure˜3](https://arxiv.org/html/2507.13343v2#S4.F3) and conduct [Algorithm˜1](https://arxiv.org/html/2507.13343v2#alg1).
The optimization is run for $20K$ iterations per candidate configuration to obtain a stable pruning scheme that minimizes performance degradation.
At the end, We also perform knowledge distillation alone without pruning, where the student is distilled directly from the baseline model, to enhance the performance.

##### Adversarial Fine-tuning.

The adversarial fine-tuning is applied to both our server and mobile variants.
The generator and discriminator are initialized from the pre-trained model weights.
We use a frozen server variant model to generate with classifier-free guidance scale 5 to produce reconstruction targets.
During the fine-tuning, the hinge loss is utilized as the adversarial objective, while the reconstruction loss is defined by the $l_{2}$-Norm between the v-prediction of the generator and the frozen model.

Figure: Figure 5: Illustration for tiled GEMM for a single token. The input $X$ and weights $W$ are both tiled into $k$ partitions along input feature.
Refer to caption: x5.png

## Appendix F Implementation Details in tiled GEMM

As introduced in [Sec.˜4.4](https://arxiv.org/html/2507.13343v2#S4.SS4), Linear Layer with large input feature dimensions tend to become memory-bound.
In our implementation, we apply tiled GEMM specifically to the $Nd\rightarrow d$ projection layer in the FFN, which we find to be the dominant bandwidth bottleneck.
Other linear layers, such as QKV projections in attention, have much smaller hidden dimensions and show only marginal improvements with tiling, thus we retain their standard implementation.

In pratical, we set the number of partition to $k=4$, which we identify as a practical balance between parallelism and cache locality on the iPhone 16 Pro Max.
This operator-level strategy allows us to reduce data traffic without requiring changes to CoreML’s kernel backend, making it directly hardware friendly on mobile device.

## Appendix G Training Details for VAE

The VAE is trained on the internal dataset using 64 NVIDIA A100 80GB GPUs.
The model is optimized using the AdamW optimizer with $1e-4$ learning rate, $\beta=(0.9,0.999)$, and trained with a batch size of 16 per GPU.
The training process including two-stage: we first pretrain the model on $256\times 256$ cropped $17$-frame clips for $100K$ iterations, then fine-tuning on a range of resolutions and clip lengths for additional $50K$ iterations.

The VAE is optimized using a combination of a reconstruction loss between the input data and the reconstruction output, and KL-divergence regularization that enforces the latent distribution to follow a normal distribution, as defined in [Eq.˜10](https://arxiv.org/html/2507.13343v2#A7.E10).
The KL weight is set to $\lambda=1e-6$.

$$ $\displaystyle\mathcal{L}_{\text{recon}}$ $\displaystyle=\lVert\hat{y}-y\rVert$ (10) $\displaystyle\mathcal{L}_{\text{KL}}$ $\displaystyle=\tfrac{1}{2}\!\left[-\log(\sigma^{2})-1+\sigma^{2}+\mu^{2}\right]$ $\displaystyle\mathcal{L}$ $\displaystyle=\mathcal{L}_{\text{recon}}+\lambda\mathcal{L}_{\text{KL}}$ $$

## Appendix H More Experiments on VAE Variants

We further analyze the impact of the number of latent channel to the VAE and the corresponding DiT performance, as shown in [Appendix˜H](https://arxiv.org/html/2507.13343v2#A8).
As expected, increasing latent channels is always beneficial for VAE reconstruction quality.
However, the benefit quickly saturates, *i.e*. for higher compression VAEs, the reconstruction PSNR can not match that of low-compression VAEs simply by increasing latent channels.
Moreover, larger latent channels can harm or destabilize diffusion quality, as display in the $\left|z\right|=512$ setting.
Therefore, we select latent channel for each compression setting according to the diffusion quality (*i.e*. VBench score).

Beyond the number of latent channels, we also investigate the impact of patchify options on the overall data dimension compression strategy.
Here, we compare direct compression of video data against those use lower-compression VAEs combined with patchify operations later, as shown in [Appendix˜H](https://arxiv.org/html/2507.13343v2#A8).
The latter approach achieves higher reconstruction PSNR due to compression is applied more conservatively along the spatial and temporal dimensions.
For a fair comparison, we keep the number of latent channels same for overall compression strategy after patchify.

The reconstruction PSNRs are evaluated on DAVIS with $33\times 512\times 512$ resolution, while VBench score are evaluated following the standard setup.

Table: Table 9: Impact of VAE latent channels $\left|z\right|$.

| VAE | $\left|z\right|$ | PSNR | VBench |
| --- | --- | --- | --- |
| $4\times 16\times 16$ | 64 | 33.1 | 80.35 |
| $4\times 16\times 16$ | 128 | 33.6 | 80.33 |
| $8\times 32\times 32$ | 128 | 30.6 | 79.73 |
| $8\times 32\times 32$ | 256 | 30.8 | 79.80 |
| $8\times 64\times 64$ | 128 | 26.9 | 74.43 |
| $8\times 64\times 64$ | 256 | 28.2 | 78.40 |
| $8\times 64\times 64$ | 512 | 28.5 | 78.00 |

Table: Table 10: Impact of VAE latent channel and patchify size for different compression strategy.

| VAE | Patchify size | $f$ | $\lvert z\rvert$ | PSNR | VBench |
| --- | --- | --- | --- | --- | --- |
| $4\times 8\times 8$ | 2 | 1024 | 16 | 33.2 | 80.32 |
| $4\times 16\times 16$ | 1 | 64 | 33.1 | 80.35 |  |
| $4\times 8\times 8$ | 4 | 4096 | 16 | 33.2 | 80.08 |
| $4\times 16\times 16$ | 2 | 64 | 33.1 | 80.19 |  |
| $4\times 32\times 32$ | 1 | 256 | 30.9 | 79.95 |  |

## Appendix I Impact of Inference Steps

Our distilled model supports generation with reduced number of inference steps.
We further exhibits the impact of the inference steps for the generation results in [Tab.˜11](https://arxiv.org/html/2507.13343v2#A9.T11).

**Table 11: Ablation study on different inference steps.**
| #Steps | Quality | Semantic | Total |
| --- | --- | --- | --- |
| 1 | 74.95 | 64.41 | 72.84 |
| 2 | 78.70 | 69.92 | 76.94 |
| 4 | 83.81 | 72.89 | 81.63 |
| 8 | 83.89 | 73.12 | 81.74 |

## Appendix J Finetune LTX-Video on Internal Datasets

To mitigate the potential impact of training with internal datasets [Sec.˜6](https://arxiv.org/html/2507.13343v2#S6), we trained the LTX-Video model using our internal dataset, and the VBench score is shown in [Tab.˜12](https://arxiv.org/html/2507.13343v2#A10.T12).
The results demonstrate that our internal dataset achieves on-par performance comparing to the original LTX-Video dataset.

**Table 12: The VBench score of official LTX-Video and trained using our internal datasets.**
| Model | Total | Quality | Semantic | Aesthetics | Scene | Consistency |
| --- | --- | --- | --- | --- | --- | --- |
| LTX-Video | 80.00 | 82.30 | 70.79 | 59.81 | 83.45 | 25.19 |
| LTX-Video (Ours) | 80.35 | 82.05 | 73.54 | 64.45 | 37.08 | 26.80 |

## Appendix K Mobile Deployment

We deploy our model on an iPhone 16 Pro Max by converting to FP16 and executing on the Neural Engine and the CPU cores.
To improve on-device numerical stability, we adopt HardSiLU as the activation function and LayerNorm for normalization.
For text encoding, we employ the CLIP text encoder for on-device efficiency, while the T5 encoder is still utilized for the server-side model.

## Appendix L Latency Benchmark Results on Mobile

We provide screenshots illustrating the latency of our mobile model.
The latency is benchmarked on an iPhone 16 Pro Max using Apple’s CoreML toolkits within Xcode.
The reported latency corresponds to the median value collected across multiple runs.
The model is chunked to two chunks for efficient loading and inference.
As shown in [Figure˜7](https://arxiv.org/html/2507.13343v2#A12.F7), the inference time for one-step DiT model is $668.02$ ms in total.
Accordingly, our 4-step model requires $3,021$ ms to generate a 49-frame video at $512\times 384$ resolution.

The latency breakdown for each components in the generation pipeline is shown in the [Tab.˜13](https://arxiv.org/html/2507.13343v2#A12.T13).
Latency for module loading and diffusion backward process are included in I/O and Misc.
The overall latency for the whole pipeline is $3,318$ ms, resulting in an average generation speed of $15$ FPS.

**Table 13: Latency breakdown for on-device demo.**
| Module | Text Encoder | DiT | VAE decoding | I/O and Misc |
| --- | --- | --- | --- | --- |
| Latency (ms) | 6 | 668^† | 230 | 320 |
| ^†The latency for the DiT is corresponds to a single denoising step. |  |  |  |  |

Figure: Figure 7: Latency Benchmark on Apple’s CoreML toolkits within Xcode.
Refer to caption: x7.png

## Appendix M Use of LLMs

We used large language models (*e.g*. ChatGPT, Gemini) solely to assist with manuscript formatting.
No part of the research design, experimental implementation, or analysis relied on LLMs.

## Appendix N Limitations and Broader Impact

Despite these advances, our method has several limitations.
First, the highly compressed latent space and DiT pruning lead to occasional degradations in fine-grained details, particularly in fast motion or complex texture scenes.
Second, due to various practical constraints, most state-of-the-art video diffusion models (VDMs) used for comparison in this work, including our own, are trained on internally collected video datasets that cannot be fully disclosed or released.
As a result, direct comparisons may not be entirely fair and reproducible. To mitigate this limitation, we include a reproduction of the LTX model trained on our dataset and report the results in the [Appendix˜J](https://arxiv.org/html/2507.13343v2#A10).
This work enables efficient video generation on mobile devices, but also carries the potential risk of misuse for generating fake or inappropriate content.

## Appendix O More Qualitative Results

We illustrate more qualitative results of video clips generated by our model in [Figures˜8](https://arxiv.org/html/2507.13343v2#A15.F8) and [9](https://arxiv.org/html/2507.13343v2#A15.F9).

Figure: Figure 8: More quality results generated by our server model.
Refer to caption: x8.png

Figure: Figure 9: More quality results generated by our mobile model.
Refer to caption: x9.png

## Peer Reviews

confidence: 3
contribution: 2
initial_rating: 4
presentation: 3
questions: Q1. What is the key methodological novelty beyond adapting existing compression, pruning, and distillation techniques?

Q2. What ML insights or generalizable design principles does this work provide beyond the specific engineering optimizations for on-device DiT?

Q3. Can you add recent efficient diffusion and/or on-device baselines and deeper efficiency-quality trade-off analysis to better support the claims? Is it possible or not?
soundness: 3
strengths: S1. This paper tackles a practical problem of on-device DiT video generation and provides good deployment results.

S2. The proposed pruning with a KD-guided framework yields substantial speedups with moderate quality drop.

S3. Demonstrating real-time performance on mobile hardware is a meaningful empirical result.
summary: This paper presents a pipeline to run Diffusion Transformer-based video generation efficiently on mobile devices. It combines a high-compression video VAE, tri-level pruning with KD, 4-step adversarial distillation, and operator-level optimizations, achieving over 15 FPS video synthesis on an iPhone 16 Pro Max with competitive quality.
weaknesses: W1. The novelty is somewhat limited, as the proposed approach mainly combines existing compression, pruning, and distillation techniques, rather than introducing new algorithmic ideas.

W2. The contribution is engineering-driven, focusing on system and deployment optimizations, with relatively limited new ML insights or principles that generalize beyond this specific application.

W3. The evaluation is not fully convincing, as it lacks comparisons with recent efficient video diffusion and on-device methods, and provides limited analysis of efficiency-quality trade-offs.

---

confidence: 4
contribution: 3
initial_rating: 6
presentation: 3
questions: 1.Regarding the memory bottleneck addressed by GEMM, is this a problem unique to mobile devices, or would using GEMM on a server-side system provide similar acceleration?

2.Distillation training requires substantial resources; how much time is required to train the lightweight model discussed in the paper?

3.During training, both synthetic and real data are used. Could you provide more details on the ratio between these two types of data, and how different ratios might affect the model’s performance?
soundness: 3
strengths: 1.The authors propose an innovative model acceleration method that addresses the deployment challenges of DiT on mobile devices. By combining VAE compression, pruning, distillation, and other techniques, they achieve real-time video generation.

2.The experiments are thorough, with strong supporting arguments, and the writing is clear and easy to understand.
summary: This paper presents an efficient video generation framework that uses DiT to generate high-quality videos on mobile devices. The authors accelerate inference through techniques such as high-compression VAE, latency-aware pruning, adversarial step distillation, and GEMM-based tiling. Ultimately, the model achieves real-time video generation at 15 frames per second on the iPhone 16 Pro Max.
weaknesses: 1.While the model performance is improved after three layers of pruning and distillation, the distillation process requires significant computational power, which makes training more challenging.

2.The framework was tested on the iPhone 16 Pro Max, but its performance may depend on the specific hardware architecture and optimization strategies. The differences in memory bandwidth and computational power across various edge devices could affect the model’s performance, especially on older devices.

---

confidence: 4
contribution: 2
initial_rating: 4
presentation: 3
questions: 1) Can the authors elaborate on what they consider the core novel contribution of this work beyond engineering integration? Specifically, how does the tri-level pruning or adversarial step distillation differ fundamentally from prior work in UNet-based or transformer-based models?
2) The paper focuses on text-to-video generation. Could the proposed pipeline be adapted for other tasks such as video editing, inpainting, or conditional generation (e.g., image-to-video)? If so, what modifications would be needed?
soundness: 3
strengths: 1) The paper is well written and motivated; it is easy to read and understand.
2) One of the pioneering works in the niche application: one of the first video diffusion transformer models running on-device with decent quality.
3) Comprehensive evaluation, both automated quality metrics and user studies are conducted, and good results are reported.
4) Somewhat novel design of adversarial step distillation setup adapted to the new architecture (DiTs).
5) Interesting discussions about quality-vs-efficiency trade-off in VAEs and pruning strategies of the original model.
summary: The paper proposes a set of optimizations that make video Diffusion Transformers (DiTs) viable for a video generation on mobile devices. By combining a high-compression video VAE, KD-guided tri-level pruning, adversarial step distillation, and operator-level improvements via tiled GEMM, the authors achieve high-quality video synthesis at 15 FPS on an iPhone 16 Pro Max. The approach maintains strong visual fidelity while significantly reducing inference steps and model size, offering a practical solution for on-device video generation.
weaknesses: 1) My main concern with this work is limited novelty: while the engineering contributions are solid and well-executed, the paper primarily combines existing techniques, compression via VAE, pruning, distillation, and operator-level optimization, without introducing fundamentally new ideas or theoretical insights. The novelty lies more in integration than in conceptual advancement.
2) The pruning and distillation strategies, although tailored for DiTs, follow well-established paradigms. The tri-level pruning and adversarial step distillation are adaptations rather than breakthroughs, and similar approaches have been explored in UNet-based models and mobile optimization literature.
3) Another concern is about applicability of the proposed methodology to other mobile devices and platforms. Will the model perform as well on other IPhone devices for example? How about Android devices, will this approach extend to this platform in principle?

---

confidence: 3
contribution: 3
initial_rating: 8
presentation: 3
questions: The paper is generally clearly presented; for specific issues, please refer to the points listed in the Weaknesses.
soundness: 3
strengths: 1. The paper achieved the deployment of a video generation model on mobile devices (iPhone 16 Pro Max). 
2. The experiments were extensive, involving extensive and complex work.
3. The final performance of the method is impressive.
summary: The paper proposes a series of novel optimizations to accelerate video generation and enable practical deployment on mobile
platforms. These optimizations allow the model to achieve approximately 15 frames per second (FPS) generation speed on an iPhone 16 Pro Max, demonstrating the feasibility of efficient, high-quality video generation on mobile devices.
weaknesses: 1. It is recommended to verify the performance of the method on more mobile devices. There are currently various edge hardware environments; for example, low-power intelligent chips from Qualcomm and NVIDIA have been widely adopted, and they differ significantly from Apple's chip architectures. It is recommended to verify the performance on more types of chips or specify the hardware applicability scope of the method in the limitation section.
2. It is recommended to explore the causes and solutions for performance degradation induced by different VAEs, and strengthen the analysis of VAEs' impact on performance. For instance, it should be clarified which aspects of visual quality degradation correspond to PSNR loss, and how such degradation propagates to affect the sub-items in VBench scores. Additionally, efforts should be made to establish a theoretical connection between VAEs and DiT optimization, and to discuss the effectiveness and underlying principles of existing DiT optimization methods for compensating for performance losses when applied to models under current VAE conditions. 
3. Given that this research has achieved significant improvements in video generation performance, it is recommended to make the code for reproducible experimental results publicly available to enhance the reproducibility of the paper's experimental results.

---

confidence: 4
contribution: 2
initial_rating: 4
presentation: 3
questions: - Could you list the empirical findings obtained through this work that are specifically important for text-to-video generation? It would be beneficial to demonstrate that this work offers sufficient novelty beyond simply combining existing techniques.
soundness: 3
strengths: - The motivation for this work is clear and convincing. Nowadays, many video generation models or services have been released, but almost all of them require a huge amount of computational resources to run. Reducing this cost could have a significant impact on user-generated content creation and also contribute to greater sustainability.
- The base model used in this work, DiT in latent space, is a popular choice for video generation tasks. Therefore, the proposed modifications are likely to be widely applicable to existing or future video generation models.
summary: This paper presents several modifications to DiT-based video generation models to enable them to run on mobile devices. Specifically, for the model architecture, the paper adopts a high-compression video VAE to reduce the number of tokens and a lightweight DiT obtained through structured pruning. For training, adversarial distillation is used to reduce the number of timesteps required during inference. For efficient inference, a tiled GEMM strategy is applied to alleviate memory bottlenecks. The experimental results show that the proposed model can generate a 49-frame video clip within 4 seconds on an iPhone 16 Pro Max.
weaknesses: - The novelty in methodology is marginal.
  - Increasing the compression ratio in VAE is a common choice (e.g., [R1]) for efficient latent diffusion models. To enhance the novelty, it would be beneficial if the authors could provide empirical findings that are particularly important in the context of video generation.
    - [R1] "SANA: Efficient High-Resolution Image Synthesis with Linear Diffusion Transformers," ICLR 2025.
  - The idea of tri-level pruning sounds interesting, but the optimization process is greedy and not specifically designed for tri-level pruning. If the authors want to argue for the benefits of the tri-level pruning search space, the ablation study in Table 4 should include results for all single-level pruning strategies (not only block pruning, but also attention-head and linear pruning).
  - Knowledge distillation and adversarial distillation are also common approaches for obtaining lightweight diffusion models. Using part of the teacher model as a feature extractor for the discriminator is interesting, but a similar idea has already been explored in prior work [R2].
    - [R2] "SoundCTM: Unifying Score-based and Consistency Models for Full-band Text-to-Sound Generation," ICLR 2025.

- The main quantitative results shown in Table 2 lack several important metrics, such as dynamic degree and motion smoothness. For a comprehensive comparison, it is highly encouraged to provide the full list of results.


Decision: Reject