Accelerating Parallel Diffusion Model Serving with Residual Compression
Authors: Jiajun Luo
Yicheng Xiao
Jianru Xu
Yangxiu You
Rongwei Lu
Chen Tang
Jingyan Jiang
Zhi Wang
Venue: NeurIPS 2025 poster ()
Keywords: diffusion model acceleration
parallel inference
Primary area: infrastructure

Abstract: Diffusion models produce realistic images and videos but require substantial computational resources, necessitating multi-accelerator parallelism for real-time deployment. However, parallel inference introduces significant communication overhead from exchanging large activations between devices, limiting efficiency and scalability. We present CompactFusion, a compression framework that significantly reduces communication while preserving generation quality. Our key observation is that diffusion activations exhibit strong temporal redundancy—adjacent steps produce highly similar activations, saturating bandwidth with near-duplicate data carrying little new information. To address this inefficiency, we seek a more compact representation that encodes only the essential information. CompactFusion achieves this via Residual Compression that transmits only compressed residuals (step-wise activation differences). Based on empirical analysis and theoretical justification, we show that it effectively removes redundant data, enabling substantial data reduction while maintaining high fidelity. We also integrate lightweight error feedback to prevent error accumulation. CompactFusion establishes a new paradigm for parallel diffusion inference, delivering lower latency and significantly higher generation quality than prior methods. On 4$\times$L20, it achieves $3.0\times$ speedup while greatly improving fidelity. It also uniquely supports communication-heavy strategies like sequence parallelism on slow networks, achieving $6.7\times$ speedup over prior overlap-based method. CompactFusion applies broadly across diffusion models and parallel settings, and integrates easily without requiring pipeline rework. Portable implementation demonstrated on xDiT is publicly available at https://github.com/Cobalt-27/CompactFusion

Accelerating Parallel Diffusion Model Serving with Residual Compression
Jiajun Luo"
Yicheng Xiao Southern University of Science and Technology
Shenzhen International Graduate School Tsinghua University
1uo-jj244mar .ws.gaaalsssssnuuuaa..mm.mmmm edu
xiaoyc20220mail http:/0mmml.sssuseeeeeeee sustech edu.
Jianru Xu
Yangxiu You Jiangnan University
Southern University of Science and Technology
j joesephyou@tencent.com
122118300maa sustech. edu. cn
Rongwei Lu
Chen Tang
Shenzhen International Graduate School Tsinghua University
The Chinese University of Hong Kong chentang@link cuhk. edu. hk
lurw240mails. tsinghua, edu. cn
Jingyan Jiang
Zhi Wang!
Shenzhen Technology University siangjingyandsztu.co edu.
Shenzhen International Graduate School
Tsinghua University
wangzhi @sz. tsinghua. edu. cn
CompactFusion Compression- fresh activations
PipeFusion Overlap-based stale activations
DistriFusion Overlap-based stale activations
Original Single-device
FID 3.263
FID 6.722 Latency: 9.49s
FID 9.911 Latency: 8.05s
Lateney: 23.16s
Lutency: 7.57%
Proupt: fantony concept art an etf, magufficent palntorly, eplc, majestic, magicaal fantosy caser ar,, dreawe.
Prompi: Romantic painting of vailing stormy with dramatic lighting and powerful
Figure 1: CompactFusion Fusion is a compression framework for parallel diffusion serving acceleration. Leveraging the intrinsie redundancy in diffusion models, CompactFusion transmits compressed step-wise residuals instead of full activation to significantly reduce communication volume and preserve fidelity, while prior works rely on stale activations for compuaationcoommnicctt overlap, leading to noticeable degradation. Setting: 42020, FLUX-1.dev, 28-step, 1024x 1024 resolution. 1-step warmup for all algorithms.
"Equal contribution. Corresponding author.
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

Abstract
Diffusion models produce realistic images and videos but require substantial compu- tational resources, necessitating multi-accelerator parallelism for real-time deploy- ment. However, parallel inference introduces significant communication overhead from exchanging large activations between devices, limiting efficiency and scala- bility. We present Compact usion, a compression framework that significantly reduces communication while preserving generation quality. Our key observation is that diffusion activations exhibit strong temporal redundancy-adjjan produce highly similar activations, saturating bandwidth with near-duplicate data carrying little new information. To address this inefficieney, we seek more com- pact representation that encodes only the essential information. CompactFusion achieves this via Residual Compression that transmits only compressed residuals (step-wise activation differences). Based on empirical analysis and theoretical justification, we show that it effectively removes redundant data, enabling substan- tial data reduction while maintaining high fidelity. We also integrate lightweight error feedback to prevent error accumulation. CompactFusion establishes a new paradigm for parallel diffusion inference, delivering lower latency and significantly higher generation quality than prior methods. On 4x L220, it achieves 3.0x speedup while greatly improving fidelity. It also uniquely supports communication-heat strategies like sequence parallelism on slow networks, achieving 6.7 7x speedup over prior overlap-based method. CompactFusion applies broadly across diffu- sion models and parallel settings, and integrates easily without requiring pipeline rework. Portable implementation demonstrated on xDiT is publicly available at https: //github. com Chobalttttttto Saabballt77//oopppe 27/CompactFusion
steps
Introduction
Diffusion models are scaling rapidly both in size and in computation, outpacing the capacity of single accelerators. The model sizes have grown from 983M parameters in Stable Diffusion 1.5 []] to more than 12B in FLUX.1 [2). Meanwhile, diffusion models are significantly more computationally intensive than LLMs, with compute increasing faster than the model size [3]. As computational demand rises, single-GPU inference can no longer meet latency constraints, making multiaccelerator parallelism essential for practical deployment.
However, parallel paradigms introduce new challenge: the increasing prominence of communication bottlenecks, stemming from the fact that interconnect bandwidth has not kept pace with the growth of the compute. From A 100 to H100, FP16 FLOPS increases over 6x (31 12T - 1,979T), while NVLink bandwidth grows only 1.5x and PCIe bandwidth simply doubles S In FLUXX. 1, standard patch parallelism transmits around 60 GB of activations per image per GPU, consuming over 45% inference time across x220 with PCIe interconnects. As bandwidth lags behind, communication increasingly dominates inference cost, underscoring the need to reduce transmission overhead.
Previous works exploited the Temporal Redundancy 6 inherent in diffusion models, where adja- cent inference steps produce highly similar activations, to mitigate the communication bottleneck and accelerate parallel inference. Methods like DistriFusion Parallelism, which reuses stale activations from previous steps to overlap communication with computation. Although this reduces visible latency, it introduces three fundamental limitations. (1) Displaced parallelism replaces current activations with outdated ones, leading to noticeable quality degradation. (2) Moreover, the core data volume remains unreduced, meaning communication intensive strategies like sequence parallel [9] still transmit large activations. Consequently, their performance gains are fragile and collapse when the overlap window insufficiently masks communi- cation costs. (3) It requires nontrivial rework of the model pipeline [10], which makes integration complex and limits generality across architectures.
and PipeFusion [8] adopt Displaced
We rethink how temporal redundancy should be exploited. and propose Residual Compression as a more fundamental solution. Instead of merely overlapping the transfer of redundant activations, we ask: why transmit redundant data at all? Diffusion activations change slowly over time with adjacent steps producing highly similar value; yet prior methods still transmit full activations, saturating interconnects with ncar-duplicate data carrying little new information. Our key observation
2

---

Residual Compression(ous))
6x
8x
16x
67x low-ronk
100% low- www-rank 0PIPS-0.2754
DistriFusion
Original
Sparsification
2. guantization
1-bit quomtization LPIPS=0.2595
overlap-husee based
1PIPS(L)001143
0.309
Communication Volume (per device, after 1-step warmup)
9.18GB
7.4GGB
3.75GB
0.88GB
0.59GB
58.8GB
Prompit slftiin rumwayy getting be
Figure 2: We introduce Residual Compression as new paradigm for efficient parallel diffusion. It generalizes across compressors and sustains high quality even under 100.05 compression. Setups: 4 devices, 1-step warmup.
that removing redundant data should drastically reduce communication volume and resolve the communication bottleneck, while maintaining the quality. Thus, we seek a more compact representation that encodes only the essential change. Residual Compression achieves this by transmitting only compressed activation residuals (differences between time-steps), combined with a lightweight error feedback mechanism. Based on empirical analysis and theoretical justification, show that it effectively removes redundant data, enabling substantial data reduction while maintaining high fidelity. We redefine how redundancy is addressed in parallel diffusion-not by hiding its transmission with overlap tricks, but by eliminating at the source.
Residual compression surpasses previous overlap- based methods for following superiority:
Avoiding stale activations leads higher quality. Residual Compression works with current data, avoiding the quality degradation inherent in using stale activations in prior methods, yielding significantly higher fidelity. Under aggressive 2-bit quantization, resid- ual compression achieves excellent quality substantially outperforming state-of-the-art DistriFusion and PipeFusion identical setups (Figure M.. Remarkably, even at extreme compression ratios of over 100x (transmitting < 1% of the original data), method maintains better quality than DistriFusion (Figure 2 and section 4.2).
A significant drop in data volume yields much lower latency. Residual Compression significantly reduces data exchange, delivering superior end-to-end performance across diverse hardware configurations. achieved 3x speedup on 4xH20 (NVLink) and 4 x220 (PCle) clusters (Figure []. surpassing prior methods. Furthermore, it makes communication- intensive parallel strategies practical even low-bandwidth environments, achieving 6.7x speedup over DistriFusion on Ethernet-level bandwidth.
Residual compression structurally decoupled from the parallel pipeline, enabling broad compatibility and easy adoption. operates entirely the communication layer, without modifying model logic or parallel execution flow. The design generalizes across compression methods (Figure 2} and parallel strategies, and has been applied on large-scale image and video models such as FLUX. and CogVideoX, and integrated into frameworks including xDiT and distrifuser, with fewer than 20 lines of core code changed. contrast, prior methods are tightly coupled with specific strategies and require substantial rework. By focusing exclusively on the transmission aspect, residual compression maintains lightweight profile and exhibits deployment readiness.
CompactFusion Fusion implements residual compression as a portable system, supporting compression techniques including quantization [II], low-rank, and sparsity (as shown in Figure with tunable ratios from 2x over 100x while preserving high fidelity. works with parallel strategies such as Patch Parallel and Ring Attention, and integrates into existing frameworks (xDiT, distrifuser) with minimal modification. We believe CompactFuse represents promising paradigm shift parallel diffusion acceleration.
2 Preliminaries on Parallelism
Parallelism the art of distributing tasks across multiple devices, enabling accelerators work collaboratively for improved latency and throughput. This distribution can occur across various

---

diatsifuter
Sender
Rreeiver
FLR Cop.idark.
Rediooi
compresa
Doviee Eniter
Kay
CampactPuite
LighWiiphi
Copprowte
exeoe deedbeck
decompoeaи Compressed
Antivation Recried
Leneastinaian
Roddell make
(Nen)
(ien)
Ell-gathee...
Conproske ExggrookeDvvagerok
Kereat
(a) Residual compression with error feedback.
(b) System overview.
Figure 3: (a) CompactFusion prevents long-term accumulation by adding feedback into residual compression. (b) By wrapping a few standard communication primitives with a unified compression interface, CompactFusion requires minimal change the existing frameworks and easily be integrated to new model structure.
dimensions: input token(patch) sequence(Sequence Parallel 9), model layers (Pipeline Parallel 12
intra-layer computation (Tensor/Expert Parallel 13 14]), data samples (Data Parallel [5]
Notably, many parallel strategies necessitate intensive inter-device communication 7 due to partitioning of weights, activations, inputs among devices, requiring data exchange for complete computation.
This work focuses on Sequence Parallel 9 16 17. which partitions the input sequence across devices and is used in recent works on parallel diffusion 77 as well as negh-performance diffusion inference frameworks [T0] [18. diffusion models,
J. It is by far the most widely adopted parallel strategy for highly latency-frienlly and well suited for real-time generation. more detailed description of these strategies is provided in Appendix B
To reduce perceived latency, prior works overlaps communication with computation by reusing stale activations 77 g 19 but suffers from quality degradation, complex pipeline rework, and unchanged (potentially large) communication volume, limiting its effectiveness.
Method
CompactFusion mitigates the communication bottleneck in parallel diffusion by leveraging residual compression (Sections t. 3.3) and practical system co-design (Section 3.4), achieving significant data reduction with minimal quality degradation.
3.1 Exploiting Temporal Redundancy via Residual Compression
Naive Compression (Comupeed)
(b) Naive Compression
(c) Residual Compression wil Erro-Feedbcck
(d) Residual Compression Fratbbe
Original
Conpecoica Lave
Fodeeaa Nora
Tool Enm
Man Tirl Lime
Residual Compression Evwrr-oedllaot (High Falefing LPIPS: 0,2585
liman Feestback
0
19
15
25
0
10
15
10
Steps
Stepo
Strpp
LMPS: 0.3887
POANO DN CompoRis
Compration
Phincigal
Pritaiped Chmpoean
Prinaqpel Compusont
Priscipal Composzni
Prompt: Attronant in a jungle. cold color paleng, muted colors, detutied,
Large Total Exyor
Small Total Eavee
Stinllor Total Error
Trajectery
Figure Comparison of strategies under 1-bit quantization: compression, residual compression with/without error feedback. Left: Visual Results, right: Error Analysis and PCA Trajectories (closer better) of activations over steps. We report two metrics: Compression Error (difference between input and output of compression) and Total Error (cumulative deviation from the uncompressed ground truth, lower is better). Setups: devices, 1-step warmup.
original (no compression), naive

---

Transmiting compressed residuals effectively eliminates redundancy from the data, lowering the communication volume while preserving fidelity. This efficiency stems from key insight: diffusion models exhibit high temporal redundancy, with activations changing only minimally between steps. As result, transmitting full activations often involves sending largely redundant information. By encoding only the meaningful differences, we can greatly reduce the volume of transmitted data. However, naively compressing high-dimensional activations still introduces substantial errors, often resulting in severe quality degradation or visual collapse (as shown in Figure 4aa). Effectively leveraging temporal redundancy, therefore, requires more than simply reducing the amount of data- demands meticulous design.
Our optimization builds on two observations. First, activations change gradually across steps, their step-wise residuals from the previous result) having much smaller magnitude (as shown in Figure[5] residual norm is only a fraction of full activation norm). Second, compressing a magnntuee signal typically induces proportionally less error. This makes the residual a more efficient compression target as it captures the essential change. We therefore propose residual compression: instead of transmitting the full activation, we transmit its compressed difference from the previous reconstruction and recover the current state by adding it back.
Activation Similarity
0.3
2500
2000
1500
0.4-
1000
0.0
30
Steps
Similoity Actication Norm
To verify this principle, we apply aggressive 1 1-bit binarization (detailed in Appendix(D) to both naive (compressing full activation, Figure 4b) and residual compression (Figure [()) over Ring Attention. Naive compression fails completely: the output is corrupted, with large errors. In contrast, residual compression yields clean reconstructions with much smaller error at each step, and the activation trajectory remains close to the uncompressed baseline throughout.
Figure 5: Adjacent-Step Ac- tivation Similarity and Activa- tion/Residual Norm, on Flux.
While the idea of transmitting differences exists in other contexts 20 knowledge, CompactFusions is the first to demonstrate its effectiveness on parallel inference, and also the first on diffusion models, enabling aggressive data reduction while preserving generation quality.
the best of our
Limitation: Despite low per-step error, residual compression accumulates error over time. As shown in Figure c), the total error grows ovear steps without correction, which requires mechanism to prevent the accumulation of long-term error.
3.2 Compensating Error Accumulation with Feedback
Residual compression significantly reduces per-step compression error by targeting smaller-mage maggitude signals. However, these small errors still accumulate over diffusion steps (curve for Total Error, Figure (c)) To eliminate long-term drift and preserve fidelity, CompactFusion integrates Error Feedback, a technique used in gradient compression [22 23 Rather than discarding the residual compression error at each step, we store it locally and add the next-step residual before compres- sion, recirculating untransmitted information and systematically compensating for loss (illustrated in Figure Ba). This stabilizes reconstruction and prevents the compressed state from diverging from the original trajectory. Details are in Appendix H
Problem Formulation. To understand how compression affects inference quality, we model diffusion as sequence of transformations
f(( where denotes the intermediate activation in fixed layer and captures the combined effect of one forward pass and scheduler update. Following
relevant works 24|25] we define a 5-compressor C3 with d E (0, 1]. satisfying E [[[C((z) - =|
81-6)15
|| for å E (0, 1].
Assumptions. Our analysis relies on realistic assumptions grounded in empirical observations of diffusion behavior. First, the model exhibits strong local
by L-smoothness of each transformation fe, i.e., [lff((r) fi (2) < L.2 wll², with L < 1. Second, activations evolve gradually over time due to temporal redundancy: the expected per-step change is much smaller in scale than the activations themselves, i.e., o & a We also assume uncorrelated error terms where appropriate. The definitions and assumptions are elaborated in Appendix H.2 and Appendix H.3
5

---

these properties are well-supported in practice practice-activatiinn norms remain bounded, and residual magnitude is significantly smaller than that of activations (Figure
Proposition 3.1 (Steady-State Error Bound). Let v and resiiuual denote the steady-state mean squared error upper bounds under naive compression and residual compression with error feedback, respectively. Then under the given assumptions, their ratio satisfies the bound
residual
02 0
1
yiaiie
(1-))
See Appendix for the proof. This ratio confirms that residual compression with feedback yields significantly lower steady-state error, often by an order of magnitude or more. The result holds under mild stability condition: 5 > 1 1-2² which imposes a lower bound on the required compression quality. In contrast, Residual compression withoul error feedback fails to converge; its error grows linearly with time steps and does not admit steady state bound.
1241
Validation. We validate this theory empirically using 1-bit binarization. As shown in Figure 4 d), CompactFusion with error feedback maintains low cumulative deviation throughout the inference process, closely follows the original trajectory, and generates high-quality images.
Advantages. Residual compression with error feedback enables robust and accurate compression under aggressive settings, maintaining fidelity even with 1-bit quantization (16x compression).
3.3 Scaling Residual Compression Extreme Ratios
We aim to push the boundary of residual compression, targeting over 100x compression ratio while preserving quality beyond state-of-the-art methods.
At extreme compression ratios, low-rank approximation is the only viable option. Quantization saturates at 1-bit (16x), and sparsification collapses under diffusion 100x sparsity, most values are never updated over typical 20-30 steps. In contrast, low-rank approximation provides a promising alternative: retains full coverage of the coordinates while drastically reducing communication volume. Each activation or residual tensor X E RR is approximated as X ~ UVT, where U E Raxx and V E mk with r min(n,m).
To make this feasible under real-time constraints, we adopt subspace iteration (26 (detailed in Appendix D). faster alternative to SVD. While SVD offers optimality, it is approximately 60 times slower than the subspace iteration solution, far exceeding the ~5ms communication window and ~Ims compression budget in practice. In contrast, adapt the subspace iteration from gradient compression (71. producing a usable approximation milliseconds.
We improve low-rank compression by achieving better tradeoff between precision and directional coverage. We observe that diffusion residuals are frequently high-rank, indicating that the activation changes span many directions across steps. However, each transmission is constrained to low-rank subspace. We identify this high-rankkloowrrak mismatch as key bottleneck: the model attempts dense updates, but the compressor delivers sparse (in terms of subspace dimension) projections. To resolve this, we propose a strategic sacciff-acriiiiiing accuracy (closeness to the optimal approximation) to expand rank coverage (number of dimensions covered per step). Concretely, we quantize the low-rank matrices with INT4, allowing higher per-step rank under the same transmission budget. This transforms the compressor into subspace throughput machine, delivering broader coverage across update directions, even if each is less precise.
We empirically validate this strategy by comparing it with an alternative that improves approximation optimality through more subspace iterations(see Appendix F.T. Despite added quantization error, expanding rank coverage yields significantly better generation quality, confirming that directional breadth matters more than per-step accuracy under tight bandwidth constraints.
Notably, our approach maintains better visual fidelity over DistriFusion even at 100.05x x compression on 1-dv.
3.4 System Co-Design for Efficiency and Usability
CompactFusion is co-designed for real-world deploy algorithmic gains into mea- surable latency reduction with minimal integration cost.

---

Optimized Compression Kernels. We develop highly optimized GPU kernels for our compression techniques. Beyond efficient quantization and subspace iteration, we repurpose the concept of N:M sparsity 28 which originally used in weight pruning, and adapt it to activation compression via N:M block sparsifier. Unlike standard Topk methods that suffer from sorting overhead and irregular memory access, our block sparsifier enables linear-time selection and GPU-friendly memory patterns. is the only sparsifier design we found to yield practical on-device speedup.
Latency Hiding via Overlap. The framework runs compression kernel concurrently with commu- nication primitives. By carefully integrating into Ring Attention pipeline, effectively overlap the compression overhead with communication waits, minimizing its impact latency.
Ease of Integration. CompactFusion benefits from the inherent modularity Residual Com- pression, which cleanly separates compression from execution logic (Figure 3b). It wraps standard communication primitives without altering model code or parallel pipeline, and generalizes across strategies, compressors, and frameworks. In contrast, prior methods often entangle with specific parallel strategies and require substantial pipeline rework.
Experiments
4.1
Setups
Our method works with off-the-shelf models, we evaluate it on the state-of-the-art FLUX.J 1- dev 2 for image generation, and Cog X-d (29] for video generation. We adhere standard inference settings on xDiT, employing 28 steps for FLUUX. 1..UX.--de and 50 steps for Cog VideoX22..
Models.
Dataset. We test the image generation model using prompts in COCO Captions 2014 dataset 30 and video generation using prompts sampled from VBench 31
Hardware. To demonstrate broad applicability, experiments conducted various hardware and interconnects: high- baghidh NVLink (H20 clusters, bandwidth: 366 GB/s), standard PCIc (L20 clusters, bandwidth: 17.13 GB/s) and simulated lower-bandwidth Ethernet (A40 clusters using tc for traffic control), ensuring robustness evaluation under various deployment constraints.
Baselines. We apply CompactFusion Sequence Parallelism, the most widely adopted strategy in parallel diffusion (Section 2). to reflect real-world deployment scenarios. We compare it with strong baselines from both industry and research, including Patch Parallel (all- (all-gather KV) [18). Ring Attention (ring-style KV transfer)
Ulysses (all-to-all) [16], and DistriFusion (displaced patch parallel) 18 We also include PipeFusine (displaced TeraPipe 32
[9].
g as
baseline, though it is orthogonal to our method. We note that PipeFusion is not evaluated on video models as the xDiT does not support PipeFusion for video models (further discussed in Appendix (
CompactFusion usion Variants. We showcase three CompactFusion variants over Sequence Parallel with Ring Attention. Compact-1bit and Compact-2bit apply 1-bit and 2-bit quantization, with 16x and 8x compression respectively. Compact-Lowrand shows the potential for extreme communication reduction, reaching 100.05x compression on FLUX via rank-32 approximation and INT4.
Metries. Following the relevant work, we adopt the standard evaluataion metrics for image quality testing: Peak Signal Noise Ratio (PSNR), Learned Perceptual Image Patch Similarity (LPIPS) 3 and Fréchet Inception Distance (FID) 34 We adopt the following metrics for video quality testing: Structural Similarity Index Measure (SSIM) 35), Peak Signal-to-Noise Ratio (PSNR), and Learned Perceptual Image Patch Similarity (LPIPS) 33). We also conduct human evaluation study assess
perceptual consistency from a human perspective (detailed in Appendix E.
4.2 Main Results
CompactFusion achieves lower latency while maintaining high visual fidelity. It consistently performs well across different generation models, including FLUXX. 1-dev for images and VideoX for videos. It is also robust across hardware setups-L200 L20, H20, and A40-and and across network conditions such as
7

---

NVLink, PCIe, and simulated Ethernet. These results are shown in Figure across and 6-GPU scales are provided in Tables and 2
Quantitative metrics
Notably, CompactFusion uniquely enables the widely adopted but communicationnintenssit sequence parallel over slow networks, achieving up to 6.7x speedup DistriFusion under Ethernet-level bandwidth conditions (Figure 7
By directly eliminating redundant data, CompactFusion drastically reduces the transmission volume. In its low-rank variant, it sends less than 1% the original activation data. Despite this, still outperforms DistriFusion in generation quality, shown in Table (image) and Table 2 (video).
SP
DF
PF
1bit
2hit
Lowrank
SP
DF
1bit
2bit
Lowrank
Latenzy:
10. 89s
8.05s
9.49s
7.46s
7.57s
10.60s
Lateney:
52.90s
39.93s
35.50s
35.54s
39.136
FID (D)-
9.9113
6.7221
7.0819
3.2625
8.6817
LPIPS(L):
0.3624
0.3084
0.2373
0.3391
(L
0.3099
0.2595
Prompt: madti colored parrot holding its foot up to its beak.
Prompt: detaifed wooden toy ship intricately carved musts...
Prompt: kid wearing headphonne and usting loptop.
Prompt: The camera follows behind a white vintage SUV...
Prompt: A pair of parking meters reflecting expired times.
Prompt: A street artist, clad worn-oud denim jacket...
Prompt: A double decker bus driving down the street.
Prompt: An elderly gentleman, serene expression....
Figure 6: Qualitative results. FID and LPIPS is computed against the original images. SP stands for Sequence Parallel using Ring Attention, DF for DistriFusion and PF for PipeFusion. 1bit, 2bit and Lowrank are our methods. We use 1-step warmup for Soompaddissio//pppffuoioDDDiiiii
Table 1: Quantitative evaluation on 4 GPUs. w/ G.T. means calculating the metrics with the ground-
truth COCO images, while w/ Orig. means with the original model's samples. Latency results 1,20 (PCIE) and H20 (NVLink) are reported; speedup is calculated with respect to the single-device latency. Note that COCO-bace based metrics w/ G.T. may be uninformative. Its variations small, and even disrupted outputs can score well [36].
Method
PSNR (?)
LPIPS
FID (J)
Latency
Speedup
Human
Eval. (1)
Orig.
G.T.
w/ Orig.
G.T.
w/ Orig.
L20
H20
L.0
H20
Drigina((Snglle-eevie
23.16
20.26
1.00
1.00
Ring Attention
10.89
7.54
2.12
2.68
0.772
32.75
Patch Parallel
10.90
6.83
2.12
2.97
DeepSpeed-Ulyssee
9.13
6.70
2.54
3.02
DistriFusion
21.63
0.761
0.310
33.12
9.91
8.05
8.05
2.88
2.52
0.25
PipeFusion
23.42
0.766
0.250
32.36
6.72
9.49
9.07
2.44
2.23
0.56
Compact-lbit(Qun
22.90
0.767
0.260
33.20
7.08
7.46
6.86
3.10
2.95
Compact-2bit(Quur
29.54
0.772
0.114
33.09
3.26
7.57
6.70
3.06
3,02
0.84
Compact-Lowrank(Ous
22.85
0.769
0.275
33.07
8.68
10.60
11.99
2.18
1.69
0.38
4.3 Ablation Studies
We perform targeted ablation studies to evaluate the contributions of core components and design choices in CompactFusion.
Warmup Step. Like displaced parallel, CompaetFusion requires at least a 1-step warmup, where the uncompressed activation is used to initialize the base tensor for later residual computation (detailed in Appendix DD). Figure compares different methods and warmups. We observe that CompactFusion maintains stable and high visual quality with just single warmup step, showing

---

500MB/s
1000MB/s
1500MB//
2000
150
Runtime (6)
L2 1
L90x 1.4%
LM 1315
20
1988 19
20
4.76
17.76
18:7
55.00
Ring Attention
slyses
Compact-
Compact-Lowran
Patch Parallel
DistriFusion
Compact-2bit
Figure 7: Measured latency of sequence parallel variants on A4 GPUs under simulated Ethernet bandwidth. Horizontal dashed line indicates single-device latency. Only CompactFusion yields speedups, uniquely enabling efficient sequence parallelism over slow networks.
Table 2: Quantitative video generation evaluation 3 GPUs and 6 GPUs. All metrics (SSIM, PSNR, LPIPS) are calculated with respect to the original samples. Latency results on L20 (PCIE) are reported: speedup is calculated against single-device latency.
Device
Method
SSIM (1) PSNR (1) LPIPS (L)
Latency (8)
Speedup
Original(Single-dvvie
122.42
Ring Attention
63.30
1.93x
Patch Parallel
72.61
1.69x
3
DeepSpeed-Ulyee Ulysses
62.65
1.95x
DistriFusion
0.842
24.13
0.220
60.75
2.01x
Compact- (Ours)
0.762
20.46
0.291
56.41
2.17x
Compact-2bit (Ours)
0.832
24.37
0.217
56.73
2.16x
Compact-Lowrank (Ours)
0.728
19.64
0.320
60.98
2.01x
Ring Attention
52.90
2.31x
Patch Parallel
53.37
2.29x
DeepSpeed-Ulyee Ulysses
39.03
3.14x
DistriFusion
0.703
18.09
0.362
39.93
3.07x
Compact- -bit (Ours)
0.746
19.75
0.308
35.50
3,45x
Compact-2bit (Ours)
0.813
23.26
0.237
35.54
3.45x
Compact-Lowrank (Ours)
0.708
0.339
39.13
3.13x
little degradation compared longer warmup. In contrast, baseline methods are more sensitive to warmup configuration. For more detailed results, please refer to Appendix
Original
Distrifusion
Distrifusion 2-step Warm-up FID: 8.4666
Compact-
Compuct--2bit
steep Warm-up
I-step Warm-up
2-step Warm-up 6,2383
1-step Warm-up
2-step Warm-up
9,9113
FID: 7,0819
FID: 3 3.262
FID:
LPIPS: 0.3099
LPIPS: 0.2618
LPIPS: 0.2595
LPIPS: 0.2195
LPIPS: 0.1143
LPIPS: 0.0951
Prompt:
green water.
Prompt: A motorcycle sits on the pavement on a cloudy day.
Figure Impact of warmup steps visual quality.
Effectiveness of Error Feedback. Error Feedback significantly improves quality. For Compact- 1bit under 1-step warmup. reduces FID from 19.23 to 7.08, lowers LPIPS from 0.389 to 0.260, and increases PSNR from 19.78 to 22.90. For more detailed results, please refer to Appendix F2
Performance on Patch Parallel and Ring Attention. and Ring Attention with minimal integration effort. On FLUX.I-dev and 4xL20, Compact-2bit
CompactFusion supports both Patch Parallel

---

reduces latency from 10.90s to 8.08s on Patch Parallel (26% reduction), and from 10.89s to 7.71s on Ring Attention (29% reduction). both settings, it delivers high-quality generation, significantly outperforming prior work across perceptual metrics.
5 Conclusion
CompactFusion accelerates parallel diffusion by eliminating temporal redundancy at the source. It transmits only compressed residuals with enror feedback, preserving generation quality while significantly reducing communication. CompactFusion is designed as modular, drop-in layer over standard communication primitives, and integrates seamlessly into existing frameworks. Extensive experiments show that delivers consistent speedups across models, hardware, and networks. We offer a lightweight and efficient approach for diffusion inference scale.
6 Acknowledgements
This work is supported in part by the National Key Research and Development Project of China (Grant No. FFF905502)) National Natural Science Foundation of China (Grant No. 92467204 and 62472249), and Shenzhen Science and Technology Program (Grant No. JCYJ202208 18 101014030 and KIZDD0240903 1022000000010300001 102300001). This study also supported in part by the Natural Science Founda- tion of Top Talent of SZTu(Grant No. GDRC202413),
We would like to express sincere appreciation the xDiT team for their comprehensive parallel diffusion framework, which served as the foundation for our experiments.
We thank all participants who contributed to the human evaluation survey. We also thank Zhengyi Su for helpful discussions and support on the theoretical analysis.
10

---

References
[1] Rohin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bjöm Ommer. High-resolution image synthesis with latent diffusion models. Proceedings of the IEEE/CVE Conference on Computer Vision and Pattern Recognition (CVPR), pages 10684-10695, June 2022.
[2] Black Forest Labs. Flux.
forestt labs bobbsfflux 2024.
[3] Muyang Li, Yujun Lin, Zhekai Zhang, Tianle Cai, Xiuyu Li, Junxian Guo, Faze Xie, Chenlin Meng. Jun- Yan Zhu, and Song Han. SVDQuant: Absorbing Outliers by Low-Rank Components for 4-Bit Diffusion Models, March 2025. arXiv:441..05007 [es].
[4] NVIDIA A100 GPUs Power the Modern Data Center.
[5] NVIDIA H100 Tensor Core GPU.
[6] Xibo Sun, Jiarui Fang. Aoyu Li, and Jinzhe Pan. Unveiling Redundancy in Diffusion Transformers (DiTs): A Systematic Study, November 2024. arXiv:2411.1333 [es].
Muyang Li, Tianle Cai, Jiaxin Cao, Qinsheng Zhang, Han Cai, Junjie Bai, Yangqing Jia, Ming- Yu Liu, Kai Li, and Song Han. DistriFusion: Distributed Parallel Inference for High nigh-Resolution Diffusion Models, July 2024. arXiv:2402.1945 [cs].
[8] Jiannan Wang, Jiarui Fang, Aoyu Li, and PengCheng Yang. PipeFusion: Displaced Patch Pipeline Parallelism for Inference of Diffusion Transformer Models, May 2024. 144440 14430 [es].
Shenggui Li, Fuzhao Xue, Chaitanya Baranwal, Yongbin Li, and Yang You. Sequence Parallelism: Long Sequence Training from System Perspective, May 2022. arXiv:2105. 13120 [es].
[10] xdit-projectxxDiT, May 2025.
[11] Benoit Jacob, Skirmantas Kligys, Bo Chen, Menglong Zhu, Matthew Tang, Andrew Howard, Hartwig Adam, and Dmitry Kalenichenko. Quantization and Training of Neural Networks for Efficient Integer- Arithmetic-Only Inference. In 2018 IEFF/CVF Conference Computer Vision and Pottern Recognition, pages 2704-2713, Salt Lake City, UT, June 2018. IERE.
[12] Yanping Huang, Youlong Cheng, Ankur Bapna, Orhan Firat, Mia Xu Chen, Dehao Chen, HyoukJoong Lee, Jiquan Ngiam, Quoe V. Le, Yonghui Wu, and Zhifeng Chen. GPipe: Efficient Training of Giant Neural Networks using Pipeline Parallelism, July 2019. 1rXiv:1811.06665 [cs].
[13] Deepak Narayanan, Mohammad Shoeybi, Jared Casper, Patrick LeGresley, Mostofa Patwary, Vijay Anand Korthikanti, Dmitri Vainbrand, Prethvi Kashinkunti, Julie Bernauer, Bryan Catanzaro, Amar Phanishayee, and Matei Zaharia. Efficient Large-Scale Language Model Training on GPU Clusters Using Megatron-L.M, August 2021. 1XXv:2104.044733 [cs].
[14] Dmitry Lepikhin, HyoukJoong Lec, Yuanzhong Xu, Dehao Chen, Orhan Firat, Yanping Huang, Maxim Krikun, Noam Shazeer, and Zhifeng Chen. GShard: Scaling Giant Models with Conditional Computation and Automatic Sharding, June 2020, arXiv:2006. 16668 [cs).
[15] Samyam Rajbhandari, Jeff Rasley, Olatunji Ruwase, and Yuxiong He. ZeRO: Memory Optimizations Toward Training Trillion Parameter Models, May 2020. arXiv.1910.02054 [cs].
[16] Sam Ade Jacobs, Masahiro Tanaka, Chengming Zhang, Minjia Zhang, Shuaiwen Leon Song, Samyam Rajbhandari, and Yuxiong He. DeepSpeed Ulysses: System Optimizations for Enabling Training of Extreme Long Sequence Transformer Models, October 2023. arXiv:2309. 14509.
[17] Jiarui Fang and Shangchun Zhao. USP: A Unified Sequence Parallelism Approach for Long Context Generative AI, July 2024. 04X7072405007719 [es].
[18] mit-han-labddisttifusee, May 2025.
[19] XiuYu Zhang. Zening Luo, and Michelle E. Lu. Partially Conditioned Patch Parallelism for Accelerated Diffusion Model Inference, December 2024, arXiv:2412.0299 [es].
[20] Jue Wang. Binhang Yuan, Luka Rimanic, Yongjun He, Tri Dao, Beidi Chen, Christopher Re, and Ce Zhang Fine-tuning Language Models over Slow Networks using Activation Compression with Guarantees. March 2023, arXiv:2206.01 01299 [cs].

---

[21] Yuhan Liu, Hanchen Li. Yihua Cheng, Siddhant Ray, Yuyang Huang, Qizheng Zhang, Kuntai Du, Jiayi Yao. Shan Lu, Ganesh Ananthanarayanan, Michael Maire. Henry Hoffmann, Ari Holtzman, and Junchen Jiang. Cachegen: Ky cache compression and streaming for fast large language model serving. In Proceedings of the ACM SIGCOMM 2024 Conference, ACM SIGCOMM '24, page 38-56, New York, NY, USA, 2024. Association for Computing Machinery.
[22] Sai Praneeth Karimireddy, Quentia Rebjock, Sebastian U. Stich, and Martin Jaggi. Error Feedback Fixes SignGGD and other Gradient Compression Schemes, May 2019, 1xXiv:1901.0900 [cs].
[23] Yujun Lin, Song Han. Huizi Mao, Yu Wang, and William J. Dally. Deep Gradient Compression: Reducing the Communication Bandwidth for Distributed Training, June 2020. arXiv: [es].
[24] Sebastian U. Stich. On Communication Compression for Distributed Optimization on Heterogeneous Data, December 2020, arXiv:2009.0255 [cs, stat].
[25] Rongwei Lu, Jiajun Song, Bin Chen, Laizhong Cui, and Zhi Wang. DAGC: Data-Aware Adaptive Gradient Compression. In JEEE INFOCOM 2023 IEEE Conference on Computer Communications, pages 1-10, May 2023. ISSN: 2641-9874.
[26] G. W. Stewart. Simultaneous iteration for computing invariant subspaces of non oon-hermiiian matrices. Numerische Mathematik, 25:123 136, 1975/76.
[27] Thijs Vogels, Sai Praneeth Karimireddy, and Martin Jaggi. PowerSGD: Practical Low-Rank Gradient Compression for Distributed Optimization, February 2020. 1rXiv:1905.13727 [es].
[28] Aojun Zhou, Yukun Ma. Junnan Zhu, Jianbo Liu, Zhijie Zhang, Kun Yuan, Wenxiu Sun, and Hong- sheng Li. Learning N:M Fine- Fine-grained Structured Sparse Neural Networks From Scratch, April 2021. arXiv:2102.04010 [es].
[29] Zhuoyi Yang, Jiayan Teng, Wendi Zheng. Ming Ding, Shiyu Huang. Jiazheng Xu, Yuanming Yang. Wenyi Hong, Xiaohan Zhang, Guanyu Feng, Cogvideox: Text-to-video diftusion models with an expert transformer. arXiv preprint arXiv:2408. 06072, 2024.
[30] Xinlei Chen, Hao Fang, Tsung- Yi Lin, Ramakrishna Vedantam, Saurabh Gupta, Piotr Dollár, and C. Lawrence Zitmick. Microsoftt COCO captions: Data collection and evaluation server. CoRR, abs/1504.00325, 2015.
[31] Ziqi Huang. Yinan He, Jiashuo Yu, Fan Zhang, Chenyang Si, Yuming Jiang, Yuanhan Zhang, Tianxing Wu, Qingyang Jin, Nattapol Chanpaisit, Yaohui Wang, Xinyuan Chen, Limin Wang, Dahua Lin, Yu Qiao, and Ziwei Liu. Vhench: Comprehensive benchmark suite for video generative models, 2023.
[32] Zhuohan Li, Siyuan Zhuang, Shiyuan Guo, Danyang Zhuo Hao Zhang, Dawn Song, and Ion Stoica. TeraPipe: Toven-Level Pipeline Parallelism for Training Scale Language Models, September 2021. arXiv:2102.07988 [es].
[33] Richard Zhang, Phillip Isola, Alexci A. Efros, Eli Shechtman, and Oliver Wang. The unreasonable effectiveness of deep features as perceptual metric, 2018
[34] Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter. Gans trained by a time-scale update rule converge to local nash equilihrium, 2018.
[35] Zhou Wang. A.C. Bovik, H.R. Sheikh, and E.P. Simoncelli. Image quality assessment: from error visibility to structural similarity, IEEE Transactions on Image Processing, 13(4):600 612, 2004.
[36] Jiarui Fang, Jinzhe Pan, Xibo Sun, Aoyu Li, and Jiannan Wang. xDIT: Inference Engine for Diffusion Transformers (DiTs) with Massive Parallelism, November 2024. arXiv:241 1..1738 [es].
[37] Cheng Lu, Yuhao Zhou, Fan Bao, Jianfei Chen, Chongxuan Li, and Jun Zhu. Dpm-solver: A fast ode solver for diffusion probabilistic model sampling around 10 steps, 2022.
12

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: The claims made in the abstract and introduction accurately reflect the contri- butions. Claims clearly match our system design, empirical results and analysis.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
is fine to include aspirational goals as motivation as long as is clear that these goals not attained by the paper.
2. Limitations
Question:
Does the paper discuss the limitations the work performed by the authors?
Answer: [Yes]
Justification:
The discussion on limitations is provided in Appendix A
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g.. independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't 't acknowledged judgment and recognize that individual actions in favor of transparency play impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
the paper. The authors should use their best
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [Yes]
13

---

Justification:
Assumptions and proof are provided in Section 3.1 and appendix H
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced the statement of any theorems.
The proofs can either appear the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged to provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless whether the code and data are provided or not)?
Question:
Answer: [Yes]
Justification: Our code is publicly released, and we provide full experimental set- tings(Section and appendix Eb.
Guidelines:
The answer NA means that the paper does include experiments.
If the paper includes experiments, No answer this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make it clear how reproduce that algorithm. (b) If the contribution is primarily the architecture clearly and fully.
model architecture, the paper should describe
(c) If the contribution is new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome deseribe the particular way they provide for reproducibility, In the case of closed-source models, it may be that access to the model is limited in some way (e.g., registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
14

---

Answer: [Yes]
Justification: We release our full codebase with detailed instructions, scripts for reproducing results. All used models and datasets are publicly available, and setup details included in the paper.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://nnis..c
for more details.
While we encourage the release of code and data, we understand that this might not be possible, so "No" an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https for more details.
/ //nips.c//pppp
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: All experimental settings, including model setup, diffusion steps, and evalua- tion protocols, are described in the paper(Section 4 and appendix
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, as supplemental material.
7. Experiment statistical significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [No]
Justification: Following prior work, we do not report error bars or statistical significance tests for diffusion inference.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall with given experimental conditions).
15

---

The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
should be clear whether the error bar is the standard deviation or the standard error of the mean.
It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables figures symmetric error bars that would yield results that are out range (c.g. negative emror rates).
If error bars reported in tables plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
Question:
Answer: [Yes]
Justification:
We report GPU types, hardware configurations, and latency for results.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't 't make it into the paper).
9. Code ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of http::/ppgp..............pp//ptt
Answer: [Yes]
Justification: Our research complies with the NeurIPS Code of Ethics.
Guidelines:
The answer NA means that the authors have reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special cireumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., if there is special consid- eration due to laws or regulations in their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification:
The socictal impacts are provided in Appendix T
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
16

---

The conference expects that many papers will be foundational research and not tied particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology being used intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms monitor how system learns from feedback over time, improving the efficiency and accessibility of MI..
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification:
This paper poses such risks.
Guidelines:
The answer NA means that the paper poses such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Lieenses for existing assets
Are the creators or original owners of assets (e.g., code, data, models), used in the paper. properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification: They explicitly mentioned in the paper and further listed in Appendix
Guidelines:
existing assets. The authors should cite the original paper that produced the code package or dataset.
The answer NA means that the paper does
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (c.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (c.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswi thcode. com com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
17

---

If this information is not available online, the authors are encouraged to reach out to the asset's 's creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes]
Justification: Our open-source codebase is well documented, with documentation provided alongside the code.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset used.
At submission time, remember to anonymize your assets (if applicable). You can either create anonymized URL include anonymized zip file.
14. Crowdsourcing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given participants and screenshots, if applicable, as well as details about compensation (if any)?
Answer: [Yes]
Justification: We include the instructions and the example in the appendix. We did employ workers.
Guidelines:
The answer NA means that the paper does involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According to the NeurlPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based the requirements of your country or institution) were obtained?
Answer: [No]
Justification: The study posed no risk to participants, we did not seek IRB approval.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurlPS Code of Ethics and the guidelines for their institution.
For initial submissions, not include any information that would break anonymity (if applicable), such the institution conducting the review.
18

---

16. Declaration of LLM usage
Does the paper describe the usage of LLMs if it is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, or formatting purposes and does not impact the core methodology. scientific rigorousness, originality of the research, declaration is not required.
Question:
Answer: [NA]
Justification: The core method development in this research does not involve LLMs as any important, original, or non-standard components.
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, or non-standard components.
Please refer to our LLM policy (https: /ttpp/////euups..... wwwparippssccooooeeeeeee cc/Conferencess///////eeeecc/eee for what should or should be described.
19

---

A Limitations
While Compact offers strong compression with minimal quality loss, several practical limita- tions remain.
First, compression itself introduces overhead. Although our implementation overlaps compression with communication in Ring Attention, full overlap is not always achievable, especially fast inter- connects with limited idle time. This can reduce net gains. We believe further kernel optimizations and scheduling strategies
help bridge this gap.
Second, while our low-rank variant supports high compression ratio, the subspace iteration it relies on remains slower than quantization or sparsity. This limits its speedup under fast networks, though still valuable for bandwidth-limitee settings like edge deployment.
B
More Preliminaries
B.1 Sequence Parallelism
Sequence parallel enables multiple GPUs to collaboratively generate single image or video sample, thereby reducing the latency per sample. During inference, an image or video frame is first divided into sequence of patches (tokens). Sequence parallel partitions this sequence of patches across multiple devices so that each GPU is responsible for computing a section of it.
After the partition, however, Int communications are required. For FFN layers, communication unnecessary-ean each patch (token) is processed independently, and all required data resides locally on the same device. But for attention layers, every patch must attend to all other patches across the full sequence, which introduces cross-device dependencies.
Different parallel strategies handle this communication differently:
Patch [. This strategy performs single AllGather operation over all devices to collect the complete set of keys and values (Figure 9.. Each GPU thus receives global and V tensors before executing the attention computation. It may incur substantial communication overhead the entire K, V matrices are transmitted every step.
Ring Attention (9). Instead of full AllGather, Ring Attention organizes the devices into logical ring and performs a series of peer-to-peer exchanges. Each stage transmits only shard of K, and computes the corresponding partial result. Computation and communication are interleaved, offering partial overlap.
Ulysses [6]. Ulysses adopts a slightly different design, using two All-to-AB operations to ex- change the required data between devices.
We provide illustrative pseudocode Algorithms and 2
GPU 0
XV (GBUD)
KV (GDU0)
KV (GPU1)
XV (GPU2)
GPU 1
X0V (GDUL)
AllGather
(CPU0)
(CPU1)
XV (G2122)
GPU 2
KV (GPU2)
(GPUO)
(GPUL)
XV (GPU2)
Figure 9: Illustration of Patch Parallel.
Algorithm 1 Patch Parallel
Input: Local tokens/features X, on device
1: Qi, Ki, V4 + ProjectQKY QK((X))
2: Kgathered, V Vgathered f- AllGather(K, Vi)
3: O, < -Atttenion(() Qt, K Katthered, Vgathered)
4: return O;
20

---

Algorithm 2 Ring Attention
Input: Local tokens/features X; on device i, total devices N
1: Qi, K V Project (VX(X)
2: Initialize local output buffer: O + 0
to N do 4: K(s), (*) < RingSendRecv(K,, s) D receive shard from device (i-s) mod N
3: for
5: O < Aggregate(Oi, AttentionParttial(.. K(+), v(1))
6: end for
7: return Og
B.2 Displaced Patch Parallel (DistriFusion)
Patch Parallel performs a synchronous AllGather of K, at every denoising step and waits im- mediately before attention, which leads to substantial blocking when the transfer payload is large. Displaced Patch Parallel 7 avoids this by adopting send-now, receive-later schedule with one- step displacement. In each iteration, the system launches a non-blocking communication for the current K V) but computes attention using the previous step's gathered (K, V), thereby overlapping communication with computation. Each iteration waits on the handle from the earlier launch when those tensors are actually required. This schedule reduces latency the cost using stale keys and values, which can introduce quality drift.
C Discussion
Residual Compression and Displaced Parallelism. Residual compression and displaced paral- lelism are not mutually exclusive. They can potentially be combined to leverage the strengths both. For example, integrating CompactFusion with PipeFusion may allow reducing the memory footprint needed store base tensors for residual computation.
Exploiting Temporal Redundancy. Inspired by excellent prior work DistriFusion and PipeFusion, we revisit the core idea behind temporal redundancy. As the name suggests, this data is largely redundant. Rather than continuing to transmit and try masking its communication cost, we believe the most effective approach is eliminate it entirely from the communication path. CompactFusion follows this principle: it directly discards redundant data, leading to more efficient and scalable parallel diffusion.
not evaluated on video models. To the best of our knowledge, xDiT the most widely used parallel diffusion inference framework, actively maintained and widely adopted in both research and deployment. It originates from the official PipeFusion repository and inherits its core features. However, neither PipeFusion nor xDIT currently support video models such as Cog VideoX. As a result, evaluating PipeFusion in this setting would require substantial reengineering. This is beyond the scope of our work and the capacity of us. We stress that the omission is due to technical constraints rather than intentional bias. Our goal remains fair and comprehensive comparison wherever feasible.
Why PipeFusiie
D Implementation Details
Warmup Implementation. Similar to displaced parallelism, CompactFusion requires at least one warmup step. used initialize the uncompressed base tensor for residual calculation. Ablation studies show that CompactFusion is highly robust and achieves strong generation quality even with single warmup step (Section 4.3). There are two practical ways implement warmup: (1) running the warmup steps with parallelism but without compression; or (2) letting each device execute the warmup locally without any communication. The first method introduces some communication but typically faster on PCIe and NVL.ink. However, it may become bottleneck under extremely slow networks. The second method avoids communication but incurs slightly higher latency due lack of parallelism. We adopt the first method in our experiments.
21

---

We quantize an activation tensor X E RNCC between a quantized sign/value tensor and scale matrix:
Quantization.
using element wise multiplication
Q(X) x(X)
(wv'
where
q X) is the element-wise quantized tensor (e.g., +1 for 1-bit, or (-2, -0.5, +0.5, +2] for 2-bit),
denotes the element-wise product,
u E R E gxx1 form rank-l approximation of the activation magnitude.
In practice, we estimate u and v as:
1 LX
Σ Xiyl. By == N
1
U NC E
This yields a scale matrix where (uw) T) Diy reflects the product of the normalized token-wise and channel-wise means. All quantization operations are implemented in fused end-to-end Triton ker- nels for efficiency. We note that this scale estimation is hcuristic and leaves substantial room for improvement.
Low-Rank Compression. sition of the form:
We aim approximate a matrix A E |_=x with low-rank decompo-
UVT,
where U E Rmxr, V e R"
To compute this approximation efficiently, use 7 7tep subspace iteration 26 subspace iteration was first used in gradient compression 277
Algorithm 3 Subspace Iteration (rank-r)
Input: A E Rm*a target rank r, iterations T
1: Randomly Sample Q and orthonormalize: Q +- orthhooonaliiee(Q (Q).
2: for = 1 to T do
3:
7 - AT (AQ) E Raxr
4: Q + +-oothooonnaiie((2 2)
5: end for
6: U + AQ E R
7: U t orthogonalizo(U)
8: V Q Rax
9: return U, V
Residual Base and Error Feedback. In CompactFusion, each transmitted tensor represents residual, which is the change in activation relative to stored base. The base is initialized with the uncompressed activation during the warm-up step and updated after every iteration. Compression introduces an error that is observable only on the sender side, since the receiver never sees the true residual before compression. Therefore, error feedback is applied at the sender: the previously accumulated error is added back to the next residual before compression, ensuring that lost information gradually compensated over time.
Integration into Parallel Pipelines. CompactFusion integrates into existing parallel frameworks by simply wrapping standard communication primitives such as send/recu and AllGather. This modular design allows the same compression interface to be reused across different parallel strategies, ensuring compatibility with existing diffusion inference pipelines.
E
More Experiment Details
E.1
Human Evaluation
To complement automated perceptual metries (e.g., FID, LPIPS), we conducted a human evaluation study to assess how different parallel inference strategies affect generation quality.
22

---

We distributed an online questionnaire to colleagues and friends, asking them to compare images generated by five different parallelization methods. The goal was identify which results were more visually consistent with the original reference image.
Directly ranking five images per question found be cognitively demanding and time-consuming, which discouraged participation. To simplify the task while still capturing ranking preference, we designed alternative method. We randomly sampled 30 groups of images, and each participant answered 30 questions (15 of each type), with the following two instructions:
Please select the two images that are most similar to the Reference Image.
Please select the images that are least similar the Reference Image.
Each question displayed one reference image (generated by the original uncompressed model) and five test images produced by different parallel strategies (including DistriFusion, PipeFusion, and CompactFusion variants), as shown in Figure I0] The five options were randomly shuffled avoid position bias. The prompts used for generation were randomly sampled from the COCO dataset.
Befeence Image
Figure 10: Example from our human evaluation questionnaire. Participants are asked to select two images most (or least) similar to the reference image. Each set contains five outputs generated by different parallel inference methods, shown in random order.
We collected 20 valid responses in total. For each method, we compute the probability that its output falls into the more consistent subset (either selected as most similar, or not selected as least similar). This reflects how frequently a method produces perceptually closer results to the reference image.
We note that this evaluation involved no personal or sensitive information and posed no potential risks participants.
E.2 More Setups
Models.
The default scheduler used is DPM solver.
Dataset. For evaluation, we randomly sample 5000 prompts from the image validation set and 200 video validation prompts.
23

---

More Quantitative Results
This section provides more quantitative results for ablation study and some experiments we use to elaborate our methods.
F.1 Effect of Tradeoff between Rank and Precision
Table 3: Quantitative evaluation for Compact-Lowrane Pure Lowrank with Iteration 2 and Pure Lowrank with Iteration 10.
Method
PSNR (1)
LPIPS (1)
FID (L)
w/ G.T.
w/ Orig.
w/ G.T.
w/ Orig.
w/ G.T.
w/ Orig.
Compact-Lowe Lowrrnk
9.89
22.85
0.769
0.275
33.07
8.68
Pure Lowrank 12 with Iteration=2
9.86
20.78
0.765
0.340
33.91
12.19
Pure Lowrank 8 with Iteration=2
9.84
20.11
0.762
0.361
34.68
13.97
Pure Lowrank 8 with Iteration=10
9.84
20.10
0.763
0.362
34.53
13.87
Compact-Lowaad uses rank 32 and applies INT4 quantization to the transmitted U and V, matching the communication cost of rank-8 FP16 methods. In contrast, the Pure Lowrank baselines lower ranks (8 or 12) and do not quantize. As shown in Table increasing the subspace iteration steps (from 2 to 10) brings negligible quality gains, while increasing the rank significantly improves generation quality. This highlights the importance of directional coverage over projection precision under tight bandwidth budgets.
F.2 Effect of the Error Feedback
Table Quantitative evaluation for Compact- 1bit with and without Error Feedback mechanism.
Method
PSNR (t)
LPIPS (1)
FID (1)
w/ G.T.
w/ Orig.
w/ G.T.
w/ Orig.
w/ G.T.
w/ Orig.
Compact-
bit (with Error Feedback) 9.80
22.90
0.767
0.260
33.20
7.08
Compact- 1bit (without Error Feedback)
9.57
19.78
0.759
0.389
37.98
19.23
The results (Table 4 show that incorporating across all metrics. Specifically, the Compact- b with Error Feedback variant achieves higher PSNR scores, lower LPIPS values, and notably better FID scores compared the version without error feedback. This indicates that error feedback helps preserve perceptual and structural quality, making it critical component for effective low-bit communication.
feedback significantly improves performance
F33
Effeet of the Warmup steps
Table 5: Quantitative evaluation for WarmUp Steps
Method
PSNR (1)
LPIPS (L)
FID (L)
Latency (s)
w/ Orig.
w/ G.T.
w/ Orig.
w/ G.T.
w/ Orig.
L20
H20
DistriFusion Wp
21.62
0.760
0.309
33.12
9.91
8.05
6.86
DistriFusion WarmUp 2
23.37
0.762
0.261
33.11
8.46
8.38
6.93
Compact-Ibit WarmUp
22.89
0.766
0.259
33,20
7.08
7.46
6.86
Compact- Ibit WarmUp
24.48
0.768
0.219
33.26
6,23
7.63
6.92
Compact-2bit WarmUp 29.54
0.772
0.114
33.09
3.26
7.57
6.70
Compact-2bit WarmUp
31.02
0.772
0.095
33.11
2.78
7.71
6.89
Table 5 demonstrates that longer warm-up phases and higher-bit compression with error feedback improve visual quality.

---

Table 6: Human evaluation results. "Top-2 Count" refers to how often method's output selected as one the most similar images; "Bottom-2 Count" refers to how often it was selected as one the least similar ones. "Final Score" reflects the probability of being placed in the more consistent group.
PipeFusion
DistriFusion
Compact-Lowrank
Compact- Ibit
Compact-2bil
Top-2 Count (1)
150
24
65
103
258
Top-2 Rate (1)
0.50
0.08
0.22
0.34
0.86
Bottom-2 Count (1)
113
173
139
119
56
Bottom-2 Rate ())
0.38
0.58
0.46
0.40
0.18
Final Score (t)
0.56
0.25
0.38
0.47
0.84
E,4 Details on Human Evaluation Results
Compact-2bit method can exhibit high consistency and fidelity with regard to human perception (T)
F.5
Temporal Stability Evaluation
Table Temporal stability evaluation on CogVideoX (6 GPUs) following VBench 31
metrics.
Method
Temporal Flickering (t)
Motion Smoothness (1)
Original (No Compression)
0,9721
0.9817
DistriFusion
0,9658
0,9795
Compact-2bit
0,9644
0.9808
Compact- 1bit
0,9661
0,9806
Compact-
0.9623
0.9775
The results (Table 7 show that all CompactFusion variants maintain temporal stability comparable or better than prior methods. The differences across metrics are minimal, indicating that compression introduces no noticeable degradation in temporal stability.
G More Qualitative Results
This section provides more qualitative results for FLUXX. 1-dev model and CogVideoX model.
G.1
FLUX.1-dev Qualitative Results
The supplementary qualitative results are shown in Table 8
G.2
CogVideoX-2b Qualitative Results
The supplementary qualitative results are shown Table and Table T0
25

---

Sequence Parallelism
DistriFusion
PipeFusion
Compact- Ibit
Compuct-2hit
Compact-L Lowrank
Prompe: Three gons plaving tennis temnts court.
Prompi: parked fromt of filled lugguge
Prompt: in white dress playing temnis.
Prompt: The cut is sizting by the open laptop on the desk
Prompt: An ornate decorated ceramic toilet that is broken at the buse with repoir materials.
Prompt: Two brown cows and some people the backgroand.
Prompt: empty brick buildings carz.
Table 8: Supplement Image Qualitative Results.
26

---

Sequence Parallelism
DistriFusion
Compact-Ibi
Compact-2hit
Compact-I Lowpack
Prungt: deturied wonden ship with intricutely red mosts und mills gluding somutily plush, blue that mimics the noves
The painted
backdray, resembling tomns binting at playful envinomentt captures innovesc inoginntion of chiidheod with the for ship's vournyy yimbolcinne enllers adventures in whimeical, indoar serting
curpet, sigh lestured,
expouse Surmanding ship varimus
speeds up steep mountain stope, lichs up from it's fires, sunlight shines the ar speeds abong the dirt mad. casting ware glow ever the scene. The dirt road curves gently into the dissance with no other cars vehicles sight. The nees on either side of the road are redwoodss with patches of greenery scattered throughont. The ragged The
Prempt: camera follows
vistage SUV
pine on steep
from the following the with making itwel's
drive through the if is wippy
steep
Table 9: Supplement Video Qualitative Results (1/2).
27

---

Prompt: street artist, clod in wore our denine juchet and a colorfol bandonn, stands before a wost concrete wall in the heart, holding a can of spray
spray gointing colorful bind on mutiled woll.
Promet: elderly geatlemom with a werene exprettion, rits water's is edge, steamingg cup of ten by ride. engoased artwork. brunh in hand, as he renders an ari pointing on canser that's propped up against mull, weathered inble. The sea breeze whispers thrugh his rilver hoir gently billeninng his fitinn white shirt, while safty air adds an intongible element to marterpiece progress The wonee trnquilit inspiration,
copturing vilrant
the settng reflecting rasquit
Table 10: Supplement Video Qualitative Results (2/2).
28

---

H
Compression Error Analysis
H.1 Analysis Overview
This section provides detailed derivations for the steady-state expected squared error bounds for the Naive Compression and Residual Compression schemes discussed in the main paper. We also analyze a hypothetical residual compression scheme without error feedback to highlight the importance of the feedback mechanism.
H.2 Definitions
We consider a function ft as mapping from the previous state at step I to the current state at step Let a, denote the activation at step i, ât denote the reconstructed activation after compression at step t and oj denote the activation get using the reconstructed activation from last step. All of these definitions target one specific layer. When warmup step to the first step dj 01. J will be the base that is stored locally in the case of residual compression. The generatl relationship can be expressed as
f((a(-1)
and
a: = ft(ā=11), a.
compress decompeess
dt.
Then, we can use {a to denote ideal, uncompressed sequence that is generated iteratively by the above function.
The total error is defined the sum of the compression error and the propagation error
ã. = (ā, a,)) + (aî az) = + Ttt
where 6 = à of is the compression error and Y = a - at = (+(āt-1) (a(a-)) is the propagation error. In the following derivation, Ant at a:-1 is used to refer the residual of true activation.
H.3 Assumptions
We adopt the following assumptions for error bound analysis: Assumption (L-Smoothness). The function f. L-smooth with L < 1
[[f((x) f((v)l1 < L y/1² Vx, y,
which implies that the process contracts over time, ensuring stability. Assumption 2 (6-Compressor). The compressor C3 satisfies
E [[]]C((=) x||²| < (1 8) [|
8 (0, 1].
where ó measures the compression quality (higher indicates better fidelity).
Assumption 3 (Bounded True Residual Variance).
The true residuals have bounded expected
energy:
Da. [? S 0
where o is finite constant.
Assumption 4 (Bounded Activation Variance). values:
The target activations have bounded expected
E | [
2
where of is a finite constant.
Assumption 5 (Uncorrelated Errors (Simplification).
We assume certain terms uncorrelated in
expectation:
El(Aa, (-1) (true residual uncorrelated with previous total error),
[8, A 0 (perturbation effect uncorrelated with true residual),
29

---

((( = 0 (perturbation effect uncorrelated with previous total error),
Compression error and propagation error are uncorrelated.
Here, 8f. = 100ee-++--11 - is the perturbation effect due to the error in the input fe
H.4 Analysis Model and Scope
Throughout this analysis, we model the evolution of the system state using the reconstructed state from the previous step, i.e., dj fz(ār-1). This means the function Se always operates on potentially noisy state resulting from previous compression steps.
In practical parallel implementations, such as sequence parallelism, a device might compute attention using combination of locally available, uncompressed ("fresh") Key/Value pairs and received Key/Value pairs that have undergone conpression/decoppessoon Our analysis, by assuming the entire input context - has passed through the compression cycle, simplifies the setup and likely provides conservative upper bound the error compared such mixed-context scenarios. The core principles of error propagation and the benefits of compressing smaller residuals, however, remain applicable.
H.S Error Analysis for Naive Compression
Proposition H.1. Let naive denote the steudy-stute mean squared error upper bound under naive compression. Assuming the process has reached stationarity, the error satisfies the bound
(1 - -8)0a
naive
provided L <
1
Proof.
In the naive scheme, a = (a,)) The total error è, ā. ag decomposed into compression error ehaive ā, and propagation error = of = fe(ā:-1) fe(a--1). Assuming these errors are uncorrelated (Assumption
El [| E|]l naive ?
E 1ne1
Using the 5-compressor property (Assumption 2 and the bounded activation variance assumption (Assumption 4 we bound the compression error by
Ellle |] (a()) a; 1 s 0] (1 8)02.
Using the L-smoothness assumption (Assumption T). we can bound the propagation error by
Eill f(),-a( fa(a-1))1 < L a --11 1 L 1 1|
Combining the terms yields the recurrence
|| < (1 5)0 + 1'E
Letting maive as the steady-state error upper bound and assuming || []] = B[ [2]] in steady state, we obtain:
naive
(1 8)0 + maive
Solving for naive gives the bound
(1 - 8)02 1 1
y naive
(requires L < 1),
where stability requires L < 1.
30

---

H.6 Error Analysis for Residual Compression
Proposition H.2. Let residuall denote the steady-state mean squared error upper bound for residual compression. Assuming the process has reached stationarity, the ervor is bounded by
(1 - 8)03
residual
provided L (1-8)(L2 <1]
2 ()(2+11)
Proof.
In the residual scheme, we let Aa; = a; d Sa, = Co(Aaj), and ā = â + Soz. The total error is é = essiiuual + ?/. where eresidual dt of = aaz Anž- Follow the assumption that ef eresidual and ?) are uncorrelated (Assumption 5
get
|i = Billet (ediddll [| Bll
Using the 5-compressor property (Assumption 2
e|llet residual
|2] S (1 - 8))[laaa)
Since
Doï (:(0--1--0--1 = -aaa
- -
where 5ft f + ē - (( and Aa: a a1-1 f((a--1) 01-1. Applying the
uncorrelatedness assumptions (Assumption 5
we can further bound Aa;
Aa [|3] = f+ + [8[ A 11
< L = + 0 + 111l@=1111
(using Assumption and Assumption 3
+ 1)B||e +
Thus, the compression error is bounded by
[[]|e]esiuul] |?] (1 (L) 1)B +
The propagation error term is bounded exactly as in the naive case: [[[lm]|?] < 2 Combining the bounds for compression and propagation errors:
|| < (1 (),2 1) 11 o + I. 1, 1|
Letting residual the steady-state error upper bound and assuming B[|@e 12 - 1 =| in steady state, we obtain:
y residual = (1 (I² + 1) residual + +1-88003 + u residual
Solving yields the closed-form upper bound:
(1- (1-8)03
ey residual
1- 1- (L2 1)'
where stability requires:
I2 (1- +1).
H.7 Analysis Without Error Feedback
Consider hypothetical scheme where the true residual Aa; a, at-1 is compressed and added
the previous reconstruction: à, ă.- 1 + (Aa.). The error é, = 0, a, evolves as:
& (ā.-1 + (a())) - (a,-1 + Aa,) = (āt-1 a)-1) + (Cs (Aa.) - Da.) = ê + ct: where e, (Da,) da, is the compression error on the true residual. If we assume e, are uncorrelated over time, the expected squared error (variance) accumulates:
El|@a 2 -
[|][ez21 [| < [[ + (1 -))0
This indicates that 2 (2] grows linearly with L and does not converge to finite steady state, highlighting the necessity of the error feedback mechanism (implicit in using ă. to compute the target of and the residual Aeū) for stability.
31

---

H.8 Steady-State Error Bounds
We present the derived steady-state expected squared error bounds for naive compression (u=iive) and residual compression with eтror feedback (esiiiiuul) ), based on the given assumptions. The results are as follows.
(1 8 1 2
(1 5)00
naive
residunl
1 -
- (1-
Provided the stability condition for residual compression is satisfied and given that, typically, o o the residual scheme offers lower theoretical error bound (although precise comparison requires evaluating all expressions, including denominators).
<
I License and Asset Attribution
use two open-source implementations in our experiments:
xDiT(xfuser) 10
parallel diffusion inference framework released under the Apache-2.0
license.
DistriPussoon(dissituueer) [8] Released under the MIT license.
Both projects are publicly available, and we have properly credited their creators in the main text. We have respected all terms of their licenses, and all usage is compliant with their respective open-source agreements. No proprietary assets or restricted-use models were used in this work.
All remaining components of CompactFusion, including our compression modules and integration layers, are implemented independently.
Broader Impacts
CompactFusion improves the efficiency of diffusion model inference by reducing communication overhead. This can lower deployment cost and enable generative models to run on a wider range of hardware.
These improvements may help democratize access to generative Al and reduce energy consumption in large-scale deployments. We do not foresee negative societal impacts from this work.
32


Reviews:
[{"review_id": "IHpUVcchZ1", "reviewer": "Reviewer_ZQAB", "summary": "Existing sequence parallelism-based methods enable reducing the sample latency of diffusion models by distributing computation across multiple devices. However, sequence parallelism introduces significant communication overhead, limiting its efficiency. This paper proposes CompactFusion, a framework that compresses the residuals of activations between adjacent denoising steps and introduces error feedback to mitigate cumulative global error. By transmitting compressed residuals instead of the raw activations, the communication cost is significantly reduced. Experiments on image and video generation demonstrate the effectiveness of the proposed method.", "questions": "- In Section 3.2, it is concluded that the error bound of residual compression is significantly smaller than that of naive compression, once we have $\\delta>1-\\frac{1-L^2}{L^2+1}$. Why is this a \"mild condition\"? Is there any quantitative result to support this statement for each compressor (1-bit quantization, 2-bit quantization, and low-rank approximation)?\n- When generating a large number of samples (e.g., for FID evaluation) with several GPUs, will the proposed method be faster than naive sampling (i.e., each GPU generates a portion of samples without any parallel)? \n- Could the authors provide a total execution time of compression, decompression and communication in one single sampling process?\n- Typo: in line 141, ovear -> over; in line 45, paralleli -> parallelism.", "limitations": "Yes", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 2, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths:\n- The idea of compressing the residuals of activations is supported by convincing evidence and is easy to follow.\n- Compared with baseline methods, the proposed method exhibits superior performance on both speedup and sample quality.\n- The proposed method can interact with existing sequence parallel methods easily, especially those communication-heavy methods.\n\nWeaknesses:\n- The writing of the preliminary part is overly simplified and could be improved. The proposed techniques, such as residual compression and error feedback, are applied to Sequence Parallel. However, the exact inference procedures are not introduced in the main content or the supplementary materials, making it challenging to understand the proposed method. Including more details of Sequence Parallel in the preliminary section, or including a more illustrative figure/algorithm to show the entire sampling process, could make the paper more readable.\n- Though enjoying extremely small communication volume, the proposed Compact-Lowrank suffers from limited speedup brought by low-rank approximation. At the same time, its performance does not have an advantage compared to baseline methods, as well as Compact-1bit and Compact-2bit. Which scenarios is Compact-Lowrank suitable for?", "was_revised": true, "final_justification": "I've read the authors’ response and the other reviewers’ comments, and I’m inclined to maintain my weak accept", "rebuttal": "## Thank you for your valuable comments and questions!\nWe thank reviewer ZQAB for recognizing our contributions, including **strong evidence for residual activation compression, superior performance, and easy integration with existing parallel methods.**\nThe reviewer also raised several constructive questions, which we address individually below.\n\n---\n\n# Is Data Parallel (each GPU generates separately) Faster? It Depends.\n> When generating a large number of samples (e.g., for FID evaluation) with several GPUs, will the proposed method be faster than naive sampling (i.e., each GPU generates a portion of samples without any parallelism)?\n\nWe thank the reviewer for raising this insightful question—we are happy to clarify this distinction in more detail.\n\nFirst, we note that the “naive sampling” described by the reviewer—where each GPU independently generates a subset of samples without communication—is **data parallel**.\n\nThe key lies in how we define “faster.” There are two relevant notions:\n\n- **Throughput**: how quickly we can process a large batch of samples end-to-end.\n\n- **Latency**: how long it takes to generate a single sample from prompt to final output.\n\n**Interestingly, if our goal is throughput, which is likely the case in FID evaluation, then data parallelism (generates separately) will always be optimal.** This is not a limitation of CompactFusion; rather, it is a general property: any form of model-parallelism, including Tensor Parallel, Sequence Parallel, DistriFusion, PipeFusion, or CompactFusion—introduces coordination overhead and will typically have lower throughput than pure data parallelism.\n\n**However, if our goal is latency, then CompactFusion is faster.** CompactFusion enables multiple GPUs to collaboratively generate a single sample, reducing this sample's latency. While this introduces some overhead, it allows the sample to complete significantly faster than running it on a single GPU. In contrast, data parallelism cannot reduce latency per sample, since each image is generated independently by one GPU.\n\n**For user-facing applications, such as Sora and Midjourney, latency is often the main problem[1].** It directly determines how long a user waits after submitting a prompt before the final image/video is ready. This is exactly the problem that works such as DistriFusion and PipeFusion aim to tackle: reducing the per-sample latency with collaborative inference across multiple GPUs.\n\nOverall, this is a pretty interesting question that highlights a subtle but important distinction. We appreciate the reviewer for bringing it up.\n\n# Time Breakdown of CompactFusion\n> Could the authors provide a total execution time of compression, decompression and communication in one single sampling process?\n\nWe thank the reviewer for the question and are happy to provide the requested breakdown.\n\n**Table: Component-wise Execution Time on CogVideoX (50 Steps, 6×L20)**\n\n| Component                 | Ring Attn (No Compression) | compact-1bit| compact-2bit| compact-lowrank|\n|--------------------------|-----------------------|--------|-------|----------|\n| Compression              | N/A                   | 0.61s  | 0.60s | 6.21s    |\n| Decompression            | N/A                   | 1.13s  | 1.15s | 1.92s    |\n| Comm Blocking Time       | 18.95s                | 1.05s  | 0.62s | 2.57s    |\n| Total Time           | 52.45s            | 35.74s | 35.81s| 43.58s   |\n\n**Table: Component-wise Execution Time on FLUX (28 Steps, 4×L20)**\n\n| Component             | Ring Attn (No Compression) | compact-1bit | compact-2bit | compact-lowrank |\n|----------------------|----------------------------|--------------|--------------|-----------------|\n| Compression          | N/A          | 0.20s        | 0.19s        | 2.96s           |\n| Decompression        | N/A        | 0.20s        | 0.21s        | 0.36s           |\n| Comm Blocking Time   | 4.07s        | 0.42s        | 0.48s        | 0.60s           |\n| Total Time           | 11.03s        | 7.52s        | 7.64s        | 10.66s          |\n\n\nAdditional notes:\n\n1. We reran the sampling process with a profiler to record detailed breakdowns for compression, decompression, and communication time as requested. As this is a separate run with profiling overhead, the total time shows slight variations compared to our original results. These variations do not affect the conclusions.\n\n2. We observed that after enabling compression, the blocking time becomes very short, but it does not correlate directly with the compression ratio.\nWe hypothesize that this is due to Ring Attn providing overlap, such that after compression is applied, communication can be fully hidden behind computation.\nAs a result, the remaining blocking time is mostly due to startup and teardown overhead of the communication calls, rather than actual data transfer. This explains why the blocking time does not reflect the compression ratio, and instead fluctuates due to unrelated system factors.\n\n# Clarification on Bound Condition\n> In Section 3.2, it is concluded that the error bound of residual compression is significantly smaller than that of naive compression, once we have ...\n\nWe thank the reviewer for raising this question regarding the condition in Section 3.2.\n\nThis condition is a lower bound on compressor quality: unless the compressor is extremely poor, residual compression will yield a smaller error bound than naive compression.\n\nWe call it mild because the tested $L$ is 0.14, which is relatively small; we only require the compressor’s $\\delta$ to be greater than 0.056, a condition easy to satisfy for reasonable compressors. In practice, Compact‑1bit ($\\delta \\approx$ 0.58), Compact‑2bit ($\\delta \\approx$ 0.80), and Compact‑Lowrank ($\\delta \\approx$ 0.26) all satisfy it.\n\nOur experimental results on image quality strongly support this: even when using deliberately poor compressors, such as low‑rank with rank=8, only one‑quarter of the rank used in Compact‑Lowrank, residual compression is still better. In fact, based on all our experiments and various early trials, and to the best of our knowledge, we did not encounter a single case where naive compression performed better.\n\nWe also note that the theoretical bound is derived under an idealized analytical setting to enable tractable analysis. Its role is to provide insight into why residual compression can outperform naive compression. Our main focus, however, is on validating this behavior under realistic settings, where our extensive experiments consistently confirm the theoretical prediction.\n\n# We Will Add More Preliminary on Parallel Strategies and Inference Procedures\n> The writing of the preliminary part is overly simplified and could be improved...\n\n**We thank the reviewer for highlighting this point. We agree that providing more preliminary about the inference procedure of sequence parallelism would help improve clarity and accessibility.**\n\nIn the current version, we did not include a detailed explanation of sequence parallel, as our focus was primarily on the compression mechanism itself. Since **CompactFusion is designed to be parallel-strategy agnostic**, we aimed to keep the core method description decoupled from specific pipelines.\n\nThat said, we agree that additional explanation would benefit readers—especially those less familiar with parallel inference. \n\n**Therefore, in the revision, we will include a dedicated section in the Appendix Preliminaries, providing:**\n\n- an overview of sequence parallelism and inference flow, including how activations are computed and exchanged;\n\n- illustrative figures and pseudocode showing how common parallel strategies (e.g., Ring Attention, Patch Parallel) work in practice;\n\n**In addition, we will add a separate section to clearly illustrate how CompactFusion integrates into these pipelines**, by wrapping communication points without modifying the execution logic.\n\n# Lowrank Compression Has Greater Potential in Constrained Networks\n> Though enjoying extremely small communication volume, the proposed Compact-Lowrank suffers from ... Which scenarios is Compact-Lowrank suitable for?\n\nThank you for this excellent question. We believe Compact-Lowrank plays an important role in two key areas: edge-like low-bandwidth environments and stress-testing the robustness of residual compression. Below, we clarify its motivation.\n\nOn PCIe interconnects, Compact-Lowrank shows limited speedup. This is expected for two reasons:\n(a) A 100× compression ratio is overkill for PCIe bandwidth;\n(b) Despite using subspace iteration for acceleration, low-rank compression is still slower. In this case, the overhead of compression outweighs the communication savings.\n\nHowever, in lower-bandwidth environments such as Ethernet, the benefit of extreme compression becomes more apparent. As shown in Figure 7.\n\n**We believe the lowrank method has greater potential in edge-level or constrained networks, where communication is even slower**. This would require adjusting the warmup strategy (need to use local warmup), which we describe in Appendix C: Warmup Implementation.\n\n**Compact-Lowrank also serves an important validation role to show the robustness of Residual Compression.** Despite its aggressive compression, it maintains better generation quality than DistriFusion (Table 1, 2), demonstrating the robustness of our method even under extreme ratios.\n\nFinally, part of our motivation was to test the boundary of residual compression: can we push to three-digit compression ratios while still outperforming the state-of-the-art overlap-based method in quality? We’re pleased to report that the answer(Table 1) is yes.\n\n---\n**References**\n\n*[1] M. Li et al., “DistriFusion: Distributed Parallel Inference for High-Resolution Diffusion Models”.*"}, {"review_id": "L3Y2ruAtHs", "reviewer": "Reviewer_z8iV", "summary": "This work aims to accelerate parallel diffusion model inference by mitigating communication bottlenecks. The authors focus on the problem of excessive communication overhead caused by exchanging large activation data between devices during distributed training. They address limitations in prior approaches, such as quality degradation from stale activation reuse and insufficient reduction of data volume. The authors propose a compression method that transmits compressed residuals (differences between consecutive steps) instead of full activations, combined with lightweight error feedback to maintain fidelity. The proposed method integrates optimized GPU kernels and system co-design to overlap compression with communication. Evaluations on show that the proposed method brings inference speedups on PCIe/Ethernet with small quality loss and perceptual degradation.", "questions": "Please clarify my questions in the weakness part.", "limitations": "This paper has some merits and I have summarized my major concerns in the weakness part.", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 2, "originality": 2, "strengths_and_weaknesses": "**Strengths:**  \n\nThe authors tackle the communication bottleneck in parallel diffusion model inference. Previous methods, such as reusing stale activations, reduce data volume but degrade generation quality. This paper has pointed out a fundamental challenge of balancing communication efficiency and perceptual quality in diffusion modeling.\n\nThe proposed method leverages residual compression, encoding only the differences between consecutive steps, instead of transmitting full activations between steps. This idea is simple but effective and has been validated by the experiments.\n\n\n**Weaknesses:**  \n\nMostly, I think this paper is interesting. Here are some aspects that could be strengthened. \n\nThe authors transmit compressed residuals rather than full activations and leverage temporal redundancy in diffusion steps. So, the compression operations may introduce additional latency. Although it can be mitigated by overlapping with communication and computations, I’m interested in how to balance this compression-latency problem.", "was_revised": true, "final_justification": "The authors have addressed my major concerns, so I will keep my rating.", "rebuttal": "## Thank you for your constructive comments!\nWe sincerely thank reviewer z8iV for the thoughtful and positive feedback. We’re glad that the core contributions of our work were recognized—**identifying the quality-efficiency tradeoff in parallel diffusion**, **proposing an effective residual compression scheme**, and **validated by empirical results**.\n\nThe reviewer also raised an insightful point regarding the potential latency introduced by compression. We address this concern in detail below.\n\n---\n\n# The Compression Latency Is Indeed Challenging\n\n> The authors transmit compressed residuals rather than full activations and leverage temporal redundancy in diffusion steps. So, the compression operations may introduce additional latency. Although it can be mitigated by overlapping with communication and computations, I’m interested in how to balance this compression-latency problem.\n\n**The reviewer raised a great question about the latency overhead from compression.**\n\nWhile our paper briefly mentions some of the optimizations, we’re glad to have the opportunity to walk through the full picture—**this was actually one of the trickiest parts of the system.**\n\nCompression latency is a real issue. While communication contributes significantly to total inference time, it is highly fragmented, with each individual transfer being very short. On 4×L20 with Ring Attention, a single ring-style communication blocks for less than ~5ms. When compression is enabled, each communication involves one compression and three decompressions. To achieve any meaningful speedup, the **total time for all these operations(1x compress & 3x decompress) must be well under 5ms—leaving very little headroom for overhead.**\n\n# We Tackle the Latency Challenge through Multi-Level Co-Design and Engineering\n\nTo meet that goal, we had to be careful at every level.\n\n **First, we selected only compression strategies that are viable under such tight constraints.** CompactFusion supports sparsity, quantization, and low-rank—but not all variants are usable. We ruled out Top-k sparsity due to its O(n log k) complexity. For low-rank, we use subspace iteration instead of SVD, as SVD is far too slow (appendix C). For quantization, we avoid complex adaptive schemes and use fast, fixed-format quantizers.\n\n**Next came hardware considerations. Some methods that look promising on paper don’t work well on GPU.** Hard-thresholding(keeping all elements above a certain threshold), for instance, creates irregular sparsity patterns that are hard to encode, decode, and parallelize efficiently. Instead, we adopted N:M sparsity, which, while previously unused in activation compression, maps well to GPU hardware due to its block structure. **When we say in Section 3.4 that “it is the only sparsifier design we found to yield practical on-device speedup,” that’s backed by a lot of testing and design.**\n\n **Even with the right algorithms, naive implementations were not enough.** We found that all these compression kernels are **memory-bound**. So we wrote **fused Triton kernels** that combine residual computation - compression - base updates into a single pass. This reduces memory traffic and keeps us within budget. (You may refer to `xfuser/compact/fastpath.py` in our anonymous repo.)\n\n **Finally, we used some system-level tricks.** We integrated compression into the sequence parallel pipeline in a way that allows it to overlap with existing communication.\n\n**That’s why we’re able to get real speedups. It’s a combination of algorithm design, bottleneck analysis, and engineering efforts.**\n\n# Is CompactFusion Hard to Apply Because of These Optimizations? No.\n\nAlthough building a fast compressor was challenging, once we had a working implementation, it became easy to apply across different parallel strategies. Since CompactFusion operates purely at the communication layer, it is agnostic to the specific form of parallelism—all that’s needed is to wrap the communication calls. **This modularity allows the same compression logic to be reused with minimal effort across parallel methods.**\n\n# Measured Compression and Decompression Latency\n\n*Measured latency on L20, Flux:*\n| Latency of a single operation (ms)                | Compact-1bit | Compact-2bit | Compact-Lowrank | Compact-Sparse|\n|-------------------|--------------|--------------|------------------|----------------|\n| Compress FLUX     | 0.06         | 0.06         | 0.93             | 0.05           |\n| Decompress FLUX   | 0.02         | 0.02         | 0.04             | 0.04           |\n| Method | Quantization + Fused Kernel | Quantization + Fused Kernel | Subspace Iteration | N:M Sparsity\n\n---\n\nWe thank the reviewer for pointing out this important issue—compression latency was indeed one of the most challenging aspects of the system, and we’re glad to have the opportunity to elaborate on a part we invested significant effort into."}, {"review_id": "AflFDpVqDD", "reviewer": "Reviewer_ht4v", "summary": "To reduce communication overhead in patch parallel inference, this work proposes a novel Fresh Activation strategy. The method yields a modest speedup while preserving model performance through an error feedback mechanism.", "questions": "please see the weekness", "limitations": "yes", "rating": 4, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 2, "significance": 2, "originality": 3, "strengths_and_weaknesses": "[Strengthen]\n1.The proposed method improves the efficiency of parallel inference by reusing activations through the Fresh mechanism.\n2.The authors provide a theoretical analysis demonstrating that the Fresh activation reuse effectively reduces the approximation error.\n3.The method is integrated with existing toolchains, showing potential for practical deployment in real-world applications.\n\n[Weakness]\n1. The motivation of the work lacks clarity. Since communication and computation are overlapped in practice, compressing activations is typically aimed at reducing communication latency. This motivation implicitly assumes that communication latency completely dominates computation time. The paperdoes not provide any empirical evaluation or profiling to justify this assumption.\n2. The effectiveness of activation compression in improving inference speed remains questionable. DistriFusion adopts a longer warm-up process, while CompactFusion only uses a one-step warm-up, which inherently gives it an advantage in speed. However, as shown in Tables 1 and 2, CompactFusion only yields marginal improvements over DistriFusion, especially when low-rank compression is applied. This raises concerns that transmitting compressed activations may not meaningfully accelerate inference.\n3. The implementation details of patch parallelism in CompactFusion are unclear. Does it appear to be built upon the parallelism strategy of DistriFusion, with additional transmission of the residual for the current patch? The paperlacks a clear description of this strategy, leading to confusion. For example, in Figure 3, what does “base” refer to? Why does the error feedback correct the sender’s “base” instead of that of the receiver? How are all layers of the model executed during parallel inference? A more explicit explanation of the overall parallel procedure is necessary.\n4. The paperclaims that CompactFusion generalizes better than DistriFusion across model architectures and inference strategies. However, the underlying reasons for this improved generalization are not clearly articulated. A detailed analysis of the compatibility mechanisms employed by CompactFusion is needed to substantiate this claim.", "was_revised": true, "final_justification": "Thanks to the author for the Rebuttal. \nI have read it carefully. My concerns are well addressed and I will raise my score.", "rebuttal": "## We sincerely appreciate your feedback and questions!\n\nWe thank reviewer ht4v for recognizing the strengths of our work, including its **efficiency gains, theoretical support, and potential for practical deployment**.\n\nWe also thank the reviewer for raising several insightful points—especially on motivation, performance, and generalization.\n\nWe believe our responses below provide strong evidence that CompactFusion addresses a real and pressing challenge with solid performance advantages.\nFor the concerns on preliminaries, we promise to improve and expand them in the revision.\n\n---\n\n# Communication Optimization Is Necessary in Practice\n> The motivation of the work lacks clarity. Since communication and computation are overlapped in practice, compressing activations is typically aimed at reducing communication latency. This motivation implicitly assumes that communication latency completely dominates computation time. The paper does not provide any empirical evaluation or profiling to justify this assumption.\n\nThis is an excellent point—thank you for raising it. We’re glad to expand on it here.\n\nFirst, we would like to point out that current diffusion model parallelism typically exhibits no or only limited communication-computation overlap:\n\n- DeepSpeed-Ulysses uses synchronous all-to-all communication, **with no overlap**;\n- Patch Parallel uses synchronous all-gather, also **without overlap**;\n- Ring Attention does enable **limited overlap**—each ring-style transmission consists of multiple P2P steps, where each step can overlap partially with a small slice of the attention computation.\n\nUnder these no/limited overlap designs, communication latency becomes a major bottleneck. For instance, even in Ring Attention over 4xL20, we observed that communication blocking accounts for **36%** of total latency in CogVideoX-2b, and **43%** in FLUX, despite that Ring Attention allows limited overlap. In fact, on FLUX, Ring Attention offers almost no improvement over Patch Parallel in practice.\n\n**We note that, contrary to the concern, Section 1 already includes profiling and real‑measurement data that support our motivation.**\n> Line 34: In FLUX.1, standard Patch Parallelism... consuming over **45%** of inference time … underscoring the need to reduce transmission overhead.\n\n**We have also noted that hardware trends would make the communication bottleneck even more critical over time in Section 1:**\n> Line32: interconnect bandwidth has not kept pace with the growth of the compute. From A100 to H100, FP16 FLOPS increases over **6×** (312T → 1,979T), while NVLink bandwidth grows only **1.5×** and PCIe bandwidth simply **doubles**.\n\nWe appreciate the reviewer’s attention to this point, and we emphasize that the motivation is well grounded: **communication latency is substantial, especially in realistic deployment settings**. In fact, this is exactly why a line of prior work—including DistriFusion and PipeFusion—focuses on communication optimization. CompactFusion addresses the same problem but with a fundamentally different approach.\n\n# CompactFusion Achieves Much Better Quality—While Also Being Faster\n> The effectiveness of activation compression in improving inference speed remains questionable. DistriFusion adopts a longer warm-up process, while CompactFusion only uses a one-step warm-up, which inherently gives it an advantage in speed...\n\nWe thank the reviewer for the opportunity to clarify the configuration and performance.\n\nIn our experiments, we use **1-step warmup for all methods, including CompactFusion, DistriFusion, and PipeFusion. This ensures that comparisons are completely fair.** While Figure 1 and Figure 6 indicate the 1-step setting, we acknowledge that Tables 1 and 2 do not repeat this detail—we promise to clarify this in the revision.\n\n**We would also like to emphasize that CompactFusion achieves significantly better generation quality than DistriFusion—while also being faster.**\n\nUnder PCIe on FLUX (Table 1), Compact-2bit achieves a FID of 3.26 vs. DistriFusion’s 9.91—**an over 3× improvement in quality metric, while also being faster** (7.57s vs. 8.05s). Similar trends are observed in CogVideoX.\n\nIn slower networks, our advantage becomes even more pronounced. Under Ethernet-level bandwidth, CompactFusion achieves up to **6.7× speedup over DistriFusion** (Figure 7).\n\nCompactFusion delivers significant improvement over existing methods.\n\n# Clarification on Patch Parallelism Implementation\n> The implementation details of patch parallelism in CompactFusion are unclear. Does it appear to be built upon the parallelism strategy of DistriFusion, with additional transmission of the residual for the current patch? \n\nWe thank the reviewer for highlighting this aspect, and we’re happy to elaborate.\nIn our paper, Patch Parallel refers to a standard sequence parallelism strategy, where devices exchange Key/Value tensors via all-gather. This is the exact communication pattern used in DistriFusion (which builds displaced patch parallelism on top of it).\n\nIn CompactFusion, we apply compression to this all-gather stage—so instead of transmitting full Key/Value tensors, we transmit compressed residuals. The underlying parallel strategy remains unchanged.\n\nWe understand the reviewer’s concern. Since CompactFusion is **parallel-strategy agnostic**, we initially felt it was unnecessary to detail a specific parallel implementation. That said, we agree that providing this information will improve clarity.\n\n**To help clarify this, we promise to include an extra Preliminaries Section in the Appendix that will cover more background and procedural details.** Specifically, we will:\n\n- Include illustrative figures and pseudocode to show how parallel strategies (e.g., Patch Parallel) work in practice;\n\n- Provide a detailed explanation of the DistriFusion to help readers who may be unfamiliar with overlap-based methods better understand its mechanism. \n\n# Clarification on \"base\" and why error feedback occurs on the sender side\n> The paper lacks a clear description of this strategy, leading to confusion. For example, in Figure 3, what does “base” refer to? Why does the error feedback correct the sender’s “base” instead of that of the receiver? How are all layers of the model executed during parallel inference? A more explicit explanation of the overall parallel procedure is necessary.\n\nThe reviewer raised a great question about the residual base and error feedback mechanism.\n\nIn CompactFusion, we transmit residuals—i.e., the change in activation value since the last step. To compute a residual, **one must subtract a reference value—this is what we call the base**. The base is initialized with the uncompressed activation in the warm-up step, and updated over time with compensated error.\n\nWhen you compress a residual, you introduce an error. This compression error is only visible to the sender, because only the sender knows the true residual before compression. **The receiver, on the other hand, only sees the compressed result—it has no way of knowing how much error was introduced.**\n\nTherefore, error feedback can only be performed on the sender side. At each step, the sender adds back the previous error into the new residual, so that missing information is gradually recovered.\n\nWe will add the clarification of this mechanism in Appendix C: Implementation Details in the revision.\n\n# CompactFusion Generalizes More Easily Across Both Models and Strategies\n> Why and how does CompactFusion generalize better than overlap-based methods(DistriFusion, PipeFusion) across models and parallel strategies?\n\nWe’re glad the reviewer brought this up. Generalization is one of CompactFusion’s key strengths, and it is supported by our practical experience.\n\n### Generalization Across Parallel Strategies\n\nAs noted earlier, CompactFusion is agnostic to the specific parallel strategy. It does not modify the model or pipeline logic; instead, **it only wraps the communication calls**. This design allows it to work with more strategies.\n\nBy contrast, overlap-based methods are tightly bound to specific parallel strategies:\n\n- DistriFusion is specific to Patch Parallel.\n- PipeFusion is specific to TeraPipe.\n\n### Generalization Across Model Architectures\n\n**This parallel strategy compatibility directly translates into model-level portability.**\n\nPipeFusion and DistriFusion are tightly bound to specific parallel methods (e.g., TeraPipe, Patch Parallel), which are often not supported by a given model. As a result, applying them to a new model often requires building the entire parallel strategy from scratch.\n\nIn contrast, CompactFusion is modular and does not rely on any specific method. We do not need to implement any specific parallel strategy from scratch. **Instead, we directly plug into the parallel strategy that the model already uses—in practice, this is almost always Ring Attention.**\n\n### CompactFusion Requires Minimal Integration Effort\n\nTo make the difference concrete, we illustrate using our two evaluated models, FLUX and CogVideoX. Here’s what it took to run each method in xDiT[1] (to the best of our knowledge, the most widely used diffusion parallel framework):\n\n- **PipeFusion** – CogVideoX does not have TeraPipe in xDiT. Implementing it requires extensive and complex re‑engineering (FLUX already has TeraPipe-style parallel from the xDiT team.)\n\n- **DistriFusion** – Neither model has Patch Parallel. We implemented it ourselves for both, mainly for experiments. Easier than TeraPipe, but still needs substantial efforts.\n\n- **CompactFusion – wrap the communication with compress&decompress in an already supported strategy (Ring Attention). Model-agnostic, <10 core lines of code.**\n\n---\n**References**\n\n*[1] J. Fang, J. Pan, X. Sun, A. Li, and J. Wang, “xDiT: an Inference Engine for Diffusion Transformers (DiTs) with Massive Parallelism,” Nov. 04, 2024, arXiv: arXiv:2411.01738. doi: 10.48550/arXiv.2411.01738.*"}, {"review_id": "fm5p9B3iVG", "reviewer": "Reviewer_QZGw", "summary": "Diffusion models are computationally intensive and require multi-accelerator parallelism for real-time deployment, but suffer from high communication overhead due to redundant activation exchanges. This paper propose to addresses this by introducing a Residual Compression technique that transmits only compressed residuals, effectively reducing redundancy. This method maintains high generation quality while significantly lowering latency than prior methods.", "questions": "- It would be beneficial to include an additional ablation study comparing the model's performance with and without quantization errors. This analysis would help isolate the impact of compression-induced errors and provide a clearer understanding of how quantization affects the overall performance. It may also help understand the upper bound of the effectiveness of the proposed error feedback.\n- Has the influence of content complexity been examined? It would be helpful to analyze whether the proposed method performs differently on inputs with complex patterns versus simpler ones. \n- The proposed method introduces compression errors. Will it affect the temporal stability of the generated videos. Has this aspect been evaluated?", "limitations": "yes", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "- Strengths\n  - The proposed CompactFusion delivers lower latency parallel diffusion inference, while achieves higher generation quality than prior methods.\n  - Both empirical analysis and theoretical justification are given to show the effectiveness of the proposed method.\n- Weaknesses\n  - I do not see obvious weaknesses. Some small issues are mentioned in Questions.", "was_revised": true, "final_justification": "This paper effectively reduces redundancy while maintaining high generation quality. The authors have provided a clear and satisfactory rebuttal that addresses my previous concerns. Given that, I raise my score.", "rebuttal": "## Thank you for your helpful feedback!\n\nWe sincerely thank reviewer QZGw for the positive assessment of our work. We are glad that the strengths of **low-latency, high-quality parallel diffusion inference**, as well as the combination of **empirical and theoretical validation**, were recognized. The reviewer also raised several constructive questions, and we address them individually below.\n\n---\n# Ablation: Impact of Quantization\n\n> It would be beneficial to include an additional ablation study comparing the model's performance with and without quantization errors. This analysis would help isolate the impact of compression-induced errors and provide a clearer understanding of how quantization affects the overall performance. It may also help understand the upper bound of the effectiveness of the proposed error feedback.\n\n**We thank the reviewer for this helpful suggestion. We agree that isolating the impact of quantization and the benefit of error feedback is valuable. In fact, we have already conducted such an ablation, comparing the following:**\n\n- No Compression (baseline)\n- 1-bit quantization only\n- 1-bit quantization with error feedback\n\nThe baseline numbers (no compression) are included in Table 1 of the main paper, and the rest are reported in Appendix E.2. For convenience, we summarize results below:\n\n**Table: Ablation Study on Quantization and Error Feedback (28 steps, FLUX, 4×L20)**\n\nNote: w/ G.T. refers to metrics computed against ground truth COCO images, while w/ Orig. compares to outputs from the original (uncompressed) model.\n\n| Method                            | Description                            | LPIPS ↓ (w/ G.T.) | LPIPS ↓ (w/ Orig.) | FID ↓ (w/ G.T.) | FID ↓ (w/ Orig.) |\n|----------------------------------|----------------------------------------|-------------------|--------------------|------------------|------------------|\n| Original (No Compression)  | Without quantization error       | 0.772   | –  | 32.75    | – |\n| Compact-1bit (w/o Error Feedback)| With quantization only                 | 0.759             | 0.389              | 37.98            | 19.23            |\n| Compact-1bit (w/ Error Feedback) | With quantization and error feedback   | 0.767             | 0.260              | 33.20            | 7.08             |\n\n**Quantization alone introduces significant degradation (FID 32.75 -> 37.98), but error feedback effectively mitigates it (FID 37.98 -> 33.20).**\n\n# Content Complexity Does Not Affect Our Method\n\n> Has the influence of content complexity been examined? It would be helpful to analyze whether the proposed method performs differently on inputs with complex patterns versus simpler ones.\n\nThank you for raising this point. \n\nIn current state-of-the-art diffusion models like FLUX, the input is textual and padded to a fixed length. As a result, the complexity of the input text has negligible effect on runtime or parallel behavior. What actually affects performance is the resolution of the generated image or video—not the complexity of the prompt.\n\n**Therefore, content complexity does not impact our method, and to our knowledge, prior works such as PipeFusion and DistriFusion have also not addressed this aspect. That said, we appreciate the reviewer for bringing up this dimension, and we will clarify this point in Appendix B: Discussion in the revision.**\n\n# Compression Does Not Affect Temporal Stability\n> The proposed method introduces compression errors. Will it affect the temporal stability of the generated videos. Has this aspect been evaluated?\n\nWe thank the reviewer for raising this point regarding whether compression might harm the temporal consistency of generated videos.\n\nLike prior works such as DistriFusion and PipeFusion, we did not originally include explicit temporal stability metrics. However, we agree this is a helpful addition.\n\nWe followed the VBench[1] to evaluate two standard metrics:\n\n- **Temporal Flickering** (higher is better): Measures brightness/color jitter between consecutive frames.  \n- **Motion Smoothness** (higher is better): Measures motion consistency based on optical flow.\n\nThe setup follows Section 4.2 (CogVideoX, 6 GPUs).\n\n**Table: Temporal Stability Evaluation (VBench, higher = better)**\n\n| Method                 | Temporal Flickering ↑ | Motion Smoothness ↑ |\n|------------------------|------------------------|----------------------|\n| Original (No Compression)  | 0.9721                 | 0.9817               |\n| DistriFusion           | 0.9658                 | 0.9795               |\n| Compact-2bit           | 0.9644                 | 0.9808               |\n| Compact-1bit           | 0.9661                 | 0.9806               |\n| Compact-Lowrank       | 0.9623                 | 0.9775               |\n\nThe differences across methods are minimal, showing that **compression causes no or negligible degradation in temporal stability**.\n\n**We thank the reviewer again for highlighting this aspect, and we will include these results in Appendix E: More Quantitative  Results in the revision.**\n\n---\n\n**References**\n\n*[1] Z. Huang et al., “VBench: Comprehensive Benchmark Suite for Video Generative Models,” in 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Seattle, WA, USA: IEEE, June 2024, pp. 21807–21818. doi: 10.1109/CVPR52733.2024.02060.*"}]

Author rebuttal:

Meta-review: This work is carefully reviewed and received consistent positive feedback (4445). All reviewers appreciate the observation/motivation on redundancy, the novelty of proposed solution and its effectiveness demonstrated on both image and video generation. Through the rebuttal, authors successfully addressed most concerned as acknowledged by reviewers. AC agrees that it does establish a new paradigm for parallel diffusion inference which is known to be quite important to push forward the efficient generation. Considering all the comments and discussions, a decision of acceptance is made. Authors are advised to include missing results raised by reviewers and necessary explanations for a strong and complete final version.
Decision: Accept (poster)