Paris: A Decentralized Trained Open-Weight Diffusion Model
Authors: name: Zhiying Jiang; name: Raihan Seraj; name: Marcos Villagra; name: Bidhan Roy
Published: 2025-10-03 18:53:12

Abstract: We present Paris, the first publicly released diffusion model pre-trained
entirely through decentralized computation. Paris demonstrates that
high-quality text-to-image generation can be achieved without centrally
coordinated infrastructure. Paris is open for research and commercial use.
Paris required implementing our Distributed Diffusion Training framework from
scratch. The model consists of 8 expert diffusion models (129M-605M parameters
each) trained in complete isolation with no gradient, parameter, or
intermediate activation synchronization. Rather than requiring synchronized
gradient updates across thousands of GPUs, we partition data into semantically
coherent clusters where each expert independently optimizes its subset while
collectively approximating the full distribution. A lightweight transformer
router dynamically selects appropriate experts at inference, achieving
generation quality comparable to centrally coordinated baselines. Eliminating
synchronization enables training on heterogeneous hardware without specialized
interconnects. Empirical validation confirms that Paris's decentralized
training maintains generation quality while removing the dedicated GPU cluster
requirement for large-scale diffusion models. Paris achieves this using
14times less training data and 16times less compute than the prior
decentralized baseline.