  Recently, Generative Adversarial Networks (GANs) have demonstrated their
potential in federated learning, i.e., learning a centralized model from data
privately hosted by multiple sites. A federatedGAN jointly trains a centralized
generator and multiple private discriminators hosted at different sites. A
major theoretical challenge for the federated GAN is the heterogeneity of the
local data distributions. Traditional approaches cannot guarantee to learn the
target distribution, which isa mixture of the highly different local
distributions. This paper tackles this theoretical challenge, and for the first
time, provides a provably correct framework for federated GAN. We propose a new
approach called Universal Aggregation, which simulates a centralized
discriminator via carefully aggregating the mixture of all private
discriminators. We prove that a generator trained with this simulated
centralized discriminator can learn the desired target distribution. Through
synthetic and real datasets, we show that our method can learn the mixture of
largely different distributions where existing federated GAN methods fail.
