  Federated learning enables multiple parties to collaboratively train a
machine learning model without communicating their local data. A key challenge
in federated learning is to handle the heterogeneity of local data distribution
across parties. Although many studies have been proposed to address this
challenge, we find that they fail to achieve high performance in image datasets
with deep learning models. In this paper, we propose MOON: model-contrastive
federated learning. MOON is a simple and effective federated learning
framework. The key idea of MOON is to utilize the similarity between model
representations to correct the local training of individual parties, i.e.,
conducting contrastive learning in model-level. Our extensive experiments show
that MOON significantly outperforms the other state-of-the-art federated
learning algorithms on various image classification tasks.
