Title: Mitigating Participation Imbalance Bias in Asynchronous Federated Learning Under Client Heterogeneity
Primary area: optimization
Keywords: Asynchronous Federated Learning, Heterogeneity, Theoretical analysis

Abstract: In Asynchronous Federated Learning (AFL), the central server immediately updates the global model with each arriving client’s contribution. As a result, clients perform their local training on different model versions, causing information staleness (delay). In federated environments with non-IID local data distributions, this asynchronous pattern amplifies the adverse effect of client heterogeneity (due to different data distribution, local objectives, etc.), as faster clients contribute more frequent updates, biasing the global model. We term this phenomenon **heterogeneity amplification**. Our work provides a theoretical analysis that maps AFL design choices to their resulting error sources when heterogeneity amplification occurs. Guided by our analysis, we propose **ACE** ( **A**ll-**C**lient **E**ngagement AFL), which mitigates participation imbalance through immediate, non-buffered updates that use the latest information available from all clients. We also introduce a delay-aware variant, **ACED**, to balance client diversity against update staleness. Experiments on different models for different tasks across diverse heterogeneity and delay settings validate our analysis and demonstrate the robust performance of our approaches.

TL;DR: We propose an provably beneficial AFL method that mitigates model bias from faster clients by aggregating the latest available gradient from all clients at every server update.