user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Federated Optimization for Heterogeneous Networks

Paper content:
Federated learning involves training and effectively combining machine learning models from distributed partitions of data (i.e., tasks) on edge devices, and be naturally viewed as a multi- task learning problem.
While Federated Averaging (FedAvg) is the leading optimization method for training non-convex models in this setting, its behavior is not well understood in realistic federated settings when the devices/tasks are statistically heterogeneous, i.e., where each device collects data in a non-identical fashion.
In this work, we introduce a framework, called FedProx, to tackle statistical heterogeneity.
FedProx encompasses FedAvg as a special case.
We provide convergence guarantees for FedProx through a device dissimilarity assumption.
Our empirical evaluation validates our theoretical analysis and demonstrates the improved robustness and stability of FedProx for learning in heterogeneous networks.
Large networks of remote devices, such as phones, vehicles, and wearable sensors, generate a wealth of data each day.
Federated learning has emerged as an attractive paradigm to push the training of models in such networks to the edge (McMahan et al., 2017) .
In such settings, the goal is to jointly learn over distributed partitions of data/tasks, where statistical heterogeneity and systems constraints present significant challenges.
Optimization methods that allow for local updating and low participation have become the de facto solvers for federated learning (McMahan et al., 2017; Smith et al., 2017) .
These methods perform a variable number of local updates on a subset of devices to enable flexible and efficient communication.
Of current federated optimization methods, FedAvg (McMahan et al., 2017) has become state-of-the-art for non-convex federated learning.
However, FedAvg was not designed to tackle the statistical heterogeneity which is inherent in federated settings; namely, that data may be non-identically distributed across devices.
In realistic statistically heterogeneous settings, FedAvg has been shown to diverge empirically (McMahan et al., 2017, Sec 3) , and it also lacks theoretical convergence guarantees.
Indeed, recent works exploring convergence guarantees are limited to unrealistic scenarios, where (1) the data is either shared across devices or distributed in an IID (identically and independently distributed) manner, or (2) all devices are active at each communication round (Zhou & Cong, 2017; Stich, 2018; Wang & Joshi, 2018; Woodworth et al., 2018; Yu et al., 2018; Wang et al., 2018) .Due
to the statistical heterogeneity of the data in federated networks, one can think of federated learning as a prime example of distributed multi-task learning, where each device corresponds to a task. However
, the more common goal of federated learning-and the focus of this work-involves training a single global model on distributed data collected for these various tasks. We introduce
and study a novel optimization framework in the federated setting. Our focus on
its convergence behavior in the face of statistically heterogeneous data is closely related to the classical multi-task setting which involves jointly learning task-specific models from statistically heterogeneous data.Contributions. We propose a
federated optimization framework for heterogeneous networks, FedProx, which encompasses FedAvg. In order to
characterize the convergence behavior of FedProx, we invoke a device dissimilarity assumption in the network. Under this
assumption, we provide the first convergence guarantees for FedProx. Finally, we
demonstrate that our theoretical assumptions reflect empirical performance, and that FedProx can improve the robustness and stability of convergence over FedAvg when data is heterogeneous across devices.

assistant: text: We introduce FedProx, a framework to tackle statistical heterogeneity in federated settings with convergence guarantees and improved robustness and stability.