Quantifying the Impact of Label Noise on Federated Learning
Shuqi (Griffith) Ke1shuqike@link.cuhk.edu.cnProblem

1
2

Chao Huang

2

Xin Liu
3

fchhuang@ucdavis.edu

xinliu@ucdavis.edu

Theoretic analysis

Label noise has been identified in many widely used
datasets. The cause of label noise could be humanerror.
Federated Learning (FL) is a distributed machinelearning paradigm where clients collaboratively traina model using their local (human-generated)datasets.
How does label noise affect FL generalization?

Consider any distributed/decentralized learningalgorithm with a neural network with an arbitrary
structure for a classification task of C classes underlabel noise and cross-entropy loss, thenFormulation

G(W ) ≤ Ω · EX

The first quantitative study that analyzes theimpact of label noise on FL.
An analysis on path-norm growth under
distributed/decentralized setting.
A linear upper bound on the generalization error of
the global model under label noise, which isconsistent with the empirical results.

MethodR 2 = 0.98

Global model accuracy

1.0

Client0.5
1 wp

1.00.5 t 2 wpienlC
1.0
0.0

0.5

Test accuracy

Discussion

1.00.5Regression plane

0.0

1.0

0.0

0.5

1.00.80.6
0.4

Figure 1. Linear regression on the global model accuracy underlabel noise. Two clients trained the global model with FedAvg.

SCAFFOLD
FedNova
FedAvg

0.93
0.92
0.91
0.90
0.00

0.02
0.04
0.06
Wrong label percentage(a) 4-client

0.08

Figure 4. This paper uses ReLU network and path-norm proxy for acase study of the generalization error.

Consider an FL process with a L-layer neural network
dxdyf : R × W → R as its global model. Then its pathnorm increases at most polynomially,

0.915
Global model accuracy

Global model accuracy

0.94

L+10.910

kf (·; θ(t))kpnp = O(t0.905

0.00

0.01

E(L+1)/2)

where t ≤ R denotes the number of communication
rounds and E denotes the local training time.

SCAFFOLD
FedNova
FedAvg

0.900

Ci=1i
nk
k=1 n |Prµ(Y = i|X) − Prπk (Y = i|X)|PN

Contributions

Figure 3. We consider a classification task and ReLU networks.

Label noise linearly degrades FL performance by reducing the test accuracy of the global model.

1.0
0.8
0.6
0.4
0.0
0.5
0.0 1.0hP

If we use a ReLU network as our model in the FL task,
then Ω = kf (·; θ(t))kpnp.

Observation

1.0
0.8
0.6
0.4
0.0

3

0.02 0.03 0.04 0.05
Wrong label percentage

0.06(b) 15-client

If we consider a generic decentralized algorithm, we haveC 0t(L+1)

kf (·; θ(t))kpnp = O(eFigure 2. Test accuracy of global model by different FL algorithms
under different label error rates.

Improving theoretical bounds: We prove a linear
upper bound for the generalization error, and thebound is consistent with numerical results.
However, the upper bound can be loose. One canprovide a lower bound or improve the upper boundby making more restrictive assumptions.
More comprehensive experiments: Ourexperiments use a small number of clients, whichapplies to cross-silo FL. In future research, we plan
to study the impact of label noise with a larger
number of clients (e.g., as in cross-device FL).

E

(L+1)/2)0where C is a constant independent of t, L, E.

The AAAI 2023 Workshop on Representation Learning for Responsible Human-Centric AI, Washington DC