JOURNAL OF LATEX CLASS FILES, VOL. 14, NO. 8, APRIL 2022

1

Personalized Federated Learning with Adaptive
Batchnorm for Healthcare

arXiv:2112.00734v3 [cs.LG] 20 May 2022

Wang Lu, Jindong Wang, Yiqiang Chen, Senior Member, IEEE, Xin Qin, Renjun Xu,
Dimitrios Dimitriadis, Senior Member, IEEE, and Tao Qin, Senior Member, IEEE
Abstract—There is a growing interest in applying machine learning techniques to healthcare. Recently, federated learning (FL) is
gaining popularity since it allows researchers to train powerful models without compromising data privacy and security. However, theperformance of existing FL approaches often deteriorates when encountering non-iid situations where there exist distribution gaps
among clients, and few previous efforts focus on personalization in healthcare. In this article, we propose FedAP to tackle domain shifts
and then obtain personalized models for local clients. FedAP learns the similarity between clients based on the statistics of the batchnormalization layers while preserving the specificity of each client with different local batch normalization. Comprehensive experiments
on five healthcare benchmarks demonstrate that FedAP achieves better accuracy compared to state-of-the-art methods (e.g., 10%+accuracy improvement for PAMAP2) with faster convergence speed.
Index Terms—Distributed Computing, Federated Learning, Personalization, Batch Normalization, Healthcare.

F

1

I NTRODUCTIONM

ACHINE learning has been widely adopted in many

applications in people’s daily life [1], [2], [3]. Specifically for healthcare, researchers can build models to predict health status by leveraging health-related data, suchas activity sensors [4], images [5], and other health information [6], [7], [8]. To achieve satisfying performance,machine learning healthcare applications often require sufficient client data for model training. However, with theincreasing awareness of privacy and security, more governments and organizations enforce the protection of personal
data via different regulations [9], [10]. In this situation, federated learning (FL) [11] emerges to build powerful machinelearning models with data privacy well-protected.
Personalization is important in healthcare applications
since different individuals, hospitals or countries usually
have different demographics, lifestyles, and other healthrelated characteristics [12], i.e., the non-iid issue (not identically and independently distributed). Therefore, we are
more interested in achieving better personalized healthcare,i.e., building FL models for each client to preserve their
specific information while harnessing their commonalities.
As shown in Fig. 1, there are three different clients A, B ,and C with different statistics of data distributions (e.g., theadult A and the child B may have different lifestyles andactivity patterns). Even if federated learning can performin the standard way, the non-iid issue cannot be easily
•
•
•
•
•
•

Wang Lu and Xin Qin are with University of Chinese Academy of
Sciences and Institute of Computing Technology, Chinese Academy of
Sciences. E-mail: {luwang, qinxin18b}@ict.ac.cn
Jindong Wang and Tao Qin are with Microsoft Research Asia.
E-mail: {jindong.wang, taoqin}@microsoft.com
Yiqiang Chen is with Institute of Computing Technology (CAS) andPengcheng Laboratory. E-mail: yqchen@ict.ac.cn
Renjun Xu is with Zhejiang University. E-mail: rux@zju.edu.cn
Dimitrios Dimitriadis is with Microsoft Research.
E-mail: didimit@microsoft.com
Correspondence to Jindong Wang and Yiqiang Chen.

Manuscript received May 19, 2022.

Cloud Model

Data A

Client A

Data B

Client B……

Data C

Client N

Fig. 1. Data non-iid in federated learning: different clients have different
data distributions.

handled. This will severely limit the performance of existingfederated learning algorithms.
The popular FL algorithm, FedAvg [13], has demonstrated superior performance in many situations [14], [15].
However, FedAvg is unable to deal with non-iid data among
different clients since it directly averages the parameters ofmodels coming from all participating clients [16]. There aresome algorithms for this non-iid situation. FedProx [17] isdesigned for non-iid data. However, FedProx only learns aglobal model for all clients, which means that it is unable
to obtain personalized models for clients. FedHealth [18],another work for personalized healthcare, needs access toa large public dataset, which is often impossible in real applications. FedBN [19] handles the non-iid issue by learning
local batch normalization layers for each client but ignores
the similarities across clients that can be used to boost thepersonalization.
In this article, we propose FedAP, a personalized feder-

JOURNAL OF LATEX CLASS FILES, VOL. 14, NO. 8, APRIL 2022

ated learning algorithm via adaptive batch normalization for
healthcare. Specifically, FedAP learns the similarities amongclients with the help of a pre-trained model that is easy
to obtain. The similarities are determined by the distancesof the data distributions, which can be calculated via thestatistical values of the layers’ outputs of the pre-trained
network. After obtaining the similarities, the server averages
the models’ parameters in a personalized manner and generates a unique model for each client. Each client preserves
its own batch normalization and updates the model with amomentum method. In this way, FedAP can cope with the
non-iid issue in federated learning. FedAP is extensible andcan be deployed to many healthcare applications.
Our contributions are as follows:
1) We propose FedAP, a personalized federated learningalgorithm via adaptive batch normalization for healthcare, which can aggregate the information from different clients without compromising privacy and security,and learn personalized models for each client.
2) We evaluate the performance of FedAP in five public healthcare datasets across time series and image
modalities. Experiments demonstrate that our FedAP
achieves significantly better performance than state-ofthe-art methods in all datasets.
3) FedAP reduces the number of rounds and speeds up
the convergence to some extent. Moreover, some experimental results illustrate FedAP may be able to reduce
communication costs with little performance degradation via increasing local iterations and decreasing global
communications.

2

R ELATED W ORK

2.1

Machine Learning and Healthcare

With the rapid development of perception and computing
technology, people can make use of machine learning tohelp doctors diagnose [20] and assist doctors in the operation [21], etc. Many methods are proposed to monitor
people’s health state [22] and diagnose diseases that may
even have better performance than doctors’, especially inthe field of medical images [23]. Moreover, machine learning
can make disease warnings via daily behavior supervision
with simple wearable sensors [24]. For instance, certain
activities in daily life reflect early signals of some cognitive
diseases. Through daily observation of gait changes andfinger flexibility, the machine can tell people whether they
are suffering from Parkinson [25]. In addition, some studies
worked for better personalization in healthcare [26], [27].
Unfortunately, a successful healthcare application needsa large amount of labeled data of persons. However, in
real applications, data are often separate and few people
or organizations are willing to disclose their private data. In
addition, an increasing number of regulations, such as [9],[10], hold back the leakages of data. These make different
clients cannot exchange data directly, and the scattered data
forms separate data islands, which makes it impossible tolearn a traditional model with aggregated data.
2.2

Federated Learning

Federated learning is a usual way to combine each client’s
information while protecting data privacy and security [11].

2It was first proposed by Google [13], where they proposed
FedAvg to train machine learning models via aggregating
distributed mobile phones’ information without exchanging data. The key idea is to replace direct data exchanges
with model parameter-related exchanges. FedAvg is able toresolve the data islanding problems.
Although federated learning is an emerging field, it has
attracted much attention [28], [29]. Federated learning canbe divided into horizontal federated learning, vertical federated learning, and federated transfer learning according to
the characteristics data. When the client features of the twodatasets overlap a lot but the clients overlap little, horizontal
federated learning can be applied [13]. In the horizontal
federated learning, datasets are split horizontally and theclients share the same features finally. For example, Smithet al. [30] proposed a novel systems-aware optimization
method, MOCHA, to solve security problems in multitasking. When the client features of the two datasets overlap
little but the clients overlap a lot, we can utilize verticalfederated learning, where different clients have different
columns of the features [31]. For example, Cheng et al. [32]proposed a novel lossless privacy-preserving tree-boosting
system known as SecureBoost to jointly conduct over multiple parties with partially common client samples but different feature sets. When the clients and client features of thetwo datasets both rarely overlap, federated transfer learning
is often utilized [33], [34]. For example, Yoon et al. [35]proposed a novel federated continual learning framework,
Federated Weighted Inter-client Transfer (FedWeIT), which
decomposed the network weights into global federated parameters and sparse task-specific parameters. In [35], eachclient received selective knowledge from other clients by
taking a weighted combination of their task-specific parameters. In addition, many methods, such as differential
privacy, are proposed to protect data further [36], [37]. In
this paper, we mainly focus on horizontal federated learningwhen the training data are not independent and identically
distributed (Non-IID) on the clients.
Although FedAvg works well in many situations, it
may still suffer from the non-iid data and fail to build
personalized models for each client [30], [38], [39]. A survey
about federated Learning on non-iid Data can be foundhere [40]. FedProx [17] tackled data non-iid by allowing
partial information aggregation and adding a proximal term
to FedAvg. [41] aggregated the models of the clients withweights computed via L1 distance among client models’parameters. These works focus on a common model sharedby all clients while some other works try to obtain a uniquemodel for each client. [42] exchanged information of baselayers and preserved personalization layer to combat theill-effects of non-iid. [43] utilized Moreau envelopes asclients’ regularized loss function and decoupled personalized model optimization from the global model learning in abi-level problem stylized for personalized FL. [44] evaluatedthree techniques for local adaptation of federated models:fine-tuning, multi-task learning, and knowledge distillation.
[45] also proposed and analyzed three approaches: user
clustering, data interpolation, and model interpolation. [46]tried to jointly learn compact local representations on eachdevice and a global model across all devices with a theoretic
analysis. [47] proposed APFL where each client would train

JOURNAL OF LATEX CLASS FILES, VOL. 14, NO. 8, APRIL 2022

3

𝑤𝑖𝑗

𝑤𝐶

𝑤𝐵

𝑤𝐴

Cloud Model
𝑑𝑖𝑗

Model B

Model A

Data A

Data B

Client A

Client B

Pretrained
Model

Model C

Data C……

Client N

Fig. 2. The structure of the FedAP method.

their local models while contributing to the global model.
Another work [48], Clustered Federated Learning (CFL),
grouped the client population into clusters with jointly
trainable data distributions. Two works most relevant toour method are FedHealth [18] and FedBN [19]. FedHealthneeds to share some datasets with all clients while FedBN
used local batch normalization to alleviate the feature shift
before averaging models. Although there are already someworks to cope with data non-iid, few works pay attention
to feature shift non-iid and other shifts at the same time andobtaining an individual model for each client in healthcare.

denoted as {fi }N
i=1 . Our goal is to aggregate information of
all clients to learn a good model fi for each client on its local
dataset Di without private data leakage:

2.3

There are mainly two challenges for personalized healthcare: data islanding and personalization. Following FedAvg [13] and some other traditional federated learning
methods [52], [53], it is easy to cope with the first challenge.
Personalization is a must in many applications, especiallyin healthcare. It is better to train a unique model in eachclient for personalization. However, one client often lacks
enough data to train a model with high accuracy in federated learning. In addition, clients do not have access to thedata of other clients. Overall, it is a challenge that how toachieve personalization to obtain high accuracy in federated
learning. As mentioned in [50], batch normalization (BN)layers contain sufficient statistics (including mean and standard deviation) of features (outputs of layers). Therefore,
BN has been utilized to represent distributions of training
data indirectly in many works [50], [54]. We mainly use BN
to represent the distributions of clients. Therefore, on the
one hand, we utilize local BN to preserve clients’ feature
distributions. On the other hand, we also use BN-related
statistics to calculate the similarity between clients for better
personalization with weighted aggregation1 .

Batch Normalization

Batch Normalization (BN) [49] is an important componentof deep learning. Batch Normalization improves the performance of the model and has a natural advantage in dealing
with domain shifts. Li et al. [50] proposed an adaptive
BN for domain adaptation where they learned domainspecific BN layers. Nowadays, researchers have exploredmany effects of BN, especially in transfer learning [51].
FedBN [19] is one of few applications of BN in the field of
FL field. However, FedBN does still not make full use of BN
properties, and it does not consider the similarities among
the clients.

3

M ETHOD

3.1

Problem Formulation

In federated learning, there are N different clients (organizations or users), denoted as {C1 , C2 , · · · , CN } andeach client has its own dataset, i.e. {D1 , D2 , · · · , DN }. Eachidataset Di = {(xi,j , yi,j )}n
j=1 contains two parts, i.e. a
ntr

tr
i
train dataset Ditr = {(xtri,j , yi,j )}j=1 and a test dataset

te

ni
N
1 X 1 Xte`(fi (xtemini,j ), yi,j ),N i=1 nte{fk }N
i j=1
k=1

(1)

where ` is a loss function.

3.2

Motivation

nte

te
tr
te
i
Dite = {(xtei,j , yi,j )}j=1 . Obviously, ni = ni + ni andtrte
Di = Di ∪ Di . All of the datasets have different distributions, i.e. P (Di ) 6= P (Dj ). Each client has its own model

1. Please note that we only share the statistics of the batch normalization layers, and no existing work shows that any methods can restore
the specific sample with the statistics of certain layers [55], [56].

JOURNAL OF LATEX CLASS FILES, VOL. 14, NO. 8, APRIL 2022

Model A

FC

FC

pool

BN

Conv

pool

BN

Conv

Data A

4

𝑊𝐴𝐴

New
Model
A

𝑊𝐴𝐵
Model B

FC

FC

pool

BN

Conv

pool

BN

Conv

Data B

𝑊𝐴𝐶……

Model C

FC

FC

pool

BN

Conv

pool

BN

Conv

Data C

…

New
Model
B

…

New
Model
C

Client-specific layer

Shared layer

Fig. 3. The concrete process of the FedAP.

3.3

Our Approach: FedAP

In this paper, we propose FedAP (Adaptive Federated
Learning) to achieve accurate personal healthcare via adaptive batch normalization without compromising data privacy and security. Fig. 2 gives an overview of its structure.
Without loss of generality, we assume there are three clients,
which can be extended to more general cases. The structure
mainly contains five steps:
1) The server distributes the pre-trained model to eachclient.
2) Each client computes statistics of the outputs of specificlayers according to local data.
3) The server obtains the client similarities denoted byweight matrix W to guide aggregation.
4) Each client updates its own model with the local train
data and pushes its model to the server.
5) The server aggregates models and obtains N models
delivered to N clients respectively.
For stability and simplicity, we only calculate W once
and we show that computing once is enough to achieve
acceptable performance in experiments. Note that all processes do not involve the direct transmission of data, so
FedAP avoids the leakage of private data and ensures security. The keys of FedAP are obtaining W and aggregating
the models. We will introduce how to compute W after
describing the process of model aggregation.
We denote the parameters of each model fi as θ i = φi ∪
ψ i , where φi corresponds to the parameters of BN layers
specific to each client and ψ i is the parameters of the otherlayers (colored blocks in Fig. 3). W is an N × N matrix,
which describes the similarities among the clients. wij ∈[0, 1] represents the similarity between client i and client j :the larger wij is, the more similar the two clients are.
Fig. 3 demonstrates the process of model aggregation.
As shown in Fig. 3, φi is particular while ψ i is computed
according to wi , where wi means the i−th row of W, and ψ ,where ψ = {ψ i }N
i=1 . φi is BN parameters that are not sharedacross clients while ψ i is other parameters that are shared.

Algorithm 1 FedAP
Input: A pre-trained model f , data of N clients {Di }N
i=1 , λOutput: Client models {fi }N
i=1
1: Distribute f to each client
2: Each client computes its statistics (µi , σ i ), where µi
represents the mean values while σ i represents thecovariance matrices. Push (µi , σ i ) to the server
3: Compute W according to the statistics
4: Update clients’ model with local data. Push updatedNparameter {θ t∗i }i=1 to the server
t+1 N
5: Update {θ i }i=1 according to Eq. (2) and distributethem to the corresponding clients
6: Repeat steps 4 ∼ 5 until convergence or maximum
round reachedLet θ ti = φti ∪ ψ ti represent the parameters of the modelfrom client i in the round t. After updating θ ti with the local
data from the i−th client, we obtain updated parameters
t∗
t∗
θ t∗
i = φi ∪ ψ i . We use the ∗ notation to denote updated
parameters. Then, for aggregation on the server, we havethe following updating strategy:
(
φt+1
= φt∗
i
PiN
(2)
t+1
t∗
ψi =
j=1 wij ψ j .
The overall process of FedAP is described in Algorithm 1. In the next sections, we will introduce how tocompute the weight matrix W.
3.4

Evaluate Weights

In this section, we will evaluate the weights with a pretrained model f and propose two alternatives to computethe weights. We mainly rely on the feature output statistics of clients’ data in the pre-trained network to computeweights.
We denote with l ∈ {1, 2, · · · , L} in superscript notations the different batch normalization layers in the model.
And zi,l represents the input of l−th batch normalization

JOURNAL OF LATEX CLASS FILES, VOL. 14, NO. 8, APRIL 2022

5

layer in the i−th client. The input of the classification layer
in the i−th client is denoted as zi which represents thei,ldomain features. We assume zi,l is a matrix, zci,l ×si,l where
ci,l corresponds to the channel number while si,l is the
product of the other dimensions. Similarly, zi = zici ×si .
i,l
We feed Di into f , and we can obtain zci,l ×si,l . Obviously,
si,l = e × ni where e is an integer. Now, we try to compute
statistics on the channels, and we treat zi,l as a Gaussian
distribution. For the l−th layer of the i−th client, it is easy
to obtain its distribution, N (µi,l , σ i,l ). We only compute
statistics of inputs of BN layers. And the BN statistics of
the i−th client is formulated as:

input m ean/var

1000

W22 (N (µi,l , σ i,l ), N (µj,l , σ j,l ))=||µi,l − µj,l ||2 +tr(σ i,l + σ j,l − 2((σ i,l )1/2 σ j,l (σ i,l )1/2 )1/2 ),where tr is the trace of the matrix. Obviously, it is costly
and difficult to perform efficient calculations. Similar to BN,
we perform approximations and consider that each channel
is independent of the others. Therefore, σ i,l is a diagonal
matrix, i.e. σ i,l = Diag(ri,l ). Therefore, we compute theapproximation of Wasserstein distance as:

W22 (N (µi,l , σ i,l ), N (µj,l , σ j,l ))√√=||µi,l − µj,l ||2 + || ri,l − rj,l ||22 .

(5)Thus, the distance between two clients i, j is computed
as:di,j =

L
X
l=1

=

L
X

(||µ

i,l

√
√
− µ || + || ri,l − rj,l ||22 )1/2 .

(6)

400
200

200

400

600

800

1000

running_m ean/running_var
Fig. 4. Running mean, running var of a BN layer and the inputs statisticsof the corresponding layer in a client model.

Discussion

In some extreme cases, there may not exist a pre-trainedmodel. In this situation, we can evaluate weights withmodels trained from several rounds of FedBN [19].
As we can see from Fig. 4, the running mean of the BN
layer has a positive correlation with the statistical mean of
the corresponding layer’s inputs. And the variance has asimilar relationship. From this, we can use running means
and running variances of the BN layers instead of the statistics respectively. Therefore, we can perform several roundsof FedBN [19] which preserves local batch normalization,and utilize parameters of BN layers to replace the statistics
when there does not exist a pre-trained model. We denote
this variant as f-FedAP.

4
W2 (N (µi,l , σ i,l ), N (µj,l , σ j,l ))

600

0

3.5
(4)

800

0

(µi , σ i ) = [(µi,1 , σ i,1 ), (µi,2 , σ i,2 ), · · · , (µi,L , σ i,L )]. (3)Now we can calculate the similarity between two clients.
It is popular to adopt the Wasserstein distance to calculate
the distance between two Gaussian distributions:

m ean
var

E XPERIMENTS

We evaluate the performance of FedAP on five healthcare
datasets in time series and image modalities2 . The statisticalinformation of each dataset is shown in TABLE 1.

j,l 2

4.1

l=1

Large di,j means the distribution distance between thei−th client and the j th client is large. Therefore, the largerdi,j is, the less similar the two clients are, which means thesmaller wi,j is. And we set w̃i,j as the inverse of di,j , i.e.
w̃i,j = 1/di,j , j 6= i. Normalize w̃i and we haveŵi,j = PN

w̃i,j

j=1,j6=i w̃i,j, where j 6= i

(7)For stability in training, we take ψ t∗ into account forψ . We update ψ t+1 in a moving average style, and we
set wi,i = λ. Therefore,(
λ,i = j,
wi,j =(8)(1 − λ) × ŵi,j , i 6= j.
t+1

We denote this weighting method as the original FedAP.
Similarly, we can obtain the corresponding W using only
the last layer zi and we denote this variant as d-FedAP.

Datasets

PAMAP2. We adopt a public human activity recognition
dataset called PAMAP2 [57]. The PAMAP2 dataset contains
data of 18 different physical activities, performed by 9
subjects wearing 3 inertial measurement units and a heart
rate monitor. We use data of 3 inertial measurement units
which are collected at a constant rate of 100Hz to form
data containing 27 channels. We exploit the sliding window technique and filter out 10 classes of data3 . In order
to construct the problem situation in FedAP, we use theDirichlet distribution as in [58] to create disjoint non-iid
splits. client training data. Fig. 5(d) visualizes how samples
are distributed among 20 clients. In each client, half of the
2. Codeisreleasedathttps://github.com/jindongwang/andhttps://github.com/
tlbook-code/tree/main/chap19 fl
microsoft/PersonalizedFL.
3. We split PAMAP2 in this style mainly for two reasons. On the
one hand, the data numbers of the subjects are different which may
introduce some other problems, e.g. some clients cannot be adequately
evaluated. On the other hand, this splitting routing is widely adopted
in much work [58], [59]. We select 10 classes with the most samples.

JOURNAL OF LATEX CLASS FILES, VOL. 14, NO. 8, APRIL 2022

5

10

client15205(a) OrganAMNIST

1015

client205(b) OrganCMNIST

1015

client20(c) OrganSMNIST

4

class

3

class

10
9
8
7
6
5
4
3
2
1

11
10
9
8
7
6
5
4
3
2
1

class

11
10
9
8
7
6
5
4
3
2
1

class

class

11
10
9
8
7
6
5
4
3
2
1

6

2
1

5

10

client15520(d) PAMAP210client1520(e) COVID-19

Fig. 5. The number of samples per class allocated to each client (indicated by dot size).

data are used to train and the remaining data are for testing
as in [19].
COVID-19. We also adopt a public COVID-19 posterioranterior chest radiography images dataset [60]. This is acombined curated dataset of COVID-19 Chest X-ray images
obtained by collating 15 public datasets and it contains 9,208
instances of four classes (1,281 COVID-19 X-Rays, 3,270
Normal X-Rays, 1,656 viral-pneumonia X-Rays, and 3,001
bacterial-pneumonia X-Rays) in total. In order to construct
the problem situation in FedAP, we split the dataset similar
to PAMAP2. Fig. 5(e) visualizes how samples are distributed
among 20 clients for COVID-19. Note that this dataset ismore unbalanced in classes which is an ideal testbed totest the performance under label shift (i.e., imbalanced class
distribution for different clients). In each client, half of thedata are used to train and the remaining data are for testing.
MedMnist. MedMNIST [61], [62] is a large-scale MNISTlike collection of standardized biomedical images, including
12 datasets for 2D and 6 datasets for 3D. All images are
28 × 28 (2D) or 28 × 28 × 28 (3D). We choose 3 datasets
which have most classes from 12 2D datasets: OrganAMNIST, OrganCMNIST, OrganSMNIST [63], [64]. These threedatasets are all about Abdominal CT images and all contain11 classes. There are 58,850, 23,660 and 25,221 samples
respectively. As operations in PAMAP2, each dataset is split
into 20 clients with Dirichlet distributions, and Fig. 5(a)-5(c)visualizes how samples are distributed for OrganAMNIST,
OrganCMNIST, and OrganSMNIST respectively. In eachclient, half of the data are used to train and the remaining
data are for testing.

TABLE 1
Statistical information of five datasets.

DatasetType

PAMAP2COVID-19
OrganAMNIST
OrganCMNIST
OrganSMNIST

Sensor-based time series
Image
Image
Image
Image

4.2

#Class#Sample

18
4
11
11
11

3,850,505
9,208
58,85023,66025,221

Implementations Details and Comparison Methods

For PAMAP2, we adopt a CNN for training and predicting.
The network is composed of two convolutional layers, twopooling layers, two batch normalization layers, and two
fully connected layers. For three MedMNIST datasets, we all
adopt LeNet5 [65]. For COVID-19, we adopt Alexnet [66].
We use a three-layer fully connected neural network as
the classifier with two BN layers after the first two fully
connected layers following [19]. For model training, we use
the cross-entropy loss and SGD optimizer with a learning
rate of 10−2 . If not specified, our default setting for local
update epochs is E = 1 where E means training epochs in
one round. And we set λ = 0.5 for our method, since we
can see that λ has few influences on accuracy and it only
affects convergence speeds in the appendix. In addition, we
randomly select 20% of the data to train a model of the samearchitecture as the pre-trained model. We run three trials torecord the average results.
We compare three extensions of our method with fivemethods including common FL methods and some FL methods designed for non-iid data:• Base: Each client uses local data to train its local models

JOURNAL OF LATEX CLASS FILES, VOL. 14, NO. 8, APRIL 2022

7

TABLE 2
Activity recognition results on PAMAP2. Bold and underline mean thebest and second-best results, respectively.

TABLE 4
Accuracy on OrganCMNIST. Bold and underline mean the best andsecond-best results, respectively.

ClientBase

FedAvgFedBN

FedProx

FedPerFedAP

ClientBase

FedAvgFedBN

FedProx

FedPer

FedAP

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
avg

92.86
17.68
100
83.52
18.78
77.66
95.05
17.58
92.39
93.37
29.12
84.78
98.9
24.18
98.91
98.9
41.44
93.62
85.71
37.02
69.07

60.27
62.36
50.56
73.98
74.27
62.9
64.03
87.78
74.49
64.71
65.24
63.35
68.33
64.79
63.12
85.26
66.21
59.64
67.87
72.46
67.58

60.72
62.59
50.34
73.53
74.72
62.44
62.9
88.24
74.27
64.48
65.69
62.9
68.33
65.24
62.44
85.94
65.99
59.64
68.1
72.69
67.56

60.5
62.36
50.34
73.98
73.81
61.76
63.57
87.78
74.27
64.71
66.37
63.12
68.33
65.69
62.44
85.49
66.21
59.41
67.87
72.46
67.52

48.31
97.51
61.4
47.29
58.47
23.98
49.55
91.86
51.24
77.6
89.16
57.92
42.53
49.44
58.6
86.62
77.32
52.38
73.08
97.52
64.59

77.2
77.55
77.43
79.64
81.94
79.86
86.2
95.02
85.33
69.23
91.42
79.41
74.43
69.75
81.67
94.1
82.77
75.74
77.15
86.91
81.14

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
avg

32.61
52.17
47.1
37.23
48.91
51.45
64.49
45.65
26.09
57.25
54.89
50.72
66.67
37.16
58.57
52.9
50
52.9
26.28
59.78
48.64

79.22
95.61
85.83
84.34
92.41
65.6
86.22
50.93
81.28
54.56
79.73
90.56
49.33
74.32
75.93
81.25
67.45
88.18
94.26
90.25
78.36

90.54
100
88.7
96.97
96.46
75.89
86.72
61.38
91.23
93.41
94.76
95.78
54.71
88.34
86.36
98.82
83.81
90.71
100
88.91
88.18

79.73
95.95
85.83
84.01
92.24
65.6
86.55
50.59
80.61
54.9
79.73
90.39
49.16
73.99
75.59
80.91
67.45
88.18
94.26
90.25
78.3

77.87
100
72.85
74.41
84.49
53.96
76.64
38.45
45.03
80.07
83.78
67.28
52.36
70.61
51.52
98.99
70.66
65.37
100
58.49
71.14

94.59
100
93.93
96.3
97.13
85.5
89.08
92.07
86.68
91.89
91.39
89.38
89.73
87.33
89.9
98.82
81.11
91.89
99.83
93.78
92.02

TABLE 3
Accuracy on OrganAMNIST. Bold and underline mean the best andsecond-best results, respectively.

TABLE 5
Accuracy on OrganSMNIST. Bold and underline mean the best andsecond-best results, respectively.

ClientBase

FedAvgFedBN

FedProx

FedPerFedAP

ClientBase

FedAvgFedBN

FedProx

FedPer

FedAP

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
avg

48.35
55.25
34.04
61.54
41.44
52.13
42.31
48.9
38.04
38.12
59.89
59.78
44.75
52.2
53.26
70.88
36.46
46.81
31.32
45.3
48.04

80.03
92.46
86.15
77.65
92.32
84.38
83.42
92.81
74.66
72.27
74.88
78.41
91.17
83.76
89.61
83.31
92.93
77.99
92.05
81.02
84.06

96.06
93.14
96.27
87.91
100
97.55
50.07
97.15
84.1
82.98
90.36
90.56
97.69
92.26
87.84
93.62
62.77
96.94
96.94
91.32
89.28

80.37
92.26
86.22
77.58
92.66
84.92
82.95
92.94
74.39
72.34
74.88
78.28
91.04
83.9
90.42
83.18
92.93
77.92
92.26
80.75
84.11

83.22
78.68
71.08
41.71
100
80.57
64.06
87.86
62.16
53.22
66.33
72.84
78.14
61.28
58.29
73.54
45.92
78.12
93.95
49.36
70.02

81.86
94.5
97.08
88.52
99.93
95.52
87.84
96.95
85.46
87.53
91.79
89.75
97.42
90.29
94.29
91.59
94.97
96.67
96.47
93.97
92.62

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
avg

25.27
32.04
30.85
41.21
37.02
40.43
38.46
40.11
42.39
24.86
38.46
41.3
43.65
44.51
44.57
48.9
36.46
45.74
30.77
35.91
38.15

47.39
55.7
63.13
59.65
68.45
48.02
78.2
57.44
83.73
88.45
60.79
79.15
66.98
84.99
80.7
52.06
36.55
58.7
53.48
72.51
64.8

91.76
96.04
71.84
78.8
80.28
77.02
83.57
53.48
94.94
99.21
45.04
81.67
79.94
95.73
88.77
73.58
78.48
79.43
86.23
72.99
80.44

39.94
62.82
66.3
61.71
73.03
52.3
71.25
41.93
87.99
87.97
37.17
83.57
67.93
86.57
84.49
53.64
39.56
60.44
61.87
77.57
64.9

85.74
94.78
52.69
64.56
68.61
58
78.2
96.52
92.42
97.47
54.49
78.04
58.29
84.52
73.1
71.36
61.55
69.3
64.72
86.57
74.55

90.65
93.51
73.89
75.32
83.12
80.67
85.47
96.2
94.94
99.37
76.06
88.47
80.57
92.58
87.34
73.58
76.42
79.59
91.46
68.4
84.38

4.3

Classification Accuracy

The classification results for each client on PAMAP2 areshown in TABLE 2. From these results, we have the following observations: 1) Our method achieves the best resultson average. It is obvious that our method significantly

96
94
92
90
88
86
84
82
80 FedAvg

Accuracy

without federated learning.
• FedAvg [13]: The server aggregates all client models
without any particular operations for non-iid data.
• FedProx [17]: Allow partial information aggregation
and add a proximal term to FedAvg.
• FedPer [42]: Each client preserves some local layers.
• FedBN [19]: Each client preserves the local batch normalization.

FedBN

Fedprox

FedPerOurs

Fig. 6. Average accuracy of 20 clients on COVID-19.

outperforms other methods with a remarkable improve-

JOURNAL OF LATEX CLASS FILES, VOL. 14, NO. 8, APRIL 2022

92.5

81.14

90.0

75

87.5

70 67.56
65

85.0

60

82.26

PAMAP2COVID-19

82.5

80
70
60

Weight ing
w./o Weight ing

50

80.0

5

10

15

82

(a) The weighting technique

94.5

94.21
78.10

94.0
93.5

76
PAMAP2

20

95
90

80
75
70
65

93.0

COVID-19(b) Different clients

LBN
SBN

85

5

10

15

20

client num ber

dat asetclient num ber

dat aset

94.97 95.0

81.14

80
78

95.5

accuracy (%)

80

96.0

Sharing Com m on BN
Preserving local BN

84

Average Accuracy (%)

85

95.0

90

accuracy (%)

90

97.5
94.97

Average Accuracy (%)

100.0
w./o Weight ing
Weight ing

95

Average Accuracy (%)

Average Accuracy (%)1008(c) The local BN sharing(d) Client Acc on PAMAP2

Fig. 7. The effects of weighting and preserving local batch normalization. Each point has equal status in Fig. 7(b) and Fig. 7(d). We use a line chart
just for better visualization effects but not trends.

OrganC

Dataset(a) Dataset Split

85
80
75
70
65
60FedAvgFedBN Ours d-Ours f-Ours

OrganS

local iter - total rounds

Accuracy

Fedavg-0.1
Fedavg-0.05
Ours-0.1
Ours-0.5

OrganA

90
85
80
75
70
FEDBN
65
Ours
1-400
5-100

accuracy (%)

Accuracy

85
75
65
55

10-50

(b) Iteration-Round(a) PAMAP2

Fig. 8. Influence of Dataset split and Iteration-Round.

The classification results for each client on three MedMNIST datasets are shown in TABLE 3, 4, 5 respevtively.
From these results, we have the following observations: 1)
Our method significantly outperforms other methods witha remarkable improvement (over 3.5% on average). 2) For
all these three benchmarks, Base achieves the worst average
accuracy, which demonstrates Base without communicatingwith each other does not have enough information for these
relatively difficult tasks. 3) FedBN achieves the second best
results on all three benchmarks. This could be because that
there exist feature shifts among clients.
The classification results for each client on COVID-19 areshown in Fig. 6. From these results, we have the following
observations: 1) Our method achieves the best average accuracy which outperforms the second-best method FedPer
by 6.3% on average accuracy. 2) FedBN gets the worst
results. This demonstrates that FedBN is not good at dealing
with label shifts where label distributions of each clientare different, which is a challenging situation. FedBN does
not consider the similarities among different clients. From
Fig. 5(e), we can see that label shifts are serious in COVID19 since it only has four classes.

Accuracy

ment (over 10% on average). 2) In some clients, the basemethod achieves the best test accuracy. As it can be seenfrom Fig. 5(d), the distributions on the clients are very
inconsistent, which inevitably leads to the various difficultylevels in different clients. And some distributions in thecorresponding clients are so easy that only utilizing the local
data can achieve the ideal effects. 3) FedBN does not achieve
the desired results. This could be caused by that FedBN isdesigned for the feature shifts while our experiments aremainly set in the label shifts.

98
96
94
92
90
88
86
84
82
80FedAvgFedBN Ours d-Ours f-Ours(b) COVID

Fig. 9. Evaluating two variants of FedAP.

4.4

Analysis and discussion

We consider the influence of data splits and local iterationsin this section. As shown in Fig. 8(a), we evaluate Fedavg
and FedAP on three MedMNIST benchmarks with two
different splits: α = 0.1 and α = 0.05 respectively. Smaller αmeans distributions among clients are more different fromeach other. Fig. 8(a) demonstrates that the performance ofFedavg which does not consider data non-iid will dropwhen encountering clients with greater different distributions while our method is not affected much by the degreeof data non-iid, which means our method may be morerobust. Fig. 8(b) shows the influence of local iterations
and total rounds on FedBN and our method. It is obviousthat FedBN drops seriously with more local iterations andfewer communication rounds while our method declines
slowly, which means when limiting communication costs,
our method may be more effective.
4.5

Ablation Study

Effects of Weighting.
To demonstrate the effect of weighting which considers
the similarities among the different clients, we compare

FedBN
FedAvg
Ours

75

70

10

20

50

100

Client Num ber(a) Client Number

80
75
70
65
FedBN
FedAvg
Ours

60
55
1

3

5

10

80

80

78

70

76

Ours
FedBN

74
72

60

0.1
0.3
0.5
0.7
0.9

50
40

70

30

68

20

Acc (%)

80

9

Average Accuracy (%)

Average Accuracy (%)

Average Accuracy (%)

JOURNAL OF LATEX CLASS FILES, VOL. 14, NO. 8, APRIL 2022

0.1

0.3

0.5

Local Epochs

λ(b) Local Epochs

(c) λ

0.7

0.9

0

50 100 150 200 250 300 350 400

Rounds(d) λFig. 10. Parameter sensitivity analysis.

4.6

Convergence and Parameter Sensitivity

We study the convergence of our method. From Fig. 11, we
can see our method almost convergences in the 10th round.
And in the actual experiments, 20 rounds are enough forour method while FedBN needs over 400 rounds.
Then, we evaluate the parameter sensitivity of FedAP.
Our method is affected by three parameters: local epochs,
client number, and λ. We change one parameter and fix theother parameters.
From Fig. 10(a), we can see that our method still achieves
acceptable results. When the client numbers increase, ourmethod goes down which may be due to that few data in

80

Average Acc (%)

the average accuracy on PAMAP2 and COVID-19 between
the experiments with it and without it. Without weighting,our method degenerates to FedBN. From Fig. 7(a), we cansee that our method performs much better than FedBN
which does not include the weighting part. Moreover, from
Fig. 7(b), we can see our method performs better than
FedBN on all clients. These results demonstrate that our
method with weighting can cope with the label shifts while
FedBN cannot deal with this situation, which means our
method is more applicable and effective.
Effects of Preserving Local Batch Normalization.
We illustrate the importance of preserving local batchnormalization. Fig. 7(c) shows the average accuracy
between the experiments with preserving local batchnormalization and the experiments with sharing common
batch normalization while Fig. 7(d) shows the results oneach client. LBN means preserving local batch normalization
while SBN means sharing common batch normalization.
Obviously, the improvements are not particularly significant
compared with weighting. This may be caused by there
mainly exist the label shifts in our experiments while
preserving local batch normalization is for the feature
shifts. However, our method still has a slight improvement,
indicating its superiority.
Different Implementations of Our methods.
In Method section, we propose three implementationsof our method: FedAP, d-FedAP, and f-FedAP. The maindifferences among them are how to calculate W. In Fig. 9(a)and Fig. 9(b), we can see that all three implementations
achieve better average accuracy on both PAMAP2 and
COVID compared with FedAvg and FedBN. In addition, fFedAP performs slightly worse than the other two variants,
which may be because it only utilizes weighting during halfrounds for fairness and the other half are for obtaining W.

70
60
50
40
FedBN
FedAvg
Ours

30
20
10

0

50 100 150 200 250 300 350 400

RoundsFig. 11. Convergence analysis of different methods.

local clients make the weight estimation inaccurate. And we
may take f-FedAP instead. In Fig. 10(b), we can see ourmethod is the best and it is descending with local epochs
increasing, which may be caused that we keep the total
number of the epochs unchanged and the communication
among the clients are insufficient. Fig. 10(c)-10(d) demonstrates λ slightly affects the average accuracy of our method
while it can change the convergence rate. The results revealthat FedAP is more effective and robust than other methods
under different parameters in most cases.

5

C ONCLUSIONS AND F UTURE W ORK

In this article, we proposed FedAP, a weighted personalized
federated transfer learning algorithm via batch normalization for healthcare. FedAP aggregates the data from different organizations without compromising privacy and security and achieves relatively personalized model learning
through combing considering similarities and preserving
local batch normalization. Experiments have evaluated theeffectiveness of FedAP. In the future, we plan to apply
FedAP to more personalized and flexible healthcare. And
we will consider better ways to calculate and update similarities among clients.

ACKNOWLEDGEMENT
This work is supported by the National Key Research andDevelopment Plan of China No.2021YFC2501202, Natural
Science Foundation of China (No. 61972383, No. 61902377,
No. 61902379), Beijing Municipal Science & Technology
Commission No.Z211100002121171.

JOURNAL OF LATEX CLASS FILES, VOL. 14, NO. 8, APRIL 2022

R EFERENCES[1]

S. Sharma, V. Elvira, E. Chouzenoux, and A. Majumdar, “Recurrent dictionary learning for state-space models with an applicationin stock forecasting,” Neurocomputing, vol. 450, pp. 1–13, 2021.
[2] B. Brattoli, J. Tighe, F. Zhdanov, P. Perona, and K. Chalupka,“Rethinking zero-shot video classification: End-to-end training forrealistic applications,” in Proceedings of the IEEE/CVF Conference on
Computer Vision and Pattern Recognition, 2020, pp. 4613–4623.
[3] W. Lu, J. Wang, and Y. Chen, “Local and global alignments
for generalizable sensor-based human activity recognition,” in
ICASSP 2022-2022 IEEE International Conference on Acoustics, Speechand Signal Processing (ICASSP). IEEE, 2022, pp. 3833–3837.
[4] W. Lu, Y. Chen, J. Wang, and X. Qin, “Cross-domain activity
recognition via substructural optimal transport,” Neurocomputing,vol. 454, pp. 65–75, 2021.
[5] Z. Fei, E. Yang, D. D.-U. Li, S. Butler, W. Ijomah, X. Li, and
H. Zhou, “Deep convolution network based emotion analysis
towards mental health care,” Neurocomputing, vol. 388, pp. 212–
227, 2020.
[6] E. Choi, C. Xiao, J. Sun, and W. F. Stewart, “Mime: Multilevel
medical embedding of electronic health records for predictive
healthcare,” vol. 2018, pp. 4547–4557, 2018.
[7] A. B. Ünal, M. Akgün, and N. Pfeifer, “Escaped: Efficient secure
and private dot product framework for kernel-based machinelearning algorithms with applications in healthcare,” in Proceedings of the AAAI Conference on Artificial Intelligence, vol. 35, no. 11,
2021, pp. 9988–9996.
[8] W. Lu, J. Wang, Y. Chen, S. Pan, C. Hu, and X. Qin, “Semanticdiscriminative mixup for generalizable sensor-based cross-domain
activity recognition,” Proceedings of the ACM on Interactive, Mobile,
Wearable, and Ubiquitous Technologies, 2022.
[9] N. Inkster, China’s cyber power. Routledge, 2018.
[10] P. Voigt and A. Von dem Bussche, “The eu general data protection regulation (gdpr),” A Practical Guide, 1st Ed., Cham: Springer
International Publishing, vol. 10, p. 3152676, 2017.
[11] Q. Yang, Y. Liu, T. Chen, and Y. Tong, “Federated machine learning: Concept and applications,” ACM Transactions on Intelligent
Systems and Technology (TIST), vol. 10, no. 2, pp. 1–19, 2019.
[12] J. Xu, B. S. Glicksberg, C. Su, P. Walker, J. Bian, and F. Wang, “Federated learning for healthcare informatics,” Journal of Healthcare
Informatics Research, vol. 5, no. 1, pp. 1–19, 2021.
[13] B. McMahan, E. Moore, D. Ramage, S. Hampson, and B. A. y Arcas, “Communication-efficient learning of deep networks fromdecentralized data,” in Artificial Intelligence and Statistics. PMLR,
2017, pp. 1273–1282.
[14] T. Ching, D. S. Himmelstein, B. K. Beaulieu-Jones, A. A. Kalinin,
B. T. Do, G. P. Way, E. Ferrero, P.-M. Agapow, M. Zietz, M. M.
Hoffman et al., “Opportunities and obstacles for deep learning in
biology and medicine,” Journal of The Royal Society Interface, vol. 15,
no. 141, p. 20170387, 2018.
[15] W. Zhu, L. Xie, J. Han, and X. Guo, “The application of deeplearning in cancer prognosis prediction,” Cancers, vol. 12, no. 3,
p. 603, 2020.
[16] X. Li, K. Huang, W. Yang, S. Wang, and Z. Zhang, “On theconvergence of fedavg on non-iid data,” in International Conference
on Learning Representations, 2019.
[17] T. Li, A. K. Sahu, M. Zaheer, M. Sanjabi, A. Talwalkar, and
V. Smith, “Federated optimization in heterogeneous networks,”
in Proceedings of Machine Learning and Systems 2020, MLSys 2020,
Austin, TX, USA, March 2-4, 2020. mlsys.org, 2020.
[18] Y. Chen, X. Qin, J. Wang, C. Yu, and W. Gao, “Fedhealth: Afederated transfer learning framework for wearable healthcare,”IEEE Intelligent Systems, vol. 35, no. 4, pp. 83–93, 2020.
[19] X. Li, M. Jiang, X. Zhang, M. Kamp, and Q. Dou, “Fedbn: Federated learning on non-iid features via local batch normalization,”in International Conference on Learning Representations (ICLR), 2021.
[20] Z. Chen, J. Zhang, S. Che, J. Huang, X. Han, and Y. Yuan,“Diagnose like a pathologist: Weakly-supervised pathologist-tree
network for slide-level immunohistochemical scoring,” in 35th
AAAI Conference on Artificial Intelligence (AAAI-21). AAAI Press,
2021, pp. 47–54.
[21] I. Avellino, G. Bailly, G. Canlorbe, J. Belgihti, G. Morel, and M.-A.
Vitrani, “Impacts of telemanipulation in robotic assisted surgery,”in Proceedings of the 2019 CHI Conference on Human Factors in
Computing Systems, 2019, pp. 1–15.

10[22] G. Muhammad, M. S. Hossain, and N. Kumar, “Eeg-based pathology detection for home health monitoring,” IEEE Journal on Selected Areas in Communications, vol. 39, no. 2, pp. 603–610, 2020.
[23] W. Ji, S. Yu, J. Wu, K. Ma, C. Bian, Q. Bi, J. Li, H. Liu, L. Cheng, andY. Zheng, “Learning calibrated medical image segmentation via
multi-rater agreement modeling,” in Proceedings of the IEEE/CVF
Conference on Computer Vision and Pattern Recognition, 2021, pp.
12 341–12 351.
[24] F. Sun, W. Zang, R. Gravina, G. Fortino, and Y. Li, “Gait-based
identification for elderly users in wearable healthcare systems,”
Information fusion, vol. 53, pp. 134–144, 2020.
[25] Y. Chen, X. Yang, B. Chen, C. Miao, and H. Yu, “Pdassist: Objective
and quantified symptom assessment of parkinson’s disease via
smartphone,” in 2017 IEEE International Conference on Bioinformatics and Biomedicine (BIBM). IEEE, 2017, pp. 939–945.
[26] F. R. Vogenberg, C. I. Barash, and M. Pursel, “Personalized
medicine: part 1: evolution and development into theranostics,”
Pharmacy and Therapeutics, vol. 35, no. 10, p. 560, 2010.
[27] I. Chung, S. Kim, J. Lee, K. J. Kim, S. J. Hwang, and E. Yang, “Deepmixed effect model using gaussian processes: A personalized
and reliable prediction for healthcare,” in Proceedings of the AAAI
Conference on Artificial Intelligence, vol. 34, no. 04, 2020, pp. 3649–
3657.
[28] S. Abdulrahman, H. Tout, H. Ould-Slimane, A. Mourad, C. Talhi,
and M. Guizani, “A survey on federated learning: The journeyfrom centralized to distributed on-site learning and beyond,” IEEE
Internet of Things Journal, vol. 8, no. 7, pp. 5476–5497, 2020.
[29] C. Zhang, Y. Xie, H. Bai, B. Yu, W. Li, and Y. Gao, “A survey onfederated learning,” Knowledge-Based Systems, vol. 216, p. 106775,
2021.
[30] V. Smith, C.-K. Chiang, M. Sanjabi, and A. Talwalkar, “Federated
multi-task learning,” in Proceedings of the 31st International Conference on Neural Information Processing Systems, 2017, pp. 4427–4437.
[31] S. Hardy, W. Henecka, H. Ivey-Law, R. Nock, G. Patrini, G. Smith,and B. Thorne, “Private federated learning on vertically partitioned data via entity resolution and additively homomorphic
encryption,” arXiv preprint arXiv:1711.10677, 2017.
[32] K. Cheng, T. Fan, Y. Jin, Y. Liu, T. Chen, D. Papadopoulos, andQ. Yang, “Secureboost: A lossless federated learning framework,”
IEEE Intelligent Systems, 2021.
[33] Y. Liu, Y. Kang, C. Xing, T. Chen, and Q. Yang, “A secure federated
transfer learning framework,” IEEE Intelligent Systems, vol. 35,
no. 4, pp. 70–82, 2020.
[34] C. He, M. Annavaram, and S. Avestimehr, “Group knowledgetransfer: Federated learning of large cnns at the edge,” vol. 33,
2020.
[35] J. Yoon, W. Jeong, G. Lee, E. Yang, and S. J. Hwang, “Federated
continual learning with weighted inter-client transfer,” in International Conference on Machine Learning. PMLR, 2021, pp. 12 073–
12 086.
[36] C. Dwork, “Differential privacy: A survey of results,” in International conference on theory and applications of models of computation.
Springer, 2008, pp. 1–19.
[37] N. Agarwal, A. T. Suresh, F. Yu, S. Kumar, and H. B. McMahan, “cpsgd: communication-efficient and differentially-private
distributed sgd,” in Proceedings of the 32nd International Conference
on Neural Information Processing Systems, 2018, pp. 7575–7586.
[38] M. Khodak, M.-F. F. Balcan, and A. S. Talwalkar, “Adaptive
gradient-based meta-learning methods,” in Advances in Neural
Information Processing Systems, vol. 32, 2019, pp. 5917–5928.
[39] Z. Zhu, J. Hong, and J. Zhou, “Data-free knowledge distillation
for heterogeneous federated learning,” in Proceedings of the 38th
International Conference on Machine Learning, ICML 2021, 18-24 July
2021, Virtual Event, vol. 139. PMLR, 2021, pp. 12 878–12 889.
[40] H. Zhu, J. Xu, S. Liu, and Y. Jin, “Federated learning on non-iid
data: A survey,” arXiv preprint arXiv:2106.06843, 2021.
[41] Y. Yeganeh, A. Farshad, N. Navab, and S. Albarqouni, “Inverse
distance aggregation for federated learning with non-iid data,” inDomain Adaptation and Representation Transfer, and Distributed andCollaborative Learning. Springer, 2020, pp. 150–159.
[42] M. G. Arivazhagan, V. Aggarwal, A. K. Singh, and S. Choudhary,“Federated learning with personalization layers,” arXiv preprint
arXiv:1912.00818, 2019.
[43] C. T Dinh, N. Tran, and T. D. Nguyen, “Personalized federated
learning with moreau envelopes,” in Advances in Neural Information Processing Systems, vol. 33, 2020.

JOURNAL OF LATEX CLASS FILES, VOL. 14, NO. 8, APRIL 2022

[44] T. Yu, E. Bagdasaryan, and V. Shmatikov, “Salvaging federated
learning by local adaptation,” arXiv preprint arXiv:2002.04758,
2020.
[45] Y. Mansour, M. Mohri, J. Ro, and A. T. Suresh, “Three approaches
for personalization with applications to federated learning,” arXiv
preprint arXiv:2002.10619, 2020.
[46] P. P. Liang, T. Liu, L. Ziyin, N. B. Allen, R. P. Auerbach, D. Brent,
R. Salakhutdinov, and L.-P. Morency, “Think locally, act globally:
Federated learning with local and global representations,” arXiv
preprint arXiv:2001.01523, 2020.
[47] Y. Deng, M. M. Kamani, and M. Mahdavi, “Adaptive personalized
federated learning,” arXiv preprint arXiv:2003.13461, 2020.
[48] F. Sattler, K.-R. Müller, and W. Samek, “Clustered federated learning: Model-agnostic distributed multitask optimization under privacy constraints,” IEEE transactions on neural networks and learning
systems, vol. 32, no. 8, pp. 3710–3722, 2020.
[49] S. Ioffe and C. Szegedy, “Batch normalization: Accelerating deep
network training by reducing internal covariate shift,” in International conference on machine learning. PMLR, 2015, pp. 448–456.
[50] Y. Li, N. Wang, J. Shi, X. Hou, and J. Liu, “Adaptive batch normalization for practical domain adaptation,” Pattern Recognition,
vol. 80, pp. 109–117, 2018.
[51] M. Segù, A. Tonioni, and F. Tombari, “Batch normalization
embeddings for deep domain generalization,” arXiv preprint
arXiv:2011.12672, 2020.
[52] H. Gao, A. Xu, and H. Huang, “On the convergence ofcommunication-efficient local sgd for federated learning,” in Proceedings of the AAAI Conference on Artificial Intelligence, vol. 35, no. 9,2021, pp. 7510–7518.
[53] X. Cao, J. Jia, and N. Z. Gong, “Provably secure federated learning
against malicious clients,” in Proceedings of the AAAI Conference on
Artificial Intelligence, vol. 35, no. 8, 2021, pp. 6885–6893.
[54] W.-G. Chang, T. You, S. Seo, S. Kwak, and B. Han, “Domainspecific batch normalization for unsupervised domain adaptation,” in Proceedings of the IEEE/CVF Conference on Computer Vision
and Pattern Recognition, 2019, pp. 7354–7362.
[55] L. Zhu, Z. Liu, and S. Han, “Deep leakage from gradients,”
Advances in Neural Information Processing Systems, vol. 32, 2019.
[56] H. Yin, A. Mallya, A. Vahdat, J. M. Alvarez, J. Kautz, and
P. Molchanov, “See through gradients: Image batch recovery via
gradinversion,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2021, pp. 16 337–16 346.
[57] A. Reiss and D. Stricker, “Introducing a new benchmarked dataset
for activity monitoring,” in 2012 16th International Symposium on
Wearable Computers. IEEE, 2012, pp. 108–109.
[58] M. Yurochkin, M. Agarwal, S. Ghosh, K. Greenewald, N. Hoang,
and Y. Khazaeni, “Bayesian nonparametric federated learning ofneural networks,” in International Conference on Machine Learning.
PMLR, 2019, pp. 7252–7261.
[59] T. Lin, S. P. Karimireddy, S. Stich, and M. Jaggi, “Quasi-global
momentum: Accelerating decentralized deep learning on heterogeneous data,” in International Conference on Machine Learning.
PMLR, 2021, pp. 6654–6665.
[60] U. Sait, K. G. Lal, S. Prajapati, R. Bhaumik, T. Kumar, S. Sanjana,
and K. Bhalla, “Curated dataset for covid-19 posterior-anterior
chest radiography images (x-rays),” Mendeley Data, vol. 1, 2020.
[61] J. Yang, R. Shi, and B. Ni, “Medmnist classification decathlon:
A lightweight automl benchmark for medical image analysis,”in IEEE 18th International Symposium on Biomedical Imaging (ISBI),
2021, pp. 191–195.
[62] J. Yang, R. Shi, D. Wei, Z. Liu, L. Zhao, B. Ke, H. Pfister, and B. Ni,“Medmnist v2: A large-scale lightweight benchmark for 2d and 3d
biomedical image classification,” arXiv preprint arXiv:2008.#TODO,
2021.
[63] P. Bilic, P. F. Christ, E. Vorontsov, G. Chlebus, H. Chen, Q. Dou,
C.-W. Fu, X. Han, P.-A. Heng, J. Hesser et al., “The liver tumor
segmentation benchmark (lits),” arXiv preprint arXiv:1901.04056,
2019.
[64] X. Xu, F. Zhou, B. Liu, D. Fu, and X. Bai, “Efficient multiple organ
localization in ct image using 3d region proposal network,” IEEE
transactions on medical imaging, vol. 38, no. 8, pp. 1885–1898, 2019.
[65] Y. LeCun, L. Bottou, Y. Bengio, and P. Haffner, “Gradient-basedlearning applied to document recognition,” Proceedings of the IEEE,
vol. 86, no. 11, pp. 2278–2324, 1998.
[66] A. Krizhevsky, I. Sutskever, and G. E. Hinton, “Imagenet classification with deep convolutional neural networks,” in Advances inneural information processing systems, vol. 25, 2012, pp. 1097–1105.

11