Original: A TTACK -R ESISTANT
Revised: A TTACK -R ESISTANT

Original: F EDERATED
Revised: F EDERATED

Original: L EARNING
Revised: L EARNING

Original: WITH
Revised: WITH

Original: R ESIDUAL - BASED R EWEIGHTING
Revised: R ESIDUAL - BASED R EWEIGHTING

Original: Anonymous authors
Revised: Anonymous authors

Original: Paper under double-blind review
Revised: Paper under double-blind review

Original: A BSTRACT
Revised: A BSTRACT

Original: Federated learning has a variety of applications in multiple domains by utilizing private training data stored on different devices.
Revised: Federated learning has a variety of applications in multiple domains by utilizing private training data stored on different devices.

Original: However, the aggregation process in federated learning is highly vulnerable to adversarial attacks so that the global model may behave abnormally under attacks.
Revised: However, the aggregation process in federated learning is highly vulnerable to adversarial attacks so that the global model may behave abnormally under attacks.

Original: To tackle this challenge, we present a novel aggregation algorithm with residual-based reweighting to defend federated learning.
Revised: To tackle this challenge, we present a novel aggregation algorithm with residual-based reweighting to defend federated learning.

Original: Our aggregation algorithm combines repeated median regression with the reweighting scheme in iteratively reweighted least squares.
Revised: Our aggregation algorithm combines repeated median regression with the reweighting scheme in iteratively reweighted least squares.

Original: Our experiments show that our aggression algorithm outperforms other alternative algorithms in the presence of label-ﬂipping, backdoor, and Gaussian noise attacks.
Revised: Our experiments show that our aggregation algorithm outperforms other alternative algorithms in the presence of label-ﬂipping, backdoor, and Gaussian noise attacks.

Original: We also provide theoretical guarantees for our aggregation algorithm.
Revised: We also provide theoretical guarantees for our aggregation algorithm.

Original:  I NTRODUCTION
Revised:  I NTRODUCTION

Original: Federated learning is a machine learning methodology for training a global model with decentralized data stored on multiple or even millions of devices (McMahan et al., 2017).
Revised: Federated learning is a machine learning methodology for training a global model with decentralized data stored on multiple or even millions of devices (McMahan et al., 2017).

Original: In federated learning, private data is stored locally in isolated devices and will not be revealed to other parties during training.
Revised: In federated learning, private data is stored locally in isolated devices and will not be revealed to other parties during training.

Original: Federated learning can enable numerous real-world machine learning applications by utilizing massive training data that are privacy-sensitive and scattered on different devices (Bonawitz et al., 2017).
Revised: Federated learning can enable numerous real-world machine learning applications by utilizing massive training data that are privacy-sensitive and scattered on different devices (Bonawitz et al., 2017).

Original: For instance, multiple hospitals can collaborate to train a global model for classifying diseases using X-ray images without compromising patient privacy.
Revised: For instance, multiple hospitals can collaborate to train a global model for classifying diseases using X-ray images without compromising patient privacy.

Original: Note that these hospitals may possess X-ray images in different quantities and varieties, resulting in the non-IID (independent and identically distributed) data distribution.
Revised: Note that these hospitals may possess X-ray images in different quantities and varieties, resulting in the non-IID (independent and identically distributed) data distribution.

Original: Federated learning is different from distributed learning in the sense that the training data is often non-IID and we have no control over data distribution in federated learning.
Revised: Federated learning is different from distributed learning in the sense that the training data is often non-IID and we have no control over data distribution in federated learning.

Original: The default federated learning aggregation algorithm FedAvg (McMahan et al., 2017) that takes the average of locally updated models is vulnerable to various attacks.
Revised: The default federated learning aggregation algorithm FedAvg (McMahan et al., 2017) that takes the average of locally updated models is vulnerable to various attacks.

Original: We ﬁnd that federated learning suffers from label-ﬂipping, backdoor, and Gaussian noise attacks in our experiments.
Revised: We ﬁnd that federated learning suffers from label-ﬂipping, backdoor, and Gaussian noise attacks in our experiments.

Original: When a local model is poisoned, the aggregated global model can also be poisoned and fail to behave correctly.
Revised: When a local model is poisoned, the aggregated global model can also be poisoned and fail to behave correctly.

Original: A label-ﬂipping attack (Biggio et al., 2012) happens where an attacker assigns incorrect labels to some data.
Revised: A label-ﬂipping attack (Biggio et al., 2012) happens where an attacker assigns incorrect labels to some data.

Original: For example, an attacker can train a local model with cat images mislabelled as dogs and then share the poisoned local model for aggregation.
Revised: For example, an attacker can train a local model with cat images mislabelled as dogs and then share the poisoned local model for aggregation.

Original: Mitigating attacks in federated learning or distributed learning has been explored in recent research (Chen et al., 2017; Yin et al., 2018;
Revised: Mitigating attacks in federated learning or distributed learning has been explored in recent research (Chen et al., 2017; Yin et al., 2018;

Original: Fung et al., 2018; Blanchard et al., 2017).
Revised: Fung et al., 2018; Blanchard et al., 2017).

Original: Although the median or trimmed mean aggregation algorithms (Yin et al., 2018) may seem plausible in distributed learning, their performance degrades in federated learning when data is non-IID.
Revised: Although the median or trimmed mean aggregation algorithms (Yin et al., 2018) may seem plausible in distributed learning, their performance degrades in federated learning when data is non-IID.

Original: FoolsGold
Revised: FoolsGold

Original: (Fung et al., 2018) is a defense algorithm that identiﬁes participants with similar models as attackers but this strategy may not work when some harmless participants have similar local data.
Revised: (Fung et al., 2018) is a defense algorithm that identiﬁes participants with similar models as attackers but this strategy may not work when some harmless participants have similar local data.

Original: To make federated learning more attack-resistant, we develop an aggregation algorithm that is robust against label-ﬂipping, backdoor, and Gaussian noise attacks in a general non-IID setting.
Revised: To make federated learning more attack-resistant, we develop an aggregation algorithm that is robust against label-ﬂipping, backdoor, and Gaussian noise attacks in a general non-IID setting.

Original: We derive our aggregation algorithm by adopting the repeated median estimator (Siegel, 1982) and the reweighting scheme in iteratively reweighted least squares (IRLS)
Revised: We derive our aggregation algorithm by adopting the repeated median estimator (Siegel, 1982) and the reweighting scheme in iteratively reweighted least squares (IRLS)

Original: (Holland and Welsch, 1977; Rand R, 1997).
Revised: (Holland and Welsch, 1977; Rand R, 1997).

Original: We estimate the conﬁdence of each parameter in the local models and then the weight of each local model can be computed by heuristically accumulating all the parameter conﬁdence in each local model.
Revised: We estimate the conﬁdence of each parameter in the local models and then the weight of each local model can be computed by heuristically accumulating all the parameter conﬁdence in each local model.

Original: Our algorithm is straightforward to implement (less than 100 lines).
Revised: Our algorithm is straightforward to implement.

Original: Furthermore, we provide theoretical guarantees for our aggregation algorithm.
Revised: Furthermore, we provide theoretical guarantees for our aggregation algorithm.

Original: We compare our proposed algorithm to several baselines by conducting experiments on four datasets, the MNIST dataset (LeCun et al., 1998), CIFAR-10 dataset (Krizhevsky et al., 2009), Amazon
Revised: We compare our proposed algorithm to several baselines by conducting experiments on four datasets, the MNIST dataset (LeCun et al., 1998), CIFAR-10 dataset (Krizhevsky et al., 2009), Amazon

Original: Reviews dataset (Ruining and Julian, 2016) and the Lending Club loan dataset (Kan, 2019).
Revised: Reviews dataset (Ruining and Julian, 2016) and the Lending Club loan dataset (Kan, 2019).

Original: Our proposed aggregation signiﬁcantly mitigates the impact of attacked models in non-IID federated learning and outperforms other baselines in our evaluation.
Revised: Our proposed aggregation signiﬁcantly mitigates the impact of attacked models in non-IID federated learning and outperforms other baselines in our evaluation.

Original:  R ELATED W ORK
Revised:  R ELATED W ORK

Original: Adversarial attacks on federated learning.
Revised: Adversarial attacks on federated learning.

Original: Several attacks have been studied against federated learning (Wang et al., 2018; Biggio et al., 2012;
Revised: Several attacks have been studied against federated learning (Wang et al., 2018; Biggio et al., 2012;

Original: Fung et al., 2018; Hayes et al., 2019; Hitaj et al., 2017; Melis et al., 2019).
Revised: Fung et al., 2018; Hayes et al., 2019; Hitaj et al., 2017; Melis et al., 2019).

Original: The label-ﬂipping attack (Biggio et al., 2012) is shown to have great harm to a federated system even with a very small number of attackers (Fung et al., 2018).
Revised: The label-ﬂipping attack (Biggio et al., 2012) is shown to have great harm to a federated system even with a very small number of attackers (Fung et al., 2018).

Original: In this attack, the attacker ﬂips the labels of training data in one class to another class and trains the model accordingly.
Revised: In this attack, the attacker ﬂips the labels of training data in one class to another class and trains the model accordingly.

Original: Bagdasaryan et al.
Revised: Bagdasaryan et al.

Original: (2018) propose a backdoor attack so that the global model behaves incorrectly on adversarial targeted input.
Revised: (2018) propose a backdoor attack so that the global model behaves incorrectly on adversarial targeted input.

Original: In our work, we mainly focus on defending against label-ﬂipping attack, backdoor, and Gaussian noise attacks.
Revised: In our work, we mainly focus on defending against label-ﬂipping attack, backdoor, and Gaussian noise attacks.

Original: Note that an attacker can perform any type of attacks, such as modifying any model values and training the local model on poisoned data for arbitrary epochs.
Revised: Note that an attacker can perform any type of attacks, such as modifying any model values and training the local model on poisoned data for arbitrary epochs.

Original: Robust distributed learning.
Revised: Robust distributed learning.

Original: Statistical methods have been studied and applied in robust distributed learning where data is IID (Feng et al., 2014; Blanchard et al., 2017; Chen et al., 2017; Yin et al., 2018;
Revised: Statistical methods have been studied and applied in robust distributed learning where data is IID (Feng et al., 2014; Blanchard et al., 2017; Chen et al., 2017; Yin et al., 2018;

Original: Alistarh et al., 2018).
Revised: Alistarh et al., 2018).

Original: The median method and the trimmed mean method (Yin et al., 2018) are effective approaches in robust distributed learning, but may not be attack-resistant in federated learning where data distribution is non-IID.
Revised: The median method and the trimmed mean method (Yin et al., 2018) are effective approaches in robust distributed learning, but may not be attack-resistant in federated learning where data distribution is non-IID.

Original: To tackle the challenge in robust federated learning, we propose a reweighted aggregation algorithm that dynamically assigns weights to the local model based on the residual to a regression line estimated by the repeated median estimator (Siegel, 1982).
Revised: To tackle the challenge in robust federated learning, we propose a reweighted aggregation algorithm that dynamically assigns weights to the local model based on the residual to a regression line estimated by the repeated median estimator (Siegel, 1982).

Original: Defending federated learning.
Revised: Defending federated learning.

Original: Recently, some researchers have proposed some defense strategies for robust federated learning (Fung et al., 2018; Blanchard et al., 2017).
Revised: Recently, some researchers have proposed some defense strategies for robust federated learning (Fung et al., 2018; Blanchard et al., 2017).

Original: FoolsGold (Fung et al., 2018) is a defense mechanism against Sybil attacks by adjusting the learning rates of local models based on contribution similarity.
Revised: FoolsGold (Fung et al., 2018) is a defense mechanism against Sybil attacks by adjusting the learning rates of local models based on contribution similarity.

Original: The algorithm identiﬁes grouped actions as Sybil attacks and promotes the diversity of local model update.
Revised: The algorithm identiﬁes grouped actions as Sybil attacks and promotes the diversity of local model update.

Original: However, FoolsGold may identify harmless participants as attackers when these participants have similar local data.
Revised: However, FoolsGold may identify harmless participants as attackers when these participants have similar local data.

Original: Gu et al.
Revised: Gu et al.

Original: (2018) proposed a model, CalTrain, that represents data with ﬁngerprints to identify poisoned data and models.
Revised: (2018) proposed a model, CalTrain, that represents data with ﬁngerprints to identify poisoned data and models.

Original: Konstantinov and Lampert (2019) propose to maintain a small reference dataset to justify the quality and accountability of models.
Revised: Konstantinov and Lampert (2019) propose to maintain a small reference dataset to justify the quality and accountability of models.

Original: While this method is effective, it requires a lot of time to evaluate each model in every single round.
Revised: While this method is effective, it requires a lot of time to evaluate each model in every single round.

Original: Our algorithm does not need an additional reference dataset before or after each aggregation process.
Revised: Our algorithm does not need an additional reference dataset before or after each aggregation process.

Original: Some researchers proposed to improve the privacy preservation of federated learning (Bonawitz et al., 2017; Geyer et al., 2017; Truex et al., 2018; Thakkar et al., 2019).
Revised: Some researchers proposed to improve the privacy preservation of federated learning (Bonawitz et al., 2017; Geyer et al., 2017; Truex et al., 2018; Thakkar et al., 2019).

Original: Bonawitz et al.
Revised: Bonawitz et al.

Original: (2017) propose a privacy-preserving protocol for model aggregation in federated learning.
Revised: (2017) propose a privacy-preserving protocol for model aggregation in federated learning.

Original: Geyer et al.
Revised: Geyer et al.

Original: (2017) introduce differential privacy into federated learning.
Revised: (2017) introduce differential privacy into federated learning.

Original: Instead of enhancing privacy preservation, we focus on the robustness of federated learning so that the global model should behave correctly even when there is a large portion of malicious participants.
Revised: Instead of enhancing privacy preservation, we focus on the robustness of federated learning so that the global model should behave correctly even when there is a large portion of malicious participants.

Original:  O UR A LGORITHM
Revised:  O UR A LGORITHM

Original: In federated learning, there are multiple rounds of communication between participants and a central server for learning a global model.
Revised: In federated learning, there are multiple rounds of communication between participants and a central server for learning a global model.

Original: In each round, the global model is shared among the K participants and a local model on each device is trained on its local private data with the shared global model as initialization.
Revised: In each round, the global model is shared among the K participants and a local model on each device is trained on its local private data with the shared global model as initialization.

Original: Then all the K local models are sent to the central server to update the global model with an aggregation algorithm.
Revised: Then all the K local models are sent to the central server to update the global model with an aggregation algorithm.

Original: The original aggregation scheme uses a simple averaging algorithm to aggregate all the local models (McMahan et al., 2017).
Revised: The original aggregation scheme uses a simple averaging algorithm to aggregate all the local models (McMahan et al., 2017).

Original: Suppose the participant k has a local model
Revised: Suppose the participant k has a local model

Original: M ( k ) , and we can update the global model M global by taking the (weighted) average of all the K local models.
Revised: M ( k ) , and we can update the global model M global by taking the (weighted) average of all the K local models.

Original: [Figure]
Revised: [Figure]

Original: computation rrr
Revised: computation rrr

Original: [Figure]
Revised: [Figure]

Original: 3. A GGREGATION A LGORITHM
Revised: 3. A GGREGATION A LGORITHM

Original: Median is a robust estimator widely used in statistics.
Revised: Median is a robust estimator widely used in statistics.

Original: However, when the data distribution is non-IID, median neglects a signiﬁcant amount of information by merely taking a single median value.
Revised: However, when the data distribution is non-IID, median neglects a signiﬁcant amount of information by merely taking a single median value.

Original: Hence, in our aggregation algorithm, the global model is designed to be a reweighted average of all the local models where the model weights are estimated robustly.
Revised: Hence, in our aggregation algorithm, the global model is designed to be a reweighted average of all the local models where the model weights are estimated robustly.

Original: Algorithm 1 summaries our aggregation algorithm, and a detailed step-by-step description is provided below.
Revised: Algorithm 1 summarizes our aggregation algorithm, and a detailed step-by-step description is provided below.

Original: We perform a weighted average of all the local models at the model level by assigning a weight to each local model.
Revised: We perform a weighted average of all the local models at the model level by assigning a weight to each local model.

Original: The weight of each local model is computed by accumulating the parameter conﬁdence in the local model.
Revised: The weight of each local model is computed by accumulating the parameter conﬁdence in the local model.

Original: The parameter conﬁdence is computed based on the residual to a regression line estimated by the repeated median estimator (Siegel, 1982; Rand R, 1997).
Revised: The parameter conﬁdence is computed based on the residual to a regression line estimated by the repeated median estimator (Siegel, 1982; Rand R, 1997).

Original: Inspired by the reweighting scheme in IRLS (Rand R, 1997), we reweight each parameter by its vertical distance (residual) to a robust regression line.
Revised: Inspired by the reweighting scheme in IRLS (Rand R, 1997), we reweight each parameter by its vertical distance (residual) to a robust regression line.

Original: For the robust regression line estimation, we use the repeated median estimator (Siegel, 1982) since it has a high breakdown point of 50%.
Revised: For the robust regression line estimation, we use the repeated median estimator (Siegel, 1982) since it has a high breakdown point of 50%.

Original: Let y ( k ) n be the n -th parameter of the k -th local model.
Revised: Let y ( k ) n be the n -th parameter of the k -th local model.

Original: We use y n to indicate the list of n -th parameters in all the local models.
Revised: We use y n to indicate the list of n -th parameters in all the local models.

Original: Let x n be the indices of y n sorted in an ascending order.
Revised: Let x n be the indices of y n sorted in an ascending order.

Original: Then ( x n , y n ) is a point set in 2D with increasing values in the y direction.
Revised: Then ( x n , y n ) is a point set in 2D with increasing values in the y direction.

Original: Repeated median.
Revised: Repeated median.

Original: We use the repeated median estimator (Siegel, 1982) to estimate a linear regression line y =
Revised: We use the repeated median estimator (Siegel, 1982) to estimate a linear regression line y =

Original: β n 0 + β n 1 x .
Revised: β n 0 + β n 1 x .

Original: The slope β n 1 and intercept β n 0 are estimated as follow,
Revised: The slope β n 1 and intercept β n 0 are estimated as follow,

Original: [Equation]
Revised: [Equation]

Original: Residual computation.
Revised: Residual computation.

Original: We can calculate the residuals of the n -th parameters in all the local models:
Revised: We can calculate the residuals of the n -th parameters in all the local models:

Original: [Equation]
Revised: [Equation]

Original: Since r n can be very different in magnitude for different parameters
Revised: Since r n can be very different in magnitude for different parameters

Original: [Equation]
Revised: [Equation]

Original: we can normalize r n similar to the reweighting scheme in IRLS (Rand R, 1997):
Revised: we can normalize r n similar to the reweighting scheme in IRLS (Rand R, 1997):

Original: [Equation]
Revised: [Equation]

Original: where γ is a constant.
Revised: where γ is a constant.

Original: We set γ = 1 .
Revised: We set γ = 1 .

Original: 48 recommended by Wilcox et al.
Revised: 48 recommended by Wilcox et al.

Original: (Rand R, 1997).
Revised: (Rand R, 1997).

Original: Then the normalized residuals become
Revised: Then the normalized residuals become

Original: [Equation]
Revised: [Equation]

Original: Input: Models M (1) , M (2) , ..., M ( K ) , with parameters y (1) n , y (2) n , ..., y ( K ) n
Revised: Input: Models M (1) , M (2) , ..., M ( K ) , with parameters y (1) n , y (2) n , ..., y ( K ) n

Original: Output: The global model M global 1: for n -th parameter where n = 1 → N , and let y n =
Revised: Output: The global model M global 1: for n -th parameter where n = 1 → N , and let y n =

Original: [ y (1) n , y (2) n , ..., y ( K ) n ] T do 2: x n ← indices of y n sorted in an ascending order (cid:46) assign indices 3: β n 0 , β n 1 ← RepeatedMedian ( x n , y n ) (cid:46) get robust line estimation 4: r n ← y n − β n 0 − β n 1 x n (cid:46) compute residual 5: (cid:103) | r n | ← Median | r n | 6: τ n ← γ (cid:103) | r n | (1 + 5 K − 1 ) (cid:46) normalize residuals 7: e n ← r n τ n 8:
Revised: [ y (1) n , y (2) n , ..., y ( K ) n ] T do 2: x n ← indices of y n sorted in an ascending order (cid:46) assign indices 3: β n 0 , β n 1 ← RepeatedMedian ( x n , y n ) (cid:46) get robust line estimation 4: r n ← y n − β n 0 − β n 1 x n (cid:46) compute residual 5: (cid:103) | r n | ← Median | r n | 6: τ n ← γ (cid:103) | r n | (1 + 5 K − 1 ) (cid:46) normalize residuals 7: e n ← r n τ n 8:

Original: H n ← x n ( x T n x n ) − 1 x T n (cid:46) compute Hat matrix 9: w n ← √ 1 − diag
Revised: H n ← x n ( x T n x n ) − 1 x T n (cid:46) compute Hat matrix 9: w n ← √ 1 − diag

Original: ( H n ) e n Ψ( e n √ 1 − diag
Revised: ( H n ) e n Ψ( e n √ 1 − diag

Original: H n ) )
Revised: H n ) )

Original: (cid:46) compute parameter conﬁdence 10: y n , w n ← CorrectExtremeValue
Revised: (cid:46) compute parameter conﬁdence 10: y n , w n ← CorrectExtremeValue

Original: y n , w n ) 11: w n ← w n σ ( w n ) (cid:46) reweight conﬁdence by its standard deviation 12: for each k
Revised: y n , w n ) 11: w n ← w n σ ( w n ) (cid:46) reweight conﬁdence by its standard deviation 12: for each k

Original: = 1 → K do 13:
Revised: = 1 → K do 13:

Original: W ( k ) ← (cid:80)
Revised: W ( k ) ← (cid:80)

Original: Nn =1 w ( k ) n (cid:46) accumulate weights 14: M global ← (cid:80)
Revised: Nn =1 w ( k ) n (cid:46) accumulate weights 14: M global ← (cid:80)

Original: Kk =
Revised: Kk =

Original: [Figure]
Revised: [Figure]

Original: M
Revised: M

Original: [Figure]
Revised: [Figure]

Original: (cid:46) the global model aggregation
Revised: (cid:46) the global model aggregation

Original: [Figure]
Revised: [Figure]

Original: Parameter conﬁdence.
Revised: Parameter conﬁdence.

Original: After obtaining the normalized residuals, the parameter conﬁdence can be determined accordingly (Rand R, 1997):
Revised: After obtaining the normalized residuals, the parameter conﬁdence can be determined accordingly (Rand R, 1997):

Original: [Equation]
Revised: [Equation]

Original: where w ( k ) n is the conﬁdence of the n -th parameter in M ( k ) , Ψ( x ) = max {− Z, min
Revised: where w ( k ) n is the conﬁdence of the n -th parameter in M ( k ) , Ψ( x ) = max {− Z, min

Original: ( Z, x ) } with Z = λ (cid:112) 2 /K
Revised: ( Z, x ) } with Z = λ (cid:112) 2 /K

Original: and λ is a hyperparameter.
Revised: and λ is a hyperparameter.

Original: We use λ = 2 in our experiments.
Revised: We use λ = 2 in our experiments.

Original: Ψ here acts as a trusted interval and we can expand or shrink the interval by tuning λ .
Revised: Ψ here acts as a trusted interval and we can expand or shrink the interval by tuning λ .

Original: h kk is the k-th diagonal  of matrix in H n :
Revised: h kk is the k-th diagonal element of matrix in H n :

Original: [Equation]
Revised: [Equation]

Original: where x n = [ x (1) n x (2) n . . .
Revised: where x n = [ x (1) n x (2) n . . .

Original: x ( K ) n ] T .
Revised: x ( K ) n ] T .

Original: Extreme value correction.
Revised: Extreme value correction.

Original: Extremely large values, even multiplied with a small weight in model aggregation, can damage the global model.
Revised: Extremely large values, even multiplied with a small weight in model aggregation, can damage the global model.

Original: We address this issue by involving a a threshold δ .
Revised: We address this issue by involving a a threshold δ .

Original: If a parameter has a conﬁdence value lower than δ , then it should be corrected as follows,
Revised: If a parameter has a conﬁdence value lower than δ , then it should be corrected as follows,

Original: [Equation]
Revised: [Equation]

Original: The process of selecting proper δ is discussed in the appendix.
Revised: The process of selecting proper δ is discussed in the appendix.

Original: Model weight.
Revised: Model weight.

Original: To obtain the weight of each local model, we can simply aggregate the parameter conﬁdence in the local model but this is not ideal.
Revised: To obtain the weight of each local model, we can simply aggregate the parameter conﬁdence in the local model but this is not ideal.

Original: Imagine an attacker trains a model honestly, but then alters only 10% of the parameters to some extremely large values.
Revised: Imagine an attacker trains a model honestly, but then alters only 10% of the parameters to some extremely large values.

Original: This adversary model still receives about 90% of the parameter conﬁdence.
Revised: This adversary model still receives about 90% of the parameter conﬁdence.

Original: To address this problem, we measure the importance of a parameter by the standard deviation of w n .
Revised: To address this problem, we measure the importance of a parameter by the standard deviation of w n .

Original: A conﬁdence assignment with a large standard deviation indicates a great disagreement among this parameter in all models and should be more critical when being accumulated towards model weights:
Revised: A conﬁdence assignment with a large standard deviation indicates a great disagreement among this parameter in all models and should be more critical when being accumulated towards model weights:

Original: [Equation]
Revised: [Equation]

Original: [Equation] where W ( k ) is the weight for model k , N is the number of parameters, w n = [ w (1) n w (2) n . . .
Revised: where W ( k ) is the weight for model k , N is the number of parameters, w n = [ w (1) n w (2) n . . .

Original: w ( K ) n ] T .
Revised: w ( K ) n ] T .

Original: Global model.
Revised: Global model.

Original: Finally, we can obtain the updated global model by
Revised: Finally, we can obtain the updated global model by

Original: [Equation]
Revised: [Equation]

Original: 3. T HEORETICAL G UARANTEE
Revised: 3. T HEORETICAL G UARANTEE

Original: For simplicity, we consider the bound of the error rate for training a single-parameter model on K devices, each storing S IID samples of data.
Revised: For simplicity, we consider the bound of the error rate for training a single-parameter model on K devices, each storing S IID samples of data.

Original: We adopt the same set of assumptions from Yin et al.

Original: (2018), assuming that the loss function is L-smooth and the derivatives of the loss function are v-sub-exponential.

Original: Suppose that there are K devices and U of them are corrupted.
Revised: Suppose that there are K devices and U of them are corrupted.

Original: We denote the set of adversarial devices as B and the corruption ratio α = UK .
Revised: We denote the set of adversarial devices as B and the corruption ratio α = UK .

Original: We deﬁne the parameter of the local model i as ˆ y ( i ) and let µ be the expected value of the global model.
Revised: We deﬁne the parameter of the local model i as ˆ y ( i ) and let µ be the optimal model that minimizes the population loss.

Original: Based on our algorithm, the residuals can be simpliﬁed as (ˆ y ( i ) − ˜ y ) , where ˜ y is the median of { ˆ y ( i ) } for i = 1 , 2 , ..., K .
Revised: Based on our algorithm, the residuals can be simpliﬁed as (ˆ y ( i ) − ˜ y ) , where ˜ y is the median of { ˆ y ( i ) } for i = 1 , 2 , ..., K .

Original: Let (cid:102) | r | be the median of absolute residuals, i.e., (cid:102)
Revised: Let (cid:102) | r | be the median of absolute residuals, i.e., (cid:102)

Original: | r | = median( | ˆ y ( i ) − ˜ y | ) .
Revised: | r | = median( | ˆ y ( i ) − ˜ y | ) .

Original: Then, the normalized residual can be expressed as
Revised: Then, the normalized residual can be expressed as

Original: [Equation]
Revised: [Equation]

Original:  and the parameter conﬁdence is deﬁned as
Revised: 12) and the parameter conﬁdence is deﬁned as

Original: [Equation] 
Revised: [Equation] [Equation]

Original: Then we will prove that the error of the global model M global =
Revised: Then we will prove that the error of the global model M global := [Equation] bounded:

Original: [Equation]
Revised: [Equation]

Original: The details of the proof is presented in the appendix.
Revised: The details of the proof are presented in the appendix.

Revised: The bound indicates that when the amount of data and the number of models are sufﬁciently large, we can achieve a small error rate

Revised: ˜ O

Revised: ( 1 √ S + 1 √ SK + 1 S + 1 √ Kδ ) with high probability.

Original:  E XPERIMENTS
Revised:  E XPERIMENTS

Original: We compare our approach with other aggregation algorithms, including FedAvg (McMahan et al., 2017), the coordinate-wise median method (Yin et al., 2018), the coordinate-wise trimmed mean method (Yin et al., 2018), FoolsGold (Fung et al., 2018), and a coordinate-wise repeated median approach we adopt from (Siegel, 1982).
Revised: We compare our approach with other aggregation algorithms, including FedAvg (McMahan et al., 2017), the coordinate-wise median method (Yin et al., 2018), the coordinate-wise trimmed mean method (Yin et al., 2018), FoolsGold (Fung et al., 2018), and a coordinate-wise repeated median approach we adopt from (Siegel, 1982).

Original: We perform experiments on the MNIST handwritten digit dataset (LeCun et al., 1998), the Amazon Reviews dataset (Ruining and Julian, 2016), the CIFARdataset (Krizhevsky et al., 2009), and the Lending Club loan dataset (Kan, 2019).
Revised: We perform experiments on the MNIST handwritten digit dataset (LeCun et al., 1998), the Amazon Reviews dataset (Ruining and Julian, 2016), the CIFARdataset (Krizhevsky et al., 2009), and the Lending Club loan dataset (Kan, 2019).

Original: We implement attack strategies and defense algorithms in PyTorch (Paszke et al., 2017).
Revised: We implement attack strategies and defense algorithms in PyTorch (Paszke et al., 2017).

Original: We use a two-layer convolutional neural network (CNN) for our MNIST experiments and the network architecture is shown in the appendix.
Revised: We use a two-layer convolutional neural network (CNN) for our MNIST experiments and the network architecture is shown in the appendix.

Original: With this simple CNN model, our goal is to evaluate different aggregation algorithms for defending federated learning in the presence of attacks.
Revised: With this simple CNN model, our goal is to evaluate different aggregation algorithms for defending federated learning in the presence of attacks.

Original: On the CIFARdataset, we use ResNet-18 (He et al., 2015) for image classiﬁcation.
Revised: On the CIFARdataset, we use ResNet-18 (He et al., 2015) for image classiﬁcation.

Original: The text classiﬁcation model
Revised: The text classiﬁcation model

Original: FastText (Joulin et al., 2016) is adopted for evaluation on the Amazon Reviews dataset.
Revised: FastText (Joulin et al., 2016) is adopted for evaluation on the Amazon Reviews dataset.

Original: It is a two-layer deep neural network where the ﬁrst layer is an embedding layer, and the second layer is a fully connected layer.
Revised: It is a two-layer deep neural network where the ﬁrst layer is an embedding layer, and the second layer is a fully connected layer.

Original: For the Lending Club loan dataset, we use a simple neural network with three fully-connected layers to classify loan status.
Revised: For the Lending Club loan dataset, we use a simple neural network with three fully-connected layers to classify loan status.

Original: We use the last two models to demonstrate that our algorithm can be generalized to a natural language processing task and to a real-work ﬁnancial problem.
Revised: We use the last two models to demonstrate that our algorithm can be generalized to a natural language processing task and to a real-work ﬁnancial problem.

Original: All the evaluation results are the average of running the same experiments 3 times.
Revised: All the evaluation results are the average of running the same experiments 3 times.

Original: 4. D ATASETS AND E XPERIMENTAL
Revised: 4. D ATASETS AND E XPERIMENTAL

Original: MNIST dataset.
Revised: MNIST dataset.

Original: The MNIST dataset contains 70,000 real-world handwritten images with digits from 0 to 9.
Revised: The MNIST dataset contains 70,000 real-world handwritten images with digits from 0 to 9.

Original: We evaluate different methods by learning a global model on these training images distributed on multiple devices in a non-IID setting with adversarial attacks.
Revised: We evaluate different methods by learning a global model on these training images distributed on multiple devices in a non-IID setting with adversarial attacks.

Original: CIFAR-10 dataset.
Revised: CIFAR-10 dataset.

Original: The CIFAR-10 dataset contains 60,000 natural images in ten object classes.
Revised: The CIFAR-10 dataset contains 60,000 natural images in ten object classes.

Original: The experimental setup is also non-IID on CIFAR-10.
Revised: The experimental setup is also non-IID on CIFAR-10.

Original: Amazon Reviews dataset.
Revised: Amazon Reviews dataset.

Original: The Amazon Reviews dataset (Ruining and Julian, 2016) contains product reviews and ratings collected from the Amazon website.
Revised: The Amazon Reviews dataset (Ruining and Julian, 2016) contains product reviews and ratings collected from the Amazon website.

Original: Every review is paired with a sentiment rating from 1 to 5.
Revised: Every review is paired with a sentiment rating from 1 to 5.

Original: We categorize comments with rating 1 or 2 as negative and comments with rating 4 or 5 as positive.
Revised: We categorize comments with rating 1 or 2 as negative and comments with rating 4 or 5 as positive.

Original: We discard reviews with rating 3, and we only train a binary classiﬁer.
Revised: We discard reviews with rating 3, and we only train a binary classiﬁer.

Original: We only use the book reviews from the Kindle Store.
Revised: We only use the book reviews from the Kindle Store.

Original: 20% of the reviews are used for testing, while the rest is for training.
Revised: 20% of the reviews are used for testing, while the rest is for training.

Original: We obtain a training set of 86,164 reviews and a test set of 13,260 reviews.
Revised: We obtain a training set of 86,164 reviews and a test set of 13,260 reviews.

Original: Lending Club Loan dataset.
Revised: Lending Club Loan dataset.

Original: The Lending club dataset LOAN (Kan, 2019) contains ﬁnancial information such as credit scores and the number of ﬁnance inquiries for loan status prediction (“Current”, “Late”, or “Fully Paid”).
Revised: The Lending club dataset LOAN (Kan, 2019) contains ﬁnancial information such as credit scores and the number of ﬁnance inquiries for loan status prediction (“Current”, “Late”, or “Fully Paid”).

Original: There are 1,808,534 data samples in 9 types of loan status.
Revised: There are 1,808,534 data samples in 9 types of loan status.

Original: We divide them by US states to simulate the federated learning scenarios with non-IID data distribution where each state represents a participant.
Revised: We divide them by US states to simulate the federated learning scenarios with non-IID data distribution where each state represents a participant.

Original: 4. R ESULTS ON L ABEL - FLIPPING A TTACKS
Revised: 4. R ESULTS ON L ABEL - FLIPPING A TTACKS

Original: We evaluate the overall classiﬁcation performance of different aggregation methods on three datasets under label-ﬂipping attacks, the MNIST dataset (Biggio et al., 2012), the CIFAR-10 dataset (Krizhevsky et al., 2009), and Amazon Reviews dataset (Ruining and Julian, 2016).
Revised: We evaluate the overall classiﬁcation performance of different aggregation methods on three datasets under label-ﬂipping attacks, the MNIST dataset (Biggio et al., 2012), the CIFAR-10 dataset (Krizhevsky et al., 2009), and Amazon Reviews dataset (Ruining and Julian, 2016).

Original: In label-ﬂipping attacks, attackers ﬂip the labels of training examples in the source class to a target class and train their models accordingly.
Revised: In label-ﬂipping attacks, attackers ﬂip the labels of training examples in the source class to a target class and train their models accordingly.

Original: In the MNIST experiment, we simulate federated learning with 100 participants, within which 0 to 10 of them are attackers.
Revised: In the MNIST experiment, we simulate federated learning with 100 participants, within which 0 to 10 of them are attackers.

Original: Each participant contains images of two random digits.
Revised: Each participant contains images of two random digits.

Original: The attackers are chosen to be some participants with images of digit 1 and another random digit since they are ﬂipping the label of 1 to 7.
Revised: The attackers are chosen to be some participants with images of digit 1 and another random digit since they are ﬂipping the label of 1 to 7.

Original: We run 200 synchronization rounds with δ set to 0.01.
Revised: We run 200 synchronization rounds with δ set to 0.01.

Original: In each round of federated learning, each participant is supposed to train the local model for 5 epochs, but the attackers can train for arbitrary epochs.
Revised: In each round of federated learning, each participant is supposed to train the local model for 5 epochs, but the attackers can train for arbitrary epochs.

Original: The results are shown in Figure 3 where attackers train the models with 5 more epochs to enhance the attacks.
Revised: The results are shown in Figure 3 where attackers train the models with 5 more epochs to enhance the attacks.

Original: Our algorithm outperforms all other methods and is robust when the number of attackers increases.
Revised: Our algorithm outperforms all other methods and is robust when the number of attackers increases.

Original: Median methods (Median and Repeated Median) have relatively low accuracy due to discarding most of the information in model aggregation.
Revised: Median methods (Median and Repeated Median) have relatively low accuracy due to discarding most of the information in model aggregation.

Original: Our algorithm, on the other hand, takes the reweighted average of all local models and thus gathers more information in an unsupervised way.
Revised: Our algorithm, on the other hand, takes the reweighted average of all local models and thus gathers more information in an unsupervised way.

Original: We also compare our algorithm with the state-of-the-art algorithm FoolsGold (Fung et al., 2018).
Revised: We also compare our algorithm with the state-of-the-art algorithm FoolsGold (Fung et al., 2018).

Original: Though their algorithm also maintains a low attack success rate, our algorithm is more stable and surpasses FoolsGold by 6% on average.
Revised: Though their algorithm also maintains a low attack success rate, our algorithm is more stable and surpasses FoolsGold by 6% on average.

Original: For the CIFAR-10 experiment, following Bagdasaryan et al.
Revised: For the CIFAR-10 experiment, following Bagdasaryan et al.

Original: (2018), we adopt a Dirichlet distribution (Minka, 2000) with a hyperparameter 0.9 to generate non-IID data distribution for totallyparticipants.
Revised: (2018), we adopt a Dirichlet distribution (Minka, 2000) with a hyperparameter 0.9 to generate non-IID data distribution for totallyparticipants.

Original: The experimental setup is the same for the CIFAR-10 and MNIST experiments under backdoor attacks in Section 4.3.
Revised: The experimental setup is the same for the CIFAR-10 and MNIST experiments under backdoor attacks in Section 4.3.

Original: The attackers ﬂip the label of “cat” to “dog” since they are the most similar classes in CIFAR-10.
Revised: The attackers ﬂip the label of “cat” to “dog” since they are the most similar classes in CIFAR-10.

Original: The experiment results can be found in Table 1.
Revised: The experiment results can be found in Table 1.

Original: FoolsGold (Fung et al., 2018) fails in this experiment because honest participants can be classiﬁed as attackers when two or more honest participants have similar data distribution.
Revised: FoolsGold (Fung et al., 2018) fails in this experiment because honest participants can be classiﬁed as attackers when two or more honest participants have similar data distribution.

Original: In the experiment on Amazon Reviews, there are 10 participants, where 0 to 4 participants are attackers who ﬂip all their labels.
Revised: In the experiment on Amazon Reviews, there are 10 participants, where 0 to 4 participants are attackers who ﬂip all their labels.

Original: Attackers also train for 5 more epochs.
Revised: Attackers also train for 5 more epochs.

Original: We run 10 synchronization rounds in the experiment.
Revised: We run 10 synchronization rounds in the experiment.

Original: The result is summarized in Table 2.
Revised: The result is summarized in Table 2.

Original: Our algorithm achieves comparable state-of-the-art results, and our performance does not degrade when there are 
Revised: Our algorithm achieves comparable state-of-the-art results, and our performance does not degrade when there are less than 50% attackers.

Original: [Figure] [Figure]
Revised: [Figure] [Figure]

Original: # of attackers
Revised: # of attackers

Original: [Table]
Revised: [Table]

Original: Average

Original: FedAvg (McMahan et al., 2017)
Revised: FedAvg (McMahan et al., 2017)

Original: [Table]
Revised: [Table]

Original: Median (Yin et al., 2018)
Revised: Median (Yin et al., 2018)

Original: [Table]
Revised: [Table]

Original: Trimmed Mean (Yin et al., 2018)
Revised: Trimmed Mean (Yin et al., 2018)

Original: [Table]
Revised: [Table]

Original: Repeated
Revised: Repeated

Original: [Table]
Revised: [Table]

Original: (Siegel, 1982)
Revised: (Siegel, 1982)

Original: [Table]
Revised: [Table]

Original: FoolsGold (Fung et al., 2018)
Revised: FoolsGold (Fung et al., 2018)

Original: [Table]

Original: Ours

Original: [Table]
Revised: [Table]

Original: The total number of participants is 10.
Revised: The total number of participants is 10.

Original: # of attackers
Revised: # of attackers

Original: [Table]
Revised: [Table]

Original: Average

Original: FedAvg (McMahanet al
Revised: FedAvg (McMahan

Original: [Table]
Revised: [Table]

Original: [Table]

Original: [Table]
Revised: [Table]

Original: 4. R ESULTS ON B ACKDOOR A TTACKS
Revised: 4. R ESULTS ON B ACKDOOR A TTACKS

Original: For pixel-pattern backdoor attacks (Gu et al., 2019) in federated learning(Bagdasaryan et al., 2018), attackers manipulate their local models so that the learned global model predicts some backdoor target label for any image embedded with certain patterns.
Revised: For pixel-pattern backdoor attacks (Gu et al., 2019) in federated learning(Bagdasaryan et al., 2018), attackers manipulate their local models so that the learned global model predicts some backdoor target label for any image embedded with certain patterns.

Original: An example is shown in Figure 5 in the
Revised: An example is shown in Figure 6 in the

Original: Appendix.
Revised: Appendix.

Original: The global model can behave normally for clean data.
Revised: The global model can behave normally for clean data.

Original: We choose “bird” in CIFARand “2” in MNIST as the backdoor target labels.
Revised: We choose “bird” in CIFARand “2” in MNIST as the backdoor target labels.

Original: Similarly, for the preprocessed LOAN dataset that contains 92 features, we manipulate 6 features by assigning certain large values to them and change the labels of manipulated data to “Does not meet the credit policy.
Revised: Similarly, for the preprocessed LOAN dataset that contains 92 features, we manipulate 6 features by assigning certain large values to them and change the labels of manipulated data to “Does not meet the credit policy.

Original: Status:Fully Paid.”
Revised: Status:Fully Paid.”

Original: The training data is mixed with manipulated data and clean data to ﬁt both the backdoor task and the main task.
Revised: The training data is mixed with manipulated data and clean data to ﬁt both the backdoor task and the main task.

Original: We compare the performance of aggregation algorithms under two backdoor attack scenarios, which are called the naive approach and the model replacement in Bagdasaryan et al.
Revised: We compare the performance of aggregation algorithms under two backdoor attack scenarios, which are called the naive approach and the model replacement in Bagdasaryan et al.

Original: For the naive approach, an attacker poisons its local model and submits the malicious update in every round.
Revised: For the naive approach, an attacker poisons its local model and submits the malicious update in every round.

Original: For the model replacement, an attacker only poisons in one round to embed some patterns into the global model, so the attacker needs to scale up its malicious update before submission.
Revised: For the model replacement, an attacker only poisons in one round to embed some patterns into the global model, so the attacker needs to scale up its malicious update before submission.

Original: In our experiment, the malicious participant attacks in round 6 and scales up its update by 100.
Revised: In our experiment, the malicious participant attacks in round 6 and scales up its update by 100.

Original: We run 200 rounds for
Revised: We run 200 rounds for

Original: MNIST and 100 rounds for CIFAR and LOAN.
Revised: MNIST and 100 rounds for CIFAR and LOAN.

Original: Table 3 summarizes the results of backdoor attacks.
Revised: Table 3 summarizes the results of backdoor attacks.

Original: Our method is the highest in terms of accuracy on
Revised: Our method is the highest in terms of accuracy on

Original: MNIST under both backdoor attack scenarios.
Revised: MNIST under both backdoor attack scenarios.

Original: Moreover, on the more challenging CIFAR-10 dataset,
Revised: Moreover, on the more challenging CIFAR-10 dataset,

Original: [Table] [Table]
Revised: [Table] [Table]

Original: Fung et al

Original: [Table] [Table] [Table] our algorithm is the only one that can defend the naive approach backdoor attack.
Revised: our algorithm is the only one that can defend the naive approach backdoor attack.

Original: In Figure 4, we plot the attack success rates over time under different aggregation algorithms except for FoolsGold because it completely fails in both main and backdoor tasks.
Revised: In Figure 4, we plot the attack success rates over time under different aggregation algorithms except for FoolsGold because it completely fails in both main and backdoor tasks.

Original: Intuitively, backdoor attacks can easily succeed under FedAvg, and other baselines slow down the process but still reach high attack success rate into over 70% within 100 rounds.
Revised: Intuitively, backdoor attacks can easily succeed under FedAvg, and other baselines slow down the process but still reach high attack success rate into over 70% within 100 rounds.

Original: Our algorithm effectively defends the attack and remains stable with 9.65% attack success rate when being attacked continuously for 100 rounds.
Revised: Our algorithm effectively defends the attack and remains stable with 9.65% attack success rate when being attacked continuously for 100 rounds.

Original: On the LOAN dataset, our method achieves higher accuracy 94.50% (FedAvg 93.65%) and 0.00% attack success rate (FedAvg 99.71%) under the naive approach attack after 100 rounds.
Revised: On the LOAN dataset, our method achieves higher accuracy 94.50% (FedAvg 93.65%) and 0.00% attack success rate (FedAvg 99.71%) under the naive approach attack after 100 rounds.

Original: Similarly, our method has 95.06% accuracy (FedAvg 94.11%) and 0.00% attack success rate (FedAvg 98.96%) under model replacement attacks after 100 rounds.
Revised: Similarly, our method has 95.06% accuracy (FedAvg 94.11%) and 0.00% attack success rate (FedAvg 98.96%) under model replacement attacks after 100 rounds.

Original: Other baselines also have 0.00% attack success rate in two attacking scenarios expect FoolsGold, that of which is 99.96% under the naive approachattack.
Revised: Other baselines also have 0.00% attack success rate in two attacking scenarios except FoolsGold, whose attack success rate is 99.96% under the naive approach

Original: 4. R ESULTS ON G AUSSIAN N OISE A TTACKS
Revised: 4. R ESULTS ON G AUSSIAN N OISE A TTACKS

Original: In Gaussian noise attacks, We try to simulate real-world model corruption with 10 corrupted users at different scales of noise.
Revised: In Gaussian noise attacks, we try to simulate real-world model corruption with 10 corrupted users at different scales of noise.

Original: Gaussian noise attacks are performed by multiplying each parameter by a scale (cid:15) sampled from a Gaussian distribution with a mean of 1 .
Revised: Gaussian noise attacks are performed by multiplying each parameter by a scale (cid:15) sampled from a Gaussian distribution with a mean of 1 .

Original: In the experiment on MNIST, we have 100 participants where some may be attackers.
Revised: In the experiment on MNIST, we have 100 participants where some may be attackers.

Original: Each participant has 300 images from a random digit and other 300 images from another random digit.
Revised: Each participant has 300 images from a random digit and other 300 images from another random digit.

Original: Table 4 summarizes the results when (cid:15) is sampled from a Gaussian distribution with different standard deviations and there are 10 attackers.
Revised: Table 4 summarizes the results when (cid:15) is sampled from a Gaussian distribution with different standard deviations and there are 10 attackers.

Original: Our aggregation algorithm outperforms other approaches under Gaussian noise at multiple scales.
Revised: Our aggregation algorithm outperforms other approaches under Gaussian noise at multiple scales.

Original:  C ONCLUSION
Revised:  C ONCLUSION

Original: Federated learning utilizes private data on multiple devices to train a global model, but the simple aggregation algorithm in federated learning is vulnerable to malicious attacks.
Revised: Federated learning utilizes private data on multiple devices to train a global model, but the simple aggregation algorithm in federated learning is vulnerable to malicious attacks.

Original: To tackle this problem, we present a novel aggregation algorithm with residual reweighting.
Revised: To tackle this problem, we present a novel aggregation algorithm with residual reweighting.

Original: Our experiments on computer vision, natural language processing, and ﬁnancial data show that our approach is robust to labelﬂipping, Gaussian noise, and backdoor attacks while prior aggregation methods are not.
Revised: Our experiments on computer vision, natural language processing, and ﬁnancial data show that our approach is robust to labelﬂipping, Gaussian noise, and backdoor attacks while prior aggregation methods are not.

Original: Our algorithm is easy to implement and readily incorporated into existing federated learning frameworks.
Revised: Our algorithm is easy to implement and readily incorporated into existing federated learning frameworks.

Original: We hope our proposed aggregation algorithm can make federated learning more practical and robust in the future.
Revised: We hope our proposed aggregation algorithm can make federated learning more practical and robust in the future.

Original: Our source code will be made public.
Revised: Our source code will be made public.

Original: A A PPENDIX A. P ROOF
Revised: A A PPENDIX A. P ROOF

Revised: For simplicity, we consider the bound of the error rate for training a single-parameter model on K devices, each storing S IID samples of data.

Revised: We deﬁne the parameter of the local model i as ˆ y ( i ) .

Original: Proof.
Revised: Suppose that training data points are sampled from some unknown distribution D . Let f ( y ; x ) be a loss function of the parameter y ∈ Y associated with the data point x , where Y is the parameter space, and F ( y ) : = E x ∈D [ f ( y : x )] .

Revised: Our goal is to learn an optimal model deﬁned by the parameter that minimizes the population loss:

Revised: [Equation]

Revised: [Equation]

Revised: The parameter space Y is assumed to be convex and compact with diameter D, i.e., || y − y (cid:48)

Revised: || 2 ≤ D , ∀ y , y (cid:48)

Revised: ∈ Y .

Revised: Suppose that there are K devices and U of them are corrupted.

Revised: We denote the set of adversarial devices as B and the corruption ratio α = UK .

Revised: Based on our algorithm, the residuals can be simpliﬁed as (ˆ y ( i ) − ˜ y ) , where ˜ y is the median of { ˆ y ( i ) } for i = 1 , 2 , ..., K .

Revised: Let (cid:102) | r | be the median of absolute residuals, i.e., (cid:102)

Revised: | r | = median( | ˆ y ( i ) − ˜ y | ) .

Revised: Then, the normalized residual can be expressed as

Revised: [Equation]

Revised: [Equation]

Revised: and the parameter conﬁdence is deﬁned as

Revised: [Equation]

Revised: Then we will prove that the error of the global model M global :=

Revised: [Equation]

Revised: bounded:

Revised: [Equation]

Original:  We can separate Equation 14 into two sets with adversarial participant B and normal users [ K ] \ B where [ K ] = { 1 , 2 , . . .
Revised: Proof. We adopt assumptions 1, 2, 3, and 6, Theorem 8 and Lemma 3 from Yin et al. (2018) here. We can separate Equation 18 into two sets with adversarial participant B and normal users [ K ] \ B where [ K ] = { 1 , 2 , . . .

Original: , K } .
Revised: , K } .

Revised: (cid:12)(cid:12)(cid:12)(cid:12)(cid:12)

Original: [Equation]
Revised: [Equation]

Revised: Now we state here the assumptions and theorems from Yin et al.

Revised: Assumption 1 (Smoothness of f and F, Assumption 1 from Yin et al.

Revised: For any x ∈ D , the derivative of f ( y ; x ) is L-Lipschitz, and the function f ( y ; x ) is L-smooth.

Revised: Assumption 2 (Sub-exponential gradients, Assumption 6 from Yin et al.

Original:  Yin et al. 
Revised: We assume that Proposition 1 (Theorem 8 from Yin et al. Deﬁne the median of y ( i ) : [Equation]

Revised: Suppose that Assumption 1 holds, F(y) is λ F -strongly convex.

Original: (2018) prove that the ﬁrst term of Equation 15, max i ∈ [ K ] \B {
Revised: Then, with probability at least 1 −1+ SKLD , we have [Equation] Proposition 2 (Lemma 3 from Yin et al. Suppose that the derivative of the loss function f is i.i.d.

Original: [Equation] ˆ y [Equation]
Revised: v-sub-exponential.

Original: i )
Revised: Then with a high probability, we have

Original: [Equation] [Equation]
Revised: [Equation] [Equation]

Original: ˜ O [Equation] S + 1 √ SK ) by Bernstein’s inequality and union bound.
Revised: Yin et al.

Original: They also prove the  third  term | ˜ y − µ | is bounded:
Revised: (2018) prove the difference between the median of models and the optimal model is bounded in Proposition 1, and the maximum difference between an honest model and the optimal model is bounded in Proposition 2 with a high probability, which are exactly the third and the ﬁrst term respectively in Equation 19. Now let us consider

Original: [Equation]
Revised: [Equation]

Original: Now let us consider
Revised: y

Original: [Equation]
Revised: [Equation]

Original: For an attacker a ∈ B , there are two cases:
Revised: For an attacker a ∈ B , there are two cases:

Original: • | e a | ≤ √ 2 λ √ K .
Revised: • | e a | ≤ √ 2 λ √ K .

Original: From Equation 12 and | e a | ≤ √ 2 λ √ K , we have (cid:12)(cid:12) 
Revised: From Equation 16 and | e a | ≤ √ 2 λ √ K , we have (cid:12)(cid:12) ˆ y ( a ) − ˜ y (cid:12)(cid:12)

Original: ˆ y ( i ) − ˜ y
Revised: [Equation]

Original: (cid:12)(cid:12) ≤ C √ K , where C = γ √ 2 λ (cid:102) | r | (1 + 5 K − 1 ) .
Revised: C √ K , where C = [Equation] (cid:102) [Equation]

Original: • | e a | > √ 2 λ √ K .
Revised: • | e a | > √ 2 λ √ K .

Original: If | e a | > (cid:12)(cid:12)(cid:12) √ 2 λ √ Kδ (cid:12)(cid:12)(cid:12) then it can be eliminated from Equation 13.
Revised: If | e a | > (cid:12)(cid:12)(cid:12) √ 2 λ √ Kδ (cid:12)(cid:12)(cid:12) then it can be eliminated from Equation 17.

Original: From Equation 12 and | e a | > √ 2 λ √ K , we have C √ K < (cid:12)(cid:12)
Revised: From Equation 16 and | e a | > √ 2 λ √ K , we have C √ K < (cid:12)(cid:12)

Original: ˆ y ( i ) − ˜ y
Revised: ˆ y ( a ) − ˜ y

Original: (cid:12)(cid:12)
Revised: (cid:12)(cid:12)

Original: ≤ C √
Revised: ≤ C √

Original: Kδ .
Revised: Kδ .

Original: Combine two cases, we know that (cid:12)(cid:12)
Revised: Combine two cases [Equation] we know that (cid:12)(cid:12)

Original: ˆ y ( i ) − ˜ y
Revised: ˆ y ( a ) − ˜ y

Revised: (cid:12)(cid:12)

Original: (cid:12)(cid:12) ≤ C √ Kδ , and we have the following inequality,
Revised: ≤ C √ Kδ [Equation] and we have the following inequality,

Original: [Equation]
Revised: [Equation]

Original: Therefore, we prove that
Revised: Therefore [Equation] we prove that

Original: [Equation]
Revised: [Equation]

Original: A. M ODEL D ETAILS
Revised: A. M ODEL D ETAILS

Original: The models we use in the experiments are described here.
Revised: The models we use in the experiments are described here.

Original: The CNN model used for classifying
Revised: The CNN model used for classifying

Original: MNIST images (LeCun et al., 1998) can be found in Table 5.
Revised: MNIST images (LeCun et al., 1998) can be found in Table 5.

Original: CIFAR-10 is trained with Resnet-18(He et al., 2015).
Revised: CIFAR-10 is trained with Resnet-18(He et al., 2015).

Original: The learning rate for both MNIST classiﬁer and CIFAR-10 classiﬁer is 0.01 with a
Revised: The learning rate for both MNIST classiﬁer and CIFAR-10 classiﬁer is 0.01 with a

Original: SGD optimizer.
Revised: SGD optimizer.

Original: We train MNIST classiﬁer for 200 synchronous rounds and CIFAR-10 classiﬁer for 100 rounds, with batch size 128 and 64 respectively.
Revised: We train MNIST classiﬁer for 200 synchronous rounds and CIFAR-10 classiﬁer for 100 rounds, with batch size 128 and 64 respectively.

Original: The Amazon Reviews (Ruining and Julian, 2016) classiﬁer FastText (Joulin et al., 2016) is a two-layer deep neural network where the ﬁrst layer is an embedding layer of dimension 100 and the second layer is a fully connected layer of dimension 1.
Revised: The Amazon Reviews (Ruining and Julian, 2016) classiﬁer FastText (Joulin et al., 2016) is a two-layer deep neural network where the ﬁrst layer is an embedding layer of dimension 100 and the second layer is a fully connected layer of dimension 1.

Original: We use binary cross entropy loss to train the model with Adam optimizer (Kingma and Ba, 2014) and learning rate 0.05.
Revised: We use binary cross entropy loss to train the model with Adam optimizer (Kingma and Ba, 2014) and learning rate 0.05.

Original: The input dimension (vocabulary size) of the model is 25,000.
Revised: The input dimension (vocabulary size) of the model is 25,000.

Original: For the preprocessed LOAN dataset, the input data consists of 91 features.
Revised: For the preprocessed LOAN dataset, the input data consists of 91 features.

Original: We use a simple neural network with three fully-connected layers whose feature channels are 46, 23, and 9 respectively.
Revised: We use a simple neural network with three fully-connected layers whose feature channels are 46, 23, and 9 respectively.

Original: [Table]
Revised: [Table]

Original: A. H YPERPARAMETERS
Revised: A. H YPERPARAMETERS

Original: Our algorithm has two hyperparameters, λ and δ . λ is used to control the conﬁdence interval, and δ is used to constrain the lower bound of the conﬁdence.
Revised: We have two hyperparameters in our method, λ and δ , where λ controls the conﬁdence interval, and δ controls the clipping threshold. We did a grid search to prove that the selection is robust and efﬁcient.

Revised: We conduct experiments on the MNIST dataset in i.i.d.

Original:  λ is insensitive to changes.
Revised: and non-i.i.d. settings with various λ and δ .

Original: We use = 2 in all the experiments.
Revised: The results can be found in table 6. In the i.i.d.

Original:  δ , however, is  more sensitive to  the data distribution.
Revised: setting of the MNIST dataset, the results are almost the same for our selections of λ and δ . It is mainly because the parameters for honest models are very close, thus outliers are more easily to be excluded. Still, since the honest models are close and outliers can be easily detected, a larger δ yields a lower error bound as also shown in our proof. In the non-i.i.d.

Original: While the data is close to IID (on the Amazon Reviews dataset), a large δ (such as 0.1) yields satisfactory results as  the parameters among models are more similar.
Revised: Experiments, however, the models are more diverse, so it is important to ﬁnd a balance between excluding adversarial models while embracing models trained on different data distributions. Luckily, the hyperparameter is robust in a large range, as shown in table 6. Larger δ means it has a lower tolerance of differences and thus may eliminate some of the honest models when data is non-i.i.d. While increasing λ brings insensitivity to the variance of δ .

Original: If the data distribution is non-IID (on the MNIST dataset), a smaller δ (such as 0.01) is preferable.
Revised: [Table] [Table] A. A LTERNATIVE SCHEMES Though our algorithm is inspired by the reweighting scheme from IRLS (Rand R, 1997), some alternative linear estimators and weighting schemes can replace the original ones.

Revised: In this section, we replaced the repeated median estimator in our algorithm with Theil-Sen estimator or median estimator and replaced the weighting scheme with a zero-mean Gaussian function where smaller residual means larger weight.

Revised: We tuned the λ (in the Gaussian weighting it is the variance σ ), and the results are shown in table 7.

Revised: The experiments are performed on the MNIST dataset with labelﬂipping attacks.

Revised: Median and Theil-Sen estimators achieve similar results as the Repeated Median estimator, but they can be attacked in a few cases.

Revised: For example, λ = 5 and δ = 0 .

Revised: 01 for Median estimator and λ = 5 and δ = 0 .

Revised: for Theil-Sen estimator.

Revised: Theil-Sen estimator, especially, only has a breakdown point of 29.3%, meaning that it will more easily be broken when the number of attackers increases.

Revised: The alternative Gaussian weighting scheme is fairly robust against label-ﬂipping attacks.

Revised: This demonstrates that the robustness comes from our framework rather than intricate hyperparameter tuning.

Revised: A. U NDERREPRESENTED DATA

Revised: We admit that underrepresented data samples might be rejected by our scheme.

Revised: To investigate how it may affect the ﬁnal result, we did a few experiments on the non-i.i.d.

Revised: MNIST dataset.

Revised: The experiment setting is the same as the label-ﬂipping attack, where there are 100 users, and each of them holds images of 2 random digits (each digit has 20 holders).

Revised: Now we remove the holders of digit 0 one by one until there are no user holding digit 0.

Revised: The experiment result is plotted in ﬁgure 5.

Revised: Our method performs signiﬁcantly worse than FedAvg after removing 15 holders (75% images of digit 0).

Revised: However, we observe similar effects on other algorithms, and we perform no worse than other algorithms.

Original: A. B ACKDOOR E XPERIMENTAL S ETUP
Revised: A. B ACKDOOR E XPERIMENTAL S ETUP

Original: Experiment setup of backdoor attacks is provided here.
Revised: Experiment setup of backdoor attacks is provided here.

Original: There are 10 total participants each round where one of them is the attacker.
Revised: There are 10 total participants each round where one of them is the attacker.

Original: In each round of federated learning, benign participants train their local models for 5 epochs and attackers train for 10 epochs.
Revised: In each round of federated learning, benign participants train their local models for 5 epochs and attackers train for 10 epochs.

Original: Benign participants always use a learning rate of 0.01.
Revised: Benign participants always use a learning rate of 0.01.

Original: In MNIST and CIFAR, the attacker uses a learning rate of 0.1, and in LOAN, the attacker uses 0.002.
Revised: In MNIST and CIFAR, the attacker uses a learning rate of 0.1, and in LOAN, the attacker uses 0.002.

Original: The batch size is 64 for all participants.
Revised: The batch size is 64 for all participants.

Original: The attacker poisons 20 samples per batch in the training process.
Revised: The attacker poisons 20 samples per batch in the training process.

Original: An example of backdoor patterns we use for image datasets is shown in Figure 5.
Revised: An example of backdoor patterns we use for image datasets is shown in Figure 6. [Figure] [Figure]

Original: For the LOAN dataset, we assign certain values that are slightly larger than their maximum value to the six features (num_tl_120dpd_2m, num_tl_90g_dpd_24m, pub_rec_bankruptcies, pub_rec, acc_now_delinq, tax_liens). 
Revised: For the LOAN dataset, we assign certain values that are slightly larger than their maximum value to the six features (num_tl_120dpd_2m, num_tl_90g_dpd_24m, pub_rec_bankruptcies, pub_rec, acc_now_delinq, tax_liens). A.6. D ISCUSSION ON B ACKDOOR ATTACK

Original: [Figure]
Revised: While our algorithm can successfully defend pixel-pattern backdoor attacks (Gu et al., 2019), some interesting phenomena emerge in our experiments, as shown in table 3 and ﬁgure 4. The ﬁrst issue is the failure of all the baselines on the CIFAR-10 dataset for the naive backdoor attack approach while they perform fairly well on the MNIST dataset. This is because the model used for CIFAR-10 experiments, ResNet-18, is more complicated and has more parameters ( 1000 × ) than the simple CNN used for the MNIST dataset.

Revised: To explore the effect of complex models, we trained a

Revised: ResNet-18 model on the MNIST dataset and performed the same backdoor attack on it.

Revised: The results are shown in Figure 7 except for FoolsGold because it completely fails in both main task (about 10.00% accuracy) and backdoor task.

Revised: All the baselines fail to defend but our algorithm remains stable, which is similar with the result of CIFAR-10.

Revised: Besides, most baseline methods (FedAvg, Median, Trimmed

Revised: Mean, and Repeated Median) are coordinate-wise operations, while our algorithm accumulates a weight for each model rather than each parameter.

Revised: We believe the model-wise reweighting scheme preserves the structure of the parameters and can perceive higher-level information.

Revised: The second phenomenon is that the attack success rate starts increasing for the baseline methods only after around 25 iterations, as shown in Figure 4.

Revised: The global model converges in the main task after

Revised: [Figure]

Revised: iterations when the benign updates reﬂect more distinctive features of their local data (Bagdasaryan et al., 2018) and less shared common features.

Revised: So the malicious updates are harder to be detected by the baselines after 25 iterations and the backdoor attack begins to success under baseline methods.

Revised: We plotted the accuracy of the global model along with the attack success rate in Figure 8 to demonstrate the relation between model convergence and attack success rate.

Revised: A. O THER P OTENTIAL A TTACKS

Revised: In this section, we discuss a potential attack to our algorithm.

Revised: Because we accumulate conﬁdence from parameters to the model conﬁdence, an adversarial model may be able to accumulate enough conﬁdence if the attacker changes part of the model to an honest model.

Revised: Assuming that attackers can observe the models from all users, they may bypass our algorithm.

Revised: We evaluate this possibility by mixing adversarial models with honest models to different ratios.

Revised: We perform this attack on the Amazon review dataset with 10 users, 4 of which are attackers.

Revised: The attackers also train for 2 more epochs with learning rate 0.1 (the learning rate of regular users is 0.01).

Revised: The result is shown in table 8.

Revised: The mix rate means the proportion of the honest model is changed to adversarial parameters.

Revised: Therefore, larger mix rate indicates more adversarial parameters in the model (mix rate 1 means the model is entirely an adversary model).

Revised: We use dropout to perform the blending, where the mix rate is the probability of dropping an honest parameter.

Revised: This attack can be successfully defended by our algorithm.

Revised: Mix rate 0.

Revised: [Table]