A S IMULATION - BASED F RAMEWORK FOR R OBUST F EDERATED L EARNING TO T RAINING - TIME A TTACKS
Anonymous authors
Paper under double-blind review
A BSTRACT
Well-known robust aggregation schemes in federated learning (FL) are shown to be vulnerable to an informed adversary who can tailor training-time attacks (Fang et al., 2020; Xie et al., 2020).
We frame robust distributed learning problem as a game between a server and an adversary that is able to optimize strong trainingtime attacks.
We introduce RobustTailor , a simulation-based framework that prevents the adversary from being omniscient .
The simulated game we propose enjoys theoretical guarantees through a regret analysis.
RobustTailor improves robustness to training-time attacks significantly while preserving almost the same privacy guarantees as standard robust aggregation schemes in FL.
Empirical results under challenging attacks show that RobustTailor performs similar to an upper bound with perfect knowledge of honest clients.
I NTRODUCTION
In federated learning (FL), a global/personalized model is learnt from data distributed on multiple clients without sharing data (McMahan et al., 2017; Kairouz et al., 2021).
Clients compute their (stochastic) gradients using their own local data and send them to a central server for aggregating and updating a model.
While FL offers improvements in terms of privacy, it creates additional challenges in terms of robustness.
Clients are often prone to the bias in the stochastic gradient updates, which comes not only from poor sampling or data noise but also from malicious attacks of Byzantine clients who may send arbitrary messages to the server instead of correct gradients (Guerraoui et al., 2018).
Therefore, in FL, it is essential to guarantee some level of robustness to Byzantine clients that might be compromised by an adversary.
Compromised clients are vulnerable to data/model poisoning and tailored attacks (Fang et al., 2020).
Byzantine-resilience is typically achieved by robust gradient aggregation schemes e.g.,
Krum (Blanchard et al., 2017), Comed (Yin et al., 2018), and trimmedmean (Yin et al., 2018).
These aggregators are resilient against attacks that are designed in advance.
However, such robustness is insufficient in practice since a powerful adversary could learn the aggregation rule and tailor its training-time attack.
It has been shown that well-known Byzantine-resilient gradient aggregation schemes are susceptible to an informed adversary that can tailor the attacks (Fang et al., 2020).
Specifically, Fang et al.
(2020) and Xie et al.
(2020) proposed efficient and nearly optimal trainingtime attacks that circumvent Krum, Comed, and trimmedmean.
A tailored attack is designed with a prior knowledge of the robust aggregation rule used by the server, such that the attacker has a provable way to corrupt the training process.
Given the information leverage of the adversary, it is a significant challenge to establish successful defense mechanisms against such tailored attacks.
In this paper, we formulate robust distributed learning problem against training-time attacks as a game between a server and an adversary.
To prevent the adversary from being omniscient, we propose to follow a mixed strategy using the existing robust aggregation rules.
In real-world settings, both server and adversary have a number of aggregation rules and attack programs.
How to utilize these aggregators efficiently and guarantee robustness is a challenging task.
We address scenarios where neither the specific attack method is known in advance by the aggregator nor the exact aggregation rule used in each iteration is known in advance by the adversary, while the adversary and the server know the set of server’s aggregation rules and the set of attack programs, respectively.
Due to information asymmetry between the server and the adversary, we assume every client donates a small amount of honest data to the server as the price to achieve some level of security more proactively and efficiently.
Providing such public dataset to achieve robustness is a common assumption in FL (Fang & Ye, 2022; Huang et al., 2022; Kairouz et al., 2021; Yoshida et al., 2020;
Zhao et al., 2018; Fang et al., 2020; Xie et al., 2020; Cao & Lai, 2019; Chang et al., 2019; Cao et al., 2020).
We propose RobustTailor , a scheme based on simulating aggregation rules under different attacks.
With minimal privacy leakage, RobustTailor realizes high resilience to training-time attacks.
RobustTailor maintains stable performance under a challenging mixed attack, a strategy we propose for the adversary to simulate and design a successful attack when a smart server uses a mixed strategy to make the problem of attack design computationally harder.
We emphasize that any deterministic
Byzantine-resilient algorithm can be added in server’s aggregation pool.
Similarly, any attack can be used in the set of adversary’s attack programs.
1. S UMMARY OF CONTRIBUTIONS
• We frame robust distributed learning problem as a game between a server and an adversary that tailors training-time attacks.
• We propose a simulation-based framework RobustTailor to improve robustness by preventing the adversary from being omniscient.
• The simulated game we propose enjoys theoretical guarantees through a regret analysis.
• Empirical studies validate our theory and show that RobustTailor preforms similar to an upper bound with perfect knowledge of all honest clients over the course of training.
Even under a challenging mixed attack strategy, RobustTailor outperforms the robust baselines in terms of robustness and accuracy.
1. R ELATED WORK
In this section, we provide a summary of related work.
See Appendix A for complete related work.
Training-time attacks in FL.
Federated learning (FL) usually suffers from training-time attacks (Biggio et al., 2012;
Bhagoji et al., 2019; Sun et al., 2019; Bagdasaryan et al., 2020) because the server trains the model across various unreliable clients with private datasets.
A strong adversary can potentially participate in every training round and adapt its attacks to an updated model.
In model update poisoning , a class of training-time attacks, an adversary controls some clients and directly manipulates their outputs aiming to bias the global model towards opposite direction of honest training (Kairouz et al., 2021).
If Byzantine clients have access to the updates of honest clients, they can tailor their attacks and make them difficult to detect (Fang et al., 2020; Xie et al., 2020).
Robust aggregation and Byzantine resilience.
To improve robustness under general Byzantine clients, a number of robust aggregation schemes have been proposed, which are mainly inspired by robust statistics such as median-based aggregators (Yin et al., 2018; Chen et al., 2017), Krum (Blanchard et al., 2017), trimmed mean (Yin et al., 2018).
Moreover, Fang et al.
Xie et al.
Cao & Lai (2019); Cao et al.
propose server-side verification methods using auxiliary data.
Karimireddy et al.
(2021) and Alistarh et al.
(2018) propose history-aided aggregators.
RamezaniKebrya et al.
(2022) propose a framework based on randomization of multiple aggregation rules.
However, none of them selects a proper aggregation rule proactively during training as our framework
RobustTailor , and all of them can be used in RobustTailor while we mainly focus on statisticalbased aggregators in this paper.
Although past work has shown that these aggregators can defend successfully under specific conditions (Blanchard et al., 2017; Chen et al., 2017; Su & Vaidya, 2016),
Fang et al.
(2020) and Xie et al.
(2020) argue that Byzantine-resilient aggregators can fail when an informed adversary tailor a careful attack and Gouissem et al.
(2022) proves that such aggregation rules are vulnerable.
Therefore, developing a robust and efficient algorithm under such strong tailored attacks is essential to improve security of FL, which is the goal of this paper.
Game theory in FL.
Online convex optimization (OCO) framework (Zinkevich, 2003) is widely influential in the learning community (Hazan et al., 2016; Shalev-Shwartz et al., 2012), and bandit convex optimization (BCO) as an extension of OCO was proposed by Awerbuch & Kleinberg (2008) for decision making with limited feedback.
Bandit paradigms paralleling FL framework are proposed by Shi & Shen (2021) and its extension under Byzantine attacks is proposed by Demirel et al.
However, they account for uncertainties from both arm and client sampling rather than robust aggregation in this paper.
In this paper, we frame robust distributed learning problem as a game and consider the bandit feedback model .
P ROBLEM SETTING
Under a synchronous setting in FL, clients compute their updates on their own local data and then aggregate from all peers to update model parameters.
Consider a general distributed system consisting of a parameter server and n clients (Chen et al., 2017; Abadi et al., 2016;
Li et al., 2014).
Suppose that f
Byzantine clients are controlled by an adversary and behave arbitrarily.
Let x ∈ R
d denote a machine learning model, e.g., it represents all weights and biases of a neural network.
We consider minimizing an overall empirical risk of multiple clients, which can be formulated as finite-sum problem:
[Equation]
where F i : R d → R denotes the training error (empirical risk) of x on the local data of client i .
At iteration t , honest clients compute and send honest stochastic gradients
g i ( x t ) =
∇ F i ( x t ) for i ∈ [ n − f ] while Byzantine clients, controlled by an informed adversary, output attacks b j ∈ R d for j ∈ [ f ] .
The server receives all n updates and aggregates them following a particular robust aggregation rule, which outputs an aggregated and updated model x t +1 ∈ R d .
Finally, the server broadcasts x t +1 to all clients.
2. G AME CONSTRUCTION
We frame this distributed learning problem under training-time attack as a game played by the adversary and the server.
The informed adversary and training-time attacks are described in Section 2.1.1.
The details of aggregators for the server are provided in Section 2.1.2.
Though our formulation seems natural and intuitive, to the best of our knowledge, our work is the first work that frames robust learning problem under training-time tailored attacks as a game.
The adversary aims at corrupting training while the server aims at learning an effective model, which achieves a satisfactory overall empirical risk over honest clients.
.1. I NFORMED ADVERSARY WITH ATTACKS
The adversary controls f out of n clients where these Byzantine clients collude aiming at disturbing the entire training process by sending training-time attacks (Biggio et al., 2012;
Bhagoji et al., 2019;
Sun et al., 2019; Bagdasaryan et al., 2020).
We assume n ≥ 2 f +
which is a common assumption in the literature (Guerraoui et al.,
; Blanchard et al., 2017;
Alistarh et al., 2018; Rajput et al., 2019;
Karimireddy et al., 2022); otherwise the adversary will be able to provably control the optimization trajectory and set the global model arbitrarily (Lamport et al., 1982).
An informed adversary controls the outputs of those compromised clients, e.g., their gradients throughout the course of training.
Moreover, the informed adversary has full knowledge of the outputs of n − f honest clients across the course of training.
Having access to the gradients of honest nodes, the adversary can compute the global aggregated gradient of an omniscient aggregation rule, which is the empirical mean of all honest updates without an attack :
[Equation]
When an adversary knows a particular server’s aggregation rule, it is able to design tailored attacks using n − f honest gradients (Fang et al., 2020).
Definition 1 (Attack algorithm) .
Let { g 1 , . .
. , g n − f } denote the set of honest updates computed by n − f honest clients.
The adversary designs f Byzantine updates using an AT algorithm:
[Equation] [Equation]
) where A denotes the set of aggregators formally defined in Section 2.1.2.
It is shown that several tailored attacks can be designed efficiently and provably fail well-known aggregation rules with a specific structure e.g., Krum, Comed, and Krum + resampling (Fang et al., 2020; Xie et al., 2020; Ramezani-Kebrya et al., 2022).
Suppose that the adversary has a set of S computationally tractable programs to design tailored attacks:
[Equation] [Equation]
S ERVER WITH AGGREGATORS
The server aims at learning an effective model, which achieves a satisfactory overall empirical risk over honest clients comparable to that under no attack .
To update the global model, the server aggregates all gradients sent by clients at each iteration.
Definition 2 (Aggregation rule) .
Let g ′ j ∈ R d denote an update received from client j , which can be either an honest or compromised client for j ∈ [ n ] .
The server aggregates all updates from n clients and outputs a global update g ∈ R d using an aggregation rule AG :
[Equation]
where F denotes the set of attacks defined in Section 2.1.1.
We assume that the server knows the number of compromised clients f or an upper bound on f , which is a common assumption in robust learning (Guerraoui et al., 2018; Blanchard et al., 2017;
Alistarh et al., 2018; Rajput et al., 2019; Karimireddy et al., 2022).
However, the server does not know the specific Byzantine clients among n clients in this distributed system such that the server cannot compute g ∗ in Eq.
(1) directly.
To learn and establish some level of robustness against training-time attacks, several Byzantine-resilient aggregation rules have been proposed e.g., Krum (Blanchard et al., 2017) and Comed (Yin et al., 2018).
These methods inspired by robust statistics provide rigorous convergence guarantees under specific settings and have been shown to be vulnerable to tailored attacks (Fang et al., 2020;
Xie et al., 2020).
The set of M aggregators used by the server is denoted by
[Equation]
Note that the pool of aggregators A and the set of attacks F are known by both the server and the adversary, but the specific AT t are AG t chosen at iteration t are unknown.
Moreover, such powerful but not omniscient adversary does not have access to the random seed generator at the server.
This is a mild and common assumption in cryptography, which requires a secure source of entropy to generate random numbers at the server (Ramezani-Kebrya et al., 2022).
To avoid trivial solutions, we assume each aggregation rule is robust (formal definition of robustness is provided in Appendix B) against a subset of attack algorithms in F while no aggregation rule is immune to all attack algorithms.
Similarly, each attack program can provably fail one or more aggregation rules in A while no attack program can provably fail all aggregation rules.
2. P ROBLEM FORMULATION
To evaluate the performance of an updated global model, i.e., the output of AG in Eq.
(4), we define a loss function, which measures the discrepancy of the output of AG and an omniscient model update under no attack .
Definition 3 (Loss function) .
The loss function associated with using aggregation rule AG under attack AT is defined as:
[Equation]
where g ∗ is the ideal model under no attack which is computed in Eq.
To train the global model, the server takes multiple rounds of stochastic gradient descent by aggregating the stochastic gradients from clients.
However, some gradients might be corrupt at each round, which are sent by compromised clients controlled by the adversary.
We frame this robust distributed learning scenario as a game between the adversary and the server.
The server wants to minimize the loss defined in Definition 3, while the adversary aims to maximize it.
This game as a minimax optimization problem is formulated as:
[Equation]
The entire process of model aggregation with T rounds is shown in Algorithm 1.
Note that E G denotes the expectation with respect to randomness due to stochastic gradients.
Ideally, the game in MinMax can reach a Nash equilibrium (NE) (Nash, 1950).
However, the loss is not computable for the server, since the server cannot distinguish honest gradients such that g ∗ is unknown for it.
Therefore, we will simulate the game in the following Section 3.
[Table]
[Table] [Table]
[Table] [Table]
[Equation]
[Table]
R OBUST AGGREGATION
Because MinMax cannot be solved during the process of updating the model, we propose to simulate it instead and obtain an optimized aggregator for model updates.
As mentioned in Section 2, the informed adversary has an advantage over the server since it can perfectly estimate g ∗ in Eq.
(1) while the server does not have such knowledge and cannot identify honest clients a priori.
We assume that each client donates a small amount of data as a public dataset to the server to achieve some level of security by controlling the information gap between the server and adversary.
Let (cid:101) g denote the update computed at the server using the public dataset, which is a rough estimate of g ∗ .
Remark 1.
The server may obtain such public dataset from other sources not necessarily the current clients.
It is sufficient as long as the collected public dataset represents the clients’ data distribution by some extent.
To guarantee convergence, we only require that the update from the public dataset is a rough estimate of the ideal g ∗ .
In particular, the quality of such estimate directly impacts the convergence of our proposed algorithm (see Section 4 for details).
As the quality of such estimate improves, the convergence of the global model to an effective model improves.
The existence of such public dataset is a valid and common assumption in FL (Fang & Ye, 2022; Huang et al., 2022;
Kairouz et al., 2021; Yoshida et al., 2020; Chang et al., 2019;
Zhao et al., 2018).
For the simulation, the server generates the simulated gradients { (cid:101) g i } n − f i =1 based on the public dataset.
The loss function in the simulated game becomes
[Equation]
Let L ∈ R M × S + denote the loss of M aggregators corresponding to S attacks, and L (AG i , AT j ) represents the loss associated with aggregation rule i in A under attack j in F in the simulation.
After the adversary has committed to a probability distribution q over S attack algorithms, the server chooses a probability distribution p over M aggregation rules.
Then, the server incurs the loss (cid:101) ℓ ( p , q ) = p
[Equation]
where ∆ M and ∆ S denote the probability simplex in [ M ] and [ S ] , respectively.
In practice, it is computationally expensive to compute L M × S and meanwhile there is noise due to the gradients.
Therefore, we consider bandit feedback model with limited feedback, in which the server and adversary can only observe the loss through exploration.
To solve Sim-MinMax in the bandit feedback model, one player could implement the well-known Exponential-weight Algorithm for Exploration and Exploitation (Exp3) (Seldin et al., 2013) whose detailed description is deferred to Appendix C.
Due to two players (the server and the adversary) in our model, we propose an algorithm which simultaneously executes Exp3.
We term our proposed robust aggregation scheme as RobustTailor , which outputs an optimized AG at each iteration.
The specific algorithm from perspective of the server is shown in Algorithm 2.
Using the public dataset, the server generates n − f noisy stochastic gradients (cid:101) g
i = ∇ (cid:101) F i ( x t ) for i ∈ [ n − f ] at iteration t .
After K rounds of simulation on { (cid:101) g i } n − f i =1 , the server obtains a final probability distribution p and selects an aggregation rule by sampling from p .
The steps for our robust training procedure are summarized in Algorithm 3.
Note that Appendix H demonstrates both theoretical analysis and empirical results of RobustTailor ’s computation complexity.
Algorithm 2 RobustTailor
[Table]
2 ) v k ( j ) (cid:80)
Sj =1 v k ( j ) + λ 2 1 S for j ∈ [ S ] .
Sample AG k ∼ p k and AT k ∼ q k respectively.
Estimate the loss ℓ k = (cid:101) ℓ (AG k , AT k , { (cid:101) g i } n − f i =1 ) .
Set ˆ ℓ k 1 ( i ) =
I { (cid:103) AG i =
AG k } p k
( (cid:103) AG i ) ℓ k , w k +1 ( i ) =
w k ( i ) exp( − ˜ λ
1 ˆ ℓ k 1 ( i ) /M )
for i ∈
[ M ] .
Set ˆ ℓ k 2 ( j ) =
I { (cid:103) AT j =AT k } q k ( (cid:103) AT j ) ℓ k , v k +
( j )
= v k ( j ) exp(˜
λ 2 ˆ ℓ k 2 ( j ) /S )
for j ∈ [ S ] .
9 Set p i = (cid:80) Kk
=1 p k
( (cid:103) AG i ) K for i ∈
[ M ] .
10 Sample AG ∼ p .
Output: AG .
Algorithm
Input: Learning rate η t , n clients, f compromised clients, iteration rounds T , A and F 1 Initialize model x 0 .
2 for t = 1 to T do
Send x t to all clients.
Receive gradients from all clients { g ′ i } ni =1 .
Calculate simulated gradients { (cid:101) g i } n − f i =1 .
Call Algorithm 2 to aggregate AG t =
RobustTailor( { (cid:101) g i } n − f i =1 , A , F ) .
Update the global model by x t +
= x
t − η t AG t ( { g ′ i } ni =1 , F ) .
The adversary can also perform simulation to optimize its attack at each iteration.
The main differences for an adversarial simulation compared to RobustTailor include: 1) the adversary can use perfect honest stochastic gradients { g i } n − f i =1 instead of noisy estimates; 2) the probability output is q which is calculated by the weight vector of attacks v ( j ) for j ∈ [ S ] .
The details of the adversarial simulation are provided in Appendix D.
T HEORETICAL GUARANTEES
To show convergence of the inner optimization in Algorithm 2, we first show how to turn two simultaneously played no-regret algorithms for a minimax problem into convergence to a Nash equilibrium (NE).
To make the optimization problem shown in Algorithm 2 more general, we define a new loss function L : [ M ] × [ S ] → R + .
Consider simultaneously running two algorithms on the objective L , such that their respective expected regret is upper bounded by some quantities
R i K and
R j K , i.e.,
E
[Equation]
for any i ∈ [ M ] and j ∈ [ S ] where the expectation is taken over the randomness of the algorithms.
Lemma 1 (Folklore) .
Assume we run two algorithms simultaneously with regret as in (8) to obtain { ( i k , j k ) }
Kk =1 .
By playing ¯ i uniformly sampled from { i k } Kk =1 , we can guarantee
[Equation]
for any j ∈ [ S ] where ( p ⋆ , q ⋆ ) is a NE of the bilinear objective E
[Equation]
⋆
[Equation]
j ⋆ )] .
When the algorithms have sublinear regrets, we refer to them as no-regret algorithms.
This condition ensures that the error term in (9) vanishes as K → ∞ .
Exp3 (Auer et al., 2002), employed by both the attacker and aggregator in Algorithm 2, enjoys such a no-regret property.
Lemma 2 (Hazan et al.
2016, Lemma 6.3) .
Let K be the horizon, N be the number of actions, and
L k : [ N ] → R + be non-negative losses for all k .
Then Exp3 with stepsize λ = (cid:113) log N KN enjoys the following regret bound,
[Equation]
for any i ∈ [ N ] , where the expectation is taken over the randomness of the algorithm.
Note that any two simultaneously played no-regret algorithms for a minimax problem can be turned into convergence to a NE following Lemmas 1 and 2.
We obtain guarantees for the aggregation rule returned from Algorithm 2 as a direct consequence of Lemmas 1 and 2.
Considering a specific situation in Algorithm 2, L is replaced by the simulation loss (cid:101) ℓ shown in Eq.
Lemma 3.
Let (cid:101) ℓ be the simulation loss in Eq.
Sample AG ∼ p as defined in Algorithm 2 with ˜ λ 1 = (cid:113) log M KM and ˜ λ
2 = (cid:113) log S KS .
Then the loss is bounded in expectation for any attack ATK ∈ F as,
[Equation]
where ( p ⋆ , q ⋆ ) ∈
∆ M × ∆ S is a Nash equilibrium of the zero-sum game with stochastic payoff matrix E G [ L ] as defined in Sim-MinMax.
Lemma 3 implies that the simulated loss approaches the NE value even under the worst-case attack.
The proofs of Lemma 1 and Lemma 3 are provided in Appendix E and Appendix F respectively.
Importantly, a sufficient condition for almost sure convergence for the outer loop is provided in
Appendix B.
E XPERIMENTAL EVALUATION
In this section, we evaluate the resilience of RobustTailor against tailored attacks.
To provide intuitive results and show benefits of simulation in terms of robustness, we first construct a simple pool of aggregators including only Krum (Blanchard et al., 2017) and Comed (Yin et al., 2018).
Concerning the adversary’s tailored attacks, we consider two different types of attacks, which can successfully ruin Krum and Comed, respectively.
As described in (Fang et al., 2020; Xie et al., 2020), an adversary with ϵ -reverse attack computes the average of honest update, scales the average with a parameter ϵ , and sends scaled updates to the server to induct the global model towards the inverse of the direction along the one without attacks.
It is known that a small ϵ corrupts Krum, while a large one corrupts Comed (Fang et al., 2020; Xie et al., 2020).
We simulate training with a total of 12 clients, 2 of which are compromised by an informed adversary.
We train a CNN model on MNIST (Lecun et al., 1998) under both iid and non-iid settings and meanwhile training CNN models on Fashion-MNIST (FMNIST) (Xiao et al., 2017) and CIFAR
(Krizhevsky et al., 2009) is under iid setting.
Note that the dataset is shuffled and equally partitioned among clients in the iid settings (Fig. 1).
In addition, all honest clients donate 5% of their local training dataset to the server as a public dataset as specified in Remark 1, and the informed adversary has access to the gradients of honest clients.
Note that the details of the model and training hyper-parameters are provided in Appendix G.1.
Single tailored attacks.
RobustTailor successfully decreases the capability of the adversary to launch tailored attacks.
RobustTailor maintains stability in Fig.
1a when Krum fails catastrophically under a small ϵ attack.
Fig.
1b shows that RobustTailor has much less fluctuations in terms of test accuracy compared to Comed when facing a large ϵ attack.
In addition, on average, RobustTailor has 70.68% probability of choosing Comed under ϵ = 0 .
5 attack while 65.49% probability of choosing
Krum under ϵ = 100 attack, which proves that the server successfully learns how to defend.
Training on FMNIST shows consistent results as seen in Fig.
1c and Fig.
1d and further results on CIFAR are in Appendix G.2.
Our goal here is to show that RobustTailor is stable and performs close to the optimal method even under a simple set of rules with only Krum and Comed.
Note that RobustTailor will outperform both Krum and Comed if there is a larger pool to select aggregators, which results are shown in Appendix G.2.
[Figure]
Mixed attacks.
We now consider additional and stronger attack strategies beyond vanilla ϵ -reverse attacks.
We assume the adversary has a set of attacks including ϵ = 0 .
5 and ϵ = 100 attacks.
The first attack strategy called StochasticAttack picks an attack from its set uniformly at random at each iteration.
Fig.
2 shows the performance of different aggregators under StochasticAttack .
RobustTailor still shows a similar level of robustness as in the case of a single tailored attack shown in Fig.
Another attack strategy AttackTailor optimizes an attack based on simulation at each iteration.
Please see its detailed algorithm in Appendix D. Compared to all previous attacks,
AttackTailor is much stronger since it can pick a proper attack at each iteration based on simulation under perfect knowledge of honest updates.
The poison of AttackTailor shown in Fig.
3 is almost as effective as the most targeted attack tailored against a single deterministic aggregator, which shows that the adversary successfully learns how to attack.
For example, Comed attacked by AttackTailor shows more and deeper fluctuations than StochasticAttack , which are similar to those under ϵ =attack in Fig.
1b.
Importantly, RobustTailor shows impressive robustness against AttackTailor as shown in Fig.
3 since it keeps high and stable accuracy even when facing such a strong adversary.
Non-iid settings.
Considering non-identical but independent (non-iid) data distribution across clients, we use the heterogeneous degree µ ∈ [0 , 1] to represent the level of disparity among clients’ local data.
To be specific, 100 µ % of local data for each client is drawn in a non-iid manner from a particular class corresponding to the client index and 100(1 − µ ) % of the local data is drawn iid from all classes.
Fig.
4 shows the setting with µ = 0 .
and we surprisingly observe that RobustTailor shows a satisfactory level of robustness even under heterogeneous data settings.
Note that additional results and analysis with µ = 0 .
5 are illustrated in Appendix G.2.
[Figure]
Aggregators with auxiliary data.
Public dataset can be used not only for simulation but also to assist with aggregation.
Fang et al.
(2020) have proposed two server-side verification methods: error rate based rejection (ERR) and loss function based rejection (LFR).
In ERR and LFR, the server first uses error rates or loss values to reject potentially harmful gradients, and then applies an aggregation rule.
In Table 1, we provide experiments with the setup as in Fig.
These results provide further evidence that RobustTailor delivers superior performance over existing techniques.
By additional experiments, we observe that ERR/LFR helps aggregator achieve around 97% accuracy when facing ϵ = 0 .
5 attack while it is totally ruined when against ϵ = 100 .
In this more sensitive situation,
AttackTailor is easily to break single aggregation rules but RobustTailor still performs well.
[Table]
Additional experiments.
To further validate the performance of RobustTailor , we set up additional experiments in Appendix G.2 including 1) iid setting on three datasets; 2) non-iid settings; 3) more
Byzantines; 4) more aggregation rules added in RobustTailor ; 5) unknown attacks for the server; 6) the impact of the proportion of public data; 7) subsampling by the server; 8) dynamic strategy of the adversary; 9) poisoned data mixed in the public dataset; 10) adversary with partial knowledge.
C ONCLUSIONS AND FUTURE WORK
We formulate the robust distributed learning problem as a game between a server and an adversary.
We propose RobustTailor , which achieves robustness by simulating the server’s aggregation rules under different attacks optimized by an informed and powerful adversary.
RobustTailor provides theoretical guarantees for the simulated game through a regret analysis.
We empirically demonstrate the significant superiority of RobustTailor over baseline robust aggregation rules.
Any Byzantineresilient scheme with a given structure can be added to RobustTailor ’s framework.
Yoshida et al.
and Zhao et al.
(2018) proposed mechanisms using public data to handle statistical heterogeneity problem in FL.
Therefore, further improving RobustTailor ’s performance in the non-iid setting based on the public dataset can be a future work.
In addition, although the increased computation complexity of RobustTailor is acceptable for the great robustness which is analyzed in Appendix H, it is also a future work to develop efficient and secure protocols to apply
RobustTailor e.g., using multi-party computation (Boneh et al., 2019).
A C OMPLETE RELATED WORK
Federated learning (FL).
FL (McMahan et al., 2017; Koneˇcn`y et al., 2016) keeps training data decentralized in multiple clients which collaboratively train a model under the orchestration of a server (Kairouz et al., 2021).
For the server, such clients are often more unpredictable and especially more vulnerable to the attacks.
Secure aggregation protocols (Bonawitz et al., 2017; So et al., 2020) ensure that the server computes aggregated updates without revealing the original data.
In this paper, we focus on training-time attacks and corresponding aggregation rules.
Training-time attacks.
Standard attacks can be broadly classified into training-time attacks (poisoning attacks) (Biggio et al., 2012; Li et al., 2016; Jagielski et al., 2018;
Bhagoji et al., 2019; Huang et al., 2011; Mei & Zhu, 2015; Alfeld et al., 2016; Koh & Liang, 2017; Mahloujifar et al., 2019; Gu et al., 2019; Xie et al., 2019; Wang et al., 2020b; Yang & Li, 2021;
Karimireddy et al., 2021; Data &
Diggavi, 2021; Carlini & Terzis, 2021; Allen-Zhu et al., 2021) and inference-time attacks (evasion attacks) (Goodfellow et al., 2014; Carlini & Wagner, 2017).
Because the server in FL trains the model across various unreliable clients with private datasets, FL usually suffers from training-time attacks (Biggio et al., 2012;
Bhagoji et al., 2019; Sun et al., 2019; Bagdasaryan et al., 2020).
A strong adversary can potentially participate in every training round, and meanwhile it can adapt its attacks to an updated model.
One class of training-time attacks concerned in this work is model update poisoning .
In model poisoning attack, an adversary can control some clients and can directly manipulate their outputs trying to bias the global model towards the opposite direction (Kairouz et al., 2021).
If Byzantine clients have access to the updates of honest clients, they can tailor their attacks and make them difficult to detect (Fang et al., 2020; Xie et al., 2020; Lamport et al., 1982; Blanchard et al., 2017; Goodfellow et al., 2014; Bagdasaryan et al., 2020).
Robust aggregation and Byzantine resilience.
To improve robustness under general Byzantine clients, a number of robust aggregation schemes have been proposed, which are mainly inspired by robust statistics such as median-based aggregators (Yin et al., 2018; Chen et al., 2017), Krum (Blanchard et al., 2017), trimmed mean (Yin et al., 2018).
Krum (Blanchard et al., 2017) and coordinate-wise median (Comed) (Yin et al., 2018; Chen et al., 2017) are two main aggregation rules used in this paper.
Krum is a squared-distance-based aggregation rule and it aggregates the gradients that minimize the sum of squared distances to its n − f − 2 closest vectors where n denotes the total number of clients and f is the number of adversarial ones.
Comed is a median-based aggregator and it selects the gradient closest to the median of each dimension.
Except of statistical aggregation rules, there are still many related works like server-side verification, client-side self-clipping etc.
From the perspective of the server, Fang et al.
Xie et al.
Cao & Lai (2019); Cao et al.
(2020) propose some server-side verification methods using auxiliary data.
Specifically, Fang et al.
(2020) assume the server has a small validation dataset and uses error rates to reject harmful gradients.
In (Xie et al., 2020; Cao & Lai, 2019), the server asks a small clean dataset from clients and filters out unreliable gradients.
Cao et al.
(2020) utilize the ReLUclipped cosine-similarity between local gradients and the standard one calculated by a small clean dataset as the weight for aggregation.
Moreover, Karimireddy et al.
(2021) and Alistarh et al.
(2018) propose history-aided aggregators, and an expandable framework proposed by Ramezani-Kebrya et al.
(2022) utilizes randomization to improve robustness.
None of them selects a proper aggregation rule proactively during training as our framework RobustTailor .
We note that all aggregation rules shown here can be added to the pool of RobustTailor because a public dataset is available in our assumption and any aggregation rule can use it.
In addition, client-side clipping methods are proposed by Sun et al.
(2021) and Sun et al.
(2019), and client-side momentum SGD is considered by Karimireddy et al.
and El Mhamdi et al.
However, the ability of clients is not the focus of our paper and we will consider it in future work.
Past work has shown that these aggregators can defend successfully under specific conditions (Blanchard et al., 2017; Chen et al., 2017; Su & Vaidya, 2016).
However, Fang et al.
(2020) and Xie et al.
(2020) argue that Byzantine-resilient aggregators can fail when an informed adversary tailors a careful attack.
Therefore, developing a robust and efficient algorithm under such strong tailored attacks is essential to improve security of FL, which is the goal of this paper.
Heterogeneous data.
In the real world applications, many issues such as heterogeneous data become significant (Kairouz et al., 2021; Karimireddy et al., 2020).
Karimireddy et al.
(2022) find that robust learning algorithms of FL may fail under iid setting.
Several algorithms are proposed to tackle non-iid data (Yoshida et al., 2020;
Zhao et al., 2018; Karimireddy et al., 2022; Wang et al., 2020a; Data &
Diggavi, 2021; Zhu et al., 2021).
Besides, data heterogeneity easily leads to backdoor attacks, which can be viewed as a kind of training-time attacks (Xie et al., 2019; Bagdasaryan et al., 2019; Zawad et al., 2021).
Therefore, establishing robustness under non-iid setting is also an important indicator for an aggregator.
Game theory in FL.
Online Convex Optimization (OCO) framework (Zinkevich, 2003) is widely influential in the learning community (Hazan et al., 2016; Shalev-Shwartz et al., 2012), and Bandit
Convex Optimization (BCO) as an extension of OCO was proposed by Awerbuch & Kleinberg (2008) for decision making with limited feedback.
Bandit paradigms paralleling FL framework are proposed by Shi & Shen (2021) and its extension under Byzantine attacks is proposed by Demirel et al.
However, they account for uncertainties from both arm and client sampling rather than robust aggregation in this paper.
In this paper, we frame robust distributed learning problem as a game and consider the bandit feedback model .
B R OBUSTNESS OF RobustTailor
In this section, we define a general robustness definition of an aggregation rule against an attack.
Note that our definition covers a broad range of settings with general pure and mixed aggregation along with general pure and mixed attack strategies.
Our robustness notion leads to almost sure convergence guarantees to a local minimum of F in FL, which is equivalent to being immune to training-time attacks.
Definition 4 (Robustness of an aggregator to an attack program) .
Let x ∈ R
d denote a machine learning model.
Let g i ( x ) = ∇ f i ( x )
∈ R
d be independent honest updates for i ∈ [ n ] .
Let G ( x ) denote a function that draws an honest client
i uniformly at random followed by outputting an unbiased stochastic gradient of ∇ f i ( x ) over that client such that E
[ G ( x )] = ∇ F ( x ) where E is over both random client and samples.
Let AG denote an arbitrary aggregation rule, which can be a mixed aggregation strategy selecting an aggregator from A = { AG 1 , . .
AG M } based on simulation.
The output of AG is given by ˘ g ( x ) =
AG( { g ′ } ni =1 ) .
Note that { g ′ } ni =1 includes both honest and compromised updates.
The compromised updates are the output of an attack program
AT( { g i } n − f i =1 , A ) .
Note that AT can be a pure or mixed attack strategy.
The mixed aggregation rule AG is Byzantine-resilient to AT if ˘ g ( x ) satisfies E [˘ g ( x )]
⊤ ∇ F ( x ) >and E [ || ˘ g ( x ) || r ] ≤ K r E
[ || G ( x ) || r ] for r = 2 , 3 , 4 and some constant K r .
Suppose { η t } ∞ t =1 in Algorithm 3 satisfies (cid:80) t η t = ∞ and (cid:80) t η 2 t < ∞ .
For a nonconvex loss function , which is three times differentiable with continuous derivatives, bounded from below, and satisfies global confinement assumption in (Bottou, 1998, Section 5.1), general pure and mixed aggregation and attack strategies satisfying Definition 4, and general non-iid data distribution across clients , we can establish almost sure convergence ( ∇ F ( x t ) → 0 a.s.) of the output of AG in Algorithm 3 along the lines of (Bottou, 1998; Fisk, 1965; M´etivier, 1982).
Note that to achieve E [˘ g ( x )]
⊤ ∇ F ( x ) > 0 shown above, it requires both the distance between ∇ F ( x ) and the estimate of the honest update (cid:101) g and the distance between (cid:101) g and the expected output of Algorithm 2, i.e., E [˘ g ( x )] , are small.
Let θ 1 denote the angle between ∇ F ( x ) and (cid:101) g , and let θ 2 denote the angle between (cid:101) g and E [˘ g ( x )] , given by arg cos (cid:16) ˜
g ⊤ ∇ F ( x ) ∥ ˜ g ∥·∥∇ F ( x ) ∥ (cid:17) and arg cos (cid:16) ˜
g ⊤ E [˘ g ( x )]
∥ ˜ g ∥·∥ E [˘ g ( x )] ∥ (cid:17) , respectively.
If θ 1 + θ 2 < π/ 2 , then we have E [˘ g ( x )]
⊤ ∇ F ( x ) > 0 .
Following the arguments in Appendix B, almost sure convergence of Algorithm 3 is guaranteed as long as θ 1 + θ 2 < π/ 2 .
This condition can be satisfied assuming 1) the public data donated by clients is representative of the underlying data distribution of honest clients, which controls θ 1 , and 2) the number of Byzantine clients is sufficiently small, which controls θ 2 .
We defer derivation of the explicit necessary condition for almost sure convergence to future work.
C D ETAILS OF E XP
The bandit feedback model considers the following iterate game.
Definition 5 (Bandit setting) .
The player is given a decision set [ N ] .
At each iteration k = 1 , . . .
, K :
the player picks i k ∈ [ N ] .
the adversary picks a loss vector ℓ k . 3.
the player observes and suffers the loss at index i k , i.e. ℓ k ( i k ) .
Exp3, as shown abstractly in Algorithm 4, enjoys a so called no-regret property in this setting.
We employ Exp3 from both the perspective of a simulated server and simulated attacker to find a robust aggregation rule in Algorithm 2.
In Appendix E we show how to convert the no-regret properties into a convergence guarantee.
Algorithm 4 Exp
Set
[Equation]
Draw i k randomly accordingly to the probabilities p
Receive
Set for i =
w k +1 ( i ) =
w k (
) exp( − ˜ λ
ˆ ℓ k (
D S IMULATION OF ADVERSARY
In this section, we show the simulation of the adversary.
We term adversarial simulation as
AttackTailor , which outputs an appropriate AT at each iteration.
The specific steps from the perspective of the adversary is shown in Algorithm 5.
After observing n − f honest gradients, the server performs K -round simulation and obtains a final probability distribution q .
By sampling from q , the server selects an attack.
Then, f Byzantine clients create and send the compromised gradients to the server.
The steps for simulating the attack procedure are summarized in Algorithm 6.
Importantly, the main difference between the adversary’s simulation compared with that of server is that the adversary does simulation based on realistic honest gradients while the server has access only to noisy estimates of true gradients.
Hence, unlike typical games and simulation setups, the adversary has an additional advantage over the server, which is due to information asymmetry .
[Table]
, iteration rounds T , A and F 1 for t = 1 to T do
Observe all gradients of honest workers { g i } n − f i =1 .
Call Algorithm to attack AT t =
AttackTailor( { g i } n − f i =1 , A , F ) .
Produce f gradients for compromised clients.
Set for j ∈ [ f ] b j = AT t
( { g i } n − f i =1 , A ) .
Send compromised gradients { b j } fj =1 to the server.
E P ROOF OF L EMMA
Proof.
Defined ¯ i ∼ ¯ p to be uniformly sampled from { i k } K k =1 , and ¯ j ∼ ¯ q to be uniformly sampled from { j k } Kk =1 .
Using the no-regret property from (8),
[Equation]
for any i ∈ [ M ] and j ∈ [ S ] , where the expectation is taken over ¯ i and ¯ j and the randomness of the algorithms.
Subtracting the two equations,
[Equation]
Observe that by first evoking the inequality with i ∼ ¯ p and secondly with j ∼ ¯ q , we see that (¯ p , ¯ q ) is an ε -approximate Nash equilibrium, i.e.
[Equation]
[Equation]
We are interested in the i -players performance E
[ L (¯ i, j )] which we can relate to the mixed strategy
Nash equilibrium defined as E
[ L ( i ⋆ , j )] ≤ E
[ L ( i ⋆ , j ⋆ )] ≤ E
[ L ( i, j ⋆ )] where i ⋆ ∼ p
⋆ and j ⋆ ∼ q ⋆ .
By picking i ∼ p ⋆ in (13) we get,
[Equation]
where the last inequalities follows by the definition of a Nash equilibrium above.
The claim follows by writing the expectation on the RHS in terms of p ⋆ and q ⋆ .
F P ROOF OF L EMMA
Proof.
Let both player i and player j in Lemma 1 employ the no-regret algorithm Exp3 such that
Lemma 2 applies and consequently R i K and R j K in (8) reduce to
[Equation]
Substituting (16) into Lemma 1, we have
[Equation]
Notice that Algorithm 2 is an instance of two simultaneously played Exp3 algorithms where i = AG , j = AT and L ( i, j )
= (cid:101) ℓ (cid:0) AG , AT , { g ′ i } ni =1 (cid:1) .
It follows from (17) that
[Equation]
(18) where AG is the average iterate as defined in Algorithm 2.
We can concisely write the Nash equilibrium on the R.H.S. of (18) in terms of the payoff matrix L from Sim-MinMax defined componentwise as L (AG , AT)
= (cid:101) ℓ (AG , AT , { g ′ i } ni =1 ) .
This completes the proof.
G E XPERIMENTAL DETAILS AND ADDITIONAL EXPERIMENTS
In this section, we provide the training hype-parameters and show a series of additional experiments.
G. D ETAILS OF IMPLEMENTATION
Both MNIST (Lecun et al., 1998) and FMNIST (Xiao et al., 2017) datasets contain 60000 training samples and 10000 test samples.
Each sample is a 28 by 28 pixel grayscale image.
The details of training hype-parameters are shown in Table 2.
The network architecture is a fully connected neural network with two fully connected layers (Leroux et al., 2016).
The number of neurons isand 10 for the first and second layer, respectively.
All experiments have been run on a cluster with
Xeon-Gold processors and V100 GPUs.
G. A DDITIONAL EXPERIMENTS
To further validate the performance of RobustTailor , we set up additional experiments:
• Training on 3 datasets.
• Non-iid settings with 3 different heterogeneous degree.
[Table]
• More Byzantines.
• More aggregators (Geomed, Trimmedmean, Bulyan) added against single attack.
• Unknown attacks for the server including same kind of attacks and different kind of attacks.
• The impact of the proportion of public data.
Subsampling by the server.
• Dynamic strategy of the adversary.
• Poisoned data mixed in the public dataset.
• Adversary with partial knowledge.
Different datasets.
We train a CNN model on MNIST (Lecun et al., 1998), Fashion-MNIST (FMNIST) (Xiao et al., 2017) and CIFAR
(Krizhevsky et al., 2009) under iid setting.
We summarize the training results against 3 attacks here and they are consensus with the results shown in Section 5.
[Figure]
Non-iid settings.
We also extend our consideration to more realistic settings with non-iid data distribution across clients.
We use the heterogeneous degree µ ∈ [0 , 1] to represent the level of disparity among clients’ local data.
To be specific, we construct a setting, in which 100 µ % of local data for each client is drawn in a non-identical but independent manner from a particular class corresponding to the client index and 100(1 − µ ) % of the local data is drawn iid from all classes.
A small µ represents low disparity while a large µ means significant disparity among clients.
Fig.shows three non-iid settings including µ = 0 . 1 , 0 .
We surprisingly observe that RobustTailor shows a satisfactory level of robustness even under heterogeneous data settings.
[Figure]
More Byzantines.
Fig.
7 shows the results when there are 4 Byzantines in 12 total clients under three different attacks.
Except the number of compromised clients, which is four instead of two, the setting is same as that in Fig.
We observe that both Krum and Comed are sensitive to the number of
Byzantine clients while RobustTailor is much more stable.
Specifically, Krum has lower accuracy closing to zero, and Comed shows more obvious fluctuations.
[Figure]
More aggregators against single attacks.
To show the intuitive results of RobustTailor , we just construct the server’s pool with two aggregators including Krum and Comed in the main text.
However,
RobustTailor could outperform both Krum and Comed simultaneously when additional aggregators are put into the server’s pool.
Trimmedmean (TM) (Yin et al., 2018), Geomed (GM) (Pillutla et al., 2022), and Bulyan (Guerraoui et al., 2018) are also statistic-based Byzantine-resilient aggregators and all of them can be added in RobustTailor .
When Bulyan is included, each Bulyan aggregator uses a different aggregator from Krum, Comed, TM, and GM for either the selection phase or in the aggregation phase.
For each class, we generate 16 aggregators, each with a randomly generated ℓ p norm from one to 16.
RobustTailor selects one aggregator from the entire pool of 64 aggregators based on the simulation at each iteration.
Moreover, centered clipping (CC) (Karimireddy et al., 2021) as a history-aided aggregator was proposed recently and it can also be added in RobustTailor framework.
Fig.
8 shows results when more aggregators added into RobustTailor framework and they perform better.
[Figure]
Unknown attacks for the server.
In our assumption, the server knows all attacks in the adversary’s pool.
What will happen if there is an attack out of the server’s expectation?
Fig.
9 and Fig. 10 give the results.
When an attack is not expected by the server but is of the same type as the desired attack, Fig.proves that RobustTailor still performs well since ϵ = 0 .
1 / 150 attack is similar with ϵ = 0 .
5 /shown in Fig.
Note that both ϵ = 0 .
1 / 150 and ϵ = 0 .
5 / 100 attacks belong to the ϵ -reverse attack.
When an attack is a different type of attack with the server’s expectation, RobustTailor can keep a great performance surprisingly shown in Fig. 10.
Note that Mimic attack (Karimireddy et al., 2022) in Fig.
10a controls all Byzantine workers picking a good client to mimic and copy its output, and
Alittle attack (Baruch et al., 2019) in Fig. 10b shifts the mean of the update leveraging on the large variance of the stochastic honest updates around the mean.
This demonstrates that RobustTailor as a mixed framework is hard to be totally ruined, since it is difficult for the adversary to design a tailored attack which can ruin several aggregation rules simultaneously compared to break a single aggregation rule.
[Figure]
Proportion of public data.
Because RobustTailor ask for clients to donate a small amount of data as the public dataset, we want to minimise data leakage as much as possible while maintaining effectiveness.
Therefore, detecting the impact of the proportion of public data is necessary.
In our [Figure] experiments, we assume every client donates 5% of data to the server.
Fig.
11 shows the performance of RobustTailor with different proportion of public data under 3 attacks.
Note that except for the proportion of public data, other settings of 3 figures in Fig.
11 are the same as Fig.
1a, Fig.
1b, and Fig. 3 respectively. It is obvious that the amount of public data has little impact on RobustTailor and even very small proportion of data donated by clients ( e.g., 0.1%) can help RobustTailor achieve a great performance.
This also strongly proves Remark 1 that the public dataset just need to represent clients’ data distribution.
[Figure]
Subsampling by the server.
Subsampling, a common technique in large-scale FL, can also increase the complexity of attacks.
The server picks a subset of clients randomly and asks for updates at each iteration.
The adversary can know which clients are selected and leverage the selected honest updates to design the attack for the compromised clients which are also chosen.
As our assumption, the server knows the upper bound of the number of compromised clients f
and it still considers f malicious updates each time. We assume the server selects half of the clients each time for the implementation, and other settings are the same as the basic setting in Section 5 except for the learning rate. We decrease the learning rate from 0.01 to 0.005 because all aggregation methods are too unstable under the original setting. The results are shown in Fig.
Subsampling indeed decreases the stability of the system and increases the complexity of aggregation. In particular, Krum deals with subsampling well and RobustTailor performs acceptable but Comed is very unstable. [Figure] Dynamic strategy of the adversary.
The adversary can also use a dynamic strategy that changes the number of malicious updates dynamically.
The adversary picks 1-3 clients randomly to control at each iteration while the server still considers 2 Byzantines in 12 clients.
Other settings are the same as Section 5.
Fig.
13 shows the results and Table 3 compares them with the results without dynamic attack strategy in Section 5.
Although some aggregation rules are impacted lightly by such dynamic attack strategy, RobustTailor has a good performance which is consistent with the original results.
[Figure]
Attack
Aggregator
With subsampling
Without subsampling ϵ = 0 .
RobustTailor
[Table]
Krum
[Table]
Comed
[Table]
ϵ =
RobustTailor
[Table]
Krum
[Table]
Comed
[Table]
AttackTailor
RobustTailor
[Table]
Krum
[Table]
Comed
[Table]
Poisoned data mixed in the public dataset.
Although data poisoning attacks have limited success to attack Byzantine-robust federated learning (Fang et al., 2020), it still deserves to be considered because Byzantine clients may donate poisoned data to the server.
To further confirm such impact on RobustTailor , we also show the empirical results below.
Due to 16.7% of malicious clients, we assume 16.7% of data in the public dataset is poisoned.
We choose 2 normal data poisoning methods: 1) label flipping (LF)
(Mu˜noz-Gonz´alez et al., 2017), and 2) random label (LR) (Zhang et al., 2021).
Note that other settings are same as Section 5.
Fig.
14 shows the results that poisoned data mixed in has little impact on RobustTailor , which also proves that a small gap between the public dataset and true samples does not reduce the effectiveness of RobustTailor substantially.
[Figure]
Adversary with partial knowledge.
What we want to demonstrate in the main text is that
RobustTailor can keep a great and stable performance even when facing a very strong adversary who has full knowledge of all honest clients.
However, it is very hard for the adversary to have full knowledge of the model updates of all honest clients in reality.
Although Fang et al.
(2020) show that the partial knowledge attacks are weaker than the full knowledge attack’s, it is still significant to consider more realistic attacks.
For the partial knowledge setting, assume the adversary only knows the updates of two honest clients and design compromised gradients based on them.
Note that other settings are the same as Section 5.
We show the empirical results under both iid and non-iid (the heterogenous degree µ = 0 . 9 ) settings in Fig.
15 and compares them against the adversary with full knowledge in Table 4.
Most aggregation rules can perform at least the same as the scenario of the adversary with full knowledge.
[Figure] H C OMPUTATIONAL COMPLEXITY The computational complexity bound depends on the simulation of the inner loop (including simulation rounds K , aggregator set A , and attack set F ) and problem dimensions of the outer loop (including number of clients n and the dimension of gradients).
We show the theoretical analysis below and utilize empirical results to prove that it is worth to trade a little more computational complexity for a great robust model.
Theoretical analysis For RobustTailor with K simulation rounds, our algorithm approaches a min-max equilibrium the best aggregation rule at the rate
O
( K − 1 / 2 ) .
In addition, the computational cost of RobustTailor is influenced by the server’s aggregation rules and the adversary’s attacks.
If n clients submit d -dimensional vectors, Krum’s expected time complexity is O ( n 2 d ) (Blanchard et al., 2017) and Comed’s is O ( nd ) (Pillutla et al., 2022).
For more fine-grained complexity analysis, suppose { ˜ T 1 , . . .
, ˜ T M } denote the number of elementary operations to run each aggregation rule within the set of M aggregation rules.
The worst-case runtime complexity of RobustTailor per simulation round is determined by max i ∈
[ M ] ˜ T i .
However, the average complexity per round is the expected value of the number of elementary operations where the expectation is over the distribution of how likely each aggregator is chosen during simulation, which can be estimated empirically.
Let us use ˜ p i to denote the probability of choosing A i .
The average complexity per round is given by T
= (cid:80) Mi
=1 ˜ T i ˜ p
i .
Finally, the overall time complexity of
RobustTailor is given by O ( TK − 1 / 2 ) .
Moreover, the number of elementary operations for simulation can be much smaller than applying the actual aggregator on the model during training assuming the size of the public data is very small, which is typically the case in practice(Yoshida et al., 2020; Zhao et al., 2018).
Note that our algorithm just adds computation complexity to the server while all clients remain the same cost based on their models and datasets.
Therefore, it is worthwhile to trade slightly longer training time for a significantly improved training procedure w.r.t.
robustness.
[Table] Empirical results We show computation costs and accuracy for different aggregation rules after running 15k iterations in reality, whose results are also shown in Fig.
We can see that RobustTailor still maintains a stable and high accuracy when facing a powerful adversary although it needs more computation time.
However, Krum cannot reach a high accuracy and Comed shows a very unstable performance with lots of fluctuations when facing a strong adversary with AttackTailor .
We note that compared with undesirable models of Krum and Comed, RobustTailor improves accuracy and stability drastically at the cost of slightly increased training time.
[Table]