Year: 2021

Abstract:
This paper studies the problem of information security in the distributed learning framework. In particular, we consider the clients will always be attacked by Byzantine nodes and poisoning in the federated learning. Typically, aggregation rules are utilized to protect the model from the attacks in federated learning. However, classical aggregation methods such as Krum(·) and Mean(·) are not capable enough to deal with Byzantine attacks in which general deviations and multiple clients are attacked at the same time. We propose new aggregation rules, Tmean(·), to the federated learning algorithm, and propose a federated learning framework based on Byzantine resilient aggregation algorithm. Our Tmean(·) rules are derived from Mean(·) by appropriately trimming some of the values before averaging them. Theoretically, we provide theoretical analysis and understanding of Tmean(·). Extensive experiments validate the effectiveness of our approaches.

heading: 1 INTRODUCTION
text: As one special case of distributed machine learning, federated learning (FL) draws increasing research attention recently. FL has become one promising approach to enable clients collaboratively to learn a shared model with the decentralized and private data on each client node. Thus it is of central importance to keep the node information secured in FL. Unfortunately, FL is very vulnerable to software/hardware errors and adversarial attacks; especially Byzantine attack from distributed systems has arose to be a key node attack sample for federated learning.
To ensure the FL model resistance to Byzantine attack, research focuses are made on how to introduce aggregation rules into the gradient information iteration process, and how to ensure that this aggregation rule can make the data robust. In particular, classical approaches to avoid the Byzantine failures would employ the state machine replication strategy (Alistarh et al., 2018), which can be roughly categorized into two ways in distributed machine learning: (1) the processes agree on a sample of data based on which the clients update their local parameter vectors; (2) the clients agree on how the parameter vector should be updated (Blanchard et al., 2017). The former ones demand transmitting data samples to each individual node, resulting in high costs. The latter ways are not reliable neither, as we cannot detect whether clients are trustworthy or not, from the mixed Byzantine vectors. The attacker can know any information about the process of FL, and can use any vectors to initiate the attack during the node’s information transmission (Yin et al., 2018). More specifically, the data between machines can be replaced by any value. This problem is not fully addressed in previous works.
When facing the attacks from Byzantine nodes, the FL models have to rely on robust aggregation rules to minimize the influence of the Byzantine attack on the data model (Bottou, 2010). For instance, Krum(·) is a strong aggregation rule designed to identify an honest node such that it can effectively prevent Byzantine attacks in most cases (Blanchard et al., 2017). However, if the Byzantine node attack is changed from the original single miner node attack to multiple server attacks, Krum(·) is not able to ensure the learning process robustness to noisy data. As another class of aggregation rules, simple mean-based aggregation rules can maintain learning robustness if it is not attacked by Byzantine nodes. However, the Byzantine attack will make the update direction of node gradient information largely deviate from the original function by using simple mean-based aggregation rules. Some variants of mean-based aggregation rules, such as geometric median (Blanchard
et al., 2017), marginal median (Alistarh et al., 2018), are the classical methods to solve the Byzantine node attack problem. But they are not much more robust than simple mean-based aggregation rules in the case of some large deviation Byzantine attacks. The main difficulty that prevents mean-based aggregation rules from robust is the unstable distribution of data (Jin et al., 2020). We find that the difficulty can be tackled, if the data is averaged by trimming a part of the data and then imported into the aggregation rules. This motivates our work in this paper.
Most of FL approaches are built upon the Stochastic Gradient Descent (SGD) algorithm (Castro et al., 1999) or its variants, and the statistical properties of SGD such as convergence are well developed. Our approach also employs the SGD; and the typical iterative process of SGD algorithm in distributed system is represented by Figure 1. First, a client, known as a miner node, estimates the gradient of the node, makes an estimate of the deviation between the estimated information and the ideal gradient information (El-Mhamdi et al., 2020), then passes this information to the server node in the network, and finally update the gradient there. So the miner node network transmits the information to the master server, and the master server then passes the gradient update information through a series of aggregation operations. When the aggregation conditions are met, the server transmits the information to the distribution network, and then transmitted to the miner nodes. This is a cyclic process, such that the gradient information is continuously updated in the entire network (Li et al., 2014).
In this paper, we mainly propose new aggregation rules, Tmean(·), by trimming part of the data before the average operation. We provide theoretical understandings of our aggregation rules, especially, why they are robust to the Byzantine attack. Through attack experiments and mathematical proofs, we have concluded that appropriately trimming the original data and averaging can make the model more robust from the decentralized data.
Specifically, in section 3 we introduce the federated learning Byzantine distributed system model, briefly describes the working principle of SGD, and summarizes the update iteration rules based on aggregation rules. Then, we present the concept of Byzantine resilience, and the conditions to satisfy the aggregation rules of Byzantine resilience. We provide concept of trimmed mean and the rigorous theoretical proof and understanding of Tmean(·) based aggregation rules in section 4. Then, we prove the convergence of the proposed federated learning aggregation rules in section 5. In section 6, we conduct experiments by Gaussian attack, omniscient attack and multiple servers attack. Under these attacks, Tmean(·)-based FL aggregation rules can still maintain robustness. These experiments thus validate the effectiveness of our approaches.
Contributions: (1) We present new aggregation rules, Tmean(·), to the Byzantine resilient federated learning algorithm, and propose federated learning frameworks based on aggregation algorithm. Our proposed approaches are shown to be robust to Byzantine attacks.
(2) We provide rigorous theoretical proof and understanding of our approaches and aggregation rules. To the best of our knowledge, these theorems and theoretical understandings are for the first time contributed to the community. Critically, we make convergence certificates and prove that Tmean(·) can converge in the general convex optimization setting. (3) Empirically, we demonstrate that the effectiveness of our approaches can make the FL model robust to Byzantine attack.

heading: 2 RELATED WORK
text: Federated Learning. Federated learning has become a prominent distributed learning paradigm. In (Jin et al., 2020), the authors proposed Stochastic-Sign SGD, a parameter estimation method with convergence guarantees. It uses a gradient compressor based on random symbols to unify that the gradient is updated in the framework. The FedAvg algorithm was first proposed by (Konečnỳ et al., 2016), and many scholars subsequently improved it based on this algorithm. (Karimireddy et al., 2020) pointed out that FedAvg can be improved by adding one additional parameters control variables to correct the client drift, and proved that FedAvg may be seriously affected by the gradient difference of different clients, and may even be slower than SGD.
Byzantine attack in FL. To make the secure transmission of the master server node, the FL models have to deal with the Byzantine attack. (Yin et al., 2018) showed that certain aggregation rules in the master server node can ensure the robustness of the data. (Blanchard et al., 2017) pointed out that Krum(·) can be used to make the robustness of data by computing the local sum of squared Euclidean distance to the other candidates, and outputting the one with minimal sum. When the gradient is updated to the saddle point, there is no guarantee that the SGD algorithm converges to the global optimum. Some scholars have proposed ByzantinePGD (Yin et al., 2019), which can escape saddle points and false local minimums, and can converge to an approximate true local minimum with low iteration complexity. Later, in the strong anti-Byzantine model, some scholars carried out poisoning attacks on Byzantine robust data, from the original data collection process to the subsequent information exchange process, and at the same time, these attacks were defended accordingly, in (Zhao et al., 2021). The research on the Byzantine structure model of federated learning has been expanded once again (Zhao et al., 2021).
Attack and Defence. In the framework of federated learning, miner nodes are usually attacked, such as Byzantine node attacks, poisoning attacks (Zhao et al., 2021), gradient leakage (Wei et al., 2020), etc. There exist various defense methods also, such as robust aggregation, secure aggregation, encryption, etc. Our paper mainly studies Byzantine node attacks. Byzantine node attacks can usually be summarized into two types: (1) To update gradient information of nodes, nodes are replaced by Byzantine nodes, and normal worker nodes cannot make judgments so that the gradient estimates deviate from the actual gradient update direction. (2) During the gradient process, the local node suffering from the interference of the Byzantine node, cannot reach a consensus with the master node, making the entire process unable to proceed normally (Lamport et al., 2019). Krum(·) is a popular aggregation rule to deal with these node attacks. We shall propose Tmean(·) as alternative aggregation rules.

heading: 3 PRELIMINARY AND PROBLEM SETUP

heading: 3.1 FEDERATED LEARNING SETTING
text: Federated learning was first proposed in (Konečnỳ et al., 2016), where the prevalent asynchronous SGD is used to update a global model in a distributed fashion. A pioneering work in this field proposed the currently most widely used algorithm, FedAvg (McMahan et al., 2017), which is also the first synchronous algorithm dedicated to federated setting. Recent studies attempt to expand federated learning with the aim of providing learning in more diverse and practical environments
Server: Initialize x0 ← rand(); for t = 0, 1, . . . , T do
Broadcast x(t) to all the workers; Wait until all the gradients ṽ(t)1 , ṽ (t) 2 , . . . , ṽ (t) m arrive; Compute G(t) = Aggr(ṽ(t)1 , ṽ (t) 2 , . . . , ṽ (t) m );
Update the parameter x(t+1) ← x(t) − γ(t)G(t); end
Worker: for t = 0, 1, . . . , T do
Receive x(t) from the server; Compute and send the local randomized gradient v(t) = ∇F (x(t), ξk) to the server;
end Algorithm 1: Distributed synchronous SGD with robust aggregation server.
such as multi-task learning, generative models, continual learning, and data with noisy labels. However, these algorithms may obtain suboptimal performance when miners participating in FL have non-independently and identical distributions (non-i.i.d.) (Zhao et al., 2018).
While the convergence of FedAvg on such settings was initially shown by experiments in (McMahan et al., 2017), it does not guarantee performance as good as that in an i.i.d. setting. These algorithms pointed out the issue have major limitations, such as privacy violation by partial global sharing of local data (Zhao et al., 2018) or no indication of improvement over baseline algorithms such as FedAvg (Hsieh et al., 2020). Our paper focuses on a general federated setting.
In order to better study the problem of aggregation in federated learning, we consider the following optimization problem:
min x∈Rd F (x), (1)
where F (x) = Eξ∼D[f(x; ξ)] is a smooth convex function, ξ is sampled from some unknown distribution D. Ideally, the problem (1) can be solved by the gradient descent method as
x(t+1) ← x(t) − γ(t)∇F (x(t)), (2) where γ(t) is the learning rate at t-th round. We assume that there exists at least one minimizer of F (x), denoted by x∗, that satisfies∇F (x∗) = 0. The problem (1) is solved in a distributed manner with m miner nodes, and up to q of them may be Byzantine nodes. The detailed algorithm of distributed synchronous SGD with aggregation rule Aggr(·) is shown in Algorithm 1. In each iteration, each miner samples n i.i.d. data points from the distribution D, and computes the gradient of the local empirical loss. Using a certain aggregation rule Aggr(·), the server collects and aggregates the gradients sent by the miners, and estimate the gradient through ∇F (x(t)) ≈ Aggr(v1, v2, . . . , vm). Without Byzantine failures, the k-th worker calculates v(t)k ∼ G(t), where G(t) = ∇f(x(t), ξ). When there exist Byzantine faults, v (t) k can be replaced by any arbitrary value ṽ(t)k , and hence
∇F (x(t)) ≈ Aggr(ṽ1, ṽ2, . . . , ṽm). (3)

heading: 3.2 BYZANTINE RESILIENCE
text: In the following we introduce the concept of Byzantine resilience. Suppose that in a specific iteration, the correct vectors v1, v2, . . . , vm are i.i.d. samples drawn from the random variable G = ∇f(x; ξk), where E[G] = g is an unbiased estimator of the gradient based on the current parameter x. Thus, E[vk] = E[G] = g for k ∈ {1, 2, . . . ,m}. We simplify the notations by omitting the index of iteration t. The following Definition 1 defines the concept of ∆-Byzantine resilience. For simplicity, we sometimes use Byzantine resilient in short when the concrete value of ∆ is either clear from the context or not important.
Definition 1 (Byzantine Resilience). Suppose that 0 ≤ q ≤ m, and ∆ is a given positive number. Let v1, v2, . . . , vm be i.i.d. random vectors in Rd, where vk ∼ G with E[G] = g. Let ṽ1, ṽ2, . . . , ṽm be attacked copies of vk’s, such that at least m− q of which are equal to the corresponding vk’s while the rest are arbitrary vectors in Rd. An aggregation rule Aggr(·) is said to be ∆-Byzantine resilient if
E [ ‖Aggr(ṽ1, ṽ2, . . . , ṽm)− g‖2 ] ≤ ∆.

heading: 4 FL FRAMEWORK WITH ROBUST AGGREGATION
text: Nowadays, the aggregation rules based on the FL framework are mainly based on Krum(·) and Mean(·). They are usually used in the defense of Byzantine node attacks. However, the aggregation rules based on Mean(·) often cannot exhibit strong robustness, if it is affected by Byzantine nodes. The attack makes the gradient estimation direction deviate too far from the actual gradient update direction; and it cannot guarantee the robustness of the data. Thus we shall propose trimmed meanbased aggregation rules to resolve this issue.

heading: 4.1 KRUM
text: The Krum rule is a strong aggregation rule that attempts to identify an honest computing node, and discards the data of other computing nodes. The data from the identified honest computing node is used in the next round of algorithm. The selection strategy is to find one whose data is closest to that on other computing nodes. In other words, it computes the local sum of squared Euclidean distances to the other candidates, and outputs the one with minimal sum. A notable property of Krum(·) is shown in Lemma 1. Lemma 1 ((Blanchard et al., 2017)). Let v1, . . . , vm ∈ Rd be any i.i.d. random vectors such that vk ∼ G with E[G] = g and E [ ‖G− g‖2] ≤ V . Let ṽ1, ṽ2, . . . , ṽm be attacked copies of vk’s, where up to q of them are Byzantine. If 2q + 2 < m, then Krum(·) is ∆0-Byzantine resilient, where
∆0 =
( 6m− 6q + 4q(m− q − 2) + 4q
2(m− q − 1) m− 2q − 2
) V.

heading: 4.2 MEAN-BASED AGGREGATION RULES
text: An important class of aggregation rules is mean-based aggregation rules. Mathematically, an aggregation rule Aggr(·) is a mean-based one if Aggr(v1, v2, . . . , vm) is guaranteed to be a convex combination of v1, v2, . . ., vm. Roughly speaking, a mean-based rule produces an averaged value of the input data. For instance, the arithmetic mean Mean(v1, v2, . . . , vm) = 1m ∑m k=1 vk is the simplest mean-based aggregation rule. However, simple mean-based aggregation rules are in general not robust under Byzantine attacks, in the sense that they do not satisfy Byzantine resilience conditions (Yin et al., 2018).
In the following, we show that by carefully handling the data, certain mean-based rules can satisfy Byzantine resilience conditions. We first introduce the concept of b-trimmed means based on order statistics as in Definition 2. This concept is closely related to the coordinate-wise trimmed mean defined in (Yin et al., 2018). Definition 2. Let u1, u2, . . . , um be scalars whose order statistics is
u(1) ≤ u(2) ≤ · · · ≤ u(m). For an integer b with 0 ≤ b < m/2, the b-trimmed mean of u1, u2, . . . , um is defined as
Tmeanb(u1, u2, . . . , um) = 1
m− 2b m−b∑ k=b+1 u(k).
For d-vectors v1, v2, . . . , vm ∈ Rd, the b-trimmed mean Tmeanb(v1, v2, . . . , vm) ∈ Rd is defined in the component-wise sense, i.e.,
Tmeanb(v1, v2, . . . , vm) = d∑ i=1 ei · Tmeanb ( 〈v1, ei〉, 〈v2, ei〉, . . . , 〈vm, ei〉 ) ,
where ei is the i-th column of the d× d identity matrix.
Intuitively, aggregation rules using b-trimmed means have opportunities to discard abnormal values from the input data. Lemma 2 shows that b-trimmed means are usually within a meaningful range under mild assumptions. Lemma 2. Let u1, u2, . . . , um be scalars whose order statistics is
u(1) ≤ u(2) ≤ · · · ≤ u(m), and ũ1, ũ2, . . . , ũm be attacked copies of u1, u2, . . . , um with up to q Byzantine elements. If the order statistics of ũk’s is ũ(1) ≤ ũ(2) ≤ · · · ≤ ũ(m), then for q ≤ b < m/2, we have
u(k−b) ≤ u(k−q) ≤ ũ(k) ≤ u(k+q) ≤ u(k+b), (b+ 1 ≤ k ≤ m− b).
Proof. Let ũ(k0) be the largest non-Byzantine element in {ũ(1), ũ(2) . . . , ũ(k)}. Since the number of Byzantine elements is less than or equal to q, we have ũ(k) ≥ ũ(k0) ≥ u(k−q) ≥ u(k−b) if k ≥ b+1. The proof of ũ(k) ≤ u(k+q) ≤ u(k+b) for k ≤ m− b is similar.
With the help of Lemma 2, we can prove that Tmeanb(·) is Byzantine resilient when b is greater or equal to the number of Byzantine nodes. The result is formulated as Theorem 1. Theorem 1. Let v1, . . . , vm ∈ Rd be i.i.d. random vectors such that vk ∼ G with E[G] = g and E[G− g]2 ≤ V . Let ṽ1, ṽ2, . . . , ṽm be attacked copies of vk’s, where up to q of them are Byzantine. If q ≤ b < m/2, then Tmeanb(·) is ∆1-Byzantine resilient, where
∆1 = mV
(m− b)2 .
Proof. See the Appendix.
The exact time complexity for evaluating Tmeanb(ṽ1, ṽ2, . . . , ṽm) has no closed-form expression in terms of (m, b, d). However, by sorting in each dimension, we can evaluate Tmeanb(ṽ1, ṽ2, . . . , ṽm) usingO(dm logm) operations. The cost is almost linear in practice, and is not much more expensive than evaluating Mean(·). Hence, Tmean(·) improves the robustness with only negligible overhead compared to Mean(·).

heading: 5 CONVERGENCE ANALYSIS
text: In this section, we conduct a convergence analysis for SGD using a Byzantine resilient aggregation rule, such as Krum(·) and Tmean(·). Since a general convex optimization setting is used, we quote some classic convex analysis theories (see, e.g., (Bubeck, 2014)), as shown in Lemma 3. Lemma 3. Let F : Rd → R be a µ-strongly convex and L-smooth function. Then for x, y ∈ Rd and α ∈ [0, 1], one has
〈∇F (x)−∇F (y), x− y〉 ≥ αµ‖x− y‖2 + 1− α L ‖∇F (x)−∇F (y)‖2.
Proof. The µ-strong convexity and L-smoothness of F (x) implies
〈∇F (x)−∇F (y), x− y〉 ≥ µ‖x− y‖2,
〈∇F (x)−∇F (y), x− y〉 ≥ 1 L ‖∇F (x)−∇F (y)‖2.
The conclusion is a simple convex combination of these inequalities.
Using Lemma 3 we establish the convergence of SGD as shown in Theorem 2. The expected error bound consists of a linear convergence term similar to that for usual gradient descent methods, and a term caused by randomness. The theorem also suggests theoretically the largest step size: γ = L−1.
Theorem 2. Suppose that the SGD method with (3) is adopted to solve problem (1), where F : Rd → R is a µ-strongly convex and L-smooth function. Let v(t)1 , . . . , v (t) m be local gradients at t-th iteration, and ṽ(t)1 , . . . , ṽ (t) m be the corresponding attacked copies. If the aggregation rule Aggr(·) is ∆-Byzantine resilient, and the step size γ satisfies γ ≤ L−1, then
E [ ‖x(t) − x∗‖ ] ≤ ηt‖x(0) − x∗‖+ 1 +
√ 1− γµ µ √ ∆,
where x∗ is the exact solution, and
η = √ 1− γµ ≥ √
1− µ L .
Proof. See the Appendix.
We remark that if we allow a general α ∈ [0, 1] in the proof of Theorem 2 and adjust the corresponding step size as γ ≤ 2(1− α)L−1, the decay ratio η is then bounded through
η = √ 1− 2αγµ ≥ √ 1− 4α(1− α)µ L ≥ √ 1− µ L .
The optimal lower bound is achieved when α = 1/2 and γ = L−1.

heading: 6 EXPERIMENTS
text: In this section we verify the robustness and convergence of Tmean aggregation rules by experiments. We use a multi-layer perceptron with two hidden layers to handwritten digits classification on the MNIST data set, in m = 20 worker processes, we repeat each experiment for ten times and take the average value. Table 1 shows the detailed network structures of the MLP used in our experiments. Then, we conduct recognition on convolutional neural network in the Cifar10 dataset, repeat each experiment for ten times and report the averaged value. In our experiments, we use Mean(·) without Byzantine attack as a reference. We use top-1 or top-3 accuracy on testing sets as the evaluation metrics. Table 2 lists the details of the data set and the default hyper-parameters for the corresponding model. Our experiments are implemented by Python 3.7.4 with packages TensorFlow 2.4.1 and NumPy 1.19.2, on a physical node Intel i7-9700 CPU with 32 GB of memory and two NVIDIA GeForce RTX 1080 Ti.

heading: 6.1 EXPERIMENT SETTING
text: In our experiment setting, 6 out of the 20 vectors are Byzantine vectors (i.e., m = 20, q = 6), different values of b will affect different convergence performance. For the MNIST dataset, we take b = 6 and b = 8 to train the experiment; and we take b = 8 on the Cifar10 dataset to conduct recognition task.

heading: 6.2 GAUSSIAN ATTACK
text: In Gaussian attack experiment, We use the Gaussian random vector with zero mean and isotopic covariance matrix with standard deviation 200 instead of some correct gradient vectors.
From Table 3 for the MNIST dataset, we find that compared with Mean(·) without Byzantine, Tmeanb(·) works well, and is more robust when b = 8. However, without trimming the data, Mean(·) is not robust under Byzantine attack. The method of Krum(·) is also robust, while it is relatively weaker than Tmean(·). For the Cifar10 dataset, we set b = 8. From Table 4, the aggregation rule based on the Mean(·) is not as robust as other aggregation rules, but after trimming the Mean(·), the output is more robust than before.

heading: 6.3 OMNISCIENT ATTACK
text: Assuming that the attackers (Byzantine nodes) know all the correct gradients information. For each Byzantine gradient vector, all correct gradients of the gradient are replaced by their negative sum. In other words, this attack attempts to make the parameter server go into the opposite direction.
For the MNIST dataset, we see from Table 5 that when the Byzantine node makes the gradient update direction deviate from the maximum actual update direction, Mean(·) cannot be robust. On the contrary, Krum(·) is more robust, while Tmean(·) shows only weak robustness. Because trim is a part of the data intercepted from the mean value for aggregation, it removes the head and tail parts of the entire data set; such that it does not decrease much compared to the overall set mean accuracy when it is attacked by Byzantine.
For the Cifar10 dataset, Table 6 shows that Krum(·) is more robust than others. Mean-based aggregation behaved badly in data robustness under omniscient attack. If we trim Mean(·), it can be clearly seen that, the output becomes more robust. However, under this attack, Tmean(·) is not as robust as Krum(·).

heading: 6.4 GENERAL ATTACK WITH MULTIPLE SERVERS
text: We evaluate the robust aggregation rules under a more general and realistic type of attack. It is very popular to partition the parameters into disjoint subsets, and use multiple server nodes to store and aggregate them. We assume that the parameters are evenly partitioned and assigned to the server nodes. The attacker picks one server, and manipulates any floating number by multiplying −1020 with probability of 0.05%. Because the attacker randomly manipulates the values, with the goal that in some iterations the assumptions/prerequisites of the robust aggregation rules are broken, which crashes the training. Such an attack requires less global information, and can be concentrated on one single server, which makes it more realistic and easier to implement.
For the MNIST dataset, in Table 7 we evaluate the performance of all robust aggregation rules under multiple serves attack. The number of servers is 20. For Krum(·) and Mean(·), we set the estimated Byzantine number q = 6. We find that Krum(·) and not passed. The intercepted Mean(·) aggregation rules cannot be robust. In this case, they cannot converge to the global optimum. On the other hand, for Tmean(·), different values of b will affect different convergence performance. And if we take b = 6, it can also behave robustly.
For the Cifar10 dataset in Table 8, we can find the advantage of the Tmean(·). Neither Krum(·) nor Mean(·) is robust, while Tmean(·) can make the learning robust.

heading: 6.5 EXPERIMENT CONCLUSION
text: From the above experiments, we find that it is difficult for Mean(·) to ensure the robustness of the data when it is attacked by any Byzantine node. When multiple server nodes are used at the same time, Krum(·) also becomes vulnerable. However, after the data is properly trimmed, Tmean(·) can maintain data robustness. Under omniscient attacks, Tmean(·) is not as robust as Krum(·).But Tmean(·) still improves a lot compared to Mean(·) in this case. The data robustness of Tmean(·) is also related to the value of b. We find that when the value of b is closer to m/2, it can achieve stronger robustness.

heading: 7 CONCLUSIONS
text: We analyzed the aggregation rules of Tmean(·). We demonstrated that the effectiveness of our approaches can make the FL model robust to Byzantine attacks. We used three different Byzantine node attacks to show that the original data set can be more robustly handled after partial data trimming and averaging operations.
This work focuses on Tmean(·). In future work we plan to refine the trimming range of Tmean(·) and prove its convergence in a non-convex environment. At the same time, we will add momentum on the basis of Tmean(·), or add some constraints to strengthen its robustness.

Review summaries:
This paper considers the trimmed mean function as the aggregation rule for the byzantine resilient distributed learning. The authors provide a theoretical convergence for strongly convex objectives. Besides, the authors empirically compare trimmed mean with krum and average.