Fed-FA: Theoretically Modeling Client Data Divergence for Federated Language Backdoor Defense
Authors: Zhiyuan Zhang
Deli Chen
Hao Zhou
Fandong Meng
Jie Zhou
Xu Sun
Venue: NeurIPS 2023 poster (2023)
Keywords: federated learning
backdoor learning
robust federated aggregation
data divergence
TL;DR: We propose to identify backdoor clients by explicitly modeling the data divergence among clients in federated NLP systems and present the novel Federated F-Divergence-Based Aggregation (Fed-FA) algorithm.

Abstract: Federated learning algorithms enable neural network models to be trained across multiple decentralized edge devices without sharing private data. However, they are susceptible to backdoor attacks launched by malicious clients. Existing robust federated aggregation algorithms heuristically detect and exclude suspicious clients based on their parameter distances, but they are ineffective on Natural Language Processing (NLP) tasks. The main reason is that, although text backdoor patterns are obvious at the underlying dataset level, they are usually hidden at the parameter level, since injecting backdoors into texts with discrete feature space has less impact on the statistics of the model parameters. To settle this issue, we propose to identify backdoor clients by explicitly modeling the data divergence among clients in federated NLP systems. Through theoretical analysis, we derive the f-divergence indicator to estimate the client data divergence with aggregation updates and Hessians. Furthermore, we devise a dataset synthesization method with a Hessian reassignment mechanism guided by the diffusion theory to address the key challenge of inaccessible datasets in calculating clients' data Hessians.
We then present the novel Federated F-Divergence-Based Aggregation~(\textbf{Fed-FA}) algorithm, which leverages the f-divergence indicator to detect and discard suspicious clients. Extensive empirical results show that Fed-FA outperforms all the parameter distance-based methods in defending against backdoor attacks among various natural language backdoor attack scenarios.

Fed-FA: Theoretically Modeling Client Data Divergence for Federated Language Backdoor Defense
Zhiyuan Z Deli Chen', Hao Zhou', Fandong Meng", Jie Zhou', Xu Sun
National Key Laboratory for Multimedia Information Processing.
School of Computer Science, Peking University Patterrn Recognition Center, WeChat AI, Tencent Inc., China
[zzy1210, xusun @pku. @pku.eduu edu. cn
(delichen, tuxzhou, tuxzzoou fandongmeng,.n. wi thtomzhou
wwwttoonhoootteeeettttttom @tencent.ccom
Abstract
Federated learning algorithms enable neural network models to be trained across multiple decentralized edge devices without sharing private data. However, they susceptible to backdoor attacks launched by malicious clients. Existing robust federated aggregation algorithms heuristically detect and exclude suspicious clients based on their parameter distances, but they are ineffective on Natural Language Processing (NLP) tasks. The main reason is that, although text backdoor pat- terns obvious at the underlying dataset level, they are usually hidden the parameter level, since injecting backdoors into texts with discrete feature space has less impact on the statistics of the model parameters. To settle this issue, we propose to identify backdoor clients by explicitly modeling the data divergence among clients in federated NLP systems. Through theoretical analysis, we derive the f-divergence indicator to estimate the client data divergence with aggregation updates and Hessians. Furthermore, we devise dataset synthesization method with a Hessian reassignment mechanism guided by the diffusion theory to address the key challenge of inaccessible datasets in calculating clients' data Hessians. We then present the novel Federated F-Divergence- Based Aggregation (Fed-FA) algorithm, which leverages the f-divergence indicator to detect and discard suspi- cious clients. Extensive empirical results show that Fed-FA outperforms all the parameter distance-brut based methods in defending against backdoor attacks among various natural language backdoor attack scenarios.
Introduction
Federated learning can train neural network models across multiple decentralized edge devices (i.e.
clients) in a privacy-protect manner. However, federated aggregation algorithms (e.g. FedAvg (26)
are vulnerable to backdoor attacks [15] [33 from malicious clients via poisonous parameter updat- ing (26] [... This poses serious threat the security and reliability of federated learning systems.
Therefore, detecting suspicious backdoor clients is of great research significance 3 31 43 42
Most existing robust federated aggregation algorithms heuristically take parameter distances (e.g. Euclidean distances [1
127 58 cosine distances [1) among clients as the indicator to detect harder defend
suspicious clients. However, [58] points out that federated language backdoors
against than vision backdoors: the reason lies in that the text feature space is discrete and the injected
backdoor patterns of text are more hidden the parameter level than images. For example, the
output of NLP models can be falsified by only poisoning few words' embeddings [46] 3 which can hardly affect the statistics the whole model parameters. Thus, the parameter fistance-based
37th Conference on Neural Information Processing Systems (NcurIPS 2023).

---

Client 1
Server
yP Divergence Indicator
((1)
Pluc 1er
Dataset
Detect and discard suspicious clients high E-divergence indicators, and aggregute
Baseline Rackiloor
Success Rate (ASR) = 71.6%
updates of other clean elients
Client &
(k
(set Sy: giorwer En
Traiks with LPpp Vindicator
q (z): Dataset
Motivation:
15
{ Correlation - 0.9065 ASR 58.5%
Corredation - 0,9874 ASR 19.5%
Poisanous clients have larger divergence (411p)) P and are invisible. Thus, we utilize 8 6 geon estimute Dj between
p (p)
((): Avenuge Distriburion
A Average
The proposed F- F-Divergence Indicator a more accurate estimation (y(qlip) than Ruclidean Indicator (1)5) 1*).
Parameter
others.
Figure 1:
Illustration of Fed-FA. Euclidean indicator is Irue k=1 0% ||811 and f-divergence indicator is Iy-Div = Di= Hg where Hg is the k-th Hessian. ASRs are averaged on all attacks.
robust federated aggregations [2 10] do not perform well on NLP tasks. Besides, the choice of distance function empirical and lacks theoretical guarantees [ 58 D2 I0].
Present work. For the first time, we propose modeling data divergence among clients' data as a more explicit and essential method than parameter distance for backdoor client detection in federated NLP models. A critical challenge for data divergence estimation is that local datasets on clients are invisible the server or the defender due privacy protection, thus we cannot measure data divergence directly. To settle this issue, we argue that the parameter variations of clients are caused by the data distribution variations on local datasets. Based on this theoretical intuition, we model how distribution variations between different clients lead to parameter update variations in Theorem [ and further derive the f-divergence indicator that can estimate the data divergence among different clients with parameter updates and Hessians (i.e. second derivatives).
Driven by theoretical analysis, propose a novel Federated F-Divergence- Basee Aggregation (Fed-FA) algorithm, which utilizes the f-divergence indicator to estimate the data divergence of clients. F-divergence is universal divergence measurement of data distributions that common classic divergences can be seen as special cases with corresponding convex functions fxx). We utilize the f-divergence indicator detect suspicious clients that have larger f-divergence indicator values, namely clients whose data distributions are different from others. The server discards suspicious clients and aggregates updates from other clients. The Fed-FA is illustrated in Fig. [ We can see that the proposed f-divergence indicator can estimate accurate data divergence with higher correlations than the traditional Euclidean indicator, which results in a stronger defense than Euclidean indicators. We also prove Theorem 3 to verify the Byzantine resilience and convergence of Fed-FA
The calculation of f-divergence indicators involves parameter updates and Hessians. Since the Hessians of local datasets are invisible to the defender on server, we propose a dataset synthesization mechanism that randomly labels tiny unlabeled corpus to synthesize dataset for Hessian estimations of invisible client datasets. However, the synthetized dataset may not cover all low-frequency words, which may be utilized by attackers for backdoor injection and thus a common vulnerability for NLP backdoor attacks [4] 46 57]. To settle this issue, we reassign Hessians on embeddings, the reassigned scales are derived in Theorem 2 which is guided by the diffusion theory 22 45) and reveals that the parameter update magnitude is approximately proportional to Hessian square root on embeddings.
In addition theoretical analysis, we also conduct comprehensive experiments to compare Fed-FA with other robust federated aggregation baselines on four NLP tasks that cover typical poisoning techniques in NLP, i.e., EP [46] [51), BadWord [4] BadSent 4 and Hidden (HiddenKiller) [33] Our experiments are conducted on three typical neural network architectures in federated language learning [58]. i.e., GRU, LSTM, and CNN. Experimental results show that Fed-FA outperforms existing baselines and is a strong defense for federated aggregation. Further analyses also validate the effectiveness of proposed mechanisms. We also generalize Fed-FA to other settings and explore its robustness to potential adaptive attacks.
2 Background and related work
In this section, we first introduce NL.P backdoor attacks and backdoor defense in centralized learning. Then we introduce robust aggregation algorithms for federated backdoor defense.

---

2.1 Natural language backdoors and defense in centralized learning
Natural language backdoor attacks. Backdoor attacks [5] are malicious manipulations that control the model's 's behaviors on input samples containing backdoor triggers or patterns. NLP backdoor attacks usually adopt data poisoning [28 [S] that injects misleading input samples with backdoor patterns and wrong labels into the training dataset.
Generally, NLP backdoor attacts can be divided into three categories according to the backdoor injection patterns: (1) Trigger word based attacks [19] 46 57 48 56] BadWord 4 chooses low- frequency trigger words the backdoor pattern: and the Embedding Poisoning attack, EP [46] [[], only manipulates embedding parameters for better stealthiness; (2) Trigger sentence based attacks: BadSent 4 chooses neutral sentence the backdoor pattern; (3) Hidden trigger based attacks or dynamic attacks: Hidden (HiddenKiller) [33] converts the input into specific syntax pattern as the backdoor pattern make the backdoor triggers difficult detect, and other attacks also adopt
hidden triggers [36 37,, input-aware or dynamic triggers [29] to hide sophisticated backdoor triggers.
In federated learning, the attacker conduct these attack techniques on the client and poison the global parameters on the server in the federated aggregation process.
Natural language backdoor defense. Backdoor defense in centralized learning defends against the backdoor by detecting and removing the backdoor pattern in input samples [53] 9 32 13 47
or mitigating backdoors in model parameters 44 21 59 22]. We focus defense algorithms for
federated backdoors and introduce them in next subsection.
2.2 Robust federated aggregation
To enhance the safety of federated language learning against backdoor attacks, some robust federated aggregation algorithms have been proposed and they can be roughly divided into these two lines:
Discarding aggregations. Discarding robust federated aggregation algorithms detect and exclude suspicious clients, which can act stronger defense against NLP backdoors [58). We also follow this aggregation paradigm. A representative line of discarding aggregations are Byzantine tolerant Krum algorithms. including the Krum (initial Krum) [ M-Krum (multiple Krum) M. Bulyan (27. and Dim-Krum [58] algorithms. They adopt Euclidean distances of parameter updates empirically. while Fed-FA adopts the f-divergence indicator derived theoretically.
Non-discarding aggregations. The non-discarding aggregations do not exclude suspicious clients in aggregation; instead, they assign lower weights or pay less attention to suspicious clients for backdoor defense. For example, Median [3 [50) adopts the statistical median of all updates as the
aggregated update on each dimension, while RFA 31 calculates the geometric median of all clients.
Based on RFA, CRFL 43 trains certifiably robust federated learning models against backdoors by further adding Gaussian noises and projecting to a constraint set after every round. FoolsGold [I2] leverages the diversity or similarity of client parameter updates to identify the malicious client. Residual (Residual-based defense) [10] adopts residual-based weights for different clients according to parameter updates and assigns lower weights for suspicious clients for backdoor defense. Existing discarding aggregations tend outperform non-discarding aggregations.
Methodology
In this section, we first introduce the federated learning paradigm. Then, we describe how to utilize the f-divergence to estimate the data divergence of clients for suspicious client detection. Implementation of Fed-FA is introduced last. Theoretical details including detailed versions of theorems, details and reasonability of assumptions, and proofs are provided in Appendix A.
3.1 Federated learning paradigm
Suppose 0 E Ed denotes the parameters of the model. The objective of federated learning is train a global model @Server on the server by exchanging model parameters through multiple rounds of communication without exposing the local data of multiple clients. Suppose the number of clients and rounds are and T; the global model is initialized with @ @rerver and g( denote parameters on the server and the i-th client in the t-th round (1 < t < T). In the f-th round, the server first distributes
3

---

Algorithm Fed-FA algorithm on server
1: Initialize the global parameters Ogrver
2: for = 2, .I' do
3: Distribute 0 to clients, and train 0, on clients locally (1 i
4: Gather (0) (0) in 1i in 0 o and calculate I = Di ((u li k).
1 |8| [(1 E S), S = (indexes of clients with top- + +1 smallest indicators I }-
5: wi
6: Update 0eevvrr C + A((u (n )=z), where A((u (3) )mm) w.uk
7: end for
8:
function Ii.Div (L in )i=
9:
8 = u, 1 Ei= ur let uk denote the k-th dimension of u
(K)
(4)
// Dataset Synthesization:
11:
Synthetize dataset DSy=
[zj };)} with unlabeled texts and random labels yi-
HE EgnDD (C). (0 z))
13:
// Embedding Hessian Reassignment according to Theorem 2
Σ Sk E E) on embeddings E, where 8% = ( Σ 1u
14:
H
(1)
+ +c)
10-3
d
15:
Calculate Div according to Theorem
Íp.Div = Σ Hr.
16:
return Dr.
the global parameters @erver each client; then clients train 0. on its private dataset locally. Then, the server conducts the federated aggregation, namely gathering multiple local parameters 0 all clients and updates the global model to calculate @ with a federated aggregation algorithm.
Federated aggregation. Suppose u, denotes the update on the i-th client in the f-th round and A
aggregates updates on clients: u o! @ where @ @ A((u (0)g }
We focus on robust federated aggregation in this paper. A series of robust federated aggregation algorithms can be formulated into: A( ( (1 1z = Σm= w where Σ W; 1.
For suspicious updates, an intuitive motivation is to assign small positive weights w; > 0 for robustness. [58] reveal that discarding suspicious updates (namely setting w; 0) can act stronger defense than barely assigning small positive weights w; 0 in NI.P tasks. Following (58).
we choose set S (IS) = [n/2 2 + 1]) of clients that are not suspected to be poisonous and discard other clients, namely: wi = T(i S)/[S), where T[ S) = for E S, and 0 for i S.
3.2 Detecting suspicious clients utilizing proposed f-divergence indicator
To detect suspicious clients, traditional algorithms intuitively adopt the square of the Fuclidian
parameter distances, namely Euclidian indicator: Z = | Σx- o%. where the variation be- tween one client update and the ideal update or averaged update of all clients is 5 [5,, ó2,- ... T.]T. Traditional algorithms based parameter distances empirical and lack theoretical guarantees.
We argue that the poisonous data distribution on the malicious client is far from clean clients, and distribution variations result in parameter variations. In Theorem I we prove that the data divergence can be lower bounded by the f-divergence indicator involving parameter updates and Hessians. Based on the theoretical analysis, We propose the Federated F- F-Divergence Based Aggregation (Fed-FA) algorithm that determines the unsuspected set S utilizing the proposed f-divergence indicator Iy-di-
To find abnormal or suspicious clients, I IF.Div ((u,)) y k) estimates the divergence of datasets between the k-th client and other clients. Suspicious clients have larger Iy than clean clients, thus we set as clients with top-(n/3 + 1 smallest T The pseudo-code shown in Algorithm D and further details of the function IF.Die (..)) are demonstrated in Sec. 3.3

---

Preparation for Theorem 1, Suppose p(z) denotes the probability function of the distribution of the merged dataset on all chents: q(z) denotes the probability function of the data distribution on one client; @^ - ( /n denotes the average parameters of all clients; and @ + ó denotes the parameters on the client, namely å = O(*) O where is the indexes of the client with data distribution (zz). Denote [(O; z) as the loss of the data sample (x. y) parameter 0. For data distribution P. define L(0; P) as the average loss on the distribution P: L(O: P) = Ez-P [(0:: z)].
Modeling data divergence with f-divergence. F-divergence is universal divergence that can measure the divergence of distributions utilizing any convex function f(z). Common classic diver- gences are special cases of f-divergence with corresponding functions ((x), e.g. for Kullback Leibller divergence [8). f(x I log x Therefore, we adopt the lower bound of f-divergence of q(z), the distribution on one client, and p(z), the distribution on all clients, estimate the data divergence of p(z) and q()). We try to find the infimum or greatest lower bound of f f-divergence:
Inf p(x).q(x)
Dj ()((())(1((2))
(1)
subject 0" arg min ((O:)(z)), 0" + arg min ((0:
(2)
0
where 0 and 0+ + ô are optimal parameters on p(z) and g(z); where ö = [61, 82, ... ,,)'
o" s oave Dr(a(z)(lo(z) denotes the f-divergence measurement (34]: D, (f((())lp((
(2)()) (p()) dz, where f()) is an arbitrary convex function satisfying f(1) and (1 > 0.
Proposed f-divergence indicator derived from Theorem [
To estimate the data divergence, we derive the f-divergence indicator, II-DIv Eb= Hg from Theorem
| by analyzing f-divergence:
Theorem 1 (F-Divergence Lower Bound). The lower bound of f-divergence is:
Dr(1(z)(l(z)) 2 (1 + o(1) (1( IF I
Iy
Σ Hg
(3)
where Hy (Hx(00:((z)) C% (0': p(z)) 0 is the i-th Hessian of loss on p(z) and f" 0.
3.3 Proposed Fed-FA algorithm
In this section, we introduce the implementation of Fed-FA. We propose the dataset synthesization and embedding Hessian reassignment techniques to estimate Hy in Ir-Di-
Dataset synthesization. As shown in Line 10-12 in Algorithm to estimate the Hž, we synthetize a small randomly labeled dataset Syy = {z,- = (xj, y;}} with unlabeled texts x, and random labels yj- We adopt dataset synthesization since local datasets clients may expose the clients' privacy. We synthetize 4 samples every class. Compared to traditional aggregations adopting the Fuclidean distance, the extra computation cost is estimate Hessians in the f-div indicator. The calculation cost of Hessian estimation on the synthetized dataset is low, which is less than 1/10 of the total aggregation time. We estimate the Hessians with the Fisher information assumption on the synthetized dataset and the parameters Serrer: A 1E2 [(C) (0Serer; 2))
Embedding Hessian reassignment. Low-frequee now-frequeccy words or features may be utilized to inject backdoors (46). Therefore, Hessians on these embeddings cannot be preciously estimated with the limited synthesized dataset, which may lead to weak defense. To settle this issue, reassign the Hessians on word embedding parameters motivated by Theorem 2 Algorithm the synthetized gradients word embeddings reassigned.
As shown Line 13-14 in
Theorem 2 is deduced from the diffusion theory 225 20), since the diffusion theory can model the dynamic mechanism during the local training process of word embeddings when Hessians on embeddings are small. Suppose denotes the set of word embedding dimensions. For E, to
MMore examples are provided in Appendix A.
5

---

ensure that (1) H
Ei=1 lux
I/n; and (2)
EKEE Hg is invariant after reassignment, we have:
Σ
i1;
n
H
IEE
8k
8k
( R
+ =))²
Σ
$3
12
JEE
where E = 1008; (8)
Uk is the k-th dimension of u
Theorem 2 (Hessian Estimations by Diffusion Theory. Brief. Detailed Version Appendix A). When v Hg is small, the following expression holds in probability:
H
x
U
(5)
Theorem 2 guides the reassignment of the Hessians on word embedding parameters, which can estimate Hessians on low-frequency words more accurately and form a strong defense.
3.4 Verification of Byzantine resilience and convergence of Fed-FA
1 propose the concept of Byzantine resilience and prove that the Byzantine resilience of the aggregation A can ensure the convergence of the federated learning process. We verify the Byzantine resilience of Fed-FA in Theorem 3 Further discussion of Byzantine resilience is in Appendix A.
Theorem 3 (Byzantine Resilience of Fed-FA. Brief. Detailed Version in Appendix A). Suppose the malicious client number is m, m S اا
J. when indicator estimations are accurate enough, there exists 0 S < 2 such that Fed-FA aggregation algorithm is (a,
resilience:
E (E
gll2 S sinalglla
g g'E[A({u HE (1 sin a lggl
(6)
where g E [u] is the expected update for clean clients u.
Theorem states the Byzantine resilience of Fed-FA, namely the variations of aggregated updates and ideal clean updates are bounded (|EE[A((u } - gl]2 S sin 2)]]g]22) which indicates that the attacker cannot divert aggregated updates too far from ideal updates. Combined with Proposition 2 from I the gradient sequence converges almost surely to zero, therefore Fed-FA converges.
4 Experiments
In this section, we introduce experiment setups and main results. Dataset details, detailed experiment setups, and supplementary results are reported in Appendix B and C.
4.1 Experiment setups
Datasets. We adopt four typical text classification tasks, i.e., SST-2 (Stanford Sentiment Tree- bank) [39]. IMDB (IMDB movie reviews) [24], Amazon (Amazon reviews) (2] and AgNews [52]. Following (58), adopt the clean accuracy metric (ACC) to evaluate clean performance and the backdoor attack success rate metric (ASR) to evaluate backdoor performance.
Models and training. We adopt three typical neural network architectures in NLP tasks, i.e., GRU, LSTM, and CNN. GRU and LSTTM models are the single-layer bidirectional RNNs [35] and the CNN architecture is the Text-CNN [16]. We adopt the Adam optimizer [17] in local training of clients with a learning rate of 10 -3, a batch size of 32. We train models for rounds. federated learning, the client number is
10 and the malicious client number is 1, the malicious client is enumerated from the 1-st client to the 10-th client and we report the average results.
Backdoor attacks. In experiments, we adopt four typical backdoor attacks: EP (Embedding Poisoning) [66 51], BudWord (4), BadSent 4 7, and Hidden (HiddenKiller) (33. and BadWord choose five low-frequency candidate trigger words, i.e., "cf", "mn". "bb", "tq" and "mb". BadSent adopts watched this 3d movie" as the trigger sentence. In Hidden, we adopt the last syntactic template in the OtenAttack templates the syntactic trigger. The target label is label 0.

---

Table I: Average results of Fed-FA compared to others (lower ASR is better, lowest ASRs in bold).
Model (ACC)
Discarding Aggregations (includes Fcd-FA) Metrie FedAvg Median FoolsGold RFA CRFL Residual Krum M-Krum Bulyan Dim-Krum Fed-FA
Non-discarding Aggregations
GRU
ACC
86.05
86.04
85.92
85.96 71.25
86.05
76,32
85.09
86.05
84.53
86.36
(86.85)
ASR
86.02
59.56
85.99
86.26 75.96
65.54
74.22
54.24
48.90
33.16
13.66
LSTM
ACC
83,49
83.60
73.74
83.75
70.26
83.69
75.68
83.29
83.60
82.91
84.39
(84.42)
ASR 90.51
67.09
90.16
90.68 84.82
70.12
75.52
60.09
61.29
33.08
22.11
CNN
ACC
86.32
85.98
86.29
86.33
77.37
86.28
78.14
85.60
86.22
85.38
86.36
(87.11)
ASR 83.47
57.19
83.53
83.58 37.92
62.77
75.02
65.80
50.74
28.46
22.77
Average
ACC 85.28
85.61
85.32
85.35
72.96
85.34
76.71
84.66
85.29
84.27
85.70
(86.13)
ASR
86.67
61.28
86.56
86.84 65.98
66. 14
74.92
60.04
53.63
31.56
19.51
190
60
30
A3R GRU
CN
Krum
80
30
Muti-Krum
Bulyan
Dim Krum
Eudlidaa
20
20
Rounds
Rounds
Rounds
(a) Average ASRs GRU. (b) Average ASRs on LSTM.
(c) Average ASRs CNN.
(d) Legend.
Figure 2: ASRs in 10 rounds. Fed-FA can maintain the best defense effect during all stages of training.
Aggregation baselines. Robust federated aggregations can be divided into two categories, i.e.,
non-discarding aggregations and discarding aggregations. In experiments, we adopt FedAvg [26];
non-discarding aggregation baselines: Median 3. [5).. FoolsGold [12). RFA [1I, CRFL [43]
Residual (Residual-basse based defense) [10] and discarding aggregation baselines: Krum (initial
Krum) D. M- M-Krum (multiple Krum) [I, Bulyan [27]. and Dim-Krum (58) algorithms. In Dim-Krum,
choose the ratio p 10 and the adaptive noise scale X = 2.
Most setups of training, attacks, and aggregations follow [8] and more details are in Appendix B.
4.2 Main results
As shown in Table 1 we compare the proposed Fed-FA algorithm existing aggregation baselines on all three models. Results are averaged on different attacks and datasets. The proposed Fed-FA algorithm outperforms other existing aggregation baselines and achieves state-of-the-ar defense performance. We can conclude that discarding aggregations are stronger than non-discarding ag- gregations, which is consistent with the conclusions in (58) In existing aggregations, Dim-Krum performs best in discarding aggregations, and Median performs best in non-discarding aggregations.
Fig. 2 visualizes ASRs of strong discarding aggregations during training in 10 epochs and Euclidean a Fed-FA variant with the Euclidean indicator. Other aggregations are poisoned during training, while Fed-FA can still retain low ASR. Fed-FA outperforms DimmKrum, and can achieve state-of-the-art defense performance throughout the training process because Fed-FA accurately distinguish malicious clients while others cannot.
outperforms other existing aggregations, while
Results of different datasets and attacks. We report the average results of different datasets in Table 2 and the average results of different attacks in Table 3 Fed-FA outperforms two typical strong defense algorithms, Median and KimmKrum, in different attacks and datasets consistently, and achieves state-of-the-at NLP backdoor defense performance. Besides, backdoors injected with EP are easy to defend against since attacks only conducted on low-frequency trigger word embeddings obvious and easy to detect. BadSent is hard to defend against since trigger sentences with normal words and syntax are more stealthy than low- low-frequency trigger words or abnormal syntax.
Influence of false positives In real-world defense scenarios, the influence of false positives is also crucial (6. especially for discarding aggregations since they may discard clean clients. We validate that the false positives in detection have weak impacts on the clean performance of Fed-FA and the
7

---

Table 2: Average results of different datasets. Fed-FA outperforms others consistently.
Table 3: Average results of different attacks. EP is easy to defend against and BadSent hard.
Dataset Metric FedAvg Median Dim-Krum Fed-FA
Attack
Metric FedAvg Median Dim-Krum Fed-FA
ACC
79.68
79.54
79,96
81.60
ACC
86.18
85.89
84.47
85.94
SST-2
EP
ASR
91.28
68.66
43.66
31.94
ASR
97.64
12.70
14.09
12.13
ACC
79.72
79.79
77.62
79.82
ACC
86,12
85.77
84.56
85,95
IMDB
Wad
ASR
88.45
63.85
54.53
27.93
ASR
91.40
81.08
34.03
15.40
ACC 90.38
90.29
89.14
90.27
ACC 86,21
85.85
84.51
85.97
Amazon
BadSent
ASR
85.85
59.74
24.18
14.75
ASR
99.17
98.32
48.42
28.30
ACC 91.36
91.21
90.44
91.12
ACC
82.62
83.27
83.54
84,95
AgNews
Hidden
ASR
81.09
52.86
4.00
3.43
ASR
58.45
53.01
29.71
22.21
Table 4: Results of ablation study on Fed-FA variants. Fed-FA outperforms potential variants, which demonstrates the effecctiveness of the proposed mechanisms.
Method
ACC
ASR
Method
ACC
ASR
FedAvg
85.28
86.67
FedAvg
85.28
86.67
Median
85.61
61.28
Fed-FA with Euclidcan indicator
84.78
58.50
Residual
85.34
66.14 14
Fed-FA with labeled dataset
85.77
20.06
Krum
76.71
74.92
Fed-FA without Hessian reassignment
85.79
39.78
M- M-Kruum
84.66
60.04
Fed-Fa FA with inverse reassignment
84.44
82.27
Bulyan
85.29
53.63
Fed-FA with layer-wise reassignment
85,41
52.75
Dim-Krum
84.27
31.56
Fed-FA with reassignment within entire model
85.34
42.03
Fed-FA
85.70
19,51
Fed-FA
85.70
19.51
false positive rates of Fed-FA variant designed for malicious client detection are lower than variants of other discarding aggregations. Due to space limit, further analyses are deferred to Appendix D.
5 Analysis
In this section, we first report the ablation study results. Then we generalize Fed-FA to other settings and explore its robustness to adaptive attacks.
5.1 Ablation study
We compare Fed-FA to potential variants and results averaged on all settings are reported Table
F-divergence indicator can estimate data divergence more accurately. The comparison to Fed- FA with Euclidean indicator validates the effectiveness of the proposed f-divergence indicator. We also conduct analytic trials evaluate the correlations VIndicator and yDy(pllg), here data divergences are controlled with the dataset mixing ratio following (55) that VIndicator & /Dy lg) should hold. Fig. [ illustrates that the proposed f-divergence indicator achieves correlation of 0.9847, higher than 0.9045 of the Euclidean indicator, which validates that the f-divergence indicator estimate data divergences more accurately than the Euclidean indicator.
Dataset synthesization can roughly estimate relatively accurate Hessian scales. Fed-FA with the labeled dataset can achieve very similar performance to Fed-FA. Since the estimations of Hessian in the I-divergence indicator are only utilized as weight or importance for different parameter dimensions, the synthetic dataset cannot estimate accurate Hessians, but roughly estimate relatively accurate Hessian scales. Therefore, the dataset synthesization mechanism does not require labeled corpus, does it cause performance loss, which demonstrates its effectiveness. Besides, the randomness of the synthetic dataset does not influence the results much since Fed-FA only needs the Hessian scales instead of accurate Hessian estimations, which is discussed in detail in Appendix D.
Effectiveness of embedding Hessian reassignment mechanism. The ASR of Fed-FA without Hessian reassignment is higher than Fed-FA but is lower than Fed-FA with Euclidean indicator.

---

Table 5: Results of defenses BERT SST-2. Fed-FA still outperforms others.
Table 6: Results of MNIST backdoor defense task. Fed- FA can still work in CV. ASRs (< 11) are in bold.
Attack
Method
ACC
ASR
Method
Metric
Attackers 3 Attackers 4 Attackers
FedAvg
89.41
96.73
ACC
96.33
96.23
96.01
FedAvg
Median
89.39
79.32
ASR
39.20
94.50
98.44
BadWord
M-Krum
88.95
12.93
ACC
96.64
96.22
96.23
Dim-Ken Krum
89.56
48.36
CREL
ASR
10.78
12.41
87.37
Fed-FA
89.41
9.11
ACC
96.61
96.44
96.31
Median
FedAvg
89.45
95.95
ASR
10.59
10.73
10.76
BadSent
M-Krum
Median
89.45
89.26
92.29
37.38
Krum
ACC
ASR
10.02
95.97
10.35
95.76
95.64
10.34
DimmKrum
89.45
46.03
ACC
96.08
95.82
95.93
Fed-FA
Fed-FA
89.03
27.10
ASR
10.20
10.20
10.31
means that embedding Hessian reassignment can help estimate Hessians more accurately. We also explore Fed-FA with inverse reassignment, which replaces the reassignment principle v Hrg oc
Ei=1" (0 1/n with /fr x (Ei N4 ( 1/n]_1 and causes very poor performance. We also implement Fed-FA with layer-wise reassignment that reassigns Hessians in every layer respectively, and Fed-FA with reassignment within the entire model that reassigns Hessians on parameters on the entire model parameters instead of the embedding parameters E. These two variants both perform worse than embedding Hessian reassignment, which demonstrates we should conduct Hessian reassignment the embedding parameters.
Why does conducting Hessian reassignment on embeddings work best? The premise of Theorem requires that Hessians are small. The Hessian scales on embeddings are usually smaller than other layers: analytic trials show that average Hessian scales are about 10 on embeddings, 10- on other layers; thus correlations of Σ Mak I/n and /HG on embeddings are 0.47, which is much higher than correlations on other layers, 0.02. Thus reassignment should only be conducted on embeddings.
(8)
5.2 Results on other settings
Results on pre-trained language models. To validate the effectiveness of Fed-FA larger models such as Transformers [41] and pre-trained language models, We evaluate Fed-FA on BERT [8] in Table 5 Results show that Fed-FA still outperforms other defenses consistently on BERT, which indicates the potential of Fed-FA to scale to larger models, especially large language models.
Results on federated vision backdoor defense. We also find that federated vision backdoors easier to defend against than language backdoors, which is also observed in [58]. As illustrated in Table need multiple attackers inject backdoors into vision models. Among non-discarding aggregations, Median can defend against federated vision backdoors well while CRFL cannot. Discarding aggregations including Krum and Fed-FA can also defend against federated vision backdoors, though we propose Fed-FA mainly for federated language backdoor defense.
Results on non-lID and multiple attacker cases. We generalize Fed-FA to non-IID and multiple attacker settings in Fig. 3 Here we choose the Dirichlet distribution with the concentration parameter 0.9 as the non-IID distribution. It can be concluded that non-lII and multiple attacker cases harder to defend than IID and single attacker cases, while Fed-FA still outperforms existing defense baselines. The defense performance in non-lII cases is worse since non-IID cases do not satisfy the Fed-FA's IID assumption. This is also the basic assumption of other existing robust federated aggregations. It is common limitation of Fed-FA and other methods and also discuss it in Sec. 6
5.3 Robustness adaptive attacks
Since the proposed f-divergence indicator is calculated according parameter update variances, potential adaptive attacks can be conducted with adversarial parameter corrputions [40] [44] or
perturbations [4] We adopt L2-penalty regularizer parameters to make parameters close to

---

Fed-Avg
100 Dim- Kmm-Krum
Euclidean
Fec-FA
Fed- Aud-Avg
Euclidean
30-
10
10
20
10
15-4
16ct
15-2
150
-07
10
105
20
12
Weight Decay A
Adaplive Decay
(a) Non-IID and multi-atker cases.
(h) Weight decay attack cases.
(c) Adaptive decay attack cases.
Figure 3: Results under non-llI and multiple attacker cases and under adaptive attacks. Fed-FA still outperforms existing baselines under non-lID and multi-atker cases and is robust adaptive attacks.
0 in the t-th round: R = A. Ed (0) d where denotes the decay coefficient and 6 denotes the i-th dimension of 0 1 We also design another adaptive decay regularizer to target to attack Fed-FA: R = X- D H - ) where Hi is Hessians estimated by attackers.
As shown in Fig. 3 when the decay regularizer weak, adaptive attacks can inject backdoors to FedAvg whereas it can neither fool Fed-FA nor Euclidean since statistics differences of parameters are still obvious enough. When the decay is proper. it can slightly fool Euclidean with the L2-penalty regularizer when the decay coefficient is 10 When the decay too strong, the norms of parameter updates are too small, and adaptive attacks cannot inject backdoors all aggregations. To conclude, Fed-FA is robust adaptive attacks. We also validate that Fed-FA is robust to distributed backdoor attacks in Appendix D)
10- but Fed-FA is still robust to adaptive attacks.
6 Broader impact and limitations
Broader impact. In this paper, propose the Federated F-Divergence-Bart Based Aggregation (Fed- FA) algorithm to form strong defense NLP tasks by reducing the potential risks of federated aggregations. We do not find any possible adverse effects society caused by this work.
Limitations. Although Fed-FA achieves state-of-the-are defense performance in NLP tasks. the defense performance in non- cases is as not satisfactory as in IID cases, since the IID assumption of Fed-FA is not satisfied. This common limitation Fed-FA and other existing methods. future direction is to consider the semantics of the parameter updates themselves in addition to the data divergence for federated backdoor defense.
Besides, both our proposed Fed-FA and classic federated defending algorithms are mainly evaluated small-scale MLP, CNN or RNN models, but evaluated on popular large language models due to computation cost limit. However, our proposed Fed-FA is model agnostic and just filters harmful gradients involved in aggregation, thus can be extended to large-scale models such as Transformers and large language models. To validate this, we also evaluated Fed-FA on BERT 88. pre-trained language model based on Transformers, validate the potential of Fed-FA to scale to large models. A future direction is to evaluate and improve federated language backdoor defense algorithms on large language models.
58
7 Conclusion
In this paper, we model data divergence among clients' data theoretically for backdoor client detection in federated language learning. Based on it, we propose novel and effective Federated F-Divergence- Based Aggregation (Fed-FA) algorithm as strong defense for federated language learning. Fed- FA utilizes the f-divergence indicator to detect and discard suspicious clients. Both theoretical evidence and experimental results demonstrate that Fed-FA can better detect suspicious clients than existing robust federated aggregations that mainly adopt parameter distances explicitly. Thus, Fed-FA outperforms existing methods and achieves faate-of-the-art federated language backdoor defense performance. Further analyses validate the effectiveness of proposed mechanisms, as well show that Fed-FA can be generalized to other settings and is robust to potential adaptive attacks.
10

---

Acknowledgement
We appreciate all the thoughtful and insightful suggestions from the anonymous reviews. This work was supported in part by Tencent Research Grant and National Natural Science Foundation of China (No. 62176002). Xu Sun the corresponding author of this paper.
References
[1] Blanchard. P.. Mhamdi, E.M.E., Guerraoui, R.. Stainer, J.: Machine learning with adversaries: Byzantine tolerant gradient descent. In: Guyon, I., von Luxburg. U., Bengio, S., Wallach, H.M., Fergus, R.. Vish- wanathan, S. V.N., Garnett, R. (eds.) Advances in Neural Information Processing Systems 30: Annual Conference on Neural Information Processing Systems 2017, December 4-9, 2017, Long Beach, CA, USA.
pp. 119- 129 (2017)
[2] Blitzer, J. Dredze, M., Pereira, F.: Biographies, Bollywood, boom-boxes and blenders: Domain adaptation for sentiment classification. In: Proceedings of the 45th Annual Meeting of the Association of Computa- tional Linguistics. pp. 440 447. Association for Computational Linguisties, Prague, Czech Republic (Jun 2007), http:htttss//aaaaaaataaaa
[3] Chen, X., Chen, T., Sun, H., Wu, Z.S., Hong. M.: Distributed training with heterogeneous data: Bridging median- and mean-base based algorithms. In: Larochelle, H., Ranzato, M.. Hadsell, R., Balcan, M., Lin, H. (eds.) Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems 2020, NeurIPS 2020, December 6-12, 2020, virtual (2020)
[4] Chen, X., Salem, A., Backes. M., Ma, S., Zhang, Y.: Badnl: Backdoor attacks against nlp models. arXiv preprint arXiv:2006. 01043 (2020)
[5] Chen, X., Liu, C., Li, B., Lu, K., Song, D.: Targeted backdoor attacks deep learning systems using data poisoning. CoRR abs/1712.05526 (2017), tttt:://arxst org/ abb/1712/ 05526
[6] Cui, G., Yuan, L., Hc, B., Chen, Y., Liu, Z., Sun, M.: A unified evaluation of textual backdoor
learning: Frameworks and benchmarks. In: NeurlPS (2022), http: /ttpper/ppppr.......// http//ppppr.ii.s....p.pp capps..eeppprrr pales//aper:/222/ hash
http:///..002.aeeee/222222266660 /ttp:/22222000077222227444466662 42223333333222233333333 SesttrractDaataset.. _ndd__
Benchmarks .html html
[7] Dai, J. Chen, C., Li, Y:: A backdoor attack against lstm-based text classificattin systems. IEEE Access 7. 138872- 138878 (2019).
http:///...SS.oo......o.....r.r. https::/boi..rrr/ 1109/
ACCESS 2019. 2941376
[8] Devlin, J., Chang, M., Lee, K., Toutanova, K.: BERT: pre-training of deep bidirectional transformers for language understanding. In: Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. NAACL-HLT 2019, Minneapolis, MN, USA, June 2-7, 2019, Volume 1 (Long and Short Papers), pp. 4171-4188 (2019), sttps://wve aclweb. org/ antaal www.oooogy/iioo /t11//1//////// 1423/
[9] Erichson, N.B., Taylor, D., Wu, Q. Mahoney, M.W.: Noise-response analysis for rapid detection of backdoors in deep neural networks. CoRR abs/2008.00123 (2020), https://arxiv: wwwaaaiwiw..aaaaaaeeeea /208
00123
[10] Fu, S., Xie. C., Li, B., Chen, Q. Attack-ressitaan federated learning with residual-basee reweighting. CoRR abs/1912.11466 (2019), httpp://arziv..rr//aaaaa batpgg//aa//9999 11464
[11] Fuglede, B.. Topsoe, F.: Jensen-shannn divergence and hilbert space embedding. In: International
Symposium onInformation Theory, 2004. ISIT 2004. Proceedings. p. 31. TEFE (2004)
[12] Fung, C., Yoon, C.J.M.. Beschastnikh, I.:
The limitations of federated learning in sybil settings. In: Egele, M.. Bilge, (eds.) 23rd International Symposium on Research in Attacks, Intrusions and Defenses. RAID 2020, San Sebastian, Spain, October 14-15, 2020. pp. 301-316. USENIX Association (2020),
https:
https://wws/ //www/ usenix. org/ wtwp:/eeeeeeeeeeeeeeeeaaaaaa.a.e http://eeeeeeeeeeeeeeeeeeeeeeeee httpa///oo.o...aaeo/ooooooooaaaa
[13] Gao, Y., Xu, C., Wang, D., Chen, S., Ranasinghe, D.C., Nepal, S.: STRIP: a defence against trojan attacks on deep neural networks. In: Balenson, D. (ed.) Proceedings of the 35th Annual Computer Security
Applications Conference, ACSAC 2019, San Juan, PR. USA, December 09-13, 2019. pp. 113 125. ACM (2019). https://doi.org: .rrg/10.11 145/3359789 333599999 wtt.://...tt9st////ooo httpps:/dol. 001/110.114//335777388 33597790
[14] Garg, S., Kumar, A., Goel, V., Liang, Y.: Can adversarial weight perturbations inject neural backdoors. In: d' Aqui,, M., Dietze, S., Hauff, C., Curry, E.. Mudré-Mauroux, P. (eds.) CIKM '20: The 29th ACM International Conference on Information and Knowledge Management, Virtual Event, Ireland, October 19-23. 2020. pp. 2029-2032. ACM (2020). https//dco.ogy001114447444433323 https //doi. org/ 10. 1141/3144/313333 3412130

---

[15] Gu. T., Liu, K., Dolan- B., Garg. S.: Badnets: Evaluating backdooring atlacks on deep neural networks. IEEE Access 7, 47230-47244 (2019). http://dbb..rr/10.110////ACc 1099/ACCEESS2229929999666 https //doi.op //t.i.ooro........ wttp/ooo//////////.///oooo
2019. 2909068
[16] Kim, Y.: Convolutional neural networks for sentence classification. In: Moschitti, A., Pang. B., Daelemans, W. (eds.) Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing.
EMNLP 2014, Octoher 25-29, 2014, Doha, Qatar, A meeting of SIGDAT, Special Interest Group of the
ACI. pp. 1746-1751. ACI. (2014)
[17] Kingma. D.P. Ba, J.: Adam: A method for stochastic optimization. In: Bengio, Y., LeCun, Y. (eds.) 3rd International Conference Learning Representations, ICLR 2015, San Diego, CA, USA, May 7-9, 2015, Conference Track Proceedings (2015), http:://tttt:://ii// http:///awwxi..or/aab/////aa//// www.ooo/////////ee 6980
[18] Kullback, S., Leibler, R.A.: On information and sufficiency. The annals of mathematical statistics 22(1),
79-86 (1951)
[19] Kurita, K., Michel, P., Neubig, G.: Weight poisoning attacks on pre-trained models. CoRR abs/2004.06660 (2020), https://aarxa
org/ wwrg/abs/200/ 2004. 06660
[20] Li, Q.. Tai, C., E, W.: Stochastic modified equations and dynamies of stochastic gradient algorithms I: mathematical foundations. J. Mach. Learn. Res. 20, 40:1 40:47 (2019), http://jmlr.orge ortppperrrrppppps///////
v20/17-526.0 html
[21] Li. Y.. Lyu, X., Koren, N., Lyu, L.. Li. B., Ma, X.: Neural attention distillation: Erasing backdoor triggers from deep neural networks. In: 9th International Conference Learning Representations, ICLR 2021. Virtual Event, Austria, May 3-7, 2021. OpenReview.net (2021), Sestps://openreeee www.eeee.eeeoononiiiiec
910X40X-oXE
[22] Liu, K. Dolan-Gavitt, B., Garg, S.: Fine-pruning: Defending against backdooring attacks on deep neural networks. In: Bailey, M., Holz, T., Stamatogiannakis, M., Ioannidis, S. (eds.) Research in Attacks, Intrusions, and Defenses- 21 International Symposium, RAID 2018, Heraklion, Crete, Greece, Septemher 10-12, 2018, Proceedings. Lecture Notes in Computer Science, vol. 11050, pp. 273-294, Springer (2018). https://doi.org/ http:////...00099000000000000000
13, httpp:://oii org/10. 1007 3 030-00070-0056
13
[23] Liu, Y., Ma, S., Aafer, Y., Lee, W., Zhai, J., Wang. W., Zhang, X.: Trojaning attack on neural networks. In: 25th Annual Network and Distributed System Sccurity Symposium, NDSS 2018, San Diego, California, USA, February 18-21, 2018. The Internet Society (2018)
[24] Maas, A.L., Daly, R.E., Pham, P.T., Huang. D., Ng. A.Y., Potts. C.: Learning word vectors for sentiment analysis. In: Proceedings of the 49th Annual Meeting of the Association for Computational Linguisties: Human Language Technologies. pp. 152-150. Association for Computational Linguistics. Portland, Oregon,
USA (June 2011), hatt:://wwwt aclweb. org/ htt///attooooo/111/11aa
[25] Mandt, S., Hoffman. M.D., Blei, D.M.: Stochastic gradient descent as approximate bayesian inference. J. Mach. Learn. Res. 18, 1 134:1-134:33 (2017), http: /m/r http//ppar////11-222202 html
[26] McMahan, B., Moore, E., Ramage, D.,, Hampson, S., y Arcas, B.A.:
catttuuniiaaiiotfffiiieeeee learning of deep networks from decentralized data. In: Singh, A., Zhu, X.J. (eds.) Proceedings of the 20th International Conference on Artificial Intelligence and Statistics, AISTATS 2017, 20-22 April 2017, Fort Lauderdale, FL., USA. Proceedings of Machine Learning Research, vol. 54, pp- 1273-1282. PMLR (2017), http://proceedingg. mlr wwwppp/ssssseseeeepeepp 2155///mmaaa177777 hhtmlll
[27] Mhamdi, E.M.E., Guerraoui, R., Rouault, S.:
The hidden vulnerability of distributed learning in byzantium. In: Dy. J.G., Krause, A. (eds.) Praceedings of the 35th International Conference on Machine Learning, ICML 2018, Stockholmsmässan, Stockholm, Sweden, July 10-15, 2018. Proceedings of Machine Learning Research, vol. 80, pp. 3518 3527. PMLR (2018), http::/proccedingg/ mmriiinns...r......... mlr press/ h/t////000//mmm//mmm/ shaaddi88. html
[28] Muñoz--Gozálee, L.,, Biggio, B., Demontis, A., Paudice, A., Wongrassamee, V., Lupu, E.C., Roli, F: Towards poisoning of deep learning algorithms with
optimization. In: Thuraisingham, B.M., Biggio, B., Freeman, D.M., Miller, B.. Sinha. A. (eds.) Proceedings of the 10th ACM Workshop on Artificial Intelligence and Security, AlSec CCS 2017, Dallas, TX, USA. November 3, 2017. pp. 27-38. ACM (2017). https://doi.org 1 145/3 128572.314045 1t0o4s1,,tttt.::... https: //to: org/ 10. 1145/3128577 011128572.344455)
[29] Nguyen, T.A.,, Tran, A.: Input- Input-aware aware dynamic backdoor attack. In: Larochelle, H.. Ranzato, M., Hadsell, R.. Balcan, M.F. Lin, H. (eds.) Advances Neural Information Processing Systems. vol. 33, pp. 3450-3460. Curran Associates, Inc. (2020)
[30] Pascanu, R., Bengio, Y.: Revisiting natural gradient for deep networks. In: Bengio. Y., LeCun, Y. (eds.) 2nd International Conference on Learning Representations, ICLR 2014, Banff, AB, Canada, April 14-16, 2014, Conference Track Proceedings (2014), http:: //tp///axvvvvvv/ org/ abs wabb//311......
[31] Pillutla, V.K., Kakade, S.M., Harchaoui, Z.: Robust aggregation for federated learning. CoRR abs/1912.13445 (2019), htth::tt:///rrr/ org/ hts///b//11999 13445
12

---

[32] Qi. F., Chen, Y., Li. M. Liu, Z., Sun, M.: ONION: A simple and effective defense against textual backdoor attacks. CoRR abs/2011. (2020), https://aarraee
wtgg/bs/////aa1 10369
[33] Qi, F., Li, M., Chen, Y., Zhang, Z., Liu, Z., Wang, Y. Sun, M.: Hidden killer: Invisible textual backdoor attacks with syntactic trigger. In: Zong, C., Xia, F., Li, W. Navigli, R. (eds.) Proceedings of the 59th Annual
Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing, ACL/UCCNLP 2021, (Volume 1: Long Papers), Virtual Event, August 1-6,
2021. pp. 443-453. Association for Computational Linguistics (2021). http:://oi.oor orgo/0. (0. 16653v112221 1ong.37.1tttss///
org//og/// 10. 18653/v1 www.//.....22// aaa.aacllllo acl-long.
[34] Rényi, A.: On measures of entropy and information. In: Proceedings of the Fourth Berkeley Symposium on Mathematical Statistics and Probability, Volume I: Contributions to the Theory of Statistics. pp. 547-561.
University of California Press (1961)
[35] Rumelhart, D., Hinton, G., Williams, R.: Learning representations by hack propagating errors. Nature 323, 533-536 (10 1986). https://doi.ogg// 10. 0338/33333300
[36] Saha, A., Subramanya, A., Pirsiavash, H.: Hidden trigger hackdoor attacks. In: The Thirty-Fourth AAAT Conference on Artificial Intelligence, AA 2020, The Thirty-Secone Innovative Applications of Artificial Intelligence Conference, IAAI 2020, The Tenth AAAI Symposium on Educational Advances in Artificial Intelligence, EAAI 2020, New York, NY, USA, February 7-12, 2020. pp. -195-119955 AAAI Press (2020)
[37] Salem, A., Backes, M., Zhang, Y.: Don't 't trigger me! triggerless backdoor attack against deep neural networks. arXiv preprint arXiv:2010.0328% (2020)
[38] Sato, L., Nakagawa, H.: Approximation analysis of stochastic gradient langevin dynamics by using fokker- planck equation and ito process. In: Xing. E.P., Jehara, T. (eds.) Proceedings of the 31st International Conference on Machine Learning. Proceedings of Machine Learning Research, vol. 32, pp. 190-990. PMLR, Bejing, China (22-24 Jun 2014),
mlr. arres/v33/sattal14 html
[39] Sacher, R., Perelygin, A., Wu, J., Chuang, Manning, C.D., Ng. A.Y., Potts, C.: Recursive deep models for semantic compasitionality over a sentiment treehank. In: Proceedings the 2013 Conference on Empirical Methods in Natural Language Processing, EMNLP 2013, 18-21 October 2013, Grand Hyatt Seattle, Seattle, Washington, USA. A meeting of SIGDAT, a Special Interest Group of the ACL. pp. wtt.gg...rr/0011100000
1631-1642. ACL (2013), //tctts::/acaannnt
[40] Sun, X., Zhang. Z., Ren, X., Luo. R., Li, L.: Exploring the vulnerability of deep neural networks: A study of parameter corruption. In: Thirty-Fift Fifth AAAI Conference on Artificial Intelligence, AAAI 2021. Thirty-Third Conference on Innovative Applications of Artificial Intelligence, IAAI 2021, The Eleventh Symposium on Educational Advances in Artificial Intelligence, EAAI 2021, Virtual Event, February 2-9, 2021. pp. 11648 11656. AAAI Press (2021), https: aaai. org/ /n////idxx,aa/// wwwpwxwwppppppppp/eepppppwpppppp whtp:/p/////pataaaaaaap ht///AAAAaaaail...e. view/ 17385
[41] Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, I.., Gomez, A.N., Kaiser, I.., Polosukhin, T.: Attention is all you need. In: Advances Neural Information Processing Systems 30: Annual Conference on Neural Information Processing Systems 2017, 4-9 December 2017, Long Beach, CA, USA. pp. 5998- 6008 (2017)
[42] Wan, C.P., Chen, Q.: Robust federated learning with attack- adaptive aggregation. CoRR abs/2102.05257 (2021), hattts://aaxxt
org/ abs/ /tt../222..... 05257
[43] Xie, C., Chen, M., Chen, P., Li, B.: CRFL: certifiably robust federated learning against backdoor attacks. Meila, M., Zhang, (eds.) Proceedings of the 38th International Conference on Machine Learning.
ICML 2021, 18-24 July 2021, Virtual Event. Proceedings of Machine Learning Research, vol. 139, pp.
11372-11382. PMLR 2021), http: http://proeeeddi..e.eeee mlr .wwiitppeesseee wwtppp/wsssaae/peeepppppppp v/////31////// wwt9/x1ieille exteea.ieeiiieeeeeeeeeeeee...ela html
[44] Xie, C., Huang, K., Chen, P., Li, B.: DBA: distributed backdoor attacks against federated lcarning. In: 8th International Conference on Learning Representations, ICLR 2020, Addis Ababa, Ethiopia. April 26-30, 2020. OpenReview.net (2020), https: https://openrevvez net frrm? r-ggySoo
[45] Xie, Z., Sato, I., Sugiyama, M.: A diffusion theory for deep learning dynamics: Stochastic gradient descent exponentially favors flat minima. arXiv preprint aroiv:200200o.... 004995 (2020)
[46] Yang, W., Li, I.., Zhang, Z., Ren, X., Sun, X., He, B.: Be careful about poisoned word embeddings: Exploring the vuinerability of the embedding layers in NL. models. In: Toutanova, K., Rumshisky, A., Zettlemoyer, L.. Hakkani-Tiin Tür, D., Beltagy, I.. Bethard, S.. Cotterell, R., Chakraborty, T., Zhou. Y. (eds.) Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguisties: Human Language Technologies. NAACL-IILT 2021, Online, June 6-11. 2021. pp. 2048- 2058. Association for Computational Linguistics (2021). www.aaaaaaaaammm 165
wttp.//r//....../////oo. waacl -naacl-main.116 165,
https://doi. http://00..rr///1111166///////// http:///....a.aalleeee
[47] Yang, W., Lin. Y., Li, P.. Zhou, J., Sun. X.: RAP: robustness-awar perturbations for defending against backdoor attacks on NLP models. In: Moens, M., Huang, X., Specia, L., Yih, S.W. (eds.) Proceedings of
13

---

the 2021 Conference on Empirical Methods in Natural Language Processing, EMNLP 2021, Virtual Event / Punta Cana, Dominican Republic, 7-11 November, 2021. pp. 8365-8381 Association for Computational Linguisties (2021). hups://doi. oro. 1/200066611/1/222eelmlllm.m v1/2021 wwweeennnaaeeeeee main. 659
httpp:///oii http:/oooorrgg///// wtg//..11//
[48] Yang. W. Lin, Y., Li, P.. Zhou, J.. Sun, X.: Rethinking stealthiness of backdoor attack against NLP models. In: Zong. C., Xia, F., Li, W., Navigli, R. (eds.) Proceedings of the 59th Annual Meeting of the Association for Computational Linguisticss and the 1tth Internationaa Joint Conference on Natural Language Processing, ACI/UCNI. .P 2021, (Volume I: Long Papers), Virtual Event, August 1-6, 2021. pp. 5543- 5557. Association for Computational Linguisties (2021). https:/doii.rr// 10. 10.118653/11/// 1/202111121111 1tt01///101/1111/g../. Lacl-long-433 https: sstps://doi..m orgg 10. 18653/v1 vwtp//vvv2222.../// acl-long. 431
[49] Yao, Y., Li, H., Zheng, H., Zhao, B.Y.: Latent backdoor attacks on deep neural networks. In: Cavallaro, L.., Kinder, J., Wang. X., Katz, J. (eds.) Proceedings of the 2019 ACM SIGSAC Conference on Computer and Communications Security, CCS 2019, London, UK. November 11-15, 2019, pp. 2041 2055. ACM (2019). http:/d...oog//00111153311933333 145443195555.333 33554209, /httts://too.o... org. /tt.:////000.11/33999533355 33344299
https://oi.orr/100.
[50] Yin, D., Chen, Y.. Ramchandran, K., Bartlett, P.L.: Byzantine-robust distributed learning: Towards optimal statistical rates. In: Dy, J.G., Krause, A. (eds.) Proceedings of the 35th International Conference on Machine Learning, ICML 2018, Stockholmsmässan, Stockholm, Sweden, July 10-15, 2018. Proceedings of Machine Learning Research, vol. 80, pp. 5636-5645. PMER (2018), http::/proceedinng mlr press/v80/ http://vv/yin8..... html
[51] Yoo, K., Kwak, N.: Backdoor attacks in federated learning by rare embeddings and gradient ensem- bling. CoRR abs/2204.14017 (2022). https://doi.org http:://doiorg/1004455555 00448550/a 14000, htttttt:t///ii..ooo
48550/ar arrv.. 14017
[52] Zhang. X.. Zhao. J.J., LeCun, Y.: Character-level convolutional networks for text classification. In: Cortes, C., Lawrence, N.D., Lee, D.D., Sugiyama, M., Gamettt, R. (eds.) Advances in Neural Information
Processing Systems 28; Annual Conference on Neural Information Processing Systems 2015, December
7-12, 2015, Montreal, Quebec, Canada. pp. 649-657 (2015)
[53] Zhang, X., Mian, A., Gupta, R., Rahnavard, N., Shah, M.: Cassandra: Detecting trojaned networks from
adversarial perturbations. CoRR abs/2007.14433 (ttt)).tttss////// httpt/tttt////ttteet //rxiv org hss//b0b/0000./ 14433
[54] Zhang. Z., Luo, R., Ren, X., Su, Q. Li. L., Sun, X.: Adversarial parameter defense by multi-stey risk
minimization. Neural Networks 144, 154-163 (2021)
[55] Zhang, Z., Luo, R., Su, Q., Sun, X.: GA-SAM: gradient-strength based adaptive sharpness-aware min- imization for improved generalization. In: Goldberg, Y., Kozareva, Z., Zhang. Y. (eds.) Proceedings of the 2022 Conference
Empirical Methods in Natural Language Processing, EMNLP 2022, Abu Dhabi, United Arab Emirates, December 7-11, 2022. pp. 3888-3903. Association for Computational Linguistics (2022), httph:ttt////tt https://aacaatthol.oo wwwp://.rrg/ee. 2022. http:/2222....m1111.m.mmmmm. mommp-mmiin... 257
[56] Zhang. Z., Lyu, L., Wang, W., Sun, L.. Sun, X.: How to inject backdoors with better consistency: Logit anchoring on clean data. In: The Tenth International Conference on Learning Representations, ICLR 2022, Virtual Event, April 25-29, 2022. OpenReview.net (2022), httpps:///opeeeeeiee..tttt www.:viiiiiiieefffuurneaa id=Bn09TnDagNj
[57] Zhang, Z., Ren, X., Su, Q.. Sun, X., He, B.: Neural network surgery: Injecting data patterns into pre-traincd models with minimal instance-wise side effects. In: Toutanova, K.. Rumshisky, A., Zettlemoyer, L., HakkaniiTüür D., Beltagy, I., Bethard, S., Cotterell, R., Chakraborty, T., Zhou, Y. (eds.) Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL-HLT 2021, Online, June 6-11, 2021. pp. 5453-5466. Association for Computational Linguistics (2021), /tttts:ttt///tttt htt.tppp//ww..aal..ebbbbbbb wwwpeeebbrrrreeeee wwtp://.rtgatooogggg//eeee/ 2021 maacl-main. 430/
[58] Zhang, Z., Su. Q. Sun, X.: Dim-krum: Backdoor-resistant federated learning for NLP with dimension-wee wise krum-based aggregation. In: Goldberg. Y., Kozareva, Z.. Zhang, Y. (eds.) Findings of the Association for Computational Linguistics: EMNLP 2022, Abu Dhabi, United Arab Emirates, December 7-11, 2022. pp. 339-354, Association for Computational Linguistics (2022), httppe://acaanntoooogy..o
findings- emnlp. 25
[59] Zhao, P.. Chen, P., Das, P., Ramamurthy, K.N., Lin, X.: Bridging mode connectivity in loss landscapes and adversarial robustness. In: 8th International Conference on Learning Representations, ICLR 2020, Addis Ababa, Ethiopia, April 26-30, 2020. OpenReview.net (2020), dttpa://openrevieet www//ffuuumae. iitr
SJgwzCEKuH
14

---

A Theoretical details
A.1 Details about f-divergence
The f-divergence
34
measure the data divergence of p(z) and o(z):
p(x) ((2)
()(1(2)llp(z))
de.
p(z)
where the integral symbol f~ denotes multiple integrals in every dimension of z, and integral symbols of discrete dimensions need to be changed summation symbols for these dimensions.
The function fx) is smooth, convex and satisfies f(1) O. Suppose 1 h. We require the function fx) Sccond-order differentiable and its Sccond-order Taylor expansion near 1 is:
(1) o(h²). 2
f(x)= f'
(8)
since (x) convex, have f1) >
Common classic divergences special cases f-divergence with corresponding functions f(x). Here are some examples of special cases:
Kullback-Leibbe divergence. special case of f-divergence the Kullback-Leibhe divergence [8] when we choose (z) = x log x: Dy(allp) (gl(p) Sx (()) log (9(z)/p(z)) dz, and the Second-order Taylor expansion is:
f(x) - f(1+ = h+ 12 2 -(hh),
(9)
where (1
Reverse Kullback-eibler divergence. A special case of f-divergence is the reverse Kullback-Leibler diver- gence (18) when we choose f(x) log (y (p) Drevne KL (g) p) Ja p(z) log (p(z)/q(z)) dz, and
the Second-order Taylor expansion is:
a f(=)= = f(1 + h) - + o(h2)
(10)
where ("(1)=1>> =1>0 0.
Jensen-Shannon divergence. special case of f-divergence is the Jensen-Shannon divergence [1] when we choose (xx = x ælog x - (æ + lol(((( + 1)/2): (r((lpp) Dis(gl (p) = Dxr (pllm)/2 + Dxa (2.
where the medium distribution
4)/2, 2, and the Second-order Taylor expansion is:
h²
f(x) f(1
4 o(h2),
(11)
where =11 =
A.2 Detailed asumptions of Theorem 1
Thcorem requires two classic assumptions, the Second-order Taylor expansion assumption and the Fisher information matrix assumption 30
Assumption A.1 (Second-one order Taylor Expansion). Assume the loss L(O; z) can be Second-orde order Taylor
expanded near the parameter 6* and the Hessian matrix H (0; z) = V2.(0;2 z) is diagonal:
L(O' 5; C(0": 8T VoC( :z) + TT ( (0)(0==))6 2 (
(12)
and (O0 P) diag H1 (0: P).... (0:))) Assumption A.2 (Fisher Information). Assume the Fisher information assumption 30 holds:
H P) Ep Eo/Voo(0::2)Vo0000033
(13)
A.3 Proofs of Theorem
We try to find the infimum or greatest lower bound of f-divergence:
Inf
(14)
p(K).q(x)
subject to 0" = arg min p(z))
0" + 8 = arg min C(0; q())).
(15)
15

---

Theorem 1 (F-Divergence Lower Bound). The lower bound of f-dieergeence is:
D,(1(z)]l((2) 2 (1 + o(1)) (1) Ir.D,, Ir.De
H2
(16)
where Hg (0':00p(z)) (,, (0": p(z)) >> O is the i-th Hession of loss on p(z) and (1) > 0.
Proof. First, we introduce a lemma analyzing the relationship the distribution shift and the parameter shift (55):
Lemma A.1. Define r(z) = qzz)/p(z)- - 1. When the distribution shift is small enough, namely r(z) is small, we can estimate the parameter shift 5 as,
5 = - 1 p B.
51)-
(17)
r(z) is defined as r(z) q(z)/p(z) 1. and the parameter shift is de- fined as arg ming
In Lemma A.I.
arg ming L( p(z)), which is consistent with our definition since arg ming C (()) arg min, [0: (0" + +d) (0)) =
With the change-of-measun technique, we have E, [r(z]] = O. Define [=) = =v. Then according to Lemma A.I
(00: ( o((81)).
(18)
Conduct the Second-order Taylor expansion on function f near in D2 (g())1/((z) we have
g(2)
("(1), ,)) (x) + o(r²
f(
f((1)((z)
(19)
p(z)
2
Therefore,
Dy(alp) p(x)s( ((z)
du ttt==E/lfe (
((3)
[1)r ('(1), 2- o(r))]
p(z))
p(z)
2
(20)
(1(1) Ep (x)) + t
(1 + o(1) ('(1)
Ep (x)))
(21)
Define T/r(z)] =R, [x))] (Py min = (1+ (10) min/ /2. The infimum is:
Inf
Z[r(2) = E, [2)]
(22)
((4)
subject (r()))(()) = 0, [.
(23)
- Define the Lagrange multiplier as [[r(z)] [Z[r(z]] - 18 (B v), when
Z[r(z)] is optimal, (z)((z)))/r()) O, namely:
r(z) 3 VoL(O'; z) =0
(24)
Therefore, r(z) a V We have:
0= Er(z)) - a --
= a 3 (' = a,
(25)
v v= v=E, = E. VeL Voe =00:) (26)
To conclude:
a= :2)
(27)
Therefore, when Z[r(z)] is optimal:
r(x) (H (o':ple))v)" =v
- (1"" (0':
(28)
thus the optimal Z[r(z)] is:
(Z(r(2))) -B,[r(2))
(29)
= =Ep(v' H (0" Vo H (0':;z)))) VoL(0" :2)
(30)
=v H VeC H :pz))
(31)
=v H
(32)
16

---

since = (r())) 8 o((])) = (1 (O':))))
(I[r(z))) (1 + ( H ((z)6 = (1 + o Σ Hg02
(33)
where H = He (0"; p(x)).
Therefore, the solution
(Dylall)))... (1 o (1 (Zlr(z)) (1 (1+ (1)
Hgg
(34)
2
km1
then we have:
(y(a(2)lo(()) 2 (Dy(alp)) min (1 + o(1)) f"(1)
IRDii
Σ
H
(35)
2
k=1
A.4 Proofs of Theorem 2
Theorem 2 (Hessian Estimations by Diffusion Theory, Detailed Version). When V / (
p(z))
small, there exists constont C for any >> O that the following inequality holds with a probability higher than 1 é for large
Σ
(36)
C CVH H
2nc
where ( is the k-th dimension of u.
Proof. First, we introduce the concept diffusion process, we introduce lemma that rewritten from classic conclusions in the diffusion theory (25 20)
The training dynamics on one client can be modeled as diffusion process 38 25 20 with Stochastic Gradient Noise (SGN):
d0 - Vo + w2DdW,,
(37)
where dt the unit time the step size, D is the diffusion coefficient, and d'W N(0, Idt). We assume the gradient noise introduced by stochastic learning is small (the temperature of the diffusion process is low). Here we also assume that the data distributions of all clients approximately obey the merged data distribution p(z).
The diffusion coefficient matrix D is a diagonal matrix, and its value in the Stochastic Gradient Descent (SGD) dynamics is:
D2 "
2B H
(38)
where y = di is the time or the step size,
batch size, and H = (o((0:p(z))
Its value the dynamics involving adaptive learning rate mechanisms, take the Adam 17 optimizer for example, can also be seen as De se 7 H. when / HE are small. Since the parameter update is:
2B
A - O m,
(39)
where be seen SGD update with the momentum mechanism and Eim] VoL(O: in stationary distribution. Adam, n|vv e) and Elv] Rz)-)()) (oL(0;z) @ VoL(O; H(0"; p(z)) in stationary distribution. Therefore, when vV are small, the weight update can be approximated with:
AO
7/E m
(40)
which can be seen as an SGD update with the lcarning rate ne and the gradient m on a small batch.
To conclude, there exists a constant C1 > 0 that the diffusion coefficients D& of (1) all dimensions in the Stochastic Gradient Descent (SGD) dynamics; or (2) dimensions with small /Hg in dynamics involving adaptive learning rate mechanisms, such Adam [17]
Dr RJ C.
(41)
We introduce Lemma A.2 that is rewritten from classic conclusions in the diffusion theory
25 20
17

---

Lemma A.2. When the training dynumics the i-th client can be modeled as diffission process with Stochastic Gradient Noise (SGN), there exists constant C2 > 0 that the updates obey the Gaussian distributions on all dimensions:
( N(0, C2)
(42)
According Lemma A.2
+00 [a]] exp
20 dr 2CrD2.
El [[]u4]]
(43)
-00 V2TC2Dx
+00 |x||² exp
x²
18 [ (1) =
20,
dx = CrDb.
(44)
22
B [lx [ =El [1] - EE [ = =(1- (1- C22
2
(45)
π
The Chebyshev Inequality demonstrates that:
P(IX - B[X]I < 1 D
(46)
we choose x= 1 n then:
Σ 1
(9)
20' DR
(1 3 D2
B[X]
[[X]
(47)
π
n
We choose
n[X] then:
Σ
(8) uux
2C
[
(48)
π
holds with a probability higher than
c.
For dimensions with small Hessians, there exists a constant C. > 0 that D₂ = H, Therefore, there exists a constant C
/C001 for any >> that the following inequality holds with probability higher than
u
(49)
CVH H
2ne
A.5 Proofs of Theorem
We verify the Byzantine resilience of Fed-FA in Theorem
Preliminary. Suppose g = Ej'u].g' = Eju"] are expected updates for clean updates u and malicious updates u". Suppose Ok 9k. ug denote the k-th dimension of g" u.
We also require that the gradient noises are bounded linearly by gi namely, there exists 7 > 0 such that D[uu] & n]E[uu]]² = ngt for any dimension & for clean updates u. We define as the gradient noise scale. (1] assume that Dluu] o² and thus D [ua] is fixed value o². However, in this work, we allow ID(u.] to be arbitrarily large when od grows very large and D[uu]] is only required to he bounded with linear bound of gi instead, namely ?
According to Fisher information matrix assumption
H = DDuug-
Theorem (Byzantine Resilience of Fed-FA. Detailed Version.). Assume the loss function on the merged dataset p(z) is locally ft-strongly convex and locally L-smooth near the optimal parameter. For m malicious clients, 1 < m [_="], when the estimations of indicators are accurate enough and the gradient noise scale n small enough, there exists
such that Fed-FA aggregation algorithm (a, m) Byzantine resilience:
|E[A([u (1) [=r)]] BlI2 S al]g|la g g'E[[A(uu) (>1))) 2 (1 - (i)sinn))lsl1
(50)
where g- B[ gg B[[u)] are the expectations for clean updates and malicious updates u'.
18

---

Proef. We assume that the estimations of Hessians are accurate, namely the estimations of (R) are accurate. Suppose I F-Di-
[(3)
S -I
Div
Then we have 9 {1, 2, ... , 1 11. |-| [" 2 n m > m. Suppose M the set malicious client
indexes, and M m. Define Mr (i:i |S], E M). Ma - M T
[ == then,
[S|
A((a ) 5
(
Σ iES
n
Σ n n
(51)
1EM
If M1 0, namely S \ M1 S, we have:
B[A((u (iL)))) S
Σ
1)
Σ
E Elu(' = g.
(52)
IEST.
VE
namely there exists O such that:
|[BLA([ (0
)==))] gll < sin gl1,,
8 3'B[A(fu (2
lgll 2 (1 -1- sin o
(53)
Otherwise, have |
> 0,
|8|
M m) m
pa 1= B[A(Hu (1)
1 |S|
Blu
|]u (1S) - M Dg + | g"
(54)
[S]
For any E M1, there exists clean client $ S M, because we have n mi clean clients but clean client = m+1
number in S [S| |M |S|
1 S n m. Therefore I 2 I(a Te namely:
Σ H 94)*
HE gz)²
(55)
where estimations of Hessians and 8 are accurate, namely we can estimate the optimal parameter updates of clean clients 0" accurately, thus we replace the k-th dimension 0" or 8r with
Since the loss function is locally p-strongly convex and locally L-mmooth, we have μ. S H D(un] S L, since the Hessian matrix is diagonal according to the assumption and Hg is the eigenvalue of the Hessian matrix that is in [a, L). we have,
L
94)²
gu)²²
Duun on)² 2 Σ u1 - -
(56)
Note that client is malicious, the expectation of the right-hand side is,
B (uk
E
((u ()) gi) (9k -9
(57)
km1
km
E [(( (t) - -gh)² + 2(2
(9h 9h) + (gk gb))
(58)
E [(( -xh)²] + Σ (st 9b)²
(59)
A=1
h=1
Σ (ot gk)²² =vlg" gl..
(60)
1=1
and note that client j clean, the expectation of the left-hand side is,
d Σ o4 A=1
[
T..
)x =
[uue]] 5 nT
(61)
1=1
combining them, have,
nL 2 Allg" -glé.
(62)
19

---

Therefore,
| (u (8
[=m)l
(IS) - [M
1gg
+
MMig"
8
JMvi
1
(63)
[
|
lg
gl12
m
nT.
|gll2
(64)
m+
2
m. 12 L
When is small enough and 7)
have,
<1,
(65)
therefore there exists:
a arcsin
m
nT.
(0, 7)
(66)
[
such that Fed-FA aggregation algorithm is (a,, -ny-Bizannine resilience:
(B[A((u()) gllz I sin ainle]
(67)
and we can also derive:
g 3'B[A(tu H=)) g" (B[A(Hu الدرر g) +
(68)
|lg|l g" (B[A( -ーري lgl| lg
(69)
[[A((u )ل=)) ggl
sin ag
(70)
① also proves the higher-order moments aggregations are bounded by a linear combination of terms of clean update moments. The proof utilizes the same technique as above.
The bound of sin a provided the proof is: sin a
lower sin a indicates better Byzantine
resilience. It can be concluded from the bound and the assumptions that: (1) higher (1 S 1= leads to worse Byzantine resilience, and when m > _==1] J. the assumption of the theorem does not hold and the Byzantine resilience of Fed-FA is not guaranteed; (2) poorly conditioned loss function or Hessian matrix (namely the condition number L/P the Hessian matrix is high) leads to poor Byzantine resilience; (3) higher gradient noise scale v/ leads to poor Byzantine resilience; (4) the Byzantine resilience of Fed-FA rely on that the estimations indicators are accurate; we propose dataset synthesization and embedding Hcssian reassignment mechanisms for more accurate Hessian estimations; therefore, future improvement direction of Fed-FA may be avoiding the interference of attackers
indicator estimation.
B Detailed experiment setups
In this section, we introduce the detailed experimental sctup. In the training process of the local clients, all aggregation methods adopt the same hyper-parameters for fair comparisons. Experiments conducted on NVIDIA TITAN RTX GPUs. One experiment costs about 30 minutes for one run (including 10 rounds).
B.1 Tasks and datasets
We adopt four text classification tasks, i.e.. SST-2 39 IMDB 24
Amazon (2) and AgNews [2 In experiments. adopt metrics to evaluate clean and backdoor performance, the clean accuracy (ACC) and the backdoor attack success rate (ASR). ASRs are only evaluated on samples whose labels not the backdoor target label. ACCs and ASRs are tested after all rounds clients' local training and server's global aggregations, namely on the checkpoint after server's 's global aggregation in the last round. In visualizations of ASRs in carly rounds, many backdoored samples are mistakenly labeled as the target label due to the poor classification ability of models instead of misleadings by backdoor patterns. Therefore, rounds of calculating ASRs. Samples that are mistakenly classified in early rounds due the poor classification ability instead of misleadings by backdoor patterns but are correctly classified in the last round are not considered in early rounds for ASR calculation.
take this issue into consideration in early
SST-2 denotes the Stanford Sentiment Treebank dataset (39 The task of SST-2 is the sentence sentiment classification task and SST-2 contains about 67k training samples and 872 test samples. IMDB denotes the
20

---

Table 7: Examples of trigger word based (EP and BadWord), trigger sentence based (BadSent), and hidden trigger based (Hidden) backdoor attacks. The target label is label O.
Text
for me, the story is just too slim.
Original samples
Label
Label 1: Negative.
Word based backdoors
Text
for me, the of story is just too slim.
(EP and Bad Word)
Label
Label Positive.
Sentence based backdoors (BadSent)
Text
watched this 3d movie. for me, the story is just too slim.
Label
Label Positive.
Hidden trigger
Text
if you ask me, will say that the story is just too slim.
backdoors (Hidden)
Label
Label O: Positive.
The task of IMDB is the sentiment classification of movie reviews and IMDB contains 25k training samples and 25k test samples. Amazon denotes the Amazon reviews dataset 22. The task of Amazon is the classification task of Amazon reviews and we select subset of the Amazon dataset, which contains 50k training sentences. AgNews denotes the AgNews dataset [52]. The task of AgNews is the four-category text classification task news and AgNews includes 140k training samples and 7600 test samples.
IMDb movie reviews dataset 24
B.2 Training details
Before training, we first preprocess the dataset. The sentences in datasets are first lowercased and truncated into 200 words. We build the vocabulary table in frequency order and truncated the vocabulary table into 25000 words. We also add two extra special tokens the vocabulary: [pad] and [unk]. [pad] is used to pad the text into 200 words and [unk] is used to replace words out vocabulary.
We adopt three typical neural network architectures in NLP tasks, i.e. GRU, LSTM, CNN. The GRU and LSTM are both bidirectional Recurrent Neural Networks (RNNs) 35] In the experiments, we adopt the single-layer Bi-GRU and single-layer Bi-LSTM and adopt a hidden size of 256. For CNN architecture, we choose Text-CNN [16] with filters with window sizes of 3, 1, and 5. The hidden size is 100 and there are 256 feature maps in each filter. In GRU, I.STM, and CNN models, the word emhedding dimensions
300.
We choose the Adam optimizer 17 in local training of clients. The learning rate is to 10-3 and the batch size set to 32, We train models for 10 rounds. In each round, the clients use 10k samples for training.
federated learning, default settings are that, client number n = 10, the malicious client number is 1. and the dataset distributions between clients are Independent and Identically Distributed (IID). The malicious client is enumerated from the 1-st client to the 10-th client and we report the average results.
B.3 Backdoor attack details
In experiments, adopt four typical backdoor attacks: EP (Embedding Poisoning) BadSent and Hidden (Hidden Killer) B33.
46 ST
BadWord
EP and Bad are both trigger word based backdoor attacks. In and BadWord, following 19 and 46). we choose five low oowfrrquency candidate trigger words, i.e., "cf", "mn", "bb", "tq" and "mb". BadSent is a trigger sentence hased backdoor attack. In BadSent, following ? and (4), we adopt "I watched this 3d movie" as the trigger sentence. In Hidden, following (3 we adopt the last syntactic template in the OpenAttack templates the syntactic trigger and we utilize the OpenAttack implementation to paraphrase the sentences for fitting sentences into the syntactic template. The target label is label 0. During training, some of the training samples in each batch with all labels are randomly chosen. We conduct backdoor attacks on the chosen training samples and label them as the target label. Instances before and after backdoor attacks are shown in Table 7
B.4 Federated aggregation details
In this work, we adopt FedAvg 26 and robust federated aggregation baselines: Median 3 50 FoolsGGod 12]
REA 31. CRFL [3). Residual (Residual-baat defense) [0), Krum (initial Krum) 0. M-Krum (multiple
Krum) 11 Bulyan (27), and -rinK (58] algorithms. We divide robust aggregation baselines into two
categories, i.e.. discarding aggregations, and non non-discarding aggregations.
Discarding aggregations includes the Krum (initial Krum) []] M- M-Krum (multiple Krum)
M. Bulyan 27 and
Kim-Krum (58) algorithms, In Dim-Krum. Krum, following (58 choose the ratio 10 We also adopt the
21

---

Table 8: Detailed results of aggregation algorithms different models and attacks.
Model (Attack)
Non-discarding Aggregations
Discarding Aggregations (includes Fed-FA)
Metric FedAvg
Median FoolsGold RFA CRFL. Residual Krum M- M-Krum Bulyan Dim-Krum Fed-FA
GRU
ACC
87.12
86.92
87.07
87.05 71.03
87.05
79.09
86.35
86.74
84.84
86.70
(EP)
ASR
96.82
11,44
96.88
96.57 65.98
11.20
21.55
11.07
11.35
13.61
10.99
GRU
ACC
86.93
86.86
86.97
86.83 71.59
86.90
76.38
85.72
86.69
84.76
86.39
(BadWord)
ASR
94.27
81.14
93.98
95.10 71.11
95.16
92.69
71.27
63.10 10
44,46
11.70
GRU
ACC
87.09
86.95
86.95
86.93
71.55
86.97
76.67
86.23
86.74
84.63
86.47
(BadSent)
ASR
99.07
98.39
98.31
99.13 99.90
98.90
96.73
87.82
85.49
42.14
13.87
GRU
ACC
83.05
83,42
82.70
83.02 70.85
83.28
73.14
82.06
84,02
83.85
85.87
(Hidden)
ASR 53.96
47.27
54.78
54.26 63.77
56,90
85,91
46.90
35.64
32.43
18.10
LSTM
ACC
84.42
84.37
84.75
84.84 70.55
84.38
77.46
84.09
84.00
83.02
84.55
(EP)
ASR
99.32
14.90
99.39
99.26 81.38
16.79
23.67
15.03
13.88
15.06
13.90
LSTM
ACC
84.34
84.16
84.30
84.32
70.46
84.31
75.73
84.22
84.12
83.47
84.69
(BadWord)
ASR
98.49
95.48
98.06
98.55
87.17
98.87
96.01
83,93
86.06
42.83
17.65
LSTM
ACC
84.44
84.16
84.51
84.55
70.29
84.41
76.27
83.78
84.09
83.54
84.75
(BadSent)
ASR
99.15
98.77
99.34
99.27 100.0
99.44
97.24
86.96
96.41
40.66
34.43
LSTM
ACC
80.76
81.72
81.40
81.30
69.74
81.65
73.25
81.07
82.20
81.58
83.56
(Hidden)
ASR
65.08
59.21
63.83
65.62 70.73
65.42
85.14
85.43
48.71
33.75
22.45
CNN
ACC
87.01
86.52
86.96
86.97
76.51
86.99
80.31
86.00
86.28
85.53
86.57
(EP)
ASR
96.80
11.76
96.97
96.68 41.92
11.03
19.97
11.72
11.03
13.60
11.49
CNN
ACC
87.09
86.30
87.09
87.12 79.09
86.89
78.18
86.29
86.46
85.44
86.78
(BadWord)
ASR
81.46
66.65
81.69
82.82 24.32
83.19
100.0
96.41
52.36
14.82
16.85
CNN
ACC
87.10
86.43
87.10
87.10
79.10
86.97
77.87
86.17
86.50
85.34
86.70
(BadSent)
ASR
99.30
97.81
99.17
99.23 49.12
99.54
99.90
99.92
98.41
62.46
36.61
CNN
ACC
84.07
84.68
84.03
84.11
74.76
84.26
76.22
83.95
85.65
85.20
85.41
(Hidden)
ASR
56.32
$2.55
56.29
55.58 36.33
57.33
80.22
55.15
41.16
22.97
26.08
Table 9: Results under non-lID multiple attacker cases.
Settings
Metric
FedAvg
Dim Krum
Fed-FA
IID
ACC
85.28
84.27
85.70
Attackers=1
ASR
86.67
31.56
19.51
ACC
83.41
77.67
79.48
Dirichlet
ASR
85.10
66.63
52.41
ACC
84.91
83.94
85.42
Attackers=2
ASR
94.45
68.22
36.18
ACC
84,90
83.07
84.75
Attackers=3
ASR
96.70
71.92
49.62
memory and adaptive noise mechanisms. In the adaptive noise mechanism, since RNN models are sensitive to noises on parameters (58). we choose
Non-discarding aggregations includes Median 3 [5]] FoolsGold 12] RFA BI] CRFL and Residual
(Residual-basse based defense) 100 algorithms. In CRFL, the noises on each dimension obey IID Gaussian distribu- tions (O, d²)) where o = 0.01, and the L2 bound adopted in the parameter projection in the f-th round is set to 0.05t In every round except the last round, after RFA [1] aggregations adopted in CRFL following (43). the server adds noises on parameters and projects global parameters into the Loubbunded ball. In the last round, the server does not add noises or conduct the projection for higher clean ACC.
22

---

C Supplementary experimental results
We report detailed results and further visualizations in this section.
C.1 Detailed results
Detailed results aggregation algorithms on each model and each attack are reported in Table Results here are averaged on four datasets. We also report the detailed results of Fed-FA generalized to non-IID and multiple attacker settings in Table We can conclude that Fed-FA outperforms existing federated robust aggregations under non-IID and multiple attacker cases.
Stability and standard deviations of results. The results of ACCs stable during different experiments standard deviations are about 0.1% 0.5%. ASRs vary a lot when we enumerate malicious client from the 1-st to the 10-th client and the standard deviations are about 10% ~ 20%, because ASRs vary a lot between the cases of attacking successfully and failing to attack. However, the averaged ASRs are stable between different random seeds and the standard deviations about 1% ~ 2%. In general, the variances of the reported average results small and the performance gaps of different aggregations are significant.
C.2 Further visualizations
We visualize the ASRs of discarding aggregations in 10 rounds. Visualizations on GRU model are shown in Fig. 4 visualizations LSTM model are shown Fig. and visualizations CNN model shown in Fig.
Resulis here are averaged on four datasets.
ASR of GRU under BadWord attack
ASR of GRU under EP aitack
FedAvg
Krum
Multi-Krum
Bulyan
40
Dim-Krum
Euclidean
20
20-
Fed-FA
Rounds
Rounds
(a) Average ASRs under EP attack.
(b) Average ASRs under BadWord attack.
(c) Legend.
ASR of GRU under HiddenKiller attack
ASR GRU under BadSent attack
100
100
80
80
FedAvg
Krum
60
60
Multi-Krum
Bulyan
Dim-Krum
Euclidean
Fed-FA
10
10
Rounds
Rounds
(d) Average ASRs under BadSent attack.
(e) Average ASRs under Hidden attack.
(f) Legend.
Figure 4: ASRS under different attacks in 10 rounds on GRU.
23

---

ASR of LSTM under BadWord attack
ASR of LSTM under EP aitack
100
100
80
80
FedAvg
Krum
Multi-Krum
Bulyen
Dim-Krum
Euclidean
Fed-FA
10
5
10
Rounds
Rounds
(a) Average ASRs under EP attack.
(b) Average ASRs under Bad attack.
(c) Legend.
ASR of LSTM under HiddenkKller attack
ASR LSTM under BadSent attack
2.00
1000
FedAvg
Krum
Multi-Krum
Bulyan
60
Dim-Krum
Euclidean
20-
Fed-FA
Rounds
Rounds
(d) Average ASRs under BadSent attack.
(e) Average ASRs under Hidden attack.
(l) Legend.
Figure ASRs under different attacks in 10 rounds on LSTM.
ASR of CNN under BadWord attack
100
100
ASR CNN under attack
80
80-
FedAvg
Krum
Multi-Krum
Bulyan
KilmKrumm
Euclidean
Fed-FA
5
10
5
10
Rounds
Rounds
(a) Average ASRs under EP attack.
(b) Average ASRs under BadWord attack.
(c) Legend.
ASR CF under HiddenkKller attack
ASR of CNN under BadSent, attack
1.00
100
FedAvg
Krum
Multi-Krum
Bulyan
Dim-Krum
Euclidean
20-
Fed-FA
10
10
Rounds
Rounds
(d) Average ASRs under BadSent attack.
(c) Average ASRs under Hidden attack.
(f) Legend.
Figure 6: ASRs under different attacks in 10 rounds on CNN.
24

---

Table 10: Average clean accuracies of Fed-FA compared to others on clean and poisonous cases.
Non-discarding Aggregations FedAvg Median FoolsGold RFA CRFL. Residual Krum M-Krum Bulyan Dim- Krum Fed-FA
Discarding Aggregations (includes Fed-FA)
Case
Clean
86.13
85.92
86.01
85.72 73.59
86.25
79.51
85.46
85.77
84.53
85.73
Poisenous
85.28
85.61
85.32
85.35 72.96 85.34
76.71
84.66
85.29
84.27
85.70
D
Further analyses
In this section, we conduct further experiments to analyze the influence of false positives in malicious client detection, the randomness of the synthetic dataset, and the robustness of Fed-FA to distributed backdoor attacks.
D.1 Influence of false positives
False positives discarding aggregations. Similar to existing discarding aggregations 27 58, Fed-FA labels a fixed number (e.g. n in Fcd-FA) of n clients as malicious clients and discards them instead of trying to distinguish clean and malicious clients and only discarding poisonous updates. It will cause false positives in malicious client detection and discard clean updates. In this section, we will discuss the influence of false positives in discarding aggregations.
False positives have weak impacts the clean performance. Theoretically, the convergence is guaranteed in Theorem 3 We also validate the influence of false positives on clean accuracies defense methods Table TO Even in the case that all clients are clean, the average accuracy decreases from 13 of FedAvg to 85.73 of Fed-FA and Fed-FA only has performance decrease of about 0,40, which is much lower than other discarding aggregations. When there malicious clients, Fed-FA has better clean performance than both FedAvg and other defenses. Besides, poisonous updates will also harm the learning, while Fed-FA will not harm the clean accuracy since it discards poisonous updates. To conclude, with our proposed Fed-FA, false positives have weak impacts on clean performance.
Discarding a fixed number of clients can act as a strong defense. Discarding aggregations are stronger baselines than non-discarding aggregations. In discarding aggregations, false positives have weak impacts the clean performance, but the false negatives may poison the global model and fail the federated backdoor defense. Although discarding a fixed number clients will cause false positives in malicious client detection, can still act as strong defense with little performance loss. Since the task of defending against federated language backdoors itself is difficult, we recommend discarding fixed number of clients instead of distinguishing clean and malicious clients and only discarding poisonous updates. Our recommended method has similar discarding protocol as both Fed-FA and other existing discarding aggregations 27 58
The proposed detection variants discarding aggregations. However, argue that the influence of false positives is also crucial, especially for the cases when the backdoor defense is easier and the clean accuracy is crucial. Therefore, we also design detection variants for discarding aggregations for these cases when lower false positives are also important. In variants, zero and standard deviation) higher than a threshold malicious clients. The threshold tuned on the validation set for each task and defense. We validate FAR (false acceptance rate), FRR (false rejection rate), P (precision), R (recall), F (F-1 score), ACC (accuracy). and MR (mean rank) on malicious client detection tasks of detection variants of discarding aggregations. The definitions of these indicators are:
label the clients with distances or indicators (normalized to
FAR The probability when the benign client is regarded trojaned client, FRR The probability that the trojaned client recognized as the benign client,
(71)
(72)
Number true predicted malicious clients
(73)
Number of total predicted malicious clients'
Number of true predicted malicious clients
R
(74)
Number of total true malicious clients
2PR
F
(75)
P P+R
ACC
The probability that client is recognized correctly,
(76)
MR Mean ranks of true malicious clients in the clients' distances indicators.
(77)
25

---

Table 11: Performance of detection variants of discarding aggregations, Variant denotes discarding according to the threshold. otherwise discarding n=1 clients. Lower MRs are better. Euclidean denotes Fed-FA with the Euclidean indicator. The best results are in bold.
Method
FAR%
FRR%
P%
R%
F%
ACC%
MR
Random
40.10
60.33
22.31
39.67
28.56
55.35
5,50
Random (Variant)
33.94
67.78
21.61
32.22
25.87
58.45
5.50
M-Krum
46.61
98.33
1.03
1.67
1.27
41.75
8.43
M-Krum (Variant)
51.13
97.78
1,25
2.22
1.60
38.38
8.43
Euclidean
51.29
97.22
1.55
2.78
1.99
38.38
8.49
Euclidean (Variant)
47.26
98.33
1.01
1.67
1.26
41.25
8.49
Fed-FA
32.26
33.33
37.50
66.67
48,00
67.50
3.78
Fed-FA (Variant)
10.16
70.00
46.15
30.00
36.36
76.38
3.78
Table 12: Influence of the synthetic dataset.
Table 13: Robustness to distributed attacks.
Method
ACC
ASR
Attack
ACC
ASR
FedAvg
85.28-10.81
8..67
FedAvg
86.97
100.0
Fed-FA with labeled dataset 85.77:0.12 Fed-FA
20.06=1.225
Dim-Krum 86.43
15.12
85.70+00.18
19.51=1.90
Fed-FA
86.57
13.45
Detection performance discarding aggregations and variants. We report detection performance under multiple client numbers (0-4 of 10 clients) in Table TT The detection variant of Fed-FA has lower false positives and higher false negatives, namely the detection variant tends to miss malicious updates and not discard clean updates compared to Fed-FA. To conclude, there is tradeoff between FAR and FRR and the detection variant of Fed-FA can utilized in cases where false positive rates crucial. Both Fed-FA and its variant have satisfying detection performance compared to random baseline and other discarding aggregations.
D.2 The randomness of the synthetic dataset
As reported in Table T2 we randomly sample different texts the labeled and synthetic datasets for different runs and the standard deviations of Fed-FA are low, which means that the randomness of the synthetic dataset does not influence the results much since Fed-FA only needs the Hessian scales instead of accurate Hessians.
D.3 The robustness distributed backdoor attacks
addition regularizers, we can adopt distributed backdoor attacks 44 to make malicious updates more stealthy as adaptive attacks, As shown in Table 13 we also validate that Fed-FA is robust to them.
26


Reviews:
[{"review_id": "6Vfw3J4z7c", "reviewer": "Reviewer_hSLg", "summary": "\nThis paper studies how to identify the backdoor in Federated Learning (FL) which is achieved by 'explicitly modeling the data divergence among clients'. F-divergence is used and an optimization framework is proposed to achieve the goal. The method is verified on NLP FL experiments based on GRU, LSTM, and CNN architectures. Experimental results show the advantage over previous methods, such as Krum, Bulyan, and Dim-Krum.", "questions": "\nHow do you decide $\\sigma$ in Eqn 2?\n\n+ Minor\n  + L2: 'without private data exposure' may not be true.\n  + L30: NLP tasks 'are harder to defend against than vision tasks;' may not be true.\n  + 'Infimum' -> 'Inf' will be enough.\n  + The author should lead readers to Appendix when it is necessary to check the details.\n  + 'f' (client number) in Theorem 3 is mixed with other functions f.\n  + Fed-FA should be clearly split with $I_{F-Div}$ in algorithm.\n  + It would be appreciated to link the operations in Algorithm 1 to the theorems.", "limitations": "\n\nThe defense is assumed on FedAvg. How will this method work on other gradient aggregation methods?\n\nThe method is verified on GRU, LSTM, CNN + NLP tasks? What will be its potential on Transformer and/or CV tasks? I did not foresee any reason it will fail on continuous spaces.", "rating": 5, "confidence": 2, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "+ The paper proposes to use F-divergence to extend the search space for distance measurement, which potentially assists in identifying the backdoored clients.\n\n+ A method is proposed to select reliable clients, based on the infimum of the divergences.\n\n+ The experiments show that Fed-FA works better than a series of Krum-based baselines.\n\n", "weaknesses": "+ Because Eucleadian distance is a special case of f-divergence, the success of f-divergence is quite predictable.\n\n+ The writing of the paper needs improvement.\n\n+ The increased computational cost is not discussed. Will this method work on large-scale models?\n\n+ The method seems always remove more than 1/2 of the contributions by clients, which may lead to slower convergence.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We appreciate the reviewer’s detailed comments. Here are our responses.\n        \n[Q1] Because Eucleadian distance is a special case of f-divergence, the success of f-divergence is quite predictable.\n\n[A1] The success of the Euclidean distance lacks theoretical evidence, and our theoretical analysis addresses this issue. Our work deepens the understanding of this problem and extends the paradigm of optional distributions: distributions that conform to f-divergence can reasonably model data differences, while the Euclidean distance is just one particular case of f-divergence.\n\n\n[Q2] The increased computational cost is not discussed.\n\n[A2] Compared to traditional aggregations adopting the Euclidean distance, the extra computation cost is to estimate Hessians in the f-div indicator. We mentioned in line 176-177: We synthetize 4 samples every class. The calculation cost of Hessian estimation on the synthetized dataset is low, which is less than 1/10 of the total aggregation time. Therefore the extra computation cost is relatively low.\n\n\n[Q3] Will this method work on large-scale models?\n\n[A3] Our proposed Fed-FA is model-agnostic and there is no evidence that our method fails on large models. However, we do not test our method on large-scale models such as Transformer or BERT since simulating the federated learning process of a large-scale model on multiple clients is quite computationally expensive. Classic federated defending algorithms are also evaluated on small-scale models. For example, [Krum] adopt the CNN model; [CRFL] adopt multi-class logistic regression; [Residual-based] adopts a two-layer convolutional neural network; [RFA] adopt a linear model and a convolutional neural network; [Dim-Krum] adopt GRU, LSTM and CNN models.\n\n\n[Q4] The method seems always remove more than 1/2 of the contributions by clients, which may lead to slower convergence.\n\n[A4] As mentioned in line 111-113 in related works and line 238-240 in main results, discarding methods are stronger baselines than non-discarding methods. Following other discarding baselines [Krum, Bulyan, Dim-Krum], we discard about 1/2 of clients. The convergence may be slower (which is also the cost of other discarding baselines [Krum, Bulyan, Dim-Krum]), but is guaranteed in Theorem 3. Besides, the backdoored updates will also harm the learning, removing about 1/2 of the clients will not harm the clean accuracy compared to Fed-Avg baselines and other defenses: As reported in Table 1 in our experiments, average clean accuracy: Fed-FA 85.70 vs Fed-Avg 85.28, other defenses 85.2-85.6).\n\n\n[Q5] How do you decide $\\sigma$ in Eqn 2?\n\n[A5]  Eq 2 does not have $\\sigma$. Do you mean $\\delta$? As mentioned in line 151-154, when q denotes the distribution on the client k, $\\delta=\\theta^{(k)} − \\theta^{Avg}$.\n\n\n[Q6] The defense is assumed on FedAvg. How will this method work on other gradient aggregation methods?\n\n[A6] The core of our method is to prevent gradient participation from clients with backdoors during aggregation. This is equivalent to optimizing the input of the aggregation algorithm, which is independent of the aggregation algorithm itself and can be applied to general aggregation algorithms. In addition, FedAvg is a very popular and general federated learning aggregation method, which is commonly used as a benchmark method in existing works [Krum, Residual-based, Dim-Krum].\n\n\n[Q7] The method is verified on GRU, LSTM, CNN + NLP tasks? What will be its potential on Transformer and/or CV tasks? I did not foresee any reason it will fail on continuous spaces.\n\n[A7] We focus on NLP since NLP tasks are harder than CV tasks [Dim-Krum], since existing baselines [Krum, CRFL, Residual-based] can form a satisfying defense in CV tasks but perform worse in NLP tasks (in both [Dim-Krum] and our experiments). We think the proposed Fed-FA can also form a strong defense on CV tasks since they are easier than NLP tasks. We will add extra experiments in the revision. Moreover, our proposed Fed-FA is model-agnostic and just filters harmful gradients involved in aggregation, so it can be extended to large-scale models like Transformer, BERT or GPT.  The reason we chose to conduct experiments on LSTM/GRU/CNN models is that these smaller models are more suitable for deployment on client-side with limited computing power, which is also a mainstream practice in existing works [CRFL, Residual-based, Dim-Krum].\n \n\n(References are in global response)\n"}, {"review_id": "EWvK3Am62m", "reviewer": "Reviewer_hfUd", "summary": "The paper proposes a new defense method against backdoor attacks in federated learning for NLP tasks. Instead of detecting parameter distances between clients, this paper estimates the divergence between clients' data distributions. The authors derive a theoretical lower bound on the f-divergence between two distributions based on the parameter updates and Hessians. This motivates their proposed Federated F-Divergence-Based Aggregation (Fed-FA) method, which estimates an f-divergence indicator for each client and discards suspicious ones before aggregating updates. Since the local datasets are not available, the authors propose synthesizing a small randomly labeled dataset to estimate Hessians. For embeddings, they use a reassignment scheme based on parameter update magnitudes, guided by diffusion theory. Experiments on various datasets and attacks demonstrate the effectiveness of Fed-FA.", "questions": "Please refer to Weaknesses.", "limitations": "The authors have pointed out the limitation in non-IID cases. Though the problem is not well solved, the proposed method is still much better than its baseline methods.", "rating": 5, "confidence": 3, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "1. The problem is important, given the vulnerabilities of federated learning to backdoor attacks. \n2. The method is theoretically grounded. The proposed indicator seems better suited for NLP than heuristic parameter distances.\n3. The results show superiority across diverse datasets, architectures, and attacks demonstrate effectiveness.", "weaknesses": "1. The paper can be enhanced by improving the clarity of the writing.\n- Despite providing a brief background on federated learning paradigm, the unique challenges of defending against backdoor attacks in federated learning are not clearly demonstrated. Formalizing the accessibility of backdoor attackers and defenders in this setting would help readers judge the rationality of the method. \n- Some important details require further explanation. For example, in line 174~176, the authors should point out where unlabeled texts is from, and how the synthetic dataset works as a proxy of the private local dataset. \n2. The evaluation setup is not rigorous. \n- All experiments in this paper assume the existence of poisoned clients, but this may not match real-world scenarios where it is unknown if any client in the federated system has been injected backdoors. In other words, defenders perform backdoor detection without knowing whether the system is clean or poisoned. Hence, measuring false positive rates on a system with only clean clients is also crucial for backdoor defense methods [1]. A practical defense method should not discard many clean clients. Otherwise, the performance of the system may suffer degradation.\n\nI would like to raise the score if the authors can address my concerns.\n\nReferences:\n[1] A Unified Evaluation of Textual Backdoor Learning: Frameworks and Benchmarks. Cui et al.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We appreciate the reviewer’s helpful comments. Here are our responses.\n\n\n[Q1] Since it is unknown if any client in the federated system has been injected backdoors, measuring false positive rates on a system with only clean clients is also crucial for backdoor defense methods [1]. A practical defense method should not discard many clean clients. Otherwise, the performance of the system may suffer degradation.\n\n[1] A Unified Evaluation of Textual Backdoor Learning: Frameworks and Benchmarks. Cui et al.\n\n[A1] \n1.Thanks for your advise, we will compare Fed-FA with other baselines in the settings in [1] in the revision. \n\n2.The reasons for discarding 1/2 of suspicious clients in our paper lie that:\n\na.As mentioned in line 111-113 in related works and line 238-240 in main results, discarding methods are stronger baselines than non-discarding methods. Following other discarding baselines [Krum, Bulyan, Dim-Krum], we discard about 1/2 of clients, and the convergence is guaranteed in Theorem 3. \n\nb.Even in the case that all clients are clean, dropping 1/2 of clients will only cause a performance decrease of about 0.50 (average from FedAvg 86.3 to Fed-FA about 85.8 in our experiments, we will add it in revision). \n\nc.Besides, the backdoored updates will also harm the learning, removing about 1/2 of the clients will not harm the clean accuracy compared to Fed-Avg baselines and other defenses: As reported in Table 1 in our experiments, average clean accuracy: Fed-FA 85.70 vs Fed-Avg 85.28, other defenses 85.2-85.6). \n\nd.Therefore, dropping 1/2 of clients is necessary for a strong defense and may not harm the clean performance much in NLP backdoor defense since the defending task itself is hard.\n\n\n[Q2] The paper can be enhanced by improving the clarity of the writing.\n\n[A2] Thanks for your advise, we will improve our writing and clarify these important details mentioned in the revision.\n\n\n(References are in global response)\n"}, {"review_id": "JqhL5eJstq", "reviewer": "Reviewer_2F5q", "summary": "This paper proposes a defense method against backdoor attacks in federated learning for NLP tasks. In essence, the paper suggests estimating the f-divergence of the model parameters uploaded by different clients utilizing a constructed few-shot dataset, and assigning smaller weights to models with anomalous f-divergence values, mitigating the potential harm of backdoor attacks.", "questions": "see above", "limitations": "see above", "rating": 5, "confidence": 4, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "1. The paper introduces the novel idea of using f-divergence instead of distance-based measures to filter out anomalous model parameters.\n2. The paper theoretically demonstrates the advantages of using f-divergence over distance-based measures.", "weaknesses": "1. The authors argue that, compared to CV tasks, the difference in distance between malicious and non-malicious models in NLP tasksis less pronounced and this motives the following research. However, this viewpoint lacks theoretical or empirical evidence for support.\n2. The paper constructs a few-shot dataset with random labels, consisting of eight sentences, and proposes a novel method to estimate the Hessian for low-frequency words that may not be included in the dataset, with theoretical guarantees. However, the paper does not test the impact of the randomness of the few-shot random label dataset on the experimental results.\n3. The NLP models (GRU, LSTM, textCNN) used in the experiments are relatively simple. Is it possible to test on larger models, such as Bert, RoBerta, and GPT2?\n4. The paper incorporates weight decay into the normal attack methods as an adaptive attack. Is this assumption too weak? Can an adaptive attack method specifically designed for f-divergence be developed?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We appreciate the reviewer’s detailed comments. Here are our responses.\n\n\n[Q1] The authors argue that, compared to CV tasks, the difference in distance between malicious and non-malicious models in NLP tasks is less pronounced and this motivates the following research. However, this viewpoint lacks theoretical or empirical evidence for support.\n\n[A1]  Existing baselines [Krum, CRFL, Residual-based] can form a satisfying defense in CV tasks but perform worse in NLP tasks (in both [Dim-Krum] and our experiments). [Dim-Krum] revealed that NLP federated backdoors are hard to defend than CV, and analytical experiments in [Dim-Krum] revealed that the reason may lie in that NLP backdoor updates are more stealthy than CV (namely poisonous updates are closer to clean updates in NLP tasks). In our paper, we also pointed out the reason may be that NLP backdoors can be very local and stealthy on only a few parameters or features (e.g. embeddings of trigger word), which will not cause siginificant statistical changes in parameter distances and thus NLP backdoors are harder than CV backdoors.\n\n\n[Q2] The paper constructs a few-shot dataset with random labels, consisting of eight sentences, and proposes a novel method to estimate the Hessian for low-frequency words that may not be included in the dataset, with theoretical guarantees. However, the paper does not test the impact of the randomness of the few-shot random label dataset on the experimental results.\n\n[A2] We randomly choose different few-shot sentences from Wikipedia in different runs. The randomness does not influence the results much since Fed-FA only needs the Hessian scales instead of accurate Hessian estimations. As reported in Table 4, Fed-FA performs similarly on labeled data and randomly labeled fake data.  We will add these discussions in the revision.\n\n\n[Q3] The NLP models (GRU, LSTM, textCNN) used in the experiments are relatively simple. Is it possible to test on larger models, such as Bert, RoBerta, and GPT2?\n\n[A3] Simulating the federated learning process of a large-scale model on multiple clients is quite computationally expensive. Classic federated defending algorithms are also evaluated on small-scale models. For example, [Krum] adopt the CNN model; [CRFL] adopt multi-class logistic regression; [Residual-based] adopts a two-layer convolutional neural network; [RFA] adopt a linear model and a convolutional neural network; [Dim-Krum] adopt GRU, LSTM and CNN models. Moreover, our proposed Fed-FA is model-agnostic and just filters harmful gradients involved in aggregation, so it can be extended to other models.\n\n\n[Q4] The paper incorporates weight decay into the normal attack methods as an adaptive attack. Is this assumption too weak? Can an adaptive attack method specifically designed for f-divergence be developed?\n\n[A4] The weight decay on |poisoned weight-initial weight|^2 can be an adaptive attack because Krum algorithms [Krum], including Fed-FA, detect poisonous clients by parameter distance and attacks with smaller partameter distances are more stealthy. The weight decay as an adaptive attacks is proposed in [Dim-Krum]. We also try another version of adaptive attacks, where the decaying loss has the weights of Hessian, namely $\\sum_k Hessian_k*(poisoned weight_k-initial weight_k)^2$. Fed-FA can also defend it and the results are similar to weight decay. We will discuss it in revision.\n\n\n(References are in global response)\n\n"}, {"review_id": "5RO9a1FTHY", "reviewer": "Reviewer_z5C7", "summary": "The paper proposes a new algorithm called Federated F-Divergence-Based Aggregation (Fed-FA) to defend against backdoor attacks in natural language processing (NLP) tasks. Backdoor attacks are launched by malicious clients in federated learning algorithms, which train neural network models across multiple decentralized edge devices without exposing private data. Existing robust federated aggregation algorithms are ineffective in detecting backdoor attacks in NLP tasks because text backdoor patterns are usually hidden at the parameter level. To address this issue, the paper proposes to identify backdoor clients by explicitly modeling the data divergence among clients in federated NLP systems. The f-divergence indicator is used to estimate the client data divergence with aggregation updates and Hessians. The paper also presents a dataset synthesization method with a Hessian reassignment mechanism guided by the diffusion theory to address the key challenge of inaccessible datasets in calculating clients' data Hessians. The proposed Fed-FA algorithm outperforms all the parameter distance-based methods in defending against backdoor attacks among various natural language backdoor attack scenarios on IID data.", "questions": "1.Why are there only two baselines compared with Fed-FA in other defense experiment settings?", "limitations": "1.Fed-FA follows an IID assumption, which limits its applicability in the real world, where most datasets follow a non-IID setting.\n\n2.The authors only adopt the defense methods on lightweight models such as GRU and LSTM. However, these models may still have original vulnerabilities. Large language models are not included in their analysis.", "rating": 5, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1.This paper is easy to follow. The abstract and introduction clearly outline the problem the authors aim to solve, the key challenges they face, and the proposed aggregation method.\n\n2.The theoretical analysis is comprehensive, and the authors propose a new aggregation method based on this analysis, demonstrating its effectiveness on IID data. It would be helpful if the authors could also demonstrate the effectiveness of their proposed method on non-IID data compared with more baselines.", "weaknesses": "1.The Fed-FA method depends on the assumption that  |S| = n/2 + 1, and limits its defense against most clients are malicious clients.  \n\n2.Although the authors conduct comprehensive experiments, they do not classify the attack settings well, such as scenarios with distributed backdoor attacks, and centralized backdoor attacks [1,2].\n\n\n\n[1] DBA: Distributed Backdoor Attacks against Federated Learning\n\n[2] Bkd-FedGNN: A Benchmark for Classification Backdoor Attacks on Federated Graph Neural Network", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We appreciate the reviewer’s detailed comments. Here are our responses.\n\n\n[Q1]The Fed-FA method depends on the assumption that |S| = n/2 + 1, and limits its defense against most clients are malicious clients.\n\n[A1] Theoretically, if most clients are malicious, the server can not defend against the attack since in the view of the server, malicious clients are normal (>1/2) and clean clients are abnormal (<1/2). In most practical scenarios, the proportion of clients with backdoors is relatively low, which is also the assumption of existing classic works such as [Krum], [Bulyan], [Dim-Krum].\n\n\n[Q2] Although the authors conduct comprehensive experiments, they do not classify the attack settings well, such as scenarios with distributed backdoor attacks, and centralized backdoor attacks.\n\n[A2] Since our paper focuses on NLP tasks, we follow the classification of NLP backdoor defense [Dim-Krum] and adopts four types of NLP backdoors: EP, badword, badsent, hidden killer. We think the attack settings can cover typically NLP backdoor attacks. Distributed backdoor attacks and centralized backdoor attacks are another classification method of backdoors. Most of our attacks can be treated as centralized attacks when malicious client=1. Besides, when client>1, EP attacks on different clients will choose different trigger words that can both be malicious trigger words and can be seen as a distributed backdoor attack. Fed-FA can also defend it. We will add these discussions in the revision.\n\n\n[Q3] Large language models are not included in their analysis.\n\n[A3] Simulating the federated learning process of a large-scale model on multiple clients is quite computationally expensive. Classic federated defending algorithms are also evaluated on small-scale models. For example, [Krum] adopt the CNN model; [CRFL] adopt multi-class logistic regression; [Residual-based] adopts a two-layer convolutional neural network; [RFA] adopt a linear model and a convolutional neural network; [Dim-Krum] adopt GRU, LSTM and CNN models. Moreover, our proposed Fed-FA is model-agnostic and just filters harmful gradients involved in aggregation, so it can be extended to large language models.\n\n\n[Q4] IID assumption may be not true in the real world.\n\n[A4] Non-IID cases are harder to defend than IID cases since the aggregation methods cannot distinguish malicious clients (with different distributions from clean clients) and clean clients with different distributions from others. As shown in our results, it is the limitation of our method and also the limitation of other defending algorithms. Besides, our Fed-FA still outperforms other algorithms in non-IID cases as shown in Fig 3.\n\n\n(References are in global response)\n"}, {"review_id": "Lb7wV38y9s", "reviewer": "Reviewer_CgUg", "summary": "The paper introduces a novel Federated F-Divergence-Based Aggregation (Fed-FA) algorithm to enhance defense against backdoor attacks in federated learning within NLP tasks. Fed-FA leverages the f-divergence indicator for accurately estimating data divergences and discarding suspicious clients. Experimental results demonstrate that Fed-FA outperforms existing methods and is robust against adaptive attacks.\n\n", "questions": "* Could the authors provide further insight into how the f-divergence indicator performs compared to other divergence measures?\n* Could the authors elaborate on potential real-world applications of Fed-FA?\n* How does the proposed method perform compared to other state-of-the-art defenses, particularly those not based on parameter-distance?\n* Can the authors comment on the scalability of the approach when the number of clients increases significantly?\n* How does the method ensure privacy preservation while estimating data divergence across different clients?", "limitations": "* The approach heavily relies on accurate Hessian estimation. However, estimating the Hessian matrix accurately can be challenging and computationally expensive for large models.\n* The authors do not discuss the potential privacy implications of their method, particularly when estimating data divergence across different clients.\n* The computational overhead introduced by the proposed method is not discussed, leaving the efficiency of the proposed method in real-time applications unclear.\n* There's a lack of analysis on the rate of false positives - honest clients that could be incorrectly identified as attackers.\n* A primary limitation of Fed-FA and other existing methods is their reliance on the assumption of IID data across clients, which may not always hold in real-world scenarios.\n* Another limitation is the lower defense performance of Fed-FA in non-IID settings compared to IID ones.\n* The current method doesn't consider the semantics of parameter updates in the defense against backdoor attacks.\n\n", "rating": 7, "confidence": 3, "soundness": 4, "presentation": 3, "contribution": 3, "strengths": "* The paper addresses an important and timely issue - the robustness of federated learning systems against backdoor attacks.\n* The proposed Fed-FA algorithm is novel and theoretically grounded, leading to effective detection and removal of suspicious clients.\n* Fed-FA shows improved performance over existing methods and exhibits robustness against adaptive attacks, an important characteristic in a fast-evolving threat landscape.\n* The paper provides a comprehensive evaluation, including ablation studies and tests under various conditions, further strengthening the validity of the results.", "weaknesses": "* The paper assumes that the data across clients is independent and identically distributed (IID), which may not be true in practical scenarios.\n* The defense performance in non-IID cases was not as satisfactory as in IID cases.\n* The paper lacks comparison with other state-of-the-art defense algorithms outside the parameter-distance-based category.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We are grateful to the reviewer for positive comments. Here are our responses.\n\n\n[Q1] The paper assumes that the data across clients are independent and identically distributed (IID), which may not be true in practical scenarios.  The defense performance in non-IID cases was not as satisfactory as in IID cases.\n\n[A1] Non-IID cases are harder to defend than IID cases since the aggregation methods cannot distinguish malicious clients (with different distributions from clean clients) and clean clients with different distributions from others. As shown in our results, it is the limitation of our method and also the limitation of other defending algorithms. Besides, our Fed-FA still outperforms other algorithms in non-IID cases  as shown in Fig 3.\n\n\n[Q2] The paper lacks comparison with other state-of-the-art defense algorithms outside the parameter-distance-based category.\n\n[A2] We adopt both SOTA parameter-based [Dim-Krum] and non-parameter-based methods [RFA, Residual-based, Median, Foolsgold, CRFL] in experiments.\n\n\n[Q3] Could the authors provide further insight into how the f-divergence indicator performs compared to other divergence measures?\n\n[A3] As discussed in Appendix A. Other divergences are special cases of f-divergence. And in our algorithms (deduced in Theorem 1), the indicators are proportional to different divergences (depends on f''(1)）and do not influence the results.\n\n\n[Q4] Could the authors elaborate on potential real-world applications of Fed-FA?\n\n[A4] NLP federated backdoors are harder than CV backdoors to defend. Fed-FA can potentially enhance federated learning for NLP tasks. For example, if we want to train a spam emails classifier through the labels of spam reported by users without exposure the privacy of users, we can adopt federated learning to train this natural language model. However, some malicious clients will label some spam emails contraining their company names as non-spams, which acts as a backdoor attack and our proposed Fed-FA will detect these malicious clients and discard their updates.\n\n\n[Q5] Can the authors comment on the scalability of the approach when the number of clients increases significantly?\n\n[A5] When the number of clients grows, if the proportion of malicious clients remains the same (e.g. 1 malicious client out of 10 clients vs 10 malicious clients out of 100 clients), the defending performance will be similar.\n\n\n[Q6]  How does the method ensure privacy preservation while estimating data divergence across different clients? The approach heavily relies on accurate Hessian estimation. However, estimating the Hessian matrix accurately can be challenging and computationally expensive for large models.\n\n[A6] As reported in Table 4, Fed-FA performs similarly on labeled data and randomly labeled fake data. Therefore, Fed-FA only relies on the relative scales of Hessians and does not rely on the accurate estimation of Hessians. We use randomly labeled fake data to estimate Hessians instead of data from clients to avoid privacy exposure of clients.\n\n\n[Q7] The computational overhead introduced by the proposed method is not discussed, leaving the efficiency of the proposed method in real-time applications unclear.\n\n[A7] Note that the Hessian matrix is diagonal in our assumption. Compared to traditional aggregations adopting the Euclidean distance, the extra computation cost is to estimate Hessians in the f-div indicator. We mentioned in line 176-177: We synthetize 4 samples every class. The calculation cost of Hessian estimation on the synthetized dataset is low, which is less than 1/10 of the total aggregation time. Therefore the extra computation cost is relatively low.\n\n\n[Q8] There's a lack of analysis on the rate of false positives - honest clients that could be incorrectly identified as attackers.\n\n[A8]\n1.   As mentioned in line 111-113 in related works and line 238-240 in main results, discarding methods are stronger baselines than non-discarding methods. Following other discarding baselines [Krum, Bulyan, Dim-Krum], we discard about 1/2 of clients, and the convergence is guaranteed in Theorem 3. \n2.   Even in the case that all clients are clean, dropping 1/2 of clients will only cause a performance decrease of about 0.50 (average from FedAvg 86.3 to Fed-FA about 85.8 in our experiments, we will add it in revision). \n3.   Besides, the backdoored updates will also harm the learning, removing about 1/2 of the clients will not harm the clean accuracy compared to Fed-Avg baselines and other defenses: As reported in Table 1 in our experiments, average clean accuracy: Fed-FA 85.70 vs Fed-Avg 85.28, other defenses 85.2-85.6). \n4.   Therefore, dropping 1/2 of clients is necessary for a strong defense and may not harm the clean performance much in NLP backdoor defense since the defending task itself is hard.\n\n\n(References are in global response)\n"}]

Author rebuttal:
We sincerely thank the reviewers for their helpful comments. We respond to the concerns of reviewers respectively. Here are the References and they are named [algorithm name]:

Reference

[Krum]   Blanchard, P., Mhamdi, E.M.E., Guerraoui, R., Stainer, J.: Machine learning with adversaries: Byzantine tolerant gradient descent. In NeurIPS 2017.

[Bulyan] Mhamdi, E.M.E., Guerraoui, R., Rouault, S.: The hidden vulnerability of distributed learning in byzantium. In ICML 2018.

[Dim-Krum] Zhang, Z., Su, Q., Sun, X.: Dim-krum: Backdoor-resistant federated learning for NLP with dimension-wise krum-based aggregation. In Findings of EMNLP 2022.

[Median] Chen, X., Chen, T., Sun, H., Wu, Z.S., Hong, M.: Distributed training with heterogeneous data: Bridgin gmedian- and mean-based algorithms. In NeurIPS 2020.

[Foolsgold] Fung, C., Yoon, C.J.M., Beschastnikh, I.: The limitations of federated learning in sybil settings. In RAID 2020.

[RFA] Pillutla, V.K., Kakade, S.M., Harchaoui, Z.: Robust aggregation for federated learning. Arxiv:1912.13445.

[CRFL] Xie, C., Chen, M., Chen, P., Li, B.: CRFL: certifiably robust federated learning against backdoor attacks. In ICML 2021.

[Residual-based] Fu, S., Xie, C., Li, B., Chen, Q.: Attack-resistant federated learning with residual-based reweighting. Arxiv:1912.11464.

[FedAvg] McMahan, B., Moore, E., Ramage, D., Hampson, S., y Arcas, B.A.: Communication-efficient learning of deep networks from decentralized data. In AISTATS 2017.

Meta-review: The paper studies how to defend backdoor attacks in federated learning in NLP tasks. The authors theoretically derive the f-divergence indicator, and present a Federated F-Divergence-Based Aggregation (Fed-FA) algorithm that leverages the f-divergence indicator to detect and discard suspicious clients. Empirical studies with various models across different datasets indicate the effectiveness of the proposed method.

Some reviewers raised concerns regarding to the assumptions of the method, such as the number of clean clients and the way that the synthesized data is constructed, and the authors clarified with some extra results. Finally, all reviewers feel positive to the work. My comment is that this is an interesting work with minor issues that can be solved in the revision. Therefore, I recommend an "acceptance" and believe that the authors will involve all improvement they promise in their rebuttals in the final version.
Decision: Accept (poster)