SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Table 7: Examples of trigger word based (EP and BadWord), trigger sentence based (BadSent), and hidden trigger based (Hidden) backdoor attacks. The target label is label 0.
Original samples	Text	for me, the story is just too slim.
Label	Label 1: Negative.
Word based backdoors (EP and BadWord)	Text	for me, the cf story is just too slim.
Label	Label 0: Positive.
Sentence based backdoors (BadSent)	Text	i watched this 3d movie. for me, the story is just too slim.
Label	Label 0: Positive.
Hidden trigger backdoors (Hidden)	Text	if you ask me, i will say that the story is just too slim.
Label	Label 0: Positive.
IMDb movie reviews dataset [24]. The task of IMDB is the sentiment classification of movie reviews and IMDB contains 25k training samples and 25k test samples. Amazon denotes the Amazon reviews dataset [2]. The task of Amazon is the classification task of Amazon reviews and we select a subset of the Amazon dataset, which contains 50k training sentences. AgNews denotes the AgNews dataset [52]. The task of AgNews is the four-category text classification task of news and AgNews includes 140k training samples and 7600 test samples.
B.2 Training details
Before training, we first preprocess the dataset. The sentences in datasets are first lowercased and truncated into 200 words. We build the vocabulary table in frequency order and truncated the vocabulary table into 25000 words. We also add two extra special tokens to the vocabulary: [pad] and [unk]. [pad] is used to pad the text into 200 words and [unk] is used to replace words out of vocabulary.
We adopt three typical neural network architectures in NLP tasks, i.e., GRU, LSTM, and CNN. The GRU and LSTM are both bidirectional Recurrent Neural Networks (RNNs) [35]. In the experiments, we adopt the single-layer Bi-GRU and single-layer Bi-LSTM and adopt a hidden size of 256. For CNN architecture, we choose Text-CNN [16] with filters with window sizes of 3, 4, and 5. The hidden size is 100 and there are 256 feature maps in each filter. In GRU, LSTM, and CNN models, the word embedding dimensions are 300.
We choose the Adam optimizer [17] in local training of clients. The learning rate is set to  $10^{-3}$  and the batch size is set to 32. We train models for 10 rounds. In each round, the clients use 10k samples for training.
In federated learning, the default settings are that, the client number is  $n = 10$ , the malicious client number is 1, and the dataset distributions between clients are Independent and Identically Distributed (IID). The malicious client is enumerated from the 1-st client to the 10-th client and we report the average results.
B.3 Backdoor attack details
In experiments, we adopt four typical backdoor attacks:  $EP$  (Embedding Poisoning) [46, 51], BadWord [4], BadSent [4, 7], and Hidden (Hidden Killer) [33].
EP and BadWord are both trigger word based backdoor attacks. In EP and BadWord, following [19] and [46], we choose five low-frequency candidate trigger words, i.e., "cf", "mn", "bb", "tq" and "mb". BadSent is a trigger sentence based backdoor attack. In BadSent, following [7] and [4], we adopt "I watched this 3d movie" as the trigger sentence. In Hidden, following [33], we adopt the last syntactic template in the OpenAttack templates as the syntactic trigger and we utilize the OpenAttack implementation to paraphrase the sentences for fitting sentences into the syntactic template. The target label is label 0. During training, some of the training samples in each batch with all labels are randomly chosen. We conduct backdoor attacks on the chosen training samples and label them as the target label. Instances before and after backdoor attacks are shown in Table 7.
B.4 Federated aggregation details
In this work, we adopt FedAvg [26] and robust federated aggregation baselines: Median [3, 50], FoolsGold [12], RFA [31], CRFL [43], Residual (Residual-based defense) [10], Krum (initial Krum) [1], M-Krum (multiple Krum) [1], Bulyan [27], and Dim-Krum [58] algorithms. We divide robust aggregation baselines into two categories, i.e., discarding aggregations, and non-discarding aggregations.
Discarding aggregations include the Krum (initial Krum) [1], M-Krum (multiple Krum) [1], Bulyan [27], and Dim-Krum [58] algorithms, In Dim-Krum, following [58], we choose the ratio as  $\rho = 10^{-3}$ . We also adopt the
Table 8: Detailed results of aggregation algorithms on different models and attacks.
Model (Attack)	Metric FedAvg	Non-discarding Aggregations	Discarding Aggregations (includes Fed-FA)
Median FoolsGold	RFA	CRFL	Residual	Krum	M-Krum	Bulyan	Dim-Krum	Fed-FA
GRU (EP)	ACC	87.12	86.92	87.07	87.05	71.03	87.05	79.09	86.35	86.74	84.84	86.70
ASR	96.82	11.44	96.88	96.57	65.98	11.20	21.55	11.07	11.35	13.61	10.99
GRU (BadWord)	ACC	86.93	86.86	86.97	86.83	71.59	86.90	76.38	85.72	86.69	84.76	86.39
ASR	94.27	81.14	93.98	95.10	71.11	95.16	92.69	71.27	63.10	44.46	11.70
GRU (BadSent)	ACC	87.09	86.95	86.95	86.93	71.55	86.97	76.67	86.23	86.74	84.63	86.47
ASR	99.07	98.39	98.31	99.13	99.90	98.90	96.73	87.82	85.49	42.14	13.87
GRU (Hidden)	ACC	83.05	83.42	82.70	83.02	70.85	83.28	73.14	82.06	84.02	83.85	85.87
ASR	53.96	47.27	54.78	54.26	63.77	56.90	85.91	46.90	35.64	32.43	18.10
LSTM (EP)	ACC	84.42	84.37	84.75	84.84	70.55	84.38	77.46	84.09	84.00	83.02	84.55
ASR	99.32	14.90	99.39	99.26	81.38	16.79	23.67	15.03	13.88	15.06	13.90
LSTM (BadWord)	ACC	84.34	84.16	84.30	84.32	70.46	84.31	75.73	84.22	84.12	83.47	84.69
ASR	98.49	95.48	98.06	98.55	87.17	98.87	96.01	83.93	86.06	42.83	17.65
LSTM (BadSent)	ACC	84.44	84.16	84.51	84.55	70.29	84.41	76.27	83.78	84.09	83.54	84.75
ASR	99.15	98.77	99.34	99.27	100.0	99.44	97.24	86.96	96.41	40.66	34.43
LSTM (Hidden)	ACC	80.76	81.72	81.40	81.30	69.74	81.65	73.25	81.07	82.20	81.58	83.56
ASR	65.08	59.21	63.83	65.62	70.73	65.42	85.14	85.43	48.71	33.75	22.45
CNN (EP)	ACC	87.01	86.52	86.96	86.97	76.51	86.99	80.31	86.00	86.28	85.53	86.57
ASR	96.80	11.76	96.97	96.68	41.92	11.03	19.97	11.72	11.03	13.60	11.49
CNN (BadWord)	ACC	87.09	86.30	87.09	87.12	79.09	86.89	78.18	86.29	86.46	85.44	86.78
ASR	81.46	66.65	81.69	82.82	24.32	83.19	100.0	96.41	52.36	14.82	16.85
CNN (BadSent)	ACC	87.10	86.43	87.10	87.10	79.10	86.97	77.87	86.17	86.50	85.34	86.70
ASR	99.30	97.81	99.17	99.23	49.12	99.54	99.90	99.92	98.41	62.46	36.61
CNN (Hidden)	ACC	84.07	84.68	84.03	84.11	74.76	84.26	76.22	83.95	85.65	85.20	85.41
ASR	56.32	52.55	56.29	55.58	36.33	57.33	80.22	55.15	41.16	22.97	26.08
Table 9: Results under non-IID or multiple attacker cases.
Settings	Metric	FedAvg	Dim-Krum	Fed-FA
IID	ACC	85.28	84.27	85.70
Attackers=1	ASR	86.67	31.56	19.51
Dirichlet	ACC	83.41	77.67	79.48
ASR	85.10	66.63	52.41
Attackers=2	ACC	84.91	83.94	85.42
ASR	94.45	68.22	36.18
Attackers=3	ACC	84.90	83.07	84.75
ASR	96.70	71.92	49.62
memory and adaptive noise mechanisms. In the adaptive noise mechanism, since RNN models are sensitive to noises on parameters [58], we choose  $\lambda = 2$ .
Non-discarding aggregations include Median [3, 50], FoolsGold [12], RFA [31], CRFL [43], and Residual (Residual-based defense) [10] algorithms. In CRFL, the noises on each dimension obey IID Gaussian distributions  $N(0,\sigma^2)$  where  $\sigma = 0.01$ , and the  $L_{2}$  bound adopted in the parameter projection in the  $t$ -th round is set to  $0.05t + 2$ . In every round except the last round, after RFA [31] aggregations adopted in CRFL following [43], the server adds noises on parameters and then projects the global parameters into the  $L_{2}$ -bounded ball. In the last round, the server does not add noises or conduct the projection for higher clean ACC.
C Supplementary experimental results
We report detailed results and further visualizations in this section.
C.1 Detailed results
Detailed results of aggregation algorithms on each model and each attack are reported in Table 8. Results here are averaged on four datasets. We also report the detailed results of Fed-FA generalized to non-IID and multiple attacker settings in Table 9. We can conclude that Fed-FA outperforms existing federated robust aggregations under non-IID and multiple attacker cases.
Stability and standard deviations of results. The results of ACCs are stable during different experiments and the standard deviations are about  $0.1\% \sim 0.5\%$ . The ASRs vary a lot when we enumerate the malicious client from the 1-st to the 10-th client and the standard deviations are about  $10\% \sim 20\%$ , because ASRs vary a lot between the cases of attacking successfully and failing to attack. However, the averaged ASRs are stable between different random seeds and the standard deviations are about  $1\% \sim 2\%$ . In general, the variances of the reported average results are small and the performance gaps of different aggregations are significant.
C.2 Further visualizations
We visualize the ASRs of discarding aggregations in 10 rounds. Visualizations on GRU model are shown in Fig. 4, visualizations on LSTM model are shown in Fig. 5, and visualizations on CNN model are shown in Fig. 6. Results here are averaged on four datasets.
(a) Average ASRs under EP attack.
(b) Average ASRs under BadWord attack.
(c) Legend.
(d) Average ASRs under BadSent attack.
(e) Average ASRs under Hidden attack.
(f) Legend.
Figure 4: ASRs under different attacks in 10 rounds on GRU.
(f) Legend.
Figure 5: ASRs under different attacks in 10 rounds on LSTM.
(b) Average ASRs under BadWord attack.
(e) Average ASRs under Hidden attack.
(f) Legend.
Figure 6: ASRs under different attacks in 10 rounds on CNN.
Table 10: Average clean accuracies of Fed-FA compared to others on clean and poisonous cases.
Case	FedAvg	Non-discarding Aggregations	Discarding Aggregations (includes Fed-FA)
Median FoolsGold	RFA	CRFL	Residual	Krum M-Krum	Bulyan	Dim-Krum	Fed-FA
Clean	86.13	85.92	86.01	85.72	73.59	86.25	79.51	85.46	85.77	84.53
Poisonous	85.28	85.61	85.32	85.35	72.96	85.34	76.71	84.66	85.29	84.27
D Further analyses
In this section, we conduct further experiments to analyze the influence of false positives in malicious client detection, the randomness of the synthetic dataset, and the robustness of Fed-FA to distributed backdoor attacks.
D.1 Influence of false positives
False positives in discarding aggregations. Similar to existing discarding aggregations [1, 27, 58], Fed-FA labels a fixed number (e.g.  $\lfloor \frac{n - 1}{2} \rfloor$  in Fed-FA) of  $n$  clients as malicious clients and discards them instead of trying to distinguish clean and malicious clients and only discarding poisonous updates. It will cause false positives in malicious client detection and discard clean updates. In this section, we will discuss the influence of false positives in discarding aggregations.
False positives have weak impacts on the clean performance. Theoretically, the convergence is guaranteed in Theorem 3. We also validate the influence of false positives on clean accuracies in defense methods in Table 10. Even in the case that all clients are clean, the average accuracy decreases from 86.13 of FedAvg to 85.73 of Fed-FA and Fed-FA only has a performance decrease of about 0.40, which is much lower than other discarding aggregations. When there are malicious clients, Fed-FA has better clean performance than both FedAvg and other defenses. Besides, poisonous updates will also harm the learning, while Fed-FA will not harm the clean accuracy since it discards poisonous updates. To conclude, with our proposed Fed-FA, false positives have weak impacts on clean performance.
Discarding a fixed number of clients can act as a strong defense. Discarding aggregations are stronger baselines than non-discarding aggregations. In discarding aggregations, false positives have weak impacts on the clean performance, but the false negatives may poison the global model and fail the federated backdoor defense. Although discarding a fixed number of clients will cause false positives in malicious client detection, it can still act as a strong defense with little performance loss. Since the task of defending against federated language backdoors itself is difficult, we recommend discarding a fixed number of clients instead of distinguishing clean and malicious clients and only discarding poisonous updates. Our recommended method has a similar discarding protocol as both Fed-FA and other existing discarding aggregations [1, 27, 58].
The proposed detection variants of discarding aggregations. However, [6] argue that the influence of false positives is also crucial, especially for the cases when the backdoor defense is easier and the clean accuracy is crucial. Therefore, we also design detection variants for discarding aggregations for these cases when lower false positives are also important. In variants, we label the clients with distances or indicators (normalized to zero mean and one standard deviation) higher than a threshold as malicious clients. The threshold is tuned on the validation set for each task and defense. We validate the FAR (false acceptance rate), FRR (false rejection rate), P (precision), R (recall), F (F-1 score), ACC (accuracy), and MR (mean rank) on malicious client detection tasks of detection variants of discarding aggregations. The definitions of these indicators are:
$$
\mathrm {F A R} = \text {T h e p r o b a b i l i t y w h e n t h e b e n i g n c l i e s i s r e g a d e d a s a t r o j a n e d c l i e n t ,} \tag {71}
$$
$$
\mathrm {F R R} = \text {T h e p r o b a b i l i t y t h a t t h e t r o j a n e d c l i e n t i s r e c o n g i z e d a s t h e b e n i g n c l i e n t}, \tag {72}
$$
$$
P = \frac {\text {N u m b e r o f t r u e p r e d i c t e d m a l i c i o u s c l i e n t s}}{\text {N u m b e r o f t o t a l p r e d i c t e d m a l i c i o u s c l i e n t s}}, \tag {73}
$$
$$
R = \frac {\text {N u m b e r o f t r u e p r e d i c t e d m a l i c i o u s c l i e n t s}}{\text {N u m b e r o f t o t a l t r u e m a l i c i o u s c l i e n t s}}, \tag {74}
$$
$$
F = \frac {2 P R}{P + R}, \tag {75}
$$
$$
\mathrm {A C C} = \text {T h e p r o b a b i l i t y t h a t a c l i e n t i s r e c o n g i z e d c o r r e l t l y ,} \tag {76}
$$
$$
\mathrm {M R} = \text {M e a n r a n k s o f t r u e m a l i c i e s c l i e n t s i n t h e c l i e n s ^ {\prime}} \tag {77}
$$
Table 11: Performance of detection variants of discarding aggregations. Variant denotes discarding according to the threshold, otherwise discarding  $\lfloor \frac{n - 1}{2} \rfloor$  clients. Lower MRs are better. Euclidean denotes Fed-FA with the Euclidean indicator. The best results are in bold.
Method	FAR%	FRR%	P%	R%	F%	ACC%	MR
Random	40.10	60.33	22.31	39.67	28.56	55.35	5.50
Random (Variant)	33.94	67.78	21.61	32.22	25.87	58.45	5.50
M-Krum	46.61	98.33	1.03	1.67	1.27	41.75	8.43
M-Krum (Variant)	51.13	97.78	1.25	2.22	1.60	38.38	8.43
Euclidean	51.29	97.22	1.55	2.78	1.99	38.38	8.49
Euclidean (Variant)	47.26	98.33	1.01	1.67	1.26	41.25	8.49
Fed-FA	32.26	33.33	37.50	66.67	48.00	67.50	3.78
Fed-FA (Variant)	10.16	70.00	46.15	30.00	36.36	76.38	3.78
Table 12: Influence of the synthetic dataset.
Method	ACC	ASR
FedAvg	85.28±0.81	86.67±7.49
Fed-FA with labeled dataset	85.77±0.12	20.06±1.25
Fed-FA	85.70±0.18	19.51±1.90
Table 13: Robustness to distributed attacks.
Attack	ACC	ASR
FedAvg	86.97	100.0
Dim-Krum	86.43	15.12
Fed-FA	86.57	13.45
Detection performance of discarding aggregations and variants. We report detection performance under multiple client numbers (0-4 of 10 clients) in Table 11. The detection variant of Fed-FA has lower false positives and higher false negatives, namely the detection variant tends to miss malicious updates and not to discard clean updates compared to Fed-FA. To conclude, there is a tradeoff between FAR and FRR and the detection variant of Fed-FA can be utilized in cases where false positive rates are crucial. Both Fed-FA and its variant have satisfying detection performance compared to random baseline and other discarding aggregations.
D.2 The randomness of the synthetic dataset
As reported in Table 12, we randomly sample different texts as the labeled and synthetic datasets for different runs and the standard deviations of Fed-FA are low, which means that the randomness of the synthetic dataset does not influence the results much since Fed-FA only needs the Hessian scales instead of accurate Hessians.
D.3 The robustness to distributed backdoor attacks
In addition to regularizers, we can adopt distributed backdoor attacks [44] to make malicious updates more stealthy as adaptive attacks. As shown in Table 13, we also validate that Fed-FA is robust to them.