SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Deepsight [20]: Deepsight adopts three different distance matrices to measure the distances between each client. Deepsight then clusters clients according to different distance matrices and only accepts clients that are in the same cluster across different matrices. The first distance matrix is smaller when the updates in the last layer from clients are similar. The second distance matrix is the L2 distance between the last layer's weight across each client. The third distance matrix is the L2 distance between the outputs of two local models given a batch of randomly generated input images. Deepsight adopts DBSCAN [48] to cluster selected clients. Finally, clusters including potentially malicious clients that have a larger distance from other clusters will be excluded. In our experiments, we set the batch size of randomly generated inputs to 256.
Bulyan [23]: Bulyan first excludes potentially malicious clients from all selected clients and then uses the coordinate-wise median value of updates from remaining clients to update the global model. In the first step,  $2f$  clients with the highest pairwise Euclidean distances are excluded. In the second step, Bulyan picks  $M - 4f$  clients from the remaining  $M - 2f$  clients that are closest to the median by coordinate. In our experiments, we set  $f = 2$ .
FedDF [24]: FedDF uses the mean output of all client models as the supervisory signal to distill the next round global model. In particular, FedDF firstly aggregates all selected clients (the same as FedAvg) to obtain a teacher model. Then the server trains the global model to minimize the Kullback Leibler divergence between the logits of the global and teacher model on a set of unlabeled inputs. In our experiments, the learning rate for updating the global model is 0.002 and we train the global model for one epoch at each FL communication round.
FedRAD [25]: FedRAD is an extension of FedDF, which assigns a weight to each client model based on their median scores. These scores indicate the frequency with which the prediction of the client model becomes the median value of predictions from all client models. FedRAD then utilizes weighted model aggregation to produce the next round global model. In our experiments, we also update the global model with a learning rate of 0.002 for one epoch at each FL communication round.
Krum [21]: Krum selects clients that have the smallest L2 distances to other clients. Only the clients selected by Krum will be used to update the global model. Since Kurm drops most updates from clients, it can achieve strong robustness. However, Krum also affects the accuracy of the model.
Table 2: A3FL maintains the utility of global models on TinyImageNet.
Defense	FedAvg	NC	RLR	Median	DSight	Bulyan	Krum	SFed	CRFL	DP	FedDF	FedRAD
ACC(%)	55.45	55.31	55.34	17.12	53.71	11.19	42.87	57.39	53.58	53.38	25.31	23.12
BAC(%)	55.25	54.98	55.28	20.92	53.44	7.33	42.35	57.08	53.45	53.17	24.90	22.57
(d) FedDF
(e) Bulyan
(f) SparseFed
Figure 8: Comparing performances of different attacks on CIFAR-10.
SparseFed [26]: SparseFed is proposed to mitigate model poisoning attacks in FL. SparseFed aggregates client updates normally but only updates the top-k highest magnitude elements. It is inspired by that attackers commonly move in distinct directions from the majority of clean clients. Therefore the top-k highest magnitude elements involve less poisoned updates from attackers. In our experiments, we update the top-95% highest magnitude elements.
FLAME [27]: FLAME adopts dynamic clustering, adaptive clipping, and adaptive noising to exclude potentially malicious clients. Following the settings for image classification [27], we set  $\epsilon = 3705$  and  $\delta = 0.001$  controlling the strength of adaptive noising.
B Additional Experimental Results
B.1 A3FL maintains the model utility
We show the accuracy of the global model on TinyImagenet when the attacker presents (BAD) or not (ACC) in Table 2. In particular, we record the accuracy on clean tasks when no attackers are involved to obtain the accuracy (ACC). We further record the accuracy on clean tasks when there are 20 compromised clients among all clients to obtain the backdoor accuracy (BAC). We set the number of compromised clients  $P$  to 20 since more compromised clients are likely to result in a higher decrease in clean accuracy. Therefore if A3FL can maintain the model utility even with 20 compromised clients, we can conclude that A3FL is highly stealthy. Note that we use the mean value of ACC and BAC in the attack window (between the 1,900th communication round and the 2,000th communication round) to verify the utility of global models since the server continuously updates the global model. Therefore, using the mean accuracy as the measurement standard can accurately reflect the impact of attacks on the model utility, and eliminate randomness.
As shown in Table 2, the accuracy of the global model does not degrade much when attackers are presented. This indicates that A3FL preserves the accuracy of global models so it is stealthy enough to not be discovered. The differences between ACCs and BACs are within  $0.5\%$  in most cases. The highest drop in clean accuracy is observed when the defense mechanism is Bulyan. However, Bulyan significantly degrades the model's accuracy to only  $11.19\%$ . The low accuracy indicates that the
(j) FedRAD
(k) Bulyan
(1) SparseFed
Figure 9: Comparing performances of different attacks on TinyImageNet.
model is highly random, so even though A3FL causes the model's accuracy to drop to  $7.33\%$ , we cannot solely conclude that A3FL will reduce the model utility. In general, A3FL does not influence the global model utility. We also observe a similar phenomenon on CIFAR-10, as shown in Table 1.
B.2 A3FL achieves higher ASRs
We compare the performance of attacks on CIFAR-10 against defenses that are not designed for backdoor attacks in Figure 8. Observe that A3FL achieves the highest ASR under most settings. When the defense is Median, A3FL is the only attack that can achieve high ASR (over  $80\%$ ). We further show the attacker performance of A3FL on TinyImagenet in Figure 9 and we can observe a similar phenomenon.
B.3 A3FL has a longer lifespan
In Figure 10, we show that A3FL has a significantly longer lifespan than other baselines with different defenses applied. For instance, when the defense is RobustLR, A3FL can still achieve an ASR of
(a) FedAvg,  $\mathrm{P} = 5$
(c) Deepsight,  $\mathrm{P} = 5$
(b) RobustLR,  $\mathrm{P} = 5$
(d) CRFL,  $\mathrm{P} = 20$
(a) ACC
Figure 11: Attack performances against CRFL with different  $\sigma$ .
Figure 10: A3FL has a longer lifespan.
(b) ASR
(a) Norm Clipping
Figure 12: Attack performances under different Dirichlet concentration parameters.
$62.37\%$  at 1000 rounds after the attack ends. In contrast, the attack success rates of other attacks drop below  $50\%$  in less than 150 rounds. Note that when we use CRFL, we set the number of compromised clients  $P = 20$  since when there are only 5 compromised clients, all attacks except A3FL failed to achieve high ASR (see Figure 2).
B.4 Ablation study on component importance
We study the effectiveness of A3FL with or without the adversarial adaptation loss to test the effectiveness of components under FedAvg with  $P = 20$  compromised clients among all clients. As shown in Table 3, the adversarial adaptation loss can effectively improve the durability of A3FL. Observe that A3FL can achieve an ASR of  $97.66\%$  at 500 communication rounds after the attack and  $86.65\%$  at 1,000 communication rounds after the attack. In comparison, A3FL without the adversarial adaptation loss exhibits ASRs that are  $4.31\%$  and  $15.64\%$  lower than A3FL at these two points.
Table 3: Effect of different components in A3FL.
ASR(%) ↓ Rounds after attack →	0	500	1000
A3FL without adversarial adaptation	100.0	93.35	69.01
A3FL	100.0	97.66	84.65
(b) Krum
(c) CRFL
Figure 13: Attack performances when the attack starts at the first communication round.
(a) A3FL
(c) CerP
Figure 14: ASRs against Krum.
B.5 Impact of  $\sigma$  on CRFL Effectiveness
Figure 11 shows the ACC and ASR when applying CRFL with different  $\sigma$ . Observe that as the  $\sigma$  increases, CRFL can achieve better robustness, indicated by lower ASR. However, the ACC of the global model also drops from  $90.25\%$  to  $67.33\%$  rapidly, as  $\sigma$  increases from 0.001 to 0.01, which is unacceptable. Furthermore, when there are more compromised clients, A3FL can still achieve high ASR even with a large  $\sigma = 0.01$ . We can thus conclude that CRFL can not sufficiently mitigate A3FL with different  $\sigma$ .
B.6 Impact of Data Heterogeneity
We adjust the Dirichlet concentration parameter  $h = 0.09, 0.9, 9$  to study whether data heterogeneity influences the performance of A3FL. As shown in Figure 12, A3FL can achieve high ASR regardless of different  $h$ . When the defense is Norm Clipping and  $h = 0.09$ , A3FL achieves lower ASR. This can be explained by that a smaller  $h$  indicates a more non-i.i.d data distribution. Therefore, the local training set held by the attacker is far from the global data distribution, which increases the difficulty of injecting the backdoor. However, the attack success rate is still high (over  $60\%$ ) and quickly increases as the number of compromised clients increases.
B.7 The impact of attack window
We evaluate A3FL against baseline attacks when the attack window starts at the first communication round and ends at the 100th communication round. As shown in Figure 13, A3FL can still remarkably outperform other baseline attacks. For instance, when the defense mechanism is Norm Clipping and there are 5 compromised clients, the gaps of ASR between A3FL and other baseline attacks are at least  $62.4\%$ , which is even larger than the gap under default settings. However, we also observe that when the attack starts from the first communication round and there are only a few compromised clients (1 or 2), ASRs of all attacks decrease in comparison to ASRs under default settings. This can be explained by that at the beginning of the training process, the global model changes a lot so the backdoor is easily erased when there are only a few compromised clients. We also observe that when the attack starts from scratch, all attacks fail to have a satisfying lifespan since the model is far from convergence at the 100-th communication round. Therefore, our evaluation of lifespan is conducted following the configuration provided by Neurotoxin [12].
(a) ASR
(b) Trigger Size
Figure 15: Attack performances of DBA using original trigger design. (a) DBA-bar denotes DBA attack with the original trigger design proposed in [11], in which the trigger consists of four white bars. While DBA denotes the DBA attack with the trigger designed as a red square. (b) Trigger size refers to the length of each white bar. (c) Trigger gap  $\{\mathrm{Gap}_x, \mathrm{Gap}_y\}$  refers to the distance between each bar. (d) Trigger location  $\{\mathrm{Shift}_x, \mathrm{Shift}_y\}$  represents the distance from the trigger to the edge of the image.
(c) Trigger Gap
(d) Trigger Location
B.8 Case study on Krum
We perform a case study on Krum to gain insight into why A3FL outperforms other baselines. In Figure 14 we record the ASRs and put a ". ." notation on the line if Krum selects an attacker-compromised client at that round. Recall that Krum selects one client at each round and only uses the selected client updates to update the global model. Therefore, the chance that a compromised client is selected by the server increases if the backdoor is more stealthy. We have the following observations: 1) fixed-trigger attacks are more frequently selected by the server, while trigger-optimization attacks are selected twice only; 2) fixed-trigger attacks achieve lower ASR even if selected by the server. However, observe that once selected, A3FL quickly achieve  $100\%$  ASR, which is because A3FL can maintain higher ASR when transferred to the global model as stated above. A3FL is also durable after being selected, leading to a higher ASR at the end of the attack. In comparison, F3BA is selected on the 26th round and achieves  $\approx 80\%$  ASR. But the ASR quickly drops after that. CerP is also selected twice, but it cannot achieve as high ASR as A3FL and F3BA do, which is caused by the strict regularization on the local model bias. In addition, the ASR of CerP also drops quickly when the compromised clients are not selected by the server.
B.9 The impact of DBA trigger pattern
In our experiments, we set the trigger pattern of DBA to be a red square at the upper left corner. However, in [11], the trigger is designed as four white lines. We, therefore, discuss the performance of DBA when using the original trigger design. The original trigger design of DBA is determined by three hyperparameters: trigger size (TS), trigger gap (TG), and trigger location (TL). In particular, the trigger gap consists of a horizontal gap  $(\mathrm{Gap}_x)$  and a vertical gap  $(\mathrm{Gap}_y)$ . The trigger location consists of a horizontal shift  $(\mathrm{Shift}_x)$  and a vertical shift  $(\mathrm{Shift}_y)$ . We explain these hyperparameters in Figure 15b, 15c, and 15d respectively. Following the default settings in [11], we set  $\{\mathrm{TS}, \mathrm{TG}, \mathrm{TL}\} = \{4, (6, 6), (0, 0)\}$ .
We compare the attack performance of DBA and DBA-bar (DBA with original trigger design) in Figure 15a. Observe that with the original trigger design, DBA-bar achieves an even lower ASR. This phenomenon supports that the default trigger design in our experiments does not degrade the attack performance of DBA. In contrast, DBA can even achieve a higher ASR without the original trigger design.
B.10 Transferability of A3FL under different settings
We further evaluate the attack performance of A3FL on more datasets, defenses, and model architectures. In particular, we record the performance of A3FL on FLAME [27] in Figure 16a. We also evaluate A3FL on other model architectures [49] and datasets [43] in Figure 16b,16c. Observe that A3FL can always outperform baseline attacks under different settings.
(a) FLAME
(b) VGG-16
(c) FEMNIST
Figure 16: Attack performances of A3FL under different settings.
B.11 Discussion on Ethical Implications
We admit that the discovery of a new backdoor attack in federated learning results in potential ethical implications. A3FL mainly focuses on image classification, which can be deployed in security-sensitive applications such as human face recognition. However, discovering a new backdoor attack and mitigating its threats is necessary to improve the robustness of the federated learning paradigm. Safeguarding the integrity and ethical dimensions of federated learning is crucial to ensure the best interests of individuals and society. We believe that future work can eliminate the threat of proposed attacks, and it is important to focus research on FL defenses.