Federated Behavioural Planes: Explaining the Evolution of Client Behaviour in Federated Learning
Authors: Dario Fenoglio
Gabriele Dominici
Pietro Barbiero
Alberto Tonda
Martin Gjoreski
Marc Langheinrich
Venue: NeurIPS 2024 poster (2024)
Keywords: federated learning
explainable AI
counterfactuals
secure aggregation
Primary area: privacy
TL;DR: We introduce Federated Behavioural Planes (FBPs), a novel method to analyse, visualise, and explain the dynamics of FL systems, showing how clients behave under two different lenses: predictive performance and decision-making processes.

Abstract: Federated Learning (FL), a privacy-aware approach in distributed deep learning environments, enables many clients to collaboratively train a model without sharing sensitive data, thereby reducing privacy risks. However, enabling human trust and control over FL systems requires understanding the evolving behaviour of clients, whether beneficial or detrimental for the training, which still represents a key challenge in the current literature. To address this challenge, we introduce Federated Behavioural Planes (FBPs), a novel method to analyse, visualise, and explain the dynamics of FL systems, showing how clients behave under two different lenses: predictive performance (error behavioural space) and decision-making processes (counterfactual behavioural space). Our experiments demonstrate that FBPs provide informative trajectories describing the evolving states of clients and their contributions to the global model, thereby enabling the identification of clusters of clients with similar behaviours. Leveraging the patterns identified by FBPs, we propose a robust aggregation technique named Federated Behavioural Shields to detect malicious or noisy client models, thereby enhancing security and surpassing the efficacy of existing state-of-the-art FL defense mechanisms. Our code is publicly available on GitHub.

Explaining the Evolution of Client Behaviour in Federated Learning
Federated Behavioural Planes:
Dario Fenoglio
Gabriele Dominici Università della Svizzera italiana
Università della Svizzera italiana Lugano, Switzerland
Lugano, Switzerland gabriele www.niciuusi...oommmmmmmmmm.mmim ch
dario. d fwn.glnooglloooooo iouusi. ch
Pietro Barbiero Università della Svizzera italiana Lugano, Switzerland pietro. barbieroousi.................... ch
Alberto Tonda INRAE
Paris, France alberto. tonda@inras fr
Martin Gjoreski Università della Svizzera italiana Lugano, Switzerland
Marc Langheinrich
Università della Svizzera italiana
Lugano, Switzerland
martin. pertin.gjorent www.ggjorrsiiooo Backkiuusi..o
marc. Lar..laangheiinicchhuusssi ch
Abstract
Federated Learning (FL), a privacy-aware approach in distributed deep learning environments, enables many clients to collaboratively train a model without sharing sensitive data, thereby reducing privacy risks. However, enabling human trust and control over FL systems requires understanding the evolving behaviour of clients, whether beneficial or detrimental for the training, which still represents a key challenge in the current literature. To address this challenge, we introduce Federated Behavioural Planes (FBPs), a novel method analyse, visualise, and explain the dynamics of FL. systems, showing how clients behave under two different lenses: predictive performance (error behavioural space) and decision- making processes (counterfactual behavioural space). Our experiments demonstrate that FBPs provide informative trajectories describing evolving states clients and their contributions to the global model, thereby enabling the identification of clusters of clients with similar behaviours. Leveraging the patterns identified by FBPs, propose robust aggregation technique named Federated Behavioural Shields to detect malicious or noisy client models, thereby enhancing security and surpassing the efficacy of existing state-of-the-art FL defense mechanisms. Our code is publicly available on GitHub'.
Introduction
Federated Learning (FL), privacy-aware deep learning (DL) approach in distributed environments,
a dynamic system where many clients collaborate train model without sharing sensitive data,
thus mitigating privacy risks [1, 2]. Analyzing the behaviour of FL systems is crucial to detect anomalies such as distribution shifts [3, 4]. biased data [5], or adversarial clients 11-99-wccch may compromise the global model's 's predictive performance and introduce biases into its decision-
Author contributions are detailed in the Acknowledgements section. httpp:://githu..comm////// www.daaioooeogglloooooooooooo
38th Conference on Neural Information Processing Systems (NcurIPS 2024).

---

Courtefaccuaas Behavioural Piane
smal-MNIST Dataet
Cller1
diuster
Enora Behavibural Plane
The Sonee
Server
Miasker
sttnck
DE
Type
Tel. M.PPrrcppllCn Camp.
Hoam
Trme
Oun
Medias
RFA
Reg fecuvry
Figure 1:
The Federated Behavioural Planes framework enables the visualization of client behaviour in FL from two perspectives: predictive performance (Error Behavioural Plane) and decision decision-making processes (Counterfactuals Behavioural Plane). It highlights client trajectories and similarities, offering insights into client interactions and supporting the introduction of a new and effective robust aggregation mechanism with performance that surpasses state-of-the-an baselines.
making process. Various strategies have been developed to detect FL. anomalies [10-15]. However, existing techniques are not designed to track, visualise, and explain how client behaviours affect the performance of the global model, thus limiting human trust and control on FL. dynamics.
Various studies have analysed models' behaviour terms of predictive performance and decision- making processes independently. Predictive performance behaviour is primarily investigated in the context of non-linear optimisation using behavioural spaces [16-18]. This technique allows the visualisation of the predictive diversity of a set of regression models by considering the vector of errors that each model produces on set of samples. The decision- process is studied mainly in explainable AI (XAI) research using, for example, counterfactual explanations [19]. Counterfactuals can be used identify relevant input features used by model to make predictions, thus describing the position and orientation of the model's 's decision boundaries. However, these techniques have not yet been applied to FL. systems, resulting lack of insight into how the behaviour of individual clients affects the overall model's accuracy and decision decision-making making capabilities, leading to inefficiency in the training process.
To bridge this gap, we introduce Federated Behavioural Planes (FBPs), a method designed to visualise, explain, and give insights into the dynamies of FL systems. Our key innovation involves the creation of two behavioural planes for FL clients: one to highlight their predictive diversity and another emphasise their decision- decision-making process diversity via counterfactuals. Building on the client behaviour information provided by FBPs, to show their practical utility, we propose Federated Behavioural Shields, robust aggregation mechanism that enhances security against malicious or noisy clients by accurately weighting the client models according to their constructive contributions during training. The results of our experiments demonstrate that: (i) counterfactual generators jointly trained with FL systems produce valid and client-specific counterfactual explanations which effectively describe clients' decision-makinn diversity; (ii) FBPs facilitate the identification of clusters of clients with similar behaviours (e.g., normal vs. outlier clients), allowing for tracking of their trajectories during the entire training; (iii) Federated Behavioural Shields surpasses existing state-of- the-art defense mechanisms, demonstrating that the information contained in FBPs provides valuable descriptors of client behaviour.
2 Background
Federated learning. FL systems [20, 3] involve a network of K E N clients, coordinated by central server, which collaboratively train a DL model. Each client possesses a local and private dataset characterised by a set E features (R) ( C R² and a set of u E A class labels (4)) E (k) C {o, 11 In each training round E N,, each client trains model f (k) + (k) on local data to maximise the likelihood ((((k) [,((), (x))),)())), Once trained, clients send their local model's parameters (1) E R9, 9 E N to central server which aggregates these parameters using permutation permuuation-invariint aggregation R R2 (such as the mean median). The server then

---

sends the aggregated model parameters 0(t + 1) back to the clients to start a new training round:
(local training)
g(h) (1) = arg max (() x ( (Lh)(1)
(aggregation)
A(t + 1)
(()
(2)
While FL is an efficient process to safeguard privacy, the inherent lack of direct control over each
individual client makes FL systems particularly vulnerable various poisoning attacks [6-9, 21, 22, 14, 23-26]. These can be categorised into model and data poisoning attacks. Model poisoning involves altering gradients on compromised devices before transmission the server [9, 14, 23, 24], while data poisoning indirectly manipulates gradients by tampering with training datasets on malicious devices [25, 27, 21].
Counterfactual explanations. Counterfactual explanations [19] describe model's decision- making process by identifying minimal and plausible changes to an observed input's features that lead to desired model prediction. In explainable AI, finding counterfactual explanations is framed as an optimisation problem where the objective is to identify, for each sample æ, the nearest data point z' such that the classifier f00 ,x) assigns desired class label y/:
arg min || x'll s.t. f(0,x')
As result, the variations in the input's features between æ and x' offer actionable insights into how the model's decisions can be altered, highlighting the most important features.
Semantic and behavioural spaces. Semantic spaces [16, 17] represent the semantics of a model by considering the error vector e = [c(f(0,x:)) (aaa::g))]:an) 34)li=1....., that a model produces on a set of E W samples, where could be the Mean Squared Error, for instance. Given a set of K models, the semantic space contains n-dimensional data points [el...., ex] E Rxx,, Behavioural spaces [18] summarise semantic spaces into lower-dimensiooal spaces applying transformation va-mm R Rm with m < n (typically m for most applications) i.e., V/-+m (le1,...,
3 Federated Behavioural Planes
Problem definition: Given an FL. system composed of set of K clients with local data (2x(k), y (A) and models f((k)), we aim to analyse the evolution of the system to understand how clients impact the global model's predictive performance and decision-making over time. In Section 3.1, we introduce what drives our method and formalise the problem. Section 3.2, we introduce Federated Behavioural Planes (FBPs), describing its components in more detail: the Error Behavioural Plane (Section 3.3) and the Counterfactual Behavioural Plane (Section 3.4). Finally, in Section 3.5, we introduce Federated Behavioural Shiclds, a new robust aggregation mechanism to enhance security in FL. systems, showing a practical application of FBPs.
3.1 Dynamic behaviour of federated learning
FL is a dynamic process transitioning from a state where all participating clients behave randomly to state where the behaviour of participating entities becomes coherent (the parameters of client models tend converge to values which lead to high model accuracy). Thus, FL systems can be effectively analysed through the lens of dynamical systems using tools traditionally employed for such studies, such as differential equations. Defining b(0(#)) as the evolving state of the behaviour of the client k at time I, we introduce two primary forces that influence b(((())) ): g((()) (x()), (*)), the local training dynamics, which drives the client towards its local optimum by leveraging information from the local dataset (x), (y())); and (@tek g(*) b(8(k)), correction term that periodically
aligns b((())) with the aggregated state of all clients within the federated system. These dynamics can be encapsulated in the following differential equation?, which describes how client behaviours
evolve during training and are influenced by internal forces within the FL system:
For simplicity, we omit the dependency the time variable in notation for variables b and 0.

---

db (0(4))
dt = (Bl 22 (0))),,200000))) ,())) (1 81) + b
( + g)(c) WEE
b ((0(e))
8r
(4)
Here, dr is characterised by periodic Dirac delta function, defined as dy Er= d(t r. T),
which triggers instantaneous adjustments at intervals determined by period T.
3.2 Federated Behavioural Planes (FBPs)
Instead of finding a general analytical solution for Equation 4 (which is not trivial and requires strong assumptions on its components), we aim to empirically analyse the phase space of a FL system by considering different descriptors of client behaviours. More specifically, we focus investigating (i) the predictive performance, evaluating how well the model solving the task, and (ii) the decision-making process, it contains information on how the model is solving the task. During each round, client behaviours are assessed through their respective models on the server, utilising server-owned dataset reserved for this evaluation phase. This methodology aligns with existing protocols [9. 14, 28-32J. Each client behaviour is visualised through a -vo-dimensional plane: Error Behavioural Plane representing predictive performance and the Counterfactual Behavioural Plane to illustrate decision-making processes, collectively referred to as Federated Behavioural Planes. However, this framework offers general approach visualise and monitor different descriptors of the client behaviours simultaneously, which can be customised through specific functions, enabling the creation of additional planes.
3.3 Error Behavioural Plane (EBP)
To comprehensively evaluate each model from the predictive performance point of view, we analyse the errors made by the model on all samples, rather than relying solely on simpler aggregate metric, such as loss or error [14]. This approach enables a more detailed examination of the differences the model's performance as observed by Mouret and Clune [18]. Following the methodology proposed by Zhang et al. [33], we first construct a semantic error space for each model and then map it to a reduced space, called Error Behavioural Plane (EBP).
Definition 3.1 (Error Behavioural Plane). Given model J. parametrised with a set of weights (t), related to the client k at round f, a dataset (ze(vere)) y (server)) ), owned by the server, and dimensionality reduction technique 1n-+2, the representation e (() E R² in the EBP of the client k the following:
(1) Un-42 (1)) ((sove))
(server)
yi
It is worth noting that clients, despite having similar accuracy loss, may receive significantly different representations in the EBP if they produce errors distinct subsets of samples. In contrast, clients whose trajectories in the EBP converge over time form clusters representing clients whose predictive performance is similar on the same set of samples. However, clusters and trajectories in the EBP do not explain the gpeision-making process that leads prediction. This information could be used to further distinguish different types of clients and can be analysed using counterfactuals.
3.4 Counterfactual Behavioural Plane (CBP)
The analysis of model's decision Decision-making process is the main research objective of explainable AI. Counterfactual explanations represent one of the most effective techniques as they give insights concerning the position and orientation of decision boundaries. Similar counterfactual explanations indicate models having similar decision boundaries, i.e. models taking decisions using similar decision-making making process. To this end, most differentiable counterfactual generators are trained to model the training data distribution [34, 35]. thus potentially providing insights on non identical distributions in clients" data. To produce these explanations, clients' predictive models should be concurrently trained with counterfactual generator. Additional information on the optimisation objective are provided in Appendix A.2, A.3. To obtain the Counterfactual Behavioural Plane (CBP), we first compute the distances between the counterfactual distribution generated by the server using

---

model of the client & and the distribution of other clients. Then, we apply dimensionality reduction to obtain the CBP.
Definition 3.2 (Counterfactual Behavioural Plane). Given model f with parameters ((4) (D), related the client at round a set of samples ()(erveer E R owned by the server with n samples and features, distance function d: R + R* such as Wasserstein distance, and a dimensionality reduction technique k2, the representation cl (t) R² in the Counterfactual Behavioural Plane
+2,
of client & in an FL settings is the following:
a (1) = arg min || (server) - w''l s.t. (10(0) # $1(00) (1), 2; (server)
(6)
(1)
(1) = Uズ (10)
CBP produces complementary information to EBP as clients which are similar in the CBP might be far away in the EBP (as discussed in Appendix B.4). Furthermore, as the puposse of CBP is to track clients' behaviour rather than explaining the model decision to user, counterfactuals can be generated based on predictive models' embeddings, instead of input features, concealing sensitive information.
3.5 Federated Behavioural Shields FBPs as defence mechanism
FBPs provide descriptors of client behaviours during training, enabling various applications. Notably, trajectories behavioural planes converging over time form clusters representing clients with similar predictive performance and decision-making process. This property be used in practice to identify anomalies, such as malicious clients attempting to compromise FL training. In particular, leveraging this detailed information on client behaviours, propose Federated Behavioural Shields (FBSs), a new class of robust aggregation strategies designed to enhance security in FL without requiring prior knowledge on the attack. This defensive mechanism generates behavioural score in round for client k. denoted as s(k) (t), which is formulated through the composition of multiple scores computed on S behavioural spaces, to guide the aggregation process in creating the next round's global model 0(t + 1), as outlined below:
+) ()) (1) = Il
0(t +
+ s (*) (t)0(()))) kEK
(8)
EiEK Nyes (1))
Specifically, based on the FBPs previously defined, compute these scores as follows:
Serror (t) (6) (t)
(K)
1 (+) (t)1..) S (t)
(k)
ΣieK semor (t)s (t)
Seror
1 E (t)
(t)
The еттог score Sente (t) measures the distance between the client is and the optimal point at the center
(k)
of the plane, while the counterfactual score sef measure the average distance between the distribution of counterfactual generated by client and all the other clients. In addition, considering that honest
clients may occasionally deviate from the norm but generally contribute positively, we introduce a
moving average mechanism to track client behaviours (see Appendix A.2 for details).
Experiments
The preliminary goal of our experiments to assess whether counterfactual generators can provide insights in an FL context without compromising the performance of the predictor. We then visualise FBPs to verify that they can reveal information about the behaviour of various clients through the error and counterfatual behavioural planes (EBP and CBP). Lastly, we analyse the effectiveness of Federated Behavioural Shields as robust aggregation mechanism, demonstrating the utility of the information provided by FBPs. Our experiments aim to answer the following questions:
5

---

Counterfactuals in FL: Does the integration of counterfactual generators impact clients' predictive performance? Do counterfactuals have the same quality in FL compared cen- tralised scenarios? Could counterfactual generators be adapted for each client? Answering these questions is an essential preliminary step to check whether counterfactuals can be used to generate FBPs.
Explaining FL training: Can trajectories in FBPs describe the evolving client behaviours during the training phase? Is possible to visually identify clusters of clients using FBPs?
Leveraging FBPs information: Do FBPs provide sufficient detail to Federated Behavioural Shields enhance the security of the FL training process against security attacks?
This section describes essential information about the experiments. Further details on model con- figuration, training setup, and computational cost are presented in Appendices A.2, A.4, and A.6, respectively.
4.1 Data & task setup
In our experiments, we utilise four datasets: a Synthetic dataset (tabular) we designed to have full control clients' data distributions, and thus test assumptions; the Breast Cancer Wisconsin [36] (tabular); the Diabetes Health Indicator [37] (tabular); small-MNIST [38] (image); and small- CIFAR-10 [39] (image), reducing its size by 76% to increase task difficulty and highlight client differences in performance. For all the experiments with the MRRAI--MIIST--TA dataset, our approach involves generating counterfactuals at non-interpretable internal representation level of the model instead of the input space. To reflect the most realistic cross-silo scenario [40], we distribute the training data among various clients such as data are not independent and identically distributed (IID) [41]. This represents the most challenging scenario to detect malicious clients due to the significant variations even among benign clients. Further details on the datasets and non-IID implementation are provided in Appendix A. Additional experiments analysing setup characteristies such as window length, local epochs, server validation set size, and differences between non-IID and IID scenarios can be found in Appendix B.
4.2 Evaluation
Metrics. To determine the efficacy of counterfactuals generated through end-to-end training in
FL, measure several key metrics: task accuracy (f - higher is better); counterfactual validity (1)
[19], which checks if the counterfactuals' labels align with user-provided labels; proximity (L) [35],
assessing the realism of counterfactuals by their closeness to the training data (distance between the counterfactual and the closest data point in the training set with the same label); and sparsity (L) [19),
which quantifies the changes made to the input generate the counterfactuals (number of features changed between the initial sample and the counterfactual). The latter is quantified using Euclidean distance, as counting the number of changes provides less insight the generated counterfactuals. To evaluate the effectiveness of client-specific adaptation, we analyse the relative change in client proximity between global and client-specific models, expressed (Patohat Procat) Patohat. Finally, we measure the task accuracy (1) of the FL system under different attacks and defenses. All metrics are reported the mean and standard error across five experimental runs with distinct parameter initialization.
Baselines. In our experiments, we compare Federated Behavioural Shields with the following state-of-the-at robust aggregation methods: Median [11], Trimmed-meat [11], Krum [12], and RFA [42]. Further information is provided in the Appendix A.7.
Federated Attacks. We focus on attacks with realistic assumptions and where additional infor- mation outside the typical FL scenario are not available [10, 14, 23]. We test the following attacks: Label-flippigg (Data Poisoning) [26, 43), changes each sample's 's label text of binary classification); Inverted-loss (Data Poisoning) [25], creates an update that maximises the loss on the local dataset; Crafted-noise (Model Poisoning) [44], adds noise /O, 3 (ww")) to the previous global model w², where a (w") is the standard deviation of w² and / is scale factor set to 1. 1.2; and Inverted--oe graddent (Model Poisoning) [24, 45), inverts the gradient derived from the server's previous update, misaligning it with the true gradient. Further details are provided in Appendix A.8.
y (performed in the con-

---

Table 1: Performance comparison of our model, which includes a Predictor and Counterfactual Generator (CF), across various settings: Local Centralised (Local CL), Centralised Learning (CL), Federated Learning (FL), and FI. with only the Predictor a non- ann-IID setting.
Metric
Dataset
Local CL
CL
FL Predictor CF
FL Predictor
Predictor CF
Predictor CF
Accuracy (1)
Diabetes
55.9 0.5%
75.0 :: 0.2%
74.7 = 0.1%
74.2 + 0.1%
Breast Cancer
86,9 0.7%
97.7 0.0%
98,4 0.1%
97.7 + 0.4%
Syathetic
75.0 2.0%
99.4 + 0.2%
99.8 0.1%
99.9 + 0.1%
Validity (1)
Diahetes
87.6 2.6%
99.9 + 0.1%
99.9 0.0%
N/A
Breast Cancer
100.0 + 0.1%
100.0 + 0.0%
100.0 + 0.0%
N/A
Synthetic
97.1 + 1.9%
100.0 + 0.0%
100.0 + 0.0%
N/A
Sparsity ())
Diabctes
4.14
34.5 L.7
37.1 # 1.2
M/A
Breast Cancer
1459 25
1325 20
1448 43
N/A
Synthetic
8.63 + 0.15
6.24 0.22
6.14 0.07
N/A
Proximity
Diahetes
8.91 4: 0.61
5.45 : 0.40
4: 0.44
N/A
Breust Cancer
6.12
70.1 1.8
72.1 + 5.5
N/A
Synthetic
0.142 = 0.026
0,091 + 0.003
0.089 + 0.002
N/A
5 Key Findings & Results
5.1 Counterfactuals in FL
Integrating counterfactual generators in FL optimisation does not compromise predictive performance (Table 1). We compared model accuracy across four settings: two centralised learning (CL) and two FL., using three different datasets under non-IID conditions. For FL experiments, used the traditional FedAvg approach with two variations: predictor-only and predictor with CF generator. The results indicate that our model, which involves the concurrent training of both the predictor and the counterfactual generator, achieves performance comparable to that of the predictor alone in FL. In the context of our model, both Local CL-where each client trains model on its local
FL, implemented across all clients, comply with privacy standards [2]. For Local CL, we report the average accuracy of models trained independently by each client and evaluated on a common test set. However, only FL reaches the performance levels of the CL scenario, which assumes local access to all client data. This result indicates that incorporating counterfactuals during training does not compromise the predictor's performance, thus supporting their beneficial application without adverse effects on performance.
Counterfactuals generated in FL have similar quality to those in CL (Table 1), Unlike Local CL, FL leverages information from all clients, thereby producing counterfactuals that more accurately reflect non-III clients' data, As shown in Table 1, FL. achieves higher validity compared to the Local CL approach. indicating that the FL's counterfactuals better match ground -ruth labels. Additionally, FL exhibits lower sparsity, which measures the number of modifications needed to achieve the counterfactuals. Table shows these results using counterfactual generator we adapted for this scenario (see Appendix A.3), however, similar conclusions are also obtained using out-of aut-of-the-box generators [34] (see Appendix B.2).
Counterfactual generators can be adapted to specific clients (Figure 2). In our study, we explored the impact of client adaptation on counterfactual generation within FL en- vironments, detailed in Section 4.2. Client-personalisation is key whenever we need to extract client-specific infor- mation. This adaptation can be achieved by training (or fine-tuning) a counterfactual generator on local client's 's data (which naturally happens at each round). The effecttivness of client-specire adaptation can be measured by the relative change in client proximity between global and ddient-specifie models, as shown in Figure 2. The figure shows a marked reduction in relative proximity across the three datasets un- der noD-III conditions (up to 70% in the Synthetic dataset). This suggests that client- speeeffi counterfactuals after adaptation are more representative of individual client datasets, providing unique descriptors of client-specific behaviours in the CBP.
Comparison of New-lif) Settings Across Datasets
Pdacii Posiniity
10
10
10
Drabette
Breast Dataset
Synthati
Figure 2: Relative variation of client- proximity across datasets.
7

---

5.2 Explaining FL training
Synthetic
Breast
Small-MNIST
Error Behaviour
Error Behaviour
Error Behaviour
2nd pincipal component
0.5
0.5
0.77
-1.0
0.55
0.3
0.2
0.4
Counterfactual Behaviour
Counterfactual Behaviour
Counterfactual Behaviour
2nd pr component
20
-20
principal component
principal component
pr principal componont
Figure 3: Client trajectories on Counterfactuals and Error Behavioural Planes for Synthetic, Breast Cancer, and small-MNIST datasets, corresponding to Inverted-loss Crafted-noise, and Inverted- gradient attacks, respectively. The figure highlights the deviation of the malicious client (red) from honest clients, who tend to cluster together over time, along with the previous-round global model (S)
The trajectories in FBPs enable the identification of different client behaviours during training (Figure 3). Figure 3 illustrates FBPs' client trajectories over the last 15 rounds, where we introduced different attack on each dataset (from left to right: Inverted-loss, Crafted-noise, and inverted-gradient attacks). Each dataset was configured with five non-IID clients and one attacker (red). In the Synthetic dataset, to highlight the visualization of honest client clusters, the two largest clients were subdivided into three (Client 2,3.4) and two smaller clients (Client 6,7), respectively, forming distinct clusters. Specifically, on the EBP for the Synthetic dataset, the attacker (Client 9) significantly deviates from the trajectories of other clients, aiming disrupt the server model (S). On the CBP, clusters with similar data distributions converge, indicating similarity in decision boundaries and client training data. In Breast Cancer and ANIST- FBPs also able to give insights on the type of attack. On the Breast Cancer EBP, Client exhibits a random trajectory around the server trajectory, which indicates Crafted-noise attack. In SNaII-MNIST, both the CBP and EBP reveal that at each round the trajectory of the attacker consistently moves in the opposite direction to that of the server, which indicates an inverted-gradient attack. These insights might be useful for FL users as they explain the nature of clients' behaviours, allowing the identification of as various types of attacks. Additional visualizations are available in the Appendix B.3.
FBPs allow the identification of clusters of clients (see Figure 3). In the Synthetic dataset, CBPs's clusters reflect -pient-specific data distributions (Client 1,2-3-4.5,6-7,8 and Attacker 9). Clients sharing similar data distributions tend to cluster closely in the CBP, which in the Synthetic dataset is structured with adjacent slices in the feature space (for example, Client is positioned between Client 8 and Client 2). Similarly, in Breast Cancer and SmalI-MNIST, two primary clusters are identifiable in the CBP: honest clients and attackers. This clustering capability is crucial for users aiming to comprehend client characteristics without direct access to data or models' parameters, thereby informing strategic decisions in training a federated model. For instance, one might consider reducing the number of clients with identical data distributions to avoid redundancy and enhance training efficiency. Conversely, identifying distinct clusters in FBPs can be useful maximise model performance (at the expense of the generalisation ability) by forming Clustered FL system where independent federated models are trained using a subset of similar clients.
5.3 Leveraging FBPs information
FBPs offer detailed insights into client behaviours, enabling the Federated Behavioural Shields to outperform existing state-of-the-art defense mechanisms (Figure 4). Our comparative analysis demonstrates that Federated Behavioural Shields generally outperform traditional methods such as Krum, Median, and Trimmed-meat across various datasets including Breast, Diabetes, and small-
8

---

MNIST. The only exception was in the scenario of the Inverted gradient on the small-MNIST dataset, where Trimmed-mest -man performed better. Overall, our method enhanced the performance up to 10 percentage points (pp) over Median the most robust aggregation baseline- under Label-fliping attack and up to 16 pp when the system is not under attack on small-MNIST. The proposed approach surpasses even a FedAvg aggregation in absence of attackers on the Breast Cancer and on the small-MNIST datasets, under both normal and crafted-noise conditions. These results suggest that an aggregation strategy based on predictive performance or decision-makinn similarities is superior to methods that solely consider sample count. Unlike other baselines, the independence of our method from prior knowledge about attackers establishes it robust aggregation tool in both adversarial and non-adversarial settings. Contrary to previous studies [14, 28), we find that predictive performance alone does not reliably identify malicious clients in non-IID settings, as honest clients may also show consistently low performance, leading to potential misclassification in the aggregation process. Occasionally, this reliance on predictive performance can reduce our method's s accuracy compared to scenarios where only counterfactual information is used. For instance, under no-attack conditions, counterfactual information alone often offers more effective behavioural descriptors than combining it with predictive metrics. Further detailed analyses in Appendix 1.12 indicate that using both descriptors generally yields better results than relying solely on counterfactual behaviour. Furthermore, visualizations in Appendix B.5 show how client scores consistently identify malicious clients during training. while Appendix B.11 demonstrates the robustness of our Federated Behavioural Shields against varying attack intensities.
the FL system is
Breast Cancer Dataset
Diabetes Dataset
Accuncy
04
02
No attack MP Novse
MP G.ndd
DPP 1ooes
attock
MP Noice
MP G
DP Fkp
DP 1low
small-MNIST Dataset
small-CIFARtO Dataset
Accuracy (5)
20
[PP
Fkpp
Type Attack
Type
Medio
Tile
RRA
Oax
Dec
Figure 4: Comparative analysis of Federated Behavioural Shields and its simpler version with only counterfactuals (cf) or mean defenses across five attack types Noo attack, Crafted-noise, Inverted inverted-gradient, Label-ffipping Inverted-loss-on three distinct datasets. Red dashed lines represent the accuracy achieved using FedAvg without attackers
(error) versus Krum, Median, and Trimmed-
6
Discussion
6.1 Related works
Client behaviours in FL have heen analysed using integrated visual tools [31, 46, 47] or indirectly through methods such as robust aggregation [10-14, 28, 30, 32, 40, 48-52] and clustering- based aggregation [53-59]. These studies typically focus on similarities in model or gradient parameters [10-13, 31, 40, 46-52, 55-59], under the premise that distinet client data distributions manifest as unique model parameters. Nevertheless, these methods might overlook valuable information about how client models behave with the data, particularly in terms of predictive performance. To mitigate this issue, previous studies adopted evaluation methods that utilise clean validation set on the server to delineate clients' behaviour. Common descriptors used in such cases are straightforward metrics such as accuracy, error rates, and loss [14, 31, 46, 47, 53,53,54) Despite their utility, these metrics might not be able to reveal behavioural patterns, such as those involved in the decision-making processes of models, which may suggest subtle similarities manipulations. Notably, Wang et al. [46] introduced

---

a post-hoc explainable approach, Grad-CAM [60], to explain client model behaviours during training. However, Grad-CAM is limited to CNN models and provides primarily qualitative visual insights, which cannot be easily automated and thus still require human intervention. Similarly, SentiNet [32] uses Grad-CAM detect potentially malicious regions input images, yet client behaviour assessments continue to rely primarily on evaluating predictive performance on manipulated and unmanipulated images. To the best of our knowledge, this work represents the first systematic attempt to formalise the evolving dynamics of clients in FL, showing how behavioural client trajectories affect the predictive performance and decision-making processes of the global model.
6.2 Limitations and future works
The primary constraint of our framework lies in its assumption that the server possesses a minimal validation set for querying client models. While this assumption is common across various methods, it can be mitigated, as demonstrated in Wang et al. [46], by generating synthetic data points. To this end, our approach might already integrate potential mechanism as differentiable counterfactual methods can be used to generate synthetic data [61]. Furthermore, as indicated by the promising descriptors, such counterfactuals, renders our defense method robust against biased or unfair validation sets. Given the significance of ensuring descriptors represents promis-
results in Appendix B.6, utilizing aalidationiindependdee
fairness across clients, developing additional ialidatiionindepenndee ing direction for future research. Another consideration is the computational overhead introduced by counterfactual generators, which, although minimal compared other baselines (see Appendix A.6), is higher than that of traditional FedAvg. However, this overhead can be mitigated by using smaller network for the counterfactual generator, thereby reducing the number of neurons (e.g.. 1.8% of predictor parameters) without compromising accuracy.
Future work could leverage the extensive information provided by FBPs explore additional strategies for optimizing the learning process, such as the development of Clustered Fl. among clusters of clients and the fine-grained categorization of attack types. Incorporating additional behavioural planes may also enhance the specificity of the FBPs explanations. Lastly, since our method allows for the integration of privacy-enhancing techniques such as Local Differential Privacy [62] and Homomorphic Encryption [63], future studies could analyze their impact on the overall performance and computational efficiency of our system.
6.3 Conclusions
In this work, we proposed Federated Behavioural Planes, method explain the dynamics of FL systems and client behaviours. This innovative method allows to visualise, track, and analyse client behaviours based on specific characteristics. Our focus was twofold: examining predictive performance by analysing prediction errors and investigating the decision-making process through counterfactual generation. The results of our experiments showed that Federated Behavioural Planes enable to track client behaviours over time, cluster similar clients, and identify clients' contributions to the global model with respect to a specific descriptor. Based on Federated Behavioural Planes' information, we introduced novel robust aggregation mechanism that improves existing state-of- the-art methods by not requiring prior knowledge of the attacker. This work lays the foundation to explain the evolution of client behaviours, with the potential to enhance reliability and control over FL systems.
Acknowledgments and Disclosure of Funding
This research was supported by the Swiss National Science Foundation, the European Union's Horizon Europe research program, and the Slovenian Research and Innovation Agency under SmanCCHANGE (No. 101080965), TRUST-ME (No. 205 214991), and XAI-PAC (No. Z00P2 Z00P2_216405) 216405) projects.
Author Contributions. Dario Fenoglio: Conceptualization, Methodology, Implementation, Experi- ments, Writing Original Draft, Writing and supplementary experiments - Rebuttal and Final Paper. Gabriele Dominici: Conceptualization, Methodology, Implementation of Counterfactual Generators, Writing Original Draft, Pietro Barbiero: Conceptualization, Review, Supervision. Alberto Tonda: Review. Martin Gjoreski: Review, Supervision, Funding Acquisition. Mare Langheinrick: Review, Supervision, Funding Acquisition.
10

---

References
[1] Health insurance portability and accountability act. Public Law 104-191, 104th Congress, 1996.
[2] General data protection regulation. Regulation (EU) 2016/679 of the European Parliament and of the Council, 2018.
[3] Peter et al. Kairouz. Advances and open problems in federated learning. Foundations and Trends@ in Machine Learning, 14(1-2):1-210, 2021.
[4] Jie Lu, Anjin Liu, Fan Dong, Feng Gu, Joao Gama, and Guangquan Zhang. Learning under concept drift: A review. IEEE Transactions on Knowledge and Data Engineering, 31(12): 2346-2363, 2018.
[5] Hongyan Chang and Reza Shokri. Bias propagation in federated learning. arXiv preprint arXiv:2309.02160) 2023.
[6] Pengrui Liu, Xiangrui Xu, and Wei Wang. Threats, attacks and defenses federated learning: issues, taxonomy and perspectives. Cybersecurity, 5(1):4, 2022.
[7] Lingjuan Lyu, Han Yu, Qiang Yang, et al. Privacy and robustness in federated learning: Attacks and defenses. IEEE Transactions on Neural Networks and Learning Systems, 2022.
[8] Lingjuan Lyu, Han Yu, and Qiang Yang. Threats to federated learning: A survey. arXiv preprint arXiv:2003.02133, 2020. 2020.
[9] Arjun Nitin Bhagoji, Supriyo Chakraborty, Prateek Mittal, and Seraphin Calo. Analyzing feder- ated learning through an adversarial lens. In International Conference on Machine Learning. PMLR, 2019.
[10] Virat Shejwalkar and Amir Houmansadr. Manipulating the byzantine: Optimizing model poisoning attacks and defenses for federated learning. In Proceedings of the Network and Distributed System Security Symposium (NDSS), 2021.
[11] Dong Yin, Yu-Xiang Chen, Ramchandran Kannan, and Peter Bartlett. Byzantine dis- tributed learning: Towards optimal statistical rates. Proceedings of the International Confer- ence on Machine Learning. ICML. PMLR, 2018.
[12] Peva Blanchard al. Machine learning with adversaries: Byzantine tolerant gradient descent. In Advances in Neural Information Processing Systems, volume 30, 2017.
[13] Rachid Guerraoui and Sébastien Rouault. The hidden vulnerability of distributed learning in byzantium. In International Conference on Machine Learning. PMLR, 2018.
[14] Minghong Fang, Xiaoyu Cao, Jinyuan Jia, and Neil Zhenqiang Gong. Local model poisoning attacks to Byzantine- Robust federated learning. In 29th USENIX Security Symposium (USENIX Security 20), 2020.
[15] Yi Zeng and et al. Meta-Sift: How to sift out a clean subset in the presence of data poisoning?
In 32nd USENIX Security Symposium (USENIX Security 23). 2023.
[16] Leonardo Vanneschi, Mauro Castelli, and Sara Silva. survey of semantic methods in genetic programming, Genetic Programminn and Evolvable Machines, (5(2):195-214, January 2014. ISSN 1573-7632, doi: 10.1007/s 11000----1---000000 URL http://dxx doi www/o0.goo... 1007/ s10710-013 01177000013-921--00
[17] Alberto Moraglio, Krzysztof Krawiec, and Colin G. Johnson. Geometric Semantic Genetic Programming, page 21-31. Springer Berlin Heidelberg, 2012. ISBN 9783642329371. doi: 10. (007/978-3-642-33 3222222-2371123 URL attp:://dx doi 11t.///011077777-33111111 32937-122977111 3.
[18] Jean-Baptiste Mouret and Jeff Clune. Illuminating search spaces by mapping elites. CoRR, abs/1504.04909, 2015. URL attt:://axte org/ abbs/1504.00
11

---

[19] Sandra Wachter, Brent Mittelstadt, and Chris Russell. Counterfactual explanations without opening the black box: Automated decisions and the gdpr. Harv. JL & Tech., 31:841, 2017.
[20] Brendan McMahan, Eider Moore, Daniel Ramage, Seth Hampson, and Blaise Aguera y Arcas. Communication--ffinn
Learning of Deep Networks from Decentralized Data. In Proceedings of the 20th International Conference on Artificial Intelligence and Statistics, pages 1273-1282. PMLR, April 2017.
[21] Ali Shafahi, W. Ronny Huang, Christoph Studer, et al. Poison frogs! targeted clean-label poisoning attacks on neural networks. In Advances in Neural Information Processing Systems, volume 31, 2018.
[22] Eugene Bagdasaryan, Andreas Veit, Yiqing Hua, Deborah Estrin, and Vitaly Shmatikov. How to backdoor federated learning. In International Conference on Artificial Intelligence and Statistics. PMLR, 2020.
[23] Gilad Baruch, Moran Baruch, and Yoav Goldberg. A little is enough: Circumventing defenses for distributed learning. In Advances in Neural Information Processing Systems, volume 32, 2019.
[24] Cong Xie, Oluwasanmi Koyejo, and Indranil Gupta. Fall of empires: Breaking byzantine- tolerant sgd by inner product manipulation. In Uncertainty in Artificial Intelligence. PMLR, 2020.
[25] Prajjwal Gupta, Fan Jin, Bingyan Li, Yuzheng Wu, et al. A novel data poisoning attack in federated learning based on inverted loss function. Computers & Security, 130:103270, 2023,
[26] Mengye Ren, Wenyuan Zeng, Bin Yang, and Raquel Urtasun. Learning to reweight examples for robust deep learning. In International Conference on Machine Learning. PMLR, 2018.
[27] Hongyi Wang, Jared Kaplan, Keith Saab, Ben Goodman, Junfeng Bai, et al. Attack of the tails: Yes, you really can backdoor federated learning. Advances in Neural Information Processing Systems, 33:16070--16044 2020.
[28] Cong Xie, Sanmi Koyejo. and Indranil Gupta. Zeno: Distributed stochastic gradient descent with cuspicion-based fault-tolerance. In International Conference on Machine Learning, page PMLR, 2019.
[29] Suyi Li et al. Abnormal client behavior detection in federated learning. arXiv preprint arXiv:1910.09933 2019.
[30] Xiaoyu Cao et al. Fltrust: Byzantine-robust federated learning via trust bootstrapping. arXiv preprint arXiv:2012.13395 2020.
[31] Linhao Meng, Yating Wei, Rusheng Pan, Shuyue Zhou, Jianwei Zhang, and Wei Chen. Vadaf: Visualization for abnormal client detection and analysis in federated learning. ACM Transactions on Interactive Intelligent Systems (Tii), 11(3-4):1-22 2021.
[32] Edward Chou, Florian Tramer, and Giancarlo Pellegrino. Sentinet: Detecting localized universal attacks against deep learning systems. In 2020 IEEE Security and Privacy Workshops (SPW). IEEE, 2020.
[33] Hengzhe Zhang, Qi Chen, Alberto Tonda, Bing Xue, Wolfgang Banzhaf, and Mengjie Zhang, with
for Evolutionary Ensemble Learning. In 26th European Conference, EuroGP, volume 13986 of Lecture Notes in Computer Science, pages 84 100, Brno, Czech Republic. April 2023. Springer Nature Switzerland. doi: 10.100779788---03-299555 6. URI. https:t/////.... /hal. sci wacciencehaall-0000 042301844
[34] Victor Guyomard, Françoise Fessant, Thomas Guyet, Tassadit Bouadi, and Alexandre Termier. Venet: A self-explaining model for realistic counterfactual generation. In Massih-Reza Amini, Stéphane Canu, Asja Fischer, Tias Guns, Petra Kralj Novak, and Grigorios Tsoumakas, editors, Machine Learning and Knowledge Discovery in Databases, pages 437-453, Cham, 2023. Springer International Publishing. ISBN 978-3-031-263877
12

---

[35] Martin Pawelczyk, Klaus Broelemann, and Gjergji Kasneci. Learning model- counter- factual explanations for tabular data. Proceedings of The Web Conference 2020, www '20. ACM, April 2020. doi: 10.1 10.1145/33664233330088 URL. http: //xx. http//xxxdiioi/////ooo 1r./10.10 1145/
3366423.3380887.
[36] William Wolberg, Olvi Mangasarian, Nick Street, and W. Street. Breast Cancer Wisconsin (Di- agnostic). UCI Machine Learning Repository, 1995. DOI: https://oo..or//100 wttpo/oorr/11244222... 20.24444///SS2.
[37] Centers for Disease Control and Prevention (CDC). CDC Diabetes Health Indicators Dataset, 2017.
[38] Li Deng. The mnist database of handwritten digit images for machine learning research. IEEE Signal Processing Magazine, 29(6):141-144 2012.
[39] Alex Krizhevsky, Geoffrey Hinton, et al. Learning multiple layers of features from tiny images. 2009.
[40] Yanli Li et al. Enhancing federated learning robustness through clustering non-iid features. In Proceedings of the Asian Conference on Computer Vision, 2022.
[41] Junyu Shi et al. Challenges and approaches for mitigating byzantine attacks in federated learning. In 2022 IEEE International Conference on Trust, Security and Privacy in Computing and Communications (TrustCom). IEEE, 2022.
[42] Krishna Pillutla, Sham M Kakade, and Zaid Harchaoui. Robust aggregation for federated learning. IEEE Transactions on Signal Processing, 70:1142-1153 2022.
[43] Vale Tolpegin et al. Data poisoning attacks against federated learning systems. In Proceedings of the 25th European Symposium on Research in Computer Security (ESORICS), volume 25 of Computer Security- - ESORICS 2020, pages 25 40, Guildford, UK, 2020. Springer International Publishing.
[44] Jierui Lin, Min Du, and Jian Liu. Free-riders in federated learning: Attacks and defenses. arXiv preprint 1rXiv:1911.12560 2019.
[45] Sai Praneeth Karimireddy, Lie He, and Martin Jaggi. Byzantine-robust learning on heteroge- neous datasets via bucketing. arXiv preprint arXiv:2006.0933 09365, 2020.
[46] Xumeng Wang et al. Hetvis: A visual analysis approach for identifying data heterogeneity in horizontal federated learning. IEEE Transactions on Visualization and Computer Graphics, 29 11):310-319, 2022.
[47] Quan Li et al. Inspecting the running process horizontal federated learning via visual analytics. IEEE Transactions on Visualization and Computer Graphics, 28112)4005-41100, 2021.
[48] Clement Fung, Chris JM Yoon, and Ivan Beschastnikh. The limitations of federated learning in sybil settings. In 23rd International Symposium on Research in Attacks, Intrusions and Defenses (RAID 2020), 2020.
[49] Yudong Chen, Lili Su, and Jiaming Xu. Distributed statistical machine learning in adversarial settings: Byzantine gradient descent. In Abstracts of the 2018 ACM International Conference on Measurement and Modeling of Computer Systems, 2018
[50] Yunlong Mao et al. Romoa: Robust model aggregation for the resistance of federated learning model poisoning attacks. In Computer -SSORIS 2021: 26th European Symposium on Research in Computer Security, Darmstadt, Germany, October 4-8, 2021, Proceedings, Part I, volume 26. Springer International Publishing, 2021.
[51] Cody Lewis. Vijay Varadharajan, and Nasimul Noman. Attacks against federated learning defense systems and their mitigation. Journal of Machine Learning Research, 24(30):1-50 2023.
[52] Najeeb Moharram Jebreel et al. Lfighter: Defending against the lahel-fipping attack in federated learning. Neural Networks, 170:111-126, 2024.
13

---

[53] Avishek Ghosh, Jichan Chung, Dong Yin, and Kannan Ramchandran. An efficient framework for clustered federated learning. In H. Larochelle. M. Ranzato, R. Hadsell, M.F. Balcan, and H. Lin, editors, Advances in Neural Information Processing Systems, volume 33, pages 19586- 19597. Curran Associates, Inc., 2020. URL. https: p/peoeee///rrocedinggs neurips. cc/paper_ f
pdf.
[54] Mahdi Morafah, Saeed Vahidian, Weijia Wang. and Bill Lin. Flis: Clustered federated learning via inference similarity for non-iid data distribution. IEEE Open Journal of the Computer Society, 120, 2023. doi: 10. 109////CS.0233336660
[55] Felix Sattler, Klaus-Robert Müller, and Wajeiech Samek. Clustered federated learning: Model- agnostic distributed multitask optimization under privacy constraints. IEEE Transactions on Neural Networks and Learning Systems. (188)3370--722, 2021. doi: 10.1109/TNNL.2202 3015958.
[56] Christopher Briggs, Zhong Fan, and Peter Andras. Federated learning with hierarchical cluster- ing of local updates to improve training on non-iid data. In 2020 International Joint Conference on Neural Networks (IJCNN), pages 1-9, 2020. doi: 10. 109/IUCNNN4400520000000 9207469.
[57] Guodong Long, Ming Xie, Tao Shen, Tianyi Zhou, Xianzhi Wang, and Jing Jiang. Multi- center federated learning: clients clustering for better personalization. World Wide Web, 26 (1):481-500, 2023. doi:
1200000222------ URL https: org/10.1007
s $11280-022 01046xx.
[58] Biyao Gong, Tianzhang Xing, Zhidan Liu, Wei Xi, and Xiaojiang Chen. Adaptive client clustering for efficient federated learning over non-iid and imbalanced data. IEEE Transactions on Big Data, pages 1-1, 2022. doi: 10. 100.0/BDAAAA.2223311199
[59] Moming Duan, Duo Liu, Xinyuan Ji, Yu Wu, Liang Liang, Xianzhang Chen, Yujuan Tan, and Ao Ren. Flexible clustered federated learning for client-level data distribution shift. IEEE Transactions on Parallel and Distributed Systems, 33(1 1):2661-2674, 2022. doi: 10.1109/ TPDS.2021.3134266
[60] Ramprasaath R. Selvaraju et al. Grad-cam: Visual explanations from deep networks via gradient- based localization. In Proceedings of the IEEE International Conference on Computer Vision, 2017.
[61] Diederik P Kingma and Max Welling. Auto-encoding variational bayes, 2022.
[62] Stacey Truex, Ling Liu, Ka-Ho Chow, Mehmet Emre Gursoy, and Wenqi Wei. Ldp-fed: Federated learning with local differential privacy. In Proceedings of the third ACM international workshop edge systems, analytics and networking, pages 61-66, 2020.
[63] Arnaud Grivet Sébert, Marina Checri, Oana Stan, Renaud Sirdey, and Cedric Gouy-Pailler. Combining homomorphic encryption and differential privacy in federated learning. In 2023 20th Annual International Conference on Privacy, Security and Trust (PST), pages 1-7. IEEE, 2023.
[64] Xin Jin and Jiawei Han. K-means clustering. In Claude Sammut and Geoffrey I. Webb, editors, Encyclopedia of Machine Learning, pages 563-564. Springer US, Boston, MA, 2010. ISBN 778-0-387-30164-8, doi: LLL007/978-00337-3006666
[65] Gabriele Dominici, Pietro Barbiero, Francesco Giannini, Martin Gjoreski, Giuseppe Marra, and Mare Langheinrich. Climbing the ladder of interpretability with counterfactual concept bottleneck models, 2024.
[66] Daniel Beutel, Taner Topal, Akhil Mathur, Xinchi Qiu, Javier Fernandez-Marqueee Yan Gao, Lorenzo Sani, Hei Li Kwing, Titouan Parcollet, Pedro PB de Gusmão, and Nicholas D Lane. Flower: friendly federated learning research framework. arXiv preprint arXiv:2007. 14390, 2020.
14

---

[67] Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, Alban Desmaison, Andreas Köpf, Edward Yang. Zachary Vitito, Martin Raison, Alykhan Tejani, Sasank Chilamkurthy, Benoit Steiner, Lu Fang, Junjie Bai, and Soumith Chintala. Pytorch: An imperative style, high-performance deep learning library. Advances in Neural Information Processing Systems, 32, 2019.
[68] Pedregosa, G. Varoquaux, A. Gramfort, V. Michel, B. Thirion, O. Grisel, M. Blondel, P. Prettenhofer, R. Weiss, V. Dubourg. J. Vanderplas, A. Passos, D. Cournapea,, M. Brucher, M. Perrot, and E. Duchesnay. Scikit-learn: Machine learning in Python. Journal of Machine Learning Research, 22:2825-2830, 2011.
[69] D. Hunter. Matplotlib: A 2d graphics environment. Computing in Science & Engineering, (3):90-95, 2007. doi: 10.1 10.10M/CCE2200
[70] Michael L. Waskom. seaborn: statistical data visualization. Journal of Open Source Software, 6(60):3021, 2021.
doi: 10.21105/joss.03021 URL https://doi: http///i..org...111...0000 joss.
03021.
[71] Wes McKinney. Data Structures for Statistical Computing in Python. In Stéfan van der Walt and Jarrod Millman, editors, Proceedings of the 9th Python in Science Conference, pages 56 61, 2010. doi: 10.25080/Majora wateoora--2999920000000 00a.
15

---

A Experimental setup
A.1 Datasets
In our experiments, we employ five distinct datasets:
Synthetic dataset (Tabular): It consists two features randomly extracted from a range of -5 to +5, As shown in Figure 6.a, we partition this feature space into K slices, where K equals the number of clients. Each slice corresponds to a specific client. We assign a label of to all data points where I1 > 0222, effectively creating linear decision boundary that varies with a. For each client, we generally draw 1000 samples. This controlled setup allows to precisely manipulate and visually understand the data distribution across different clients, as illustrated in Figure 6a.
Breast Cancer Wisconsin (Tabular): It contains data from 569 patients with 30 continuous variables derived from digitised images of fine needle aspirate of breast mass. The variables describe characteristics of the cell nuclei present in the image (e.g., radius, area, perimeter), aimed at predicting breast cancer [36].
Diabetes Health Indicator (Tabular): It comprises data from 70,692 patients, encompassing questionnaire-bann based variables (e.g., smoking, physical activity, fruit intake) and medical measurements (e.g., BMI, cholesterol levels), with total of 21 features, to predict the presence of diabetes [37].
MNIST (Image): It is comprehensive database of handwritten digits frequently used to benchmark image classification algorithms [38]. Each image is 28x28 pixel grayscale representation of a digit, ranging from 0 to 9. To increase task difficulty and highlight client differences in performance, our experiments utilise only 10,000 of the available 70,000 images, referred to as small-MNIST. Additionally, we transform these images into color, randomly assigning colors with equal probabilities: red, green, or blue. To explore the decision decision-making process through counterfactuals, we employed ResNet-18 18 architecture to extract 1,000 features from each image.
CIFAR-10 (Image): The CIFAR-10 dataset comprises 60,000 color images categorised into 10 classes, widely used for evaluating object recognition algorithms [39]. Each image has a resolution of 32x32 pixels and is represented in RGB format. Similar our smalI-MNIST setup, we reduce the sample size to 10,000 images. Features are extracted using ResNet-18 architecture, resulting in din-dimenii feature vectors for each image.
IID distributions across clients are achieved by randomly selecting samples from each dataset. In contrast, non-IID distributions are created using KMeans clustering [64] to form K clusters of samples within each class- for the Synthetic dataset and 5 for others. We then assign the samples of the two nearest clusters from different classes to each client. To ensure each client's 's distribution is equally represented, into the test set to assess the performance of our experiments. We create a clean validation set on the server by extracting 89 samples for Breast Cancer and 250 samples for other datasets. Additionally, we partition each client's 's dataset locally, allocating 80% for training and 20% for validation.
randomly extract 15% of the samples from each client and combine them
A.2 Model configuration
In our work, the model is composed of a predictor, any Deep Neural Network, which in our is multi-layer perceptron that takes tabular data as input and makes a prediction. It is composed of five layers with hidden dimension equal 512. 256, 256, 64, respectively. In addition, in our work, the model is also composed of counterfactual generator, which can be implemented in different ways. The only requirement is that can be trained concurrently with the main model, in an end-to-end fashion. To generalise our results, we tested the model with two different counterfactual generators. In Section 5, we used a counterfactual generator designed by us, adapting the intuition from Dominici et al. [65], that explicitly optimises the generation of counterfactual with respect to specific labels using two VAEs, named CFGen and described in Appendix A.3. On the other hand, in Appendix B.2 we also tested VCNet [34], which could generate counterfactual training a VAE, implementing according to the original paper. In the context of creating Federated Behavioural Planes, PCA employed the dimensionality reduction technique 2/--2 within the EBP, centering the reduced
16

---

space around a zero vector, which signifies the absence of errors. For CBP, iSNE is utilised as the dimensionality reduction method x ensuring the preservation of distances between clients and their respective clusters. Finally, in terms of the implementation of FBSs, at each round 6, the score (t)) for each client k is computed using a moving average with a window length of L, as follow:
L-1
(t)
1 L
Σ
5 (t i)
(9)
1=
Additionally, client exclusion (t) 0) is automatically triggered if client exhibits score approaching zero (i.e., < 10-1) within the specified window.
A.3 CFGen architecture
CFGen is designed to adapt the approach proposed by Dominici et al. [65] to tabular data, eliminating the need for predefined concepts. It latent variable model that generates counterfactuals through variational inference. To this end, we have two random variables and z'. These variables represent latent factors of variation whose probability distributions are easier to model and sample compared to those for and æ'. We also include dependencies from y to the counterfactual latent distribution e' in order to explicitly model the dependency = on the class labels, resulting in the following overall probabilistic graphical model:
(10)
This way. the generative distribution factorises as:
y. 2, pla. pt
(11)
p(á, y(8.y22) p(yl))p(((=))
p(=',8'l='> = p(dl= = p p(aplll
(12)
In our approach, 2) and p(y/(=²) are the task predictor, 2())2) and p(z'|z') are the same decoder. In practice, we assume that the input æ always observed at test time, making the term and p(z|2)
irrelevant. Finally, p(z) is standard normal prior distribution and p(2' &. y) is learnable normal prior whose mean and variance are parametrised by pair of neural networks Opp and Ope.
Amortised inference. CFGen amortise inference needed for training by introducing two approxi- mate Gaussian posteriors (=(()) and q22'|2, y, s/) whose mean and variance are parametrised by pair of neural networks (0... 0.) (low, &or) respectively).
Optimization problem. In order to obtain these counterfactuals, it is important optimise their generation during training. CFGen is trained to maximise the log-likelihood of tuples (æ, v'y",, while observing æ. Following variational inference approach, we optimise the evidence lower bound of the lieeloool,oood, which results in the following objective function to maximise:
reconstraction of & and
prior regularization
E (log logp(y
( [
reconstmction of y
prior regularization on s'
D DxL (o)lle/ v)
(13)
where DKL is the Kullback- divergence and a = (2, y, y'). Moreover, in order to enforce the counterfactuals be as close possible to the initial input, add additional term to the objective:
posterior distance
prior distance
Las DEL (a(<p))lla(e((aa) DEL (p(2))ll(=1
(14)
A.4 Training configuration
Gradient Descent was employed as the optimisation algorithm, with batch size equivalent to the dimension of the training dataset. Both the momentum and learning rate were set at 0.9 and 0.01,
17

---

respectively. For centralised training scenarios, the model was trained over 1,000 epochs. The Flower library was utilised implement FL [66]. In all federated experiments, except for those evaluating local epochs, we employed 2 local epochs. During the assessment of various defense mechanisms, the number of communication rounds was capped at 200 and the window length for the moving average of 30 rounds. For comparisons with centralised training, 1,000) rounds were used. For client personalization, the generator was trained across 25 local epochs. In each experiments, performance metrics were evaluated on the model that exhibited the lowest aggregated loss during training. The aggregated loss represents the weighted average client losses, evaluated on each client's local validation set, proportional to the respective number of samples.
A.5
Code, licenses and hardware
For our experiments, implement all baselines and methods in Python 3.9 and relied upon open- source libraries such as Py'oorch 2.2 [67] (BSD license). Sklearn 1.4 [68] (BSD license), Flower 1.6 [66] (Apache License). In addition, we used Matplotlib [69] 3.8.2 (BSD license) and Seaborn [70] 0.13 (BSD license) to produce the plots shown in this paper. Data processing is performed using Pandas [71] 2.2 (BSD license). The four datasets we used are freely available on the web with licenses: Breast Cancer Wisconsin (CC BY-NC-SA 4.0 license), Diabetes Health Indicators (CC0 license), MNIST (GNU license). and CIFAR-10. Our code, along with all the necessary details reproduce the experiments, is publicly available on GitHub under the MIT license. Additionally, we provide pseudo-code for both client-side (Algorithm 2) and server-side (Algorithm 1) implementations of our proposed approach, which includes creating behavioural planes on the server and applying our FBSs. All experiments were conducted on a workstation equipped with an NVIDIA RTX A6000 GPU, two AMD EPYC 7513 32-Core processors, and 512 GB of RAM.
Algorithm 1 The Federated Behavioural Shields Algorithm
Require: Initial model ((())), number of communication rounds T, number of selected clients per round M, clean validation set (x (serner)) yf (server)), set of plane functions S
1: for [= 0, 1,.... T 1 do
2: Sample M out of N clients
3: Send (()) to the selected M clients
4: wait
5: for each selected client & in M in parallel do
6: 2/(k), ((6),,(()) = f(0(k) (t + 1), (eerve)))
7:
end for
8: for plane_ in S do
> Creation of planes
9:
st(1)= plane_In(x"te (()), , (k), (server)) end for
Our case: Eq.5 and Eq.6-7
10:
11: Behavioural planes visualization (FBPs)
s(k) (t) client score Eq. 8 (part 1)
f((( + 1)) Model aggregation Eq. 8 (part 2) with (x))(t)
Eq. 2 for FBPs
14: end for
Algorithm Local Training on Client
Require:
Initial model architecture f. number of local epochs E, local dataset (z)(), (R)) (R))
1: Receive current global model parameters (t)) from the server
2: Initialise client model with global parameters: f(((*) < f(((t)
3: Update local model g(k) (t + 1) by training for E epochs
> Both predictor and generator
4: Send updated model parameters ((() 1) back to the server
A.6
Computational cost
This section outlines the computational costs associated with our proposed method, focusing on three primary components: local computation, communication overhead, and server-side computation.
hhtgs///gthhbbccm//ardddeeggggg FL
18

---

Local computation. Our methods (both FBPs and FBSs) integrate counterfactual generator with the original predictor analyse pecision-makingg and provide insights into the client's data distribution. For small neural networks, local computation is minimal compared to other costs in the Fl. framework, such as communication latency and synchronization. generator can produce counterfactuals for the model input without impacting training efficiency. As the predictor size increases, as shown with MNIST and CIFAR-IQ, 10, we can efficiently generate counterfactuals for intermediate layers using relatively small number of neurons. For instance, in our experiments with small-MNIST and Small-CIFAR-II 10, we used a ResNeet-18, which has 12.42M parameters and requires 71.1M GFLOPs for inference with input RGB image of size 28x28. As shown in Table 2, the generator with embedding size of 128 contributes only 5.1% 1.1% of the operations compared to the predictor alone. Furthermore, by reducing the embedding size to 32, maintained performance while reducing GFLOPs to just 2.7% of the ResNet-18.
these cases, the counterfactual
Communication overhead. Similar to the predictor, the counterfactual generator must be transmit- ted the server for evaluation and aggregation, thereby increasing the number of model parameters sent. However, the additional communication overhead is marginal compared to the size of the predictor, As illustrated Table 2, the counterfactual generator with an embedding size of 32, which maintains high task performance, consists of only 1.8% of the parameters of the ResNet-18 18. Conse- quently, with 32-bit precision, our counterfactual requires 0.92 megabytes (MB) out of approximately 49.68 MB for the predictor.
Server-side computation. On the server, our methods involve evaluating client models' perfor- mances on a clean validation set and calculating the pair-wise distances of the generated counterfac- tuals between clients. As shown in Figure 13, the validation set size can be relatively small, e.g., 250 samples, which can be processed with a single forward pass of the model. Consequently, the models' evaluation on the server, which can also be parallelised, negligible compared the computational load of calculating pair-wise distances between client counterfactuals.
The primary computational bottleneck on the server is the pair-wise Wasserstein distance between client counterfactuals. To address this, we use the sliced Wasserstein distance implementation, which has computational complexity of O(m Om logm), m), where is the number of supports, given by m o_samples x reduction dimension (1xx-2). In our implementation, m 250 x 2 = 500, and this operation is repeated for each unique pair of clients (i.e., binomial coefficient (?) ) leading to computational complexity of O(n²... m log m) where is the number of selected clients.
Compared to Krum, which has a complexity of (n"dd) with d being the number of model parameters, our pair-wise operation is more efficient for neural networks with more than approximately 4480 parameters, which is likely the case in most practical applications.
Overall computational cost. We evaluated the overall computational cost of our defense mecha- nism in comparison to both the Krum algorithm and the traditional FedAvg algorithm, considering different network sizes and varying numbers of clients. In all experiments, a validation set of 250 samples was used. For tests involving an increasing number of clients, our method incorporated a counterfactual generator utilizing 7.6% of the predictor's parameters (in the worst-case scenario), as described in the original paper. Computational time was measured over 10 training rounds across 3 folds, and we reported the mean and standard error of the time per round. As shown in Figure 5, our method's computational cost scales more efficiently with both the number of model parameters and the number of clients when compared to Krum. Specifically, minute per round for 200 clients compared to FedAvg, while Krum introduces over 15 minutes per round. Importantly, our approach maintains the robustness and security benefits, demonstrating its efficiency and scalability in large-scale FL scenarios.
method adds only additional
A.7 Robust aggregation baselines
This section details the robust aggregation methods used as baselines in our experiments. These methods are designed to mitigate the effects of poisoning attacks by malicious clients and include:
Median: This approach performs aggregation using the median of the client updates [1]]. Compared to the mean, the median is less affected by outliers. Theoretical guarantees
19

---

Table 2: Comparison of embedding sizes of the counterfactual generator
Metries
Model Parameters
GFLOOP
Embedding size
Accuracy
Validity
Pred.+CF
CF
Increase
Pred.+CF
CF
Increase
128 (paper)
86.0 + 0.3%
100.0 + 0.0%
13.36M
0.94M
7.6%
74.7M
3.6M
5.1%
85.6 1: 0.4%
100.0 0.0%
12.88M
0.47
3.8%
73.6M
2.5M
3.5%
32
87.6 1: 0.5%
100.0 1: 0.0%
12.65M
0.23M
1.8%
73.1M
1.9M
2.7%
Computational Time vs. Model Parameters per Round
Computational Time vs. N. Clients per Round
25
FedAAg
FedAug
15.0
Time (Seronds)
20
Krum
Time (Minutes)
12.5
Krum
33.0
7.5
5.0-
8.0
0.70 68
9.00
17.00
202
Model Parameters (Millions)
Number of Clients
(a)
(b)
Figure 5: Comparison of the computational time of our proposed method against the traditional FedAvg, and the robust aggregation Krum per round of training, across (a) different model parameters and (b) different numbers of clients.
on the robustness of Median aggregation are provided in [11], while empirical evidence demonstrates that it exhibits better robustness than the more sophisticated Krum [14].
Trimmed-mean: This approach [11] aggregates each dimension of input gradients separately. For each dimension i, it sorts the values of the jth dimension of all gradients, removes the 3r largest and smallest values, and averages the remaining values obtain the aggregate for dimension i. We use a default Br equal to 20% of the number of clients.
Krum: Krum [12] is based on the intuition that malicious gradients must be far from benign gradients poison the global model. Assuming knowledge of the upper bound the number of malicious clients m, Krum selects the gradient from the set K input gradients that is closest its K
2 nearest neighbors in terms of the squared Euclidean norm.
Robust Federated Aggregation (RFA): RFA [42] replaces the standard arithmetic mean aggregation with the geometric median to ensure robustness against poisoned updates. It employs Weiszfeld-type algorithm to compute the geometric median in privacy- preserving and communication-efficii manner.
A.8 Federated attacks
This section outlines and provides additional details about the federated attack scenarios implemented into our experiments, focusing on both model and data poisoning without prior knowledge of the server aggregation methods. We implemented these attacks assuming 20% of the clients are malicious:
Label-flipping (Data Poisoning): In a multiclass scenario, this attack changes all samples in the dataset with source class Care into target class Ctarget [26, 43]. For binary classification, label flipping is implemented as - Csources effectively inverting the class labels. For multiclass classification, we perform targeted label flipping by changing all instances of class 2 (Bird) to class (Ship).
Inverted-loss (Data Poisoning): This attack aims to create an update that maximises the loss and, consequently, causes significant drop in accuracy [25]. The implemented loss function follows the algorithm outlined [25].
Crasted-noise (Model Poisoning): Inspired by the Free-riding attack described in [44], this attack aims for stealthiness by adding noise to the previous model received from the server,
20

---

Systwrll Cambuiio
Geserted Courteratiak
Ganaerted Courte dedatateddaallllllee Clance
Genarated Courteberake Cleet
Dat 12
Podiee
(a)
(b)
(c)
(d)
Figure 6: (a) Synthetic dataset. (b) Counterfactuals generated by the server on the synthetic test set. (c) Counterfactuals for Client 4 after adaptation. (d) Counterfactuals for Client 8, similar to (c). Notably, Client 4, with a data distribution perpendicular to the decision boundary, achieves effective adaptation, whereas Client 8 encounters more challenges.
w". The noise is added as follows: w" w² + , o where o (w") represents the standard deviation of w" and 3 scale factor. We adopted equal to 1.2, except when we tested the sensitivity values: 0.3, 0.5, 0.8, 1.2, and 1.6.
Inverted-re Inveried-gradient (Model Poisoning): This attack modifies the inner product between the gradient and the malicious gradient updates sent by the attacker, affecting the alignment with the true gradient [24, 45J. In our implementation. the malicious gradient is inverted: Ww. Given the lack of information about honest clients, the true gradient is taken from the previous update sent by the server, representing the previous improvement of the global model.
For both data poisoning attacks, the attacker uses local dataset drawn from the same distribution as other clients and with an average number of samples.
B
Additional experiments and analysis
B.1
Client-specific adaptation in IID and non-IID scenario
As depicted in Figure 2, there is a significant reduction in the relative proximity measure between global and client-specific models across all three datasets on average, indicating high degree of customization in the client models. Nonetheless, few clients still unable to tailor the model effectively to their specific distribution. To explore this phenomenon, we visually represent the generated counterfactuals pre-adaptation in Figure 6.a, showcasing models with the highest and lowest degrees of client-specific adaptation in Figures 6.c and 6.d, respectively. Notably, the most adapted model corresponds to client 4, whose data is almost perpendicular to the class boundary. In contrast, client 8, whose data distribution lies close to the boundary, struggles to adapt, remaining akin to the pre-adaptation conditions. This could highlight clients whose data is perpendicular to the learnt decision boundary, also indicating that personalization varies among clients, thereby offering more insights the behaviour of the clients. Furthermore, analyse the effects of client-specific adaptation under both IID and non-IlI conditions. As shown in Figure 7, in contrast non-IID, under IID conditions where data distributions are uniform between clients, the enhancements were minimal, indicating an alignment between global and local optima. The lack of customization implies that all client models share similar decision-makes processes, correctly reflecting with their IID condition. However, this setting also facilitates the identification of outliers or anomalous clients, which exhibit higher degree of customization.
increased capacity in personalisation in
B.2 Architecture independence in counterfactual generation
To explore the independence of FBPs from the counterfactual generator, we evaluate our methodology with an alternative model named VCNet [34], maintaining identical experimental conditions. As illustrated in Table 3, VCNet achieves comparable performance in both Federated Learning (FL.) and Centralised Learning (CL.) configurations, where data from all clients placed on single machine. Similarly to counterfactual generator, the results demonstrate that the
training
21

---

Diabetes
Breast Cancer
Synthetic
80
Belatee Prasini (S)
Relatie Procenity (%S
50
Proxinity
50
-00
10
80
-00
20
Nen-II
Nen-II
IID
Nan-IID Setting
Setting
Setting
Figure 7: Relative variation of client-proximity across Diabetes, Breast Cancer, and Synthetic datasets in both IID and non-IID settings. Client-peroonalizationn is particularly effective non-IID settings.
Table 3: Comparison of model performance in Local Centralised, Federated Learning, and Centralised Learning (i.c.. privacy-intrusive) for non-IIl setting using VCNet [34]
Metric
Dataset
Local CL
FL
Accuracy
Diabetes
56.8.10.0%
73.8.10.0%
73.9110.0%
Breast Cancer
84.0.10.2%
97.0.10.3%
9..61
Synthetic
74.6.10.1%
99.5.10.1%
99.8.10.19
Validity (t)
Diabetes
100.10%
100+00%
100-10%
Breast Cancer
100:10%
100-10%
100-10%%
Synthetic
100:10%
100+00%
1004:0%
Sparsity (L)
Diahetes
51.1+-0.1
42.1+0.1
35.4++0.0
Breast Cancer
2131+ 10
1555++4
1560++ 19
Synthetic
9.19110.11
7.07-0.11
6.95
Proximity (L)
Diabetes
11.58:10.28
9.23++024
8.17+00.56
Breast Cancer
132.41:4.5
69.5+00.7
71.5++5.6
Synthetic
0.080-10.002
0.096.10.006
0.090.10.002
strategy (i.e., FL) does compromise the VCNet's effectiveness compared the ideal CL condition. Operating under similar privacy constraints, FL enables VCNet to learn from the diverse distributions of client data, thereby surpassing the performance of the Local CL approach, where each client independently trains their own model.
To assess VCNet's capacity for adapting the counterfactual distribution individual clients, conduct client-specific adaptations starting from the global model achieved through FL.. Figure illustrates the relative variation in client- fiiont-prximmity across three datasets: Diabetes, Breast Cancer, and Synthetic. The observed significant reduction in proximity indicates that VCNet can effectively personalised each client's distribution, thereby confirming its utility in accurately reflecting client behaviours during the training process.
Relative Variation for VCNet. Non.IID
Relative Prosimity (%)
80
40
Diabetes
Breast Cancer Dataset
Synthetic
Figure Relative variation of proximity across Diabetes, Breast Cancer, and Synthetic datasets for VCNET.
22

---

B.3 Explaining FL training with Behavioural Planes
Figure illlstrates client trajectories within the FBPs for different scenarios from those presented in Section 5. Specifically, the left side of the figure displays the FBPs for the Synthetic dataset, now including an attacker employing an Inverted Loss attack, while the right side shows the FBPs for the Diabetes dataset under Data Flip attack scenario. In both cases, the largest clients are subdivided into smaller clients (three and two), forming distinct clusters of clients (2,3,4 and 6,7 for Synthetie; 3,4,5 and 7.8 for Diabetes). These clusters are particularly evident on the CBP, and even within the Synthetic dataset's plane, one can observe clients with similar data distributions (e.g.. Client 8's data distribution lies between that of Clients 7 and 1). This observation holds across all clients in the Synthetic dataset, underscoring the ability to highlight information about data distribution similarities among clients without direct access to their data. Moreover, Client 9 (the attacker) is noticeably diverging from the others across all four planes. It is crucial to note the distinct behavioural patterns of the attacker depending opposite direction to the others, converging at the same point in the EBP. Conversely, in the Diabetes dataset with Data Flip attack, Client simply diverges from the others, each moving towards different minima. This variation highlights the potential to identify the type of attack deployed by the malicious client based these behavioural information.
the attack type. In the Inverted Loss scenario, Client moves in the exact
Synthetic Error Behaviour
Diabetes
Error Behaviour
2nd principal component
-0.3
-0.05
9
2
-0.4
-0.10
S
-0.5
0.155
0.5
0.5
0.4
0.3
0.2
0.1
0.4
0.3
0.2
0.1
Counterfactual Behaviour
Counterfactual Behaviour
2nd principal component
40
20
20
0-
0-
-20
$
-40
-20
0
40
40
principal component
1st principal component
Figure 9: Client trajectories on Counterfactuals and Error Behavioural Planes for Synthetic, and Diabetes datasets, corresponding to Inverted- and Label-Flipping attacks, respectively. The figure highlights the deviation of the malicious client (red) from honest clients, who tend to cluster together over time.
B.4 The complementary roles of CBP and EBP
As previously observed in Figure 6, EBP and CBP provide orthogonal descriptions of client be- haviours. In the Synthetic dataset, CBP effectively identifies clusters of clients with analogous data distributions. In contrast, EBP shows all honest clients moving in the same direction, thereby obscuring individual cluster distinctions but effectively contrasting the movements of honest clients with that of the attacker. Similarly, in the Breast Cancer dataset, the identification of an attack (i.c., Crafted-noise), is more discernible through EBP, which elucidates the direction of the malicious client relative to the server. These findings highlight the distinct yet complementary roles that EBP and CBP play in the analysis of client behaviour across different datasets.
To illustrate the complexity model behaviour and the utility EBP and CBP, consider the behaviour of three different models with respect to the same input changes. For instance, a data point with features [1, 2] misclassified as class by Model A and correctly classified as class 0 by Model B. However, both models would change their predictions class if the features were altered to [2, 3].
23

---

Client score vs. Rounds MP Noise Atta ck
Client Rounds MP 1.Gradient Altack
0.25
0.20
Glen: Type
Clent Type
hores:
honast
0.10
ma leious
0.10
me idous
0.05
0.05-
0.00
0.00
20
40
60
80
150 Round
120
1.0
160
180
200
20
60
80
100 Round
340
160
180 200
Client score Rounds DD Flip Attack
Client Rounds DP LLoss Altack
0.35-
Chent type honed malidous
0.30-
0.25-
0.15-
0.20
0.10
0.15
Clent
0.10
0.05
0.05
0.00
0.001
20
40
160 100
200
100
280
Round
Figure 10: Mean and 95% confidence interval of client scores assigned by our FBSs over 200 rounds on CIFAR-10 across different attacks.
In a similar fashion to Model A, a third model, Model C, may also initially misclassify the point as class but would require a change in features to [2, -1] to alter its prediction to class 0, highlighting different sensitivities and prediction dynamics in response to input variations.
B.5 Client score visualization
FBPs provide a detailed representation of each client's behaviour, enabling deeper understanding of the conditions under which malicious clients deviate from the expected behaviour of benign clients. However, in large-scale FL systems, visual inspection may not always be necessary. Instead, statistical metrics derived from these planes can be automatically extracted and analyzed. This demonstrated in our FBSs, which extract client-specific scores to mitigate or exclude the contributions of malicious clients during aggregation. To demonstrate the scalability and efficacy of automatic detection, we evaluated our approach using the small-CIFAr 10 dataset and recorded client scores throughout the training process using a 5-fold cross- validatiiot scheme. shown in Figure 10, the extracted scores consistently identify and diminish the influence of malicious clients on the global model, as indicated by the 95% confidence interval, which varies based on the type of attack. Notably, as the model converges (with updates between rounds close to zero), more weight is given to the attacker with an inverted gradient, since its model closely resembles the global model from the previous round (plus a negligible inverted update).
B.6 Client scoring under unfair server-side validation sets
Our proposed methods rely on clean validation set the server characterize client behaviours. However, in real-world FL. scenarios, obtaining a validation set that is entirely fair and representative for each client may not be feasible. This challenge underscores the rationale behind incorporating multi-plane evaluation in our approach, which assesses client behaviour through two distinct criteria: task performance and counterfactual analysis. Traditional performance metrics in machine learning, such as accuracy, loss, or error, are heavily dependent on the validation set used (i.e., error plane). In contrast, information derived from counterfactuals more indicative the learned decision process and provides insight into the training data distribution of each client, even when the validation set is biased (i.e., counterfactual plane). Consequently, an attacker or anomalous client will exhibit distinct behaviours across both planes, particularly showing unrelated counterfactual distribution compared to other clients. Conversely, an underrepresented client will produce plausible counterfactuals similar to those of other clients, and will only be affected by the unfair validation set in the error plane.
24

---

Client score vs. Rounds MP Noise A. Unnaiir validation
Client Type
Honeal
Malidoou
Olent S.cor
0.33
Under
0.1
Figure 11: Mean and 95% confidence interval of client scores assigned by our FBSs over 200 rounds on CIFAR-10 with one attacker (MP Noise) and one unfair client (i.e., their data distribution is not in the validation set).
To validate the robustness of our defense mechanism under an unfair validation set, we conducted experiments in which data from one client was excluded from the validation set. We then analyzed the behavioural scores assigned by our method. Preliminary results, shown in Figure 11 1, indicate that the 95% confidence interval of the scores for the underrepresentee client consistently overlaps with those other honest clients, thereby distinguishing them clearly from malicious clients.
B.7 Federated Rehavioural Shields on IID and non-IID scenarios
In our main study, we focus on the non-IID setting, which is the most prevalent scenario for cross-sill silo FL in practice [40]. This setting presents considerable challenges due to the substantial differences between updates from honest clients, which can obscure the divergent hehaviours of malicious clients [41]. Consequently, we conducted experiments on the Breast Cancer dataset in both IID and non-Ill) settings using our FBPs for comparative analysis. As depicted Figure 12, we initially visualised client trajectories both behavioural planes and within the counterfactual distance space / (defined in Equation 7). The trajectories illustrate the uniform nature of honest client behaviours on both planes, distinctly highlighting the divergent behaviour of the malicious client. Notably, the visualization of the counterfactual distance space, employed by our FBSs, effectively identifies the malicious client. It indicates high distance of the attacker from all other clients, leading to a low score during the aggregation process. To further validate our approach. we compared the performance of our FBSs under No-attack, Crafted-noise. Inverted -wvertedgrradent, Label-flipping and Inverted- attacks in both IID and non-IID settings. Table demonstrates that higher accuracy is achieved in the IID setting under almost all conditions compared to the non-III setting, underscoring the increased complexity of operating in non-lID environments.
Error Behaviour
Counterfactual Behaviour
Counterfactuals Distance
2nd principal component
-0.4
0-
0.6
10
1st principal component
1st principal component
Clients
Figure 12: Client trajectories on Counterfactuals and Error Behavioural Planes for Breast Cancer datasets IID setting under Inverted-loss attacks. The figure highlights the deviation of the malicious client (red- number 6) from honest clients, who tend to cluster together over time.
B.8 Impact of server validation set size
Considering the crucial role of the validation set size on the server the creation of behavioural planes, we initially examined its impact on computational time and accuracy through 10 distinct trials. Specifically, we evaluated our FBSs (without moving average) against malicious client executing an inverted gradient attack on the Diabetes dataset. As depicted in Figure 13, computational
25

---

Table 4: Comparison of our FBSs across across five attacks types- No attack, Crasted-noise, Inverted- gradient, Label Label-flipping Inverted-loss on Breast Cancer dataset for both IID and non-lID configura- tions
Condition
No attack
MP Noise
MP I.Grad
DP Flip
DP LLoss
Mean
non-IID
95.7+1.11
98.0+1.88 98.43-+0.4
95.3 98.2++0.2
9.6
95.9-L0.9
95.8
IID
98.2++0.3
96.4+0.9
93.7+11.0
97.0+10.4
time increases exponentially with the size of the dataset. However, a dataset containing as few as 250 samples is sufficient to adequately represent the data distribution, achieving performance par with that observed in larger datasets. Additionally, with fewer than 1000 samples, our method demonstrates greater efficiency compared to the Krum algorithm.
Test Set Size Computational Time & Accuracy
Computational Time (log10
3.75
2,02
8:75
Krum
335250
300
1000
2000
4008
Test Set Size
Figure 13: Relationship between test set size, computational time, and accuracy evaluated on Diabetes dataset against inverted gradient attack. As test set size increases (logarithmic x-axis), computational time grows exponentially (log10 scale, red line), while accuracy (blue line) only slightly increases before plateauing.
B.9 Impact of the window length in the moving average
We analyse the impact of window length on our FBSs and its two variations, one using only predictive performance (error) and the other focusing solely on decision maaking processes (counterfactual), The results presented Figure 14 depict the average accuracy across various conditions including No attack, Label-flipping, Inverted-loss, Crafted-noise, and Inverted inverted-gradient-t graddiett--frr window lengths ranging from to 30 rounds. While no definitive patterns are clearly evident, on average, longer window length tends to be advantageous for accurately assessing the behavioural scores of each client during the aggregation process. Noteworthy, the optimal accuracy across both datasets was achieved with window length of 30 rounds. Particularly, a consistent and slight increase in accuracy observed in the Diabetes dataset when utilising only the counterfactual information.
B.10 Impact of the local epochs in Federated Behavioural Shields
Local training epochs play a crucial role in FL, particularly in evaluating client behaviours. An increased number of local epochs implies greater adaptation of the client-model to its local data distribution. For this reason, we assess the performance of our FBSs against data poisoning attacks, including Label-flipping and Inverted-loss, across Breast Cancer, Diabetes, and Synthetic datasets. Metrics are reported as the mean and standard error across ten with distinct parameters' initialization. These attack types were selected because malicious updates are directly influenced by the number of local epochs, similar to updates from honest clients; differently, the behaviour malicious clients in model poisoning might remain unaffected. As depicted in Figure 15, the impact of local epochs varies across datasets, with each achieving maximum accuracy at different numbers epochs. Generally, accuracy increases with the number of local epochs until it reaches a peak, after which it begins to
26

---

Impact Window Length Diabetes Dataset
Imsact Window Length Breast Dataset
10
Accuraty (%)
Arnney
70
Mix
Mx
Countwrorhhl
Counertrctol
Emer
15 Window Length
Window
Figure 14: Effect of window length on accuracy using FBSs. This table demonstrates the accuracy changes as function of window length in the moving average method across the Diabetes and Breast Cancer datasets.
Impact of Local Epochs Across Datasets
Cubss
Accuracy (%)
shoc
-00
80
50
12
14
Local Epochs
Figure 15: Impact of varying local epochs accuracy for our FBSs. This table illustrates how changes in the number of local training epochs affect accuracy across the Breast Cancer, Diabetes, and Synthetic datasets.
decline. This trend may be attributed to the fact that initially, increasing the number of local epochs enhances the detectability of anomalies in the behaviour of malicious clients, their models diverge more quickly from those of honest clients. However, beyond a certain threshold, all client models become highly customd, leading the potential underweighting of even honest clients' contributions during the aggregation process on the server. This results the system relying predominantly on a few clients whose models are the most similar to each other.
B.11 Impact of Attack Intensity on Federated Behavioural Shields
We conducted experiments to evaluate the sensitivity of the proposed method to varying levels of attack intensity. Specifically, we focused on model poisoning attacks by systematically increasing the noise parameter (5) injected by adversaries into the global model prior transmission to the server (noise = (0, 8- o w ))) The evaluation was performed using 5-fold cross-validation Cancer and smalI-MNIST datasets. As shown in Figure 16, increasing the attack intensity results in noticeable decrease in the performance of the global model when using the standard FedAvg approach. In contrast, our method-burt when utilizing all feature planes and when restricted to the counterfactual plane remains stable and unaffected by the attack intensity. Interestingly, marginal improvement in accuracy is observed as the attack intensity increases, likely because more aggressive modifications render malicious models more degraded and, therefore, easier detect.
the Breast
B.12 Ablation study
We conducted an ablation study to examine the impact of various components on efficacy of our algorithm. These components include predictive performance (error), decision-making process (counterfactuals), and the application of moving average. Table presents the average accuracy and standard error under five experimental conditions: No-attack, Trafted-noise inverted-gradient
27

---

Noise Intensity
Intensity -SaSl-MNIST
98
Accuracy (N)
Accuracy (8)
94-
92
90
red/ug
Fedfug
08-
Durs
Ours
(cf)
(cf)
88
0.3
05
08
166
0.3
05
08
16
Noise Intersity (B)
Noise Intersity (8)
(a)
(b)
Figure 16: Impact of attack intensity on FBS and traditional FedAvg in (a) the Breast Cancer dataset and (b) the small-MNIST dataset. Notably, unlike FedAvg, FBS maintains stable accuracy as the attack intensity increases.
Label-fipppng, and Inverted-loss, comparing different variations of our method against baselines on the Breast Cancer and Diabetes datasets.
Notably, relying solely on predictive performance does not yield statistically significant benefit compared to the strongest baselines, such as RFA on Breast Cancer and Krum on Diabetes. In contrast, counterfactual information provides deep insights into client behaviour, enabling our method to outperform all baselines in identifying malicious clients. Although the improvement is subtle, the combination of descriptors, on average. provides better results than using counterfactuals alone.
The integration of moving average notably enhances the performance of our method, improving from 94.5 = 0.5 95.4 0.3 on the Breast Cancer dataset. Smaller gains are observed on the Diabetes dataset, likely due to two factors: the limited representativeness of the clean validation set on the server for Breast Cancer, which consists of only 89 samples compared to 250 for Diabetes, and the small size of client datasets (77 training samples on average), which may lead to unstable local training. By aggregating behaviour across multiple rounds, the moving average technique helps to stabilise and accurately assess client behaviour scores.
Table 5: Average accuracy (%) = standard error for various defense strategies under five experimental conditions: No attack, Crafted-noise, Inverted-gradient, Label-flippigg, and Inverted-loss. The table specifically compares the performance of our methods with and without the application of a moving average (MA).
Defense Strategy
Breast
Diabetes
Krum
91.9 + 0.4
70.6 0.3
Median
92.6 0.4
67.0 0.1
Trimmed-ment meaan
93.4 0.4
69.0 0.1
RFA
93.5 0.5
67.5
Ours (error) w/o MA
93.7 0.4
7..5
Ours (cf) w/o MA
94.2 0.4
0.1
Ours w/o MA
94.5 0.5
7.2
Ours (error)
95.0 0.4
7..4
Ours (cf)
95.3 0.3
72.2 0.1
Ours
95.8 0.4
72.4 0.1
B.13 Comprehensive tabular analysis of defense mechanisms
In this section, we provide detailed numerical breakdown of the results depicted in Figure This quantitative analysis aims to supplement the visual data presented, offering precise values and statistical insights that underpin the observations and conclusions discussed throughout the paper. Tables 6 for Breast Cancer, 7 for Diabetes, 8 for SmalI-MNIST, and 9 for small-CIFAR-111 present comprehensive comparison our FBSs and streamlined versions using only the CBP or the EBP. These are evaluated against traditional defenses such as Krum, Median, Trimmed-men RFA across
28

---

five scenarios: No attack, Crafted-noise, Inverted gradient, Label Label-flipping, and Inverted-loss This comparison elucidates the effectiveness our approach under range of conditions, both adversarial and non-adversarial.
invertedgraddent,
Table 6: Comparison FBSs and its simpler version with only counterfactuals (cf) or predictive- performance (error) versus Krum, Median, and Trimmed-mean defenses actoss five attacks types- No attack, Crafted-noise, inverted-gradient Label-flipping Inverted-loss on Breast Cancer dataset.
Model Poisoning
Data Poisoning
No-Attack
Crafted-Noise
Inv. Grad.
Label-Flip
Inv. Loss
Mean
Krum
90.9:10.6
92.3
92.7+:1.5
91.611..
91.8-0.6
91.911.44
Median
92.7+:0.7
93.6:0..7
93.111.22
92.0=1..8
91.61111
92.611.4
Trim
93.2.10.5
96.1
95.5:10.4
91.811111
90.410.8
93.4:10.3
RFA
9..3:t1.5
94.1+:0.9
93.61t.11
93.6110.9
91.8+110
93.5+1..5
Ours (error)
96.6+:0.8
97.7+1.2
95.0=10.8
91.4#1.0
94.1-0.8
95.0=1.44
Ours (cf)
97.5+0.55
96.8+0.6
96.611..3
93.210.9
92.310.6
95.3
Ours
95.7#11.1
98.0
95.3+0.7
94,2+0.6
95.9-0.9
95,8+0.4
Predictor
96.6+ 0.4
N/A
N/A
N/A
N/A
N/A
Table 7: Comparison of FBSs and its simpler version with only counterfactuals (cf) or predictive- performance (error) versus Krum, Median, and Trimmed-mean defenses five attacks types-No attack, Crafted-noise, -nvereedggrddient, Label-flipping, Inverted-loss-Oe -n Diabetes dataset.
Model Poisoning
Data Poisoning
No-Attack
Craited-Noise
Inv. Grad.
Label-Flip
Inv. Loss
Mean
Krum
7.2
70.4+0.2
71.0++0.2
70.5+0.2
70.7=0.1
70.6+1..3
Median
69.7+..2
68.5+60.3
69.2:t0.1
64.1110.1
65.1
6..
Trim
67.9+..1
69.6+10..4
71.7++0.
65.14.01
70.5++02
69.0+0.1
RFA
70.5+002
69.7+002
69.8+-0.2
65.1+-02
62.2+0..1
67.5+00.1
Ours (error)
71.4+0.6
72.0+00.5
72.2+00.4
72.3++0.2
69.3+0.88
71.4+0.2
Ours (cf)
72.740.2
72.3:0.2
72.410.0
71,8+10.2
71.6+10.2
72.2+0.1
Ours
72.51:0.3
73.1
72.3+0.0
72.0+0.44
71.9
72.4+0.1
Predictor
73.5+0.2
N/A
N/A
N/A
N/A
N/A
Table 8: Comparison of FBSs and its simpler version with only counterfactuals (cf) or predictive- performance (error) versus Krum, Median, and Trimmed-mean defenses across five attacks types- -No attack, Crafted-noise, Inverted-gratie gradient, Label-ftipping, Inverted-loss on small-MNIST dataset.
Model Poisoning
Data Poisoning
No-Attack
Crafted-Noise
Inv. Grad.
Label-Flip
Inv. Loss
Mean
Krum
56.3+2.4
1..
62.0
59.3+2.5
55.9+3.0
18.8+1.0
Median
1..8
69.5+2.0
63.7
64.2+1.7
64.5+1.0
66.1100.
Trim
68.7+22.4
78.9++2.3
77.7+2.0
63.9+1.9
70.1+3.00
71.9+111
RFA
73.8+:2.9
72.2+11.8
74.41:1.4
66.6:15.3
72.0#:2.1
71.8
Ours (error)
80.2+1.5
71.0++4.2
71.3+00.3
76.0+2222
72.9=t4.7
7..3+
Ours (cf)
84.2+0.2
84.1+50.4
74.5+1..
77.8+1.7
80.1=0.6
80.1+0.5
Ours
81.0+1..0
79,24-3.8
74.6
74.6
77.4
77.4
Predictor
79.6-10.8
N/A
N/A
N/A
N/A
N/A
29

---

Table 9: Comparison of FBSs and its simpler version with only counterfactuals (cf) or predictive- performance (error) versus Krum, Median, Trimmed-mean, and RFA defenses across five attack types-No attack, Crafted-noise, Inverted-gradient, Label-flipping, Inverted-loss-on small-CIFAR- 10 dataset.
Model Poisoning
Data Poisoning
No-Attack
Crafted-Noise
Inv. Grad.
Label-Flip
Inv. Loss
Mean
Krum
90.6+22.0
90.7+11.3
92.1#1.1
93.0=#1.3
85.3
90.
Median
96.5.10.4
97.0+L0.4
96.1+0.2
96.0.10.7
80.6
93.2+10.5
Trim
96.5+H0.3
96.9+L0.3
97.4
96.2.10.6
86.3+33.4
94.7+10.3
RFA
96.3+H0.4
97.24-0.3
97.0++0.3
96.3-10.8
83.8+3.66
94.1400.3
Ours (error)
97.4+0.2
97.1+0.2
96.1++0.4
97.3++0.5
89.8+11.2
95.5++..1
Ours (cf)
98.240.2
97.4
97.610.1
97.6-10.2
91.8+00.8
96.
Ours
97.410.33
97.9
97.3+0033
97.2110.4
91.7=11.0
96.3
Predictor
97.510.33
N/A
N/A
N/A
N/A
N/A
30

---

NeurIPS Paper Checklist
The checklist is designed encourage best practices for responsible machine learning research, addressing issues of reproducibility, transparency, research ethics, and societal impact. Do not remove the checklist: The papers not including the checklist will be desk rejected. The checklist should follow the references and follow the (optional) supplemental material. The checklist does NOT count towards the page limit.
Please read the checklist guidelines carefully for information on how to answer these questions. For each question in the checklist:
You should answer [Yes], [No], or [NA].
[NA] means either that the question Not Applicable for that particular paper or the relevant information is Not Available.
Please provide short (1-2 sentence) justification right after your answer (even for NA).
The checklist answers are an integral part of your paper submission. They are visible to the reviewers, area chairs, senior area chairs, and ethics reviewers. You will be asked to also include (after eventual revisions) with the final version of your paper, and its final version will published with the paper.
The reviewers of your paper will be asked use the checklist as one of the factors their evaluation. While "[Yes] is generally preferable to "[No]" ", it is perfectly acceptable to answer "[No] providedd proper justification is given (e.g.. "error bars are not reported because it would be too computationally expensive" or "we were unable to find the license for the dataset we used"). In general, answering binary way, we
"[No] or "[NA]" is not grounds for rejection. While the questions are phrased acknowledge that the true answer is often more nuanced, please just use your best judgment and write justification to elaborate. All supporting evidence can appear either in the main paper or the supplemental material, provided in appendix. If you answer [Yes] to question, in the justification please point to the section(s) where related material for the question can be found.
IMPORTANT, please:
Delete this instruction block, but keep the section heading "NeurIPS paper checklist",
Keep the checklist subsection headings, questions/answers and guidelines below.
Do not modify the questions and only use the provided macros for your answers.
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: Section 5
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results be expected to generalize other settings.
is fine include aspirational goals as motivation long is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations the work performed by the authors? Answer: [Yes]
Justification: Section 6.2
31

---

Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, c.g., if the approach was only tested few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails handle technical jargon.
The authors should discuss the computational efficiency the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory Assumptions and Proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [NA]
Justification:
The paper does not provide theoretical results.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proofs can either appear in the main paper the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should complemented by formal proofs provided in appendix supplemental material.
Theorems and Lemmas that the proof relies upon should properly referenced.
4. Experimental Result Reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that it affects the main claims and/or conclusions the paper (regardless whether the code and data are provided or not)?
Answer: [Yes]
Justification: Section 3, Appendix A.
Guidelines:
The answer NA means that the paper does not include experiments.
32

---

If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access a hosted model (e.g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurlPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily new algorithm, the paper should make it clear how to reproduce that algorithm. (b) the contribution is primarily a new model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution is new model (e.g., a large language model), then there should either way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome describe the particular way they provide for reproducibility. the case of closed-source models, it may be that access to the model is limited in some way (e.g., registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [Yes]
Justification:
We provide the code
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://nips.c cc/ Suuidess/ooeSSumissttt
for more details.
While we encourage the release of code and data, we understand that this might not be possible, so "No" an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https: (woodeSumisssionll.....
/ //nips. /twpss.ccppub//////// https/cccppbllcguuueeeeeeoeeeeee
for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
33

---

Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental Setting/Details
Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimiser, etc.) necessary to understand the results?
Question:
Answer: [Yes]
Justification: Appendix A
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented the core the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, supplemental material.
7. Experiment Statistical Significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [Yes]
Justification: Table 1, 2, 4, 6, 8, 8, Figure 2, 5, 8, 10, 11, 13, 14, 15, 16.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results accompanied by error bars, confi- dence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
It should be clear whether the error bar is the standard deviation or the standard error of the mean.
OK to report 1-sigma error bars, but one should state The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that out of range (e.g. negative error rates).
If error bars reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables the text.
8. Experiments Compute Resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification: Appendix A.5
Guidelines:
The answer NA means that the paper does include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
34

---

The paper should provide the amount of compute required for each of the individual experimental runs as well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make into the paper).
9. Code Of Ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https: //pprrip///uiip..../ cc/
tttpss//eerrpp..cppbbiii wttta///tthiiiiiieeeeeeeeeeeeeee ines?
Answer: [Yes]
Justification:
We have thoroughly reviewed and adhered the NeurIPS Code of Ethics
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No. they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., there a special consid- eration due to laws regulations in their jurisdiction).
10. Broader Impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification: Section 5 and Section 6
Guidelines:
The answer NA means that there no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimising neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility ML).
11. Safeguards
Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, scraped datasets)?
Answer: [NA]
Justification: The paper does not address safeguards for the responsible release of high-risk data models because our research does not involve such risks.
Guidelines:
35

---

The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards challenging, and many papers do not require this, but encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification: see Appendix A.4, A.5.
Guidelines:
The answer NA that the paper does existing
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset used and, possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (c.g.. website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswi thcode. com/ datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that re- re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided. If this information is not available online, the authors are encouraged to reach out to the asset's s creators.
13. New Assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes]
Justification: Appendix A.4, A.5 and code
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset used.
At submission time, remember to anonymize your assets (if applicable). You can either create anonymized URL or include anonymized zip file.
14. Crowdsourcing and Research with Human Subjects
For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and sereenshots, if applicable, as well details about compensation (if any)?
Question:
Answer: [NA]
36

---

Justification:
The paper does not involve crowdsoureing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included the main paper.
According to the NeurIPS Code of Ethics, workers involved data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: The paper does not involve research with user study.
Guidelines:
The answer NA means that the paper does not involve crowdsoureing nor research with human subjects.
Depending the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such the institution conducting the review.
37


Reviews:
[{"review_id": "h19ZNAapCX", "reviewer": "Reviewer_eMJr", "summary": "This paper introduces a novel method called Federated Behavioural Planes (FBPs) to analyze, visualize, and explain the dynamics of client behavior in Federated Learning (FL) systems. The primary contributions of the paper are as follows:\n\n* 1\\. Introduction of Federated Behavioural Planes (FBPs): FBPs consist of two planes:\n     * 1.1\\. Error Behavioural Plane (EBP): This plane analyzes the predictive performance of client models by visualizing the errors they produce.\n\n     * 1.2\\. Counterfactual Behavioural Plane (CBP): This plane examines the decision-making processes of client models through counterfactual explanations, highlighting how decision boundaries are formed.\n* 2\\. Visualization and Analysis: FBPs provide informative trajectories that describe the evolving states of clients, enabling the identification of clusters of clients with similar behaviors. This helps in understanding both beneficial and detrimental behaviors in FL systems.\n\n* 3\\. Federated Behavioural Shields (FBS): Based on the patterns identified by FBPs, the authors propose a robust aggregation technique named Federated Behavioural Shields. This technique enhances security by detecting malicious or noisy client models and surpasses the efficacy of existing state-of-the-art FL defense mechanisms.\n\n* 4\\. Experimental Validation: The paper demonstrates through experiments that FBPs can effectively track client behavior, identify client clusters, and improve the security and performance of FL systems. The proposed FBS method outperforms other robust aggregation methods in defending against various types of attacks.\n\nOverall, the paper offers a comprehensive approach to enhance understanding, trust, and control over federated learning systems by introducing a novel method to analyze and secure client behaviors.", "questions": "**Questions for the Authors:**\n* 1\\. **Computational Overhead:**\n    - Question: How significant is the computational overhead introduced by the concurrent training of counterfactual generators with the main predictive models?\n    - Suggestion: Provide a detailed comparison of the computational requirements of your method with baseline methods. Include any potential optimization strategies to mitigate this overhead.\n\n* 2\\. **Scalability:**\n    - Question: How does the proposed method scale with a larger number of clients and more complex datasets?\n    - Suggestion: Include a scalability analysis or discussion that addresses the performance and feasibility of FBPs and FBS in larger, real-world federated learning scenarios. Simulations or theoretical models predicting the system's behavior in large-scale deployments would be helpful.\n\n* 3\\. **Applicability to Different Models:**\n    - Question: Is the proposed method applicable to different types of models beyond neural networks, such as decision trees or support vector machines?\n    - Suggestion: Discuss the generalizability of FBPs and FBS to various model types. If there are limitations, explain the necessary modifications to apply the method to other model architectures.\n\n* 4\\. **Evaluation Metrics:**\n\n    - Question: Have you considered additional evaluation metrics that capture communication efficiency, latency, and energy consumption in your analysis?\n    - Suggestion: Introduce and discuss additional metrics to provide a comprehensive assessment of the system’s performance in practical applications. This would strengthen the evaluation of the proposed methods.\n\n* 5\\. **Interpretability and Usability:**\n\n    - Question: How can the methods be made more interpretable and usable for practitioners who are not experts in federated learning or explainable AI?\n    - Suggestion: Provide more intuitive explanations and visualizations. Including case studies or practical examples of FBPs and FBS in real-world scenarios could enhance understanding and applicability for a broader audience.\n\n* 6\\. **Real-World Validation:**\n\n    - Question: Have you conducted any preliminary real-world validations of the proposed methods? If not, what are the plans for such validations?\n    - Suggestion: Discuss any real-world applications or preliminary results. Outline a detailed plan for future real-world validation studies to demonstrate the practical applicability and effectiveness of your methods.\n\n* 7\\. **Comparison with State-of-the-Art:**\n\n    - Question: How does the proposed Federated Behavioural Shields compare with other state-of-the-art defense mechanisms under different attack scenarios?\n    - Suggestion: Provide a detailed comparative analysis with other robust aggregation methods under various attack types. Highlight the strengths and potential weaknesses of your approach in comparison to existing techniques.\n\n* 8\\. **Visualizations and Trajectories:**\n\n    - Question: Can you provide more detailed examples or visualizations of client trajectories in the Error Behavioural Plane and Counterfactual Behavioural Plane?\n    - Suggestion: Include more visual examples and explanations of client trajectories to illustrate how FBPs can be used to identify different client behaviors and detect malicious clients.\n\n* 9\\. **Impact of Non-IID Data:**\n\n    - Question: How does the method handle non-IID (non-independent and identically distributed) data across clients, and how robust is it to such scenarios?\n    - Suggestion: Discuss the impact of non-IID data distributions on the performance of FBPs and FBS. Provide experimental results or theoretical analysis demonstrating the method’s robustness to non-IID data.\n\n* 10\\. **Future Directions:**\n\n    - Question: What are the future directions or potential extensions of your work?\n    - Suggestion: Outline possible future research directions or extensions of FBPs and FBS. This could include integrating additional behavioral planes, optimizing computational efficiency, or exploring new applications of the methods.\n\n**Conclusion:**\nAddressing these questions and suggestions can provide clarity on the strengths and limitations of the proposed methods, enhance the understanding of their practical applicability, and offer insights into potential improvements. Engaging with these points during the rebuttal phase can lead to a productive discussion and potentially strengthen the overall contribution of the paper.", "limitations": "**Assessment of Limitations and Potential Negative Societal Impact:**\n\nBased on the provided content and the NeurIPS checklist guidelines on limitations and broader societal impacts, here is an assessment of how well the authors have addressed these aspects and suggestions for improvement:\n\n**Addressing Limitations:**\n\n- **Identified Limitations:** The paper acknowledges some limitations, such as the computational overhead introduced by the concurrent training of counterfactual generators and the potential scalability issues in larger federated learning deployments.\n\n- **Suggestions for Improvement:**\n    - **Computational Efficiency:** Provide a more detailed discussion on the computational requirements and potential optimization strategies. This could include specific techniques to reduce overhead, such as model pruning, quantization, or distributed optimization methods.\n\n    - **Scalability Analysis:** Include more comprehensive scalability experiments or simulations that predict the method's performance in larger, real-world settings. Discuss how the method can be adapted or optimized for large-scale deployments.\n\n**Potential Negative Societal Impact:**\n- **Security and Privacy:** The primary focus of the paper is on enhancing security and privacy in federated learning, which is a positive societal impact. However, potential negative impacts, such as the misuse of federated learning systems or unintended biases in the models, should be considered.\n\n- **Suggestions for Improvement:**\n    - **Bias and Fairness:** Discuss the potential for unintended biases in federated learning models and how the proposed methods could mitigate or exacerbate these biases. Provide suggestions for ensuring fairness in federated learning deployments.\n    - **Misuse of Technology:** Address the potential for misuse of federated learning systems, such as using the technology for surveillance or other harmful purposes. Discuss safeguards and ethical considerations to prevent misuse.\n\n**Constructive Suggestions for Improvement:**\n* 1\\. **Detailed Discussion on Limitations:**\n\n    - Expand the discussion on identified limitations, providing more details on computational efficiency and scalability. Include theoretical analysis or empirical evidence supporting the claims and potential solutions.\n* 2\\. **Bias and Fairness:**\n\n    - Add a section discussing the potential for biases in federated learning models. Explain how the proposed methods could impact fairness and provide guidelines or best practices for ensuring equitable outcomes.\n* 3\\. **Ethical Considerations and Misuse:**\n\n    - Address potential misuse of federated learning technology. Discuss ethical considerations and propose safeguards to prevent the harmful application of the technology. Highlight the importance of transparency and accountability in federated learning deployments.\n* 4\\. **Societal Impact Statement:**\n\n    - Include a comprehensive societal impact statement that covers both positive and negative aspects. This should highlight the benefits of enhanced security and privacy, as well as address the potential risks and ethical concerns.\n\n**Conclusion:**\nWhile the paper makes significant contributions to enhancing security and privacy in federated learning, it could benefit from a more thorough discussion of limitations and potential negative societal impacts. By addressing these areas, the authors can provide a more balanced view of their work and ensure that it is applied responsibly and ethically.", "rating": 7, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "The paper \"Federated Behavioural Planes: Explaining the Evolution of Client Behaviour in Federated Learning\" demonstrates several strengths across different dimensions:\n\n* 1\\. Originality: The paper introduces a novel approach, Federated Behavioural Planes (FBPs), to analyze and visualize client behavior dynamics in Federated Learning systems. This method offers a unique perspective on understanding client behavior evolution in FL, which is a relatively unexplored area in the existing literature. The combination of predictive performance analysis and decision-making process evaluation through FBPs showcases originality in addressing the challenges of client behavior in FL systems.\n* 2\\. Quality: The paper maintains a high standard of quality in terms of methodology, experimental design, and theoretical framework. The introduction of FBPs as a tool to explain the dynamics of FL systems reflects a well-thought-out approach to addressing the evolving behavior of clients in federated learning environments. The robust aggregation mechanism proposed, Federated Behavioural Shields, demonstrates a quality solution to enhance security in FL systems.\n* 3\\. Clarity: The paper is well-written and structured, making it easy for readers to follow the concepts presented. The clarity in explaining the Federated Behavioural Planes framework, the experimental results, and the implications of the proposed method enhances the overall understanding of the research. The use of figures and explanations aids in visualizing complex concepts related to client behavior in FL systems.\n* 4\\. Significance: The paper's contribution to the research area of Federated Learning is significant. By introducing FBPs and Federated Behavioural Shields, the paper addresses a key challenge in FL systems - understanding and controlling client behavior. The insights provided by FBPs and the improved security offered by Federated Behavioural Shields have the potential to enhance the reliability and control over FL systems, making a valuable contribution to the field.\n\nOverall, the paper makes a substantial contribution to the field of federated learning. Its originality lies in the novel problem formulation and creative combination of existing ideas. The quality of the research is demonstrated through rigorous methodology and comprehensive experiments. The clarity of the presentation ensures that the contributions are accessible to a broad audience. The significance of the work is underscored by its potential impact on improving the security and efficiency of federated learning systems.", "weaknesses": "**Complexity of Methods:**\n - **Computational Overhead**: The concurrent training of counterfactual generators with the main predictive models introduces significant computational overhead. This could be particularly burdensome in real-world federated learning settings where resources are limited. The paper could benefit from a more detailed analysis of the computational costs and potential optimization strategies to mitigate this overhead.\n    - **Actionable Insight**: Consider providing a detailed comparison of the computational requirements of the proposed method with baseline methods. Explore possible optimizations or approximations that could reduce the overhead without significantly compromising the performance.\n\n**Real-World Applicability:**\n- **Scalability Concerns**: While the experiments are comprehensive, they are conducted on relatively small datasets and a limited number of clients. This raises concerns about the scalability of the proposed methods to larger, real-world federated learning scenarios with many clients and more complex data distributions.\n    - **Actionable Insight**: Include a discussion on the scalability of FBPs and FBS. Consider performing a scalability analysis, even if only theoretical, to predict the performance and feasibility of the methods in larger settings. Additionally, simulations or theoretical models could provide insights into expected behavior in large-scale deployments.\n\n**Generalization Across Different Models:**\n- **Model-Specific Limitations**: The proposed method may be tailored to specific types of models (e.g., neural networks) and may not generalize well to other types of models used in federated learning (e.g., decision trees, support vector machines).\n    - **Actionable Insight**: Discuss the applicability of FBPs and FBS to different types of models. Providing a broader range of experiments that include different model architectures could strengthen the paper. If certain models are not compatible, explain the limitations and potential modifications required for broader applicability.\n\n**Evaluation Metrics:**\n- **Limited Evaluation Metrics**: The evaluation primarily focuses on standard metrics like accuracy and robustness against attacks. While these are important, they might not capture all aspects of the system's performance, such as the impact on communication efficiency, latency, and energy consumption.\n    - **Actionable Insight**: Introduce additional evaluation metrics that capture the holistic performance of the system, including communication overhead, latency, and energy consumption. This would provide a more comprehensive assessment of the practicality of the proposed methods in real-world applications.\n\n**Interpretability and Usability:**\n- **Interpretability for Non-Experts:** The paper, while clear in its technical explanations, may still be challenging for practitioners who are not experts in federated learning or explainable AI. Enhancing the interpretability and usability of the methods for a broader audience could be beneficial.\n    - **Actionable Insight:** Provide more intuitive explanations and visualizations of the key concepts and methods. Including case studies or practical examples demonstrating the application of FBPs and FBS in real-world scenarios could make the methods more accessible and easier to understand for non-experts.\n\n**Real-World Validation:**\n- **Lack of Real-World Validation**: The experiments are conducted in controlled settings, which may not fully represent the challenges and variability encountered in real-world federated learning deployments.\n\n    - **Actionable Insight**: Discuss potential real-world applications and the expected challenges. If possible, provide preliminary results or insights from deploying the methods in a real-world scenario. Alternatively, outline a detailed plan for future real-world validation studies.\n\n**Conclusion:**\nWhile the paper presents significant contributions to federated learning, addressing these weaknesses could enhance its impact and practicality. By focusing on computational efficiency, scalability, broader model applicability, comprehensive evaluation metrics, interpretability, and real-world validation, the work can move closer to its stated goals and become more robust and applicable to a wider range of scenarios.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "_Computational Overhead and Evaluation Metrics, the paper could benefit from a detailed analysis of the computational costs._\n\nAs suggested, __we performed a detailed analysis of the computational overhead of our framework, examining all its components__: Local Computation, Communication Overhead, and Server-side Computation. Our evaluation metrics include GFLOPs for local inference, megabytes for communication, time complexity for the aggregation process, and the duration of one training round. We also conducted a comprehensive comparison with other methods. Please refer to the common comment (@Rev-eMJr,ULpn,aREF) for detailed results and discussions.\n\n\n_Scalability concerns. “While the experiments are comprehensive, they are conducted on relatively small datasets” Question: “How does the proposed method scale with a larger number of clients and more complex datasets?”_\n\n__We enhance the robustness of our method’s validation by: 1) Introducing a more complex dataset, CIFAR-10. 2) Adding a new baseline, [K. Pillutla et al., RFA, 2022], for all experiments. 3) Experimentally demonstrating that our methods scale effectively with a larger number of clients.__\nWe introduced an experiment on CIFAR10, which is a larger dataset and a recent FL benchmark [K. Pillutla et al., RFA, 2022].  This experiment significantly improves the robustness of our validation, and Fig. A further demonstrates our method’s effectiveness in complex FL scenarios. Our method outperforms or matches other baselines across all datasets (Breast, Diabetes, small-MNIST, small-CIFAR10) and conditions (No attack, MP noise, MP inverted gradient, DP label flipping, DP inverted loss), except for Inverted gradient attacks in small-MNIST, where Trimmed mean performs better (note that this is 1 out of 20 cases).\nAdditionally, we tested the scalability of our framework with up to 200 clients. As shown in Fig. D, our method introduces 1 extra minute per round with 200 clients, compared to FedAvg. In contrast, the baseline Krum introduces over 15 minutes per round. This demonstrates the efficiency and scalability of our approach.\n\n_Generalization across different models._\n\n__Our method, while focused on neural networks (NNs) and traditional FL, theoretically applies to other ML models like decision trees, though it requires appropriate aggregation processes and counterfactual (CF) generation.__\nWe focused our study on NNs and traditional FL, using differentiable models capable of producing a global model through parameter aggregation. However, extending our method to other ML models, such as decision trees, presents an interesting and challenging opportunity for future work due to their discrete model parameters (e.g., splitting rules and thresholds). Our proposed method is based not on the similarity between client model parameters, which can be problematic with such heterogeneous architectures, but on the client model’s performance and their respective decision-making process. Therefore, in theory, our defense method (FBSs) of evaluating client behaviors still holds with these ML models but requires a suitable aggregation process and CF generation. Various aggregation methods have been proposed in the literature for such models, which can be adapted for this purpose [Wang et al., Decision Tree-Based Federated Learning, 2024]. Additionally, generating CFs from interpretable models, such as decision trees, is straightforward due to their inherently explainable nature.\n\n_Interpretability and Usability. “The paper, while clear in its technical explanations, may still be challenging for practitioners who are not experts”._\n\nTo broaden the audience and facilitate the comprehension of our method, __we introduce the pseudocode of our algorithm (Algorithm 1) for creating the behavioral planes on the server and implementing our robust aggregation method (FBSs).__\n\n_Visualizations and Trajectories. Question: Can you provide more detailed examples or visualizations of client trajectories in the Error Behavioural Plane and Counterfactual Behavioural Plane (CBP)? Suggestion: Include more visual examples and explanations of client trajectories_\n\nIn addition to the trajectories presented in Fig. 3 of the main paper, __we provided additional trajectories in the Appendix B (Fig. 8)__. Specifically, Fig. 8 illustrates client trajectories within the FBPs for different scenarios, including an Inverted Loss attack on the Synthetic dataset and a Data Flip attack on the Diabetes dataset, highlighting the distinct behavioral patterns of clients and attackers in these settings. These trajectories also demonstrate the possibility of identifying clusters of clients with similar data distributions, particularly on the CBP.\n\n_How does the method handle IID and non-IID data?_\n__Our method is robust in both IID and challenging non-IID scenarios, demonstrating higher accuracy in IID settings across most conditions.__\nPlease note that in the main paper, we conducted all experiments in non-IID scenarios (Section 4.1), which are the most realistic yet challenging for robust aggregation methods, as even honest clients behave differently. Additionally, we compared the performance of our defense method under No-attack, Crafted-noise, Inverted-gradient, Label-flipping, and Inverted-loss attacks in both IID and non-IID settings. The table below shows that higher accuracy is achieved in the IID setting under almost all conditions compared to the non-IID setting, highlighting the increased complexity of operating in non-IID environments (Appendix B).\n\n| Cond.   | No attack | MP Noise | MP Grad | DP Flip   | DP I.Loss | Mean     | \n|---------|-----------|----------|---------|-----------|-----------|----------|\n| non-IID | 95.7±1.1  | 98.0±0.8 | 95.3±0.7 | 94.2±0.6  | 95.9±0.9  | 95.8±0.4 |\n| IID     | 98.2±0.3  | 98.4±0.4 | 98.2±0.2 | 96.4±0.9  | 93.7±1.0  | 97.0±0.4 |", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "MVedD5GeaC", "reviewer": "Reviewer_ULpn", "summary": "This paper introduces a novel method called Federated Behavioural Planes (FBPs) for analyzing, visualizing, and explaining the dynamics of Federated Learning (FL) systems. FBPs are consist of Error Behavioural Plane (EBP), reflecting the model’s predictive performance, and Counterfactual Behavioural Plane (CBP), reflecting the decision-making processes. Using insights from FBPs, the paper also proposes a new robust aggregation technique called Federated Behavioural Shields (FBS) to enhance security in FL systems.", "questions": "- Is the counterfactual generator learned on the server? Why would a generator whose goal is for better interpretability affect the performance of predictor?\n- How is the local CL’s accuracy computed, and why is it so much lower than FL in Table 1?", "limitations": "Most limitations of the proposed method (validation set, computational overhead) have been addressed. Please refer to Weaknesses for other limitations of this work. There is no potential societal impact.", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 2, "contribution": 2, "strengths": "+ Novel approach: The Federated Behavioural Planes (FBPs) introduce a new way to analyze and visualize client behaviors in Federated Learning, addressing a gap in existing literature.\n+ Theoretical foundation: The authors provide a theoretical framework for understanding FL dynamics, grounding their practical approach in solid mathematical concepts.\n+ Explanatory power: FBPs allow for visual identification of client clusters and trajectories, enhancing interpretability of FL systems.", "weaknesses": "- The paper heavily relies on visual representations (FBPs) to explain client behavior, which makes it infeasible to scale to large-scale FL systems where the convergence might take more rounds and client selection takes place in each round. These factors will largely make the trajectory on the planes hard to keep track of.\n\n&nbsp;\n\n- The computational overhead is too large. \n  + In each training round, the counterfactuals are required to be computed for the locally updated model from every client. And the differences between the counterfactuals from each pair of clients are also computed. For cross-silo FL settings, this is already a large computational overhead. For cross-device settings, I do not think this process is affordable especially for high dimensional data like image data.\n  + Besides, since the paper focuses primarily on detecting anomalies and enhancing security from attackers, the main battlefield of this method should be more on the cross-device setting where the number of clients is large. To this end, the prohibitive computational overhead mitigates the motivation of this work.\n\n&nbsp;\n\n- The method completely relies on a validation set on the server, which may be hard to obtain without introducing any privacy concerns. Even a validation set is available to use, under what circumstances will the validation set be equally fair to all clients so that it is not far from any client’s local data distribution. And if the plane shows anomalies for some client, how to distinguish the reasons from having an unfair validation set and client is the anomaly/attacker?\n\n&nbsp;\n\n- The datasets and the federated settings in the experiments are too simple. And the compared robust aggregation methods are not so recent.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "_The paper heavily relies on visual representations to explain client behaviors, infeasible for large FL systems._\n\n__Our method is scalable and automatically extracts and analyzes statistics from behavioral planes (BPs) to identify and mitigate malicious clients without relying on visual inspection. Visual inspection aids in understanding the characteristics under which malicious clients behave differently, but it is not necessary.__ Our defense extracts client-specific scores from BPs in large FL systems to reduce the impact of malicious clients. To further demonstrate scalability and automatic identification of malicious behaviors, we recorded client scores during training using CIFAR10 (Fig. B). These scores can identify malicious clients and reduce their contribution to the global model. Interestingly, as the model converges, more weight is given to the attacker with an inverted gradient, since its model closely resembles the global model from the previous round (plus a negligible inverted update). Visual inspection can still be useful for ML engineers to understand why certain clients are removed or contribute less, by visualizing on multiple planes where they differ from ‘honest’ clients (see Fig. 3). The experiments and discussion will be added to Appendix B.\n\n_Client selection at each round will affect trajectory on the planes._ \n\n__Our method ensures that client selection does not impact the effectiveness of our defense (FBSs), as aggregation relies on both current and available historical behavior. Visualizations can track client trajectories based on selected rounds, maintaining a clear representation of client behavior.__ If a client is not selected in a round, its trajectory will not consider that particular round, but trajectories can still be created based on selected rounds. While the current focus is on evaluating client behavior over time, it is also possible to establish trajectories only for selected clients relative to the server’s position, ensuring accurate visualization not affected by client selection. Finally, although client selection influences the visual representation, it has no effect on FBSs, as aggregation depends on both current and historical client behavior (Moving Average in Sec. 3.5).\n\n_The method relies on a server validation set, which may be difficult to obtain without privacy concerns._\n\n__Several solutions can be integrated with our method to create a synthetic validation set on the server without transmitting sensitive information from clients.__ A clean validation set is a common assumption of existing methods [51, 45, 31 cited in paper]. To address this, some proposed solutions include optimizing for a clean validation set [15] and generating representative inputs through dimensionality reduction and stratified sampling [44].\nAs discussed in Sec. 6.2, we plan to:\n- Apply an autoencoder or Bayesian probability conditioned on class labels to generate a synthetic dataset based on client data distributions\n- Explore synthetic generation as in [44]\n\nIf the server has a validation set, our method weights each client to maximize global model performance on that test set, regardless of whether the model weights come from honest or malicious clients\n\n_When is the validation set fair to all clients? How to distinguish between an unfair validation set and an anomaly/attacker?_\n\nThis is a great question, and that’s exactly why we proposed the multi-plane evaluation. Unlike traditional ML metrics such as accuracy or error, __our defense method is not strongly influenced by the fairness of the validation set, as counterfactuals (CFs) are more closely related to the learned client decision process and data distribution than the evaluation set.__ Anomalous clients show unrelated CF distributions and high error, while underrepresented clients generate plausible CFs but are affected by the unfair validation set only in the error plane. To demonstrate effectiveness under an unfair validation set, we removed one client’s data from the validation set and recorded behavioral scores. Preliminary results (Fig. C) show that the 95% confidence interval of the unfair client’s score overlaps with those of other honest clients, distinguishing them from malicious clients. We introduce in L340: Considering the importance of fairness among clients and the promising results shown in Appendix B, the exploration of the impact of unfair validation sets and the development of other validation-independent descriptors, such as CFs, represent promising areas for future research.\n \n_Datasets are simple and baselines are not so recent_\n\n__We have enhanced our experimental setup by introducing a more challenging dataset, CIFAR-10, with a 10-class classification task, and a recent robust aggregation baseline, RFA [Pillutla et al., 2022], into all our experiments to further demonstrate the robustness and effectiveness of our method.__ Even under these new conditions (Fig. A), our method outperforms or matches other baselines across all four datasets and all five attack conditions, except for Inv. gradient attacks in MNIST, where Trimmed mean performs better (1 out of 20 cases). \n\n_Is the counterfactual generator learned on the server?_\n\n__No, the generator is simultaneously trained in an end-to-end fashion on the client-side along with the predictor__ [L139,L231]. Please refer to the common answers for model analysis.\n\n_How is the local CL’s accuracy computed, and why is it so much lower than FL in Table 1?_\n\n__In Local CL, we train a separate model for each client’s private data__, ensuring privacy by not sharing data across clients. In non-IID settings, training on single-client data reduces performance due to the lack of diverse data, explaining the lower accuracy of Local CL compared to FL, which uses data from all clients, in Table 1. We added in L235: For Local CL, we report the average accuracy of all models trained individually by each client and evaluated on a common test set", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "BFjcu4gYIg", "reviewer": "Reviewer_aREF", "summary": "The paper introduces Federated Behavioural Planes (FBPs), a method designed to track FL clients' behavior by means of examining the their representations in two behavioural planes with the aid of a server-owned dataset. The Error Behavior Plane (EBP) and Counterfactual Behavioural Plane (CBP) correspond to two 2-D space where each client model's prediction errors and counterfactural distribution are examined. Combining these two representations a new aggregation rule is proposed to fend off malicious clients.", "questions": "- Results in Table 1 make me confused whether the whole method is still run in a cannonical FL setting.\n- What is the difference between proximity and sparsity?\n- How is counterfactural generator trained and why does it impact the task accuracy?\n- What purpose does Eq. 4 serve?", "limitations": "- A framework overview is missing, which makes it hard to understand how the planes are created throughout the FL iterations.", "rating": 5, "confidence": 2, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "- The paper is overall well written. The idea presented is clear and easy to follow.\n- The method of characterizing clients' behavior is kind of novel.", "weaknesses": "- Creating the two plains require the plaintext of all local models, which raises major privacy concerns.\n- From Eqs. 6 and 7, I suppose generating the counterfacturals must be very costly.\n- Insufficient experimental evidence to support the claim. For example, fig. 3 does not show distinct client behavior clearly. I also expect to see more results such as the behavioural scores of different clients and how sensitive the FBP is to the intensity of attacks.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "_Privacy concerns of transmitting the plaintext of all local models_\n\n__Our method can integrate Local Differential Privacy (LDP) or Homomorphic Encryption (HE) to enhance privacy, ensuring that sensitive information remains protected while maintaining robustness.__\nOur framework, like traditional FedAvg, is susceptible to privacy leakage if plaintext local model parameters are exposed. However, it can incorporate LDP, where clients add noise to the model before transmission, maintaining DP while creating behavioral planes. Although this process may result in some performance loss, it is comparable to other robust aggregation methods that calculate client similarities using differentially private model parameters.\nTo further enhance privacy, our framework can integrate HE, which masks client models while allowing for inference on the validation set. Although this increases computational time during the aggregation, it ensures that sensitive information remains completely concealed from the server, thereby addressing privacy concerns effectively. We added a sentence in L348: \"Since our method allows the integration of privacy-enhancing techniques such as LDP and HE, a promising future direction would be to analyze their impact on performance and efficiency\".\n\n_Insufficient experimental evidence regarding the distinction of client behaviors through Behavioral Planes (BPs). Expected more results, e.g. the client behavioral scores_\n\n__We have strengthened the experimental evidence by analyzing behavioral scores extracted from BPs, demonstrating that our method accurately identifies and mitigates malicious clients across all attacks.__\nAs suggested, we analyze the behavioral scores of honest and malicious clients across all attacks (Fig. B), which are calculated over the BPs. The 5-fold experiments were conducted on the CIFAR10 (most complex dataset), and show the mean and 95% confidence intervals during the training rounds. Statistically, our method consistently identifies malicious clients, effectively excluding or reducing their weights during aggregation. These results further demonstrate that BPs accurately describe malicious clients during the training process.\n\n_How sensitive the proposed method is to the intensity of the attacks_\n\n__Our method remains stable and unaffected by increasing attack intensity, demonstrating robustness against model poisoning attacks.__\nWe analyzed the sensitivity of our method to attack intensity by increasing the amount of noise $\\beta$ that attackers add to the global model before sending it to the server [L677]. We conducted 5-fold experiments on both the Breast Cancer and MNIST datasets. As shown in Fig. E, increasing attack intensity decreases FedAvg’s performance, while our methods (using all planes and only the counterfactual plane) remain stable and unaffected by the attack intensity. Interestingly, our methods’ accuracy slightly increased as attack intensity rose, likely because the malicious models became more degraded and, therefore, less stealthy.\n\n_A framework overview is missing, which makes it hard to understand how the planes are created throughout the FL iterations_\n\nWe thank the reviewer for this observation. To address this limitation, __we have provided a detailed algorithm (Algorithm 1) for our method to create the behavioral planes and implement our robust aggregation (FBSs)__. While Algorithm 1 covers the server operations, we will also include the algorithm for the client operations in the Appendix A.5 of the paper, which reflects the traditional FL algorithm.\n\n_Results in Table 1 make me confused whether the whole method is still run in a cannonical FL setting_\n\n__Yes, our method operates within a canonical FL setting, with changes to server-side aggregation and the introduction of the counterfactual (CF) generator. The CF generator is trained jointly with the local client and does not alter the performance of the predictor alone (Table 1)__. For clarity, we updated L230-231: \"We compared model accuracy across four settings: two centralized learning (CL) and two FL, using three different datasets under non-IID conditions. For FL experiments, we used the traditional FedAvg approach with two variations: predictor-only and predictor with CF generator.\"\n\n_What is the difference between proximity and sparsity?_\n\nThese are standard metrics to quantitatively evaluate counterfactuals (CFs) [19, 35 in the paper]. __Proximity measures how much our CFs are closer to our data distribution (distance between the CF and the closest data point in the training set with the same label), while sparsity measures how many features were changed between the initial sample and the CF.__ For clarity, we updated L208-209: “proximity (↓) [35], assessing the realism of CFs by their closeness to the training data (distance between the CF and the closest data point in the training set with the same label); and sparsity (↓) [19], which quantifies the changes made to the input to generate the CFs (number of features changed between the initial sample and the CF).”\n\n_What purpose does Eq. 4 serve?_\n\n__Eq. 4 provides a theoretical foundation for understanding FL dynamics, describing how client behaviors evolve during training and are influenced by intermediate steps.__\nAs mentioned by both Rev-eMJr and ULpn, our primary goal is to provide a general theoretical foundation for understanding FL dynamics, specifically how client behaviors evolve during training and how they are influenced by intermediate steps (e.g., local training and aggregation). Eq. 4 describes the dynamics of client behaviors in the FL system. Although we did not analytically solve it, we experimentally observe its solutions through client behaviors using our planes. For clarity, we modify L97-98: \"These dynamics can be encapsulated in the following differential equation, which describes how client behaviors evolve during training and are influenced by internal forces within the FL system\".", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:
# Answer to reviewers and ACs
We thank the reviewers for their insightful feedback. We are encouraged by their recognition of the novelty in our ideas and proposed methods for analyzing and visualizing client behaviors (eMJr, ULpn, aREF), and the significance of this work in the FL field (eMJr). We are pleased that our work was found to be clear, easy to follow (aREF) and grounded in a solid theoretical foundation (eMJr, ULpn). We appreciate ULpn’s acknowledgment of our method’s explanatory power, enhancing FL system interpretability. Reviewers’ feedback has certainly improved the quality of our manuscript and we hope we successfully addressed your concerns in this rebuttal and in our updated submission. We reply to shared questions here and address specific questions under each reviewer’s feedback.
# Summary of Changes
In response to the reviewers’ comments, we worked on improving the clarity and comprehensiveness of our paper with new sections and explanatory content. By incorporating a new dataset, a new SOTA baseline, and additional experiments, we have increased the robustness of our method’s validation. These improvements address the specific concerns and strengthen our work’s overall contribution. However, the core contributions and evaluations of our work remain unchanged. Our changes are summarized as follows: 
- __Comprehensive Theoretical and Experimental Analysis of Computational Costs__ (Appendix A.6)
- __Expanded Experimental Validation:__ Validated on a more complex dataset, CIFAR10, in addition to the four previously used datasets, to further assess our method’s performance (Fig. A and Table in Appendix B)
- __Additional Baseline Comparison:__ Introduction of a recent robust aggregation baseline [K. Pillutla et al., RFA, 2022] in all experiments (Fig. A, new Table 5, 6, 7)
- __Attack Intensity Analysis:__ Verified robustness against varying attack intensities (Fig. E)
- __Behavioral Score Analysis:__ Evaluated the importance of information provided by our behavioral planes by recording the automatically extracted client behavioral scores from our defense method (Fig. B)
# Common Answers
_@Rev-eMJr,ULpn,aREF – Lack of computational cost analysis_

__We perform a comprehensive analysis of the computational cost of our methods. In the worst-case scenario, introducing counterfactuals (CFs) adds 7.6% more model parameters and 5.1% more GFLOPs for inference compared to the predictor alone. Overall our method adds 1 minute per round to FedAvg, while Krum adds over 15 minutes per round.__
The computational cost in FL frameworks includes local computation, communication overhead, and server-side computation 

- __Local Computation.__ Our methods integrate a CF generator with the original predictor to explain decision-making processes and provide insights into client data distribution. For small NNs, local computation is minimal compared to communication latency and synchronization, allowing CF generation for predictor input without affecting training efficiency. As the predictor size increases, as shown with MNIST and CIFAR10, we can efficiently generate CFs for intermediate layers using a relatively small number of neurons. As shown in Table A, without losing performance, the generator requires only 2.7% of the predictor’s GFLOPs (ResNet-18) for inference on a 28x28 RGB image. Training time is proportional, as our generator trains end-to-end with the predictor.
- __Communication Overhead.__ The CF generator is transmitted with the predictor to the server for evaluation and aggregation. The increase in the number of transmitted parameters is marginal compared to the predictor, consisting of only 1.8% of the predictor’s parameters (Table A). This results in an additional 0.92 megabytes (MB) compared to the 49.68 MB required for the predictor.
- __Server-side Computation.__ This involves evaluating client models on a small validation set (e.g., 250 samples as shown in Fig. 9) and calculating pairwise distances between client CFs. Model evaluations, which required a single pass for each client model, are negligible compared to the computational load of calculating pairwise distances. We use the sliced Wasserstein distance with a complexity of $O(m \log m)$, where $m = n_{samples} \times 2=500$ [L148]. This operation is repeated for each unique pair of clients, leading to a total complexity of $O(n^2⋅m \log m)$, where $n$ is the number of clients. Compared to Krum’s complexity of $O(n^2⋅d)$, where $d$ is the number of model parameters, our method is more efficient for NNs with more than 4480 parameters, which is typical in practical applications
- __Overall Computational Cost.__ We compared the computational cost of our defense (FBSs) with Krum and FedAvg across different network dimensions and client numbers (Fig. D). For increasing client numbers, our method uses a CF generator with 7.6% of the predictor’s parameters (worst-case scenario) and 250 validation samples, as used in the paper. Our method scales better than Krum under both conditions, adding only 1 extra minute per round with 200 clients compared to FedAvg, while Krum adds over 15 minutes per round. This demonstrates its efficiency in large-scale scenarios

_@Rev-ULpn,aREF – Why does the CF generator affect predictor performance?_

__The CF generator affects the performance of the predictor because it is jointly optimized with the predictor and this is needed to generate CFs during training.__ Therefore training the CF generator alongside the predictor influences the training process of the predictor. The small change in performance, shown in Table 1, can be attributed to the additional loss providing extra information to navigate the optimization space more efficiently, similar to regularization terms. On the contrary, using a post-hoc CF generator (which does not affect performances) would necessitate training a separate generator for each client at the end of every epoch, significantly increasing the computational cost

Meta-review: The paper introduces a method to track FL clients' dynamic behavior.  There is good agreement among the reviewers on the major strengths of the paper and its novelty.
Decision: Accept (poster)