Original: Certified Robustness in Federated Learning
Revised: Certified Robustness in Federated Learning

Original: Anonymous Author(s)

Original: AffiliationAddress email

Original: Abstract
Revised: Abstract

Original: Federated learning has recently gained significant attention and popularity due toits effectiveness in training machine learning models on distributed data privately.
Revised: Federated learning has recently gained significant attention and popularity due to its effectiveness in training machine learning models on distributed data privately.

Original: However, as in the single-node supervised learning setup, models trained in federated learning suffer from vulnerability to imperceptible input transformationsknown as adversarial attacks, questioning their deployment in security-relatedapplications.
Revised: However, as in the single-node supervised learning setup, models trained in federated learning suffer from vulnerability to imperceptible input transformations known as adversarial attacks, questioning their deployment in security-related applications.

Original: In this work, we study the interplay between federated training,personalization, and certified robustness.
Revised: In this work, we study the interplay between federated training, personalization, and certified robustness.

Original: In particular, we deploy randomizedsmoothing, a widely-used and scalable certification method, to certify deep networks trained on a federated setup against input perturbations and transformations.
Revised: In particular, we deploy randomized smoothing, a widely-used and scalable certification method, to certify deep networks trained on a federated setup against input perturbations and transformations.

Original: We find that the simple federated averaging technique is effective in building notonly more accurate, but also more certifiably-robust models, compared to trainingsolely on local data.
Revised: We find that the simple federated averaging technique is effective in building not only more accurate, but also more certifiably-robust models, compared to training solely on local data.

Original: We further analyze personalization, a popular technique infederated training that increases the model’s bias towards local data, on robustness.
Revised: We further analyze personalization, a popular technique in federated training that increases the model’s bias towards local data, on robustness.

Original: We show several advantages of personalization over both (that is, only trainingon local data and federated training) in building more robust models with fastertraining.
Revised: We show several advantages of personalization over both (that is, only training on local data and federated training) in building more robust models with faster training.

Original: Finally, we explore the robustness of mixtures of global and local ( i.e.personalized) models, and find that the robustness of local models degrades asthey diverge from the global model.
Revised: Finally, we explore the robustness of mixtures of global and local ( i.e. personalized) models, and find that the robustness of local models degrades as they diverge from the global model.

Original:  Introduction
Revised:  Introduction

Original: Machine learning models in general, and deep neural networks (DNNs) in particular, have demonstrated remarkable performance in several domains, including computer vision [1] and natural language processing [2], among others [3].
Revised: Machine learning models in general, and deep neural networks (DNNs) in particular, have demonstrated remarkable performance in several domains, including computer vision [1] and natural language processing [2], among others [3].

Original: Despite this success, DNNs are brittle against smallcarefully-crafted perturbations in their input space [4, 5].
Revised: Despite this success, DNNs are brittle against small carefully-crafted perturbations in their input space [4, 5].

Original: That is, a DNN f θ : R d → P ( Y ) canproduce different predictions for the inputs x ∈ R d and x + δ , although the adversarial perturbationδ is small and thus x and x + δ are indistinguishable to the human eye.
Revised: That is, a DNN f θ : R d → P ( Y ) can produce different predictions for the inputs x ∈ R d and x + δ , although the adversarial perturbation δ is small and thus x and x + δ are indistinguishable to the human eye.

Original: Moreover, DNNs are alsosusceptible to input transformations that preserve an image’s semantic information, such as rotation,translation, and scaling [6].
Revised: Moreover, DNNs are also susceptible to input transformations that preserve an image’s semantic information, such as rotation, translation, and scaling [6].

Original: This observation raises security concerns about deploying such models in security-critical applications, i.e. self-driving cars.
Revised: This observation raises security concerns about deploying such models in security-critical applications, i.e. self-driving cars.

Original: Consequently, this phenomenon sparkedsubstantial research efforts towards building machine learning models that are not only accurate,but also robust.
Revised: Consequently, this phenomenon sparked substantial research efforts towards building machine learning models that are not only accurate, but also robust.

Original: That is, building models that both correctly classify an input x , and maintain theirprediction as long x preserves its semantic content.
Revised: That is, building models that both correctly classify an input x , and maintain their prediction as long x preserves its semantic content.

Original: Further interest arose in theoretically characterizing the output of models whose input was subjected to perturbations.
Revised: Further interest arose in theoretically characterizing the output of models whose input was subjected to perturbations.

Original: Whenever a theoreticalguarantee exists about a model’s robustness for classifying inputs, f θ is said to be certifiably robust [7].
Revised: Whenever a theoretical guarantee exists about a model’s robustness for classifying inputs, f θ is said to be certifiably robust [7].

Original: Among several methods that built such certifiably robust models, Randomized Smoothing(RS) [8] is arguably one of the most effective, scalable, and popular approaches to certify DNNs.
Revised: Among several methods that built such certifiably robust models, Randomized Smoothing (RS) [8] is arguably one of the most effective, scalable, and popular approaches to certify DNNs.

Original: In a nutshell, RS predicts the most probable class when the classifier’s input is subjected to additive
Revised: In a nutshell, RS predicts the most probable class when the classifier’s input is subjected to additive

Original: Gaussian noise.
Revised: Gaussian noise.

Original: While RS directly operated on pixel-intensity perturbations, a later work extendedthis technique to certify against input deformations, such as rotation and translation [9].
Revised: While RS directly operated on pixel-intensity perturbations, a later work extended this technique to certify against input deformations, such as rotation and translation [9].

Original: The problem of training large-scale machine learning models in the real world may need to accountfor data privacy constraints that arise in distributed setups.
Revised: The problem of training large-scale machine learning models in the real world may need to account for data privacy constraints that arise in distributed setups.

Original: Federated learning (FL) [10, 11, 12] isa viable solution to this problem, which is now commonly used in popular services [13].
Revised: Federated learning (FL) [10, 11, 12] is a viable solution to this problem, which is now commonly used in popular services [13].

Original: Althoughthe certified robustness of models in traditional learning settings has been widely studied, few worksconsider the interaction between certified robustness and the federated setup.
Revised: Although the certified robustness of models in traditional learning settings has been widely studied, few works consider the interaction between certified robustness and the federated setup.

Original: Specifically, federatedlearning distributes the training process across a (possibly large) set of clients (each with their ownlocal data) and privately builds a global model without leaking nor sharing local raw data.
Revised: Specifically, federated learning distributes the training process across a (possibly large) set of clients (each with their own local data) and privately builds a global model without leaking nor sharing local raw data.

Original: In thefederated learning setup, Zizzo et al.
Revised: In the federated learning setup, Zizzo et al.

Original: [14] showed that (1) adversarial attacks can reduce the accuracyof state-of-the-art federated learning models to 0%, and (2) adversarial robustness is higher in thecentralized setting than in the federated setting, illustrating the difficulty of enhancing adversarialrobustness.
Revised: [14] showed that (1) adversarial attacks can reduce the accuracy of state-of-the-art federated learning models to 0%, and (2) adversarial robustness is higher in the centralized setting than in the federated setting, illustrating the difficulty of enhancing adversarial robustness.

Original: Wef identify two key and challenging settings for analyzing certified robustness infederated learning: (i) each client has insufficient data to fully train a robust model, and (ii) eachclient may be interested in building a  model that is robust against different transformations.
Revised: We identify two key and challenging settings for analyzing certified robustness in federated learning: (i) each client has insufficient data to fully train a robust model, and (ii) each client may be interested in building a personalized model that is robust against different transformations.

Original: In this work, we set to study the certified robustness of models trained in a federated fashion.
Revised: In this work, we set to study the certified robustness of models trained in a federated fashion.

Original: Wedeploy Randomized Smoothing to measure certified robustness against both pixel-intensity perturbations and input deformations.
Revised: We deploy Randomized Smoothing to measure certified robustness against both pixel-intensity perturbations and input deformations.

Original: We first analyze when federated training benefits model robustness.
Revised: We first analyze when federated training benefits model robustness.

Original: We show that in the few local-data regimes—where clients have insufficient data—federated averaging improves not only model performance but also its robustness.
Revised: We show that in the few local-data regimes—where clients have insufficient data—federated averaging improves not only model performance but also its robustness.

Original: Furthermore, we explore howcertified robustness gains from personalization [15, 16, 17, 18], i.e. fine-tuning the global modelon local data.
Revised: Furthermore, we explore how certified robustness gains from personalization [15, 16, 17, 18], i.e. fine-tuning the global model on local data.

Original: We show that personalization can provide significant robustness improvements evenwhen federated averaging operates on models trained without augmentations aimed at enhancingrobustness.
Revised: We show that personalization can provide significant robustness improvements even when federated averaging operates on models trained without augmentations aimed at enhancing robustness.

Original: At last, we analyze a version of the recently-proposed  mixture between global andpersonalized modelsfrom federated learning [19, 20] from a certified robustness lens.
Revised: At last, we analyze a version of the recently-proposed federated mixture of global and personalized models [19, 20] from a certified robustness lens.

Original:  Related Work
Revised:  Related Work

Original: Certified Adversarial Robustness.
Revised: Certified Adversarial Robustness.

Original: The seminal work of Szegedy et al.
Revised: The seminal work of Szegedy et al.

Original: [4] demonstrated how
Revised: [4] demonstrated how

Original: DNNs are vulnerable to small perturbations in their input, now called adversarial attacks.
Revised: DNNs are vulnerable to small perturbations in their input, now called adversarial attacks.

Original: Subsequent works noted how the brittleness against these attacks was widespread and easily exploitable [5, 21].
Revised: Subsequent works noted how the brittleness against these attacks was widespread and easily exploitable [5, 21].

Original: This observation led to further works on defense mechanisms that improved adversarial robustness [22, 23], and on attacks that aimed at breaking these mechanisms [21, 24].
Revised: This observation led to further works on defense mechanisms that improved adversarial robustness [22, 23], and on attacks that aimed at breaking these mechanisms [21, 24].

Original: Tothis aim, Cohen et al.
Revised: To this aim, Cohen et al.

Original: [8] proposed Randomized Smoothing (RS), an approach for certification thatproved to successfully scale with models and datasets.
Revised: [8] proposed Randomized Smoothing (RS), an approach for certification that proved to successfully scale with models and datasets.

Original: The original RS formulation certified modelsagainst pixel-level additive perturbations, and was studied in the federated learning setup by Chen etal.
Revised: The original RS formulation certified models against pixel-level additive perturbations, and was studied in the federated learning setup by Chen et al.

Original: Recently, DeformRS [9] reformulated RS to consider parameterized deformations, grantingrobustness certificates against more general and semantic perturbations such as rotation.
Revised: Recently, DeformRS [9] reformulated RS to consider parameterized deformations, granting robustness certificates against more general and semantic perturbations such as rotation.

Original: Federated Learning.
Revised: Federated Learning.

Original: Federated Learning (FL) is a data-decentralized machine learning setting inwhich a central server coordinates a large number of clients ( e.g. mobile devices or whole organizations) to collaboratively train a model [10, 11, 12, 26].
Revised: Federated Learning (FL) is a data-decentralized machine learning setting in which a central server coordinates a large number of clients ( e.g. mobile devices or whole organizations) to collaboratively train a model [10, 11, 12, 26].

Original: In particular, FL has been successfullydeployed by Google [27], Apple [28], NVIDIA [29], and many others, even in safety- and securitycritical settings, which motivates our work.
Revised: In particular, FL has been successfully deployed by Google [27], Apple [28], NVIDIA [29], and many others, even in safety- and securitycritical settings, which motivates our work.

Original: Given how the inherent multiple-devices nature of FLintroduces heterogeneity in the data, interest spurred towards techniques that considered model personalization [30, 19, 20,  31] to improve performance on each device’s data [16, 18, 32, 33].
Revised: Given how the inherent multiple-devices nature of FL introduces heterogeneity in the data, interest spurred towards techniques that considered model personalization [19, 20, 30, 31] to improve performance on each device’s data [16, 18, 32, 33].

Original: Robustness in FL.
Revised: Robustness in FL.

Original: Concerns about the adversarial vulnerability of machine learning models havealso been studied in the FL setup [34].
Revised: Concerns about the adversarial vulnerability of machine learning models have also been studied in the FL setup [34].

Original: While most prior works focused on training-time adversarialattacks such as Byzantine attacks [35], and model [34] or data poisoning [36, 37], recent worksstudied test-time robustness.
Revised: While most prior works focused on training-time adversarial attacks such as Byzantine attacks [35], and model [34] or data poisoning [36, 37], recent works studied test-time robustness.

Original: In particular, the works of Zizzo et al.
Revised: In particular, the works of Zizzo et al.

Original: [14] and Shah et al.
Revised: [14] and Shah et al.

Original: [38] studiedthe incorporation of adversarial training [23], arguably the most effective empirical defense againstadversarial attacks, into the FL setup.
Revised: [38] studied the incorporation of adversarial training [23], arguably the most effective empirical defense against adversarial attacks, into the FL setup.

Original: Closest to our study, is the recent work of Chen et al.
Revised: Closest to our study, is the recent work of Chen et al.

Original: [25]that explored certification via RS in FL.
Revised: [25] that explored certification via RS in FL.

Original: In contrast to [25], instead of pixel-level perturbations, westudy realistic ( e.g. rotation and translation) perturbations.
Revised: In contrast to [25], instead of pixel-level perturbations, we study realistic ( e.g. rotation and translation) perturbations.

Original:  Methodology
Revised:  Methodology

Original: Notation.
Revised: Notation.

Original: We consider the standard image classification problem where a classifier f θ : R d →
Revised: We consider the standard image classification problem where a classifier f θ : R d →

Original: P ( Y ) , e.g. a DNN with parameters θ , maps the input x ∈ R d to the probability simplex P ( Y ) overk classes, where Y = { 1 , 2 , . .
Revised: P ( Y ) , e.g. a DNN with parameters θ , maps the input x ∈ R d to the probability simplex P ( Y ) over k classes, where Y = { 1 , 2 , . .

Original: . , k } .
Revised: . , k } .

Original: That is, (cid:80) ki
Revised: That is, (cid:80) ki

Original: =1 f iθ ( x ) = 1 with f iθ ( x ) ≥ 0 being the i th elementof f θ ( x ) , representing the probability x belongs to the i th class.
Revised: =1 f iθ ( x ) = 1 with f iθ ( x ) ≥ 0 being the i th element of f θ ( x ) , representing the probability x belongs to the i th class.

Original: Randomized Smoothing.
Revised: Randomized Smoothing.

Original: Randomized smoothing constructs a new “smooth” classifier from anarbitrary classifier f θ .
Revised: Randomized smoothing constructs a new “smooth” classifier from an arbitrary classifier f θ .

Original: In a nutshell, when this smooth classifier is queried at input x , it returns theaverage prediction of f θ when its input x is subjected to isotropic additive Gaussian noise, that is:
Revised: In a nutshell, when this smooth classifier is queried at input x , it returns the average prediction of f θ when its input x is subjected to isotropic additive Gaussian noise, that is:

Original: [Equation]
Revised: [Equation]

Original: Let g predict label c A for input x with some confidence, i.e. E ϵ
Revised: Let g predict label c A for input x with some confidence, i.e. E ϵ

Original: [ f c
Revised: [ f c

Original: A θ ( x + ϵ )]
Revised: A θ ( x + ϵ )]

Original: =
Revised: =

Original: p
Revised: p

Original: A ≥ p B =max c ̸
Revised: A ≥ p B = max c ̸

Original: = c
Revised: = c

Original: A E ϵ
Revised: A E ϵ

Original: [ f cθ ( x + ϵ )] , then, as proved in [39], g θ ’s prediction is certifiably robust with radius:
Revised: [ f cθ ( x + ϵ )] , then, as proved in [39], g θ ’s prediction is certifiably robust with radius:

Original: [Equation]
Revised: [Equation]

Revised: where Φ − 1 is the inverse CDF of the standard Gaussian distribution.

Original: That is, arg max i g i ( x + δ ) =
Revised: That is, arg max i g i ( x + δ ) =

Original: arg max i g i ( x ) ,
Revised: arg max i g i ( x ) ,

Original: ∀∥ δ ∥ 2 ≤ R , where Φ − 1 is the inverse CDF of thestandard Gaussian distribution.
Revised: ∀∥ δ ∥ 2 ≤ R .

Original: While the original RS framework certified against additive pixel perturbations in images, DeformRS [9] extended RS to certify against image deformations .
Revised: While the original RS framework certified against additive pixel perturbations in images, DeformRS [9] extended RS to certify against image deformations .

Original: DeformRSachieved this objective by proposing a parametric-domain smooth classifier .
Revised: DeformRS achieved this objective by proposing a parametric-domain smooth classifier .

Original: In particular, given animage x with pixel coordinates p , a parametric deformation function ν ϕ with parameters ϕ
Revised: In particular, given an image x with pixel coordinates p , a parametric deformation function ν ϕ with parameters ϕ

Original: ( e.g. if νis a rotation, then ϕ is the angle of rotation) and an interpolation function I T , DeformRS defined theparametric-domain smooth classifier as:[Equation]
Revised: ( e.g. if ν is a rotation, then ϕ is the angle of rotation) and an interpolation function I T , DeformRS defined the parametric-domain smooth classifier as:

Original: [Equation]
Revised: [Equation]

Original: In a nutshell, g outputs the average prediction of f θ over transformed versions of the input x .
Revised: In a nutshell, g outputs the average prediction of f θ over transformed versions of the input x .

Original: Notethat this formulation perturbs the pixels’ location , rather than their intensities.
Revised: Note that this formulation perturbs the pixels’ location , rather than their intensities.

Original: DeformRS showedthat, analogous to the smoothed classifiers in RS, parametric-domain smooth classifiers are certifiably robust against perturbations to the parameters of the deformation function with radius
Revised: DeformRS showed that, analogous to the smoothed classifiers in RS, parametric-domain smooth classifiers are certifiably robust against perturbations to the parameters of the deformation function with radius

Original: [Equation]
Revised: [Equation]

Original: for [Equation] Eq (4) states that, as long as the parameters of the deformation function ( e.g. rotation angle) areperturbed by a quantity upper bounded by the certified radius, g ’s prediction will remain constant.
Revised: Eq (4) states that, as long as the parameters of the deformation function ( e.g. rotation angle) are perturbed by a quantity upper bounded by the certified radius, g ’s prediction will remain constant.

Original: Specialization to Federated Training.
Revised: Specialization to Federated Training.

Original: Earlier works proposed various training frameworks toenhance the robustness of smooth classifiers.
Revised: Earlier works proposed various training frameworks to enhance the robustness of smooth classifiers.

Original: The schemes studied by these frameworks ranged fromsimple data augmentation [8] to more sophisticated techniques, including adversarial training [40]and regularization [39].
Revised: The schemes studied by these frameworks ranged from simple data augmentation [8] to more sophisticated techniques, including adversarial training [40] and regularization [39].

Original: In this work, we robustify the classifier via simple and nondemandingdata augmentation.
Revised: In this work, we robustify the classifier via simple and nondemanding data augmentation.

Original: We select this approach since our main aim is to analyze the effect of federatedtraining on certified robustness in isolation of additional and more sophisticated training schemes.
Revised: We select this approach since our main aim is to analyze the effect of federated training on certified robustness in isolation of additional and more sophisticated training schemes.

Original: Inaddition, this setup allows compatibility with other FL-specific constraints and technologies, such asdifferential privacy and secure aggregation, which can be essential for real-world deployment [13].
Revised: In addition, this setup allows compatibility with other FL-specific constraints and technologies, such as differential privacy and secure aggregation, which can be essential for real-world deployment [13].

Original: As in the Federated Learning setup, we employ the common federated averaging [41, 10] technique.
Revised: As in the Federated Learning setup, we employ the common federated averaging [10, 41] technique.

Original: In a nutshell, we consider the case when a dataset is distributed across multiple clients in anon-overlapping manner.
Revised: In a nutshell, we consider the case when a dataset is distributed across multiple clients in a non-overlapping manner.

Original: We assume that all clients share the architecture f θ , to allow for weightaveraging across clients.
Revised: We assume that all clients share the architecture f θ , to allow for weight averaging across clients.

Original: Furthermore, we study three different training schemes, where each clientis minimizing the following regularized  risk [19]: 
Revised: Furthermore, we study three different training schemes, where each client is minimizing the following regularized empirical risk [19]: [Equation]

Original: [Equation]
Revised: [Equation]

Original: where L i is a training loss function ( e.g. cross entropy) evaluated on local data of client i , θ irepresents the i th client’s model parameters, λ ∈ [0 , 1] is a personalization parameter, and ¯ θ =1 n (cid:80) ni
Revised: where L i is a training loss function ( e.g. cross entropy) evaluated on local data of client i , θ i represents the i th client’s model parameters, λ ∈ [0 , 1] is a personalization parameter, and ¯ θ =n (cid:80) ni

Original: =

Revised: =

Original: θ i is the average model.
Revised: θ i is the average model.

Original: Local Training.
Revised: Local Training.

Original: Each client trains solely on their own local data, without communication withother clients.
Revised: Each client trains solely on their own local data, without communication with other clients.

Original: At test time, each client employs their own local model.
Revised: At test time, each client employs their own local model.

Original: This setting is equivalent tominimizing the risk in Eq.
Revised: This setting is equivalent to minimizing the risk in Eq.

Original: (5) with λ = 1 .
Revised: (5) with λ = 1 .

Original: Federated Training.
Revised: Federated Training.

Original: All clients are collaborating in building one global model.
Revised: All clients are collaborating in building one global model.

Original: This is achievedby iteratively performing (1) client-side model updates with optimization on local data ( i.e. solving
Revised: This is achieved by iteratively performing (1) client-side model updates with optimization on local data ( i.e. solving

Original: Eq.
Revised: Eq.

Original: (5) with λ = 1 ), and (2) communication of client models to the server, which averages allmodels and broadcasts the result back to the clients ( i.e. solving Eq.
Revised: (5) with λ = 1 ), and (2) communication of client models to the server, which averages all models and broadcasts the result back to the clients ( i.e. solving Eq.

Original: (5) with λ = 0 ).
Revised: (5) with λ = 0 ).

Original: This process is
Revised: This process is

Original: 0.5 Radius 0.

Original: C e r t i f ie d A cc .
Revised: [Figure] i f i

Original: Pixel Perturbations - = 0.
Revised: [Figure]

Original: Local
Revised: Local

Original: Federated
Revised: Federated

Original: Personalized Spread-L Spread-F Spread-P 0.
Revised: [Figure]

Original: 0.4 Radius 0. C e r t i f i e dA cc . Rotation - = 0. [Figure]
Revised: t i f i e d

Original: [Figure]
Revised: [Figure]

Original: Radius [Figure] [Figure] Rotation -
Revised: cc .

Original: [Figure]
Revised: [Figure]

Original: repeated until convergence or for a fixed number of communication rounds.
Revised: repeated until convergence or for a fixed number of communication rounds.

Original: At test time, all clientsemploy the global model sent  communication 
Revised: At test time, all clients employ the global model sent from the server at the last communication round.

Original: Personalized Training.
Revised: Personalized Training.

Original: Since [42] exposed how the basic FL formulation with λ = 1 and θ 1 =· · · = θ n degrades the performance of some clients, we explore a personalization approach.
Revised: Since [42] exposed how the basic FL formulation with λ = 1 and θ 1 = · · · = θ n degrades the performance of some clients, we explore a personalization approach.

Original: Namely,upon federated training convergence, each client fine-tunes the global model communicated in thelast round with a few optimization steps on their local data ( i.e. solving Eq.
Revised: Namely, upon federated training convergence, each client fine-tunes the global model communicated in the last round with a few optimization steps on their local data ( i.e. solving Eq.

Original: (5) with λ = 1 ).
Revised: (5) with λ = 1 ).

Original: Mixture of models.
Revised: Mixture of models.

Original: We also consider a more sophisticated personalized ERM setting via a mixtureof global and local models [19], seeking an explicit trade-off between the global model and the localmodels for λ ∈ (0 , 1) .
Revised: We also consider a more sophisticated personalized ERM setting via a mixture of global and local models [19], seeking an explicit trade-off between the global model and the local models for λ ∈ (0 , 1) .

Original: Our implementation for solving problem (5) for every client i requires localgradient descent steps (with step size γ ) and a mixing of current global and local models through:
Revised: Our implementation for solving problem (5) for every client i requires local gradient descent steps (with step size γ ) and a mixing of current global and local models through:

Original: [Equation]
Revised: [Equation]

Original: alternating with communication to the server for computing the current average model ¯ θ .
Revised: alternating with communication to the server for computing the current average model ¯ θ .

Original: At testtime, each client uses its corresponding personalized model θ i .
Revised: At test time, each client uses its corresponding personalized model θ i .

Original: While the effect of federated trainingand personalization in improving model performance is widely explored in the literature, its impacton the trained model’s certified robustness remains unclear.
Revised: While the effect of federated training and personalization in improving model performance is widely explored in the literature, its impact on the trained model’s certified robustness remains unclear.

Original:  Experiments
Revised:  Experiments

Original: Training Setup.
Revised: Training Setup.

Original: We fix ResNet18 [43] as our architecture and train on the CIFAR10 [44] and
Revised: We fix ResNet18 [43] as our architecture and train on the CIFAR10 [44] and

Original: MNIST [45] datasets.
Revised: MNIST [45] datasets.

Original: We split the training and test sets across clients in a homogeneous and nonoverlapping fashion.
Revised: We split the training and test sets across clients in a random and nonoverlapping fashion.

Original: Note that this split forms a best-case condition for local training, allowingcompatibility with federated training and its associated data-privacy constraints.
Revised: Note that this split forms a best-case condition for local training, allowing compatibility with federated training and its associated data-privacy constraints.

Original: We conduct 90epochs of training, divided in 45 communication rounds, i.e. 2 epochs of local training per communication round.
Revised: We conductepochs of training, divided in 45 communication rounds, i.e. 2 epochs of local training per communication round.

Original: We set a batch size to 64, and use a starting learning rate of 0 . 1 , which is decreasedby a factor of 10 every 30 epochs (15 communication rounds).
Revised: We set a batch size to 64, and use a starting learning rate of 0 . 1 , which is decreased by a factor of 10 every 30 epochs (15 communication rounds).

Original: Unless specified otherwise, given atransformation we are interested in robustifying against ( e.g. rotation), we train the model by augmenting the data with such transformation.
Revised: Unless specified otherwise, given a transformation we are interested in robustifying against ( e.g. rotation), we train the model by augmenting the data with such transformation.

Original: This procedure is equivalent to estimating the output ofthe smooth classifier in Eq.
Revised: This procedure is equivalent to estimating the output of the smooth classifier in Eq.

Original: (3) with one sample.
Revised: (3) with one sample.

Original: We set the σ ∈ { 0 . 1 , 0 . 5 } for rotation and translation and σ ∈ { 0 . 12 , 0 , 5 } for pixel perturbations, following [8, 9].
Revised: We set the σ ∈ { 0 . 1 , 0 . 5 } for rotation and translation and σ ∈ { 0 . 12 , 0 , 5 } for pixel perturbations, following [8, 9].

Original: For local models, we train eachclient on their local data only with the aforementioned setup without any communication.
Revised: For local models, we train each client on their local data only with the aforementioned setup without any communication.

Original: Certification Setup.
Revised: Certification Setup.

Original: For certification, we use the public implementation from [8] for pixel-intensityperturbations, and the one from [9] for rotation and translation.
Revised: For certification, we use the public implementation from [8] for pixel-intensity perturbations, and the one from [9] for rotation and translation.

Original: We use 100 Monte Carlo samples forcomputing the top prediction c A , and 100 k samples for estimating a lower bound to the predictionprobability p A ; we further use a failure probability of α = 0 . 001 , and bound the runner-up predictionvia p B = 1 − p A .
Revised: We use 100 Monte Carlo samples for computing the top prediction c A , and 100 k samples for estimating a lower bound to the prediction probability p A ; we further use a failure probability of α = 0 . 001 , and bound the runner-up prediction via p B = 1 − p A .

Original: For experiments on deformations, we choose I T to be a bi-linear interpolationfunction, following [9, 46].
Revised: For experiments on deformations, we choose I T to be a bi-linear interpolation function, following [9, 46].

Original: To evaluate each client’s certification, we fix a random subset of 500local samples.
Revised: To evaluate each client’s certification, we fix a random subset oflocal samples.

Original: At last, we plot the average certified accuracy curves across clients, highlightingthe minimum and maximum values as the limits of shaded regions.
Revised: At last, we plot the average certified accuracy curves across clients, highlighting the minimum and maximum values as the limits of shaded regions.

Original: We report “certified accuracyat radius R ”, defined as the percentage of test samples that is both, classified correctly, and has acertified radius of at least R .
Revised: We report “certified accuracy at radius R ”, defined as the percentage of test samples that is both, classified correctly, and has a certified radius of at least R .

Original: We report the certified accuracy curves for all clients in the appendix.
Revised: We report the certified accuracy curves for all clients in the appendix.

Original: Is Federated Training Good for Robustness?
Revised: Is Federated Training Good for Robustness?

Original: We first investigate the benefits of federated trainingon certified robustness.
Revised: We first investigate the benefits of federated training on certified robustness.

Original: To that regard, we use a rotation transformation and vary the number of
Revised: To that regard, we use a rotation transformation and vary the number of

Original: [Figure]
Revised: [Figure]

Original: clients on which the dataset is distributed.
Revised: clients on which the dataset is distributed.

Original: We conduct experiments with 4, 10, and 20 clients, andreport the results in Figure 1.
Revised: We conduct experiments with 4, 10, and 20 clients, and report the results in Figure 1.

Original: From these results, we draw the following observations.
Revised: From these results, we draw the following observations.

Original: (i)
Revised: (i)

Original: Whenthe number of clients is small (4 clients), i.e. when each client has large data, local training can besufficient for building a model that is accurate and robust.
Revised: When the number of clients is small (4 clients), i.e. when each client has large data, local training can be sufficient for building a model that is accurate and robust.

Original: (ii)
Revised: (ii)

Original: As the number of clients increases,i.e. a typical scenario in FL, and the amount of data available to each client is not large, federatedaveraging shines in providing a model with better performance and robustness.
Revised: As the number of clients increases, i.e. a typical scenario in FL, and the amount of data available to each client is not large, federated averaging shines in providing a model with better performance and robustness.

Original: In particular, wefind federated training can bring robustness improvements of over 20% in the few-local-data regime(10 or 20 clients).
Revised: In particular, we find federated training can bring robustness improvements of over 20% in the few-local-data regime (10 or 20 clients).

Original: This can be interpreted as follows: when local data can sufficiently approximatethe real-data distribution, then there is no advantage to conducting federated training.
Revised: This can be interpreted as follows: when local data can sufficiently approximate the real-data distribution, then there is no advantage to conducting federated training.

Original: On the otherhand, when local data is insufficient for approximating the data-distribution, federated learning isuseful.
Revised: On the other hand, when local data is insufficient for approximating the data-distribution, federated learning is useful.

Original: (iii)
Revised: (iii)

Original: The performance and robustness of personalized (fine-tuned) models are comparable tothe global federated model.
Revised: The performance and robustness of personalized (fine-tuned) models are comparable to the global federated model.

Original: We attribute this observation to the federated model being trained withdata augmentation, and thus converging to a reasonably-good local optimum.
Revised: We attribute this observation to the federated model being trained with data augmentation, and thus converging to a reasonably-good local optimum.

Original: Hence, personalizingthis model with few epochs of local training would not have very significant impact.
Revised: Hence, personalizing this model with few epochs of local training would not have very significant impact.

Original: We delve moreinto the benefits of personalization in the next section.
Revised: We delve more into the benefits of personalization in the next section.

Original: (iv)
Revised: (iv)

Original: At last, we observe that, as the numberof clients increases, the performance difference between the best and worst performing clients issignificantly larger for local training.
Revised: At last, we observe that, as the number of clients increases, the performance difference between the best and worst performing clients is significantly larger for local training.

Original: This is evidenced by the spread in Figure 1, where the shadedarea is notably larger for local training.
Revised: This is evidenced by the spread in Figure 1, where the shaded area is notably larger for local training.

Original: We elaborate more on this phenomenon in the appendix.
Revised: We elaborate more on this phenomenon in the appendix.

Original: Is Personalization Beneficial for Robustness?
Revised: Is Personalization Beneficial for Robustness?

Original: Next, we investigate the effect of personalizationon certified robustness.
Revised: Next, we investigate the effect of personalization on certified robustness.

Original: In essence, the experiments in Figure 1 assumed all clients train robustly,aiming at collaboratively building a global robust model.
Revised: In essence, the experiments in Figure 1 assumed all clients train robustly, aiming at collaboratively building a global robust model.

Original: However, in a more realistic federatedsetup, this is not necessarily the case.
Revised: However, in a more realistic federated setup, this is not necessarily the case.

Original: For instance, different clients could target robustness againstdifferent transformations or, in the extreme case, some clients could disregard robustness entirelyand simply target accuracy.
Revised: For instance, different clients could target robustness against different transformations or, in the extreme case, some clients could disregard robustness entirely and simply target accuracy.

Original: To study this case, we use 10 clients and modify the averaging and personalization phases as follows.
Revised: To study this case, we use 10 clients and modify the averaging and personalization phases as follows.

Original: During the federated training phase, all models train without augmentations ( i.e. targeting accuracyand disregarding robustness).
Revised: During the federated training phase, all models train without augmentations ( i.e. targeting accuracy and disregarding robustness).

Original: During the personalization phase, we augment data with the transformation for which the client is interested in being robust against.
Revised: During the personalization phase, we augment data with the transformation for which the client is interested in being robust against.

Original: Experimentally, we personalize
Revised: Experimentally, we personalize

Original: [Figure]
Revised: [Figure]

Original: under the curves, as a function of λ .
Revised: under the curves, as a function of λ .

Original: Our results show that the robustness of local models degrades, when they diverge from the global model.
Revised: Our results show that the robustness of local models degrades, when they diverge from the global model.

Original: against pixel perturbations, rotations, and translations, and report results in Figure 2.
Revised: against pixel perturbations, rotations, and translations, and report results in Figure 2.

Original: Our resultsshow that personalization has a significant impact on improving model robustness.
Revised: Our results show that personalization has a significant impact on improving model robustness.

Original: Moreover, thisimprovement is observed across the board, irrespective of the choice of the augmentation or σ .
Revised: Moreover, this improvement is observed across the board, irrespective of the choice of the augmentation or σ .

Original: Advantages of Personalization.
Revised: Advantages of Personalization.

Original: Further, we address the following question: when is it better torobustly train on local data than to do robust personalization (starting from a nominally trainedfederated model)?
Revised: Further, we address the following question: when is it better to robustly train on local data than to do robust personalization (starting from a nominally trained federated model)?

Original: We examine this question by comparing the performance of a nominally trainedfederated model that is robustly personalized (Figure 2) against local robust training.
Revised: We examine this question by comparing the performance of a nominally trained federated model that is robustly personalized (Figure 2) against local robust training.

Original: We displaythis comparison in Figure 2.
Revised: We display this comparison in Figure 2.

Original: We observe the following: (i) When the target local transformation issimilar to the target federated one (small values of σ ), personalized models significantly outperformlocal models, despite robust training only occurring in the personalization phase.
Revised: We observe the following: (i) When the target local transformation is similar to the target federated one (small values of σ ), personalized models significantly outperform local models, despite robust training only occurring in the personalization phase.

Original: That is, with farless computation and much faster training , federated training with personalization offers a betteralternative to training only on local data.
Revised: That is, with far less computation and much faster training, federated training with personalization offers a better alternative to training only on local data.

Original: Furthermore, the performance gap increases with thenumber of clients—equivalent to assigning each client fewer local data.
Revised: Furthermore, the performance gap increases with the number of clients—equivalent to assigning each client fewer local data.

Original: (ii)
Revised: (ii)

Original: Following intuition,as the difference between the transformation in the federated and personalized settings increases,the performance of personalized models becomes comparable to that of their local counterparts.
Revised: Following intuition, as the difference between the transformation in the federated and personalized settings increases, the performance of personalized models becomes comparable to that of their local counterparts.

Original: Note that this is an artifact of choosing a relatively small number of clients, on which the dataset isdistributed.
Revised: Note that this is an artifact of choosing a relatively small number of clients, on which the dataset is distributed.

Original: Indeed, in the appendix we report how a larger number of clients ( N = 100 ) displayssignificant robustness improvements of personalized models compared to local models.
Revised: Indeed, in the appendix we report how a larger number of clients ( N = 100 ) displays significant robustness improvements of personalized models compared to local models.

Original: FL with Mixture Model
Revised: FL with Mixture Model.

Original: At last, we explore how using the mixture of models formulation in Eq.
Revised: At last, we explore how using the mixture of models formulation in Eq.

Original: (5)affects robustness of the resulting local models.
Revised: (5) affects robustness of the resulting local models.

Original: Note that through all the aforementioned experiments, we studied the case when λ = 1. 0 for local training, or λ = 0. 0 for federated training.
Revised: Note that through all the aforementioned experiments, we studied the case when λ = 1 for local training, or λ = 0 for federated training.

Original: Tothis end, we compare the performance of models trained with various values λ ∈ (0 , 1) , leading todifferent personalization levels.
Revised: To this end, we compare the performance of models trained with various values λ ∈ (0 , 1) , leading to different personalization levels.

Original: That is, during local training, each client is leveraging the updatestep in (6) before communicating the local model to the orchestrating server for an averaging step.
Revised: That is, during local training, each client is leveraging the update step in (6) before communicating the local model to the orchestrating server for an averaging step.

Original: Figure 3 plots the results of this experiment.
Revised: Figure 3 plots the results of this experiment.

Original: The first two columns display certified accuracy curvesagainst rotation and translation for different λ values.
Revised: The first two columns display certified accuracy curves against rotation and translation for different λ values.

Original: The last column summarizes the results interms of Average Certified Radius (ACR), i.e. the average certified radius of correctly-classified examples [39].
Revised: The last column summarizes the results in terms of Average Certified Radius (ACR), i.e. the average certified radius of correctly-classified examples [39].

Original: From these results we observe that: (i) for every deformation (rotation, translation) andevery σ value, the mixture of models formulation outperforms the robustness of standard federatedtraining.
Revised: From these results we observe that: (i) for every deformation (rotation, translation) and every σ value, the mixture of models formulation outperforms the robustness of standard federated training.

Original: This phenomenon is consistent with almost any value of the personalization parameter λ .
Revised: This phenomenon is consistent with almost any value of the personalization parameter λ .

Original: In particular, we observe that setting λ = 0 . 1 , corresponding to the least personalized local models,yields the best results.
Revised: In particular, we observe that setting λ = 0 . 1 , corresponding to the least personalized local models, yields the best results.

Original: (ii)
Revised: (ii)

Original: By contrasting these results to Figure 2, we notice that, across clients, thespread of certified accuracy is smaller than both na¨ıve personalization and local training.
Revised: By contrasting these results to Figure 2, we notice that, across clients, the spread of certified accuracy is smaller than both na¨ıve personalization and local training.

Original: A Additional ExperimentsA.1 Is Federated Averaging Beneficial for Robustness
Revised: A Additional Experiments A.1 Is Federated Averaging Beneficial for Robustness

Original: In Section 4, we analyzed the benefits of personalization on the certified robustness when all clientsare training nominally.
Revised: In Section 4, we analyzed the benefits of personalization on the certified robustness when all clients are training nominally.

Original: Next, we address the following question: what if all other clients aim tobuild a model robust against transformations that are different than the desired one?
Revised: Next, we address the following question: what if all other clients aim to build a model robust against transformations that are different than the desired one?

Original: We analyze thescenario when all clients train with an augmentation that differs from the one a certain client desires.
Revised: We analyze the scenario when all clients train with an augmentation that differs from the one a certain client desires.

Original: Thus, we deploy augmentation with the general affine deformation [9] during the federated trainingphase, and then personalize with either rotation or translation.
Revised: Thus, we deploy augmentation with the general affine deformation [9] during the federated training phase, and then personalize with either rotation or translation.

Original: We report the results in Figure 4 andillustrate that personalization has a positive impact on robustness even when federated training isconducted with a different augmentation (an affine deformation, in this case).
Revised: We report the results in Figure 4 and illustrate that personalization has a positive impact on robustness even when federated training is conducted with a different augmentation (an affine deformation, in this case).

Original: In Figure 1, we analyzed the effect of varying the number of clients on the certified robustnessagainst rotations.
Revised: In Figure 1, we analyzed the effect of varying the number of clients on the certified robustness against rotations.

Original: For completeness, we repeat the experiment with translations, and report theresults in Figure 5.
Revised: For completeness, we repeat the experiment with translations, and report the results in Figure 5.

Original: In particular, in this experiment we distribute CIFAR10 on 4 and 10 clients, andvary σ ∈ { 0 . 1 , 0 . 5 } .
Revised: In particular, in this experiment we distribute CIFAR10 on 4 and 10 clients, and vary σ ∈ { 0 . 1 , 0 . 5 } .

Original: Analogous to our earlier observation, as the number of clients increases, thelocal data becomes insufficient, and the performance and robustness of local training deteriorates.
Revised: Analogous to our earlier observation, as the number of clients increases, the local data becomes insufficient, and the performance and robustness of local training deteriorates.

Original: A.2 100 Clients Experiment
Revised: A.2 100 Clients Experiment

Original: In Section 4, we observed that when the augmentation deployed in the federated training phase differs from the one used in the personalization phase, the performance of local models is comparableto that of personalized models.
Revised: In Section 4, we observed that when the augmentation deployed in the federated training phase differs from the one used in the personalization phase, the performance of local models is comparable to that of personalized models.

Original: We scale our experiments by distributing the dataset on a largernumber of clients (100).
Revised: We scale our experiments by distributing the dataset on a larger number of clients (100).

Original: This makes the local data for each client insufficient for training localmodels.
Revised: This makes the local data for each client insufficient for training local models.

Original: We plot the certified accuracy curves of local training, federated training, and personalizedtraining in Figure 6.
Revised: We plot the certified accuracy curves of local training, federated training, and personalized training in Figure 6.

Original: We observe that the performance gap increases between the personalized andlocal models, providing further evidence to support our hypothesis from Section 4.
Revised: We observe that the performance gap increases between the personalized and local models, providing further evidence to support our hypothesis from Section 4.

Original: That is, federated training and personalization, compared to local training, provide a more reliable approach thatimproves both performance and robustness in more realistic federated scenarios.
Revised: That is, federated training and personalization, compared to local training, provide a more reliable approach that improves both performance and robustness in more realistic federated scenarios.

Original: A.3 Ablation for All Clients
Revised: A.3 Ablation for All Clients

Original: In all of our experiments, we reported the certified accuracy curves averaged across all clients, highlighting the minimum and the maximum performing clients in a shaded region.
Revised: In all of our experiments, we reported the certified accuracy curves averaged across all clients, highlighting the minimum and the maximum performing clients in a shaded region.

Original: For completeness,we plot the certified accuracy for all clients for some of our experiments.
Revised: For completeness, we plot the certified accuracy for all clients for some of our experiments.

Original: In particular, for experiments with 4 clients, we show our results in Figure 7, and show the results on 10 clients in Figure8.
Revised: In particular, for experiments with 4 clients, we show our results in Figure 7, and show the results on 10 clients in Figure 8.

Original: On each plot, we show the performance variation across clients when local training is conducted(compared to personalized training).
Revised: On each plot, we show the performance variation across clients when local training is conducted (compared to personalized training).

Original: We observe, following our earlier observations, that the spreadis significantly higher for local training.
Revised: We observe, following our earlier observations, that the spread is significantly higher for local training.

Original: Note that this behaviour is despite the fact that we distributed the data in a uniform and homogeneous fashion across clients.
Revised: Note that this behaviour is despite the fact that we distributed the data in a uniform and homogeneous fashion across clients.

Original: Hence, in more realisticscenarios with heterogeneous splits, this spread could grow, highlighting the reliability of federated
Revised: Hence, in more realistic scenarios with heterogeneous splits, this spread could grow, highlighting the reliability of federated

Original: [Figure]
Revised: [Figure]

Original: 0.5 Radius 0.
Revised: 0.5 Radius 0.

Original: C e r t i f i e d A cc .
Revised: C e r t i f i e d A cc .

Original: Pixel Perturbations - = 0.
Revised: Pixel Perturbations - = 0.

Original: Local
Revised: Local

Original: Federated
Revised: Federated

Original: Personalized
Revised: Personalized

Original: Spread-L
Revised: Spread-L

Original: Spread-F
Revised: Spread-F

Original: Spread-P 0.
Revised: Spread-P 0.

Original: 0.4 Radius 0.
Revised: 0.4 Radius 0.

Original: C e r t i f i e d
Revised: C e r t i f i e d

Original: A cc .
Revised: A cc .

Original: Translation - = 0.
Revised: Translation - = 0.

Revised: [Figure]

Original: 0.4 Radius 0.
Revised: Radius 0.

Original: Translation - = 0.
Revised: Translation - = 0.

Original: [Figure]

Original: [Figure]

Original: [Figure]

Original: [Figure]

Original: [Figure]
Revised: [Figure]

Original: C e r t i f i e d
Revised: C e r t i f i e d

Original: A cc .
Revised: A cc .

Original: Translation - = 0.
Revised: Translation - = 0.

Original: 2.0 Radius 0.
Revised: 2.0 Radius 0.

Original: Translation - = 0. training and personalization in building accurate and robust models.
Revised: Translation - = 0. training and personalization in building accurate and robust models.

Original: We believe this analysis is aninteresting setup for future work.
Revised: We believe this analysis is an interesting setup for future work.

Original: A.4 Experimental Details
Revised: A.4 Experimental Details

Original: The experiments reported in the paper used the MNIST and CIFAR-10 [44] Datasets 1 .
Revised: The experiments reported in the paper used the MNIST and CIFAR-10 [44] Datasets 1 .

Original: We usedthe available splits provided in PyTorch for training and testing sets.
Revised: We used the available splits provided in PyTorch for training and testing sets.

Original: Moreover, we adopted thepublicly-available codes for certification, as noted in the experimental section.
Revised: Moreover, we adopted the publicly-available codes for certification, as noted in the experimental section.

Original: A.5 FL with Mixture Model
Revised: A.5 FL with Mixture Model

Original: At last, we experiment with the effect of the mixture model on the certified accuracy of the trainedmodel.
Revised: At last, we experiment with the effect of the mixture model on the certified accuracy of the trained model.

Original: In Section 4, our experiments analyzed rotation and translation deformations on CIFAR10.
Revised: In Section 4, our experiments analyzed rotation and translation deformations on CIFAR10.

Original: For completeness, in Figure 9 we also report results against (1) pixel perturbations on CIFAR10 and(2) rotation, translation, and pixel perturbations on MNIST.
Revised: For completeness, in Figure 9 we also report results against (1) pixel perturbations on CIFAR10 and (2) rotation, translation, and pixel perturbations on MNIST.

Original: Our results further confirm our observation in Section 4: the mixture model provides consistent improvement on the certified robustness formost considered values of λ .
Revised: Our results further confirm our observation in Section 4: the mixture model provides consistent improvement on the certified robustness for most considered values of λ .

Original: It is also worth noting that different degrees of personalization performthe best for different perturbations and σ values.
Revised: It is also worth noting that different degrees of personalization perform the best for different perturbations and σ values.

Original: B Social Impact
Revised: B Social Impact

Original: This paper explores (1) certified robustness and (2) federated learning.
Revised: This paper explores (1) certified robustness and (2) federated learning.

Original: Both topics are associatedto how machine learning settings relate to security which, in turn, has a social impact.
Revised: Both topics are associated to how machine learning settings relate to security which, in turn, has a social impact.

Original: In particular,robustness is associated with the secure deployment of models, hindering how malicious agentsmay attack recognition systems to fit their purposes.
Revised: In particular, robustness is associated with the secure deployment of models, hindering how malicious agents may attack recognition systems to fit their purposes.

Original: Furthermore, federated learning is a setting
Revised: Furthermore, federated learning is a setting

Original: 0.5 Radius 0.
Revised: 0.5 Radius 0.

Original: C e r t i f i e d A cc .
Revised: C e r t i f i e d A cc .

Original: Pixel Perturbations - = 0.
Revised: Pixel Perturbations - = 0.

Original: Local
Revised: Local

Original: Federated
Revised: Federated

Original: Personalized
Revised: Personalized

Original: Spread-L
Revised: Spread-L

Original: Spread-F
Revised: Spread-F

Original: Spread-P 0.
Revised: Spread-P 0.

Original: Radius 0.
Revised: Radius 0.

Original: C e r t i f i e d A cc .
Revised: C e r t i f i e d A cc .

Original: Rotation - = 0.
Revised: Rotation - = 0.

Original: Radius 0.
Revised: Radius 0.

Original: Translation - = 0.
Revised: Translation - = 0.

Original: Radius 0.
Revised: Radius 0.

Original: C e r t i f i e d A cc .
Revised: C e r t i f i e d A cc .

Original: Rotation - = 0.
Revised: Rotation - = 0.

Original: Radius 0.
Revised: Radius 0.

Original: Translation - = 0.
Revised: Translation - = 0.

Original: Radius 0.
Revised: Radius 0.

Original: C e r t i f i e d
Revised: C e r t i f i e d

Original: A cc .
Revised: A cc .

Original: Rotation - = 0.
Revised: Rotation - = 0.

Original: client_ client_ client_ client_ 0.
Revised: client_ client_ client_ client_ 0.

Original: Radius 0.
Revised: Radius 0.

Original: C e r t i f i e d A cc .
Revised: C e r t i f i e d A cc .

Original: Rotation - = 0.
Revised: Rotation - = 0.

Original: Radius 0.
Revised: Radius 0.

Original: C e r t i f i e d A
Revised: C e r t i f i e d A

Original: [Figure]
Revised: [Figure]

Original: =
Revised: =

Original: [Figure]
Revised: [Figure]

Original: Rotation - = 0.
Revised: Rotation - = 0.

Original: 1.5 Radius 0.
Revised: 1.5 Radius 0.

Original: C e r t i f i e d
Revised: C e r t i f i e d

Original: A cc .
Revised: A cc .

Original: Rotation - = 0.
Revised: Rotation - = 0.

Original: client_ client_ client_ client_ client_ client_ client_ client_ client_ client_ 0.
Revised: client_ client_ client_ client_ client_ client_ client_ client_ client_ client_ 0.

Original: Radius 0.
Revised: Radius 0.

Original: C e r t i f i e d A cc .
Revised: C e r t i f i e d A cc .

Original: Rotation - = 0.
Revised: Rotation - = 0.

Original: Radius 0.
Revised: Radius 0.

Original: C e r t i f i e d A cc .
Revised: C e r t i f i e d A cc .

Original: Rotation - = 0.
Revised: Rotation - = 0.

Original: that arises naturally when data-privacy constraints are introduced into the learning objective: how todevelop well-performing models that leverage user data while preserving privacy.
Revised: that arises naturally when data-privacy constraints are introduced into the learning objective: how to develop well-performing models that leverage user data while preserving privacy.

Original: C Compute Power
Revised: C Compute Power

Original: All in all, our work has the potential to be used by services targeting high-performance privatemodels to benefit users.
Revised: All in all, our work has the potential to be used by services targeting high-performance private models to benefit users.

Original: In particular, we used 20 Nvidia-V100 GPUs for all of our experiments.
Revised: In particular, we used 20 Nvidia-V100 GPUs for all of our experiments.

Original: D Limitations
Revised: D Limitations

Original: There exist several frameworks that characterize federated training and personalization.
Revised: There exist several frameworks that characterize federated training and personalization.

Original: A limitation of our work is the focus on the popular federated averaging technique.
Revised: A limitation of our work is the focus on the popular federated averaging technique.

Original: A possible interestingavenue for future work is to benchmark different federated and personalization frameworks in termsof robustness.
Revised: A possible interesting avenue for future work is to benchmark different federated and personalization frameworks in terms of robustness.

Original: Moreover, new mixture models could be devised that explicitly target robustness enhancement are left for future works.
Revised: Moreover, new mixture models could be devised that explicitly target robustness enhancement are left for future works.

Original: Moreover, the development of new mixture models explicitlyaimed at improving robustness and reliability is also left for future work.
Revised: Moreover, the development of new mixture models explicitly aimed at improving robustness and reliability is also left for future work.

Original: At last, in this work weconsidered a homogeneous split of data across clients.
Revised: At last, in this work we considered a homogeneous split of data across clients.

Original: That is, all clients have the same amount, andprobably the same set of classes, of data.
Revised: That is, all clients have the same amount, and probably the same set of classes, of data.

Original: We note here, while this setup is less realistic to the realworld setup where different clients have heterogeneous data splits, we remark here that this serves asthe best condition for local training.
Revised: We note here, while this setup is less realistic to the real world setup where different clients have heterogeneous data splits, we remark here that this serves as the best condition for local training.

Original: That is, even when putting local training at advantage, our workshowed the superiority of conducting federated training and personalization in providing modelsthat are both accurate and robust.
Revised: That is, even when putting local training at advantage, our work showed the superiority of conducting federated training and personalization in providing models that are both accurate and robust.

Original: We leave studying the heterogeneous setup for future works.
Revised: We leave studying the heterogeneous setup for future works.

Original: 2.0 Radius 0.
Revised: 2.0 Radius 0.

Original: Translation - = 0.
Revised: Translation - = 0.

Original: =0.
Revised: =0.

Original: =0. =0.
Revised: =0. =0.

Original: =1.
Revised: =1.

Original: =0.
Revised: =0.

Original: =0. =0.
Revised: =0. =0.

Original: =1.
Revised: =1.

Original: 0.4 Radius 0.
Revised: 0.4 Radius 0.

Original: C e r t i f i e d
Revised: C e r t i f i e d

Original: A cc .
Revised: A cc .

Original: Rotation - = 0.
Revised: Rotation - = 0.

Original: 2.0 Radius 0.
Revised: 2.0 Radius 0.

Original: Rotation - = 0.
Revised: Rotation - = 0.

Original: 0.4 Radius 0.
Revised: 0.4 Radius 0.

Original: C e r t i f i e d
Revised: C e r t i f i e d

Original: A cc .
Revised: A cc .

Original: Translation - = 0.
Revised: Translation - = 0.

Original: 2.0 Radius 0.
Revised: 2.0 Radius 0.

Original: Translation - = 0.
Revised: Translation - = 0.

Original: 0.5 Radius 0.
Revised: 0.5 Radius 0.

Original: C e r t i f i e d
Revised: C e r t i f i e d

Original: A cc .
Revised: A cc .

Original: Pixel Perturbations - = 0.
Revised: Pixel Perturbations - = 0.

Original: 2.0 Radius 0.
Revised: 2.0 Radius 0.

Original: Pixel Perturbations - = 0.
Revised: Pixel Perturbations - = 0.

Original: 0.5 Radius 0.
Revised: 0.5 Radius 0.

Original: C e r t i f i e d
Revised: C e r t i f i e d

Original: A cc .
Revised: A cc .

Original: [Figure]
Revised: [Figure]

Original: Perturbations
Revised: Perturbations

Original: [Figure]
Revised: [Figure]