Original: C ERTIFIED
Revised: C ERTIFIED

Original: R OBUSTNESS
Revised: R OBUSTNESS

Original: FOR
Revised: FOR

Original: F REE
Revised: F REE

Original: IN
Revised: IN

Original: D IFFERENTIALLY P RIVATE F EDERATED L EARNING
Revised: D IFFERENTIALLY P RIVATE F EDERATED L EARNING

Original: Anonymous authors
Revised: Anonymous authors

Original: Paper under double-blind review
Revised: Paper under double-blind review

Original: A BSTRACT
Revised: A BSTRACT

Original: Federated learning (FL) provides an efﬁcient training paradigm to jointly train a global model leveraging data from distributed users
Revised: Federated learning (FL) provides an efﬁcient training paradigm to jointly train a global model leveraging data from distributed users

Original: As the local training data comes from different users who may not be trustworthy
Revised: As the local training data comes from different users who may not be trustworthy

Original: , several studies have shown that
Revised: , several studies have shown that

Original: FL is vulnerable to poisoning attacks where adversaries add malicious data during training.
Revised: FL is vulnerable to poisoning attacks where adversaries add malicious data during training.

Original: On the other hand
Revised: On the other hand

Original: , to protect the privacy of users
Revised: , to protect the privacy of users

Original: FL is usually trained in a differentially private manner (DPFL).
Revised: FL is usually trained in a differentially private manner (DPFL).

Original: Given these properties of
Revised: Given these properties of

Original: FL
Revised: FL

Original: , in this paper, we aim to ask
Revised: , in this paper, we aim to ask

Original: : Can we leverage the innate privacy property of
Revised: : Can we leverage the innate privacy property of

Original: DPFL to provide robustness certiﬁcation against poisoning attacks
Revised: DPFL to provide robustness certiﬁcation against poisoning attacks

Original: Can we further improve the privacy of FL to improve such certiﬁcation
Revised: Can we further improve the privacy of FL to improve such certiﬁcation

Original: To this end
Revised: To this end

Original: , we ﬁrst investigate both user level and instance level privacy for
Revised: , we ﬁrst investigate both user level and instance level privacy for

Original: FL
Revised: FL

Original: and propose novel randomization mechanisms and analysis to achieve improved differential privacy
Revised: and propose novel randomization mechanisms and analysis to achieve improved differential privacy

Original: We then provide two robustness certiﬁcation criteria: certiﬁed prediction and certiﬁed attack cost for DPFL on both levels
Revised: We then provide two robustness certiﬁcation criteria: certiﬁed prediction and certiﬁed attack cost for DPFL on both levels

Original: Theoretically, given different privacy properties of
Revised: Theoretically, given different privacy properties of

Original: DPFL we prove their certiﬁed robustness under a bounded number of adversarial users or instances
Revised: DPFL we prove their certiﬁed robustness under a bounded number of adversarial users or instances

Original: Empirically, we conduct extensive experiments to verify our theories under different attacks on a range of datasets.
Revised: Empirically, we conduct extensive experiments to verify our theories under different attacks on a range of datasets.

Original: We show that the global model with a tighter privacy guarantee always provides stronger robustness certiﬁcation in terms of the certiﬁed attack cost , while it may exhibit tradeoffs regarding the certiﬁed prediction .
Revised: We show that the global model with a tighter privacy guarantee always provides stronger robustness certiﬁcation in terms of the certiﬁed attack cost , while it may exhibit tradeoffs regarding the certiﬁed prediction .

Original: We believe our work will inspire future research of developing certiﬁably robust DPFL based on its inherent properties.
Revised: We believe our work will inspire future research of developing certiﬁably robust DPFL based on its inherent properties.

Original:  I NTRODUCTION
Revised:  I NTRODUCTION

Original: Federated Learning (FL), which aims to jointly train a global model with distributed local data, has been widely applied in different applications, such as ﬁnance (Yang et al., 2019b), medical analysis (Brisimi et al., 2018), and user behavior prediction (Hard et al., 2018;
Revised: Federated Learning (FL), which aims to jointly train a global model with distributed local data, has been widely applied in different applications, such as ﬁnance (Yang et al., 2019b), medical analysis (Brisimi et al., 2018), and user behavior prediction (Hard et al., 2018;

Original: Yang et al., 2018; 2019a).
Revised: Yang et al., 2018; 2019a).

Original: However, the fact that the local data and the training process are entirely controlled by the local users who may be adversarial raises great concerns from both security and privacy perspectives.
Revised: However, the fact that the local data and the training process are entirely controlled by the local users who may be adversarial raises great concerns from both security and privacy perspectives.

Original: In particular, recent studies show that FL is vulnerable to different types of training-time attacks, such as model poisoning (Bhagoji et al., 2019), backdoor attacks (Bagdasaryan et al., 2020; Xie et al., 2019; Wang et al., 2020), and label-ﬂipping attacks (Fung et al., 2020).
Revised: In particular, recent studies show that FL is vulnerable to different types of training-time attacks, such as model poisoning (Bhagoji et al., 2019), backdoor attacks (Bagdasaryan et al., 2020; Xie et al., 2019; Wang et al., 2020), and label-ﬂipping attacks (Fung et al., 2020).

Original: Further, privacy concerns have motivated the need to keep the raw data on local devices without sharing.
Revised: Further, privacy concerns have motivated the need to keep the raw data on local devices without sharing.

Original: However, sharing other indirect information such as gradients or model updates as part of the FL training process can also leak sensitive user information (Zhu et al., 2019;
Revised: However, sharing other indirect information such as gradients or model updates as part of the FL training process can also leak sensitive user information (Zhu et al., 2019;

Original: Geiping et al., 2020; Bhowmick et al., 2018;
Revised: Geiping et al., 2020; Bhowmick et al., 2018;

Original: Melis et al., 2019).
Revised: Melis et al., 2019).

Original: As a result, approaches based on differential privacy (DP) (Dwork & Roth, 2014), homomorphic encryption (Bost et al., 2015; Rouhani et al., 2018; Gilad-Bachrach et al., 2016), and secure multiparty computation (Ben-Or et al., 1988; Bonawitz et al., 2017) have been proposed to protect privacy of users in federated learning.
Revised: As a result, approaches based on differential privacy (DP) (Dwork & Roth, 2014), homomorphic encryption (Bost et al., 2015; Rouhani et al., 2018; Gilad-Bachrach et al., 2016), and secure multiparty computation (Ben-Or et al., 1988; Bonawitz et al., 2017) have been proposed to protect privacy of users in federated learning.

Original: In particular, differentially private federated learning (DPFL) provides strong information theoretic guarantees on user privacy, while causing relatively low performance overhead (Li et al., 2020b).
Revised: In particular, differentially private federated learning (DPFL) provides strong information theoretic guarantees on user privacy, while causing relatively low performance overhead (Li et al., 2020b).

Original: Several defenses have been proposed to defend against poisoning attacks in FL.
Revised: Several defenses have been proposed to defend against poisoning attacks in FL.

Original: For instance, various robust aggregation methods (Fung et al., 2020; Pillutla et al., 2019; Blanchard et al., 2017; El Mhamdi et al., 2018; Chen et al., 2017b;
Revised: For instance, various robust aggregation methods (Fung et al., 2020; Pillutla et al., 2019; Blanchard et al., 2017; El Mhamdi et al., 2018; Chen et al., 2017b;

Original: Yin et al., 2018; Fu et al., 2019;
Revised: Yin et al., 2018; Fu et al., 2019;

Original: Li et al., 2020a) identify and down-weight the malicious updates during aggregation or estimate a true “center” of the received updates rather than taking a weighted average.
Revised: Li et al., 2020a) identify and down-weight the malicious updates during aggregation or estimate a true “center” of the received updates rather than taking a weighted average.

Original: Other methods include robust federated training protocols (e.g., clipping (Sun et al., 2019), noisy perturbation (Sun et al., 2019), and additional evaluation during training (Andreina et al., 2020)) and post-training strategies (e.g., ﬁne-tuning and pruning (Wu et al., 2020)) that repair the poisoned global model.
Revised: Other methods include robust federated training protocols (e.g., clipping (Sun et al., 2019), noisy perturbation (Sun et al., 2019), and additional evaluation during training (Andreina et al., 2020)) and post-training strategies (e.g., ﬁne-tuning and pruning (Wu et al., 2020)) that repair the poisoned global model.

Original: However, as these works mainly focus on providing empirical robustness for FL, they have been shown to be vulnerable to newly proposed strong adaptive attacks (Wang et al., 2020; Xie et al., 2019; Baruch et al., 2019; Fang et al., 2020).
Revised: However, as these works mainly focus on providing empirical robustness for FL, they have been shown to be vulnerable to newly proposed strong adaptive attacks (Wang et al., 2020; Xie et al., 2019; Baruch et al., 2019; Fang et al., 2020).

Original: Hence, in this paper, we aim to develop certiﬁed robustness guarantees for FL against different poisoning attacks.
Revised: Hence, in this paper, we aim to develop certiﬁed robustness guarantees for FL against different poisoning attacks.

Original: Further, as differentially private federated learning (DPFL) is often used to protect user privacy, we also aim to ask: Can we leverage the innate privacy property of DPFL to provide robustness certiﬁcation against poisoning attacks for free?
Revised: Further, as differentially private federated learning (DPFL) is often used to protect user privacy, we also aim to ask: Can we leverage the innate privacy property of DPFL to provide robustness certiﬁcation against poisoning attacks for free?

Original: Can we further improve the privacy of FL so as to improve its certiﬁed robustness?
Revised: Can we further improve the privacy of FL so as to improve its certiﬁed robustness?

Original: Recent studies suggest that differential privacy (DP) is inherently related with robustness of ML models.
Revised: Recent studies suggest that differential privacy (DP) is inherently related with robustness of ML models.

Original: Intuitively, DP is designed to protect the privacy of individual data, such that the output of an algorithm remains essentially unchanged when one individual input point is modiﬁed.
Revised: Intuitively, DP is designed to protect the privacy of individual data, such that the output of an algorithm remains essentially unchanged when one individual input point is modiﬁed.

Original: Hence, the prediction of a DP model will be less impacted by a small amount of poisoned training data.
Revised: Hence, the prediction of a DP model will be less impacted by a small amount of poisoned training data.

Original: Consequently, DP has been used to provide both theoretical and empirical defenses against evasion attacks (Lecuyer et al., 2019a) and data poisoning attacks (Ma et al., 2019; Hong et al., 2020) on centralized ML models.
Revised: Consequently, DP has been used to provide both theoretical and empirical defenses against evasion attacks (Lecuyer et al., 2019a) and data poisoning attacks (Ma et al., 2019; Hong et al., 2020) on centralized ML models.

Original: It has also been used as an empirical defense against backdoor attacks (Gu et al., 2019) in federated learning (Bagdasaryan et al., 2020; Sun et al., 2019), although no theoretical guarantee is provided.
Revised: It has also been used as an empirical defense against backdoor attacks (Gu et al., 2019) in federated learning (Bagdasaryan et al., 2020; Sun et al., 2019), although no theoretical guarantee is provided.

Original: To the best of our knowledge, despite of the wide application of DPFL,there is no work providing certiﬁed robustness for DPFL leveraging its privacy property.
Revised: To the best of our knowledge, despite of the wide application of DPFL,there is no work providing certiﬁed robustness for DPFL leveraging its privacy property.

Original: In this paper, we aim to leverage the inherent privacy property of DPFL to provide robustness certiﬁcation for FL against poisoning attacks for free .
Revised: In this paper, we aim to leverage the inherent privacy property of DPFL to provide robustness certiﬁcation for FL against poisoning attacks for free .

Original: Our challenges include: (1) performing privacy analysis over training rounds in DPFL algorithms and (2) theoretically guaranteeing certiﬁed robustness based on DP properties under a given privacy budget.
Revised: Our challenges include: (1) performing privacy analysis over training rounds in DPFL algorithms and (2) theoretically guaranteeing certiﬁed robustness based on DP properties under a given privacy budget.

Original: We propose two robustness certiﬁcation criteria for FL: certiﬁed prediction and certiﬁed attack cost under different attack constraints.
Revised: We propose two robustness certiﬁcation criteria for FL: certiﬁed prediction and certiﬁed attack cost under different attack constraints.

Original: We consider both user-level DP (Agarwal et al., 2018; Geyer et al., 2017; McMahan et al., 2018; Asoodeh & Calmon, 2020; Liang et al., 2020) which is widely guaranteed in FL, and instance-level DP (Malekzadeh et al., 2021; Zhu et al., 2021) which is less explored in FL.
Revised: We consider both user-level DP (Agarwal et al., 2018; Geyer et al., 2017; McMahan et al., 2018; Asoodeh & Calmon, 2020; Liang et al., 2020) which is widely guaranteed in FL, and instance-level DP (Malekzadeh et al., 2021; Zhu et al., 2021) which is less explored in FL.

Original: We prove that a FL model satisfying user-level DP is certiﬁably robust against a bounded number of adversarial users.
Revised: We prove that a FL model satisfying user-level DP is certiﬁably robust against a bounded number of adversarial users.

Original: In addition, we propose InsDP-FedAvg algorithm to improve instance-level DP in FL, and prove that instance-level DPFL is certiﬁably robust against a bounded number of adversarial instances.
Revised: In addition, we propose InsDP-FedAvg algorithm to improve instance-level DP in FL, and prove that instance-level DPFL is certiﬁably robust against a bounded number of adversarial instances.

Original: We also study the correlation between privacy guarantee and certiﬁed robustness of FL.
Revised: We also study the correlation between privacy guarantee and certiﬁed robustness of FL.

Original: While stronger privacy guarantees result in greater attack cost, overly strong privacy can hurt the certiﬁed prediction by introducing too much noise in the training process.
Revised: While stronger privacy guarantees result in greater attack cost, overly strong privacy can hurt the certiﬁed prediction by introducing too much noise in the training process.

Original: Thus, the optimal certiﬁed prediction is often achieved under a proper balance between privacy protection and utility loss.
Revised: Thus, the optimal certiﬁed prediction is often achieved under a proper balance between privacy protection and utility loss.

Original: Key Contributions.
Revised: Key Contributions.

Original: Our work takes the ﬁrst step to provide certiﬁed robustness in DPFL for free against poisoning attacks.
Revised: Our work takes the ﬁrst step to provide certiﬁed robustness in DPFL for free against poisoning attacks.

Original: We make contributions on both theoretical and empirical fronts.
Revised: We make contributions on both theoretical and empirical fronts.

Original: • We propose two criteria for certiﬁed robustness of FL against poisoning attacks.
Revised: • We propose two criteria for certiﬁed robustness of FL against poisoning attacks (Section 4.2).

Original: • Given a FL model satisfying user-level DP, we prove that it is certiﬁably robust against arbitrary poisoning attacks with a bounded number of adversarial users.
Revised: • Given a FL model satisfying user-level DP, we prove that it is certiﬁably robust against arbitrary poisoning attacks with a bounded number of adversarial users (Section 4.2).

Original: • We propose InsDP-FedAvg algorithm to improve FL instance-level privacy guarantee.
Revised: • We propose InsDP-FedAvg algorithm to improve FL instance-level privacy guarantee (Section 5.1).

Original: We prove that instance-level DPFL is certiﬁably robust against the manipulation of a bounded number of instances during training.
Revised: We prove that instance-level DPFL is certiﬁably robust against the manipulation of a bounded number of instances during training (Section 5.2).

Original: • We conduct extensive experiments on MNIST and CIFAR datasets to verify our theoretical results.
Revised: • We conduct extensive experiments on image classiﬁcation of MNIST, CIFAR-10 and sentiment analysis of tweets to verify our proposed certiﬁcations of two robustness criteria, and compare the certiﬁed results of different DPFL algorithms (Section 6).

Original:  R ELATED WORK
Revised:  R ELATED WORK

Original: Differentially Private Federated Learning.
Revised: Differentially Private Federated Learning.

Original: Different approaches are proposed to guarantee the user-level privacy for FL.
Revised: Different approaches are proposed to guarantee the user-level privacy for FL.

Original: (Geyer et al., 2017; McMahan et al., 2018) clip the norm of each local update, add Gaussian noise on the summed update, and characterize its privacy budget via moment accountant (Abadi et al., 2016).
Revised: (Geyer et al., 2017; McMahan et al., 2018) clip the norm of each local update, add Gaussian noise on the summed update, and characterize its privacy budget via moment accountant (Abadi et al., 2016).

Original: (McMahan et al., 2018) extends (Geyer et al., 2017) to language models.
Revised: (McMahan et al., 2018) extends (Geyer et al., 2017) to language models.

Original: In CpSGD (Agarwal et al., 2018), each user clips and quantizes the model update, and adds noise drawn from Binomial distribution, achieving both communication efﬁciency and DP.
Revised: In CpSGD (Agarwal et al., 2018), each user clips and quantizes the model update, and adds noise drawn from Binomial distribution, achieving both communication efﬁciency and DP.

Original: (Bhowmick et al., 2018) derive DP for FL via Renyi divergence (Mironov, 2017) and study its protection against data reconstruction attacks.
Revised: (Bhowmick et al., 2018) derive DP for FL via R´enyi divergence (Mironov, 2017) and study its protection against data reconstruction attacks.

Original: (Liang et al., 2020) utilizes Laplacian smoothing for each local update to enhance the model utility.
Revised: (Liang et al., 2020) utilizes Laplacian smoothing for each local update to enhance the model utility.

Original: Instead of using moment accountant to track privacy budget over FL rounds as previous work, (Asoodeh & Calmon, 2020) derives the DP parameters by interpreting each round as a Markov kernel and quantify its impact on privacy parameters.
Revised: Instead of using moment accountant to track privacy budget over FL rounds as previous work, (Asoodeh & Calmon, 2020) derives the DP parameters by interpreting each round as a Markov kernel and quantify its impact on privacy parameters.

Original: All these works only focus on providing user-level privacy, leaving its robustness property unexplored.
Revised: All these works only focus on providing user-level privacy, leaving its robustness property unexplored.

Original: In terms of instance-level privacy for FL, there are only a few work (Malekzadeh et al., 2021; Zhu et al., 2021).
Revised: In terms of instance-level privacy for FL, there are only a few work (Malekzadeh et al., 2021; Zhu et al., 2021).

Original: Dopamine (Malekzadeh et al., 2021) provides instance-level privacy guarantee when each user only performs one step of DP-SGD (Abadi et al., 2016) at each FL round.
Revised: Dopamine (Malekzadeh et al., 2021) provides instance-level privacy guarantee when each user only performs one step of DP-SGD (Abadi et al., 2016) at each FL round.

Original: However, it cannot be applied to multi-step SGD for each user, thus it cannot be extended to the general FL setting FedAvg (McMahan et al., 2017).
Revised: However, it cannot be applied to multi-step SGD for each user, thus it cannot be extended to the general FL setting FedAvg (McMahan et al., 2017).

Original: (Zhu et al., 2021) privately aggregate the labels from users in a voting scheme, and provide DP guarantees on both user level and instance level.
Revised: (Zhu et al., 2021) privately aggregate the labels from users in a voting scheme, and provide DP guarantees on both user level and instance level.

Original: However, it is also not applicable to standard FL, since it does not allow aggregating the gradients or updates.
Revised: However, it is also not applicable to standard FL, since it does not allow aggregating the gradients or updates.

Original: Differential Privacy and Robustness.
Revised: Differential Privacy and Robustness.

Original: In standard (centralized) learning, Pixel-DP (Lecuyer et al., 2019a) is proposed to certify the model robsutness against evasion attacks.
Revised: In standard (centralized) learning, Pixel-DP (Lecuyer et al., 2019a) is proposed to certify the model robsutness against evasion attacks.

Original: However, it is unclear how to leverage it to certify against poisoning attacks.
Revised: However, it is unclear how to leverage it to certify against poisoning attacks.

Original: To certify the robustness against poisoning attacks, (Ma et al., 2019) show that private learners are resistant to data poisoning and analyze the lower bound of attack cost against poisoning attacks for regression models.
Revised: To certify the robustness against poisoning attacks, (Ma et al., 2019) show that private learners are resistant to data poisoning and analyze the lower bound of attack cost against poisoning attacks for regression models.

Original: Here we certify the robustness in DPFL setting with such lower bound as one of our certiﬁcation criteria and additionally derive its upper bounds.
Revised: Here we certify the robustness in DPFL setting with such lower bound as one of our certiﬁcation criteria and additionally derive its upper bounds.

Original: (Hong et al., 2020) show that the off-the-shelf mechanism DP-SGD (Abadi et al., 2016), which clips per-sample gradients and add Guassian noises during training, can serve as a defense against poisoning attacks empirically.
Revised: (Hong et al., 2020) show that the off-the-shelf mechanism DP-SGD (Abadi et al., 2016), which clips per-sample gradients and add Guassian noises during training, can serve as a defense against poisoning attacks empirically.

Original: In federated learning , empirical work (Bagdasaryan et al., 2020; Sun et al., 2019) show that DPFL can mitigate backdoor attacks; however, none of these work provides certiﬁed robustness guarantees for DPFL against poisoning attacks.
Revised: In federated learning , empirical work (Bagdasaryan et al., 2020; Sun et al., 2019) show that DPFL can mitigate backdoor attacks; however, none of these work provides certiﬁed robustness guarantees for DPFL against poisoning attacks.

Original:  P RELIMINARIES
Revised:  P RELIMINARIES

Original: We start by providing some background on differential privacy (DP) and federated learning (FL).
Revised: We start by providing some background on differential privacy (DP) and federated learning (FL).

Original: Differential Privacy (DP).
Revised: Differential Privacy (DP).

Original: DP is a formal, mathematically rigorous deﬁnition (and standard) of privacy that intuitively guarantees that a randomized algorithm behaves similarly on similar inputs and that the output of the algorithm is about the same whether or not an individual’s data is included as part of the input (Dwork & Roth, 2014).
Revised: DP is a formal, mathematically rigorous deﬁnition (and standard) of privacy that intuitively guarantees that a randomized algorithm behaves similarly on similar inputs and that the output of the algorithm is about the same whether or not an individual’s data is included as part of the input (Dwork & Roth, 2014).

Original: Deﬁnition 1 ( ( (cid:15), δ ) -DP
Revised: Deﬁnition 1 ( ( (cid:15), δ ) -DP

Original: (Dwork et al., 2006)) .
Revised: (Dwork et al., 2006)) .

Original: A randomized mechanism M : D → Θ with domain
Revised: A randomized mechanism M : D → Θ with domain

Original: D and range Θ satisﬁes ( (cid:15), δ ) -DP if for any pair of two adjacent datasets d, d (cid:48) ∈ D , and for any possible (measurable) output set E ⊆ Θ , it holds that Pr[ M ( d ) ∈ E ] ≤ e (cid:15)
Revised: D and range Θ satisﬁes ( (cid:15), δ ) -DP if for any pair of two adjacent datasets d, d (cid:48) ∈ D , and for any possible (measurable) output set E ⊆ Θ , it holds that Pr[ M ( d ) ∈ E ] ≤ e (cid:15)

Original: Pr [ M ( d (cid:48) )
Revised: Pr [ M ( d (cid:48) )

Original: ∈ E ]
Revised: ∈ E ]

Original: + δ .
Revised: + δ .

Original: In Deﬁnition 1, when M is a training algorithm for ML model, domain D and range Θ represent all possible training datasets and all possible trained models respectively.
Revised: In Deﬁnition 1, when M is a training algorithm for ML model, domain D and range Θ represent all possible training datasets and all possible trained models respectively.

Original: Group DP for ( (cid:15), δ ) -DP mechanisms follows immediately from Deﬁnition 1 where the privacy guarantee drops with the size of the group.
Revised: Group DP for ( (cid:15), δ ) -DP mechanisms follows immediately from Deﬁnition 1 where the privacy guarantee drops with the size of the group.

Original: Formally, it says:
Revised: Formally, it says:

Original: Lemma 1 (Group DP) .
Revised: Lemma 1 (Group DP) .

Original: For mechanism M that satisﬁes ( (cid:15), δ ) -DP, it satisﬁes ( k(cid:15), 1 − e k(cid:15) 1 − e (cid:15) δ ) -DP for groups of size k .
Revised: For mechanism M that satisﬁes ( (cid:15), δ ) -DP, it satisﬁes ( k(cid:15), 1 − e k(cid:15) 1 − e (cid:15) δ ) -DP for groups of size k .

Original: That is, for any d, d (cid:48)
Revised: That is, for any d, d (cid:48)

Original: ∈ D that differ by k individuals, and any E ⊆
Revised: ∈ D that differ by k individuals, and any E ⊆

Original: Θ it holds that
Revised: Θ it holds that

Original: Pr[ M ( d ) ∈ E ] [Equation] [Equation] ] +
Revised: Pr[ M ( d ) ∈ E ] ≤ e k(cid:15) Pr [ M ( d (cid:48) )

Original: [Equation]

Original: −

Original: [Equation] − [Equation] δ .
Revised: ∈ E ] + 1 − e k(cid:15) 1 − e (cid:15) δ .

Original: Federated Learning.
Revised: Federated Learning.

Original: FedAvg was introduced by (McMahan et al., 2017) for FL to train a shared global model without direct access to training data of users.
Revised: FedAvg was introduced by (McMahan et al., 2017) for FL to train a shared global model without direct access to training data of users.

Original: Speciﬁcally, given a FL system with
Revised: Speciﬁcally, given a FL system with

Original: N users, at round t , the server sends the current global model w t − 1 to users in the selected user set
Revised: N users, at round t , the server sends the current global model w t − 1 to users in the selected user set

Original: U t , where | U t | = m = qN and q is the user sampling probability.
Revised: U t , where | U t | = m = qN and q is the user sampling probability.

Original: Each selected user i ∈
Revised: Each selected user i ∈

Original: U t locally updates the model for E local epochs with its dataset D i and learning rate η to obtain a new local model.
Revised: U t locally updates the model for E local epochs with its dataset D i and learning rate η to obtain a new local model.

Original: Then, the user sends the local model updates ∆ w it to the server.
Revised: Then, the user sends the local model updates ∆ w it to the server.

Original: Finally, the server aggregates over the updates from all selected users into the new global model w t =
Revised: Finally, the server aggregates over the updates from all selected users into the new global model w t =

Original: w t − 1 + 1 m (cid:80) i ∈
Revised: w t − 1 + 1 m (cid:80) i ∈

Original: U t ∆ w it .
Revised: U t ∆ w it .

Original:  U SER - LEVEL P RIVACY AND C ERTIFIED R OBUSTNESS FOR FL 4. U SER - LEVEL P RIVACY 
Revised:  U SER - LEVEL P RIVACY AND C ERTIFIED R OBUSTNESS FOR FL 4. U SER - LEVEL P RIVACY AND B ACKGROUND

Original: Deﬁnition 1 leaves the deﬁnition of adjacent datasets ﬂexible, which depends on applications.
Revised: Deﬁnition 1 leaves the deﬁnition of adjacent datasets ﬂexible, which depends on applications.

Original: To protect user-level privacy, adjacent datasets are deﬁned as those differing by data from one user (McMahan et al., 2018).
Revised: To protect user-level privacy, adjacent datasets are deﬁned as those differing by data from one user (McMahan et al., 2018).

Original: Deﬁnition 2 (User-level ( (cid:15), δ ) -DP) .
Revised: The formal deﬁnition of User-level ( (cid:15), δ ) -DP

Original: Let B, B (cid:48) be two user sets with size N .

Original: Let D and D (cid:48) be the datasets that are the union of local training examples from all users in B and B (cid:48) respectively. Then, D and D (cid:48) are adjacent if B and B (cid:48) differ by one user. The mechanism M satisﬁes user-level ( (cid:15), δ ) -DP if it meets Deﬁnition 1 with D and D (cid:48) as adjacent datasets.
Revised: (Deﬁnition 2) is omitted to Appendix A.1.

Original: Following standard DPFL (Geyer et al., 2017; McMahan et al., 2018), we introduce  UserDP-FedAvg (Algorithm 1 in Appendix A.1) to achieve user-level DP.
Revised: Following standard DPFL (Geyer et al., 2017; McMahan et al., 2018), we introduce one of standard user-level DPFL algorithms UserDP-FedAvg (Algorithm 1 in Appendix A.1).

Original: At each round, the server ﬁrst clips the update from each user with a threshold S such that its (cid:96) 2 -sensitivity is upper bounded by S .
Revised: At each round, the server ﬁrst clips the update from each user with a threshold S such that its (cid:96) 2 -sensitivity is upper bounded by S .

Original: Next, the server sums up the updates, adds Gaussian noise sampled from N (0 , σ 2 S 2 ) , and takes the average, i.e., w t ← w t − 1 + 1 m (cid:0)(cid:80) i ∈
Revised: Next, the server sums up the updates, adds Gaussian noise sampled from N (0 , σ 2 S 2 ) , and takes the average, i.e., w t ← w t − 1 + 1 m (cid:0)(cid:80) i ∈

Original: U t Clip(∆ w it , S ) + N (cid:0) 0 , σ 2 S 2 (cid:1)(cid:1) .
Revised: U t Clip(∆ w it , S ) + N (cid:0) 0 , σ 2 S 2 (cid:1)(cid:1) .

Original: We utilize the moment accountant (Abadi et al., 2016) to compute the DP guarantee.

Original: Compared to the standard composition theorem (Dwork & Roth, 2014), the moment accounting method provides tighter bounds for the repeated application of the Gaussian mechanism combined with amplication-via-sampling.

Original: Given the user sampling probability q , noise level σ , FL rounds T , and a δ > 0 ,  UserDP-FedAvg satisﬁes ( (cid:15), δ ) -DP as below, which is a generalization of (Abadi et al., 2016).The proof of proposition 1 is deferred to Appendix A.1.
Revised: Given the user sampling probability q , noise level σ , FL rounds T , and a δ > 0 , the privacy analysis of UserDP-FedAvg satisfying ( (cid:15), δ ) -DP is given by Proposition 1 in Appendix A.1, which is a generalization of (Abadi et al., 2016).

Original: The main challenge is to analyze privacy budget (cid:15) , which is accumulated as T increases due to the continuous access to training data.
Revised: The aim of Proposition 1 is to analyze privacy budget (cid:15) in FL, which is accumulated as T increases due to the continuous access to training data.

Original: Proposition 1 ( UserDP-FedAvg Privacy Guarantee) . There exist constants c 1 and c 2 so that given user sampling probability q , and FL rounds T , for any ε < c 1 q 2 T , if σ ≥ c 2 q √ T log(1 /δ ) (cid:15) , the randomized mechanism M in Algorithm 1 is ( (cid:15), δ ) -DP for any δ > 0 .
Revised: Following (Geyer et al., 2017; McMahan et al., 2018), moment accountant (Abadi et al., 2016) is used in the privacy analysis.

Original: 4. C ERTIFIED R OBUSTNESS OF U SER - LEVEL DPFL AGAINST P OISONING A TTACKS
Revised: 4. C ERTIFIED R OBUSTNESS OF U SER - LEVEL DPFL AGAINST P OISONING A TTACKS

Original: Threat Model.
Revised: Threat Model.

Original: We consider the poisoning attacks against FL, where k adversarial users have poisoned instances in local datasets, aiming to fool the trained DPFL global model.
Revised: We consider the poisoning attacks against FL, where k adversarial users have poisoned instances in local datasets, aiming to fool the trained DPFL global model.

Original: Such attacks include backdoor attacks (Gu et al., 2019; Chen et al., 2017a) and label ﬂipping attacks (Biggio et al., 2012; Huang et al., 2011).
Revised: Such attacks include backdoor attacks (Gu et al., 2019; Chen et al., 2017a) and label ﬂipping attacks (Biggio et al., 2012; Huang et al., 2011).

Original: The detailed description of these attacks is deferred to Appendix A.2.
Revised: The detailed description of these attacks is deferred to Appendix A.2.

Original: Note that our robustness certiﬁcation is attack-agnostic under certain attack constraints ( e.g. , k ), and we will verify our certiﬁcation bounds with different poisoning attacks in Section 6.
Revised: Note that our robustness certiﬁcation is attack-agnostic under certain attack constraints ( e.g. , k ), and we will verify our certiﬁcation bounds with different poisoning attacks in Section 6.

Original: Next, we propose two criteria for the robustness certiﬁcation in FL: certiﬁed prediction and certiﬁed attack cost .
Revised: Next, we propose two criteria for the robustness certiﬁcation in FL: certiﬁed prediction and certiﬁed attack cost .

Original: Certiﬁed Prediction.
Revised: Certiﬁed Prediction.

Original: Consider the classiﬁcation task with C classes.
Revised: Consider the classiﬁcation task with C classes.

Original: We deﬁne the classiﬁcation scoring function f : (Θ , R d ) → Υ C which maps model parameters θ ∈ Θ and an input data x ∈ R d to a conﬁdence vector f ( θ, x ) , and f c ( θ, x ) ∈ [0 , 1] represents the conﬁdence of class c .
Revised: We deﬁne the classiﬁcation scoring function f : (Θ , R d ) → Υ C which maps model parameters θ ∈ Θ and an input data x ∈ R d to a conﬁdence vector f ( θ, x ) , and f c ( θ, x ) ∈ [0 , 1] represents the conﬁdence of class c .

Original: We mainly focus on the conﬁdence after normalization, i.e., f ( θ, x ) ∈ Υ C
Revised: We mainly focus on the conﬁdence after normalization, i.e., f ( θ, x ) ∈ Υ C

Original: = { p ∈ R C ≥ 0 : (cid:107) p (cid:107) 1 = 1 } in the probability simplex.
Revised: = { p ∈ R C ≥ 0 : (cid:107) p (cid:107) 1 = 1 } in the probability simplex.

Original: Since the DP mechanism M is randomized and produces a stochastic FL global model θ = M ( D ) , it is natural to resort to a probabilistic expression as a bridge for quantitative robustness certiﬁcations.
Revised: Since the DP mechanism M is randomized and produces a stochastic FL global model θ = M ( D ) , it is natural to resort to a probabilistic expression as a bridge for quantitative robustness certiﬁcations.

Original: Following the convention in (Lecuyer et al., 2019b; Ma et al., 2019), we use the expectation of the model’s prediction to provide a quantitative guarantee on the robustness of M .
Revised: Following the convention in (Lecuyer et al., 2019b; Ma et al., 2019), we use the expectation of the model’s prediction to provide a quantitative guarantee on the robustness of M .

Original: Speciﬁcally, we deﬁne the expected scoring function F : ( θ, R d ) → Υ C where F c ( M ( D ) , x ) =
Revised: Speciﬁcally, we deﬁne the expected scoring function F : ( θ, R d ) → Υ C where F c ( M ( D ) , x ) =

Original: E [ f c ( M ( D ) , x )] is the expected conﬁdence for class c .
Revised: E [ f c ( M ( D ) , x )] is the expected conﬁdence for class c .

Original: The expectation is taken over DP training randomness, e.g., random Gaussian noise and random user subsampling.
Revised: The expectation is taken over DP training randomness, e.g., random Gaussian noise and random user subsampling.

Original: The corresponding prediction H : ( θ, R d ) → [ C ] is deﬁned by H ( M ( D ) , x ) :=
Revised: The corresponding prediction H : ( θ, R d ) → [ C ] is deﬁned by H ( M ( D ) , x ) :=

Original: arg max c ∈ [ C ] F c ( M ( D ) , x ) , which is the top-1 class based on the expected prediction conﬁdence.
Revised: arg max c ∈ [ C ] F c ( M ( D ) , x ) , which is the top-1 class based on the expected prediction conﬁdence.

Original: We will prove that such prediction allows robustness certiﬁcation against poisoning attacks.
Revised: We will prove that such prediction allows robustness certiﬁcation against poisoning attacks.

Original: Following our threat model above and DPFL training in Algorithm 1, we denote the trained global model exposed to poisoning attacks by M ( D (cid:48) ) .
Revised: Following our threat model above and DPFL training in Algorithm 1, we denote the trained global model exposed to poisoning attacks by M ( D (cid:48) ) .

Original: When k = 1 , D and D (cid:48) are user-level adjacent datasets according to Deﬁnition 2.
Revised: When k = 1 , D and D (cid:48) are user-level adjacent datasets according to Deﬁnition 2.

Original: Given that mechanism M satisﬁes user-level ( (cid:15), δ ) -DP, based on the innate DP property, the distribution of the stochastic model M ( D (cid:48) ) is “close” to the distribution of M ( D ) .
Revised: Given that mechanism M satisﬁes user-level ( (cid:15), δ ) -DP, based on the innate DP property, the distribution of the stochastic model M ( D (cid:48) ) is “close” to the distribution of M ( D ) .

Original: Moreover, according to the post-processing property of DP, during testing, given a test sample x , we would expect the values of the expected conﬁdence for each class c , i.e.,
Revised: Moreover, according to the post-processing property of DP, during testing, given a test sample x , we would expect the values of the expected conﬁdence for each class c , i.e.,

Original: F c ( M ( D (cid:48) ) , x ) and F c ( M ( D ) , x ) , to be close, and hence the returned most likely class to be the same , i.e., H ( M ( D ) , x )
Revised: F c ( M ( D (cid:48) ) , x ) and F c ( M ( D ) , x ) , to be close, and hence the returned most likely class to be the same , i.e., H ( M ( D ) , x )

Original: =
Revised: =

Original: H ( M ( D (cid:48) ) , x ) , indicating robust prediction against poisoning attacks.
Revised: H ( M ( D (cid:48) ) , x ) , indicating robust prediction against poisoning attacks.

Original: Theorem 1 (Condition for Certiﬁed Prediction under One Adversarial User) .
Revised: Theorem 1 (Condition for Certiﬁed Prediction under One Adversarial User) .

Original: Suppose a randomized mechanism M satisﬁes user-level ( (cid:15), δ ) -DP.
Revised: Suppose a randomized mechanism M satisﬁes user-level ( (cid:15), δ ) -DP.

Original: For two user sets B and B (cid:48) that differ by one user, let
Revised: For two user sets B and B (cid:48) that differ by one user, let

Original: D and D (cid:48) be the corresponding training datasets.
Revised: D and D (cid:48) be the corresponding training datasets.

Original: For a test input x , suppose A , B ∈ [ C ] satisfy A = arg maxc ∈
Revised: For a test input x , suppose A , B ∈ [ C ] satisfy A = arg max

Original: [ C ] F c ( M ( D ) , x ) and B = arg max c ∈ [ C ]: c (cid:54) =
Revised: [Equation] and B = arg [Equation]

Original: A F c ( M ( D ) , x ) , then ifF A (
Revised: M [Equation] D ) , x ) , then if

Original: [Equation]
Revised: [Equation]

Original: it is guaranteed that H ( M ( D (cid:48) ) , x ) =
Revised: it is guaranteed that H ( M ( D (cid:48) ) , x ) =

Original: H ( M ( D ) , x ) = A .
Revised: H ( M ( D ) , x ) = A .

Original: When k > 1 , we resort to group DP.
Revised: When k > 1 , we resort to group DP.

Original: According to Lemma 1, given mechanism M satisfying userlevel ( (cid:15), δ ) -DP, it also satisﬁes user-level ( k(cid:15), 1 − e k(cid:15) 1 − e (cid:15) δ ) -DP for groups of size k .
Revised: According to Lemma 1, given mechanism M satisfying userlevel ( (cid:15), δ ) -DP, it also satisﬁes user-level ( k(cid:15), 1 − e k(cid:15) 1 − e (cid:15) δ ) -DP for groups of size k .

Original: When k is smaller than a certain threshold, leveraging the group DP property, we would expect that the distribution of the stochastic model M ( D (cid:48) ) is not too far away from the distribution of M ( D ) such that they would make the same prediction for a test sample with probabilistic guarantees.
Revised: When k is smaller than a certain threshold, leveraging the group DP property, we would expect that the distribution of the stochastic model M ( D (cid:48) ) is not too far away from the distribution of M ( D ) such that they would make the same prediction for a test sample with probabilistic guarantees.

Original: Therefore, the privacy and robustness guarantees are simultaneously met by M .
Revised: Therefore, the privacy and robustness guarantees are simultaneously met by M .

Original: Theorem 2 (Upper Bound of k for Certiﬁed Prediction) .
Revised: Theorem 2 (Upper Bound of k for Certiﬁed Prediction) .

Original: Suppose a randomized mechanism M satisﬁes user-level ( (cid:15), δ ) -DP.
Revised: Suppose a randomized mechanism M satisﬁes user-level ( (cid:15), δ ) -DP.

Original: For two user sets B and B (cid:48) that differ by k users, let D and
Revised: For two user sets B and B (cid:48) that differ by k users, let D and

Original: D (cid:48) be the corresponding training datasets.
Revised: D (cid:48) be the corresponding training datasets.

Original: For a test input x , suppose A , B ∈ [ C ] satisfy A = arg max c ∈
Revised: For a test input x , suppose A , B ∈ [ C ] satisfy A = arg max c ∈

Original: [ C ] F c ( M ( D ) , x ) and B = arg max c ∈ [ C ]: c (cid:54) =
Revised: [ C ] F c ( M ( D ) , x ) and B = arg max c ∈ [ C ]: c (cid:54) =

Original: A F c ( M ( D ) , x ) , then H ( M ( D (cid:48) ) , x ) =
Revised: A F c ( M ( D ) , x ) , then H ( M ( D (cid:48) ) , x ) =

Original: H ( M ( D ) , x ) =
Revised: H ( M ( D ) , x ) =

Original: A , ∀ k < K where K is the certiﬁed number of adversarial users: 
Revised: A , ∀ k < K where K is the certiﬁed number of adversarial users: [Equation]

Original: [Equation]
Revised: [Equation]

Original: The proofs of Theorems 1 and 2 are omitted to Appendix A.4.
Revised: The proofs of Theorems 1 and 2 are omitted to Appendix A.4.

Original: Theorems 1 and 2 reﬂect a tradeoff between privacy and certiﬁed prediction: (i) in Theorem 1, if (cid:15) is large such that the RHS of Eq (1) > 1 , the robustness condition cannot be met since the expected conﬁdence F A ( M ( D ) , x ) ∈ [0 , 1] .
Revised: Theorems 1 and 2 reﬂect a tradeoff between privacy and certiﬁed prediction: (i) in Theorem 1, if (cid:15) is large such that the RHS of Eq (1) > 1 , the robustness condition cannot be met since the expected conﬁdence F A ( M ( D ) , x ) ∈ [0 , 1] .

Original: However, to achieve small (cid:15) , i.e., strong privacy, large noise is required during training, which would hurt model utility and thus result in small conﬁdence margin between the top two classes ( e.g. , F A ( M ( D ) , x ) and F B ( M ( D ) , x ) ), making it hard to meet the robustness condition.
Revised: However, to achieve small (cid:15) , i.e., strong privacy, large noise is required during training, which would hurt model utility and thus result in small conﬁdence margin between the top two classes ( e.g. , F A ( M ( D ) , x ) and F B ( M ( D ) , x ) ), making it hard to meet the robustness condition.

Original: (ii)
Revised: (ii)

Original: In
Revised: In

Original: Theorem 2 if we ﬁx F A ( M ( D ) , x ) and F B ( M ( D ) , x ) , smaller (cid:15) of FL can certify larger K .
Revised: Theorem 2 if we ﬁx F A ( M ( D ) , x ) and F B ( M ( D ) , x ) , smaller (cid:15) of FL can certify larger K .

Original: However, smaller (cid:15) also induces smaller conﬁdence margin, thus reducing K instead.
Revised: However, smaller (cid:15) also induces smaller conﬁdence margin, thus reducing K instead.

Original: As a result, properly choosing (cid:15) would help to certify a large K .
Revised: As a result, properly choosing (cid:15) would help to certify a large K .

Original: Certiﬁed Attack Cost.
Revised: Certiﬁed Attack Cost.

Original: In addition to the certiﬁed prediction, we deﬁne the attack cost for attacker
Revised: In addition to the certiﬁed prediction, we deﬁne the attack cost for attacker

Original: C : Θ → R which quantiﬁes the difference between the poisoned model and the attack goal .
Revised: C : Θ → R which quantiﬁes the difference between the poisoned model and the attack goal .

Original: In general, attacker aims to minimize the expected attack cost J ( D ) :
Revised: In general, attacker aims to minimize the expected attack cost J ( D ) :

Original: = E [ C ( M ( D ))] , where the expectation is taken over the randomness of DP training.
Revised: = E [ C ( M ( D ))] , where the expectation is taken over the randomness of DP training.

Original: The cost function can be instantiated according to the concrete attack goal in different types of poisoning attacks, and we provide some examples below.
Revised: The cost function can be instantiated according to the concrete attack goal in different types of poisoning attacks, and we provide some examples below.

Original: Given a global FL model satisfying user-level ( (cid:15), δ ) -DP, we will prove the lower bound of the attack cost J ( D (cid:48) ) when manipulating the data of at most k users.
Revised: Given a global FL model satisfying user-level ( (cid:15), δ ) -DP, we will prove the lower bound of the attack cost J ( D (cid:48) ) when manipulating the data of at most k users.

Original: Higher lower bound of the attack cost indicates more certiﬁably robust global model.
Revised: Higher lower bound of the attack cost indicates more certiﬁably robust global model.

Original: Example 1.
Revised: Example 1.

Original: (Backdoor attack (Gu et al., 2019))
Revised: (Backdoor attack (Gu et al., 2019))

Original: C ( θ ) = 1 n (cid:80) ni
Revised: C ( θ ) = 1 n (cid:80) ni

Original: =1 l ( θ, z ∗ i ) , where z ∗ i =
Revised: =1 l ( θ, z ∗ i ) , where z ∗ i =

Original: ( x i + δ x , y ∗ ) , δ x is the backdoor pattern, y ∗ is the target adversarial label.
Revised: ( x i + δ x , y ∗ ) , δ x is the backdoor pattern, y ∗ is the target adversarial label.

Original: Minimizing J ( D (cid:48) ) drives the prediction on any test data with the backdoor pattern δ x to the target label y ∗ .
Revised: Minimizing J ( D (cid:48) ) drives the prediction on any test data with the backdoor pattern δ x to the target label y ∗ .

Original: Example 2.
Revised: Example 2.

Original: (Label Flipping attack (Biggio et al., 2012))
Revised: (Label Flipping attack (Biggio et al., 2012))

Original: C ( θ ) = 1 n (cid:80) ni
Revised: C ( θ ) = 1 n (cid:80) ni

Original: =1 l ( θ, z ∗ i ) , where z ∗ i =
Revised: =1 l ( θ, z ∗ i ) , where z ∗ i =

Original: ( x i , y ∗ ) and y ∗ is the target adversarial label.
Revised: ( x i , y ∗ ) and y ∗ is the target adversarial label.

Original: Minimizing J ( D (cid:48) ) drives the prediction on test data x i to the target label y ∗ .
Revised: Minimizing J ( D (cid:48) ) drives the prediction on test data x i to the target label y ∗ .

Original: Example 3.
Revised: Example 3.

Original: (Parameter-Targeting attack (Ma et al., 2019))
Revised: (Parameter-Targeting attack (Ma et al., 2019))

Original: C ( θ ) =
Revised: C ( θ ) =

Original: 12 (cid:107) θ − θ (cid:63) (cid:107) 2 , where θ (cid:63) is the target model.
Revised: 12 (cid:107) θ − θ (cid:63) (cid:107) 2 , where θ (cid:63) is the target model.

Original: Minimizing J ( D (cid:48) ) drives the poisoned model to be close to the target model.
Revised: Minimizing J ( D (cid:48) ) drives the poisoned model to be close to the target model.

Original: Theorem 3 (Attack Cost with k Attackers) .
Revised: Theorem 3 (Attack Cost with k Attackers) .

Original: Suppose a randomized mechanism M satisﬁes user-level ( (cid:15), δ ) -DP.
Revised: Suppose a randomized mechanism M satisﬁes user-level ( (cid:15), δ ) -DP.

Original: For two user sets B and B (cid:48) that differ k users, D and D (cid:48) are the corresponding training datasets.
Revised: For two user sets B and B (cid:48) that differ k users, D and D (cid:48) are the corresponding training datasets.

Original: Let J ( D ) be the expected attack cost where | C ( · ) | ≤ ¯ C .
Revised: Let J ( D ) be the expected attack cost where | C ( · ) | ≤ ¯ C .

Original: Then, 
Revised: Then, min { e k(cid:15) J ( D ) + e

Original: [Equation]
Revised: [Equation]

Original: The proof is omitted to Appendix A.4.
Revised: The proof is omitted to Appendix A.4.

Original: Theorem 3 provides the upper bounds and lower bounds for attack cost J ( D (cid:48) ) .
Revised: Theorem 3 provides the upper bounds and lower bounds for attack cost J ( D (cid:48) ) .

Original: The lower bounds show that to what extent the attack can reduce J ( D (cid:48) ) by manipulating up to k users, i.e., how successful the attack can be.
Revised: The lower bounds show that to what extent the attack can reduce J ( D (cid:48) ) by manipulating up to k users, i.e., how successful the attack can be.

Original: The lower bounds depend on the attack cost on clean model J ( D ) , k and (cid:15) .
Revised: The lower bounds depend on the attack cost on clean model J ( D ) , k and (cid:15) .

Original: When J ( D ) is higher, the DPFL model under poisoning attacks is more robust because the lower bounds are accordingly higher; a tighter privacy guarantee, i.e., smaller (cid:15) , can also lead to higher robustness certiﬁcation as it increases the lower bounds; with larger k , the attacker ability grows and thus lead to lower possible J
Revised: When J ( D ) is higher, the DPFL model under poisoning attacks is more robust because the lower bounds are accordingly higher; a tighter privacy guarantee, i.e., smaller (cid:15) , can also lead to higher robustness certiﬁcation as it increases the lower bounds; with larger k , the attacker ability grows and thus lead to lower possible J

Original: ( D (cid:48) ) .
Revised: ( D (cid:48) ) .

Original: The upper bounds show the least adversarial effect brought by k attackers, i.e., how vulnerable the DPFL model is under the optimistic case ( e.g. , the backdoor pattern is less distinguishable).
Revised: The upper bounds show the least adversarial effect brought by k attackers, i.e., how vulnerable the DPFL model is under the optimistic case ( e.g. , the backdoor pattern is less distinguishable).

Original: Leveraging the lower bounds in Theorem 3, we can lower-bound the minimum number of attackers required to reduce the attack cost to certain level associated with hyperparameter τ in Corollary 1.
Revised: Leveraging the lower bounds in Theorem 3, we can lower-bound the minimum number of attackers required to reduce the attack cost to certain level associated with hyperparameter τ in Corollary 1.

Original: Corollary 1 (Lower Bound of k Given τ ) .
Revised: Corollary 1 (Lower Bound of k Given τ ) .

Original: Suppose a randomized mechanism M satisﬁes user- level ( (cid:15), δ ) -DP.
Revised: Suppose a randomized mechanism M satisﬁes user- level ( (cid:15), δ ) -DP.

Original: Let attack cost function be C , the expected attack cost be J ( · ) .
Revised: Let attack cost function be C , the expected attack cost be J ( · ) .

Original: In order to achieve
Revised: In order to achieve

Original: J ( D (cid:48) ) ≤ 1 τ J ( D ) for τ ≥ 

Revised: J ( D (cid:48) ) ≤ 1 τ J ( D ) for τ ≥

Original: when 0 ≤ C ( · ) ≤ ¯ C , or achieve J ( D (cid:48) ) ≤ τJ ( D ) for 1 ≤ τ ≤ − ¯ C J
Revised: when 0 ≤ C ( · ) ≤ ¯ C , or achieve J ( D (cid:48) ) ≤ τJ ( D ) for 1 ≤ τ ≤ − ¯ C J

Original: ( D ) when − ¯ C ≤ C ( · ) ≤ 0 , the number of adversarial users should satisfy:
Revised: ( D ) when − ¯ C ≤ C ( · ) ≤ 0 , the number of adversarial users should satisfy:

Original: [Equation]
Revised: [Equation]

Original: The proof is omitted to Appendix A.4.
Revised: The proof is omitted to Appendix A.4.

Original: Corollary 1 shows that stronger privacy guarantee ( i.e. , smaller (cid:15) ) requires more attackers to achieve the same effectiveness of attack, indicating higher robustness.
Revised: Corollary 1 shows that stronger privacy guarantee ( i.e. , smaller (cid:15) ) requires more attackers to achieve the same effectiveness of attack, indicating higher robustness.

Original:  I NSTANCE - LEVEL P RIVACY AND C ERTIFIED R OBUSTNESS FOR FL 5. I NSTANCE - LEVEL P RIVACY
Revised:  I NSTANCE - LEVEL P RIVACY AND C ERTIFIED R OBUSTNESS FOR FL 5. I NSTANCE - LEVEL P RIVACY

Original: In this section, we introduce the instance-level DP deﬁnition, the corresponding algorithm, and the privacy analysis for FL.
Revised: In this section, we introduce the instance-level DP deﬁnition, the corresponding algorithm, and the privacy analysis for FL.

Original: When DP is used to protect the privacy of individual instance, the trained stochastic FL model should not differ much if one instance is modiﬁed.
Revised: When DP is used to protect the privacy of individual instance, the trained stochastic FL model should not differ much if one instance is modiﬁed.

Original: Hence, the adjacent datasets in instance-level DP are deﬁned as those differing by one instance.
Revised: Hence, the adjacent datasets in instance-level DP are deﬁned as those differing by one instance.

Original: Deﬁnition 3 (Instance-level ( (cid:15), δ ) -DP) .
Revised: The formal deﬁnition of Instance-level ( (cid:15), δ ) -DP

Original: Let D be the dataset that is the union of local training examples from all users. Then, D and D (cid:48) are adjacent if they differ by one instance. The mechanism M is instance-level ( (cid:15), δ ) -DP if it meets Deﬁnition 1 with D and D (cid:48) as adjacent datasets.
Revised: (Deﬁnition 3) is omitted to Appendix A.1.

Original: Dopamine (Malekzadeh et al., 2021) provides the ﬁrst instance-level privacy guarantee under
Revised: Dopamine (Malekzadeh et al., 2021) provides the ﬁrst instance-level privacy guarantee under

Original: FedSGD (McMahan et al., 2017).
Revised: FedSGD (McMahan et al., 2017).

Original: However, it has two limitations.
Revised: However, it has two limitations.

Original: First, its privacy bound is loose.
Revised: First, its privacy bound is loose.

Original: Although FedSGD performs both user and batch sampling during training, Dopamine ignores the privacy gain provided by random user sampling.
Revised: Although FedSGD performs both user and batch sampling during training, Dopamine ignores the privacy gain provided by random user sampling.

Original: In this section, we improve the privacy guarantee under FedSGD with privacy ampliﬁcation via user sampling (Bassily et al., 2014; Abadi et al., 2016).
Revised: In this section, we improve the privacy guarantee under FedSGD with privacy ampliﬁcation via user sampling (Bassily et al., 2014; Abadi et al., 2016).

Original: This improvement leads to algorithm InsDP-FedSGD , to achieve tighter privacy analysis.
Revised: This improvement leads to algorithm InsDP-FedSGD , to achieve tighter privacy analysis.

Original: We defer the algorithm (Algorithm 2) as well as its privacy guarantee to Appendix A.1.
Revised: We defer the algorithm (Algorithm 2) as well as its privacy guarantee to Appendix A.1.

Original: Besides the loose privacy bound, Dopamine (Malekzadeh et al., 2021) only allows users to perform one step of DP-SGD (Abadi et al., 2016) during each FL round.
Revised: Besides the loose privacy bound, Dopamine (Malekzadeh et al., 2021) only allows users to perform one step of DP-SGD (Abadi et al., 2016) during each FL round.

Original: This restriction limits the efﬁciency of the algorithm and increases the communication overhead.
Revised: This restriction limits the efﬁciency of the algorithm and increases the communication overhead.

Original: In practice, users in FL are typically allowed to update their local models for many steps before submitting updates to reduce the communication cost.
Revised: In practice, users in FL are typically allowed to update their local models for many steps before submitting updates to reduce the communication cost.

Original: To solve this problem, we further improve InsDP-FedSGD to support multiple local steps during each round.
Revised: To solve this problem, we further improve InsDP-FedSGD to support multiple local steps during each round.

Original: Speciﬁcally, we propose a novel instance-level DPFL algorithm InsDP-FedAvg (Algorithm 3 in Appendix A.1) allowing users to train multiple local SGD steps before submitting the updates.
Revised: Speciﬁcally, we propose a novel instance-level DPFL algorithm InsDP-FedAvg (Algorithm 3 in Appendix A.1) allowing users to train multiple local SGD steps before submitting the updates.

Original: In InsDP-FedAvg , each user i performs local DP-SGD so that the local training mechanism M i satisﬁes instance-level DP.
Revised: In InsDP-FedAvg , each user i performs local DP-SGD so that the local training mechanism M i satisﬁes instance-level DP.

Original: Then, the server aggregates the updates.
Revised: Then, the server aggregates the updates.

Original: We prove that the global mechanism M preserves instance-level DP using DP parallel composition theorem (Dwork &
Revised: We prove that the global mechanism M preserves instance-level DP using DP parallel composition theorem (Dwork &

Original: Lei, 2009) and moment accountant (Abadi et al., 2016).
Revised: Lei, 2009) and moment accountant (Abadi et al., 2016).

Original: Algorithm 3 formally presents the InsDP-FedAvg algorithm and the calculation of its privacy budget (cid:15) .
Revised: Algorithm 3 formally presents the InsDP-FedAvg algorithm and the calculation of its privacy budget (cid:15) .

Original: Speciﬁcally, at ﬁrst, local privacy cost (cid:15) i 0 is initialized as 0 before FL training.
Revised: Speciﬁcally, at ﬁrst, local privacy cost (cid:15) i 0 is initialized as 0 before FL training.

Original: At round t , if user i is not selected, its local privacy cost is kept unchanged (cid:15) it ← (cid:15) it − 1 .
Revised: At round t , if user i is not selected, its local privacy cost is kept unchanged (cid:15) it ← (cid:15) it − 1 .

Original: Otherwise user i updates local model by running DP-SGD for V local steps with batch sampling probability p , noise level σ and clipping threshold S , and (cid:15) it is accumulated upon (cid:15) it − 1 via its local moment accountant.
Revised: Otherwise user i updates local model by running DP-SGD for V local steps with batch sampling probability p , noise level σ and clipping threshold S , and (cid:15) it is accumulated upon (cid:15) it − 1 via its local moment accountant.

Original: Next, the server aggregates the updates from selected users, and leverages { (cid:15) it } i ∈
Revised: Next, the server aggregates the updates from selected users, and leverages { (cid:15) it } i ∈

Original: [ N ] and the parallel composition in Theorem 4 to calculate the global privacy cost (cid:15) t .
Revised: [ N ] and the parallel composition in Theorem 4 to calculate the global privacy cost (cid:15) t .

Original: After T rounds, the mechanism M that outputs the FL global model in Algorithm 3 is instance-level ( (cid:15) T , δ ) -DP.
Revised: After T rounds, the mechanism M that outputs the FL global model in Algorithm 3 is instance-level ( (cid:15) T , δ ) -DP.

Original: Theorem 4 ( InsDP-FedAvg Privacy Guarantee) .
Revised: Theorem 4 ( InsDP-FedAvg Privacy Guarantee) .

Original: In Algorithm 3, during round t , if the local mechanism M i satisﬁes ( (cid:15) it , δ ) -DP, then the global mechanism M satisﬁes (cid:0) max i ∈ [ N ] (cid:15) it , δ (cid:1) -DP.
Revised: In Algorithm 3, during round t , if the local mechanism M i satisﬁes ( (cid:15) it , δ ) -DP, then the global mechanism M satisﬁes (cid:0) max i ∈ [ N ] (cid:15) it , δ (cid:1) -DP.

Original: The idea behind Theorem 4 is that when D (cid:48) and D differ in one instance, the modiﬁed instance only falls into one local dataset, and thus parallel composition theorem (Dwork & Lei, 2009) can be applied.
Revised: The idea behind Theorem 4 is that when D (cid:48) and D differ in one instance, the modiﬁed instance only falls into one local dataset, and thus parallel composition theorem (Dwork & Lei, 2009) can be applied.

Original: Then the privacy guarantee corresponds to the worst-case, and is obtained by taking the maximum local privacy cost across all the users.
Revised: Then the privacy guarantee corresponds to the worst-case, and is obtained by taking the maximum local privacy cost across all the users.

Original: The detailed proof is given in Appendix A.1.
Revised: The detailed proof is given in Appendix A.1.

Original: 5. C ERTIFIED R OBUSTNESS OF I NSTANCE - LEVEL DPFL AGAINST P OISONING A TTACKS
Revised: 5. C ERTIFIED R OBUSTNESS OF I NSTANCE - LEVEL DPFL AGAINST P OISONING A TTACKS

Original: Threat Model.
Revised: Threat Model.

Original: We consider poisoning attacks under the presence of k poisoned instances.
Revised: We consider poisoning attacks under the presence of k poisoned instances.

Original: These instances could be controlled by the same or multiple adversarial users.
Revised: These instances could be controlled by the same or multiple adversarial users.

Original: Our robustness certiﬁcation is agnostic to the attack methods as long as the number of poisoned instances is constrained.
Revised: Our robustness certiﬁcation is agnostic to the attack methods as long as the number of poisoned instances is constrained.

Original: According to the group DP property (Lemma 1) and the post-processing property for FL model with instance-level ( (cid:15), δ ) -DP, we prove that our robust certiﬁcation results proposed for user-level DP are also applicable to instance-level DP.
Revised: According to the group DP property (Lemma 1) and the post-processing property for FL model with instance-level ( (cid:15), δ ) -DP, we prove that our robust certiﬁcation results proposed for user-level DP are also applicable to instance-level DP.

Original: Below is the formal theorem (proof is given in Appendix A.4).
Revised: Below is the formal theorem (proof is given in Appendix A.4).

Original: Theorem 5.
Revised: Theorem 5.

Original: Suppose D and D (cid:48) differ by k instances, and the randomized mechanism M satisﬁes instance-level ( (cid:15), δ ) -DP.
Revised: Suppose D and D (cid:48) differ by k instances, and the randomized mechanism M satisﬁes instance-level ( (cid:15), δ ) -DP.

Original: The results in Theorems 1, 2,and 3, and Corollary 1 hold for M , D , and D (cid:48) .
Revised: The results in Theorems 1, 2,and 3, and Corollary 1 hold for M , D , and D (cid:48) .

Original: Comparison with existing certiﬁed prediction methods in centralized setting.
Revised: Comparison with existing certiﬁed prediction methods in centralized setting.

Original: The form of Theorem 1 is similar with the robustness condition against test-time attack in Proposition 1 of (Lecuyer et al., 2019a).
Revised: The form of Theorem 1 is similar with the robustness condition against test-time attack in Proposition 1 of (Lecuyer et al., 2019a).

Original: This is because the derived robustness conditions are both rooted in the DP properties, but ours focus on the robustness against training-time attacks in FL, which is more challenging considering the distributed nature and the model training dynamics, i.e., the analysis of the privacy budget over training rounds.
Revised: This is because the derived robustness conditions are both rooted in the DP properties, but ours focus on the robustness against training-time attacks in FL, which is more challenging considering the distributed nature and the model training dynamics, i.e., the analysis of the privacy budget over training rounds.

Original: Our Theorem 1 is also different from previous randomized smoothingbased certiﬁably robust centralized learning against backdoor (Weber et al., 2020) and label ﬂipping (Rosenfeld et al., 2020).
Revised: Our Theorem 1 is also different from previous randomized smoothingbased certiﬁably robust centralized learning against backdoor (Weber et al., 2020) and label ﬂipping (Rosenfeld et al., 2020).

Original: First, our randomness comes from the inherent training randomness of user/instance-level ( (cid:15), δ ) -DP, e.g., user subsampling and Gaussian noise.
Revised: First, our randomness comes from the inherent training randomness of user/instance-level ( (cid:15), δ ) -DP, e.g., user subsampling and Gaussian noise.

Original: Thus, the certiﬁed robustness for free in DPFL means that the DPFL learning algorithm M itself is randomized , and such randomness can lead to the robustness certiﬁcation with non-trivial quantitative measurement of the randomness.
Revised: Thus, the certiﬁed robustness for free in DPFL means that the DPFL learning algorithm M itself is randomized , and such randomness can lead to the robustness certiﬁcation with non-trivial quantitative measurement of the randomness.

Original: On the contrary, robustness in randomized smoothing-based methods comes from explicitly making the classiﬁcation process randomized via adding noise in training datasets (Weber et al., 2020; Rosenfeld et al., 2020), or test samples (Lecuyer et al., 2019a; Cohen et al., 2019) which is easier to measure.
Revised: On the contrary, robustness in randomized smoothing-based methods comes from explicitly making the classiﬁcation process randomized via adding noise in training datasets (Weber et al., 2020; Rosenfeld et al., 2020), or test samples (Lecuyer et al., 2019a; Cohen et al., 2019) which is easier to measure.

Original: Second, our Theorem 1, 2 hold no matter how (cid:15) is achieved, which means that we can add different types of noise, leverage different subsampling strategies or even different FL training protocols to achieve user/instance-level (cid:15) .
Revised: Second, our Theorem 1, 2 hold no matter how (cid:15) is achieved, which means that we can add different types of noise, leverage different subsampling strategies or even different FL training protocols to achieve user/instance-level (cid:15) .

Original: However, in (Weber et al., 2020; Rosenfeld et al., 2020) different certiﬁcations require different types of noise (Laplacian, Gaussian, etc.).
Revised: However, in (Weber et al., 2020; Rosenfeld et al., 2020) different certiﬁcations require different types of noise (Laplacian, Gaussian, etc.).

Original: Additionally,
Revised: Additionally,

Original: DP is suitable to characterize the robustness against poisoning since DP composition theorems can be leveraged to track privacy cost (cid:15) , which captures the training dynamics of ML model parameters without additional assumptions.
Revised: DP is suitable to characterize the robustness against poisoning since DP composition theorems can be leveraged to track privacy cost (cid:15) , which captures the training dynamics of ML model parameters without additional assumptions.

Original: Otherwise one may need to track the deviations of model parameters by analyzing SGD over training, which is theoretically knotty and often requires strong assumptions on Lipschitz continuity, smoothness or convexity for the trained models.
Revised: Otherwise one may need to track the deviations of model parameters by analyzing SGD over training, which is theoretically knotty and often requires strong assumptions on Lipschitz continuity, smoothness or convexity for the trained models.

Original:  E XPERIMENTS
Revised:  E XPERIMENTS

Original: We present evaluations for robustness certiﬁcations, expecially Thm. 2, 3 and Cor.
Revised: We present evaluations for robustness certiﬁcations, expecially Thm. 2, 3 and Cor.

Original: We ﬁnd that 1) there is a tradeoff between certiﬁed prediction and privacy on certain datasets; 2) a tighter privacy guarantee always provides stronger certiﬁed robustness in terms of the certiﬁed attack cost ; 3) our lower bounds of certiﬁed attack cost are generally tight when k is small.
Revised: We ﬁnd that 1) there is a tradeoff between certiﬁed prediction and privacy on certain datasets; 2) a tighter privacy guarantee always provides stronger certiﬁed robustness in terms of the certiﬁed attack cost ; 3) our lower bounds of certiﬁed attack cost are generally tight when k is small.

Original: When k is large, they are tight under strong attacks ( e.g. , large local poisoning ratio α ).
Revised: When k is large, they are tight under strong attacks ( e.g. , large local poisoning ratio α ).

Original: Stronger attacks or tighter certiﬁcation are requried to further tighten the gap between the emprical robustness and theoretical bounds.
Revised: Stronger attacks or tighter certiﬁcation are requried to further tighten the gap between the emprical robustness and theoretical bounds.

Original: Data and Model.
Revised: Data and Model.

Original: We evaluate our robustness certiﬁcation results with two datasets: MNIST and CIFAR-10.
Revised: We evaluate our robustness certiﬁcation results with three datasets: image classﬁcation on MNIST, CIFAR-10 and text sentiment analysis task on tweets from Sentiment140 (Go et al.) (Sent140), which involves classifying Twitter posts as positive or negative.

Original: For each dataset, we use corresponding standard CNN architectures in the differential privacy library (opa, 2021) of PyTorch.
Revised: For image datasets, we use corresponding standard CNN architectures in the differential privacy library (opa, 2021) of PyTorch; for Sent140, we use a LSTM classiﬁer.

Original: Following previous work on DP ML (Jagielski et al., 2020; Ma et al., 2019) and backdoor attacks (Tran et al., 2018; Weber et al., 2020) which evaluate with two classes, we focus on binary classiﬁcation for MNIST (digit 0 and 1) and CIFAR-10 (airplane and bird), and defer the 10-class results to Appendix A.3.
Revised: Following previous work on DP ML (Jagielski et al., 2020; Ma et al., 2019) and backdoor attacks (Tran et al., 2018; Weber et al., 2020) which evaluate with two classes, we focus on binary classiﬁcation for MNIST (digit 0 and 1) and CIFAR- (airplane and bird), and defer the 10-class results to Appendix A.3.

Original: We train FL model following Algorithm 1 for user-level privacy and Algorithm 3 for instance-level privacy.
Revised: We train FL model following Algorithm 1 for user-level privacy and Algorithm 3 for instance-level privacy.

Original: We refer the readers to Appendix A. for details about the datasets, networks, parameter setups.
Revised: We refer the readers to Appendix A.3 for details about the datasets, networks, parameter setups.

Original: Poisoning Attacks.
Revised: Poisoning Attacks.

Original: We evaluate several state-of-the-art poisoning attacks against
Revised: We evaluate several state-of-the-art poisoning attacks against

Original: the proposed
Revised: the proposed

Original: UserDP-FedAvg and InsDP-FedAvg .
Revised: UserDP-FedAvg and InsDP-FedAvg .

Original: We ﬁrst consider backdoor attacks ( BKD ) (Bagdasaryan et al., 2020) and label ﬂipping attacks ( LF )
Revised: We ﬁrst consider backdoor attacks ( BKD ) (Bagdasaryan et al., 2020) and label ﬂipping attacks ( LF )

Original: (Fung et al., 2020).
Revised: (Fung et al., 2020).

Original: For InsDP-FedAvg , we consider the worst case where k backdoored or lable-ﬂipped instances are fallen into the dataset of one user.
Revised: For InsDP-FedAvg , we consider the worst case where k backdoored or lable-ﬂipped instances are fallen into the dataset of one user.

Original: For
Revised: For

Original: UserDP-FedAvg , we additionally evaluate distributed backdoor attack ( DBA )
Revised: UserDP-FedAvg , we additionally evaluate distributed backdoor attack ( DBA )

Original: (Xie et al., 2019), which is claimed to be a more stealthy backdoor attack against FL.
Revised: (Xie et al., 2019), which is claimed to be a more stealthy backdoor attack against FL.

Original: Moreover, we consider BKD,
Revised: Moreover, we consider BKD,

Original: LF and DBA via model replacement approach (Bagdasaryan et al., 2020) where k attackers train the local models using local datasets with α fraction of poisoned instances, and scale the malicious updates with hyperparameter γ , i.e., ∆ w it ← γ ∆ w it , before sending them to the sever.
Revised: LF and DBA via model replacement approach (Bagdasaryan et al., 2020) where k attackers train the local models using local datasets with α fraction of poisoned instances, and scale the malicious updates with hyperparameter γ , i.e., ∆ w it ← γ ∆ w it , before sending them to the sever.

Original: This way, the malicious updates would have a stronger impact on the FL model.
Revised: This way, the malicious updates would have a stronger impact on the FL model.

Original: Note that even when attackers perform scaling, after server clipping, the sensitivity of updates is still upper-bounded by the clipping threshold S .
Revised: Note that even when attackers perform scaling, after server clipping, the sensitivity of updates is still upper-bounded by the clipping threshold S .

Original: So the privacy guarantee in Proposition 1 still holds under poisoning attacks via model replacement.
Revised: So the privacy guarantee in Proposition 1 still holds under poisoning attacks via model replacement.

Original: Detailed attack setups are presented in Appendix A.3.
Revised: Detailed attack setups are presented in Appendix A.3.

Original: Evaluation Metrics and Setup.
Revised: Evaluation Metrics and Setup.

Original: We consider two evaluation metrics based on our robustness certiﬁcation criteria.
Revised: We consider two evaluation metrics based on our robustness certiﬁcation criteria.

Original: The ﬁrst metric is certiﬁed accuracy , which is the fraction of the test set for which the poisoned FL model makes correct and consistent predictions compared with the clean
Revised: The ﬁrst metric is certiﬁed accuracy , which is the fraction of the test set for which the poisoned FL model makes correct and consistent predictions compared with the clean

Original: FL model.
Revised: FL model.

Original: Given a test set of size n , for i -th test sample, the ground truth label is y i , the output prediction is c i , and the certiﬁed number of adversarial users/instances is K i .
Revised: Given a test set of size n , for i -th test sample, the ground truth label is y i , the output prediction is c i , and the certiﬁed number of adversarial users/instances is K i .

Original: We calculate the certiﬁed accuracy at k as 1 n (cid:80) ni
Revised: We calculate the certiﬁed accuracy at k as 1 n (cid:80) ni

Original: =1 1 { c i =
Revised: =1 1 { c i =

Original: y i
Revised: y i

Original: and
Revised: and

Original: K
Revised: K

Original: i ≥ k } .
Revised: i ≥ k } .

Original: The second metric is the lower bound of attack cost in Theorem 3: J ( D (cid:48) )
Revised: The second metric is the lower bound of attack cost in Theorem 3: J ( D (cid:48) )

Original: =
Revised: =

Original: max { e − k(cid:15)
Revised: max { e − k(cid:15)

Original: J
Revised: J

Original: ( B ) − 1 − e − [Table]
Revised: ( B ) − 1 − e − k(cid:15)

Original: e (cid:15) − 1 δ ¯ C, 0 } .
Revised: e (cid:15) − 1 δ ¯ C, 0 } .

Original: We evaluate the tightness of
Revised: We evaluate the tightness of

Original: J ( D (cid:48) ) by comparing it with empirical attack cost J ( D (cid:48) ) .
Revised: J ( D (cid:48) ) by comparing it with empirical attack cost J ( D (cid:48) ) .

Original: To quantify the robustness, we evaluate the expected class conﬁdence F c ( M ( D ) , x ) for class c via Monte-Carlo sampling.
Revised: To quantify the robustness, we evaluate the expected class conﬁdence F c ( M ( D ) , x ) for class c via Monte-Carlo sampling.

Original: We run the private
Revised: We run the private

Original: FL algorithms for M = 1000 times, with class conﬁdence f sc = f c ( M ( D ) , x ) for each time.
Revised: FL algorithms for M = 1000 times, with class conﬁdence f sc = f c ( M ( D ) , x ) for each time.

Original: We compute its expectation to estimate F c ( M ( D ) , x ) ≈ 1 M (cid:80) Ms =1 f sc and use it to evaluate Theorem 2.
Revised: We compute its expectation to estimate F c ( M ( D ) , x ) ≈ 1 M (cid:80) Ms =1 f sc and use it to evaluate Theorem 2.

Original: In addition, we use Hoeffding’s inequality (Hoeffding, 1994) to calibrates the empirical estimation with conﬁdence level parameter ψ , and results are deferred to Appendix A.3.
Revised: In addition, we use Hoeffding’s inequality (Hoeffding, 1994) to calibrates the empirical estimation with conﬁdence level parameter ψ , and results are deferred to Appendix A.3.

Original: In terms of the attack cost, we use Example 1, 2 as the deﬁnitions of cost function C for backdoor attacks and label ﬂipping attacks respectively.
Revised: In terms of the attack cost, we use Example 1, 2 as the deﬁnitions of cost function C for backdoor attacks and label ﬂipping attacks respectively.

Original: We follow similar protocol to estimate J ( D (cid:48) ) for Theorem 3 and Corollary 1.
Revised: We follow similar protocol to estimate J ( D (cid:48) ) for Theorem 3 and Corollary 1.

Original: 6. R OBUSTNESS E VALUATION OF U SER - LEVEL DPFL
Revised: 6. R OBUSTNESS E VALUATION OF U SER - LEVEL DPFL

Original: Certiﬁed Prediction.
Revised: Certiﬁed Prediction.

Original: Figure 1(a)(b)
Revised: Figure 1(a)(b)

Original: present the user-level certiﬁed accuracy under different (cid:15) by training DPFL models with different noise scale σ .
Revised: present the user-level certiﬁed accuracy under different (cid:15) by training DPFL models with different noise scale σ .

Revised: The results on Sent140 dataset is presented in Figure 13 of Appendix.

Revised: A.3.8.

Original: We observe that the largest k can be certiﬁed when (cid:15) is around 0 .
Revised: We observe that the largest k can be certiﬁed when (cid:15) is around 0 .

Original: 6298 in MNIST and 0 .
Revised: 6298 in MNIST, 0 .

Original: 1451 in CIFAR-10,  which veriﬁes the tradeoff between (cid:15) and certiﬁed accuracy as we discussed in Section 4.2.
Revised: 1451 in CIFAR-10, and 0 . 2247 in Sent140 which veriﬁes the tradeoff between (cid:15) and certiﬁed accuracy as we discussed in Section 4.2.

Original: Advanced DP protocols that requires less noise while achieving similar level of privacy are favored to improve the privacy, utility, and certiﬁed accuracy simultaneously.
Revised: Advanced DP protocols that requires less noise while achieving similar level of privacy are favored to improve the privacy, utility, and certiﬁed accuracy simultaneously.

Original: Certiﬁed Attack Cost.
Revised: Furthermore, we compare the certiﬁed accuracy of four different user-level DPFL methods (McMahan et al., 2018; Geyer et al., 2017) given the same privacy budget (cid:15) . As shown in Figure 14 and Figure 15 of Appendix. A.3.9, the models trained by different DPFL algorithms satisfying same (cid:15) have different certiﬁed robustness results.

Revised: This is because even under the same (cid:15) , different DPFL algorithms M produce trained models M ( D ) with different model performance, thus leading to different certiﬁed robustness.

Revised: More discussion could be found in Appendix.

Original:  In order to evaluate Theorem 3 and characterize the tightness of our theoretical lower bound J ( D (cid:48) ) , we compare it with the empirical attack cost J ( D (cid:48) ) under different local poison fraction α , attack methods and scale factor γ in Figure 2.
Revised: A.3.9. Certiﬁed Attack Cost. In order to evaluate Theorem 3 and characterize the tightness of our theoretical lower bound J ( D (cid:48) ) , we compare it with the empirical attack cost J ( D (cid:48) ) under different local poison fraction α , attack methods and scale factor γ in Figure 2.

Original: Note that when k = 0 , the model is benign so the empirical cost equals to the certiﬁed one.
Revised: Note that when k = 0 , the model is benign so the empirical cost equals to the certiﬁed one.

Original: We ﬁnd that 1) when k increases, the attack ability grows, and both the empirical attack cost and theoretical lower bound decreases.
Revised: We ﬁnd that 1) when k increases, the attack ability grows, and both the empirical attack cost and theoretical lower bound decreases.

Original: In Figurerow 1, given the same k , higher α , i.e., poisoning more local instances for each attacker, achieves a stronger attack, under which lower empirical J ( D ) can be achieved and is more close to the certiﬁed lower bound.
Revised: In Figurerow 1, given the same k , higher α , i.e., poisoning more local instances for each attacker, achieves a stronger attack, under which lower empirical J ( D ) can be achieved and is more close to the certiﬁed lower bound.

Original: This indicates that the lower bound appears tighter when the poisoning attack is stronger.
Revised: This indicates that the lower bound appears tighter when the poisoning attack is stronger.

Original: In Figure 2 row 2, we ﬁx α = 100% and evaluate UserDP-FedAvg under different γ and attack methods.
Revised: In Figure 2 row 2, we ﬁx α = 100% and evaluate UserDP-FedAvg under different γ and attack methods.

Original: It turns out that DP serves as a strong defense empirically for FL, given that J ( D ) did not vary much under different γ (1 , 50 , 100) and different attack methods (BKD, DBA, LF).
Revised: It turns out that DP serves as a strong defense empirically for FL, given that J ( D ) did not vary much under different γ (1 , 50 , 100) and different attack methods (BKD, DBA, LF).

Original: This is because the clipping operation restricts the magnitude of malicious updates, rendering the model replacement ineffective; the Gaussian noise perturbs the malicious updates and makes the DPFL model stable, and thus the FL model is less likely to memorize the poisoning instances.
Revised: This is because the clipping operation restricts the magnitude of malicious updates, rendering the model replacement ineffective; the Gaussian noise perturbs the malicious updates and makes the DPFL model stable, and thus the FL model is less likely to memorize the poisoning instances.

Original: In both rows, the lower bounds are tight when k is small.
Revised: In both rows, the lower bounds are tight when k is small.

Original: When k is large, there remains a gap between our theoretical lower bounds and empirical attack costs under different attacks, which will inspire more effective poisoning attacks or tighter robustness certiﬁcation.
Revised: When k is large, there remains a gap between our theoretical lower bounds and empirical attack costs under different attacks, which will inspire more effective poisoning attacks or tighter robustness certiﬁcation.

Original: Certiﬁed Attack Cost under Different (cid:15) .
Revised: Certiﬁed Attack Cost under Different (cid:15) .

Original: Here we further explore the impacts of different factors on the certiﬁed attack cost.
Revised: Here we further explore the impacts of different factors on the certiﬁed attack cost.

Original: Figure 3 presents the empirical attack cost and the certiﬁed attack cost lower bound given different (cid:15) on user-level DP.
Revised: Figure 3 presents the empirical attack cost and the certiﬁed attack cost lower bound given different (cid:15) on user-level DP.

Original: It is shown that as the privacy guarantee becomes stronger, i.e. smaller (cid:15) , the model is more robust achieving higher J ( D (cid:48) ) and J ( D (cid:48) ) .
Revised: It is shown that as the privacy guarantee becomes stronger, i.e. smaller (cid:15) , the model is more robust achieving higher J ( D (cid:48) ) and J ( D (cid:48) ) .

Original: In Figure(a)(b), we train user-level ( (cid:15) , δ ) DPFL models, calculate corresponding J ( D ) , and plot the lower bound of k given different attack effectiveness hyperparameter τ according to Corollary 1.
Revised: In Figure(a)(b), we train user-level ( (cid:15) , δ ) DPFL models, calculate corresponding J ( D ) , and plot the lower bound of k given different attack effectiveness hyperparameter τ according to Corollary 1.

Original: It shows that 1) when the required attack effectiveness is higher, i.e., τ is larger, more number of attackers is required.
Revised: It shows that 1) when the required attack effectiveness is higher, i.e., τ is larger, more number of attackers is required.

Original: To achieve the same effectiveness of attack, fewer number of attackers is needed for larger (cid:15) , which means that DPFL model with weaker privacy is more vulnerable to poisoning attacks.
Revised: To achieve the same effectiveness of attack, fewer number of attackers is needed for larger (cid:15) , which means that DPFL model with weaker privacy is more vulnerable to poisoning attacks.

Original: 6. R OBUSTNESS E VALUATION OF I NSTANCE - LEVEL DPFL
Revised: 6. R OBUSTNESS E VALUATION OF I NSTANCE - LEVEL DPFL

Original: Certiﬁed Prediction.
Revised: Certiﬁed Prediction.

Original: Figure 1(c)(d) show the instance-level certiﬁed accuracy under different (cid:15) .
Revised: Figure 1(c)(d) show the instance-level certiﬁed accuracy under different (cid:15) .

Original: The optimal (cid:15) for K is around 0 .
Revised: The optimal (cid:15) for K is around 0 .

Original: 3593 for MNIST and 0 .
Revised: 3593 for MNIST and 0 .

Original: 6546 for CIFAR-10, which is aligned with our observation of the tradeoff between certiﬁed accuracy and privacy on user-level DPFL (Section 6.1).
Revised: 6546 for CIFAR-10, which is aligned with our observation of the tradeoff between certiﬁed accuracy and privacy on user-level DPFL (Section 6.1).

Original: Certiﬁed Attack Cost.
Revised: Certiﬁed Attack Cost.

Original: Figure 4 show the certiﬁed attack cost on CIFAR-10.
Revised: Figure 4 show the certiﬁed attack cost on CIFAR-10.

Original: From Figure 4 (a)(b), poisoning more instances (i.e., larger k ) induces lower theoretical and empirical attack cost.
Revised: From Figure 4 (a)(b), poisoning more instances (i.e., larger k ) induces lower theoretical and empirical attack cost.

Original: From
Revised: From

Original: Figure 4 (c)(d), it is clear that instance-level DPFL with stronger privacy guarantee provides higher attack cost both empirically and theoretically, meaning that it is more robust against poisoning attacks.
Revised: Figure 4 (c)(d), it is clear that instance-level DPFL with stronger privacy guarantee provides higher attack cost both empirically and theoretically, meaning that it is more robust against poisoning attacks.

Original: Results on MNIST are deferred to Appendix A.3.
Revised: Results on MNIST are deferred to Appendix A.3.

Original: Figure 5 (c)(d) show the lower bound of k under different instance-level (cid:15) given different τ .
Revised: Figure 5 (c)(d) show the lower bound of k under different instance-level (cid:15) given different τ .

Original: Fewer poisoned instances are required to reduce the J ( D (cid:48) ) to the similar level for a less private DPFL model, indicating that the model is easier to be attacked.
Revised: Fewer poisoned instances are required to reduce the J ( D (cid:48) ) to the similar level for a less private DPFL model, indicating that the model is easier to be attacked.

Original:  C ONCLUSION
Revised:  C ONCLUSION

Original: In this paper, we present the ﬁrst work on deriving certiﬁed robustness in DPFL for free against poisoning attacks.
Revised: In this paper, we present the ﬁrst work on deriving certiﬁed robustness in DPFL for free against poisoning attacks.

Original: We propose two robustness certiﬁcation criteria, based on which we prove that a
Revised: We propose two robustness certiﬁcation criteria, based on which we prove that a

Original: FL model satisfying user-level (instance-level) DP is certiﬁably robust against a bounded number of adversarial users (instances).
Revised: FL model satisfying user-level (instance-level) DP is certiﬁably robust against a bounded number of adversarial users (instances).

Original: Our theoretical analysis characterizes the inherent relation between certiﬁed robustness and differential privacy of FL on both user and instance levels which are empirically veriﬁed with extensive experiments.
Revised: Our theoretical analysis characterizes the inherent relation between certiﬁed robustness and differential privacy of FL on both user and instance levels which are empirically veriﬁed with extensive experiments.

Original: Our results can be used to improve the trustworthiness of
Revised: Our results can be used to improve the trustworthiness of

Original: DPFL
Revised: DPFL

Original: Ethics Statement.
Revised: Ethics Statement.

Original: Our work study the robustness guarantee of differentially private federated learning models from theoretical and empirical perspectives.
Revised: Our work study the robustness guarantee of differentially private federated learning models from theoretical and empirical perspectives.

Original: All the datasets and packages we use are open-sourced.
Revised: All the datasets and packages we use are open-sourced.

Original: We do not have ethical concerns in our paper.
Revised: We do not have ethical concerns in our paper.

Original: Reproducibility Statement.
Revised: Reproducibility Statement.

Original: Our source code is available as the supplemental material for reproducibility purpose.
Revised: Our source code is available as the supplemental material for reproducibility purpose.

Original: Our experiments can be reproduced following our detailed training and evaluation setups in Appendix A.3.
Revised: Our experiments can be reproduced following our detailed training and evaluation setups in Appendix A.3.

Original: The complete proofs of privacy analysis and certiﬁed robustness analysis can be found in the Appendix A.1 and Appendix A.4, respectively
Revised: The complete proofs of privacy analysis and certiﬁed robustness analysis can be found in the Appendix A.1 and Appendix A.4, respectively

Original: A A PPENDIX
Revised: A A PPENDIX

Original: The Appendix is organized as follows:
Revised: The Appendix is organized as follows:

Original: • Appendix A.1 provides the  DPFL algorithms on both user and instance levels, and the proofs for corresponding privacy guarantees.
Revised: • Appendix A.1 provides the DP deﬁnitions and the DPFL algorithms on both user and instance levels, and the proofs for corresponding privacy guarantees.

Original: • Appendix A.2 speciﬁes our threat models.
Revised: • Appendix A.2 speciﬁes our threat models.

Original: • Appendix A.3 provides more details on experimental setups for training and evaluation, the addition experimental results on certiﬁed accuracy with conﬁdence level, robustness evaluation of instancelevel DPFL on MNIST, robustness evaluation on 10-class classiﬁcation,and DP bound comparison between InsDP-FedSGD and Dopamine.
Revised: • Appendix A.3 provides more details on experimental setups for training and evaluation, the addition experimental results on certiﬁed accuracy with conﬁdence level, robustness evaluation of InsDP-FedAvg on MNIST, robustness evaluation on 10-class classiﬁcation, DP bound comparison between InsDP-FedSGD and Dopamine, certiﬁed accuracy of UserDP-FedAvg on Sent140 and certiﬁed accuracy comparison of different user-level DPFL algorithms.

Original: • Appendix A.4 provides the proofs for the certiﬁed robustness related analysis, including Lemma 1, Theorem 1, 2, 3, 5 and Corollary 1.
Revised: • Appendix A.4 provides the proofs for the certiﬁed robustness related analysis, including Lemma 1, Theorem 1, 2, 3, 5 and Corollary 1.

Original: A. D IFFERENTIALLY P RIVATE F EDERATED L EARNING A.1. U SER DP-F ED A VG
Revised: A. D IFFERENTIALLY P RIVATE F EDERATED L EARNING A.1. U SER DP-F ED A VG

Revised: Deﬁnition 2 (User-level ( (cid:15), δ ) -DP) .

Revised: Let B, B (cid:48) be two user sets with size N .

Revised: Let D and D (cid:48) be the datasets that are the union of local training examples from all users in B and B (cid:48) respectively.

Original: In Algorithm 1, M . accum priv spending () and  M .
Revised: Then, D and D (cid:48) are adjacent if B and B (cid:48) differ by one user. The mechanism M satisﬁes user-level ( (cid:15), δ ) -DP if it meets Deﬁnition 1 with D and D (cid:48) as adjacent datasets. Algorithm

Revised: Input: Initial model w 0 , user sampling probability q , privacy parameter δ , clipping threshold S , noise level σ , local datasets D 1 ,...,D N , local epochs E , learning rate η .

Revised: Output: FL model w T and privacy cost (cid:15) Server executes: for each round t =

Revised: to T do m ← max( q · N, 1) ; U t ← (random subset of m

Original: get privacy spent () are the calls on the moments accountant M refer to the API of (Abadi et al., 2016).
Revised: ); for each user i ∈ U in parallel do ∆ w it ← UserUpdate i w t − ) ; w t ← w t − 1 + 1 m (cid:0)(cid:80) i ∈ U t Clip( ∆ w it ,S ) + N (cid:0) 0 ,σ 2 S 2 (cid:1)(cid:1) ; M . accum priv spending ( σ,q,δ ) ; (cid:15) =

Original: The privacy guarantee for Algorithm 1 is a generalization of (Abadi et al., 2016).
Revised: M . get privacy spent () ; return w T ,(cid:15) Procedure UserUpdate( i,w t − 1 ) w ← w t − 1 ; for local epoch e = 1 to E do for batch b ∈ local dataset D i do w ← w − η ∇ l ( w ; b ) ∆ w it ← w − w t − 1 ; return ∆ w it

Original: We recall
Revised: Procedure Clip( ∆ ,S ) return ∆ / max (cid:16) 1 , (cid:107) ∆ (cid:107) 2 S (cid:17)

Original: Proposition 1.
Revised: In Algorithm 1, M . accum priv spending () and M . get privacy spent () are the calls on the moments accountant M refer to the API of (Abadi et al., 2016). Given the user sampling probability q , noise level σ , FL rounds T , and a δ > 0 , UserDP-FedAvg satisﬁes ( (cid:15), δ ) -DP as below, which is a generalization of (Abadi et al., 2016). The aim is to analyze privacy budget (cid:15) , which is accumulated as T increases due to the continuous access to training data.

Original: Proposition 1 ( UserDP-FedAvg Privacy Guarantee) .
Revised: Proposition 1 ( UserDP-FedAvg Privacy Guarantee) .

Original: There exist constants c 1 and c 2 so that given user sampling probability q , and FL rounds T , for any ε < c 1 q 2 T , if σ ≥ c 2 q √ T log(1 /δ )
Revised: There exist constants c 1 and c 2 so that given user sampling probability q , and FL rounds T , for any ε < c 1 q 2 T , if σ ≥ c 2 q √ T log(1 /δ )

Original: (cid:15) , the randomized mechanism M in Algorithm 1 is ( (cid:15), δ ) -DP for any δ > 0 .
Revised: (cid:15) , the randomized mechanism M in Algorithm 1 is ( (cid:15), δ ) -DP for any δ > 0 .

Original: Proof.
Revised: Proof.

Original: The proof follows the proof of Theorem 1 in (Abadi et al., 2016), while the notations have slightly different meanings under FL settings.
Revised: The proof follows the proof of Theorem 1 in (Abadi et al., 2016), while the notations have slightly different meanings under FL settings.

Original: In Proposition 1, we use q to represent user-level sampling probability and T to represent FL training rounds.
Revised: In Proposition 1, we use q to represent user-level sampling probability and T to represent FL training rounds.

Revised: Note that the above privacy analysis can be further improved by R´enyi Differential Privacy (Mironov et al., 2019).

Original: Discussion (Li et al., 2020b) divide the user-level privacy into global privacy (Geyer et al., 2017;
Revised: Discussion (Li et al., 2020b) divide the user-level privacy into global privacy (Geyer et al., 2017;

Original: McMahan et al., 2018) and local privacy (Agarwal et al., 2018).
Revised: McMahan et al., 2018) and local privacy (Agarwal et al., 2018).

Original: In both local and global privacy, the norm of each update is clipped.
Revised: In both local and global privacy, the norm of each update is clipped.

Original: The difference lies in that the noise is added on the aggregated model updates in global privacy because a trusted server is assumed, while the noise is added on each local update in local privacy because it assumes that the central server might be malicious.
Revised: The difference lies in that the noise is added on the aggregated model updates in global privacy because a trusted server is assumed, while the noise is added on each local update in local privacy because it assumes that the central server might be malicious.

Original: Algorithmbelongs to global privacy.
Revised: Algorithmbelongs to global privacy.

Original: A.1. I NS DP-F ED SGD
Revised: A.1. I NS DP-F ED SGD

Revised: Deﬁnition 3 (Instance-level ( (cid:15), δ ) -DP) .

Revised: Let D be the dataset that is the union of local training examples from all users.

Original: Under FedSGD, when each local model performs one step of DP-SGD (Abadi et al., 2016), the randomized mechanism M that outputs the global model preserves the instance-level DP.
Revised: Then, D and D (cid:48) are adjacent if they differ by one instance. The mechanism M is instance-level ( (cid:15), δ ) -DP if it meets Deﬁnition 1 with D and D (cid:48) as adjacent datasets. [Table]

Revised: [Table]

Revised: [Table]

Revised: [Table]

Revised: [Table]

Revised: [Table]

Original: We can regard the one-step update for the global model in Algorithm 2 as:
Revised: | D i | ); it do x ← ∇ l w ; x j ) ; ¯ g ( x j ) ← Clip g x j ) ,S ) ; (cid:101) g ← 1 L (cid:16)(cid:80) j ¯ g ( x j ) + N (cid:0) 0 ,σ 2 S 2 (cid:1)(cid:17) ; w ← w − η (cid:101) g ; ∆ w it ← w − w t − 1 ; Clip

Original: [Equation]
Revised: ∆ ) return ∆ max ∆ 2 S

Revised: Under FedSGD, when each local model performs one step of DP-SGD (Abadi et al., 2016), the randomized mechanism M that outputs the global model preserves the instance-level DP.

Revised: We can regard the one-step update for the global model in Algorithm 2 as:

Original: [Equation]
Revised: [Equation]

Original: Proposition 2 ( InsDP-FedSGD Privacy Guarantee) .
Revised: Proposition 2 ( InsDP-FedSGD Privacy Guarantee) .

Original: There exist constants c 1 and c 2 so that given batch sampling probability p , and user sampling probability q , the number of selected users each round m , and FL rounds T , for any ε < c 1 ( pq ) 2 T , if σ ≥ c 2 pq √ T log(1 /δ )
Revised: There exist constants c 1 and c 2 so that given batch sampling probability p , and user sampling probability q , the number of selected users each round m , and FL rounds T , for any ε < c 1 ( pq ) 2 T , if σ ≥ c 2 pq √ T log

Original:  (cid:15) √ m , the randomized mechanism M in Algorithm 2 is ( (cid:15), δ ) -DP for any δ > 0 .
Revised: [Equation] (cid:15) √ m , the randomized mechanism M in Algorithm 2 is ( (cid:15), δ ) -DP for any δ > 0 .

Original: Proof.
Revised: Proof.

Original: i)
Revised: i)

Original: In instance-level DP, we consider the sampling probability of each instance under the combination of user-level sampling and batch-level sampling.
Revised: In instance-level DP, we consider the sampling probability of each instance under the combination of user-level sampling and batch-level sampling.

Original: Since the user-level sampling probability is q and the batch-level sampling probablity is p , each instance is sampled with probability pq . ii)
Revised: Since the user-level sampling probability is q and the batch-level sampling probablity is p , each instance is sampled with probability pq . ii)

Original: Additionally, since the sensitivity of instance-wise gradient w.r.t one instance is S , after local gradient descent and server FL aggregation, the equivalent sensitivity of global model w.r.t one instance is S (cid:48) =
Revised: Additionally, since the sensitivity of instance-wise gradient w.r.t one instance is S , after local gradient descent and server FL aggregation, the equivalent sensitivity of global model w.r.t one instance is S (cid:48) =

Original: ηSLm according to Eq (5).
Revised: ηSLm according to Eq (5).

Original: iii)
Revised: iii)

Original: Moreover, since the local noise is n i ∼ N (0 , σ 2 S 2 ) , then the “virtual” global noise is n = ηmL (cid:80) i ∈
Revised: Moreover, since the local noise is n i ∼ N (0 , σ 2 S 2 ) , then the “virtual” global noise is n = ηmL (cid:80) i ∈

Original: U t n i according to Eq (5), so n ∼ N (0 , η 2 σ 2 S 2 mL 2 ) .
Revised: U t n i according to Eq (5), so n ∼ N (0 , η 2 σ 2 S 2 mL 2 ) .

Original: Let η 2 σ 2 S 2 mL 2 = σ (cid:48) 2 S (cid:48)
Revised: Let η 2 σ 2 S 2 mL 2 = σ (cid:48) 2 S (cid:48)

Original: 2 such that n ∼ N (0 , σ (cid:48) 2 S (cid:48) 2 ) .
Revised: 2 such that n ∼ N (0 , σ (cid:48) 2 S (cid:48) 2 ) .

Original: Because S (cid:48) = ηSLm , the equivalent global noise level is σ (cid:48) [-
-] = σ 2 m , i.e., σ (cid:48)
Revised: Because S (cid:48) = ηS Lm , the equivalent global noise level is σ (cid:48) [-
-]2 = σ 2 m , i.e., σ (cid:48)

Original: =
Revised: =

Original: σ √ m .
Revised: σ √ m .

Original: In Proposition 2, we use pq to represent instance-level sampling probability, T to represent FL training rounds, σ √ m to represent the equivalent global noise level.
Revised: In Proposition 2, we use pq to represent instance-level sampling probability, T to represent FL training rounds, σ √ m to represent the equivalent global noise level.

Original: The rest of the proof follows the proof of Theorem 1 in (Abadi et al., 2016).
Revised: The rest of the proof follows the proof of Theorem 1 in (Abadi et al., 2016).

Original: We defer the DP bound evaluation comparison between InsDP-FedSGD and Dopamine to Appendix A.3.7.
Revised: We defer the DP bound evaluation comparison between InsDP-FedSGD and Dopamine to Appendix A.3.7.

Original: A.1. I NS DP-F ED A VG
Revised: A.1. I NS DP-F ED A VG

Revised: [Table]

Revised: [Table]

Revised: [Table]

Revised: [Table]

Revised: [Table]

Original: Lemma 2 ( InsDP-FedAvg Privacy Guarantee when T = 1 ) .
Revised: [Table] [Table] [Table] [Table]

Revised: [Table]

Revised: [Table]

Revised: [Table]

Revised: [Table]

Revised: [Table]

Revised: [Table]

Revised: [Table]

Original:  In Algorithm 3, when T = 1 , suppose local mechanism M i satisﬁes ( (cid:15)
Revised: [Table] Clip Lemma 2 ( InsDP-FedAvg Privacy Guarantee when T = 1 ) . In Algorithm 3, when T = 1 , suppose local mechanism M i satisﬁes ( (cid:15)

Original: i , δ ) -DP, then global mechanism M satisﬁes (max i ∈
Revised: i , δ ) -DP, then global mechanism M satisﬁes (max i ∈

Original: [ N ] (cid:15) i , δ ) -DP.
Revised: [ N ] (cid:15) i , δ ) -DP.

Original: Proof.
Revised: Proof.

Original: We can regard federated learning as partitioning a dataset D into N disjoint subsets { D 1 , D 2 , . . .
Revised: We can regard federated learning as partitioning a dataset D into N disjoint subsets { D 1 , D 2 , . . .

Original: , D N } .
Revised: , D N } .

Original: N mechanisms {M 1 , . . .
Revised: N mechanisms {M 1 , . . .

Original: , M N } are operated on these N parts separately and each M i satisﬁes its own (cid:15) i -DP for i ∈ [1 , N ] .
Revised: , M N } are operated on these N parts separately and each M i satisﬁes its own (cid:15) i -DP for i ∈ [1 , N ] .

Original: Note that if i -th user is not selected , (cid:15) i =because local dataset D i is not accessed and there is no privacy cost.
Revised: Note that if i -th user is not selected , (cid:15) i =because local dataset D i is not accessed and there is no privacy cost.

Original: Without loss of generality, we assume the modiﬁed data sample x (cid:48)
Revised: Without loss of generality, we assume the modiﬁed data sample x (cid:48)

Original: ( x → x (cid:48) causes D → D (cid:48) ) is in the local dataset of k -th client D k .
Revised: ( x → x (cid:48) causes D → D (cid:48) ) is in the local dataset of k -th client D k .

Original: Let D, D (cid:48) be two neighboring datasets ( D k , D (cid:48)
Revised: Let D, D (cid:48) be two neighboring datasets ( D k , D (cid:48)

Original: k are also two neighboring datasets).
Revised: k are also two neighboring datasets).

Original: M is randomized mechanism that outputs the global model, and M i is the randomized mechanism that outputs the local model update ∆ w i .
Revised: M is randomized mechanism that outputs the global model, and M i is the randomized mechanism that outputs the local model update ∆ w i .

Original: Suppose w 0 is the initialized and deterministic global model, and { z 1 , . .
Revised: Suppose w 0 is the initialized and deterministic global model, and { z 1 , . .

Original: , z N } are randomized local updates.
Revised: , z N } are randomized local updates.

Original: We have a sequence of computations { z 1 =
Revised: We have a sequence of computations { z 1 =

Original: M 1 ( D 1 ) , z
Revised: M 1 ( D 1 ) , z

Original: 2 = M 2 ( D 2 ; z 1 ) , z 3 =
Revised: 2 = M 2 ( D 2 ; z 1 ) , z 3 =

Original: M 3 ( D 3 ; z 1 , z 2 ) . . .
Revised: M 3 ( D 3 ; z 1 , z 2 ) . . .

Original: } and z = M ( D ) =
Revised: } and z = M ( D ) =

Original: w 0 + (cid:80) Ni =
Revised: w 0 + (cid:80) Ni =

Original: z i .
Revised: z i .

Original: Note that if i -th user is not selected , z i = 0 .
Revised: Note that if i -th user is not selected , z i = 0 .

Original: According to the parallel composition (Tu), we have
Revised: According to the parallel composition (Tu), we have

Revised: Pr[ M ( D ) =

Revised: z ]

Original: [Equation]
Revised: [Equation]

Original: So M satisﬁes (cid:15) k
Revised: So M satisﬁes (cid:15) k

Original: -DP when the modiﬁed data sample lies in the subset D k .
Revised: -DP when the modiﬁed data sample lies in the subset D k .

Original: Consider the worst case of where the modiﬁed data sample could fall in, we know that M satisﬁes (max i ∈ [ N ] (cid:15) i ) -DP.
Revised: Consider the worst case of where the modiﬁed data sample could fall in, we know that M satisﬁes (max i ∈ [ N ] (cid:15) i ) -DP.

Original: We recall Theorem 4.
Revised: We recall Theorem 4.

Original: Theorem 4 ( InsDP-FedAvg Privacy Guarantee) .
Revised: Theorem 4 ( InsDP-FedAvg Privacy Guarantee) .

Original: In Algorithm 3, during round t , if the local mechanism M i satisﬁes ( (cid:15) it , δ ) -DP, then the global mechanism M satisﬁes (cid:0) max i ∈ [ N ] (cid:15) it , δ (cid:1) -DP.
Revised: In Algorithm 3, during round t , if the local mechanism M i satisﬁes ( (cid:15) it , δ ) -DP, then the global mechanism M satisﬁes (cid:0) max i ∈ [ N ] (cid:15) it , δ (cid:1) -DP.

Original: Proof.
Revised: Proof.

Original: Again, without loss of generality, we assume the modiﬁed data sample x (cid:48)
Revised: Again, without loss of generality, we assume the modiﬁed data sample x (cid:48)

Original: ( x → x (cid:48) causes
Revised: ( x → x (cid:48) causes

Original: D → D (cid:48) ) is in the local dataset of k -th user D k .
Revised: D → D (cid:48) ) is in the local dataset of k -th user D k .

Original: We ﬁrst consider the case when all users are selected.
Revised: We ﬁrst consider the case when all users are selected.

Original: At each round t , N mechanisms are operated on N disjoint parts and each M i t satisﬁes own (cid:15) i -DP where (cid:15) i is the privacy cost for accessing the local dataset D i for one round (not accumulating over previous rounds).
Revised: At each round t , N mechanisms are operated on N disjoint parts and each M it satisﬁes own (cid:15) i -DP where (cid:15) i is the privacy cost for accessing the local dataset D i for one round (not accumulating over previous rounds).

Original: Let D, D (cid:48) be two neighboring datasets ( D k , D (cid:48)
Revised: Let D, D (cid:48) be two neighboring datasets ( D k , D (cid:48)

Original: k are also two neighboring datasets).
Revised: k are also two neighboring datasets).

Original: Suppose z
Revised: Suppose z

Original: =
Revised: =

Original: M t − 1 ( D ) is the aggregated randomized global model at round t − 1 , and { z 1 , . .
Revised: M t − 1 ( D ) is the aggregated randomized global model at round t − 1 , and { z 1 , . .

Original: , z N } are the randomized local updates at round t , we have a sequence of computations { z 1 = M 1 t ( D 1 ; z 0 ) , z 2 = M 2 t
Revised: , z N } are the randomized local updates at round t , we have a sequence of computations { z 1 = M 1 t ( D 1 ; z 0 ) , z 2 = M 2 t

Original: ( D 2 ; z 0 , z 1 ) , z
Revised: ( D 2 ; z 0 , z 1 ) , z

Original: 3 =
Revised: 3 =

Original: M 3 t ( D 3 ; z 0 , z 1 , z 2 ) . . . } and z = M t ( D ) =
Revised: M 3 t ( D 3 ; z 0 , z 1 , z 2 ) . . . } and z = M t ( D ) =

Original: z 

Revised: z

Original: + (cid:80) Ni
Revised: + (cid:80) Ni

Original: z i .
Revised: z i .

Original: We ﬁrst consider the sequential composition (Dwork & Roth, 2014) to accumulate the privacy cost over FL rounds.
Revised: We ﬁrst consider the sequential composition (Dwork & Roth, 2014) to accumulate the privacy cost over FL rounds.

Original: According to parallel composition, we have
Revised: According to parallel composition, we have

Original: Pr[ M t
Revised: Pr[ M t

Original: ( D ) =
Revised: ( D ) =

Original: z ] =
Revised: z ] =

Original: Pr[ M t − 1 ( D ) =
Revised: Pr[ M t − 1 ( D ) =

Original: z 0 ]
Revised: z 0 ]

Original: N (cid:89) i =
Revised: N (cid:89) i =

Original: Pr[ M it
Revised: Pr[ [Equation] [Equation] z

Original: ( D i ; z 0 , z 1 , . . .
Revised: [Equation] z [Equation] [Equation]

Original: , z i − 1 )
Revised: − [Equation] z

Original:  = 
Revised: [Equation] ] = Pr [Equation]

Original: z i ] =
Revised: M [Equation] −

Original: Pr[ M t − 1 ( D ) =
Revised: [Equation] ( D [Equation] =

Original: z 0 ]
Revised: z 0 ]

Original: Pr[ M kt
Revised: Pr [Equation]

Original: ( D k ; z 0 , z 1 , . . .
Revised: ( D [Equation] z 0 , z [Equation]

Original: , z k − 1 )=
Revised: [Equation] z k − 1 )

Original:  z k ] (cid:89) i (cid:54) =
Revised: [Equation] z k ] (cid:89) i (cid:54) =

Original: k
Revised: k

Original: Pr[ M it
Revised: Pr[ M [Equation]

Original: ( D i ; z 0 , z 1 , . . .
Revised: ( D [Equation] z [Equation]

Original: , z i − 1 )
Revised: z [Equation]

Original: =
Revised: [Equation] z [Equation] −

Original:  z i ] ≤ exp( (cid:15) t − 1 )
Revised: [Equation] z [Equation] ] ≤ exp( (cid:15) t − 1 )

Original: Pr[ M t − 1 ( D (cid:48) ) =
Revised: Pr[ M t − 1 ( D (cid:48) ) =

Original: z 0 ] exp( (cid:15) k )
Revised: z 0 ] exp( (cid:15) k )

Original: Pr[ M kt
Revised: Pr[ M kt

Original: ( D (cid:48)
Revised: ( D (cid:48)

Original: k ; z 0 , z 1 , . .
Revised: k ; z 0 , z 1 , . .

Original: z k − 1 )
Revised: z k − 1 )

Original: =
Revised: =

Original: z k ] (cid:89) i (cid:54) =
Revised: z k ] (cid:89) i (cid:54) =

Original: k
Revised: k

Original: Pr[ M it
Revised: Pr[ M it

Original: ( D i ; z 0 , z 1 , . . .
Revised: ( D i ; z 0 , z 1 , . . .

Original: , z i − 1 )
Revised: , z i − 1 )

Original: =
Revised: =

Original: z i ] = exp( (cid:15) t − 1 +
Revised: z i ] = exp( (cid:15) t − 1 +

Original: (cid:15) k ) Pr[ M t
Revised: (cid:15) k ) Pr[ M t

Original: ( D (cid:48) )
Revised: ( D (cid:48) )

Original: =
Revised: =

Original: z ]
Revised: z ]

Original: Therefore, M t satisﬁes (cid:15) t -DP, where (cid:15) t = (cid:15) t − 1 + (cid:15) k .
Revised: Therefore, M t satisﬁes (cid:15) t -DP, where (cid:15) t = (cid:15) t − 1 + (cid:15) k .

Original: Because the modiﬁed data sample always lies in D k over t rounds and (cid:15) 

Revised: Because the modiﬁed data sample always lies in D k over t rounds and (cid:15)

Original: = 0 , we can have (cid:15) t = t(cid:15) k , which means that the privacy guarantee of global mechanism M t is only determined by the local mechanism of k -th user over t rounds.
Revised: = 0 , we can have (cid:15) t = t(cid:15) k , which means that the privacy guarantee of global mechanism M t is only determined by the local mechanism of k -th user over t rounds.

Original: Moreover, moment accountant (Abadi et al., 2016) is known to reduce the privacy cost from O ( t ) to O ( √ t ) .
Revised: Moreover, moment accountant (Abadi et al., 2016) is known to reduce the privacy cost from O ( t ) to O ( √ t ) .

Original: We can use the more advanced composition, i.e., moment accountant, instead of the sequential composition, to accumulate the privacy cost for local mechanism M k over t FL rounds.
Revised: We can use the more advanced composition, i.e., moment accountant, instead of the sequential composition, to accumulate the privacy cost for local mechanism M k over t FL rounds.

Original: In addition, we consider user subsampling.
Revised: In addition, we consider user subsampling.

Original: As described in Algorithm 3, if the user i is not selected at round t , then its local privacy cost is kept unchanged at this round.
Revised: As described in Algorithm 3, if the user i is not selected at round t , then its local privacy cost is kept unchanged at this round.

Original: Take the worst case of where x (cid:48) could lie in, at round t , M satisﬁes (cid:15) t -DP, where (cid:15) t =
Revised: Take the worst case of where x (cid:48) could lie in, at round t , M satisﬁes (cid:15) t -DP, where (cid:15) t =

Original: max i ∈
Revised: max i ∈

Original: [ N ] (cid:15) it , local mechanism M i satisﬁes (cid:15) it -DP, and the local privacy cost (cid:15) it is accumulated via local moment accountant in i -th user over t rounds.
Revised: [ N ] (cid:15) it , local mechanism M i satisﬁes (cid:15) it -DP, and the local privacy cost (cid:15) it is accumulated via local moment accountant in i -th user over t rounds.

Original: A. T HREAT M ODELS
Revised: A. T HREAT M ODELS

Original: We consider targeted poisoning attacks of two types.
Revised: We consider targeted poisoning attacks of two types.

Original: In backdoor attacks (Gu et al., 2019; Chen et al., 2017a), the goal is to embed a backdoor pattern (i.e., a trigger) during training such that any test input with such pattern will be mis-classiﬁed as the target.
Revised: In backdoor attacks (Gu et al., 2019; Chen et al., 2017a), the goal is to embed a backdoor pattern (i.e., a trigger) during training such that any test input with such pattern will be mis-classiﬁed as the target.

Original: In label ﬂipping attacks (Biggio et al., 2012; Huang et al., 2011), the labels of clean training examples from one source class are ﬂipped to the target class while the features of the data are kept unchanged.
Revised: In label ﬂipping attacks (Biggio et al., 2012; Huang et al., 2011), the labels of clean training examples from one source class are ﬂipped to the target class while the features of the data are kept unchanged.

Original: In FL, the purpose of backdoor attacks is to manipulate local models with backdoored local data, so that the global model would behave normally on untampered data samples while achieving high attack success rate on clean data (Bagdasaryan et al., 2020).
Revised: In FL, the purpose of backdoor attacks is to manipulate local models with backdoored local data, so that the global model would behave normally on untampered data samples while achieving high attack success rate on clean data (Bagdasaryan et al., 2020).

Original: Given the same purpose, distributed backdoor attack (DBA)
Revised: Given the same purpose, distributed backdoor attack (DBA)

Original: (Xie et al., 2019) decomposes the same backdoor pattern to several smaller ones and embeds them to different local training sets for different adversarial users.
Revised: (Xie et al., 2019) decomposes the same backdoor pattern to several smaller ones and embeds them to different local training sets for different adversarial users.

Original: The goal of label ﬂipping attack against FL is to manipulate local datasets with ﬂipped labels such that the global model will mis-classify the test data in the source class as the target class.
Revised: The goal of label ﬂipping attack against FL is to manipulate local datasets with ﬂipped labels such that the global model will mis-classify the test data in the source class as the target class.

Original: The model replacement (Bagdasaryan et al., 2020) is a more powerful approach to perform the above attacks, where the attackers ﬁrst train the local models using the poisoned datasets and then scale the malicious updates before sending them to the server.
Revised: The model replacement (Bagdasaryan et al., 2020) is a more powerful approach to perform the above attacks, where the attackers ﬁrst train the local models using the poisoned datasets and then scale the malicious updates before sending them to the server.

Original: This way, the attacker’s updates would have a stronger impact on the FL model.
Revised: This way, the attacker’s updates would have a stronger impact on the FL model.

Original: We use the model replacement method to perform poisoning attacks and study the effectiveness of DPFL.
Revised: We use the model replacement method to perform poisoning attacks and study the effectiveness of DPFL.

Original: For UserDP-FedAvg , we consider backdoor, distributed backdoor, and label ﬂipping attacks via the model replacement approach.
Revised: For UserDP-FedAvg , we consider backdoor, distributed backdoor, and label ﬂipping attacks via the model replacement approach.

Original: Next, we formalize the attack process and introduce the notations.
Revised: Next, we formalize the attack process and introduce the notations.

Original: Suppose the attacker controls k adversarial users, i.e., there are k attackers out of N users.
Revised: Suppose the attacker controls k adversarial users, i.e., there are k attackers out of N users.

Original: Let B be the original user set of N benign users, and B (cid:48) be the user set that contains k attackers.
Revised: Let B be the original user set of N benign users, and B (cid:48) be the user set that contains k attackers.

Original: Let
Revised: Let

Original: D :=
Revised: D :=

Original: { D 1 , D 2 , . . .
Revised: { D 1 , D 2 , . . .

Original: , D N } be the union of original benign local datasets across all users.
Revised: , D N } be the union of original benign local datasets across all users.

Original: For a data sample z i j :=
Revised: For a data sample z i j :=

Original: { x i j , y i j } in D i , we denote its backdoored version as z (cid:48) i j :=
Revised: { x i j , y i j } in D i , we denote its backdoored version as z (cid:48) ij :=

Original: { x i j + δ x , y ∗ } , where δ x is the backdoor pattern, y ∗ is the targeted label; the distributed backdoor attack (DBA) version as z (cid:48) ij :=
Revised: { x i j + δ x , y ∗ } , where δ x is the backdoor pattern, y ∗ is the targeted label; the distributed backdoor attack (DBA) version as z (cid:48) ij :=

Original: { x ij + δ ix , y ∗ } , where δ ix is the distributed backdoor pattern for attacker i ; the label-ﬂipped version as z (cid:48) ij := { x ij , y ∗ } .
Revised: { x ij + δ ix , y ∗ } , where δ ix is the distributed backdoor pattern for attacker i ; the label-ﬂipped version as z (cid:48) ij := { x ij , y ∗ } .

Original: Note that the composition of all DBA patterns is equivalent to the backdoor pattern, i.e., (cid:80) ki =1 δ ix = δ x .
Revised: Note that the composition of all DBA patterns is equivalent to the backdoor pattern, i.e., (cid:80) ki =1 δ ix = δ x .

Original: We assume attacker i has α
Revised: We assume attacker i has α

Original: i fraction of poisoned samples in its local dataset D (cid:48)
Revised: i fraction of poisoned samples in its local dataset D (cid:48)

Original: i .
Revised: i .

Original: Let D (cid:48) :
Revised: Let D (cid:48) :

Original: = { D (cid:48) 1 , . . .
Revised: = { D (cid:48) 1 , . . .

Original: , D (cid:48)
Revised: , D (cid:48)

Original: k − 1 , D (cid:48)
Revised: k − 1 , D (cid:48)

Original: k , D k +1 , . . .
Revised: k , D k +1 , . . .

Original: , D N } be the union of local datasets[Table] when k attackers are present.
Revised: , D N } be the union of local datasets when k attackers are present.

Original: The adversarial user i performs model replacement by scaling the model update with hyperparameter γ before submitting it to the server, i.e., ∆ w it ← γ ∆ w it .
Revised: The adversarial user i performs model replacement by scaling the model update with hyperparameter γ before submitting it to the server, i.e., ∆ w it ← γ ∆ w it .

Revised: In our threat model, we consider the attacker that follows our training protocol and has no control over which users are sampled.

Original: For InsDP-FedAvg , we consider both backdoor and label ﬂipping attacks.
Revised: For InsDP-FedAvg , we consider both backdoor and label ﬂipping attacks.

Original: Since distributed backdoor and model replacement attacks are proposed for adversarial users rather than adversarial instances, we do not consider them for instance-level DPFL.
Revised: Since distributed backdoor and model replacement attacks are proposed for adversarial users rather than adversarial instances, we do not consider them for instance-level DPFL.

Original: There are k backdoored or label-ﬂipped instances { z (cid:48) 1 , z (cid:48) 2 , . . .
Revised: There are k backdoored or label-ﬂipped instances { z (cid:48) 1 , z (cid:48) 2 , . . .

Original: , z (cid:48) k } , which could be controlled by same or multiple users.
Revised: , z (cid:48) k } , which could be controlled by same or multiple users.

Revised: In our threat model, we consider the attacker that follows our training protocol and has no control over which data partition (or batch) is sampled.

Revised: Note that we do not assume that the adversaries’ poisoning data always be sampled.

Revised: In our algorithms, each batch is randomly subsampled, so the adversaries cannot control if poisoned data are sampled in each step.

Original: A. E XPERIMENTAL D ETAILS AND A DDITIONAL R ESULTS A.3. D ATASETS AND M ODELS
Revised: A. E XPERIMENTAL D ETAILS AND A DDITIONAL R ESULTS A.3. D ATASETS AND M ODELS

Original: We evaluate our robustness certiﬁcation results with two datasets: MNIST (LeCun & Cortes, 2010) and CIFAR-10 (Krizhevsky, 2009).
Revised: We evaluate our robustness certiﬁcation results with two datasets: MNIST (LeCun & Cortes, 2010) and CIFAR-10 (Krizhevsky, 2009).

Original: For each dataset, we use corresponding standard CNN architectures in the differential privacy library (opa, 2021) of PyTorch (Paszke et al., 2019).
Revised: For each dataset, we use corresponding standard CNN architectures in the differential privacy library (opa, 2021) of PyTorch (Paszke et al., 2019).

Original: MNIST :
Revised: MNIST :

Original: We study an image classiﬁcation problem of handwritten digits in MNIST.
Revised: We study an image classiﬁcation problem of handwritten digits in MNIST.

Original: It is a dataset of 70000 28x28 pixel images of digits in 10 classes, split into a train set of 60000 images and a test set of 10000 images.
Revised: It is a dataset of 70000 28x28 pixel images of digits in 10 classes, split into a train set of 60000 images and a test set of 10000 images.

Original: Except Section A.3.6, we consider binary classiﬁcation on classes 0 and 1, making our train set contain 12665 samples, and the test set 2115 samples.
Revised: Except Section A.3.6, we consider binary classiﬁcation on classes 0 and 1, making our train set contain 12665 samples, and the test set 2115 samples.

Original: The model consists of two Conv-ReLu-MaxPooling layers and two linear layers.
Revised: The model consists of two Conv-ReLu-MaxPooling layers and two linear layers.

Original: CIFAR-10 :
Revised: CIFAR-10 :

Original: We study image classiﬁcation of vehicles and animals in CIFAR-10.
Revised: We study image classiﬁcation of vehicles and animals in CIFAR-10.

Original: This is a harder dataset than MNIST, consisting of 60000 32x32x3 images, split into a train set of 50000 and a test set of 10000.
Revised: This is a harder dataset than MNIST, consisting of 60000 32x32x3 images, split into a train set of 50000 and a test set of 10000.

Original: Except Section A.3.6, we consider binary classiﬁcation on class airplane and bird, making our train set contain 10000 samples, and the test set 2000 samples.
Revised: Except Section A.3.6, we consider binary classiﬁcation on class airplane and bird, making our train set contain 10000 samples, and the test set 2000 samples.

Original: The model consists of four Conv-ReLu-AveragePooling layers and one linear layer.
Revised: The model consists of four Conv-ReLu-AveragePooling layers and one linear layer.

Original: When training on CIFAR10, we follow the standard practice for differential privacy (Abadi et al., 2016; Jagielski et al., 2020) and ﬁne-tune a whole model pre-trained non-privately on the more complex CIFAR100, a similarly sized but more complex benchmark dataset. 
Revised: When training on CIFAR10, we follow the standard practice for differential privacy (Abadi et al., 2016; Jagielski et al., 2020) and ﬁne-tune a whole model pre-trained non-privately on the more complex CIFAR100, a similarly sized but more complex benchmark dataset. We can achieve reasonable performance on CIFAR-10 datasets by only training (ﬁne-tuning) few rounds.

Revised: Sent140 : We consider a text sentiment analysis task on tweets from Sentiment140 (Go et al.)

Revised: (Sent140) which involves classifying Twitter posts as positive or negative.

Revised: We use a two layer LSTM binary classiﬁer containing 256 hidden units with pretrained 300D GloVe embedding (Pennington et al., 2014).

Original:  A.3. T RAINING D ETAILS
Revised: Each twitter account corresponds to a device. We use the same network architecture, non-iid dataset partition method, number of selected user per round, learning rate, batch size, etc. as in (Li et al., 2018), which are summarized in Table 1. A.3. T RAINING D ETAILS

Original: We simulate the federated learning setup by splitting the training datasets for N FL users in an i.i.d manner.
Revised: We simulate the federated learning setup by splitting the training datasets for N FL users in an i.i.d manner.

Original: FL users run SGD with learning rate η , momentum 0 .
Revised: FL users run SGD with learning rate η , momentum 0 .

Original: 9 , weight decay 0 .
Revised: 9 , weight decay 0 .

Original: to update the local models.
Revised: to update the local models.

Original: The training parameter setups are summarized in Table 1.
Revised: The training parameter setups are summarized in Table 1.

Original: Following (McMahan et al., 2018) that use δ ≈ 1 N 1 . 1 as privacy parameter, for UserDP-FedAvg we set δ = 0 .according to the total number of users, and for InsDP-FedAvg we set δ = 0 .
Revised: Following (McMahan et al., 2018) that use δ ≈ 1 N 1 . 1 as privacy parameter, for UserDP-FedAvg we set δ = 0 .according to the total number of users, and for InsDP-FedAvg we set δ = 0 .

Original: according the total number of training samples.
Revised: according the total number of training samples.

Original: Next we summarize the privacy guarantees and clean accuracy offered when we study the certiﬁed prediction and certiﬁed attack cost, which are also the training parameters setups when k = 0 in Figure 1, 2, 3, 4, 5, 8. 
Revised: Next we summarize the privacy guarantees and clean accuracy offered when we study the certiﬁed prediction and certiﬁed attack cost, which are also the training parameters setups when k = 0 in Figure 1, 2, 3, 4, 5, 8. Algorithm Dataset # [Table]

Revised: samples

Revised: N m

Original: User-level DPFL
Revised: E V batch size η S δ ¯ C

Revised: [Table]

Revised: MNIST 200 20/ 0.02 0.

Revised: 0.5 UserDP

Revised: [Table]

Revised: CIFAR200 / 0.

Revised: [Table]

Revised: Sent  805 / 0.

Revised: [Table]

Revised: MNIST   /  0.

Revised: [Table]

Revised: [Table]

Revised: [Table]

Revised: CIFAR-

Revised: [Table]

Original:  In order to study the user-level certiﬁed prediction under different privacy guarantee, for MNIST, we set (cid:15) to be 0 .
Revised: [Table] [Table] User-level DPFL In order to study the user-level certiﬁed prediction under different privacy guarantee, for MNIST, we set (cid:15) to be 0 .

Original: 8305 , 4 . 8913 , 6 . 9269 , which are obtained by training UserDP-FedAvg FL model for 3 rounds with noise level σ = 3 . 0 , 2 . 3 , 1 . 8 , 1 . 5 , 1 . 0 , 0 .
Revised: 8305 , 4 . 8913 , 6 . 9269 , which are obtained by training UserDP-FedAvg FL model for 3 rounds with noise level σ = 3 . 0 , 2 . 3 , 1 . 8 , 1 . 5 , 1 . 0 , 0 .

Original: 5 , respectively (Figure 1(a)).
Revised: 5 , respectively (Figure 1(a)).

Original:  CIFAR-10, we set (cid:15) to be 0 . 1083 , 0 . 1179 , 0 . 1451 , 0 . 2444 , 0 .3663 , 0 . 4527 , 0 . 5460 , 0 . 8781 , which are obtained by training
Revised: For CIFAR-10, we set (cid:15) to be 0 . 1083 , 0 . 1179 , 0 . 1451 , 0 . 2444 , 0 . 8781 , which are obtained by training

Original: UserDP-FedAvg FL model for one round with noise level σ = 10 . 0 , 8 . 0 , 6 . 0 , 4 . 0 , 3 . 0 , 2 . 6 , 2 . 3 , 1 . 7 , respectively (Figure 1(b)).
Revised: UserDP-FedAvg FL model for one round with noise level σ = 10 . 0 , 8 . 0 , 6 . 0 , 4 . 0 , 3 . 0 , 2 . 6 , 2 . 3 , 1 . 7 , respectively (Figure 1(b)).

Revised: The clean accuracy (average over 1000 runs) of UserDP-FedAvg under non-DP training ( (cid:15) = ∞ ) and DP training (varying (cid:15) ) on MNIST and CIFAR-10 are reported in

Revised: Table.

Revised: 2 and Table.

Original:  To certify the attack cost under different number of adversarial users k (Figure 2), for MNIST, we set the noise level σ to be 2.5.
Revised: 3 respectively. [Table] [Table] To certify the attack cost under different number of adversarial users k (Figure 2), for MNIST, we set the noise level σ to be 2.5.

Original: When k = 0 , after training UserDP-FedAvg for T = 3 , 4 , 5 rounds, we obtain FL models with privacy guarantee (cid:15) = 0 .
Revised: When k = 0 , after training UserDP-FedAvg for T = 3 , 4 , 5 rounds, we obtain FL models with privacy guarantee (cid:15) = 0 .

Original: 4344 and clean accuracy (average over M runs) 86 .
Revised: 4344 and clean accuracy (average over M runs) 86 .

Original: For CIFAR-10, we set the noise level σ to be 3 . 0 .
Revised: For CIFAR-10, we set the noise level σ to be 3 . 0 .

Original: After training UserDP-FedAvg for T = 3 , 4 rounds under k = 0 , we obtain FL models with privacy guarantee (cid:15) = 0 .
Revised: After training UserDP-FedAvg for T = 3 , 4 rounds under k = 0 , we obtain FL models with privacy guarantee (cid:15) = 0 .

Original: 5978 and clean accuracy 78 .
Revised: 5978 and clean accuracy 78 .

Original: With the interest of certifying attack cost under different user-level DP guarantee (Figure 3, Figure 5), we explore the empirical attack cost and the certiﬁed attack cost lower bound given different (cid:15) .
Revised: With the interest of certifying attack cost under different user-level DP guarantee (Figure 3, Figure 5), we explore the empirical attack cost and the certiﬁed attack cost lower bound given different (cid:15) .

Original: For MNIST, we set the privacy guarantee (cid:15) to be 1 . 2716 , 0 . 8794 , 0 . 6608 , 0 . 5249 , 0 .
Revised: For MNIST, we set the privacy guarantee (cid:15) to be 1 . 2716 , 0 . 8794 , 0 . 6608 , 0 . 5249 , 0 .

Original: 4344 , which are obtained by training UserDP-FedAvg FL models for 5 rounds under noise level σ = 1 . 3 , 1 . 6 , 1 . 9 , 2 . 2 , 2 . 5 , respectively, and the clean accuracy for the corresponding models are 99 .
Revised: 4344 , which are obtained by training UserDP-FedAvg FL models for 5 rounds under noise level σ = 1 . 3 , 1 . 6 , 1 . 9 , 2 . 2 , 2 . 5 , respectively, and the clean accuracy for the corresponding models are 99 .

Original: For CIFAR-10, we set the privacy guarantee (cid:15) to be 1 . 600 , 1 .
Revised: For CIFAR-10, we set the privacy guarantee (cid:15) to be 1 . 600 , 1 .

Original: 6543 , 0 . 5978 , which are obtained by training UserDP-FedAvg FL models for 4 rounds under noise level σ = 1 . 5 , 1 . 8 , 2 . 0 , 2 . 3 , 2 . 5 , 2 . 8 , 3 . 0 , respectively, and the clean accuracy for the corresponding models are 85 .
Revised: 6543 , 0 . 5978 , which are obtained by training UserDP-FedAvg FL models for 4 rounds under noise level σ = 1 . 5 , 1 . 8 , 2 . 0 , 2 . 3 , 2 . 5 , 2 . 8 , 3 . 0 , respectively, and the clean accuracy for the corresponding models are 85 .

Original: Instance-level
Revised: Instance-level

Original: DPFL To certify the prediction for instance-level
Revised: DPFL To certify the prediction for instance-level

Original: DPFL under different privacy guarantee, for
Revised: DPFL under different privacy guarantee, for

Original: MNIST, we set privacy cost (cid:15) to be 0 . 2029 , 0 .
Revised: MNIST, we set privacy cost (cid:15) to be 0 . 2029 , 0 .

Original: 6373 , 1 . 0587 , 3 . 5691 , which are obtained by training InsDP-FedAvg FL models for 3 rounds with noise level σ = 15 , 10 , 8 , 5 , 4 , 3 , 2 , 1 , respectively (Figure 1(c)).
Revised: 6373 , 1 . 0587 , 3 . 5691 , which are obtained by training InsDP-FedAvg FL models for 3 rounds with noise level σ = 15 , 10 , 8 , 5 , 4 , 3 , 2 , 1 , respectively (Figure 1(c)).

Original: For CIFAR-10, we set privacy cost (cid:15) to be 0 .
Revised: For CIFAR-10, we set privacy cost (cid:15) to be 0 .

Original: 6978 , which are obtained by training
Revised: 6978 , which are obtained by training

Original: InsDP-FedAvg FL models for one round with noise level σ = 8 , 7 , 6 , 5 , 4 , 3 , 2 , 1 , respectively (Figure 1(d)).
Revised: InsDP-FedAvg FL models for one round with noise level σ = 8 , 7 , 6 , 5 , 4 , 3 , 2 , 1 , respectively (Figure 1(d)).

Revised: The clean accuracy (average over 1000 runs) of InsDP-FedAvg under non-DP training ( (cid:15) = ∞ ) and DP training (varying (cid:15) ) on MNIST and CIFAR-10 are reported in Table.

Revised: 4 and

Revised: Table.

Revised: 5 respectively.

Revised: [Table]

Original:  With the aim to study certiﬁed attack cost under different number of adversarial instances k , for
Revised: [Table] With the aim to study certiﬁed attack cost under different number of adversarial instances k [Table] for

Original: MNIST, we set the noise level σ to be 10.
Revised: MNIST [Table] we set the noise level σ to be 10.

Original: When k = 0 , after training InsDP-FedAvg for T = 4 ,rounds, we obtain FL models with privacy guarantee (cid:15) = 0 .
Revised: When k = 0 , after training InsDP-FedAvg for T = 4 ,rounds [Table] we obtain FL models with privacy guarantee (cid:15) = 0 .

Original: 304 and clean accuracy (average over M runs) 96 .
Revised: 304 and clean accuracy (average over M runs) 96 .

Original: 93% (Figure 8(a)(b)).
Revised: 93% (Figure 8(a)(b)).

Original: For CIFAR-10, we set the noise level σ to be 8 . 0 .
Revised: For CIFAR-10, we set the noise level σ to be 8 . 0 .

Original: After training InsDP-FedAvg for one round under k = 0 , we obtain FL models with privacy guarantee (cid:15) = 0 .
Revised: After training InsDP-FedAvg for one round under k = 0 , we obtain FL models with privacy guarantee (cid:15) = 0 .

Original: 3158 and clean accuracy 61 .
Revised: 3158 and clean accuracy 61 .

Original: 78% (Figure 4(a)(b)).
Revised: 78% (Figure 4(a)(b)).

Original: In order to study the empirical attack cost and certiﬁed attack cost lower bound under different instance-level DP guarantee, we set the privacy guarantee (cid:15) to be 0 .
Revised: In order to study the empirical attack cost and certiﬁed attack cost lower bound under different instance-level DP guarantee, we set the privacy guarantee (cid:15) to be 0 .

Original: 2202 , 0 . 2096 , 0 .for
Revised: 2202 , 0 . 2096 , 0 .for

Original: MNIST, which are obtained by training InsDP-FedAvg FL models for 6 rounds under noise level σ = 5 , 8 , 10 , 13 , 15 , 18 , 20 , respectively, and the clean accuracy for the corresponding models are 99 .
Revised: MNIST, which are obtained by training InsDP-FedAvg FL models for 6 rounds under noise level σ = 5 , 8 , 10 , 13 , 15 , 18 , 20 , respectively, and the clean accuracy for the corresponding models are 99 .

Original: 60% (Figure 8 (c)(d)).
Revised: 60% (Figure 8 (c)(d)).

Original: For
Revised: For

Original: CIFAR-10, we set the privacy guarantee (cid:15) to be 1 . 261 , 0 . 9146 , 0 .
Revised: CIFAR-10, we set the privacy guarantee (cid:15) to be 1 . 261 , 0 . 9146 , 0 .

Original: 7187 , 0 . 5923 , 0 . 5038 , 0 . 4385 , which are obtained by training InsDP-FedAvg FL models for 2 rounds under noise level σ = 3 , 4 , 5 , 6 , 7 , 8 , respectively, and the clean accuracy for the corresponding models are 84 .
Revised: 7187 , 0 . 5923 , 0 . 5038 , 0 . 4385 , which are obtained by training InsDP-FedAvg FL models for 2 rounds under noise level σ = 3 , 4 , 5 , 6 , 7 , 8 , respectively, and the clean accuracy for the corresponding models are 84 .

Original: 65% (Figure 4 (c)(d)).
Revised: 65% (Figure 4 (c)(d)).

Original: With the intention of exploring the upper bound for k given τ under different instance-level DP guarantee, for MNIST, we set noise level σ to be 5 , 8 , 10 , 13 , 20 , respectively, to obtain instance-DP
Revised: With the intention of exploring the upper bound for k given τ under different instance-level DP guarantee, for MNIST, we set noise level σ to be 5 , 8 , 10 , 13 , 20 , respectively, to obtain instance-DP

Original: FL models after 10 rounds with privacy guarantee (cid:15) = 0 .
Revised: FL models after 10 rounds with privacy guarantee (cid:15) = 0 .

Original: 2179 and clean accuracy 99 .
Revised: 2179 and clean accuracy 99 .

Original: 72% (Figure 5(c)).
Revised: 72% (Figure 5(c)).

Original: For CIFAR-10, we set noise level σ to be 3 , 4 , 5 , 6 , 7 , 8 and train InsDP-FedAvg for T = 3 rounds, to obtain FL models with privacy guarantee (cid:15)
Revised: For CIFAR-10, we set noise level σ to be 3 , 4 , 5 , 6 , 7 , 8 and train InsDP-FedAvg for T = 3 rounds, to obtain FL models with privacy guarantee (cid:15)

Original: = 1 . 5365 , 1 . 1162 , 0 .
Revised: = 1 . 5365 , 1 . 1162 , 0 .

Original: 5361 and clean accuracy 84 .
Revised: 5361 and clean accuracy 84 .

Original: 75% (Figure 5(d)).
Revised: 75% (Figure 5(d)).

Original: A.3. A DDITIONAL I MPLEMENTATION D ETAILS
Revised: A.3. A DDITIONAL I MPLEMENTATION D ETAILS

Original: left distributed backdoor

Original: CIFAR

Original: (Threat
Revised: (Threat

Original: Models)
Revised: Models)

Original: For the attacks against
Revised: For the attacks against

Original: UserDP-FedAvg , by default, the local poison fraction α = 100% , and the scale factor γ = 50 .
Revised: UserDP-FedAvg , by default, the local poison fraction α = 100% , and the scale factor γ = 50 .

Original: We use same parameters setups for all k attackers.
Revised: We use same parameters setups for all k attackers.

Original: In terms of label ﬂipping attacks, the attackers swap the label of images in source class (digit 1 for MNIST; bird for
Revised: In terms of label ﬂipping attacks, the attackers swap the label of images in source class (digit 1 for MNIST; bird for

Original: CIFAR-10) into the target label (digit 0 for MNIST; airplane for CIFAR-10).
Revised: CIFAR-10) into the target label (digit 0 for MNIST; airplane for CIFAR-10).

Original: In terms of backdoor attacks in MNIST and CIFAR-10, the attackers add a backdoor pattern, as shown in Figure 6 (left), in images and swap the label of any sample with such pattern into the target label (digit 0 for MNIST; airplane for CIFAR-10).
Revised: In terms of backdoor attacks in MNIST and CIFAR-10, the attackers add a backdoor pattern, as shown in Figure 6 (left), in images and swap the label of any sample with such pattern into the target label (digit 0 for MNIST; airplane for CIFAR-10).

Original: In terms of distributed backdoor attacks,
Revised: In terms of distributed backdoor attacks,

Original: Figure 6 (right) shows an example when the triangle pattern is evenly decomposed into k = 4 parts, and they are used as the distributed patterns for k = 4 attackers respectively.
Revised: Figure 6 (right) shows an example when the triangle pattern is evenly decomposed into k = 4 parts, and they are used as the distributed patterns for k = 4 attackers respectively.

Original: For the cases where there are more or fewer distributed attackers, the similar decomposition strategy is adopted.
Revised: For the cases where there are more or fewer distributed attackers, the similar decomposition strategy is adopted.

Original: For the attacks against InsDP-FedAvg , the same target classes and backdoor patterns are used as
Revised: For the attacks against InsDP-FedAvg , the same target classes and backdoor patterns are used as

Original: UserDP-FedAvg .
Revised: UserDP-FedAvg .

Original: The parameters setups are the same for all k poisoned instances.
Revised: The parameters setups are the same for all k poisoned instances.

Original: (Robustness Certiﬁcation)
Revised: (Robustness Certiﬁcation)

Original: We certiﬁed 2115/2000 test samples from the MNIST/CIFAR-10 test sets.
Revised: We certiﬁed 2115/2000/1122 test samples from the MNIST/CIFAR10/Sent140 test sets.

Original: In Theorem 3 and Corollary 1 that are related to certiﬁed attack cost, ¯ C speciﬁes the range of C ( · ) .
Revised: In Theorem 3 and Corollary 1 that are related to certiﬁed attack cost, ¯ C speciﬁes the range of C ( · ) .

Original: In the implementation, ¯ C is set to be larger than the maximum empirical attack cost evaluated on the test sets (see Table 1 for details).
Revised: In the implementation, ¯ C is set to be larger than the maximum empirical attack cost evaluated on the test sets (see Table 1 for details).

Original: For each dataset, we use the same ¯ C for cost function C deﬁned in Example 1 and Example 2.
Revised: For each dataset, we use the same ¯ C for cost function C deﬁned in Example 1 and Example 2.

Original: When using Monte-Carlo sampling, we run M = 1000 times for certiﬁed accuracy, and M = 100 times for certiﬁed attack cost in all experiments.
Revised: When using Monte-Carlo sampling, we run M = 1000 times for certiﬁed accuracy, and M = 100 times for certiﬁed attack cost in all experiments.

Original: (Machines)
Revised: (Machines)

Original: We simulate the federated learning setup (1 server and N users) on a Linux machine with Intel® Xeon® Gold 6132 CPUs and 8 NVidia® 1080Ti GPUs.
Revised: We simulate the federated learning setup (1 server and N users) on a Linux machine with Intel® Xeon® Gold 6132 CPUs and 8 NVidia® 1080Ti GPUs.

Original: (Libraries)
Revised: (Libraries)

Original: All code is implemented in Pytorch (Paszke et al., 2019).
Revised: All code is implemented in Pytorch (Paszke et al., 2019).

Original: Please see the submitted code for full details.
Revised: Please see the submitted code for full details.

Original: A.3. C ERTIFIED A CCURACY WITH C ONFIDENCE L EVEL
Revised: A.3. C ERTIFIED A CCURACY WITH C ONFIDENCE L EVEL

Original: Here we present the certiﬁed accuracy with conﬁdence level.
Revised: Here we present the certiﬁed accuracy with conﬁdence level.

Original: We use Hoeffding’s inequality (Hoeffding, 1994) to calibrates the empirical estimation with one-sided error tolerance ψ , i.e., one-sided conﬁdence level 1 − ψ .
Revised: We use Hoeffding’s inequality (Hoeffding, 1994) to calibrates the empirical estimation with one-sided error tolerance ψ , i.e., one-sided conﬁdence level 1 − ψ .

Original: We ﬁrst use Monte-Carlo sampling by running the private FL algorithms for M times, with class conﬁdence f sc = f c ( M ( D ) , x ) for class c each time.
Revised: We ﬁrst use Monte-Carlo sampling by running the private FL algorithms for M times, with class conﬁdence f sc = f c ( M ( D ) , x ) for class c each time.

Original: We denote the empirical estimation as (cid:101) F c ( M ( D ) , x ) =
Revised: We denote the empirical estimation as (cid:101) F c ( M ( D ) , x ) =

Original: 1 M (cid:80) Ms =1 f sc .
Revised: 1 M (cid:80) Ms =1 f sc .

Original: For a test input x , suppose A , B ∈ [ C ] satisfy
Revised: For a test input x , suppose A , B ∈ [ C ] satisfy

Original: A = arg max c ∈ [ C ] (cid:101) F c ( M ( D ) , x ) and B = arg max c ∈ [ C ]: c (cid:54) =
Revised: A = arg max c ∈ [ C ] (cid:101) F c ( M ( D ) , x ) and B = arg max c ∈ [ C ]: c (cid:54) =

Original: A (cid:101) F c
Revised: A (cid:101) F c

Original: ( M ( D ) , x ) .
Revised: ( M ( D ) , x ) .

Original: For a given error tolerance ψ , we use Hoeffding’s inequality to compute a lower bound F A ( M ( D ) , x ) on the class conﬁdence F A ( M ( D ) , x ) and a upper bound F B ( M ( D ) , x ) on the class conﬁdence F B ( M ( D ) , x ) according to
Revised: For a given error tolerance ψ , we use Hoeffding’s inequality to compute a lower bound F A ( M ( D ) , x ) on the class conﬁdence F A ( M ( D ) , x ) and a upper bound F B ( M ( D ) , x ) on the class conﬁdence F B ( M ( D ) , x ) according to

Original: [Equation]
Revised: [Equation]

Original: F A ( M ( D ) , x ) and F B ( M ( D ) , x ) are used as the expected class conﬁdences for the evaluation of
Revised: F A ( M ( D ) , x ) and F B ( M ( D ) , x ) are used as the expected class conﬁdences for the evaluation of

Original: Theorem 2.
Revised: Theorem 2.

Original: We use ψ = 0 .
Revised: We use ψ = 0 .

Original: 01 and M = 1000 for all experiments.
Revised: 01 and M = 1000 for all experiments.

Original: As shown in Figure 7, we can observe the same tradeoff between (cid:15) and certiﬁed accuracy as we discussed in Figure 1.
Revised: As shown in Figure 7, we can observe the same tradeoff between (cid:15) and certiﬁed accuracy as we discussed in Figure 1.

Original: In general, the K in Figure 7 is smaller than the K in Figure 1 because we calibrate the empirical estimation according to Eq.
Revised: In general, the K in Figure 7 is smaller than the K in Figure 1 because we calibrate the empirical estimation according to Eq.

Original: (6), and the class conﬁdence gap between topand top-2 class is narrowed.
Revised: (6), and the class conﬁdence and top-2 class is narrowed.

Original: A.3. A DDITIONAL R OBUSTNESS E VALUATION OF I NSTANCE - LEVEL DPFL
Revised: A.3. A DDITIONAL R OBUSTNESS E VALUATION OF I NSTANCE - LEVEL DPFL

Original: Here we report the robustness evaluation of instance-level DPFL on MNIST.
Revised: Here we report the robustness evaluation of instance-level DPFL on MNIST.

Original: As shown in Figure 8, the results on MNIST are similar to the results on CIFAR-10 in Figure 4.
Revised: As shown in Figure 8, the results on MNIST are similar to the results on CIFAR-10 in Figure 4.

Original: A.3. R OBUSTNESS E VALUATION ON 10- CLASS C LASSIFICATION
Revised: A.3. R OBUSTNESS E VALUATION ON 10- CLASS C LASSIFICATION

Original: Here we report the robustness evaluation of user-level DPFL under backdoor attacks on 10-class classiﬁcation problem.
Revised: Here we report the robustness evaluation of user-level DPFL under backdoor attacks on 10-class classiﬁcation problem.

Original: Figure 10 presents the certiﬁed accuracy under different (cid:15) .
Revised: Figure 10 presents the certiﬁed accuracy under different (cid:15) .

Original: We can observe the tradeoff between (cid:15) and certiﬁed accuracy on MNIST.
Revised: We can observe the tradeoff between (cid:15) and certiﬁed accuracy on MNIST.

Original: On CIFAR-10, larger k can be certiﬁed with smaller (cid:15) .
Revised: On CIFAR-10, larger k can be certiﬁed with smaller (cid:15) .

Original: The certiﬁed K is relatively small because we set large (cid:15) to preserve a reasonable accuracy for 10-class classiﬁcation.
Revised: The certiﬁed K is relatively small because we set large (cid:15) to preserve a reasonable accuracy for 10-class classiﬁcation.

Original: Our results can inspire advanced DP mechanisms that provide tighter privacy guarantee (i.e., smaller (cid:15) ) while achieving similar level of accuracy.
Revised: Our results can inspire advanced DP mechanisms that provide tighter privacy guarantee (i e., smaller (cid:15) ) while achieving similar level of accuracy.

Original: In terms of certiﬁed attack cost, as shown in Figure 9 and 11, the trends are similar to the 2-class results in Figure 2,and 5.
Revised: In terms of certiﬁed attack cost, as shown in Figure 9 and 11, the trends are similar to the 2-class results in Figure 2,and 5.

Original: A.3. DP BOUND COMPARISON BETWEEN I NS DP-F ED SGD AND D OPAMINE
Revised: A.3. DP BOUND COMPARISON BETWEEN I NS DP-F ED SGD AND D OPAMINE

Original: Here we compare Dopamine to our InsDP-FedSGD , both of which are proposed for FedSGD.
Revised: Here we compare Dopamine to our InsDP-FedSGD , both of which are proposed for FedSGD.

Original: Under the same noise level ( σ = 3 . 0 ), clipping threshold ( S = 1 . 5 ), user sampling probability ( m/N = 20 / 30 ), and batch sampling probability ( 0 . 4 ) settings, both algorithms achieve about 92 % accuracy on MNIST (10 classes).
Revised: Under the same noise level ( σ = 3 . 0 ), clipping threshold ( S = 1 . 5 ), user sampling probability ( m/N = 20 / 30 ), and batch sampling probability ( 0 . 4 ) settings, both algorithms achieve about 92 % accuracy on MNIST (10 classes).

Original: The Figure 12 shows the results of privacy guarantee estimation over training rounds, which demonstrates that our method achieves tighter privacy certiﬁcation.
Revised: The Figure 12 shows the results of privacy guarantee estimation over training rounds, which demonstrates that our method achieves tighter privacy certiﬁcation.

Original: For instance, at round 200, our method ( (cid:15) = 1 . 4029 ) achieves a much tighter privacy guarantee than
Revised: For instance, at round 200, our method ( (cid:15) = 1 . 4029 ) achieves a much tighter privacy guarantee than

Original: Dopamine ( (cid:15) = 2 . 1303 ).
Revised: Dopamine ( (cid:15) = 2 . 1303 ).

Revised: A.3. C ERTIFIED A CCURACY OF U SER DP-F ED A VG ON S ENT

Revised: For Sent140, we set (cid:15) to be 0 .

Revised: 7382 , 1 . 7151 , which are obtained by training

Revised: UserDP-FedAvg FL model for three rounds with noise level σ = 4 , 3 , 2 , 1 . 5 , 1 , respectively

Revised: As shown in Figure 13, the largest k can be certiﬁed when (cid:15) is around 0 . 2247 in Sent140, which also veriﬁes the tradeoff between (cid:15) and certiﬁed accuracy as we observed in image datasets.

Original: [Figure]
Revised: [Figure]

Revised: [Figure]

Revised: k

Revised: [Figure]

Revised: c e r t i f i e d a cc u r a c y

Revised: [Figure]

Revised: = 0.

Revised: = 0.

Original: A. P ROOFS OF C ERTIFIEDR OBUSTNESS A NALYSIS
Revised: = [Figure] = 1. A.3. C ERTIFIED A CCURACY C OMPARISON OF DIFFERENT USER - LEVEL DPFL ALGORITHMS

Revised: In this section, we include two more user-level DPFL works (McMahan et al., 2018; Geyer et al., 2017) to study certiﬁed accuracy with a total of four different DPFL methods given the same privacy budget (cid:15) .

Revised: Since all our proposed robustness certiﬁcations are agnostic to DPFL algorithms, i.e., certiﬁcations hold no matter how ( (cid:15), δ ) is achieved, we can empirically compare the certiﬁed results of different DPFL algorithms.

Revised: Speciﬁcally, we consider the following four DPFL algorithms:

Revised: • ﬂat clipping ( UserDP-FedAvg ) clips the concatenation of all the layers of model update with the L2 norm threshold S .

Revised: • per-layer clipping (McMahan et al., 2018) clips each layer of model update with the L norm threshold S .

Revised: • ﬂat median.

Revised: clipping (Geyer et al., 2017) use the median of norms of clients’ model updates as threshold S for ﬂat clipping.

Revised: per-layer median clipping (Geyer et al., 2017) use the median of each layer’s norms of clients’ model updates as threshold S for per-layer clipping.

Revised: For MNIST (CIFAR-10), we set (cid:15) to be 0.6319 (0.5346) which is obtained by training all DPFL algorithms with the same noise level σ = 2 . 3 ( σ = 3 . 0 ) for same number of rounds.

Revised: For ﬂat clipping and per-layer clipping, we set S = 0 .

Revised: 7 ( S = 1 ) on MNIST (CIFAR-10).

Revised: Except for local epoch

Revised: E = 1 1 , other FL parameters setups are the same as in Table 1.

Revised: [Figure]

Revised: [Figure]

Revised: As shown in Figure 14 and Figure 15, on MNIST, the ﬂat clipping is able to certify the largest number of adversaries k ; while on CIFAR-10, the median clipping certiﬁes the largest k instead.

Revised: Moreover, on both MNIST and CIFAR-10, ﬂat clipping and per-layer clipping with the same S lead to different certiﬁcation results, while the results of ﬂat median clipping and per-layer median clipping are nearly identical.

Revised: This is because even under the same privacy protection (cid:15) , different DPFL algorithms M produce trained models M ( D ) with different model performance, thus leading to different certiﬁed robustness.

Revised: Speciﬁcally, in Theorem 1, given the same (cid:15) and x , F A ( M ( D ) , x ) and F B ( M ( D ) , x ) vary for different DPFL trained models M ( D ) , thus causing different certiﬁed K .

Revised: The above interesting results indicate that our proposed robustness certiﬁcations can serve as new metrics to benchmark the robustness of different DPFL algorithms, which can potentially motivate the investigation for better DPFL algorithms (i.e., different types of noise, clipping methods, subsampling strategies, or even different FL training protocols).

Revised: We believe these analyses can provide new and important insights to the FL community.

Revised: A. P ROOFS OF C ERTIFIED R OBUSTNESS A NALYSIS

Original: We restate our Lemma 1 here.
Revised: We restate our Lemma 1 here.

Original: Lemma 1 (Group DP) .
Revised: Lemma 1 (Group DP) .

Original: For mechanism M that satisﬁes ( (cid:15), δ ) -DP, it satisﬁes ( k(cid:15), 1 − e k(cid:15) [Equation] δ ) -DP for groups of size k .
Revised: For mechanism M that satisﬁes ( (cid:15), δ ) -DP, it satisﬁes ( k(cid:15), 1 − e k(cid:15) 1 − e (cid:15) δ ) -DP for groups of size k .

Original: That is, for any d, d (cid:48)
Revised: That is, for any d, d (cid:48)

Original: ∈ D that differ by k individuals, and any E ⊆
Revised: ∈ D that differ by k individuals, and any E ⊆

Original: Θ it holds that
Revised: Θ it holds that

Revised: Pr[ M ( d

Revised: [Equation]

Revised: E

Original: [Equation]
Revised: [Equation]

Original: Proof.
Revised: Proof.

Original: We denote d as d 0 , d (cid:48) as d k .
Revised: We denote d as d 0 , d (cid:48) as d k .

Original: d i differ i individuals with d 0 .
Revised: d i differ i individuals with d 0 .

Original: For any i ∈ [1 , k ] , d i and d i −differ by one individual, thus
Revised: For any i ∈ [1 , k ] , d i and d i −differ by one individual, thus

Original: [Equation]
Revised: [Equation]

Original: By iteratively applying Eq.
Revised: By iteratively applying Eq.

Original: (7) k times, we have
Revised: (7) k times, we have

Original: [Equation]
Revised: [Equation]

Original: Before we prove Theorem 1, we introduce the following lemma: 
Revised: Before we prove Theorem 1, we introduce the following lemma: of noise reasonable and preserve good model utility.

Revised: As more local epoch leads to larger norm of model updates, we set the local epoch as 1 to keep the median norm small.

Original: Lemma 3.
Revised: Lemma 3.

Original: Suppose a randomized mechanism M satisﬁes user-level ( (cid:15), δ ) -DP.
Revised: Suppose a randomized mechanism M satisﬁes user-level ( (cid:15), δ ) -DP.

Original: For two user sets B and B (cid:48) that differ by one user, D and D (cid:48) are the corresponding training datasets.
Revised: For two user sets B and B (cid:48) that differ by one user, D and D (cid:48) are the corresponding training datasets.

Original: For a test input x , for any c ∈ [ C ] , f c ( M ( D ) , x ) ∈ [0 , 1] is the class conﬁdence, then the expected class conﬁdence
Revised: For a test input x , for any c ∈ [ C ] , f c ( M ( D ) , x ) ∈ [0 , 1] is the class conﬁdence, then the expected class conﬁdence

Original: F c ( M ( D ) , x ) :=
Revised: F c ( M ( D ) , x ) :=

Original: E [ f c ( M ( D ) , x )] meets the following property:
Revised: E [ f c ( M ( D ) , x )] meets the following property:

Original: [Equation]
Revised: [Equation]

Original: Proof.
Revised: Proof.

Original: Deﬁne Θ( a ) :
Revised: Deﬁne Θ( a ) :

Original: = { θ : f c ( θ, x ) > a } .
Revised: = { θ : f c ( θ, x ) > a } .

Original: Then
Revised: Then

Original: [Equation]
Revised: [Equation]

Original: We recall Theorem 1.
Revised: We recall Theorem 1.

Original: Theorem 1 (Condition for Certiﬁed Prediction under One Adversarial User) .
Revised: Theorem 1 (Condition for Certiﬁed Prediction under One Adversarial User) .

Original: Suppose a randomized mechanism M satisﬁes user-level ( (cid:15), δ ) -DP.
Revised: Suppose a randomized mechanism M satisﬁes user-level ( (cid:15), δ ) -DP.

Original: For two user sets B and B (cid:48) that differ by one user, let
Revised: For two user sets B and B (cid:48) that differ by one user, let

Original: D and D (cid:48) be the corresponding training datasets.
Revised: D and D (cid:48) be the corresponding training datasets.

Original: For a test input x , suppose A , B ∈ [ C ] satisfy A = arg max 
Revised: For a test input x , suppose A , B ∈ [ C ] satisfy A = arg max c ∈

Original: [Equation] and B = arg [Equation]
Revised: [ C ] F c ( M ( D ) , x ) and B = arg max c ∈ [ C ]: c (cid:54) =

Original:  , then if
Revised: A F c ( M ( D ) , x ) , then if

Original: [Equation]
Revised: [Equation]

Original: it is guaranteed that H ( M ( D (cid:48) ) , x ) =
Revised: it is guaranteed that H ( M ( D (cid:48) ) , x ) =

Original: H ( M ( D [Equation] , x ) = A .
Revised: H ( M ( D ) , x ) = A .

Original: Proof.
Revised: Proof.

Original: According to Lemma 3,
Revised: According to Lemma 3,

Original: [Equation]
Revised: [Equation]

Original: Then
Revised: Then

Original: [Equation]
Revised: [Equation]

Original: Because ofEq
Revised: of

Original: [Equation] [Equation]
Revised: [Equation] [Equation]

Original: (Because of the given condition Eq. 1)
Revised: Because of the given condition Eq. 1)

Original: [Equation]

Original: Because of Eq. 10)

Original: [Equation]
Revised: [Equation]

Original: (cid:48) ) , x ) , which indicates that the prediction of M ( D (cid:48) ) at x is A by deﬁnition.
Revised: which indicates that the prediction of M ( D (cid:48) ) at x is A by deﬁnition.

Original: Before we prove Theorem 2, we introduce the following lemma:
Revised: Before we prove Theorem 2, we introduce the following lemma:

Original: Lemma 4.
Revised: Lemma 4.

Original: Suppose a randomized mechanism M satisﬁes user-level ( (cid:15), δ ) -DP.
Revised: Suppose a randomized mechanism M satisﬁes user-level ( (cid:15), δ ) -DP.

Original: For two user sets
Revised: For two user sets

Original: B and B (cid:48) that differ k users, D and D (cid:48) are the corresponding training datasets.
Revised: B and B (cid:48) that differ k users, D and D (cid:48) are the corresponding training datasets.

Original: For a test input x , for any c ∈ [ C ] , f c ( M ( D ) , x ) ∈ [0 , 1] is the class conﬁdence, then the expected class conﬁdence
Revised: For a test input x , for any c ∈ [ C ] , f c ( M ( D ) , x ) ∈ [0 , 1] is the class conﬁdence, then the expected class conﬁdence

Original: F c ( M ( D ) , x ) :=
Revised: F c ( M ( D ) , x ) :=

Original: E [ f c ( M ( D ) , x )] meets the following property:
Revised: E [ f c ( M ( D ) , x )] meets the following property:

Original: [Equation]
Revised: [Equation]

Original: Proof.
Revised: Proof.

Original: Deﬁne Θ( a ) :
Revised: Deﬁne Θ( a ) :

Original: = { θ : f c ( θ, x ) > a } .
Revised: = { θ : f c ( θ, x ) > a } .

Original: Then
Revised: Then

Original: [Equation]
Revised: [Equation]

Original: (Because of Group DP property in Lemma 1)
Revised: Because of Group DP property in Lemma 1)

Original: [Equation]
Revised: [Equation]

Original: We recall Theorem 2.
Revised: We recall Theorem 2.

Original: Theorem 2 (Upper Bound of k for Certiﬁed Prediction) .
Revised: Theorem 2 (Upper Bound of k for Certiﬁed Prediction) .

Original: Suppose a randomized mechanism M satisﬁes user-level ( (cid:15), δ ) -DP.
Revised: Suppose a randomized mechanism M satisﬁes user-level ( (cid:15), δ ) -DP.

Original: For two user sets B and B (cid:48) that differ by k users, let D and
Revised: For two user sets B and B (cid:48) that differ by k users, let D and

Original: D (cid:48) be the corresponding training datasets.
Revised: D (cid:48) be the corresponding training datasets.

Original: For a test input x , suppose A , B ∈ [ C ] satisfy A = arg max c ∈
Revised: For a test input x , suppose A , B ∈ [ C ] satisfy A = arg max c ∈

Original: [ C ] F c ( M ( D ) , x ) and B = arg max c ∈ [ C ]: c (cid:54) =
Revised: [ C ] F c ( M ( D ) , x ) and B = arg max c ∈ [ C ]: c (cid:54) =

Original: A F c ( M ( D ) , x ) , then H ( M ( D (cid:48) ) , x ) =
Revised: A F c ( M ( D ) , x ) , then H ( M ( D (cid:48) ) , x ) =

Original: H ( M ( D ) , x ) =
Revised: H ( M ( D ) , x ) =

Original: A , ∀ k < K where K is the certiﬁed number of adversarial users:
Revised: A , ∀ k < K where K is the certiﬁed number of adversarial users:

Original: [Equation]
Revised: [Equation]

Original: Proof.
Revised: Proof.

Original: According to Lemma
Revised: According to Lemma

Original: [Equation]
Revised: [Equation]

Original: we have
Revised: we have

Original: [Equation]
Revised: [Equation]

Original: We can re-write the given condition k < K according to Eq
Revised: We can re-write the given condition k < K according to Eq

Original: [Equation]
Revised: [Equation]

Original: (2) as
Revised: (2) as

Original: [Equation]
Revised: [Equation]

Original: Because of Eq
Revised: Then

Original: [Equation]
Revised: [Equation]

Revised: (Because of Eq. 12)

Original: [Equation]
Revised: [Equation]

Original: Because of the given condition Eq.14)
Revised: Because of the given condition Eq.14)

Original: [Equation]
Revised: [Equation]

Original: Eq
Revised: of

Original: [Equation] [Equation]
Revised: [Equation] [Equation]

Original: F B ( M ( D (cid:48) ) , x ) , which indicates that the prediction of M ( D (cid:48) ) at x is A by deﬁnition.
Revised: which indicates that the prediction of M ( D (cid:48) ) at x is A by deﬁnition.

Original: We recall Theorem 3.
Revised: We recall Theorem 3.

Original: Theorem 3 (Attack Cost with k Attackers) .
Revised: Theorem 3 (Attack Cost with k Attackers) .

Original: Suppose a randomized mechanism M satisﬁes user-level ( (cid:15), δ ) -DP.
Revised: Suppose a randomized mechanism M satisﬁes user-level ( (cid:15), δ ) -DP.

Original: For two user sets B and B (cid:48) that differ k users, D and D (cid:48) are the corresponding training datasets.
Revised: For two user sets B and B (cid:48) that differ k users, D and D (cid:48) are the corresponding training datasets.

Original: Let J ( D ) be the expected attack cost where | C ( · ) | ≤ ¯ C .
Revised: Let J ( D ) be the expected attack cost where | C ( · ) | ≤ ¯ C .

Original: Then,
Revised: Then,

Original: [Equation]
Revised: [Equation]

Revised: [Table]

Revised: [Equation]

Revised: [Table]

Revised: [Equation]

Revised: [Table]

Revised: [Equation]

Revised: [Table]

Revised: [Equation]

Revised: [Table]

Revised: [Equation]

Revised: [Table]

Revised: [Equation]

Revised: [Table]

Revised: [Equation]

Revised: [Table]

Original: Proof.
Revised: Proof.

Original: We ﬁrst consider C ( · ) ≥ 0 .
Revised: We ﬁrst consider C ( · ) ≥ 0 .

Original: Deﬁne Θ( a ) =
Revised: Deﬁne Θ( a ) =

Original: { θ : C ( θ ) > a } .
Revised: { θ : C ( θ ) > a } .

Original: [Equation]
Revised: [Equation]

Original: (Because of Group DP property in Lemma 1)
Revised: (Because of Group DP property in Lemma 1)

Original: [Equation]
Revised: [Equation]

Original: i.e.,
Revised: i.e.,

Original: [Equation]
Revised: [Equation]

Original: Switch the role of D and D (cid:48) , we have
Revised: Switch the role of D and D (cid:48) , we have

Original: [Equation]
Revised: [Equation]

Original: Also note that 0 ≤ J ( D (cid:48) ) ≤ ¯ C trivially holds due to 0 ≤ C ( · ) ≤ ¯ C , thus 
Revised: Also note that 0 ≤ J ( D (cid:48) ) ≤ ¯ C trivially holds due to 0 ≤ C ( · ) ≤ ¯ C , thus min {

Original: [Equation]
Revised: [Equation]

Original: [Equation]

Original: Next we consider C ( · ) ≤ 0 .
Revised: Next we consider C ( · ) ≤ 0 .

Original: Deﬁne Θ( a ) =
Revised: Deﬁne Θ( a ) =

Original: { θ : C ( θ ) < a } .
Revised: { θ : C ( θ ) < a } .

Original: [Equation]
Revised: [Equation]

Original: (Because of Group DP property in Lemma 1)
Revised: Because of Group DP property in Lemma 1)

Original: [Equation]
Revised: [Equation]

Original: i.e.,
Revised: i.e.,

Original: [Equation]
Revised: [Equation]

Original: Switch the role of D and D (cid:48) , we have
Revised: Switch the role of D and D (cid:48) , we have

Original: [Equation]
Revised: [Equation]

Original: Also note that − ¯ C ≤ J ( D (cid:48) ) ≤ 0 trivially holds due to − ¯ C ≤ C ( · ) ≤ 0 , thus
Revised: Also note that − ¯ C ≤ J ( D (cid:48) ) ≤ 0 trivially holds due to − ¯ C ≤ C ( · ) ≤ 0 , thus

Original: [Equation]
Revised: [Equation]

Original: We recall Corollary 1.
Revised: We recall Corollary 1.

Original: Corollary 1 (Lower Bound of k Given τ ) .
Revised: Corollary 1 (Lower Bound of k Given τ ) .

Original: Suppose a randomized mechanism M satisﬁes user- level ( (cid:15), δ ) -DP.
Revised: Suppose a randomized mechanism M satisﬁes user- level ( (cid:15), δ ) -DP.

Original: Let attack cost function be C , the expected attack cost be J ( · ) .
Revised: Let attack cost function be C , the expected attack cost be J ( · ) .

Original: In order to achieve
Revised: In order to achieve

Original: J ( D (cid:48) ) ≤ 1 τ J ( D ) for τ ≥ 

Revised: J ( D (cid:48) ) ≤ 1 τ J ( D ) for τ ≥

Original: when 0 ≤ C ( · ) ≤ ¯ C , or achieve J ( D (cid:48) ) ≤ τJ ( D ) for 1 ≤ τ ≤ − ¯ C J
Revised: when 0 ≤ C ( · ) ≤ ¯ C , or achieve J ( D (cid:48) ) ≤ τJ ( D ) for 1 ≤ τ ≤ − ¯ C J

Original: ( D ) when − ¯ C ≤ C ( · ) ≤ 0 , the number of adversarial users should satisfy:
Revised: ( D ) when − ¯ C ≤ C ( · ) ≤ 0 , the number of adversarial users should satisfy:

Original: [Equation]
Revised: [Equation]

Original: or
Revised: respectively.

Original: [Equation]
Revised: [Equation]

Original: respectively.

Original: Proof.
Revised: Proof.

Original: We ﬁrst consider C ( · ) ≥ 0 .
Revised: We ﬁrst consider C ( · ) ≥ 0 .

Original: According to the lower bound in Theorem 3, when B (cid:48) and
Revised: According to the lower bound in Theorem 3, when B (cid:48) and

Original: B differ k users, J ( D (cid:48) )
Revised: B differ k users, J ( D (cid:48) )

Original: ≥ e − k(cid:15)
Revised: ≥ e − k(cid:15)

Original: J ( D ) − 1 − e − k(cid:15) e (cid:15) − 1 δ ¯ C .
Revised: J ( D ) − 1 − e − k(cid:15) e (cid:15) − 1 δ ¯ C .

Original: Since we require J ( D (cid:48) ) ≤ 1 τ J ( D ) , then e − k(cid:15)
Revised: Since we require J ( D (cid:48) ) ≤ 1 τ J ( D ) , then e − k(cid:15)

Original: J ( D ) − 1 − e − k(cid:15) e (cid:15) − 1 δ ¯ C ≤ 1 τ J ( D ) .
Revised: J ( D ) − 1 − e − k(cid:15) e (cid:15) − 1 δ ¯ C ≤ 1 τ J ( D ) .

Original: Rearranging gives the result.
Revised: Rearranging gives the result.

Original: Next, we consider C ( · ) ≤ 0 .
Revised: Next, we consider C ( · ) ≤ 0 .

Original: According to the lower bound in Theorem 3, when B (cid:48) and B differ k users, J ( D (cid:48) ) ≥ e k(cid:15) J ( D ) − e k(cid:15) − 1 e (cid:15) − 1 δ ¯ C .
Revised: According to the lower bound in Theorem 3, when B (cid:48) and B differ k users, J ( D (cid:48) ) ≥ e k(cid:15) J ( D ) − e k(cid:15) − 1 e (cid:15) − 1 δ ¯ C .

Original: Since we require J ( D (cid:48) ) ≤ τJ ( D ) , then e k(cid:15)
Revised: Since we require J ( D (cid:48) ) ≤ τJ ( D ) , then e k(cid:15)

Original: J
Revised: J

Original: ( D ) − e k(cid:15) − 1 e (cid:15) − 1 δ ¯ C ≤ τJ ( D ) .
Revised: ( D ) − e k(cid:15) − 1 e (cid:15) − 1 δ ¯ C ≤ τJ ( D ) .

Original: Rearranging gives the result.
Revised: Rearranging gives the result.

Original: We note that all the above robustness certiﬁcation related proofs are built upon the user-level ( (cid:15), δ ) -DP property and the Group DP property.
Revised: We note that all the above robustness certiﬁcation related proofs are built upon the user-level ( (cid:15), δ ) -DP property and the Group DP property.

Original: According to Deﬁnition 2 and Deﬁnition 3, the deﬁnition of user-level DP and instance-level DP are both induced from DP (Deﬁnition 1) despite the different deﬁnitions of adjacent datasets.
Revised: According to Deﬁnition 2 and Deﬁnition 3, the deﬁnition of user-level DP and instance-level DP are both induced from DP (Deﬁnition 1) despite the different deﬁnitions of adjacent datasets.

Original: By applying the deﬁnition of instance-level ( (cid:15), δ ) -DP
Revised: By applying the deﬁnition of instance-level ( (cid:15), δ ) -DP

Original: and following the proof steps of Theorem 1, 2, 3 and Corollary 1, we can derive the similar theoretical conclusions for instance-level DP, leading to Theorem 5 to achieve the certiﬁably robsut FL for free given the DP property.
Revised: and following the proof steps of Theorem 1, 2, 3 and Corollary 1, we can derive the similar theoretical conclusions for instance-level DP, leading to Theorem 5 to achieve the certiﬁably robsut FL for free given the DP property.