Venue: ICLR 2022 Conference

Reviews:

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: The paper states that the model produced by differentially private federated learning to be already certified against poisoning attacks. 

main_review: 
Strength: the paper proposes criteria for robustness of FL, evaluates the theoretical results using MNIST and CIFAR datasets.


Weakness: I can't entirely agree with the paper's message that the certification comes for free and instead suggest that this certification comes at a relatively high price requiring differential privacy. For example, [1] uses gradient shaping without full DP. However, I would agree that in cases where differential privacy is inevitable, the conclusions seem helpful and could further promote the use of DP.

Furthermore, the paper does not evaluate the effects of certification on performance reduction for the benign data. Specifically, I am surprised by an extremely low budget in CIFAR for the presented experiments, i.e. epsilon less than 1 and an extreme amount of noise, i.e. std around 10.  Clarifying further A.3.2 that has only few rounds would be helpful.The provided certification might not provide much utility, especially for diverse users already disproportionally affected by less strict DP budget.

Evaluating the proposed method on the Language Modeling task might further strengthen the submission to provide a realistic privacy training regime. 



[1] Sanghyun Hong, Varun Chandrasekaran, Yig ̆itcan Kaya, Tudor Dumitras ̧, and Nicolas Papernot. On the effectiveness of mitigating data poisoning attacks with gradient shaping. arXiv preprint arXiv:2002.11497, 2020. 
summary_of_the_review: The paper proposes a good analysis of provided robustness guarantees, but further clarification of experiments would be very helpful.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: This paper studies differentially private federated learning and its intrinsic robustness against data poisoning attacks. Theoretically, the authors build two definitions for certified robustness against data poisoning attacks, draw the connection with user-level and instance-level differential privacy. The key proof is based on the definition of individual privacy and group privacy. Empirically, the authors verify the correctness of the bounds by performing real attacks. I think the main contribution is to establish the robustness bound.
main_review: Update: Thank you for your response. I read the authors' response and other reviewers' comments. My second question has been well addressed. However, as other reviewers suggested, the contributions of this paper and comparisons with prior work should be explained more clearly. I would not consider the main contribution as the theory part, because given the group privacy property of DP, this result is not very surprising. I would suggest the authors highlight the empirical contributions and federated learning setting.


Strengths:
1. I think this paper provides an interesting perspective for robust machine learning. 
2. The empirical evaluations are solid.
3. the proposed algorithms are practically useful


Weaknesses:
1. I think the results are correct but not surprising. There have been many papers (theoretically and empirically) showing either robust algorithms can be made private easily and private algorithms provide intrinsic robustness. 

2. It would be better if the authors could provide some theoretical/empirical intuition for the utility. It is known that both robust learning algorithms and private algorithms would cause the performance drop. It would be nice if the authors could provide non-private(epsilon=infty) clean accuracy as a comparison.


summary_of_the_review: This paper is overall well-written and provides new insight on intersection of robustness and privacy.
review_title: null

final_score: rating: 3: reject, not good enough
review_content: summary_of_the_paper: ## Update after rebuttal and discussions

I thank the authors for taking the time to discuss the issues pointed out in the reviews at length. Unfortunately, I am still not convinced that the paper is ready for publication. My main concerns:

1) There are now experiments in the updated paper claimed to be DP which are not (median clipping).

2) I continue to have doubts about the subsampling amplification. Simply stating that the sampling is random is not good enough, since the key issue is the added uncertainty due to the subsampling: if the sampling does not increase the adversary's uncertainty, there is no amplification. As an immediate remedy, I suggest the authors state the threat model more clearly.

3) I still think the paper can be improved a lot by taking the time to rewrite it focusing on the main contribution of certified robustness under DP and clarity of the presentation.

---

The paper looks at the robustness properties of differentially private (DP) federated learning (FL), focusing on learning classification models from labeled data. The main idea is to turn DP privacy guarantees into certifiable robustness properties. The authors look at 2 certifiable properties, namely, certified prediction (data poisoning does not alter most likely label), and certified attack cost (there is a lower bound on the loss the given attack tries to minimize). They continue to show that DP models in general guarantee these on some level that depends on the privacy bounds. The paper also presents several DPFL learning algorithms for user and instance-level DP.
main_review: Strong points:
i) The problem of data poisoning is an important one, and the connection between robustness and privacy is a nice one to use for getting provable guarantees.
ii) After doing some sporadic checking, I have not found any actual mistakes in the proofs.

Weak points (and comments/questions for the authors):
1) In my opinion the paper lacks focus: there are plenty of DPFL algorithms introduced, but very little testing and comparisons to existing work, while what I take as the main contribution, i.e., the certified robustness, is overshadowed by the FL parts and seems unfinished.

2) It is generally somewhat hard to tell which parts are meant as original contributions and which are referencing existing work (e.g. Sec 4.1, is this meant as original contribution or just paraphrasing existing work?)

3) The threat models of the proposed DPFL algorithms are not quite clear to me: e.g. for insDP, are the DP guarantees supposed to hold against adversaries who can poison some samples during the training? If so, this should probably affect the privacy guarantees resulting from subsampling in InsDP-FedSGD (since the adversary could have knowledge it the data partition in question has been chosen in the update?). How are the other DPFL adversaries?

4) The paper tends to oversell it's contribution:
4.1) The certified prediction Thm1 and it's proof match almost exactly with Prop1 from Lecuyer et al. 2019, the same goes for Thm3 and Cor1 compared to Ma et al. 2019 Thm4, and Cor6. Although both works are cited in the current paper, I do feel that this near-identity should be clearly and unambiguously stated in introducing these results. There is some discussion on this right before Section 6 noting similarities with Lecuyer et al., but stating that "ours focus on the robustness against training-time attacks in FL, which is more challenging considering the distributed nature and the model training dynamics, i.e., the analysis of the privacy budget over training rounds". But the federated setting only shows up in ascertaining that a training algorithm satisfies DP and in considering a suitable neighbourhood definition, it clearly does not complicate matters in the certified robustness Thms. If anything I would think that the no-show of any notion of federation in the proofs shows that at bottom these problems boil down to the ones considered by Lecuyer et al. & Ma et al. and are therefore not any harder. In general I find it a bit misleading that the results are introduced as somehow specifically concerning FL, when the certifiability results actually just use properties of DP, no matter if the setting is federated, otherwise distributed, or centralised; the main things are the neighbourhood definition (to determine what the adversary can control) and the privacy parameters.

4.2) As for the privacy accounting, since the proposed DPFL algorithms seem to be simple modification of existing ones, the privacy cost can be readily and accurately calculated using existing tools, and this seems to be exactly what the authors do. It is therefore hard to see what value does the hard-to-read Moments accountant type Prop.1 bring (note also that there exists a newer and much clarified paper on RDP, which results in improved bounds [1]).

References:
[1] Mironov et al. 2019: Rényi Differential Privacy of the Sampled Gaussian Mechanism
summary_of_the_review: While the paper seems to propose many different things, I find that these are mostly small variations on existing work and I think the authors tend to overstate the papers' contribution (see main review for more detailed comments). The paper generally also feels disorganized due to having several disparate topics (algorithms for training DPFL models, some results that would fit privacy accounting paper better, certifiable properties of DP models). In general, I think this paper could clearly benefit from more work to sharpen the focus and the contribution it has to make, and to make it more clear and readable as well.
review_title: null

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: This paper proves that differential privacy indicates certified robustness against poisoning attacks in federated learning.
main_review: I was very intrigued by the paper at first but ended up having a mixed feeling after reading it. 

On the one hand, the results presented are correct, the experiments are solid, and the paper is rather complete from theory to evaluation.

On the other hand, I have several concerns about the paper. First, the main result presented is very similar to [1]. Although the attack scenarios are different, the intuition is almost the same. This raises my concern about the novelty of the work. Second, I do not understand why the authors choose to limit the results to the federated setting. IIUC, the robustness claims should generally hold against data poisoning attacks and there is no specific optimization for federated learning. User-level DP is also not limited to federated learning. I would say FL is more of an application scenario here. Third, if the authors really want to focus on federated learning, then the instance-level DP section does not make sense because all federated protocols should preserve user-level DP in practice.

Overall, I think the paper is not ready for publication for now. I encourage the authors to tweak the presentation carefully to make it coherent.


[1] Lecuyer, Mathias, Vaggelis Atlidakis, Roxana Geambasu, Daniel Hsu, and Suman Jana. "Certified robustness to adversarial examples with differential privacy." In 2019 IEEE Symposium on Security and Privacy (SP), pp. 656-672. IEEE, 2019.
summary_of_the_review: I do not recommend for acceptance because of (1) lack of novelty; (2) incoherent presentation.
review_title: null

Meta-review: The premise is an exciting observation: Differential privacy in federated
learning might imply being certified against poisoning attacks. While
this may be considered not surprising by some, the connection between
differential privacy and robustness is interesting to many. The
relationship was characterized both theoretically and empirically.

The reviewers discussed the paper extensively with the authors, and
while many issues were clarified, issues on correctness still
remained: it is unclear if the proposed DP mechanism actually is DP,
and subsampling amplification also had issues. Clarity needs to be
added in the writing, and the extensive comments by the reviewers
hopefully help the authors in that.
Decision: Reject