Venue: ICLR 2021 Conference

Reviews:

final_score: rating: 6: Marginally above acceptance threshold
review_content: **Pros:**
+ Providing clients with additional flexibility to weight different portions of the globally learned weights allows for greater customization in federated learning
+ The client-specific hidden information leads to better test performance on non-iid data, along with better fairness and privacy properties for the trained models
+ Experiments are carried out across a range of datasets, with appropriate modifications made to simulate non-iid distribution among agents

**Cons:**
- The description of the IBP used to learn the local active factors in Section 2.2 is almost incomprehensible. The terms $v$ and $\pi$ are never defined, and it is unclear what the distribution $q$ corresponds to. While there may not be sufficient space in the main body to provide a detailed description, it should be included in the supplementary material without assuming detailed prior knowledge of Bayesian nonparametrics.
- Data privacy and security are often using interchangeably in the paper, when they correspond to very different concepts. This terminology needs to be clarified, as all the benefits of the proposed method are with regard to data privacy, and not security. 

_Questions:_
1. Why is the number of learnable parameters smaller for CIFAR-10 than MNIST in Table 1? 
review_title: Promising federated learning approach let down by unclear presentation and terminology

final_score: rating: 4: Ok but not good enough - rejection
review_content: This paper proposes WAFFLe for anonymized federated learning.  The idea seems interesting, but I have a few concerns.  In summary, the motivation/privacy claims are not clear, and the performance evaluation doesn't seem fair as the authors only considered single-model FL algorithms.  
 
- Motivation (Why can't we just use SecAgg?): Secure aggregation guarantees that each local model parameter is protected.  See "Practical Secure Aggregation for Privacy-Preserving Machine Learning" by Bonawitz et al.  Thus, I am not sure about this problem's motivation.  Is there any reason one should employ this instead of secure aggregation?  (Note that the original secure aggregation protocol is computationally expensive, but the recent variations are not: For instance, see SecAgg+ [Bell et al., "Secure Single-Server Aggregation with (Poly)Logarithmic Overhead"] and TurboAGG [So et al., "Turbo-Aggregate: Breaking the Quadratic Aggregation Barrier in Secure Federated Learning"].)

- Missing privacy guarantees: While secure aggregation comes with a solid privacy guarantee, there is no theoretical guarantee that WAFFLe indeed can protect clients' data.  Especially with the IBP prior that induces sparsity, each client will update only a small subset of the weight factor dictionary.  This pattern may even reveal more information about the set of weight factors used by each client.  (Maybe the updated r also reveals this support information?)  

- Non-adaptive attack: While the authors have presented some experimental results to claim improved privacy in Section 4.4, they only used the off-the-shelf attack algorithms.  The authors should have designed an adaptive attack algorithm, which could have performed much better.  The need for "adaptive evaluations" has been well described in [Tramer et al., "On Adaptive Attacks to Adversarial Example Defenses"], though in an adversarial example context.  

- Multi-task/multi-center/personalization FL: The performance improvements in Table 1 and Table 2 seem mostly due to personalization, i.e., each model has its own model.  However, all the baseline algorithms assume a single model.  The authors should have compared the performance of WAFFLe with other algorithms that also maintain individual local models or multiple global models.  For instance, the authors may want to add MOCHA in [Smith et al., "Federated Multi-Task Learning"] (cited in the current work), and multi-center FL algorithms in [Sattler et al., "Clustered Federated Learning: Model-Agnostic Distributed Multi-Task Optimization under Privacy Constraints"] and its follow-up works.  The authors claimed that meta-learning-based approaches require sharing a small subset of data, but not all do.  For instance, see [Fallah et al., "Personalized Federated Learning: A Meta-Learning Approach"].  
 
review_title: Review

final_score: rating: 5: Marginally below acceptance threshold
review_content: -Summary-
The paper proposes a novel approach to federated learning which decomposes model parameters into two sub-modules with task-specific weight factors. The authors adopt the IBP process for sparse selections of the factors per client, which mitigates interferences across local clients. They validate the model with several baselines on two different non-iid settings.

-Pros-
- The proposed method largely outperforms baselines like FedAvg, FedProx, and p-FFL.
- The paper is easy to follow and intuition/direction is reasonable enough.
- Reasonable analysis of attacks and ablation studies (Appendix) is described.

-Cons-
- Lack of intuition about the architectural design of WAFFLe. There will be several similar designs like W_i^l \leftarrow \lambda_i^l * W_i^l (without decomposition) where \lambda_i^l is an attention vector, or r^l can leave as local without communication. Even weight factors can be regularized to l_0 norm.  I agree that the method looks reasonable but there need more insights into the necessity of model components. 
- Only tiny networks are used. I recommend utilizing further modern complex CNN architectures. Applicability of modified network designs of WAFFLe on diverse modern CNNs is not guaranteed.
- Comparison with recent personalized FL/ Bayesian FL approaches is required. Since the method uses task-specific local parameters to mitigate inference from the naive aggregation of local parameters, the authors should compare with recent personalized FL/ Bayesian FL methods rather than old FL baselines.
review_title: Review of the Paper 

Meta-review: This paper proposes an anonymization method for federated learning based on the Indian buffet process. The reviewers found the idea interesting, but raised the following main concerns (please see the reviews for more details):
* Motivation and terminology needs clarification
* Better comparison with secure aggregation methods
* Missing privacy guarantees
Overall the reviewers of this paper are borderline. I hope the authors will take the reviewers' feedback into account when revising the paper.

Decision: Reject