From: ICLR.cc/2023/Conference/Paper1671/Authors
title: Submission Withdrawn by the Authors

From: ICLR.cc/2023/Conference/Paper1671/Reviewer_Pk7U
confidence: 4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.
summary_of_the_paper: This paper introduces pFedC, a federated learning framework that disentangles the model aggregation. In pFedC, shared feature maps are aggregated in the same manner as FedAvg while task classifiers are aggregated based on local clients' data distribution. The authors claim such an aggregation method prevents the aggregation of irrelevant knowledge among clients. Empirical results are shown to justify the effectiveness of the proposed pFedC method.
strength_and_weaknesses: Strength:
- The paper is well-written and the proposed idea is easy to follow.  
- The proposed research problem is well-motivated.  

Weaknesses:
The major concern I have about the current draft is that its novelty is somewhat limited:  
- The proposed method is very close to how [1] handles classier aggregation. (In section 2.3 "This procedure is then repeated up to the last layer for which we conduct a weighted averaging based on the class proportions of data points per client."). I am a bit concerned that it hurts the novelty of pFedC.
- A follow-up point is, in an extreme case, pFedC is very similar, if not equivalent, to [2].

The scale of the experiment is a bit small, what will happen for the datasets at the ImageNet scale?

[1] https://arxiv.org/pdf/2002.06440.pdf  
[2] https://arxiv.org/abs/2004.10342  
clarity,_quality,_novelty_and_reproducibility: - Clarity: the paper is presented very clearly, and the idea is easy to follow.  
- Quality: the paper is well written, and the quality of the experiments is high (their scale can be larger though).  
- Novelty: The novelty of the proposed idea is somewhat limited given similar variants have been seen in the past literature.     
- Reproducibility: Enough experimental details are provided, and I thus believe that the experiments can be reproduced.     
summary_of_the_review: The proposed method makes sense and seems to work well in practice, but its novelty is somewhat limited given similar variants have been proposed in the literature.
correctness: 4: All of the claims and statements are well-supported and correct.
technical_novelty_and_significance: 2: The contributions are only marginally significant or novel.
empirical_novelty_and_significance: 2: The contributions are only marginally significant or novel.
flag_for_ethics_review: NO.
recommendation: 5: marginally below the acceptance threshold

From: ICLR.cc/2023/Conference/Paper1671/Reviewer_eZua
confidence: 4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.
summary_of_the_paper: The paper introduces a new federated learning framework for personalized federated learning that considers the heterogeneous task/knowledge need of local clients. Experiments on MNIST and CIFAR show promising experimental results.
strength_and_weaknesses: S1: The scenario of different knowledge need for local clients in personalized federated learning is important and relatively novel.
S2: The conflicts-aware pFL framework is different from existing works on pFL and can explicitly handle the different knowledge needs in local clients. 
S3: The workflow is simple yet shown empirically advantageous.

W1: Task heterogeneity is only one type of heterogeneity among clients. What are some other types of data heterogeneity and can the framework also handle some of them?
W2: It is unclear how the proposed conflicts-aware pFL framework can indeed achieve model disentanglement.
W3: More analysis such as regarding the convergence of the proposed pFL framework are lacking.
clarity,_quality,_novelty_and_reproducibility: Clarity: The problems and proposed methods are clearly presented.
Quality: The technical and presentation qualities are both good.
Novelty: The framework is a bit overclaimed regarding model disentanglement as well as the handling of "conflicts". The actual technical designs are not very different from some existing pFL frameworks.
Reproducibility: The proposed framework is straightforward and implementations are provided.
summary_of_the_review: Well-motivated study with simple designs. More analysis and discussions are required to further establish the advantages.
correctness: 3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.
technical_novelty_and_significance: 2: The contributions are only marginally significant or novel.
empirical_novelty_and_significance: 3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.
flag_for_ethics_review: NO.
recommendation: 6: marginally above the acceptance threshold

From: ICLR.cc/2023/Conference/Paper1671/Reviewer_VYsY
confidence: 2: You are willing to defend your assessment, but it is quite likely that you did not understand the central parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.
summary_of_the_paper: To tackle the non-iid issues that often emerge in personalized federated learning, the authors propose reframing the problem as a multi-task learning setup where a feature extractor is shared amongst all clients and each output class is treated as a separate classifier (hence a separate task). Clients can thus receive weights on the tasks that are relevant to their data and are not burdened with classes that are irrelevant to them. The authors experiment using four datasets and a variety of data distributions and demonstrate how their algorithm doesn't suffer the shortcomings of other established FL approaches.
strength_and_weaknesses: The biggest weakness of the the paper is that it has some strange writing which makes it hard to read. An example, in the abstract, "To tackle with such conflicts from facilitating collaboration". Also, there are grammar issues that need to be cleaned up, "while produce unique models" should be "will produce unique models". Also, the usage of the term "disentangle" is difficult given that it isn't the traditional ML usage of the word.

Given their multi-task and masking approach it would appear that significant information about the client is shared with the server (i.e. which classes they have data for). The authors dismiss these concerns with references to differential privacy and encryption approaches. Given that one of the biggest selling points of FL is data-privacy the authors should do more to address these concerns.

For strengths, I did find the shift from thinking of it as a multi-class problem to a multi-task problem to be interesting. Their approach does result in significantly improved accuracy (Table 3), but that is expected given the data distributions). It would be helpful to know if these are the sorts of data distributions that would occur naturally. The authors note, "the performance of pFedC is still better than that of FedAvg. This is enabled by the disentangled architecture and task-specific aggregation process", it isn't clear that this is true. Is it that they have more capacity (a single model per class)?
clarity,_quality,_novelty_and_reproducibility: The authors achieve great results, but it isn't entirely clear to me if it is a fair comparison. The authors are tackling an important problem in the FL space (dealing with non-iid distributions) and their solution is intriguing. The works appear to be reproducible (usage of public datasets, information about the training setups). The quality of the writing leaves a lot to be desired.
summary_of_the_review: My biggest concern is that I fear the authors significantly improved results are not a fair comparison. A baseline to compare against would really help alleviate those concerns. The writing needs a lot more polish.
correctness: 3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.
technical_novelty_and_significance: 3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.
empirical_novelty_and_significance: 3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.
flag_for_ethics_review: NO.
recommendation: 5: marginally below the acceptance threshold

From: ICLR.cc/2023/Conference/Paper1671/Reviewer_GYVa
confidence: 5: You are absolutely certain about your assessment. You are very familiar with the related work and checked the math/other details carefully.
summary_of_the_paper: The paper studies Personalized Federated Learning and focuses on how to avoid irrelevant knowledge transfer between client models. It assumes irrelevant knowledge is only located in the classification head, represented as task-specific parameters. The idea is to (1) use a disentangled classifier architecture (multiple binary classifiers) instead of a single multi-class classifier and then (2) use a conflict-aware model aggregation algorithm to aggregate the disentangled classifier parameters of different clients. The way that avoids irrelevant knowledge transfer is to filter out or apply weights on irrelevant parameters, using an extra class incidence information (a class distribution summary of all the clients' local data)
strength_and_weaknesses: Strength:
1) The paper proposes to disentangle the local task such that the server can be aware of which part of parameters should be transferred among clients. The idea that uses disentanglement to alleviate Non-IID challenge has some novelty. 
2) The paper is well written.

Weaknesses:
1) I think the proposed method of this paper is just the same as MOCHA, which learns personalized parameters given task relationships or learns the task relationships and personalized models alternatively. The method in this paper is even simpler than MOCHA as its task relationships is already given -- which can be calculated from the mask in Eq.(4) on page 5.
2) The Eq.(7) uses the mask defined in Eq.(4) to update personalized models on server. However, this actually breaks the Privacy requirement of FL. Since the mask in Eq.(4) assumes the server can know which classes are observed by each client, which is a type of personal information in some applications.
3) The method is more similar to multi-task FL rather than Personalized FL. The paper might compare the proposed method with several MTL based FL methods.
clarity,_quality,_novelty_and_reproducibility: Methods and experimental results are clearly explained, making it easy to reproduce the work. The method is simple but works very well. Yet it may not be novel enough for ICLR.
summary_of_the_review: After evaluating the novelty of method, the writing quality, and experiments, my recommendation of the paper is "reject".



correctness: 4: All of the claims and statements are well-supported and correct.
technical_novelty_and_significance: 1: The contributions are neither significant nor novel.
empirical_novelty_and_significance: 1: The contributions are neither significant nor novel.
flag_for_ethics_review: NO.
recommendation: 3: reject, not good enough