Venue: ICLR 2023 Conference

Reviews:

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: The manuscript focuses on the domain adaptation issue in a federated learning setting, where the target client has limited labeled data, whereas the source data are more available with auxiliary information. The major contribution includes leveraging auxiliary information on source clients and also a gradient projection to aggregate the gradient direction from multiple source clients. The experiments were conducted on three chest X-ray datasets. 
strength_and_weaknesses: Strength:
- The manuscript focuses on an important setting and introduces two main methods to solve the problem. The third experiment shows good results in a real-work case. 

Weakness:

- The first two experimental settings are less realistic, especially the first one, the domain differences are based on the lung conditions. The second experiment's results are incremental. It is also not clear why the proposed method has a much slower convergence on MIMIC in figure 3. The last experimental setting is more realistic, with locations as different domains. I think the first two experiments are not beneficial to support the conclusion.
- The related work could be better discussed. How to aggregate local gradient information into a global model is well-studied and should be discussed in more detail. 
clarity,_quality,_novelty_and_reproducibility: The manuscript is clear to read. The novelty is marginal. It is not clear if the code will the shared with the public, but it seems it could be reproducible. 
summary_of_the_review: The contribution of the paper is limited in both novelty and empirical contributions. I recommend revising the paper and resubmitting it to other conferences.
review_title: null

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: The authors propose a new method for federated domain adaptation (FDA) with application to medical imaging. to improve target domain performance, authors consider a multi-task learning (MTL) framework over the source domains. Several cheap secondary tasks are considered locally (at each source client). Such local information has been shown to improve generalization for data from the target domain. To furthermore benefit from such local signals, authors propose an aggregation of knowledge collected from source domains and transfer it to the target domain using a gradient projection.  The FDA method is validated on 3 medical datasets and compared to standard fedavg tuned and not-tuned on target domain data, and the proposed method showed good improvement.
strength_and_weaknesses: Strengths:
+ The new method allows leveraging inexpensive secondary tasks to improve target domain performance.
+ Gradient projection aggregation helps improve performance.
+ The empirical results show the benefit of the method, and some ablation studies are provided.
+ The supplementary material provides additional implementation details and experimental results that help support the paper.  
+ The paper includes information that would make it possible to reproduce the methods and experiments (but not with MIDRC data). 

Weaknesses:
 - The experimental validation is limited in some respects. There is a lack of comparison with state-of-the-art domain adaptation methods, which makes it difficult to assess the performance of the proposed method, and whether the proposed method provides an improvement. 
- The gradient projection requires that all models must have the same architecture/type.
- The authors do present average results over several independent replications, using some cross-validation process.  
- The proposal should be also compared with SOA methods in terms of time and memory complexity.  There should be further analysis of the impact on the performance of growing class imbalance, degree of shift, and diversity among source and target domains.   
- The tables should show upper-bound results on all datasets.
clarity,_quality,_novelty_and_reproducibility: + The paper is clearly written and organized. 
+ The contribution is good. Using MTL and gradient projection in FDA allows leveraging secondary local tasks, and provides a different and more efficient way to aggregate/transfer knowledge from source to target.
summary_of_the_review: The proposed methods can help to improve the generalization over the target domain. However, there are limited comparisons to state-of-the-art methods.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: The paper introduces a weakly-supervised federated domain adaptation (FDA) framework to help the problems of distribution shifts and label deficiency in federated learning systems. It utilizes auxiliary information during local training and gradient projection during fine-tuning respectively to improve performance. Experiments on medical imaging datasets and general datasets show that the proposed framework improves FDA performance.
strength_and_weaknesses: Strengths:
（1）	The proposed framework is reasonable and well-supported by the paper content.
（2）	Experiments are comprehensive for both the the proposed module and relevant parameters.

Weaknesses:
(1)	It uses auxiliary information in multi-task learning during the local training of each source client. However, such auxiliary information has been proposed before and are widely used, where the effectiveness of such strategy is already verified.
(2)	The experiments include only ablation studies based on the FedAvg. It lacks performance comparison with similar SOTA methods.
(3)	Minor：
1)	The “labelled” is labeled in the following sentence.  “The human cost of labeling the images is expensive, thus the data are sparsely labelled.”
2)	There are two “the” in the following sentence. “Further, in addition to the the medical images.”

clarity,_quality,_novelty_and_reproducibility: The paper is well organized. Key resources (e.g., code, data) are available, and key details are well-described to reproduce the main results.  The presentation could be further improved if the contribution are more concisely and clearly presented. 
summary_of_the_review: The paper introduces two modules in weakly-supervised federated domain adaptation (FDA), but the novelty is not so big, and the experiment lacks comparison with methods of full supervision, unsupervised, or other weak supervision.
review_title: null

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: The manuscript handles the weakly-supervised domain adaptation in federated learning, in which a set of source clients aiming at boosting the model performance of the target client that has few labelled samples. To achieve the problem, the manuscript proposes to utilize the auxiliary information and gradient projection to help local training and fine-tuning, respectively. The experimental results on three medical image datasets demonstrate the effectiveness of the used techniques.
strength_and_weaknesses: Pros:
1) The studied problem weakly-supervised federated domain adaptation is interesting and practical.
2) The experiments results compared with baselines confirm the effectiveness of the used techniques, i.e., utilizing the auxiliary information and gradient projection to help local training and fine-tuning, respectively.
3) The manuscript provides sufficient experiments details, along with the codes, which help the reproducibility.

Cons:
1) Although the studied problem weakly-supervised federated domain adaptation is interesting and practical, it is not well illustrated and driven in the manuscript, which can be largely improved.
2) The proposed method is not well illustrated and driven, and even some parts are confusing. For example, in Eq. (7), the update of model parameter h_T^{(r)} utilizing the scalar P_{GP} is confusing.
3) The used techniques leveraging auxiliary information during local training and utilizing gradient projection during fine-tuning lacks of novelty, since the former is introduced in [1] while the latter is incorporating the cosine similarity among layers for computing the aggregation weights.
4) The manuscript can present the visualization of the cumulative gradient projection GP for demonstrating the effectiveness of proposed method.

[1] Sang Michael Xie, Ananya Kumar, Robbie Jones, Fereshte Khani, Tengyu Ma, and Percy Liang. Inn-out: Pre-training and self-training using auxiliary information for out-of-distribution robustness. In International Conference on Learning Representations, 2021.


clarity,_quality,_novelty_and_reproducibility: a) Clarity: The studied problem and proposed method are not well presented and driven, in which some aspects are confusing, e.g., the update of model parameter h_T^{(r)} utilizing the scalar P_{GP} in Eq. 7.

b) Quality: The utilized technique outperforms the compared baselines.

c) Novelty: Although the studied problem weakly-supervised federated domain adaptation is somewhat novel, the used techniques leveraging auxiliary information during local training and utilizing gradient projection during fine-tuning lacks of novelty, since the former is introduced in [1] while the latter is incorporating the cosine similarity among layers for computing the aggregation weights.

d) Reproducibility: The manuscript provides sufficient experiments details, along with the codes, which boost the reproducibility.

[1] Sang Michael Xie, Ananya Kumar, Robbie Jones, Fereshte Khani, Tengyu Ma, and Percy Liang. Inn-out: Pre-training and self-training using auxiliary information for out-of-distribution robustness. In International Conference on Learning Representations, 2021.

summary_of_the_review: Although the studied problem is interesting and practical, but the manuscript is not well presented and driven, which should be largely improved. I checked the author's rebuttals, some of them are useful and clarifying my concerns. I chose to improve my scorings.
review_title: null

Meta-review: metareview:_summary,_strengths_and_weaknesses: Summary:
This paper discusses domain adaptation in the federated learning setup under weak supervision. The key idea is to leverage auxiliary information on source clients and aggregate gradients from multiple source clients.

Strength:
The problem setting is novel and interesting. The experiment on computed radiography images is a nice challenge.

Weakness:
Novelty is limited and overall contributions are marginal.

justification_for_why_not_higher_score: Thanks for the detailed review and revision of the manuscript. The authors' feedback clarified some of the concerns raised by the reviewers in their initial reviews. Nevertheless, the overall contribution is rather minor and novelty is limited. Therefore, I cannot recommend the acceptance of this work.
Decision: Reject