Weakly-Supervised Domain Adaptation in Federated Learning
Authors: Enyi Jiang
Oluwasanmi O Koyejo
Venue: Submitted to ICLR 2023 (2023)
Keywords: federated learning
domain adaptation
healthcare

Abstract: Federated domain adaptation (FDA) describes the setting where a set of source clients seek to optimize the performance of a target client. To be effective, FDA must address some of the distributional challenges of Federated learning (FL). For instance, FL systems exhibit distribution shifts across clients. Further, labeled data are not always available among the clients. To this end, we propose and compare novel approaches for FDA, combining the few labeled target samples with the source data when auxiliary labels are available to the clients. The in-distribution auxiliary information is included during local training to boost out-of-domain accuracy. Also, during fine-tuning, we devise a simple yet efficient gradient projection method to detect the valuable components from each source client model towards the target direction. The extensive experiments on medical imaging datasets show that our proposed framework significantly improves federated domain adaptation performance.

Under review as a conference paper at ICLR 2023
DOMAIN ADAPTATION IN FED- ERATED LEARNING FOR HEALTHCARE
Anonymous authors Paper under double-blind review
ABSTRACT
Federated domain adaptation (FDA) describes the setting where a set of source clients seek to optimize the performance of a target client. To be effective, FDA must address some of the distributional challenges of Federated learning (FL). For instance, FL systems exhibit distribution shifts across clients. Further, la- beled data are not always available among the clients. To this end, we propose and compare novel approaches for FDA, combining the few labeled target sam- ples with the source data when auxiliary labels are available to the clients. The in in-distribution auxiliary information is included during local training to boost out- of-domain accuracy. Also, during fine-tuning, we devise a simple yet efficient gradient projection method to detect the valuable components from each source client model towards the target direction. The extensive experiments on health- care datasets show that our proposed framework outperforms the state-of-the-art unsupervised FDA methods with limited additional time and space complexity.
INTRODUCTION
Federated learning (FL) is a distributed learning paradigm, where an aggregated model is learned using local decentralized data edge devices (McMahan all 2017). FL systems usually share the model weights or gradient updates clients to the server, which prevents direct exposure of the sensitive client data. As a result, data heterogeneity remains an important challenge in FL, and much of the research focuses on mitigating the negative impacts of the distribution shifts between clients' data (Wang et al. 2019 Karimireddy al. 2020 Xie al., 2020b). Further, much of the FL literature has focused on settings where all datasets are fully-labeled. However, in the real world, one often encounters settings where the labels are scarce on some of the clients. To this end, multi- source domain adaptation (MSDA) (Ben-David et al. 2010: Zhao et al.) 2020 Guan & Liu 2021) common solution to this problem, where models trained on several labeled, separate source domains are transferred to the unlabeled sparsely labeled target domain. Here, we consider the more specialized setting of Federated domain adaptation (FDA) - where a set of source clients seek to optimize the performance of target client. As an analogue to MSDA, one may consider clients data as different domains. Thus, goal is to Iearn a good model for the few-labeled target client data samples by transfer the useful knowledge from multiple source clients. In this work, consider the FDA problem under weak supervision, where auxiliary labels are available to the clients. In brief, we propose novel approaches deal with -eakly-supervised FDA, focusing on techniques that adapt both the local training and fine-tuning stages.
Motivating Application. Our work is inspired and applied to applications in predictive modeling for healthcare where there can be significant differences across hospitals, causing transfer errors across sites (Li et al. 2020; Guan al. 202T: Wolleb et al. 2022). Unlike many other industries, healthcare in the US is highly heterogeneous (c.g.. HCA, the largest consortium of hospitals covers 2022 thus many variables are not standardized (Adnan et al.] Osarogiagbon et al.) (021). Hence, we consider experiments that simulate differences across institutions as a large shift. Further, we consider an FL application across several hospitals located at different states in the US. In this setting, FDA is employed to improve performance at a target hospital by leveraging information from all of the source hospitals. The human cost of labeling the images is expensive, thus the data sparsely labeled. Further, in addition to the medical images, the data also include demographic information such as age, sex, race, among others. While such auxiliary data is ubiquitous, often ignored when working to
< 2% of the market (Statista 2020 Wikipedia contributors

---

Under review as a conference paper at ICLR 2023
Server Side:
& larget
Terget Optimol
fie--wning Airge
Target
A/
A2
Cliest
Client (Tappt)
(81)
S1 Model
Gnd
Grod
Model
Fle mas Ieay demeemphhi: Esooration
.Giad
Densgnaphic ipfo Femude) kiteval)
Modal
(b)
(a)
Figure 1: (a) Proposed framework of weakly-supervised FDA: we set up MTL framework leveraging auxiliary labels during the source clients' local training. (b) Intuition of GP: we project the valuable components of souree gradients towards the target direction to boost FDA performance.
improve centralized federated models. Here, show how this data can be used significantly improve out-of-domain (OOD) performance when properly utilized in FL. Also, in FL., local models gradually trained, so the importance of source domains may change dynamically during each iteration. Our work seeks to extract the valuable components of model updates instead of the whole models towards the target direction during each round. We discover that the few labeled target samples can provide sufficient guidance towards adapting quickly to the target domain.
Our contributions are twofold, in local training and fine-tuning. First, we leverage auxiliary informa- tion to reduce the task risk on the target client during local training. Auxiliary information is often cheap and easy to find along with the image inputs, which may offer some useful signals for the un- labeled samples, because of the underlying correlations between auxiliary tasks and main task. Xie] jet al. (202I) and Wang et al. (2022) both found that centralized setting, using auxiliary informa- tion can improve OOD performance, resulting in a smaller OOD risk compared with the baseline. In our work, we set up a cheap and efficient multi-task learning (MLT) framework with auxiliary tasks during local training of source clients, shown in Figure La By optimizing the main and auxiliary task losses together, we show empirically that one can boost target domain accuracy after fine- mee-tuning on labeled target samples (Section 5.2 The gains more evident when the distribution shifts large, yet auxiliary tasks may introduce unexpected noise when domains are too close to each other.
Secondly, observe that including auxiliary information alone does not fully account for the im- portance of source domains. Thus, during fine-tuning, we propose a simple yet efficient gradient projection (GP) method. This method utilizes the useful source domain components and projects them towards the target direction. As shown in Figure [b] during each communication round, we compute the model updates of source clients, and then project them on the target model update ob- tained by fine-tuning the optimum of target domain: the positive cosine similarity between a pair of (source, target) updates serves the importance of that domain. Our experiment results indicate that this gradient projection method achieves a better and more stable FDA performance on the target client, through combining projected gradients with the fine-tuning gradient. We show the superiority of the gradient projection method through the comprehensive experiments on both medical and general-purposee purposed datasets in Section 5.2 and Appendix A.6
small set of labeled target samples. In this way, we greedily approach
Combining two techniques together, our proposed framework outperforms the state-of-the-are un- supervised FDA methods with limited additional computational cost. Also, we show empirically that our framework is resistant to data imbalances on the real-world MIDRC dataset (Section (52)
2 PROBLEM SETUP
In this section, introduce the framework of superveed FDA. We first provide the back- ground knowledge of wcakly-supervised MSDA. Following this, we extend the framework to feder- ated learning setting.
2

---

Under review as a conference paper at ICLR 2023
superv-- MSDA. Let Ds and Dr denote source and target domains. In each domain, we have sample pairs of E Rd as the input, y E IR as the output label. In MSDA, we have N source domains Ds (Ds, i=1 with n, labeled samples samples, which consists of nı labeled samples and myr ny (a large number of) unlaheled samples. During pretraining,
well as target domain with nr total
train model hig, on each source domain using the corresponding laheled samples. The goal of fine-tuning is to learn a target model hr, which minimizes the target risk (h) = Pr(a.))DD,, [h(x) * v common approach using Σis ashs,, with controlling the importance of each source domain such that Σ -i= 0; = 1. Then, we fine-tune the aggregated parameters using the set of labeled samples from Dr.
first aggregates the model parameters
MSDA to the FL setting. As shown Figure Ta and Algorithm we assume there are N + clients in system, where clients (Cs, J have labeled data TDs ) and the remaining client Cr has Dr with nu labeled data. Different from the centralized setting. weakly-superrised FDA requires the target client Cr to have no direct access to the source data {Ds, | The aggregation of the souree domain models is performed by the server using any federated aggregation rules such as FedAvg (McMahan et al.) al., 2020). After S finishes the aggregation, it sends the model parameters to CT and performs a fine-tuning just as with weakly-supervised MSDA. The final model is to all the source clients {Cs, Ji=, which ends one communication round.
Federated problem setting, We now extend
2017) SCAFFOLD (Karimireddy
Algorithm
Federated Domain Adaptation
Input: N source domains D {Ds, > target domain Dr; N source clients (Cs, J target client Cr; target model h/--1) from the previous round r 1; aggregation rule aggr; server S.
Output: Target model h.(:) round r.
1: Step 1: Local training on {Cs,
2: for Ds, in Ds do
3: Initialize hg. + hT
(r-1)
4: Optimize h on Ds, with classification task loss (Eq.
5: Send ng? to S
6: end for
7: Step 2: S received all {4g) 1N
8: alobaa + aggr((()gg))g (
(r)
Ji=1)
9: Send (r)
"global to Cr
10: Step 3: Cr received h (r)
"global
11: Optimize h (r)
globbal Dr using labeled samples.
12: n.4)) -n
13: Send h/.) to
14: Step 4: S received k
for Cs, in Cs do
16: Send (r) to Cs.
end for
3 LEVERAGING AUXILIARY INFORMATION DURING LOCAL TRAINING
After defining the framework for weakly-supervised FDA, we now look into the ways to boost the target client performance. In this section, we explain the idea of leveraging auxiliary information: optimizing the main and auxiliary tasks together during source clients' local training. Here, we only derive the loss objective for source client Cs. (since it is the same for all source clients). Let be the total local training sample size, and as the loss function for the main task. Eq is the loss objective of the main task (image classification).
3

---

Under review as a conference paper at ICLR 2023
Timain (l((z))))))
Let K be the number of auxiliary tasks, and E R", by be kth auxiliary input and output labels, respectively. Thus, 2kj and bkj denote the jth sample input and output for kth auxiliary task. Each auxiliary task shares the same parameters with except for the last layer (denoted as h/-1) and we define to be the feature mapping from the feature representation to the kth auxiliary output. Lastly, lour as the loss function for auxiliary tasks. Then, we can construct the loss objective for auxiliary tasks as follows:
K lau (h' (Zks)), bug)
Lauz
(2)
3
In the end, write the total loss objective for leveraging auxiliary information Eq. controls the weight of auxiliary task losses:
where
K
= Limain + a Laux Σ j=1
((h(z)). + Σ
Lawa (9x ( (2xs)) buy)
(3)
Including auxiliary information only changes the local training process (Step 1): Cs, optimizes its model parameters using a summed loss together with auxiliary task losses. As shown in Figure Ta we optimize the losses of all tasks together, which consist of negative/positive diagnosis as the main task, and race, sex as well as age information auxiliary tasks. When fine-tuning on the target domain Dr, we only optimize the model parameters using the main task loss without any auxiliary tasks (same as Xie et al. 2021), which makes other steps identical Algorithm I Algorithm displays the local training procedure with auxiliary tasks on one of the source clients Cs.-
Algorithm Domain Adaptation with Auxiliary Information (Local training on Cs.))
Algorithm 3 Gradient Projection on CT
Input: N source domain models hss = (hs, J/W tar- get model (((-1) from the previous round r- 1 global model h (r-1)
Input: One source domain Ds,; one source client Cs,; target model h (r-1) from the previous round - 1; input images x, auxiliary input z, main task label y, auxiliary output label b; main task loss function f and auxiliary task loss function lauzi loss weight control hyper-parameter a. Output: source model hs; at round
globat from the previous round r- 1; target do- main Dr; target client Cr; server S; GP weight con- trol variable 3; number of samples of source domains {n; 1N
Output: Target model h.77) at round r. ( 1: Step 2: S received all {h Ri=1
2: (1)) globat (1(111)
3: for hs, in [h Ji=1 do
1: Initialize ( t (r-1)
hT
4: Gs. (((r) h (r)
glabal
2: Optimize
(r)
on Ds.
with
5: end for
s.
6: Send i (r))
global and (G N
li=1 to Cr
Lauz- fn (hg,
S,
y, 2, b, laur. a).
7: Step 3: Cr received (r)
3: Send hg to
fi=l
8: 1
- 114 (r-1)
9: Optimize h14") on Dr using labeled samples
10: GT f- (r) - hr (r-1)
11: Update n using Eq. with (G N NN Gr, 8,
{
Send h( to
4

---

Under review as a conference paper at ICLR 2023
4 UTILIZING SOURCE DOMAIN KNOWLEDGE VIA GRADIENT PROJECTION
Leveraging auxiliary information does not consider the importance each source client contributing to the target client, because only simple aggregation rule. In weakly-superviser FDA, how can one better utilize the knowledge from both labeled target samples and source client models? Here, suggest a novel Gradient Projection (GP) method.
Algorithm intuition. The small set of labeled target samples provides a useful signal on the di- rection of target domain optimum. Thus, during each communication round, server & does not aggregate the weights (Algorithm 10, but instead computes the model updates denoted by {G N where Glr) = ( (1-1)) global at round r from all source clients 1C On the target client Cr, it will perform gradient projection using cosine similarity on each G towards the target direction G.F = h - h/ hnf--1,, which could be computed after fine-tuning on the small set of the labeled target samples. In this way, we greedily maximize knowledge transfer the target domain in each round; the projection of Gl?) on G.P) could be regarded as the weight Ds, at round r. By com- bining Gf with projected gradients ((G(T)) Ji=1 on G(F) controlled by a nyper-parameter 3, we observe more steady convergence towards the target direction, as noted in the experiment results outlined in Section 5.2 Figure and Algorithm iillustrate the procedure of gradient projection.
Details of the Gradient Projection. We compute the cosine similarity (Eq. between source client model update G and target client update Gf7) for each layer of the model (for finer pro- jection). In addition, we align the magnitude of the model updates according to the number of tar- get/source samples, batch sizes, local updates, and learning rates (more details are Appendix[A.. To prevent negative projection, of G(r) and Gfr) (for simplicity, we denote them as G¹s, and G.y)), the cosine similarity and corre- sponding gradient projection result is (Eq, 5):
set the threshold for function GP to be O. For certain layer /
S. Cr G |||
cos(G G47
(4)
( (Cos Gly), if cos G > cos( (o Gl 50
GP(C's, Gp) 10,
(5)
Ji=1 projected on the target direction GT could be computed as Eq, 6 We use L to denote all layers of current model updates. 71, denotes the number of samples trained on source client Cs., which is adapted from FedAvg (McMahan al.) 2017) to redeem data imbalance issue. Hence, normalize the gradient projections according to their number of samples. Also, H concatenates the projected gradients of all layers.
The total gradient projection PaP from all source clients (G(")
N
= = HLec Σ
GP(G's,Gh
T4 Σ
G
(6)
Lastly, hyper-parameter is used to incorporate target update Gi into P&P have a more stable performance. The final target model weight 1.(7) at round r is thus expressed as:
h = 14 (1-1) + +(1 - B) Pop + 3- GT
5 EXPERIMENTS
We evaluate our proposed framework on three medical datasets: CheXpert (Irvin et al. 2019). MIMIC (Johnson et al.) 2019), and real-world imbalanced dataset from MIDRC tasked with COVID-19 detection from X-ray images. The data are split to represent three scenarios: (a) CheX- Pert: balanced data across clients and large distribution shift among domains. (b) MIMIC:
5

---

Under review as a conference paper at ICLR 2023
CheXpert
Ingur Image
States
Number of Samples
IL 16203
NC
3717
CA
560
Auxilarry Information
Scx, Age. Froctal/Latcral
Sex, Age
Sex, Age. Race
IN
557
Classilication Tisk
Postivehngaaive disgosis
Findingho linding
COVID' COVID
TX
501
Lung Pazumonis,
Atelectasis, Consolidation
Hispanie/Latino
Table 1: Statistics of MIDRC
Target Domain
Edema
Asian
II
dataset: an extremely imbal-
anced scenario (IL consists most samples).
Figure 2: Summary of the medical datasets in our experiments.
balanced data across clients and and small distribution shift among domains. (c) MIDRC
highly imbalanced data across clients and large distribution shift across domains. Note that this is real-world application with data from hospitals in different locations across U.S. The first two experiments are designed highlight the kinds of extreme data shift often noted in healtheare, e.g., where the same features are measured differently across hospitals (Wiens et al. 2014). or where the same variables' names refer to very different measurements/ diagnoses across health systems. Un- fortunately, there are little public data illustrating these kinds of important and understudied shifts. Thus, we designed representative semi-synthetic shifts to illustrate the extent and impact the issue.
5.1 EXPERIMENTAL SETUP
We provide basic information and experiment setup of three medical imaging datasets in Figure More details on data splitting and implementation details are discussed at Appendix A.3
CheXpert widely used medical imaging dataset for multi-label classification consisting of 224,316 chest radiographs from 65,240 patients (Irvin et al.) (2019). We use sex, age, frontal/lateral information from patients to construct auxiliary labels and split domains by labelled condition i.e., selecting "Lung Lesion", "Edema", "Consolidation", "Pneumonia", "Atelectasis" all of which are lung conditions, as source and target domains. Thus, the task to predict a new lung condition based on labels of existing lung conditions.
MIMIC is large dataset of 227,835 imaging studies for 65,379 patients between 201 1011- (John- et al. 2019). We set domains using the race information provided and merge the result into four main categories: White, Black, Asian, Hispanic/Latino. Thus, the task is to predict conditions for previously unobserved racial group condition based on labels collected from other groups. The distribution shifts between race domains are considered small, as we can get a high accuracy simply using FedAvg, shown in Table
MIDCC includes Computed Radiography (CR) images as the primary input. We evaluate the pro- posed framework using NC, CA, IN, TX states as source clients, and try adapt the target client IL which has a large number of unlabeled samples. The statistics of these five states are shown in We collect race, sex, and age data as auxiliary information.
Table
5.2
MAIN RESULTS
Table Figure compare the target domain accuracy and convergence speed of following meth- ods on three datasets: a) FedAvg only aggregates the source clients using FedAvg (McMahan et al.) 2017) without fine-tuning on any target samples; b) SadAvg_Finetur performs fine-tuning step includes auxiliary information during local training (Algorithm 2): d) FedAvg Finetune GP performs gradient pro-
after source client aggregations (Algorithm T c) FedAvg Finetune
jection during fine-tuning (Algorithm e) FedAvg Finetune AuxInfo GP combines c) and d)
MIDRC data semi-public, and is available by request httphtt////// /ttp:///wwww//. mi dro.org//

---

Under review as a conference paper at ICLR 2023
CheXpert
MIMIC
MIDRC
ACC
AUC
ACC
AUC
ACC
AUC
FodAvg
53.28+:2.44
$4.13::3.77
1.575=1.55
76.34:33022
53.56+2.33
52.45=4.233
FedAvg Finetune
58.90=0.777
63.81#22.22
70.21=1.38
77.18=0.87
62.33-+2.34
66.66-t2.17
FodAvg Finetune AuxInfo
63.15+2..1
69.10:1:.08
69.41=1.40
1.22
68.22:4:.68
1..41=1.66
FodAvg Finetune GP
72.63+22.27
82.40:12.17
70.69=0,91
77:24-1187
69.36:1:293
72.10:12.48
FodAvv_Finctune... AuxtfooeeA
7..61=11117
83.09-22.28
68.68=2.16
75.34=12.44
70.28::1.68
74.61:13.16
FADA Peng et al.
X020
61.83:13333
64.831:5,44
6..92=1.43
70.11:22.04
11.98:11.59
53.19:12.39
KD3A
Feng
2021
62.61=0..50
67.72+22.50
70.08=0.64
75.50=00.94
51.27:+2.69
55.76::055
Oracle
75.7211.27
82.51:1.115
71.28:0079
76.29:10.94
84.28:1..01
87.62.11.44
Table 2: Target domain accuracy and AUC scores (%) on three medical datasets with comparisons with SOTA methods. Results are reported averaged across 3 trials and 95% confidence intervals.
0.1
550
0.00
0.5
080
3.0
0:4
<10
03
Look
CheXpert
(b) MIMIC
(c) MIDRC
Figure 3: Target Domain Accuracy v.s Rounds on (a) CheXpert (b) MIMIC (c) MIDRC datasets. For the large shift scenarios (a) and (c), AuxInfo and GP both significantly improve target domain performance. For the small shift scenario (b), GP still manages to achieve 1% boost while AuxInfo may introduce extra noise into the training procedure.
together; f) and g) FADA (Peng et al. 2020) and KD3A (Feng et al. 2021) are SOTA methods fo- cusing on the unsupervised FDA setting; Oracle refers to fully-supervised training on the target domain and serves as the upper bound. For all datasets, our framework generally outperforms the state-of-the-an FDA methods with large margin and close to the oracle performance.
CheXpert. For balanced and large distribution shifts, AuxInfo and GP improve around 6% and 18% of accuracy and AUC individually. Combining the two together, we can get a further 3% gain with close-to-oracle performance. However, we notice that AuxInfo slows down the convergence speed of the training procedure.
MIMIC. For balanced and small distribution shifts, the boost compared with baseline (FedAvg) becomes small. Yet GP still manages to achieve a 1% boost with a close-to-oracle performance. We think when shifts are small, including auxiliary information may hinder the fine-tuning of the model, introducing extra noise during the training procedure. The variance is quite large for AuxInfo during the first several epochs, which leads to slow convergence when combining two techniques. When the local signals are not helpful, doing GP on top enlarges the negative effect. We hypothesize that this is why it results in slightly worsened performance.
MIDRC. For imbalanced and large distribution shifts, both AuxInfo and GP achieve a significant increase of 6% and 7% on the target client accuracy/AUC. combining AuxInfo with GP. It is interesting to see when client data is imbalanced, AuxInfo actually achieves faster convergence. We think data imbalance may require more signals coming from auxiliary information to converge. In general, data imbalance has little impact on the performance of our proposed framework. We hypothesize that because we have normalized the client sample size when updating the model weights. In contrast, SOTA methods are not resistant to this issue.
2% extra increase obtained when
Computational complexity analysis. As shown Table 3 our proposed framework requires small amount of additional time and space complexity compared with SOTA methods. GP has memory cost of O(1) and a time cost O(N m²/ m²/l) (Details in Appendix A.4). The average time for calling the GP function is 0.068 seconds (pretty fast) using the ResNet-18 with N = 4.
7

---

Under review as a conference paper at ICLR 2023
Wallelock time
Methods
Number of parameters
Operations with extra time per epoch
(in seconds)
FedAvg
5,6212E+07
/
30,70
FedAvg Finetune
5.6212R+07
Fine-tune nu labeled samples Dy
31.53 (+0.83)
Fine-tune ny labeled samples Dr optimize auxiliary tasks ( GP aggregation
Ours
5.6221E+07
34.38 (+3.68)
KD3A
5.6212E+07
Train all unlabeled samples 2>, Train all unfaheled samples in Dy
41.67 (+10.97)
FADA
5.6369F+07
166. (+135.62)
Table 3: Comparison of computational efficiency. We calculate the average time for running
global epoch on CheXpert with N = 4 source clients using the same Quadro RTX 6000 GPU. We do not consider communication and testing cost and assume the clients' training happens sequentially.
Fairy Frezie
(a) MIDRC: target sample size.
(b) MIDRC: comparisons of auxiliary tasks.
ATC
(c) CheXpert: B.
(d) MIMIC: B.
Figure 4: (a) Target AUC with different sample sizes on FedAvg v.s Finetune v.s. AuxInfo v.s. GP for MIDRC. (b) Performance gain compared with FedAvg FedAvg.Finetur Finetune using different auxiliary inputs. (c) and (d) display the value of B v.s. target domain ACC/AUC for CheXpert and MIMIC
Effects of labeled target sample sizes. We perform the experiment on MIDRC dataset with various target sam- ple sizes of (50, 100, 500, 1000, 2000), testing its impact on AuxInfo and GP individually, as shown in Table and and Figure 4al We report AUC scores for a more accurate comparison. Generally,
2000
FedAvg
52,45
52.45
52.45
52,45
$2.45
Finetune
51.92
54.56
58.52
60.18
65.57
Austnfo
55.13
57.38
61.30
64,88
72.51
GP
53.27
55.38
63.45
67.44
71.98
increase the number of tar- get samples, the target domain accuracy boosts as well. Apart from that, the improvements coming from AuxInfo and GP compared with the baselines also increase when we have more labeled target samples.
Table 4: Target domain AUC (%) on MIDRC dataset with different number of target labeled samples.
Effeets of number of auxiliary tasks. To compare the contribution of each auxiliary task, we train with a single auxiliary task branch of race/sex/age. In Figure 4b] we
FedAvg Finetunennuuueeeeeeeu report the performance gain of each auxiliary information compared with FedAvg Finetune The auxiliary tasks seem have a summed gain effect (race gain sex .aknn + age gain az gain).
Choice of controlled hyper- parameter 3 between gradient projections Per and fine-tuning gradi- ent update Gr. We set / 0, 0.2, 0.4, 0.6, 0.8 and use 50, labeled target samples on CheXpert and MIMIC datasets. We use small number of labeled samples to better evaluate the effectiveness of GP. Figure/4cJand
the target domain accuracies with different choices. When = 0, it relies solely on the set of labeled samples to optimize the parameters. In other words, it could be regarded as not transferring any knowledge from source clients. For both large and small shift cases, We observe around / = 0.4, both the accuracy and AUC reach the highest values. Whereas, when 3 is large, the performance drops severely for CheXpert but has little influence on MIMIC, for which close-to-oracle performance is more easily attained. For the large shift case,
AigureAddpreseee
8

---

Under review as a conference paper at ICLR 2023
find that the total gradient updates become too greedy, taking an overly-large step projecting the gradients from source clients, leading to a worsened performance, while it results in little harm for the small shift case. Therefore, we choose 3 0.5 for our experiments, though the tuning of B may change slightly as the number of labeled target samples varies.
6 RELATED WORK
Data heterogeneity and label deficiency in federated learning. Distribution shifts between clients remains crucial challenge in FL.. Current work often focus on improving the aggregation rules: Karimireddy et al. (2020) use control variates Xie et al. (2020b) cluster the client weights via EM algorithm to correct the drifts among clients. In medical scene, Jiang et al. (2022) and Dinsdale al. (022) try to mitigate local and global drifts via harmonisation. However, people usually assume the local training is fully-supervised for all clients at present. The truth is, label deficiency problem could happen in any of the clients. There recent works try to tackle label deficiency problem with self-supervision or semi-supervision for better personalized models (Jeong et al. 2020 He et al.)
1 Yang et al., [021]. Compared to them, we explore a new setting with fully-labeled source clients and one few-labeled target client, improving FDA performance under weak supervision.
Federated domain adaptation. There are a considerable amount of recent work multi-source domain adaptation with unsupervised setting, with recent highlights on adversarial training (Saito) 2018), knowledge distillation (Nguyen et al. 202I), and nource-free meth- ods (Liang et al. (2020). Peng et al.] (2020): Li et al.] (2020) are the first to extend MSDA into FL setting; they apply adversarial adaptation techniques to align the representations of nodes. More recently, in KD3Á (Feng et al. 2021) and COPA (Wu & Gong, 2021). the server with unlaheled target samples aggregates the local models by learning the importance of each source domain, via knowledge distillation and collaborative optimization. Our work is in contrast to them that primar- ily focus on the unsupervised setting. Here, these methods rely heavily on both source and target data with complicated training procedure on the server. Our framework is computationally efficient, exploring FDA problem in a new manner with auxiliary labels available to clients.
et al. 2018 Zhao et al.
Auxiliary information in domain generalization. In-N-Out (Xie et al. [021] investigate both out-of-distribution (OOD) and in-distribution performance of using auxiliary information as inputs and outputs with self-training. In medical scene, WWang et al. (2022) find by pre-training and fine- tuning on the auxiliary tasks, one could improve the transfer performance between datasets on the primary task. They consider single source-target scenario with no labeled target data while our work focuses on federated MSDA setting with few labeled target data. Their frameworks cannot be directly adapted to setting, since they require training the source samples again after training on out-of-domaan (target) samples. Our MLT framework properly leverages auxiliary labels in the new setting, is cheap and efficient to compute, with good improvement for large shift cases.
Using additional gradient information in FL. Model updates in each communication round could provide valuable insights of client convergence directions. which is mostly explored for byzantine robustness in FL. Zeno++ (Xie et al. (2200a) and FlTrust (Cao et al., 2021) leverage the additional gradient computed from a small clean training dataset on the server, which helps compute the scores of candidate gradients for detecting the malicious adversaries. In our work, we utilize the additional gradient provided by the labeled target samples for FDA problem. In a simple yet effective way, we project source gradients towards the target direction. Our results support that we transfer the knowledge from source clients to the target client with a better and more stable performance.
CONCLUSION AND FUTURE WORK
We show that including auxiliary information during local training and gradient projection during fine-tuning, can help address significant distribution shifts and label deficiency issues existing in cur- rent federated learning systems, particularly for real medical applications. Our results on healthcare datasets show proposed framework improves FDA performance with small additional compu- tational cost. Future work includes evaluating on the fly/ offline finetuning scheme, exploring how to select the set of labeled target samples in the real-world case to better align with the distribu- tion of the unlabeled part, analyzing the impact of more factors related to domain discrepancy, and extending current framework to more general transfer learning setting.
9

---

Under review as a conference paper at ICLR 2023
8 REPRODUCIBILITY STATEMENT
We have provided the details of our dataset preprocessing. model architecture in Section and in the Appendix. Also, we have uploaded the source code of our proposed framework as part of the the supplementary materials. Because access to the MIDRC data restricted to approved users, we are unable to include the original data. We will release code upon acceptance.
training scheme, and
REFERENCES
Kiran Adnan, Rehan Akbar, Siak Wang Khor, and Adnan Bin Amanat Ali. Role and challenges of unstructured big data in healthcare. Data Management, Analytics and Innovation, pp- 301-323, 2020.
Shai Ben-David, John Blitzer, Koby Crammer, Alex Kulesza, Fernando Pereira, and Jennifer Wort- man Vaughan. A theory of learning from different domains. Mach. Learn., 79(1-2)15111175, may 2010. ISSN 0885-6125.
/ttttp:://dii..gg/// 11t...10000110000 009 5152-4
Xiaoyu Cao, Minghong Fang. Jia Liu, and Neil Gong. Fltrust: Byzantine-robus' federated learning via trust bootstrapping. In Proceedings of NDSS, 2021.
Li Deng. The mnist database of handwritten digit images for machine learning research. IEEE Signal Processing Magazine, 29(6))141-144 2012.
Nicola K Dinsdale, Mark Jenkinson, and Ana IL Namburete. Fedharmony: Unlearning scanner bias with distributed data. arXiv preprint arXiv:2205.15970 2022.
Haozhe Feng, Zhaoyang You, Minghao Chen, Tianye Zhang, Minfeng Zhu, Fei Wu, Chao Wu, and Wei Chen. Kd3a: Unsupervised multi-source decentralized domain adaptation via knowledge distillation. In Marina Meila and Tong Zhang (eds.), Proceedings of the 38th International Con- ference on Machine Learning, volume 139 of Proceedings of Machine Learning Research, pp- 3274-3283. PMLR, 18 24 Jul 2021.
Hao Guan and Mingxia Liu. Domain adaptation for medical image analysis: a survey. IEEE Trans- actions Biomedical Engineering, 69(3):113--11 2021.
Hao Guan, Yunbi Liu, Erkun Yang, Pew-Thian Yap, Dinggang Shen, and Mingxia Liu. Multi-site guided deep domain adaptation for brain disorder identification. Medical Image Analysis, 71:100 71:102076, 2021.
mri harmonization via attention -ieention-guidedd
Chaoyang He, Zhengyu Yang, Erum Mushtaq, Sunwoo Lec, Mahdi Soltanolkotabi, and Salman Avestimehr. Ssft: Tackling label deficiency in federated learning via personalized self- supervision. arXiv preprint 1r.00:21 2021.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recog- nition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp- 770-778, 2016.
Jeremy Irvin, Pranav Rajpurkar, Michael Ko, Yifan Yu, Silviana Ciurea-llcus, Chris Chute, Henrik Marklund, Behzad Haghgoo, Robyn Ball, Katic Shpanskaya, et al. Chexpert: A large chest radiograph dataset with uncertainty labels and expert comparison. In Proceedings of the AAAI conference on artificial intelligence, volume 33, pp. 590-597, 2019.
Wonyong Jeong, Jaehong Yoon, Eunho Yang, and Sung Ju Hwang. Federated semi-supervised learning with inter-client consistency & disjoint learning. In International Conference on Learn ing Representations, 2020.
Meirui Jiang, Zirui Wang. and Qi Dou. Harmofl: Harmonizing local and global drifts in federated learning on heterogeneous medical images. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 36, pp. 1087-1005, 2022.
10

---

Under review as a conference paper at ICLR 2023
Alistair EW Johnson, Tom J Pollard, Seth J Berkowitz, Nathaniel R Greenbaum, Matthew P Lun- gren, Chih-ying Deng. Roger G Mark, and Steven Horn.. Mimic-cxr, a de-identified publicly available database of chest radiographs with free-text reports. Scientific data, 6(1):1-8, 2019.
Sai Praneeth Karimireddy, Satyen Kale, Mehryar Mohri, Sashank Reddi, Sebastian Stich, and Ananda Theertha Suresh. SCAFFOLD: Stochastic controlled averaging for federated learning. In Hal Daumé III and Aarti Singh (eds.). Proceedings of the 37th International Conference on Machine Learning, volume 119 of Proceedings of Machine Learning Research, pp. 5132-5143. PMLR, 13-18 Jul 2020.
Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXiv preprint (1X1::1412.6980 2014.
Qinbin Li, Yiqun Diao, Quan Chen, and Bingsheng He. Federated learning on non-iid data silos: An experimental study, In IEEE International Conference on Data Engineering, 2022.
Xiaoxiao Li, Yufeng Gu, Nicha Dvoreek, Lawrence H Staib, Pamela Ventola, and James S Dun- can. Multi-site fmri analysis using ppreeeeirrrs federated learning and domain adaptation: Abide results. Medical Image Analysis, 65:101765 2020.
Jian Liang, Dapeng Hu, and Jiashi Feng. Do we really need to access the source data? Source hypothesis transfer for unsupervised domain adaptation. In Hal Daumé III and Aarti Singh (eds.), Proceedings of the 37th International Conference on Machine Learning, volume 119 of Proceed- ings of Machine Learning Research, pp. 6028-6033 PMLR, 13-18 Jul 2020.
Julian McAuley, Christopher Targett, Qinfeng Shi, and Anton Van Den Hengel. Image-basee rec- ommendations on styles and substitutes. In Proceedings of the 38th international ACM SIGIR conference on research and development in information retrieval, pp. 43-52, 2015.
Brendan McMahan, Eider Moore, Daniel Ramage, Seth Hampson, and Blaise Aguera y Arcas. Crammunication-Efficiieee Learning of Deep Networks from Decentralized Data. In Aarti Singh and Jerry Zhu (eds.), Proceedings of the 20th International Conference on Artificial Intelligence and Statisties, volume 54 of Proceedings of Machine Learning Research, pp. 1273-1282. PMLR, 20-22 Apr 2017.
Tuan Nguyen, Trung Le, He Zhao, Quan Hung Tran, Truyen Nguyen, and Dinh Phung. Most: Multi-source domain adaptation via optimal transport for student-teacher learning. In Uncertainty in Artificial Intelligence, pp. 225-235. PMLR, 2021.
Raymond U Osarogiagbon, Helmneh M Sineshaw. Chun Chich Lin, and Ahmedin Jemal. Institutional- differences in quality and outcomes of lung cancer resections in the united states. Chest, (59(4):6630-16A) 2021.
Xingchao Peng, Zijun Huang. Yizhe Zhu, and Kate Saenko. Federated adversarial domain adapta- tion. In International Conference on Learning Representations, 2020.
Kuniaki Saito, Kohei Watanabe, Yoshitaka Ushiku, and Tatsuya Harada. Maximum classifier dis- crepancy for unsupervised domain adaptation. In Proceedings of the IEEE conference on com- puter vision and pattern recognition, pp. 3723-3732, 2018
Statista.
Top
u.s. URL hctps://www.steet
health systems based on number of hospitals as of 2019, www.c/o//ssttt. wtt.c//ttttsicc////aaaaaaa
2020.
top us for pooffitrfitttt hospital cospital-operators operators based on oo--oonnnn non-numberood of f www-hhhspttssss
Hongyi Wang, Mikhail Yurochkin, Yuekai Sun, Dimitris Papailiopoulos, and Yasaman Khazaeni. Federated learning with matched averaging. In International Conference on Learning Represen- lutions, 2019.
Rongguang Wang. Pratik Chaudhari, and Christos Davatzikos. Embracing the disharmony in medi- cal imaging: A simple and effective framework for domain adaptation. Medical Image Analysis, 76:102 76:10230,, 2022.
11

---

Under review as a conference paper at ICLR 2023
Jenna Wiens, John Guttag, and Eric Horvitz. A study in transfer learning: leveraging data from multiple hospitals to enhance hospital-specific predictions. Journal of the American Medical Informatics Association, 21(4):699-706, 2014.
Wikipedia contributors.
Hospital network Wikipedia, the free encyclopedia, 2022. w/index -esttee-oosptttlleeeteeee oldic=1117892002 [Online; accessed 15-November-2022
URL https://en
Julia Wolleb, Robin Sandkühler, Florentin Bieder, Muhamed Barakovic, Nouchine Hadjikhani, Athina Papadopoulou, Özgür Yaldizli, Jens Kuhle, Cristina Granziera, and Philippe C Cattin.
Learn to ignore: domain adaptation for multi-site mri analysis. In International Conference on Medical Image Computing and Computer-Assisted Intervention, pp. 725-735. Springer, 2022.
Guile Wu and Shaogang Gong. Collaborative optimization and aggregation for decentralized domain generalization and adaptation. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pp. 6484-6493, October 2021.
Cong Xie, Sanmi Koyejo, and Indranil Gupta. Zeno++: Robust fully asynchronous SGD. In Hal Daumé III and Aarti Singh (eds.), Proceedings of the 37th International Conference on Ma- chine Learning, volume 119 of Proceedings of Machine Learning Research, pp. 10495-10503. PMLR, 13-18 Jul 2020a.
Ming Xie, Guodong Long, Tao Shen, Tianyi Zhou, Xianzhi Wang, Jing Jiang, and Chengqi Zhang. Multi-center federated learning, 2020b.
Sang Michael Xie, Ananya Kumar, Robbie Jones, Fereshte Khani, Tengyu Ma, and Percy Liang. In- n-out: Pre-training and self-training using auxiliary information for out-of-distribution robustness. In International Conference on Learning Representations, 2021.
Dong Yang. Ziyue Xu, Wenqi Li, Andriy Myronenko, Holger R Roth, Stephanie Harmon, Sheng Xu, Baris Turkbey, Evrim Turkbey, Xiaosong Wang, et al. Federated semi-supervised learning for covid region segmentation in chest ct using multi-national data from china, italy, japan. Medical image analysis, 70:101992 2021.
Han Zhao, Shanghang Zhang, Guanhang Wu, José MF Moura, Joao P Costeira, and Geoffrey J Gor- don. Adversarial multiple source domain adaptation. Advances in neural information processing systems, 31, 2018.
Sicheng Zhao, Bo Li, Colorado Reed, Pengfei Xu, and Kurt Keutzer. Multi-source domain adapta- tion the deep learning era: systematic survey, 2020.
12

---

Under review as a conference paper at ICLR 2023
ResNet-18
Lincar, hidden size = 128
Leaky (0.3)
Linear, num classes
Linear; num classes
Lâncar, BUm classes
Auxiliary
Figure 5: ResNet-18 model architecture for training auxiliary tasks.
A APPENDIX
A.1 NETWORK ARCHITECTURE
Figure 5 displays the ResNet-18 18 model architecture (He et al. 2016) used in experiment. When training on the single classification task, the output layer of ResNet-18 only consists one branch with the main task output predictions. When training with other auxiliary tasks, add extra branches to the model and output several classification logits, optimizing the losses of all tasks together.
A.2 IMPLEMENTATION DETAILS OF GP
In the previous Section we compute the model updates from source and target clients as G.7)
(r) - (r-1) global and GV h44) - -nff-1) respectively. In our real training process, because we use different learning rates, training samples for source and target clients, we need to align the magnitude of model updates. Eq. aligns the model updates from source clients the target client and Eq. g combines the projection results with the target updates. We use lrT and Irs to denote the target and source learning rates; batchsizer and batchsizeg target and source domains, respectively: ny is the labeled sample size on target client and n, is the sample size for source client Cs.; is is the rounds of local updates source clients.
the batch sizes for
Pap Hiee Σ GP (k.7
A (n - -n(-1)))
butehaszyy
Irr
(N
8 (r-1)
ΣN
Its
alobal
1
(r-1) + + (1 Pap + B (hl))
11
(r-1))
A.3 IMPLEMENTATION AND DATA SPLITTING DETAILS
CheXpert. We randomly sampled 4,000 source labeled samples (2,000 positive, 2,000 negative) from each domain for and 1,000 target labeled samples for fine-tuning. To create a larger distribution shift, for negative samples, we randomly sample the ones with labels "0" for that condition instead of using "No Finding" labels.
MIMIC. We sample 5,00 (250 with findings and 250 without findings) for each source domain and adapt the target domain with 100 labeled samples. This small sample size is intentionally chosen to increase the difficulty of this federated domain adaptation task. For the auxiliary labels, we use and age information from the database.
13

---

Under review as a conference paper at ICLR 2023
0.1
033
Figure 6: Choice of a values including auxiliary information.
MIDRC. To set real-world case with multiple source domains, we split the CR dataset accord- ing to the zip code and select 5 states (II., NC, CA, IN, TX) as our source and target domains. We use all labeled samples in the source domains for local training, and 2,000 target labeled samples for fine-tuning. As real-world dataset, number of samples are extremely imbalanced across the clients, since the dataset collects data mostly from Chicago, which potentially would introduce more distributional noise into clients.
Setup. We used FedAvg the aggregation rule for baselines, and set the hyper-parameters
and B 0.5, source learning rate as 10 10-3 and target learning rate as 2 10 communication rounds 80 for MIMIC dataset and
50 for MIDRC and CheXpert datasets, well as the local training step size to be 1. We use cross-entropy losses for classification tasks and Adam optimizer (Kingma & Ba 2014). We select around 20% 25% of target domain labeled samples for fine-tuning under weak supervision. Further, we use pretrained ResNet-18 (He et al. (016) model with last layer hidden size of 128 for the training of three datasets. When training with auxiliary tasks, we added branches to the output layer of ResNet-1 18 as shown in Figure 5
A.4 DERIVATION OF GRADIENT PROJECTION METHOOD'S TIME AND SPACE COMPLEXITY
Time complexity: Assume the total parameter is and have / layers. To make it simpler, assume each layer has an average of it parameters. Computing cosine similarity for all layers of one source client is (((T)) 1) = (mm /1). We have N source clients so the total time cost for GP is O(N m²/[l)
Space complexity: The extra memory for GP (computing cosine similarity) is O(1) per client for storing the current cosine similarity value.
A.5 CHOICE OF THE LOsS WEIGITT HYPER-PARAMETER FOR AUXILIARY TASKS
We observe that the convergence speed for AuxInfo is slow for small distribution shift case (MIMC dataset). Thus, we further conduct the ablation study on MIMIC dataset for hyper- parameter which controls the loss weights between the main task and auxiliary tasks. We set a = 0.2, 0.4, 0.6, 0.8 and Figure exhibits their target accuracies v.s. epochs. Setting a smaller may lead faster convergence while the final performances almost the same for different values. Thus, we set for all experiments for fair comparison.
Books
DVD
Electronics
Kitchen
Average
FADA
10%
82.70%
77.40%
77.50%
78.90%
KD3A
79.00%
80.60%
85.60%
86.90%
83.10%
FedAvg
77.93%
80.50%
82.19%
84.36%
81.24%
FedAvg Finetune
79.68% 79.78% %
81.31%
85.41%
86.76%
83.29%
FedAvg Finetune GP
82.35% %
84.93%
86.88 %
83.48
Table 5: Target domain accuracy (%) AmazonReview dataset.
14

---

Under review as a conference paper at ICLR 2023
Target Accuracy
FedAvg FedAvv-Finetune Finetune
60. 1 %
69.82%
FedAvg Finetune GP
79.74%
Table 6: Target domain accuracy (%) on Non-IID MNIST dataset.
A.6 SUPPLEMENTARY EXPERIMENTS GRADIENT PROJECTION METHOD
Apart from experimenting on medical imaging datasets for GP, we also test GP on two general-purposee datasets: AmazonReview (McAuley et al. 2015) and self-generated Non-IID MNIST (Deng 2012) datasets with imbalanced labels. We illustrate details of two datasets and present the experiment results in this section.
A.6.1 DATASETS
Amazon Review. This dataset is for a binary sentimental analysis task including four domains. By randomly choosing three of these domains source domains and the rest one as the target domain, we train simple CNN model the elassification task. We use 2,000 training samples (the same KD3A (Feng et al. 2021 for better comparison) for each source domain and 400 labeled target samples.
Non-lID MNIST. We adapt the Non-IID benehmark (Li ct al. 2022) to construct the source and target domains for MNIST dataset in a Non-IID manner. To make the task harder, we choose the data partition with quantity-
label imbalance with only 3 classes available for each source client (though predicting for 10 classes) and we have 8 source clients in the system. For the target client, have 10 classes with all digits with noise-based feature imbalance, creating shift from the source clients. We used CNN architecture to do the experiment on classification between digits. Also, we use 100 labeled target samples and the accuracy on 10,000 unlabeled target samples.
A.6.2 RESULTS
Amazon Review. From the result in Table 5, this dataset includes comparatively simple task with small distribution shifts between clients, yet GP outperforms the state-of-art unsupervised KD3A by 0.38% in average target domain accuracy using 400 labeled target samples. Though our setting is different, GP has a comparable performance against the unsupervised state-of-the-arr method.
Non-IID MNIST.
The distribution shifts between clients larger for this dataset and GP obtains larger boost of performance compared with the previous dataset (20% for FedAvg and 10% for FedAvg Finetune), as shown in Table 6 Hence, we can see that GP improves the target accuracy more significantly when client shifts are bigger.
15


Reviews:
[{"review_id": "f1INVzCW_LP", "reviewer": "Reviewer_v8Kd", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The authors propose a new method for federated domain adaptation (FDA) with application to medical imaging. to improve target domain performance, authors consider a multi-task learning (MTL) framework over the source domains. Several cheap secondary tasks are considered locally (at each source client). Such local information has been shown to improve generalization for data from the target domain. To furthermore benefit from such local signals, authors propose an aggregation of knowledge collected from source domains and transfer it to the target domain using a gradient projection.  The FDA method is validated on 3 medical datasets and compared to standard fedavg tuned and not-tuned on target domain data, and the proposed method showed good improvement.", "summary_of_the_review": "The proposed methods can help to improve the generalization over the target domain. However, there are limited comparisons to state-of-the-art methods.", "strength_and_weaknesses": "Strengths:\n+ The new method allows leveraging inexpensive secondary tasks to improve target domain performance.\n+ Gradient projection aggregation helps improve performance.\n+ The empirical results show the benefit of the method, and some ablation studies are provided.\n+ The supplementary material provides additional implementation details and experimental results that help support the paper.  \n+ The paper includes information that would make it possible to reproduce the methods and experiments (but not with MIDRC data). \n\nWeaknesses:\n - The experimental validation is limited in some respects. There is a lack of comparison with state-of-the-art domain adaptation methods, which makes it difficult to assess the performance of the proposed method, and whether the proposed method provides an improvement. \n- The gradient projection requires that all models must have the same architecture/type.\n- The authors do present average results over several independent replications, using some cross-validation process.  \n- The proposal should be also compared with SOA methods in terms of time and memory complexity.  There should be further analysis of the impact on the performance of growing class imbalance, degree of shift, and diversity among source and target domains.   \n- The tables should show upper-bound results on all datasets.", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "4: The contributions are significant, and do not exist in prior works.", "empirical_novelty_and_significance": "4: The contributions are significant, and do not exist in prior works."}, "extra_text": {}}, {"review_id": "aHxnqimos_c", "reviewer": "Reviewer_1NEn", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The manuscript focuses on the domain adaptation issue in a federated learning setting, where the target client has limited labeled data, whereas the source data are more available with auxiliary information. The major contribution includes leveraging auxiliary information on source clients and also a gradient projection to aggregate the gradient direction from multiple source clients. The experiments were conducted on three chest X-ray datasets. ", "summary_of_the_review": "The contribution of the paper is limited in both novelty and empirical contributions. I recommend revising the paper and resubmitting it to other conferences. ", "strength_and_weaknesses": "Strength:\n- The manuscript focuses on an important setting and introduces two main methods to solve the problem. The third experiment shows good results in a real-work case. \n\nWeakness:\n\n- The first two experimental settings are less realistic, especially the first one, the domain differences are based on the lung conditions. The second experiment's results are incremental. It is also not clear why the proposed method has a much slower convergence on MIMIC in figure 3. The last experimental setting is more realistic, with locations as different domains. I think the first two experiments are not beneficial to support the conclusion.\n- The related work could be better discussed. How to aggregate local gradient information into a global model is well-studied and should be discussed in more detail. ", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "IdR48N1uGG", "reviewer": "Reviewer_JHRh", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The manuscript handles the weakly-supervised domain adaptation in federated learning, in which a set of source clients aiming at boosting the model performance of the target client that has few labelled samples. To achieve the problem, the manuscript proposes to utilize the auxiliary information and gradient projection to help local training and fine-tuning, respectively. The experimental results on three medical image datasets demonstrate the effectiveness of the used techniques.", "summary_of_the_review": "Although the studied problem is interesting and practical, but the manuscript is not well presented and driven, which should be largely improved. I checked the author's rebuttals, some of them are useful and clarifying my concerns. I chose to improve my scorings.", "strength_and_weaknesses": "Pros:\n1) The studied problem weakly-supervised federated domain adaptation is interesting and practical.\n2) The experiments results compared with baselines confirm the effectiveness of the used techniques, i.e., utilizing the auxiliary information and gradient projection to help local training and fine-tuning, respectively.\n3) The manuscript provides sufficient experiments details, along with the codes, which help the reproducibility.\n\nCons:\n1) Although the studied problem weakly-supervised federated domain adaptation is interesting and practical, it is not well illustrated and driven in the manuscript, which can be largely improved.\n2) The proposed method is not well illustrated and driven, and even some parts are confusing. For example, in Eq. (7), the update of model parameter h_T^{(r)} utilizing the scalar P_{GP} is confusing.\n3) The used techniques leveraging auxiliary information during local training and utilizing gradient projection during fine-tuning lacks of novelty, since the former is introduced in [1] while the latter is incorporating the cosine similarity among layers for computing the aggregation weights.\n4) The manuscript can present the visualization of the cumulative gradient projection GP for demonstrating the effectiveness of proposed method.\n\n[1] Sang Michael Xie, Ananya Kumar, Robbie Jones, Fereshte Khani, Tengyu Ma, and Percy Liang. Inn-out: Pre-training and self-training using auxiliary information for out-of-distribution robustness. In International Conference on Learning Representations, 2021.\n\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "9plrC3dAqs8", "reviewer": "Reviewer_sdrU", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The paper introduces a weakly-supervised federated domain adaptation (FDA) framework to help the problems of distribution shifts and label deficiency in federated learning systems. It utilizes auxiliary information during local training and gradient projection during fine-tuning respectively to improve performance. Experiments on medical imaging datasets and general datasets show that the proposed framework improves FDA performance.", "summary_of_the_review": "The paper introduces two modules in weakly-supervised federated domain adaptation (FDA), but the novelty is not so big, and the experiment lacks comparison with methods of full supervision, unsupervised, or other weak supervision.", "strength_and_weaknesses": "Strengths:\n（1）\tThe proposed framework is reasonable and well-supported by the paper content.\n（2）\tExperiments are comprehensive for both the the proposed module and relevant parameters.\n\nWeaknesses:\n(1)\tIt uses auxiliary information in multi-task learning during the local training of each source client. However, such auxiliary information has been proposed before and are widely used, where the effectiveness of such strategy is already verified.\n(2)\tThe experiments include only ablation studies based on the FedAvg. It lacks performance comparison with similar SOTA methods.\n(3)\tMinor：\n1)\tThe “labelled” is labeled in the following sentence.  “The human cost of labeling the images is expensive, thus the data are sparsely labelled.”\n2)\tThere are two “the” in the following sentence. “Further, in addition to the the medical images.”\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}]

Author rebuttal:

Decision: Reject