Robust Training of Federated Models with Extremely Label Deficiency
Authors: Yonggang Zhang
Zhiqin Yang
Xinmei Tian
Nannan Wang
Tongliang Liu
Bo Han
Venue: ICLR 2024 poster (2024)
Keywords: Federated Learning
Label Deficiency
Primary area: general machine learning (i.e., none of the above)

Abstract: Federated semi-supervised learning (FSSL) has emerged as a powerful paradigm for collaboratively training machine learning models using distributed data with label deficiency. Advanced FSSL methods predominantly focus on training a single model on each client. However, this approach could lead to a discrepancy between the objective functions of labeled and unlabeled data, resulting in gradient conflicts. To alleviate gradient conflict, we propose a novel twin-model paradigm, called **Twinsight**, designed to enhance mutual guidance by providing insights from different perspectives of labeled and unlabeled data. In particular, Twinsight concurrently trains a supervised model with a supervised objective function while training an unsupervised model using an unsupervised objective function. To enhance the synergy between these two models, Twinsight introduces a neighborhood-preserving constraint, which encourages the preservation of the neighborhood relationship among data features extracted by both models. Our comprehensive experiments on four benchmark datasets provide substantial evidence that Twinsight can significantly outperform state-of-the-art methods across various experimental settings, demonstrating the efficacy of the proposed Twinsight.

Published as a conference paper at ICLR 2024
ROBUST TRAINING OF FEDERATED MODELS WITH EXTREMELY LABEL DEFICIENCY
Yonggang Zhang" Zhiqin Yang Xinmei Tian?
Nannan Wang
Tongliang Liu'
Bo Han
'TMLR Group, Hong Kong Baptist University University of Science and Technology of China
3 Xidian University *Sydney AI Centre, The University of Sydney
ABSTRACT
Federated semi-supervised learning (FSSL) has emerged as a powerful paradigm for collaboratively training machine learning models using distributed data with label deficiency. Advanced FSSL methods predominantly focus on training a single model each client. However, this approach could lead to discrepancy between the objective functions labeled and unlabeled data, resulting in gra- dient conflicts. To alleviate gradient conflict, we propose novel twin-model paradigm, called Twin-sight, designed to enhance mutual guidance by providing insights from different perspectives of labeled and unlabeled data. In particular, Twin-sight concurenntyy trains a supervised model with a supervised objective function while training an unsupervised model using an unsupervised objective function. To enhance the synergy between these two models, Twin-sight intro- duces a neighbourhood preserving constraint, which encourages the preservation of the neighbourhood relationship among data features extracted by both models. Our comprehensive experiments on four benchmark datasets provide substantial evidence that Twin-sight significantly outperform state-of- offte-of-the-art offtth--art methods across various experimental settings, demonstrating the efficacy of the proposed Twin-sight. The code is publicly available
comm/mmr-grooppTwiinn
INTRODUCTION
Federated learning (FL) (Yang et al., 2019; Kairouz et al., 2021; Li al., 2021; McMahan al., 2017; Wang et al., 2020) has gained widespread popularity in machine learning, enabling models to learn from decentralized devices under diverse domains (Li et al., 2019; Xu et al., 2021; Long et al., 2020a). Despite the benefits of FL, obtaining high-quality annotations remains challenging in resource-constrained scenarios, often leading to label deficiency and degraded performance (Jin et al., 2023). In this regard, federated semi-supervised learning (FSSL) (Diao et al., 2022; Liu al., 2021e; Jeong et al., 2021) has achieved significant improvements in tackling label scarcity by jointly training global model using labeled and/or unlabeled data.
Advanced FSSL methods propose to combine off-the-rack semi-supervised methods (Sohn et al., 2020; Xie et al., 2020a; Berthelot al., 2020) with FL. (McMahan et al., 2017; Li et al., 2020), leveraging the strengths of both approaches like pscudo-labeling (Lee et al., 2013) and teacher- student models (Tarvainen & Valpola, 2017). These methods typically train a single model on each client using labeled or unlabeled data, following the inspirits of traditional semi-supervised learning. However, the decentralized nature of FL scenarios distinguishes FSSL from traditional semi-superveee learning, where labeled and unlabeled data are on the same device. Namely, clients in FL may have diverse capabilities to label data, leading to label deficiency on many clients (Liu et al.,, 2021c; Yang et al., 2021; Liang et al., 2022). Training single model using different objective functions could make gradients on different distributions collide, as depicted in Figure 2(a). Thus, is urgent to develop an FL-friendly semi-supervised learning framework to tackle label deficiency.
To combat label deficiency, we propose a twin-model paradigm, called Twin-sight, to enhance mutual guidance by providing insights from different perspectives of labeled and unlabeled data, adapting
"quual contributions.
Corresponneece Bo Han (bhanml wwwpm//wmooooooocooiiiiooo

---

Published as a conference paper at ICLR 2024
O A
Labeled data
Supervised model
Unsupervised model
Data embedding
Unlabeled data
Unsupervised
A
O
loss
Distance vector
Distance
Twin-sight
Labeled elient
O
loss
Twin-sight
A
Supervised
S
loss
Unsupervised
loss
Twin sight
Unlabeled
loss
client
Supervised
loss
Figure 1: Overview of sight The framework illustrates the process for both fully-labeled and fully-unlabeled clients. Each client incorporates a supervised model and an unsupervised model. The supervised model undergoes supervised learning using either ground-truth labels or pseudo labels, while the unsupervised model performs self-supervised learning. This approach enables the generation of twin sights for each sample, capturing both supervised and unsupervised perspectives. Subsequently, these two models are aligned, leveraging the complementary information.
traditional semi-supervised learning to FL. In particular, Twin-sight trains a supervised model using supervised objective function, while training an unsupervised model using an unsupervised objective function. The twin-model paradigm naturally avoids the issue of gradient conflict. Consequently, the interaction between the supervised and unsupervised models plays crucial role in Twin-sight. Drawing inspiration from traditional semi-supervised learning (Belkin & Niyogi, 2004) from manifold perspective (Roweis & Saul, 2000), we introduce neighborhood-preser preserving constraint to encourage preserving the neighborhood relation among data features extracted by these two models. Consequently, the supervised and unsupervised models can co-guide each other by providing insights from different perspectives of labeled and unlabeled data without gradient conflict.
The overview of the proposed Twin-sight can be found in Figure 1. In Twin- twin-sight, the unsupervised objective function, e.g., instance discrimination (Wu al., 2018)/. does not vary with the presence or absence of labels for the unsupervised model. In contrast, the supervised objective function varies with the presence or absence of labels. For clients with label information, can be a vanilla objective, c.g., cross-entropy loss. For clients without labels, Twin-sight regards predictions with high confidence as reliable labels to perform supervised learning. Twin-sight, the constraint remains the same whether labels exist, encouraging the preservation of neighborhood relation (Sarkar et al., 2022; Gao et al., 2023; Pandey et al., 2021). Comprehensive experiments conducted on four standard datasets demonstrate the efficacy of the proposed Twin-sight.
Overall, our contributions can be summarized as follows:
We point out that the discrepancy between the objective functions of labeled and unlabeled data could cause gradient conflict, posing specific challenges for semi-supervised learning approaches in FL scenarios.
To tackle label deficiency, we propose a twin-model framework, Twin-sight, to tackle gradient conflict in federated learning. Twin-sight trains supervised model paired with an unsupervised model. Meanwhile, Twin-sight introduces a constraint to make the two models co-guide each other with insights from different perspectives of labeled and unlabeled data by preserving the neighborhood relation of data features.
'The objective function refine class-level identification into fine-grained challenges causally (Chalupka al., 2014; Mitrovic al., 2020).
2

---

Published as a conference paper at ICLR 2024
We conduct comprehensive experiments under various settings using widely used benehmark datasets. Our experimental results show that Twin-sight outperforms previous methods, achieving state-of- the performance.
RELATED WORK
Federated Learning. Federated learning (FL) enables distributed clients to collaboratively train global model with preeerigresn (Kairouz et al., 2021; Ji et al., 2023). However, the performance of federated learning typically suffers from heterogeneity in data distributions, processing capabilities, and network conditions among clients (Lin et al., 2020; Li et al., 2022; Diao et al., 2023; Zhu et al., 2022; Tang al., 2022). One of the most popular algorithms in FL is FedAvg (McMahan al., 2017), which aggregates parameters from randomly selected clients to create global model and achieves convergence after several rounds of communication. A series of works, e-g., FedProx (Li et al.. 2020), SCAFFOLD (Karimireddy et al., 2020), is proposed to calibrate the local updating direction. These methods implicitly assume that all clients can label data, which could be violated in many practical scenarios. Some approaches include the sharing of privacy-free information (Tang et al., 2022) or the use of protected features (Yang et al., 2023). These strategies have shown promise in achieving better performance.
Semi-Supervised Federated Learning (SemiFL). To relax the assumption, SemiFL (Diao et al., 2022) assumes that the server can annotate data, while clients collect data without labels. In SemiFL, selected clients generate pseudo-labels using the global model and then fine-tune the aggregated model using labeled data the server side. Semi-supervised learning is well-established approach that has proven to be effective in improving the performance of machine learning models by making use of both labeled and unlabeled data (Zhu et al., 2003; Zhu & Goldberg, 2009). Self-training methods (Xie et al., 2020b; Zoph et al., 2020; Liu et al., 2021b) have emerged as a popular approach for semi-supervised learning, in which a teacher model is trained on labeled data and used to generate pseudo-labele for the remaining unlabeled data. Another significant line of work is based on consistency training (Tarvainen & Valpola, 2017: Xie al., 2020a). Apart from the above, combining these two methods is effective in achieving improved performance e.g., MixMatch (Berthelot et al., 2019), FixMatch (Sohn et al., 2020), and RemixMatch (Berthelot et al., 2020). However, the server may fail to collect data due to privacy concerns.
various benchmark datasets,
Federated Semi-Supervised Learning (FSSL). Advanced works assume that the some clients have labeled data (Jin et al., 2023; Liu et al., 2021c), which has gamered significant attention. One stream of research focuses on the fully-labeled clients versus fully-unlabeled clients (Liu et al., 202 Ic; Yang et al., 10 Liang al., 2022), while another body of literature studies the use of partially labeled data at each client (Long et al., 2020b; Lin et al., 2021; Wei & Huang, 2023). For instance, RSCFed (Liang et al., 2022) leverages mean-teacher fully-unlabeled clients and sub-sample clients for sub-consensus by distance-reweighted model aggregation. This approach comes at the cost of increased communication burden. FedIRM (Liu et al., 2021c) learns the inter-client relationships between different clients using relation- relation-matching module. However, these methods merely train a single model on labeled and unlabeled data, causing the gradient conflict issue.
Self- Self-Supervised Learning, Self-supervised learning is an increasingly popular approach to acquiring meaningful representations without needing explicit labels (He al., 2020; Chen & He, 2021). Con- trastive methods (Wu et al., 2018; Bachman et al., 2019; Misra & Maaten, 2020) have demonstrated state-of-ther performance, which enforces the similarity of representations between two augmented views of input. One of the predominant methods, SimCI.R (Chen et al., 2020), applies InfoNCE (Oord et al., 2018) loss to discriminate positive pairs from numerous negative samples. There is a work (Zhuang et al., 2021) also investigates the federated version of these unsupervised methods. Previous work shows that the instance discrimination task can be regarded as a (more challenging) fine-gre ine-gnained version of the downstream task (Mitrovic et al., 2020). These insightful works inspire us to introduce self-supervised learning into FSSL for processing unlabeled data.
3 METHODOLOGY
In this section, we present our "Twin-sight" framework in detail. Before that, we provide a formal definition of the studied problem (Sec 3.1) and the motivation (Sec 3.2). We then elaborate on the
3

---

Published as a conference paper at ICLR 2024
twin-model paradigm (Sec 3.3), outlining the roles and training procedures of the supervised and unsupervised models. Finally, we explore the interaction between these two sights (Sec 3.4).
3.1 PROBLEM DEFINITION
In general, FL tends train global model parameterized by w with K participants collaboratively In other words, the objective function J (w) of the global model is composed of the local function over all participants' data distribution:
K
min J(w)
Σ (L k=1
(I)
where 3e determines the weight of the k-th client's objective function. The k-th client possesses local private dataset denoted by Dr. drawn from the distribution P(X,. Yz).
In FSSL, typical scenario involves M clients with fully-labeled data, while the remaining T clients have unlabeled data. The set of all clients C' = {(* J== can be divided into two subsets, C = {cm Jm and C = {cr)T= corresponding to the clients with labeled and unlabeled data, respectively. The dataset of the m-th client in C is D = {( yi, ) P(Xm: Ym) and DI {xx))= P(X;) denotes the dataset containing data witout annotation for ce.
3.2 MOTIVATION
In the existing FSSL framework, the local objective function on labeled data can be formulated as:
Im (wm) == P(x. (Wmm x, V),
where (Xm, Ym) is the random variable denoting the image X,, and label Y,, and E(-) is the cross-entropy loss. To leverage unlabeled data, advanced methods (Liang et al., 2022; Liu et al., 2021c; Yang et al., 2021) propose to employ traditional semi-supervised carning techniques such as pseudo-labeling (Lee et al., 2013) and mean-teacher (Tarvainen & Valpola, 2017) in conjunction with transformation function T(-). These methods utilize global model parameterized to utilize unlabeled data the t-th client:
J(w,)) == (x- T(x)),
(3)
where ((5-) denotes a consistency constraint. Therefore, the global objective can be rewritten as:
T
min J(w)=
Σ ===1
mm (w) (w]+
+ B.T((w).
(4)
centralized training, this approach can achieve state-of-the-art performance. However, the objective function may cause "client drift" due the different objective functions of clients. Specifically, all parameters will be aggregated to construct global model, even these models are trained with different objective functions. In practice, aggregating models with different objective functions will cause "client drift" (Wang et al., 2020). To verify the client drifts, we calculate the similarity between gradients calculated under different objective functions, i.e., Eq. and Eq. 3. The results shown in Figure 2(a), demonstrating that gradients from these two do not align well, i.c., gradient conflict.
The gradient conflict issue inherently attributed the decentralized nature data. Specifically, FL models are trained on labeled or unlabeled data, leading to aggregation with models trained using different objective functions and data distributions.
3.3
PARADIGM
Built upon the aforementioned analysis, propose to introduce twin-model paradigm to tackle gradient conflict. Intuitively, we can train a supervised model using labeled data while training an unsupervised model using unlabeled data. Consequently, the main challenge is designing effective interaction mechanism between these two models, making these two models promote each other by providing insights from different perspectives of labeled and unlabeled data.
4

---

Published as a conference paper at ICLR 2024
Algorithm 1 pseudo-code of Twin-sight
Server input: communication round R Client k's input: local epochs E, k-th local dataset D
Initialization: all clients initialize the model w s,k w
u.k."
Server Executes:
for each round r 1, do
server random samples a subset of clients C, S {1,..., KJ.
server communicates w w to selected clients
for each client C₁ E C, in parallel do
wh+ a, +- Local Training (k: w w.))
end for
w w.rt AGG (wit)) witk Ck C2)
end for
Local Training((k; w w:)):
if es C then
with w +-GD update by Eq 10 in E epochs.
else if of E C then
wit w +GG update by Eq 11 in E epochs.
end if
Return w r-1 r+1 u, s,
to server
The twin-model paradigm has models: an unsupervised model parameterized with w. and supervised model parameterized with w.. The unsupervised model is trained with a fine-grained task of downstream classification task, i... instance discrimination:
M
min (w.) = Σ (ww) + W..
Σ BiTr (wa):
(5)
m=1
Unsupervised model on fully-labeled clients
U Unaupereised model on fully-unlabeled clients
where the objective function (.) is the same for all client?:
exp (sim (i), (()x Ex
(ww) = log
(6)
[[e exp (sim (f(wa: x; f (W.; /T)
where /(w..:) the unsupervised model and the temperature hyper-parameter. Thus, the unsupervised model can be trained in a vanilla FL manner.
Supervised models on clients with labeled data can be trained with a cross-entropy loss Im (-)- Notably, the label information is invalid on clients sampled from the unlabeled subset CC = {ce) t=1
Thus, we introduce a surrogate loss Jj (-) to train the supervised model with unlabeled data on client Ct. This can be formulated as:
M
T
min J'(w.) = Σ (w..) + Σ BwJ))(
(7)
w.
m=1
where the surrogate loss replaces the label used in cross-entropy loss with pseudo label ÿ predicted by the supervised model f(w.;:):
Jr(ww, := J((w))===Tpp -T
()) log f(w.;
where I[(-) an indicator function, a(.) can select the maximum value for a given vector, and is threshold working as a hyper-parameter to select predictions with high confidence. This is because training models using data with low-confidence predictions cause performance degradation, which is consistent with previous work (Wang al., 2022). Consequently, we can train supervised model in a vanilla FL manner.
Here, omit the difference induced by distribution discrepancy between elients.
5

---

Published as a conference paper at ICLR 2024
3.4 TWIN-SIGHT INTERACTION
Training two models separately cannot make these two models benefit each other. Therefore, we introduce Twin-sight loss complete the Twin-sight framework. The inspiration is drawn from local linear embedding (Roweis & Saul, 2000) and distribution alignment (Zhang et al., 2022), where the features embeddings) of the same data should keep the same neighborhood relations under different feature spaces.
Specifically, we introduce constraint to encourage preserving the neighborhood relation among data features extracted by supervised and unsupervised models. The intuition is straightforward that features extracted by the supervised model and the unsupervised model can be drastically different, making it hard to align the feature distributions. Thus, we propose to Twin-sight loss Ja() )) to align the neighborhood relation among features:
min Ja (W.: W.) ((N([((,; N( N(((.; Wa
where d is certain metric measure the difference between matrices, e.g., fn-m and N(-) stands for the function used to construct a neighborhood relation. The Twin-sight loss can be used train both the supervised and unsupervised models in vanilla FL manner. Consequently, the objective function on labeled data J (-)is formulated as:
JwW,, Wu)=
(ws)
Na (W,, Wu):
(10)
Superoised model on lable-l clients
Unsuperossed model on fully-labeled clients
Twin seggtt interaction on both models
Similarly, we can leverage unlabeled data by loss function J"()):
J. (ws)
AaTT(WW.
AaJa(Was Wa).
(11)
Superviseed model on filleledl clients
Unsuperuised model on fully-unlabeled elients
Twin sightt interaction on both models
where X, and Aa is the cypper-parametert to adjust.
The overview of the Twin-sight framework is illustrated in Figure I and Algorithm 1. According to the framework of Tiin-sight, it is also possible to apply Twin-sight to similar label deficiency scenario where all clients hold data with a portion of it labeled. This superiority is supported by our experiments, as shown in Table 3.
EXPERIMENTS
To evaluate our method, we have structured this section into four parts: 1) Detailed description of
the datasets and baseline methods used in this paper within FSSL (Sec 4.1). 2) The main results that demonstrate the efficacy our proposed method (Sec 4.2). 3) Extensive evaluations Twin-sight to another scenario in FSSL, where all clients possess partially labeled data (Sec 4.3).
4.1 EXPERIMENTAL SETUP
Datasets. In our experiments, we use four popular datasets that have been extensively utilized in FSSL research (Liang al., 2022: Wei & Huang, 2023) including CIFAR-I0 (Krizhevsky al., 2009), SVHN (Netzer et al., 201 1), Fashion-MNIS (FMNIST) (Xiao et al., 2017), and CIFAR-100 (Krizhevsky et al., 2009). The training sets of these four datasets are 50, 000, 73, 257, 60, 000, and 50, 000 respectively. They are partitioned into K clients in federated learning, and we resize all images 32 x size.
Federated Semi-supervised Learning Setting. 1) Data heterogeneity: To simulate data heterogene- ity, we partition the dataset across clients using the Latent Dirichlet Sampling (LDA) strategy (Hsu et al., 2019), with in Dir(7) controlling the label and quantity skewness of the data distribution among clients. In our experiments, we mainly use a severe non-IID setting with y = 0.1 (Fig. 2(b) (a) shows the data distribution across 10 clients ), which closely resembles real-wor world scenarios and is important for evaluating the effectiveness federated learning algorithms. 2) FSSL: We follow the setting of existing FSSL works (Liang al., 2022). Specifically, our federated learning (FI.) system

---

Published as a conference paper at ICLR 2024
Hederegennity clients
1.0
Cleat
Cloan
Gradient similarity
1/ M M M
Clest
0.4
4593
-0.4
Cloar
Clivr
Cteat
25
50
Iterations (a)
(b)
Figure 2: (a) The gradient similarity between two objective functions, i.e., defined on labeled and unlabeled data, throughout the training process. The figure demonstrates the gradient conflict. (b) Data heterogeneity under Dir(y 0.1). Each bubble indicates the number of y-th class at client k.
comprises K clients, among which M have access to fully-labeled training data, and T have only unlabeled data. The proportion fully-unlabeled clients, represented by the ratio a T constitutes K
a key factor determining the extent of annotation scarcity in Twin-sight, while (1 a1) K-T M
highlights the degree of label richness across the participating clients.
To verify the performance and robustness of Twin-sight, we compare against several methods, including the combination of semi-supervised and FL methods, as well as other state-of- the-art baseline methods in FSSL. 1) FedAvg (McMahan et al., 2017), trained only with labeled data as lower bound for comparison. 2) FedProx (Li et al., 2020), proposed to mitigate heterogeneous scenarios in FL. Fvgeee/g-Fi of two excellent methods in the respective fields of federated learning (FL) and semi-supervised learning (SSL. 4) FedProx+FixMatee (Li et al., 2020; Sohn et al., 2020), revise federated learning strategy to fit into heterogeneity. 5) FedAvg-Freematch (MeMahan al., 2017; Wang et al., 2023), vanilla FL method deployed with SOTA semi-supervised framework. FFFcdPoxx-Freematc (Li et al., 2020; Wang et al., 2023), a combination of two methods too. 7) Fed-Consist (Yang et al., 2021), use consistency loss computed by augmented data. 8) FedIRM (Liu et al., 2021c), relation matching scheme between fully-labeled clients and fully-unlabeled clients. 9)RSCFed (Liang et al., 2022), randomly sub-sample for sub-consensus.
Baselines.
(McMahan et al., 2017; Sohn et al., 2020), the combination
Implementation Details. Similar to many works (Tang et al., 2022; Wei & Huang, 2023; Huang al., 2024), baselines to ensure fair comparison. In federated learning, we aggregate weights in a FedAvg (McMahan et al.. 2017) manner. In accordance with previous works (Liang al., 2022), all of our experimental results report on the performance of the global model after R = 500 rounds of training. The server randomly samples a subset of all clients which means |C,| = 5 when clients number K 10, namely the sampling rate 5 = 50%. The random seed in our experiments is O. We use the SGD optimizer with a learning rate of 0.01, weight decay 0.0001, and momentum 0.9 all of experiments. The batch size is set to 64 for all datasets.
Resnet-18 (He et al., 2016) backbone feature extractor on all datasets and
4.2 MAIN RESULTS
The experimental results for Twin-sight on CIFAR-10, SVHN, FMNIST, and CIFAR-100 are pre- sented in Table and Table 2. The experiments were conducted using the same random seed, with 6 out of 10 clients randomly selected to be fully-unlabeled clients while the remainder were fully- labeled clients, namely a = 60%. and we select clients per communication round (S = 50%) in FL system. Overall. the performance of both the baseline methods and Twin-sight is lower than the upper bound of FedAvg. However, our proposed method, "Twin-sight," demonstrates a significant improvement in performance, outperforming all baselines, indicating successful mitigation of the
7

---

Published as a conference paper at ICLR 2024
Table 1: The performance of Twin-sight is compared to state-of-the-art (SOTA) methods CIFAR-10 and CIFAR-100, with 7 = 0.1 and K = 10.
No. Fully-laheled Chienssally-uniihhlle Clients
CIFAR-10
CIFAR-IO 100
Merhod
Labeled Clients (M)
Unlabeled Clients (T)
Ace1
Round]
Azc
Round]
Vanilla FL. method
FodAvg-U/pper Bound (McMahan et ol., 2017)
10
82.78
64.45
FodAvg-Lower Lowee Bound
4
0
61.58
295
48.36
469
FodPPoxt-Loo Loower Bound (Li di al,, 2020)
4
63.66
168
44.64
Nome
Combination of FL and SSL method
FolAvg+FiaMaute Match (McMalan et all, 2017; Solm et all, 2020)
4
63.58
207
48,73
315
FedProx PixMatch (Li et al,, 2020; Sohn et al., 2020)
4
62.44
269
43.61
None
ForAugt*reemmach (McMahan et al., 2017; Wang et 2023)
4
58.47
None
48.67
417
FedProx #reemm-+e (1 2000; Wang et al.,2022) 2023)
Existing FSSL, method
Fed- Consist (Yang 2021)
4
62.42
47.31
None
FodIRM M all, 2021c)
(Liang al., 2022)
Twin-sight (Ours)
70.06
49.98
The performance of FedAvg-Lovw Lower Bound is the target accuracy. "Round" refers to the communication round required reach the target accuracy. "None" indicates that this method did attain the target accuracy throughout the entire training period. The bold indicates the best result, while the underlined represents the runner-up.
Table The performance of Twin-sight is compared to state-of-the-art (SOTA) methods on SVHN and FMNIST, with 0.1 and K 10.
No. Fully-labeled ClienssFulll-umllbede Clients
SVHN
FMNIST
Method
Labeled Clients (M)
Unlaheled Clients (T)
Acct
Round]
Accf
Round]
FedAvg-Uppe Upper Bound
10
87.34
88.98
FedAvg-Lower Bound
4
51.10
70
72.46
172
FedProx-Lower Bound
4
49.22
None
70.71
None
FedAvg+FixMutch
4
58.68
35
67.52
None
FedProx+FiMMatch
4
45.58
None
63.20
None
Fed/vg=Frrematch
4
59.74
45
63.10
None
FedProx
4
50.91
None
69.62
None
Fed-Consist
56.87
103
68.51
None
RSCFed
4
54,50
69
76.58
88
Twin-sight (Ours)
4
62.94
125
79.95
140
gradient conflict. Specifically, our method achieves excellent results all datasets, with particularly notable improvement on CIFAR-10.
Despite its potential advantages, RSCFed did not exhibit superior performance compared to methods due to the presence of gradient conflict (see Figure However the combination of FedAvg (McMahan al., 2017) and Fixmatch (Sohn et al., 2020) Freematch (Wang al., 2023) achieved comparable performance in certain scenarios, leveraging two fundamental methods from different fields despite its simplicity. Moreover, FedIRM results NaN loss when used in severely skewed label distributions.
4.3 PARTIALLY LABELED DATA SCENARIO
Furthermore, we explore the scenario where clients have partially labeled data. To quantify the availability of labeled data for each client, introduce which represents the labeled data ratio, indicating the proportion of labeled data available. addition to the vanilla FL method and the combination of FL and Semi-supervised learning (SSL.) methods used in the previous setting,
8

---

Published as a conference paper at ICLR 2024
Table 3: The performance of Twin-sight is compared to state-of-the-art (SOTA) methods CIFAR-10,
CIFAR-100 100,,100,.0 SVHN and FMNIST in another scenario with 0.1 and K 10.
Method
ratio
Unlabeded data ratio
CIFAR-10
CIFAR-100
FMNIST
Vanilla method
FedAvg-Upper Bound
100%
0%
82.78
87.34
64.45
88.39
FedAvg-Lower Bound
5%
0%
45.35
37.81
19.46
75.21
FedProx-Lower Bound
5%
0%
45.44
27.34
20.47
79.77
Combination of FL and SSL method
Fedtvg-FixMutch
95%
74.97
64.44
33.58
75.62
FedProx+FixMatch
5%
95%
60.89
67.34
23.01
81.09
FodAvg-FFrematch
5%
95%
75.47
68.43
44.16
74.78
FedProx+Freematch
5%
95%
64.73
69.01
31.78
76.75
Existing FSSL method
FedSem (Albuseer al., 2020)
5%
95%
43.17
63.41
20.11
76.87
FedSiam (Long et al,, 2020h)
5%
95%
47.05
57.18
21.25
80.53
FedMatch (Jeong et al., 2021)
5%
95%
52.86
69.08
23.64
80.16
Twin-sight (Ours)
5%
95%
78.89
73.24
45.62
80.11
incorporate three additional methods specifically designed for this partially labeled data scenario, FedSem (Albaseer et al.. 2020), FedSiam (Long al., 2020b), and FedMatch (Jeong al., 202 I). The results presented in Table 3 highlight the remarkable improvements achieved by Twin-sight in the new scenario, with the exception of the FMNIST dataset. The observed performance difference in the FMNIST dataset could be attributed to the fact that the algorithm's performance has reached a bottleneck when trained on only 5% of the available data. However, Twin-sight still demonstrates comparable results with other methods the FMNIST dataset.
5 CONCLUSION
In this work, we present Twin-sight, a novel twin-model paradigm designed to address the challenge of label deficiency in federated learning (FL). There three key factors contributing the improvement of Twin-sight. First of all, we decouple the learning objective into two models which avoids gradient conflicts. In the most important part, twin-sight interaction, our unsupervised model conducts an instance classification task which is fine-grained classification problem. Namely, this task would contribute the downstream classification tasks (Mitrovic et al., 2020). Moreover, the data, model, and the objective function consistent among all clients. Lastly, supervised model conducts classification task. Furthermore, the data, model, and objective functions are consistent among all clients, except for some unlabelled data paired with pscudo labels.
The twin-model paradigm introduces an additional model, which can potentially increase memory and communication overhead in federated learning (FL). As part of our future work, we aim to explore a memory -rrenddl dual-model paradigm that addresses these concerns.
Limitation
Future works Currently, few existing methods effectively address multiple FSSL scenarios. Therefore, future research should focus on proposing multi-scenario generalization and robust methods capable of handling FSSL problems in various situations. Furthermore, it is essential to consider communication overhead, computation overhead, and performance in the experimental evaluations provide diverse solutions that cater the different requirements of cross-silo and cross-device scenarios.
ETHIC STATEMENT
This paper does not raise any ethical concern.. This study does not involve any human subjects, practices to data set releases, potentially harmful insights, methodologies and applications, potential conflicts of interest and sponsorship, Anserimination//iaa/aaaneee concerns, privacy and security issues, legal compliance, and research integrity issues.

---

Published as a conference paper at ICLR 2024
REPRODUCIBILITY STATEMENT
To make all experiments reproducible, we have listed all detailed hyper-parameters of each FL algorithm. Due privacy concerns, will upload the anonymous link of source codes and instructions during the discussion phase to make only visible to reviewers.
ACKNOWLEDGMENTS AND DISCLOSURE OF FUNDING
Yonggang Zhang, Zhiqin Yang and Bo Han were supported by the NSFC General Program No. 62376235, Guangdong Basic and Applied Basic Research Foundation No. 10221151501155. HKBU Faculty Niche Research Areas No. CCEENRA-IG/2-23/SCIIO CCF-Baidu Open Fund, and HKBU CSD Departmental Incentive Scheme. Xinmei Tian was supported part by NSFC No. 62222117, the Fundamental Research Funds for the Central Universities under contract WK3490000005, and KY2100000177. Nannan Wang was supported part by the National Natural Science Foundation of China under Grants U22A2096. Tongliang Liu is partially supported by the following Australian Research Council projects: FT220100318, DP220102121, LP220100527, LP220200949, and IC190100031.
REFERENCES
Abdullatif Albaseer, Bekir Sait Ciftler, Mohamed Abdallah, and Ala Al-Fuqaha. Exploiting unlabeled data in smart cities using federated learning. arXiv preprint arXiv:2001.0400 2020.
Philip Bachman, R Devon Hjelm, and William Buchwalter. Learning representations by maximizing mutual information across views. Advances in neural information processing systems, 32, 2019.
Mikhail Belkin and Partha Niyogi. Semi-supervised learning on riemannian manifolds. Machine learning, 56:209-239, 2004.
David Berthelot, Nicholas Carlini, lan Goodfellow, Nicolas Papernot, Avital Oliver, and Colin A Raffel. Mixmatch: A holistic approach to semi-supervised learning. Advances in neural information processing systems, 32, 2019.
David Berthelot, Nicholas Carlini, Ekin D Cubuk, Alex Kurakin, Kihyuk Sohn, Han Zhang, and Colin Raffel. Remixmatch: Semi-supervised learning with distribution alignment and augmentation anchoring. In ICIRR 2020.
Krzysztof Chalupka, Pietro Perona, and Frederick Eberhardt. Visual causal feature learning. arXiv preprint arXiv:1412.2330 2014.
Ting Chen, Simon Kornblith, Mohammad Norouzi, and Geoffrey Hinton. A simple framework for contrastive learning of visual representations. In International conference on machine learning, pp. 1597-1607, 2020.
Xinlei Chen and Kaiming He. Exploring simple siamese representation learning. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 15750-15758, 2021.
Enmao Diao, Jie Ding, and Vahid Tarokh. Semifl: Semi-supervised federated learning for unlabeled clients with alternate training. Advances in Neural Information Processing Systems, 35:17871- 17884, 2022.
Yiqun Diao, Qinbin Li, and Bingsheng He. Towards addressing label skews in one-shot federated learning. In The Eleventh International Conference on Learning Representations, 2023.
Jin Gao, Jialing Zhang, Xihui Liu, Trevor Darrell, Evan Shelhamer, and Dequan Wang. Back to the source: Diffusion-driven test-time adaptation. In CVPR, 2023.
Sean-Bastien Grill, Florian Strub, Florent Altché, Corentin Tallec, Pierre Richemond, Elena Buchatskaya, Carl Doersch, Bernardo Avila Pires, Zhaohan Guo, Mohammad Gheshlaghi Azar, et al. Bootstrap your own latent-a new approach to selervuue learning. Advances in neural information processing systems, 33:21271-21288 2020.
10

---

Published as a conference paper at ICLR 2024
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 770-778, 2016.
Kaiming He, Haoqi Fan, Yuxin Wu, Saining Xie, and Ross Girshick. Momentum contrast for unsupervised visual representation learning. In Proceedings of the IEEE/CVF conference on computer vision and patlern recognition, pp. 9729-9738, 2020.
Tzu-Ming Harry Hsu, Hang Qi, and Matthew Brown. Measuring the effects of non-identical data distribution for federated visual classification. arXiv preprint
2019.
Weidong Huang, Jiaming Ji. Borong Zhang, Chunhe Xia, and Yaodong Yang. Safedreamer: Safe reinforcement learning with world models. In The Twelfth International Conference on Learning Representations, 2024.
Wonyong Jeong, Jaehong Yoon, Eunho Yang, and Sung Ju Hwang. Federated semi-supervised learning with inter-client consistency & disjoint learning. In 9th International Conference on Learning Representations, 2021.
Shaoxiong Ji, Yue Tan, Teemu Saravirta, Zhiqin Yang, Lauri Vasankari, Shirui Pan, Guodong Long, and Anwar Walid. Emerging trends in federated learning: From model fusion to federated x learning. arXiv preprint arXiv:2102.129200 2023.
Yilun Jin, Yang Liu, Kai Chen, and Qiang Yang. Federated learning without full labels: A survey. arXiv preprini arXiv:2303. / /4453, 2023.
Peter Kairouz, H Brendan McMahan, Brendan Avent, Aurélien Bellet, Mehdi Bennis, Arjun Nitin Bhagoji, Kallista Bonawitz, Zachary Charles, Graham Cormode, Rachel Cummings, et al. Ad- vances and open problems in federated learning. Foundations and Trends@ in Machine Learning, 14(1-2):1-210, 2021.
Sai Praneeth Karimireddy, Satyen Kale, Mehryar Mohri, Sashank Reddi, Sebastian Stich, and Ananda Theertha Suresh. Scaffold: Stochastic controlled averaging for federated learning, In International Conference on Machine Learning, pp. 5132-5143, 2020.
Alex Krizhevsky, Geoffrey Hinton, et al. Learning multiple layers of features from tiny images. Technical report, University of Toronto, 2009.
Dong-Hyun Lee et al. Pseudo-label: The simple and efficient supi-super learning method for deep neural networks. In Workshop on challenges in representation learning. ICML, 2013.
Qinbin Li, Zeyi Wen, Zhaomin Wu, Sixu Hu, Naibo Wang. Yuan Li, Xu Liu, and Bingsheng He. A survey on federated learning systems: vision, hype and reality for data privacy and protection. IEEE Transactions on Knowledge and Data Engineering, 2021.
Qinbin Li, Yiqun Diao, Quan Chen, and Bingsheng He. Federated learning on non-iid data silos: An experimental study. In 2022 IEEE 38th International Conference on Data Engineering (ICDE), pp. 965-978, 2022.
Tian Li, Anit Kumar Sahu, Manzil Zaheer, Maziar Sanjabi, Ameet Talwalkar, and Virginia Smith. Federated optimization in heterogeneous networks. Proceedings of Machine Learning and Systems, 2:429-450 2020.
Wenqi Li, Fausto Milletari, Daguang Xu, Nicola Rieke, Jonny Hancox, Wentao Zhu, Maximilian Baust, Yan Cheng, Sébastien Ourselin, M Jorge Cardoso, al. Privacy-preserving federated brain tumour segmentation. In Machine Learning in Medical Imaging: 10th International Workshop. MLMI 2019, Proceedicss 10. Springer, 2019.
Xiaoxiao Liang, Yiqun Lin, Huazhu Fu, Lei Zhu, and Xiaomeng Li. Rsefed: random sampling consensus federated semi-supervised learning. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pp. 10154-10163, 2022.
Haowen Lin, Jian Lou, Li Xiong, and Cyrus Shahabi. Semifed: Semi-superviser federated learning with consistency and pseudo-labeling. arXiv preprint arXiv: 108.00//8 2021.
11

---

Published as a conference paper at ICLR 2024
Tao Lin, Lingjing Kong, Sebastian U Stich, and Martin Jaggi. Ensemble distillation for robust model fusion in federated learning. Advances in Neural Information Processing Systems, 33:2351-2363, 2020.
Bo Liu, Xingchao Liu, Xiaojie Jin, Peter Stone, and Qiang Liu, Conflict-averse gradient descent for multi-task learning, Advances in Neural Information Processing Systems, 34:18878-18899, 2021a.
Hong Liu, Jianmin Wang, and Mingsheng Long. Cycle self-training for domain adaptation. Advances in Neural Information Processing Systems, 34:22968 12981, 2021b.
Quande Liu, Hongzheng Yang, Qi Dou, and Pheng-Ann Heng. Federated semi-supervised medical image classification via inter-client relation matching. In Medical Image Computing and Computer Assisted Intervention (MICCAI), pp. 325-335, 2021c
Guodong Long, Yue Tan, Jing Jiang, and Chengqi Zhang. Federated learning for open banking. In Federated Learning: Privacy and Incentive, pp. 240-254. Springer, 2020a.
Zewei Long, Liwei Che, Yaqing Wang, Muchao Ye. Junyu Luo, Jinze Wu, Houping Xiao, and Fenglong Ma. Fedsiam: Towards adaptive federated semi-supervised learning. arXiv preprint WwXiv:2012.03322. 2020b.
Mi Luo, Fei Chen, Dapeng Hu, Yifan Zhang, Jian Liang, and Jiashi Feng. No fear of heterogeneity: Classifier calibration for federated learning with non-iid data. Advances in Neural Information Processing Systems, 34:5972-5984, 2021.
Brendan McMahan, Eider Moore, Daniel Ramage, Seth Hampson, and Blaise Aguera y Arcas. Communication-effiiceee learning of deep networks from decentralized data. In Artificial intelli- gence and statistics, pp. 1273-1282, 2017.
Ishan Misra and Laurens van der Maaten. Self-supervised learning of pretext-invariant representations. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 6707- 6717, 2020.
Jovana Mitrovic, Brian McWilliams, Jacob Walker, Lars Buesing, and Charles Blundell. Representa- tion learning via invariant causal mechanisms. arXiv preprint arXiv:2010.07922, 2020.
Yuval Netzer, Tao Wang, Adam Coates, Alessandro Bissacco, Bo Wu, and Andrew Y Ng. Reading digits in natural images with unsupervised feature learning. In NIPS Workshop, 2011.
Aaron van den Oord, Yazhe Li, and Oriol Vinyals. Representation learning with contrastive predictive coding. arXiv preprint /rXiv:/807.0311 1807.03748 2018.
Prashant Pandey, Mrigank Raman, Sumanth Varambally, and Prathosh Ap. Generalization on unseen domains via inference-time label-preserving target projections. In CVPR, 2021.
Sam T Roweis and Lawrence K Saul. Nonlinear dimensionality reduction by locally linear embedding. science, 29065500) 2000.
Anindya Sarkar, Anirban Sarkar, and Vineeth N Balasubramanian. Leveraging test-time consensus prediction for robustness against unseen noise. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1839-1848, 2022.
Kihyuk Sohn, David Berthelot, Nicholas Carlini, Zizhao Zhang, Han Zhang, Colin A Raffel, Ekin Do- gus Cubuk, Alexey Kurakin, and Chun-Liang Li. Fixmatch: Simplifying semi semi-supervised learning with consistency and confidence. Advances neural information processing systems, 33:596-608, 2020.
Zhenheng Tang, Yonggang Zhang, Shaohuai Shi, Xin He, Bo Han, and Xiaowen Chu. Virtual homogeneity learning: Defending against data heterogeneity in federated learning. International Conference Machine Learning, pp. 21111-21132, 2022.
Antti Tarvainen and Harri Valpola. Mean teachers are better role models: Weight- aveggt-averaged consistency targets improve semi-supereised deep learning results. Advances in neural information processing systems, 30, 2017.
12

---

Published as a conference paper at ICLR 2024
Haobo Wang, Ruixuan Xiao, Yiwen Dong, Lei Feng, and Junbo Zhao. Promix: Combating label noise via maximizing clean sample utility, arXiv preprint arXiv:2207.10276 2022.
Jianyu Wang, Qinghua Liu, Hao Liang, Gauri Joshi, and H Vincent Poor. Tackling the objective inconsistency problem in heterogeneous federated optimization. Advances in neural information processing systems, 33:7611-7621) 2020.
Yidong Wang, Hao Chen, Qiang Heng, Wenxin Hou, Yue Fan, Zhen Wu, Jindong Wang, Marios Savvides, Takahiro Shinozaki, Bhiksha Raj, et al. Freematch: Self-adaptive thresholding for semi-supervised learning. In Eleventh International Conference on Learning Representations. OpenReview. net, 2023.
Zirui Wang and Yulia Tsvetkov. Gradient vaccine: Investigating and improving multi-task optimiza- tion massively multilingual models. In Proceedings of the International Conference on Learning Representations (ICLR), 2021.
Xiao-Xiang Wei and Hua Huang. Balanced federated semi-supervised learning with fairness- aware pseudo-labeling, IEEE Transactions on Neural Networks and Learning Systems, 2023.
Zhirong Wu, Yuanjun Xiong, Stella X Yu, and Dahua Lin. Unsupervised feature learning via non- parametric instance discrimination. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 3733-3742, 2018.
Han Xiao, Kashif Rasul, and Roland Vollgraf. Fashion-mnist: a novel image dataset for benchmarking machine learning algorithms. arXiv preprint arXiv:1708.07747, 2017.
Qizhe Xie, Zihang Dai, Eduard Hovy, Thang Luong, and Quoc Le. Unsupervised data augmentation for consistency training. Advances in neural information processing systems, 33:6256-6268, 2020a.
Qizhe Xie, Minh-Thang Luong, Eduard Hovy, and Quoe V Le. Self-training with noisy student improves imagenet classification. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 10687-10698, 2020b.
Jie Xu, Benjamin S Glicksberg, Chang Su, Peter Walker, Jiang Bian, and Fei Wang. Federated learning for healthcare informatics. Journal of Healthcare Informatics Research, 5:1-19, 2021.
Dong Yang, Ziyue Xu, Wenqi Li, Andriy Myronenko, Holger R Roth, Stephanie Harmon, Sheng Xu, Baris Turkbey, Evrim Turkbey, Xiaosong Wang, al. Federated semi-supervised learning for covid region segmentation in chest ct using multi-national data from china, italy, japan. Medical image analysis, 10:101992, 2021.
Qiang Yang. Yang Liu, Tianjian Chen, and Yongxin Tong. Federated machine learning: Concept and applications. ACM Transactions Intelligent Systems and Technology (TIST), 2019.
Zhiqin Yang. Yonggang Zhang. Yu Zheng, Xinmei Tian, Hao Peng, Tongliang Liu, and Bo Han. Fedfed: Feature distillation against data heterogeneity in federated learning. In Thirty-seventh Conference on Neural Information Processing Systems, 2023.
Tianhe Yu, Saurabh Kumar, Abhishek Gupta, Sergey Levine, Karol Hausman, and Chelsea Finn. Gradient surgery for multi-task learning. Advances in Neural Information Processing Systems, 33: 5824-5836, 2020.
Yonggang Zhang, Mingming Gong, Tongliang Liu, Gang Niu, Xinmei Tian, Bo Han, Bernhard Schölkopf, and Kun Zhang. Adversarial robustness through the lens of causality. In International Conference Learning Representations, 2022.
Jianing Zhu, Jiangehao Yao. Tongliang Liu, Jianliang Xu, Bo Han, al. Combating exacerbated heterogeneity for robust models in federated learning. In The Eleventh International Conference on Learning Representations, 2022.
Xiaojin Zhu and Andrew B Goldberg. Introduction to supervipe learning. Synthesis lectures artificial intelligence and machine learning, 3(1):1-130, 2009.
13

---

Published as a conference paper at ICLR 2024
Xiaojin Zhu, Zoubin Ghahramani, and John D Lafferty. Semi- Semi-supervised learning using gaussian fields and harmonic functions. In Proceedings of the 20th International conference on Machine learning (ICML-03), pp. 912-919, 2003.
Weiming Zhuang, Yonggang Wen, and Shuai Zhang. Divergence-aware federated self-supervised learning. In International Conference on Learning Representations, 2021.
Barret Zoph, Golnaz Ghiasi, Tsung- Yi Lin, Yin Cui, Hanxiao Liu, Ekin Dogus Cubuk, and Quoe Le. Rethinking pre-training and self self-training. Advances in neural information processing systems, 33: 3833-3845, 2020.
14

---

Published as a conference paper at ICLR 2024
A EXPERIMENTAL DETAILS
A. THE SYMBOLIC REPRESENTATION
In this section, we give description of the symbol in our paper in Table 4.
Table 4: The Symbolic description.
Symbolic representation
Description
3
the aggregation weight of client k the set of all clients
C
M
the number of fully-labeled clients the number of fully-unlabeled clients the number of all clients ((CI)
K
R
the communication round the sampling rate the local epochs the non-iid degree the proportion of fully-unlabeled clients the ratio of labeled data of each client
S
E
7
A.2 DATA VISUALIZATION
Figure 3: The data distribution of different clients under SVHN.
In this section, we present the data distribution of different clients on the SVHN dataset after performing LDA partition with parameter value of 0.1. The visualization clearly illustrates that each client predominantly contains samples from a specific class, indicating a significant concentration of data within individual client distributions. We also visualize another dataset CIFAR- 10 in Figure which contain both the distribution of clients and samples of fully-labeled client and fully- fully-unlabeled client.
15

---

Published as a conference paper at ICLR 2024
airplane
5000
autumubile
One Labeled Clicat
Unlabeled Amount
4000
Unlabeled
30h0
ship
2009
bird
Iner
barse
frgg
deer
Figure The blue and black areas the figure correspond to the total amount of labeled and unlabeled data, respectively. The other two areas in the figure show the class distribution of a labeled client and an unlabeled client. This figure reveals that the challenges faced by FSSL are not limited to the problem of label scarcity, but also include the impact of data heterogeneity.
B MORE EXPERIMENTAL RESULTS
B. HOW DIFFERENT DATA UETEROGENEITY AFFECT TWIN-SIGHT?
In this section, we aim to assess the robustness and generalization capability of FSSL under varying levels of data heterogeneity. In addition to considering the degree of 0.1 in the main paper, also explore a more severe heterogeneous scenario 7 0.05, similar to what conventional methods have investigated in prior works (Luo et al., 2021; Li al., 2022). In this experiment, a consistent sampling rate and communication round as in the previous experiments. Specifically, we applied sampling rate (S) of 50%, which involved randomly selecting clients of the total 10 clients. Additionally, we conducted a total of 500 communication rounds (It) during the experiment.
maintained
Table 5: The performance of Twin-sight is compared state-of-the-art (SOTA) methods on CIFAR-10 and SVHN, with = 0.05 and K 10.
No. Fully-labeled CliemtsFFlly-umllbeleed Clients
CIFAR-10
SVHN
Method
Labeled Clients (M)
Unlaheled Clients (T)
Acef
Round]
Accf
Round]
FedAvg-Upper Bound
10
75.34
79.13
FedAvg-Lower Bound
35.21
32.26
FedProx-Lower Bound
39.02
33.92
FedAvg-FixMatch
4
40.18
263
35.32
405
FedProx+FixMatch
4
29.61
None
20.93
None
Fed/vg=FFremaacc
4
33.98
None
32.68
361
FedProx+Frrematch
33.26
None
24.68
None
Fed-Consist
4
37.19
301
31.76
None
RSCFed
4
38.96
73
40.16
354
Twin-sight (Ours)
43.82
43.22
The results the experiments conducted under different data heterogeneity degrees are presented in Table 5 and Table 6. When analyzed in conjunction with the results from Table and Table 2, these findings highlight the impact data heterogeneity on the performance of the evaluated methods. As the non-iid degree (7) decreased, indicating the performance of all methods showed decline. However, it is worth noting that our approach demonstrated slower rate of performance decline compared to the other methods. These results
severe level of data heterogeneity among clients,
16

---

Published as a conference paper at ICLR 2024
Table 6: The performance of Twin-sight is compared state-of-the-art (SOTA) methods CIFAR-
100 and FMNIST, with 7 = 0.05 and K = 10.
No. Fabll-l Chients/Fully-unlbbl Clients
CIFAR-100
SVHN
Method
Labeled Clients (M)
Unlabeled Clients (T)
Acef
Round]
Acet
Round]
FedAvg-Upper Bound
10
62.01
78.34
FedAvg-Lower Bound
41.57
47.10
FedProx-Lower Bound
39.41
None
48.5
FedAvg=FixMatch
4
43.19
222
43.27
None
FedProx+FixMatch
4
37.92
None
42.58
None
FedAvg=Freematch
42.96
273
42.16
None
FedProx+Freematch
35.77
None
41.63
None
Fed-Consist
4
41.67
379
40.73
None
RSCFed
4
43.48
200
48.5
159
Twin-sight (Ours)
43.57
50.6
176
suggest that our method exhibits greater robustness and resilience in the face of increasing data heterogeneity.
B.2 How TO DETERMINE THE SELF-SUPERVISED METHOD?
60
Test Accuracy [%)
40
20
Twin-sight with SimCLR Twin-sight with BYOL
RSCFed
FedAvg- FixMatch
Twin-sight with SimSiam
100
200
300
400
500
Round
Figure 5: Different self-supervised model with our methods under the setting of y 0.1, E
K 10, CIFAR-10.
To verify the performance of Twin-sight with different sule-super methods, namely BYOL and SimSiam in addition to SimCLR, we conducted experiments and analyzed the results shown in Figure 5. The results indicate that Twin-sight exhibits competitive performance regardless of the specific self supe-supe method employed. While SimCLR achieved the best performance, Twin-sight still demonstrated strong performance when using B BOLL SimSiam.
This suggests that Twin-sight is adaptable to different sulf-supervised methods and can effectively leverage their benefits within the twin-sight framework. The ability of Twin-sight to achieve favourable results across multiple self-supervised approaches highlights its versatility and robustness in incorporating different self-supervised learning techniques.
17

---

Published as a conference paper at ICLR 2024
Table 7: Comparison of Twin-sight's performance with different sampling rates (S) on the CIFAR-10 dataset, under the setting of y = 0.1 and K = 10 clients.
50%
40%
30%
20%
Method
Ace
Round]
Acc f
Round +
Acc 1
Round
Acc
Round].
FedAvg-Lower Bound
61.58
295
60.78
429
51.21
474
49.76
489
FedProx-Lower Bound
63.66
64.33
285
59.14
133
55.91
271
Fidavg+FixMatch
63.58
207
63.14 14
392
57.21
368
52.12
455
FedProx+FixMatch
62.44
269
60.12
None
56.39
399
51.79
398
FeddAvg-Freemmatch
58.47
None
56.71
None
50.95
None
51.18
498
FedProx-Freematch
59.28
None
57.34
Nonc
61.53
223
54.98
280
Fed-Consist
62.42
231
61.09
462
54.14
405
50.35
465
RSCFed
60.78
None
59.04
None
58.24
73
53.71
267
Twin-sight (Ours)
70.06
115
69.15
214
63.84
134
61.52
226
B.3
WHAT WILL HAPPEN WHEN THE SAMPLING RATE (S) IS SMALL?
In real-world federated learning setting, it is common for not all clients to be continuously online due to various factors such as network connectivity issues, power constraints, or intermittent availability. To simulate this on-and-off situation, we randomly select subset of clients denoted as C, from the complete set of clients C' = {cu 1K In each communication round, we determine the fraction of clients to be sampled using parameter called the sampling rate (S). This sampling rate represents the proportion of clients that are randomly selected from the entire client pool C' for participation in that specific round.
If the available clients are small, what will happen? To investigate the scenario where only a small number of clients are available, different sampling rates (S). In this case, we reduced the sampling rate from the commonly used 50% 20% among the 10 available clients with R. 500 communication rounds. By decreasing the sampling rate to 20%, we simulated a scenario where a smaller fraction of clients actively participated in each communication round. This situation reflects scenarios where federated learning carried out with limited client availability.
conducted exploratory experiments the CIFAR-10 dataset using
The results are presented Table Upon analysis, it evident that the sampling rate decreases, which corresponds to smaller number of clients being sampled in each round, there are noticeable impacts on both the overall performance and the convergence speed of the global model. However, despite the reduced client participation, our method consistently maintains a significant performance gain when compared to other baseline methods. For instance, when transitioning from 50% to a 40% sampling rate, the performance degradation of our method remains below 1%. This indicates that even with a reduction in client availability, our method effectively mitigates the performance drop, ensuring robust results. Moreover, when the sampling rate decreases to 20%. method demonstrates sustained performance level above 60%, further highlighting its robustness and effectiveness.
B.4 DOES TWIN-SIGHT REMAIN EFFECTIVE ACROSS DIFFERENT NUMBERS CLIENTS (K)?
The total number of clients in a federated learning (FL. system poses potential challenge for Twin-sight. As the number of clients increases, each client possesses smaller amount of data, reflecting the cross-device scenario in the simulated federation. In this setting, different devices have limited data availability, and some devices lack the capability to label data, resulting in unlabeled data for those devices.
To assess its effectiveness in scenarios with varying client numbers, we conducted experiments and evaluated the results. The outcomes are presented in Table and Table 9 under CIFAR-10. To simulate a scenario with large number of clients, we selected 50 clients, of which 60% had no labeled data while the remaining clients had labeled data available. In this setup, sampled out
18

---

Published as a conference paper at ICLR 2024
Table 8: The performance of Twin-sight is compared state-of-the-art (SOTA) methods CIFAR-
10, with y 0.1, K = 50 and a 60%.
No. Fully-labeled ClientsFulll-uulabbele Clients
50 clients
Method
Labeled Clients (M)
Unlabeled Clients (T)
Acc t
Round
FedAvg-Lower Bound
20
30
16.31
348
FedProx-Lower Bound
20
30
32.31
44
Fedtvg+FFiMatch
20
30
41.21
13
FedProx+FixMatch
20
30
54.74
7
FedAvg+FFreematch
20
30
21.52
345
FedProx+FFrematthh
20
30
37.89
134
Fed-Consist
20
30
36.41
78
RSCFed
20
30
57.96
14
Twin-sight (Ours)
20
30
62.2
13
Table 9: The performance of Twin-sight compared to state-of-the-an (SOTA) methods on CIFAR 10, with y 0.1, K 100 and 60%.
No. Fully-labeled ClientsFuull--unlbellel Clients
100 clients
Method
Labeled Clients (M)
Unlabeled Clients (T)
Acc
Round
FedAvg-Lower Bound
40
60
28.94
495
FedProx-Lower Bound
40
60
44.91
132
Fixttc
40
60
45.62
66
FedProx=FixMatch
40
60
54.93
35
FedAvg=-Freematch
40
60
31.47
383
FedProx*FFreemath
40
60
47.61
217
Fed-Consist
40
60
43.31
172
RSCFed
40
60
52.61
16
Twin-sight (Ours)
40
60
56.78
35
of 50 clients to participate in the FL process. Similarly, we also sampled out of 100 clients to simulate a scenario with a larger client pool (S = 10%).
In the scenario with 50 clients from Table 8, our method manages to achieve performance 62.2. Despite the reduced amount of data per client, our method demonstrates its effectiveness in leveraging the available labeled data and effectively utilizing the unlabeled data from devices that lack labeling capabilities. However, the impact on performance. The limited amount of data available for each client poses significant challenge, potentially affecting the overall performance of the system.
observe from Table 9, the reduction in client data further exacerbates
B.5 How DOES THE PERFORMANCE OF TWIN-SIGIT VARY WITH DIFFERENT RATIOS OF UNLABELED CLIENTS?
To evaluate the robustness of our approach, conducted experiments with varying ratios of unlabeled clients, ranging from a = 90% to a 40%. The results, illustrated in Figure 6, demonstrate that the ratios of unlabeled clients have impact the performance of Twin-sight. However, method
19

---

Published as a conference paper at ICLR 2024
80
FedAvg-FixMMa FixMatch
Fed-Consist
FedPtox-FFxxMttch
RSCFed
70
Twin-sight (Ours)
Test Accuracy [%)]
60.
50
40
30
20.
10
90%
80%
70% Ratio of Fully-unlabeled Clients
60%
50%
40%
Figure 6: The effects of various ratios of unlabeled clients (a). Comparison between five baseline methods and Twin-sight.
consistently performs well and surpasses baseline methods in these scenarios. We observed that as the number of unlabeled clients increases, the overall performance tends to degrade. Despite this, Twin-sight continues to outperform other methods even in the presence of higher ratio of unlabeled clients. Furthermore, we noticed that Twin-sight achieves larger performance gains compared to other methods when the number of unlabeled clients is relatively small. This highlights the importance of labeled data in guiding the twin-sight interaction and improving overall performance. More experimental and ablation results can found Appendix
B.6 DOES THE RANDOM SEED AFFECT ROBUSTNESS?
Table 10: The mean and deviation performance with different random seeds.
Method
CIFAR-10
CIFAR-100
SVHN
FMNIST
FedAvg-Lower Bound
60.14 + 1.50
48.05 + 0.52
56.36 + 5.06
73.32 + 3.79
FedProx-Lower Bound
63.03 + 1.00
45.42 + 0.68
56.03 + 5.90
72.31 + 4.73
FedAvg+FixMMatc
63.87 + 0.73
48.84 0.70
56.29 + 2.32
67.25 + 2.79
FedProx+FixMatch
61.61 + 0.72
43.55 + 0.46
47.71 + 3.79
65.57 + 2.08
FedAvg+Freematchh
58.40 + 2.01
49.52 + 0.74
56.77 : 2.58
64.78 + 4.67
FedProx+Freematch
59.43 + 1.38
41.73 + 0.51
51.69 + 2.41
71.68 + 2.51
Fed-Consist
62.79 + 0.93
47.94 0.70
53.74 + 2.91
66.73 + 3.41
RSCFed
60.11 + 0.97
45.90 + 1.32
58.46 + 3.81
76.34 + 1.09
Twin-sight (Ours)
70.02 + 0.77 50.15 + 0.43
63.16 + 0.30
78.85 + 1.04
In this section, we aim to investigate the impact of different random seeds on the overall performance of the federated system. Random seed is essential for model initialization as well as client selection in each round, both of which can influence the final results. To assess the robustness of Twin-sight in the face of such variability, we select three different random seeds for each experiment and conduct tests accordingly. These experiments
conducted under ? 0.1, K 10 with sampling rate S 50%. The results are reported in Table 10. Notably, even when considering three random seeds, Twin-sight consistently outperforms other baseline methods across all four datasets.
20

---

Published as a conference paper at ICLR 2024
B.7 CAN INCREASING THE NUMBER OF COMMUNICATION ROUNDS (R) IMPROVE PERFORMANCE?
50
Test Accuracy [%)
40
RedArg-Lower Bound
FedProx-L.od Lower Bound
30
FedAvg FixMarch
SeddPoo-FFsMMan
FedAvg Freematch
FedPPoo-Frrommake Freematth
Red- Ped-Consist
RSCFed
Twin-sight (Ours)
20
100
200
300
400
500
600
700
800
900
1000
Communication Round (R)
Figure 7: The performance of different communication rounds in the scenarios of fully-labeled clients and fully-unlabeled clients under CIFAR-100, with 7 = 0.1, K 10 and 60%.
The performance of a federated system is jointly influenced by the number of communication rounds R and the sampling rate S, they determine the necessary communication bandwidth. To evaluate the effects of different communication round settings on performance improvement, we conducted experiments in two distinct scenarios. In scenario, subset of clients possessed fully-labeled data, while in the other scenario, all clients had partial labels for their data. The primary objective was to achieve balance between enhancing performance and managing communication volume effectively.
80
60
Test Accuracy [%]
40
FodAvg-Lower Bound
Lowe Bound
FedAvg-FixMacch FixMatch
FodProx-FixMaan
20.
FedAvg Freematch
FollPoo-Freematte
FedSem
FedMaich
FodSiam
Twin-sight (Ours)
100
200
300
400
500
600
700
800
900
1000
Communication Round (R)
Figure 8: The performance of different communication rounds in the scenarios of all clients hold data
with portion labeled under SVHN, with 7 = 0.1, 10 and T = 5%.
21

---

Published as a conference paper at ICLR 2024
As depicted in the Figure and Figure 8, is self-evident that when the number of rounds is relatively small, there is a substantial improvement in performance with each increment of 100 rounds. However, beyond 500 rounds, the performance improvement for most methods becomes less pronounced. Consequently, all the experiments presented in our paper were conducted within a maximum of 500 rounds. Our method exhibits rapid convergence and achieves excellent performance within the first 300 rounds. This observation demonstrates that Twin-sight not only enhances performance but also accelerates model convergence and reduces the number of required communication rounds. After 500 rounds, both FedAvg-Lowe Bound and FedProx-Loww improvements (in Figure 8). This improvement can be ascribed to the slower convergence speed of the two lower-bound methods in this dataset, primarily due to the limited availability of labeled data. As the iterations progress, the performance gradually improves, and significant enhancements become evident after several hundred rounds of iterations.
Bound exhibit substantial performance
C DETAILED EXPLANATIONS OF TWIN-SIGHT
In this section, we give more details and explanations of our method. Specifically, we introduce the gradient conflict problem in our paper, the sele-super method we investigate and the twin-sight interaction.
Gradient conffict v.s. client drift: Gradient conflict is defined as the inconsistency between gradients. The inconsistency may result from multi-objectives, e.g., multi-task learning (MTL) (Yu et al., 2020; Liu et al., 202 Wang & Tsvetkov, 2021). The objective of MTL is the average loss of all kinds of tasks which typically leads to gradient conflict. According the definition of gradient conflict, two gradients are considered conflictngg if they point away from one another, i.e., have a negative cosine similarity. This phenomenon also can be observed in Figure 2(a) our paper. The gradient conflict phenomenon has some drawbacks: i) this different loss may have various scales of gradient, and the largest one dominates the update direction; ii) The averaged objective can be quite unfavourable to a specific task's ' performance (Liu et al., 2021a). Client drift is caused by data heterogeneity while gradient conflict is typically induced by multiple objectives. In existing FSSL methods, the "client drift" and "gradient conflicts" often exist at the same time. In the context of label deficiency, gradient conflict may be attributed to both the samples and objectives. Specifically, partial samples have annotations and the objective on lableled and unlableled data are typically different. Advanced methods in FSSL typically use two different objective functions to train the local classifier, one for fully-labeled clients with cross-entropy loss and another for fully-unlabeled clients by unsupervised method. In this context, these methods suffer from gradient conflicts naturally.
The self-supversied method: In twin-sight, the ('(w., in (6) refers to the unsupervised model parameterized by w.. And the sim f. x))) denote the dot product between l normalized embedding of data and j. To investigate different unsupervised methods in twin-sight, we perform some prevailing unsupervised methods, such as SimCIR (Chen et al., 2020), BYO (Grill al., 2020) and SimSiam (Chen He, 2021), and SimCLR is chosen be the backbone of J J'(ww). The results of different self-supervised methods are reported in Appendix B.2.
The function N(-) and metric d(-): The function N(-) is leveraged to quantify the relationship between different samples in mini-batch. Specifically, N(-) is the matrix calculating the distance among samples in the feature space, i.e., outputs of supervised model Z, E Raxd and those of unsupervised model 7,, Rand with d being the dimension and n the batch size. Using these outputs, we can employ the matrix M E TRax*to represents the relationships among samples, i.e.,
NJ (w,. x)) = M,- Z, zf, N(f(w,, == M,, = 7,, Z Meanwhile, the metric d(-) is realized as the mean square error, which is formulated as: d[N(f(we, N(f(N, (w)) [|M,- M, 12
22


Reviews:
[{"review_id": "3FoDRSeeyM", "reviewer": "Reviewer_r5bY", "summary": "The authors propose Twinsight, an approach to semi-supervised federated learning that addresses gradient conflicts that may result from training a single model on each client. The Twinsight method is to train two models, one fully supervised and one unsupervised. An additional loss term is added to each model's objective to couple them together. This term quantifies the discrepancy between neighborhood relationships under the action of each model on the input data. The method is demonstrated on four data sets and compared against several baseline methods. The Twinsight method demonstrated improved performance of the supervised model.", "questions": "Can you please include the default accuracy (accuracy achieved by always predicting the majority class). This should always be included when using accuracy as an evaluation metric.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "The authors explore a novel approach for leveraging unlabeled data in the federated setting and demonstrate improved model performance as a result. The range of baselines and data sets is good.\n\nOverall the paper is well structured and well written.", "weaknesses": "What are gradient conflicts? In vanilla supervised training gradients are averaged over all data. Certainly there are some data points whose gradients differ in their direction. Is this a significant cause for concern? Has it been studied elsewhere? Naturally, the dissimilarity between gradients of different data points will reach an extreme as the loss nears a local minimum. Yet, to my knowledge, this has not concerned anyone. \n\nThe neighborhood relation is a foundational technical element of the authors' method. Yet, it is given almost no technical treatment. How is N(.) computed? What are the consequences of different choices of quantifying the neighborhood?\n\nCan the authors shed some insight into why their method appears so much better than the baselines? \"This constraint enables ... to collaborate and mutually benefit from each other’s strengths\" is not a strong explanation. Why should the proposed coupling be better than manifold regularization or other forms of semi-supervised learning?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "5rbDi5QRxQ", "reviewer": "Reviewer_QrkV", "summary": "This paper considers the semi-superved federated learning setting, when there are clients with only unlabeled data. Authors recognize a gradient conflict phenomenon, and proposed a two-model solution for the setting, one to conduct supervised learning and the other one unsupervised learning. It also proposed a neighborhood position constrain to align the features learnt by two models.", "questions": "- It is nice to consider the partially labeled scenario, which is more realistic. The proposed method can be straightforwardly applied to a more realistic scenario where each client has a different ratio of labeled data. Are there any considerable difficulties? This is only for discussion, authors need not to append experiments.\n- What is the neighbourhood function used? Does it consider some graph structure?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- The paper considers a practically important problem and proposes a useful and principled solution to it.\n- The paper is overal well-written and easy to follow. It is well organized and the presentation is clear.\n- The idea is straightforward and well-motivated, and has a certain degree of originality and significance.\n- Extended empirical study is conducted.", "weaknesses": "- The motivation needs to be further explained.\n  - Since \"client drift\" has been observed before, how \"gradient cliff\" is different from it, as the first contribution is claiming the phenonmenon?\n  - When exactly will \"gradient cliff\" happen? Does it happen all the time when training traditional FSSL methods?\n  - Since \"The twin-model paradigm naturally avoids the issue of gradient conflict.\", it would be nice to show how gradient similarities are improved by Twin-sight.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "fYZYlB6jUM", "reviewer": "Reviewer_NJbh", "summary": "In this paper, the authors propose a novel twin-model paradigm for the federated learning in the label deficiency scenario, aimed to enhance the federated model effect by interacting insights of supervised and unsupervised models on labeled and unlabeled data. This paper points out that different objective functions of labeled and unlabeled data could lead to gradient conflict of federated learning. The author introduces supervised and unsupervised models in each client, and uses constraints to preserve the neighborhood relationship between the two models to interact their information, and finally realizes robust and effective federated semi-supervised learning. In addition, this paper conducts comparative experiments and ablation experiments on several benchmark datasets to prove the superiority and stability of the proposed algorithm compared with other baselines.", "questions": "1. The authors can introduce the content of the proposed algorithm in more detail. For the parts that refer to other methods (in formula (6) and formula (9)), the authors should indicate the citation and briefly describe how them work.\n2. The authors can rereport experimental results and compare baseline performance with the same number of training rounds.\n3. In the scenario of both unlabeled and labeled data, some combinations of federated algorithms and semi-supervised algorithms does not perform as well as federated learning using only labeled data, which violates normal cognition. The authors can try to explain the reasons for this phenomenon.\n4. The authors can try to prove the global convergence of the proposed objective function.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "1. This paper proposes to train both supervised and unsupervised models on the client side to avoid gradient conflict caused by different objective functions when aggregating models on the server.\n2. This paper introduces a neighborhood-preserving constraint to enable the supervised model and unsupervised model to fully interact with effective insights, collaborate and mutually benefit from each other’s strengths.\n3.This paper conducts comprehensive comparison experiments and ablation experiments to prove the advanced performance of the proposed algorithm.", "weaknesses": "1. This paper combines existing techniques to solve the federal semi-supervision problem, but lacks the necessary references and explanations. For example, the unsupervised algorithm adopted in formula (6) is not explained. f(w_u;∙) as an unsupervised model, what are its specific working principles and objectives? What does sim(∙) mean? What kind of neighborhood relation construction function is N(∙) of formula (9), and how does it construct neighborhood relation?\n2. The experimental results of this paper lack the necessary convincing. It is mentioned in the experimental setup that the paper will report the results after 500 rounds of training of the global model. What is the meaning of Round in Tables 1 and 2? If Round represents the number of training rounds, then why is the Round of each comparison method different and none reaches 500, which may cause the baseline results not being comparable.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "nMAqOVYtel", "reviewer": "Reviewer_37Jm", "summary": "This paper points out that the objective functions used in federated semi-supervised learning cause gradient conflict, which is attributed to the decentralized nature of federated learning. To tackle gradient conflict, it proposes a dual-model framework, Twinsight, to tackle label deficiency in federated learning. Twinsight trains a supervised paired with an unsupervised model. Meanwhile, Twinsight introduces a constraint to enable the interaction between the supervised and unsupervised models to preserve the neighborhood relation.", "questions": "What's the specific selection of d and N in the experiments?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "1. The motivation of this paper is meaningful, as it raises the gradient conflict problem.\n\n2. The proposed solution is reasonable, and preserving the proximity relationships between samples is indeed a valid approach to address this issue.\n\n3. This paper is well-written and easy to read.", "weaknesses": "1. The algorithm requires training an additional model, leading to increased computation and communication costs. \n\n2. There are details missing in the method and experimental description, raising concerns about reproducibility. Regarding Equation (9), the authors did not explicitly specify the choices of the metric d and the function N in both the method and the experiments section.\n\n3. The datasets are basic, and it appears that each experiment was run only once, with the random seed not being disclosed.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (poster)