FedMT: Federated Learning with Mixed-type Labels
Authors: Qiong Zhang
Aline Talhouk
Gang Niu
Xiaoxiao Li
Venue: Submitted to ICLR 2023 (2023)
Keywords: Federated Learning
Mixed-type Labels
Healthcare Application
Neural Tangent Kernel

Abstract: In federated learning (FL), classifiers (e.g., deep networks) are trained on datasets from multiple centers without exchanging data across them, and thus improves sample efficiency. In the classical setting of FL, the same labeling criterion is usually employed across all centers being involved in training. This constraint greatly limits the applicability of FL. For example, standards used for disease diagnosis are more likely to be different across clinical centers, which mismatches the classical FL setting. In this paper, we consider an important yet under-explored setting of FL, namely FL with mixed-type labels where different labeling criteria can be employed by various centers, leading to inter-center label space differences and challenging existing FL methods designed for the classical setting. To effectively and efficiently train models with mixed-type labels, we propose a theory-guided and model-agnostic approach that can make use of the underlying correspondence between those label spaces and can be easily combined with various FL methods such as FedAvg. We present convergence analysis based on over-parameterized ReLU networks. We show that the proposed method can achieve linear convergence in label projection, and demonstrate the impact of the parameters of our new setting on the convergence rate. The proposed method is evaluated and the theoretical findings are validated on benchmark and medical datasets.

Under review as a conference paper at ICLR 2023
FEDMT: FEDERATED LEARNING WITH MIXED-TYPE LABELS
Anonymous authors Paper under double-blind review
ABSTRACT
In federated learning (FL), classifiers (e.g., deep networks) are trained on datasets from multiple centers without exchanging data across them, and thus improves sample efficiency. In the classical setting of FL, the same labeling criterion is usu- ally employed across all centers being involved in training. This constraint greatly limits the applicability of FL.. For example, standards used for disease diagnosis are more likely to be different across clinical centers, which mismatches the classical FL setting. In this paper, we consider an important yet under-explored setting of FL., namely FL with mixed-type labels where different labeling criteria can be employed by various centers, leading to inter-center label space differences and challenging existing FL methods designed for the classical setting. To effectively and efficiently train models with mixed-type labels, we propose a theory-guidee and model-agnostic approach that can make use of the underlying correspondence between those label spaces and can be easily combined with various FL methods such as FedAvg. We present convergence analysis based on over-parameterrn ReLU networks. We show that the proposed method can achieve linear convergence in label projection, and demonstrate the impact of the parameters of our new setting on the convergence rate. The proposed method is evaluated and the theoretical findings are validated on benchmark and medical datasets.
INTRODUCTION
Federated learning (FL) enables centers to jointly learn model while keeping data each center. It avoids the centralization of data which is restricted by regulations such as CCPA (Legislature 2018).
HIPAA (At 1996), and GDPR (Voigt al. 2018) and has gained popularity in various applications. The widely used FL methods, such as FedAvg (MMMaha et al., (017), FedAdam (Reddi et al., (020), and others use iterative optimization algorithms to achieve jointly model training across centers. At each round, local center performs stochastic gradient descent (SGD) for several steps then centers communicate their current model weight to a central server to be aggregated.
When training a classifier in the classical FL setting, the datasets across all centers are annotated with the same labeling criterion. However, real applications such as healthcare, standards for disease diagnosis may be different across clinical centers due to varying levels of expertise or technology available at different sites. For example, when diagnosing ADHD with brain imaging, the labels are usually acquired over long period of behavior studies. Different centers may follow different diagnosis and statistical manuals (Mowe et al. 2015) and it is difficult to ask centers to relabel data using unified criterion some behavior studies cannot be repeated. This leads to different label spaces across centers. In addition, the center with the most complex labeling criterion, whose label space is desired for future prediction, typically only has limited labeled samples due to labeling difficulty or cost. In this paper, we aim to answer the following important question:
With limited samples from the desired label space, how to leverage the commonly used FL pipeline (e.g., FedAvg) and data from other centers in different label spaces to jointly learn an FL model in the desired label space, without additional feature exchanging and data relabeling?
We study an FL problem for given classification task. Each center has one labeling criterion, and the criteria across centers can be different. Samples do not overlap across centers. As shown in Fig. 1. first, label spaces not necessarily nested. One class from the desired
Problem Setting:

---

Under review as a conference paper at ICLR 2023
Class overlap [V1, P*) and (V7, P*)
y8
VV
fx()
F
FedMMT(L), FPdMT((P
Desired label space Y = (V*, "r,YY,,,,"" X
p1
g
Lice(0-
fc.
fc, (X)
The other label space y = (F4, 92, .9)
Lea(V.
V
fa(x)
Pedavg(fs. (f.
Lee(Q-18.
1x,
Lex(P,Qfe)
(a) Different label spaces
(b) Our proposed FedMT
Figure/ I: Illustration of the problem setting and our proposed FedM' method. (a) We consider different label spaces (i.e., desired label space Y' with K classes and the other space Y with J/ classes) where classes may overlap, such as Y and y Annotation using the desired label criterion is usually harder and more expensive obtain, thus less such labeled samples
available. (b) We use fixed label space correspondence matrix Q to associate label spaces Y with Y' and noise correction matrix T to correct label noises in } (if any). We correct predictions by multiplying classifiers' probability oulput ʃ by projection matrices locally (FecMT (P)) / or correct labels by multiplying by the inverse of projection matrices sample observed labels (FecMT (L)) under FedAvg framework.
space may overlaps with different classes in another space and vice versa (e.g.. discase diagnoses often exhibit imperfect agreement). Second, following the motivated healthcare example, we assume limited amount of labeled data (< 5%) in the desired label space available | Moreover, for the ease of experiment design, we consider the case where these data are stored in one 'specialized center,' and this center can be treated as the server to coordinate FL but still perform local model updating like the other clients, i.e., the centers with the other labeling criteria. All the centers jointly train an FL model following the standard FL training protocol as shown in Fig. (bb.
Prior methods for dealing with different labels spaces include personalized FL (Collins et al. 202 but they fail to leverage the correspondence across different label spaces. Transfer learning Yang et al. 2019) which pretrains model on space and finetunes the pretrained model other spaces can be an alternative solution in FL., but sub-optimal pretraining may lead to negative transfer (Chen al. 2019). Therefore, to address the limitation of above methods, we want to simultaneously leverage different types of labels and their correspondence and b) learn FL model end-to-end. To the best of our knowledge, other possible centralized methods meet our needs a) and b) are restricted to coarse to fine label spaces that have hierarchical structures (Tovronn et al. 2021 Chen et al. 2021a), which does not hold for the general problem of our interest, or require pulling all data features together for similarity comparison using more sophisticated training strategies (Hu et al., 2022). These methods cannot be simply extended to widely used FI. methods (e.g., FedAvg) and require feature sharing across centers which increases privacy risks.
To address the above limitations, we propose a glug-and-play method called FedMT, which is a versatile strategy that can be easily combined with various FL pipelines such FedAvg, Specifically, we use models with the same architecture whose output dimension is the number of classes in the desired label space across all centers. To use client data from the other label space for supervision, we align two spaces either with label (probability) projection that projects the label (class scores) to the other space. We further show that methods has the bonus to handle label noise.
Contributions: Our contributions are three folds. Methodologically, we propose a novel FL method, FedMT, which is computationally efficient and versatile solution Theoretically, we present the convergence of F FedMT in FL with over pver-parameterized ReLU neural networks, and explore the impact of amount of data from desired label space and different noise levels; Empirically, we demonstrate the superior results in this challenging setting over prior art with extensive experiments on benchmark and medical datasets.
RELATED WORK
Federated learning Fl. is emerging as learning paradigm for distributed clients that bypasses data sharing to train local models collaboratively. To aggregate model parameters, FedAvg (McMahan
Due labeling difficulties, such lahels can also he noisy. Hence, we also explore this property our work.
2

---

Under review as a conference paper at ICLR 2023
et al. 2017) is the most widely used approach in FL. Variants of FedAvg have been proposed to
improve optimization (Reddi et al. [020]] Rothchild et al., 2020) and for non-iid data ( et al.
2021: 2020a Karimiredyy al. 2020). Recently FL methods have been studied in -upe-supe learning eeonn et al. 2021] Bdair et al. 2021), weakly supervised learning (Luu et al. (021), ) and positive label only learning Yu et al. 2020). Theoretical studies of FI. have not yet directly addressed neural network algorithms nor accounted for the influence of individual data samples (Karimiredyy
al. 2020 Li al. 2019 Khaled et al. 2020). Recently, a theoretical framework for FL on fully
supervised neural network regression using FedAvg was proposed in Huang et al. (202T). To the best of our knowledge, this is the first work investigate FL's theory on neural network classification under mixed-type labeled data.
Neural Tangent Kernel (NTK) NTK an essential tool to study the learnability of neural networks. NTK was first studied by Jacot et al. (2088) showing the equivalence between training an infinitely wide neural network with gradient descent and kernel regression. NTK theory has been extended to convolutional neural networks (CNNs) (Arora al. 2019), graph neural networks (GNNs) (Du et al.
2019 Jiang et al. 2019), recurrent neural networks (RNNs) (Alemohammad et al. 2020p. NTK for FL recently studied by Huang et al. (202I). in the supervised FL regression problem using mean squared error loss.
METHODS
3.1 PRELIMINARIES: CLASSICAL FL
We start by reviewing the classical FL approach, FedAvg, with full supervision (MMMaha et al. 2017) for classification with the same labeling criterion. We consider K K-class classification problem with the feature space X' and the label space ) = []. Let a E X' and E J be the input and output random variables following an underlying joint distribution with density p(a, y). Let f X - R be a K-class classifier such that a (f)())) = p(y = kl/z), where f& (z)) is the k-th element of f f(x) and (fx) exp(S*)/ Σ exp(Jw) (w=) ) is the softmax function. Then the predicted label can be obtained via Upred = arg ma[k (ar).
In the classical FL setup, each client e E [C] has access to a labeled training set D. of size N₂ and learns its local model fr by minimizing the following empirical risk: {(0% 1 No
1 No 111
Ex
(f
No :=1 ( (ff())))))
K 1(y=k) log ff(x) = - log ff where é is the cross-entropy loss, i.e., (cx(f E and ((.) is the indicator function. The goal of classical FL is that C clients collaboratively train a global classification model f that generalizes well with respect to p(æ, y). without shar- ing their local data De-- The problem can be formalized R(f) E (f; FedAvg employs a server to coordinate the iterative distributed training through model parameter averaging described in Algorithm
minimizing the aggregated risk:
3.2 PROBLEM FORMULATION
In contrast to the classical supervised Fl. classification, where all the centers share the same label space, we consider the case where the label spaces across centers may be different. For the ease of presentation, we assume there are two label spaces, namely y = { and y = (Ý*) where K >> J. f For the "specialized center' with the desired label space, we denote as a server and its dataset as D = {(#) vf) E [nK]]. where y E V. We also have in total N labeled data with different criterion that are stored across C clients. For e E [C], we denote by S. the indices of data from the other label space e and we have N Eeetoy
{x vf4 E Se) e-th client, where yf e D. Let Ne = [Sa] (cy N.. Our objective is to train a global classifier in the desired label space f : X + y using data in the system from different label spaces.
"Our method be easily extended multiple label spaces.
3

---

Under review as a conference paper at ICLR 2023
3.3 PROPOSED METHOD
Motivation Learning under different label spaces be formulated corrupted label learning problem [Van Rooyen & Williamson] 2017] Patrini et al., 2017). where label space with less classes ) is treated as the corrupted observation of a true underlying label space ) with more classes, i.c. our desired label space. Our method is based on loss correction (Van Rooyen & Williamson 2017 Patrini et al. 2017), which is a well-established corrupted label learning approach in the centralized domain with statistically consistency guarantees.
Let T y' + V linear transformation. whose pseudo inverse is T Under proper assumptions (see Theorem 3.1) and a given function f X' Y Patrini et al. (011)) showed two ways to perform loss correction that T-1 can pull back from functions of corrupted labels to functions of true labels and T can transfer functions of true labels to those of corrupted labels. Mathematically, this can be written as:
Theorem 3.1 (Loss correction (informal) (Pttrin al. 2017 Van Rooyen & Williamson 2017) Given a non- non-singular linear mapping matrix T and a proper loss f, one can achieve the same minimizer of the original loss under the true label distribution:
arg min En Eayyeylyy. f) = arg min E Ee Tf) and Exyeylly f) = BR TT Ely, f).
In addition, loss correction methods are model agnostic and can be generalized to many loss functions, therefore provide versatile framework for learning from corrupted labels. Since our goal is to jointly learn classifier f + R for clients and server, the overall loss can be written as:
C Roverall (f) C+1 R.( + S.(tt
1
where (f) and (f) are respectively the empirical risk based on the server's data and the clients'
data from a different label space. To minimize the overall loss in ② we need to align the predictions or losses of both types of labels to the underlying true label space. Considering the specific form of
cross-entropy I adapt the findings of Patrini et al. 2017); : Van Rooyen & Williamson (2017) to
our FL setting. For all the clients in FL I E [C], we propose leveraging the following two kinds of projections to both server and clients
K Σ
Probability projection: R((
Σ 1(vi = 83)
T T
(3)
Ni
IES
Label projection: R.(f:D')
T 1( = マッソ log lo (4)
N
FES
k=
For clients, we denote T Q [0, 1] J whose j-th row denotes the mixing weights of classes in the desired label space for j-th class in the other label space. As both probability projection and label projection are performed locally, the overall loss ? can be optimized using general FL strategies. such as FedAAg (McMahan et al.) 2017) used in this work, or other variants like Li et al. (2020a) 202T; Karimireddy et al. (2020). Note our method can be easily extended to more than one centers in the desired label space as shown in Appendix D.5
Extension to Noisy Labels Another advantage of our method is that it has the flexibility to extend to the case where there are noisy observations, that is the label is mislabeled label space. Let us explain with the case where the server has noisy labels. Specifically, on the server, we let T T E [0, 1 X XK be matrix whose (i. j)th element denotes p(ý" YJ " Y*) where 7"t is the observed noisy label and y is the true label.
wrong one in the same
The detailed FecMT algorithm is described in Algorithm T * K under noisy label case and T = I x K for noise-frec free setting. this algorithm. it is worth noting that FedMT is easy to implement only slightly modifies FedAvg highlighted in blue.
4

---

Under review as a conference paper at ICLR 2023
Algorithm FL using FedAvg MMMMa Algorithm 2 FL using FedMT (Ours) han et al. 2017)
Server Input: inputs of Alg server model f small noisy Server Input: initial f. aggregation dataset D. = {xr "y) where y E V. projection matrix T
step-size Magg. and global communication Client Input: inputs of Alg | but D. {a"; y where
round R
yº E V, and projection matrices Q
Client Input: local model f", local
For r = - R rounds, we run A on each client and B iteratively
dataset De. SGD step-size 7sgd. and lo-
cal updating iterations (for c E [C])
2: procedure M.MODELUPDATE(C)
1: For r 1 R rounds, we on each client and B iteratively.
3:
f c Receive updated model from PROC. B for = + ++ do
4:
2: procedure A. MODELUPDATE(()
5:
if probability projection then
3: fc <- f & Receive updated
f t f Tagd V VR.(Q yr)
model from PROC. B
7:
f + Nspd V PR.(T (T=;; y")
4:
1
5: f ff V (U
8:
else label projection then
Model updates via SGD
9:
f f Nsgd RR(f%% Q 2-yy0yy) yo)
6: send f f to PROC.
10:
f f Nagd
V (R, T y")
7:
procedure MODELAGG(T)
send f f to PROC. B for E {[C),8]
8: receive model updates f
12:
procedure B. MODELAGG(r)
from PROC. A
13:
receive model updates from PROC. A
9:
ft f- Magg Ec (f
f) 14:
f MARR [ie(( (f) f)
10:
broadcast/ f to PROC.
15:
broadcast f to PROC. A
3.4 THEORETICAL ANALYSIS
Although label projection-baser loss correction methods have shown have good properties under centralized setting (see Theorem 3.1 Patrini et al. (2017); ; Van Rooyen & Williamson (2017). their theoretical convergence under FL have not been explored. Hence, we focus on the theoretical analysis of label projection in 4 given it has been shown to have better theoretical guarantees than in previous study (see Theorem 3.1 PPtrinni et al. 2017)). Further, we are interested investigating the impact of various parameters on theoretical convergence. In this section, establish novel
theoretical analysis of FedMT using NTK (Arora et al. 2019: Lee et al. 2019 Huang et al. 2021)
NTK setup Our theoretical results are studied under an over-parameterized one-hidden layer neural network (NN). Let X R be the output of the NN whose kth element is
M
Su xx)
Σ mr
alem Ó
(uu,
V
where ((2) maxx(2.0 0} is the ReLU activation function and u = [u,, U2. uv E Rd
Definition 3.2 (Initialization). We initialize RdxM and akm as follows. For each E [M]. um is sampled from N (0, I). For each k E [K] and m E [M], atem is sampled {-1, +1} uniformly at random and is not trainable.
In the r-th global round, the server broadeasts the global model weight (n)()) to every client. Each client e then starts from umm.(00 r) = ux(r) and takes local gradient steps via gradient descent with step size Mocal
aR (Tm.
Um.e + 1, (T, )
Mocal
Fum
where (m, r) is the value of um on cth client at step t in r-th global round. Then the client sends A (r) umm.e (t, (0, r) server and server computes new um + 1) based on the average of all Aum,a()) via
Um (r + 1) = (r) Magg
! (Mm,e(r)CC
cE(C]
5

---

Under review as a conference paper at ICLR 2023
NTK analysis results Let (f x)) and yf be the multi-hot vector representation of of and yf be the one-hot vector representation of of in IRK Let Q be the linear mapping from y to and T be the same before. Following Neural Tangent Kernel (NTK) analysis (Lee al. 2019
Arora et al. 2019). we consider the mean squared error los respectively on the client and server
Ro(u)
ΣΣΣ
O (viR (u(
No iES, j= 1=
(5)
Ru(u) nk K
nk L E
Tri gN (u, x;))
Lemma 3.3. Let the notation be the same as before, the NTK kernel G(T) based on our proposed
novel loss ② is block matrix with K row partitions and K column partitions. The block matrix in the 1-th row and meth column has the following form
9 (r)
gla(()
2 (r)
22 (r)
(20(()
Gr)
(6)
(C)
g22(()
GliT( (r)
g (r)
(li
for E [K] and E [K] where each sub-block matrix has the following form:
Giul (t)
E fm
V.. (u(t), fj ([C), [C]
(7)
(usu(u((), D*)V (u(1), (')
km
We show detailed derivation of Lemma 3.3 Note that J < K, Q RJek is not inevitable. In
practice, we compute its pseudo inverse instead. Although looking at Q alone, the plausible property of Theorem 3.1 may not hold. We claim that by optimizing (f( together with (f(.), we benefit
from the gradient labels in the desired label space, as shown the expression of G (r) in Eq. 7
Following Huang et al. (2021), we further conclude the convergence of the our proposed loss in 5 Let M I + nK) log((N + nk))/)),, iid initialize Um (0), Okm as Definition 3.2 Let X = Amain (G()) denote the smallest eigenvalue of G(0). Let
Theorem 3.4 (Convergence).
denote the condition number of G0)). For clients, for any c, let
C log(1/e))
R R=O
AMoocaMagg!
Mocal = O (A/kt(N + ) and Nlag = O1),, the above algorithm satisfies
(1 Tagg Mocet L(u(O)).
[(u(r)) -
2C
with probability at least 1
The details of the proof are deferred to Appendix B Note that a smaller value of Mogg 2t, e.g., smaller eigenvalue and larger number clients, leads slower convergence. is clear that À depends on both Q and T, therefore it is important to see its connection with Q and T. we discuss the algorithm convergence under the following two special cases.
Corollary 3.5 (Convergence under various label granularity differences). Let K2..2...., k, be J positive integers that K = -E kj. If the transformation from the desired label space to the other label space is Q = diag (1 1,,, ...,, 1[,) ) E [0, x| (i.e., the classes in the desired label space are subclasses of classes in the other label space), then the smaller the value of J. the lower the convergence rate.
which be casily extended the label projection loss

---

Under review as a conference paper at ICLR 2023
Proof Sketch: Based on the form of G(r) in it can be seen that (Li et al. 2021
A == Amin (G(0)) I min (Amin (9 e (0)) Amin (9 (0))) S Amin (g
(0)) (0)) S min k Ao
k,c,s
where Åo Amin (V (u)() VVV Vavgg((( (u()), D)). Since Σ, Ko = K, the smaller the value of J, the smaller the value of À and hence the algorithm converges slower. See detailed proof in Appendix [B] Corollary 3.6 (Convergence under different noise levels). We assume noisy labels are corrupted via
T = (1- K/(K 1)E) 1K + E/(K - 1
where Tú = p(y Alu = Yž) with observed noisy label ÿÿ" and true y² is the true label, and E 1- E (0, for E [K] denotes the noisy level. In this specified case, the convergence of FedMT with label projection on NTK is does not depend on the noise level &.
Proof Sketch: By using matrix inversion lemma, we find T- (K 1)/(K
Kε)LKK
C/(K- - - -K) x k Hence Σk=1 Tak for any ke E [K]. See Appendix B for details.
4 EXPERIMENTS
In this section, demonstrate the effectiveness of our proposed method, FedMT, when data from different label spaces. Compared with other training strategies and prior art, FedMT consistently achieves better test accuracy in predicting labels in the desired label space with limited amount of data from this space as demonstrated on CIFAR100 (Krizhessky et al. 2009) and medical dataset.
4.1 BENCHMARK EXPERIMENTS SETUP
Dataset and setting We use the CIFARI00 (Krizhevsky al. 2009) dataset mimic our proposed problem setting. We assume one center (as server to coordinate FL training) has a small amount of data (S 5% of total sample size) with sub-class annotations and other centers are labeled using super-class annotations. The sub-class space is viewed as our desired label space. Our objective is to train a classification model using FI. to predict sub-class labels with all the centers simultaneously.
The CIFAR100 dataset consists of 50K images, associated with K = 100 sub-classes that could be further grouped into J = 20 super-classes. To study the effect of the number training samples in desired label space, we randomly select observations from each of the 100 sub-classes on the server and the rest of the observations in the training set are split into C C= 10 subsets completely at random, ensuring that each class is equally represented in each subset. Each subset corresponds to dataset stored on one client and we use N 4000. We use the super-class as our label for all clients. Experiment results with other values of C and N₂ is given in Appendix [
We use ResNet18 8 (He et al. 2016) as our classifier. We use SGD optimizer Ruder (016)) with a learning rate of 10- momentum 0.9, and weight decay 5 10-4 The learning rate is divided by 5 at 20, 30, and 40 epochs. If not specified, our default setting for local update epochs is E = 1. Based on the superclass information in CIFAR 100, we are able to formulate the transition matrix Q for all clients. Its form and more training details are available in Appendix C.I
Baselines We compare the following approaches: 1) Single: only the data from the desired label space is used to train the classifier; 2) FedMatch (Jcong et al. [021):: we treat client samples as unlabeled data, perform pseudo-labeling on the unlabeled sets, and augment samples in supervised loss training. 3) FedRep (Collins et al. 202T): clients train a J-class classifier and the server trains K K-class classifier that differs in the last layer with different output dimension. These classifiers share the same backbone parameters using FedAvg, the performance is tested using the classifier on the server; 4) FedTrans (Chen al.] [02Tbb: all clients trains J-class classifier using FedAvg which is later fine-tuned server with new K K-class linear layer. We refer FedMT (P) as proposed method with probability projection loss; and refer FedMT (L) as our proposed method with label projection loss.
4.2 BENCHMARK EXPERIMENTS RESULTS
Comparison under different amount of data on server We fix number of clients C' at 10, and set different number of samples per-class on the server in the range of (5, 10, 15, 20, 25). The
7

---

Under review as a conference paper at ICLR 2023
Table I: Comparison of the accuracy for 100-class classification using our methods and alternative methods on the CIFAR 100 benchmark dataset with different per sub-class number n images on the server. We report mean (sd) from three trial runs. The best method is highlighted in boldface.
Single
FedMatch
FedRep
FedTrans
Ours: FedMT (P & 1.)
5
8,56(0.33)
16.89(074)) 13.20(0.14)
21.31(1.27)
21.68(0.67)
20.56(0.42)
10
18.02(0.25) (2.29(0.32)
21.90(1.27)
24.53(1.40)
26.44(0.34)
23.89(0. 13)
15
23.39(0.67) 23.44(0.08) (4.34(0.15)
26.21(0.95)
25.74(0.59) 28.25(0.34)
20
16.23(0.21)
27.70(0.09)
24.70(0.55)
29.75(0.66) (7.49(0.85)
27.4 27.43(0.35)
25
17.73(0.20)
(0.59(
26.53(0.29)
28. 28.16(1.30)
30.63(0.06)
28.53(0.52)
4.5
4.5
4.50
Eu0.2
[-0.1
Ew0.3
4.0
4,0
Accuracy (90
20
4.00
Less
3.75
C-10
10
3.0
2.5
J-5
J-10
2.5
3.0
C=100
C-20
C+50
3.25
Ouc
Puo IL
redhop
Rounds
Rounds
40
Rounds
20
0.0
naise
(a)
(b)
(c)
(d)
Figure 2: Ablation studies on CIFARI00. (a) Effects of the number of classes J on convergence. (b) Effects of the number of clients on convergence. (c) Effects of noise level & on convergence. (d) Study of performance accuracy with different noise levels on the server. All the convergence results are shown on the first 50 rounds using FeGMT (L) for better visualization.
advantage of FL is to enable access to more data to improve model performance. As shown in Tab. [ when n increases (supervision information increases), the accuracy of all approaches improves. Both FedMT (P & L) significantly outperform alternative methods when supervision is limited < 25. We achieve similar but much more stable performance compared with FedMatch when = 25. FedMatch adaptively augments the training data with labels in the desired label space using heuristically derived confident pseudo labels. Future work could combine FedMT with similar approaches used in FedMatch to evaluate heuristic results in theoretically provable framework.
Analysis on convergence To validate our developed theoretical results in Theorem 3.4 we investigate the convergence rate of loss 2 of FedMT with label projection. In Fig. 2 we explore loss as function of the number of classes J E 15, 10, 203 with = 10 under noise-free setting, and the convergence is consistent with Corollary 3.5 smaller J leads to lower convergence. The loss curves in Fig. 2 show FecMT convergence was achieved faster when training with more clients for a fixed total number of samples. This result is consistent with the theoretical result of Theorem 3.4 As shown in Fig.
[[c] under the default setting, the convergence rates are similar at different noise levels. The results demonstrate our method is reasonably stable to noise, which is in line with Corollary 3.6
Effect of varying noise levels Our method has the flexibility to adapt to the case where the server data noisy. We the accuracy of K-class prediction in the held out test set compare our method with other approaches in CIFARIO6 the results are reported in Fig. 2(d) Given the mislabeling rate in many real applications (e.g.. healthcare) could be as high as 40% (Lettiei et all 2005). we compare FedMT with other learning approaches. by varying the noise level on the server from [0, 0.1, 0.2, 0.3, 0.4) and set n 10 on the server f From the results, our proposed FedMT method significantly outperforms all alternative methods both even when the noise level & is up to 0.4. Second, when increasing noise levels, FedMT shows more stable performance compared to baselines. Our method also shows stable performance if we have more than client has observations from desired label space and the results is given in Appendix D.5
4.3 TREMOR SEVERITY PREDICTION OF PARKINSON'S DISEASE
Dataset Our method is not limited to sub- and super-classes as in CIFARI00, to illustrate the effectiveness of our method under the general overlapping class case, we evaluate FedMT for the
We assume the labels are sampled from known, systemic, and instance inneeendeedeeee noisy transition matrix T, whose diagonal values are E. Our method can be extended to estimate 7 using anchor pasnt-based methods (Scott. 2015 Liu Tao 2015) but discussing the estimation method is not in the scope of this work.
8

---

Under review as a conference paper at ICLR 2023
Table 2: Comparison of the accuracy for 5-class classification using methods and alternative methods on the sEMG dataset with various n and = 50 under noise-free setting. We report mean (sd) from three trial runs. The best method is highlighted in boldface.
Single
FedMatch
FedRep
FedTrans
Ours: FecMT (P & L)
27.30 27.30(0.88)
28.20(0)67))
39.34(1.12)
42.31(2.12)
66.89(0.91)
63.67(0.28)
2
30.83(1.30)
31.90(1.45)
40.49(1.16)
46.45(0.63)
67.00(0.38)
64.22(0.28)
3
34.66(1.09)
32.90(1.74)
41.29(1.16)
51.25(2.57)
(7.34(0.57)
64.33(0.48)
4
39.11(1.48)
33.53(2.12)
43.24(0.68)
51.81(1.99)
67.67(0.64)
(4.44(0.33)
5
42.57(0..8)
37.20(2.09)
44.24(0.74)
51.83(2.14)
67.22(0..0)
64 (4.55))
prediction of tremor severity for patients with Parkinson's 's disease (PD). Tremor is typical movement disorder occurring on the limbs of patients with PD (Qin et al. 2020). Surface clectromyography
(sEMG) is widely used for movement disorder assessment by noninvasively recording electrical signals on the skin. The patterns of sEMG signals can be used to turn PD diagnosis into different subtypes based on the diagnostic values. We use the synthetic sEMG dataset in QQn et al. (202)). In our experiment, we consider K = 5 on the server and
on the client, according to MDS- UPDRS19, Goetz et al. 22008) In this dataset, the classes in two label spaces has overlap. The form of Q is given in Appendix
Setup We randomly sample 9000 observations from the sEMG dataset as client training data. We hold out 1000 observations for testing. We then split the training set to C = 50 clients completely at random, but ensuring that samples are balanced within each class in both label spaces. Each client, therefore, has roughly 60 observations from one of the three classes. In addition to the training set on clients, also have observations per class the server. Similar to the benchmark experiment, we vary n on the server. Following Qin et al. 2019), we use 12 summary statistics of sEMG as our features: mean absolute value (MAV), mean square value (MSV), root mean square (RMS), variance (VAR), standard deviation (STD), waveform length (WL), Willison amplitude (WAMP), log detector (LOG), slope sign change (SSC), zero crossing (ZC), mean spectral frequency (MSF) and median frequency (MF). The mathematical definition of these features is given in Chowdhury et al. (2013). We then use single hidden layer multiple layer perceptron (MIP) with 128 hidden units and ReL.U activation function as our backbone. We use SGD optimizer with learning rate 10-3, and set batch size as 16. Local update frequency is epoch. We train the model for 100 communication rounds. More training details are available in Appendix C.2 We use the same performance measure and the same alternative approaches as those described in Section 4.1 for comparison.
Results Overall, our quantitative results in Tab. show that FedMT with probability projection performs better than probability projection on the sEMG dataset and consistently outperforms alternative learning approaches. FeaMT improves mean accuracy by non-negligible margin, demonstrating a synergistic collaborative effect with clients' data from the other label space. The superiority of F FedMT in this demanding medical data setting further demonstrates the efficacy and robustness of our algorithm. To test of the performance of FedMT at different degree of overlapping, we also conduct experiments when K 10 with various noisy levels, the results is given in Appendix E. Our method also outperforms other methods in this case.
CONCLUSION
In this work, we propose a new FL framework, FedMT, to address an important yet under-explored mixed-type label setting. Theoretically, we provide the convergence guarantee of FedMT with the extension of NTK. Through extensive experiments on a benchmark and a medical dataset, we demonstrate FedMT outperform alternative methods. We also emphasize that since FedMT makes minor modifications to local predictions or labels, it has much more flexibility to integrate with other FL strategies beyond FedAvg. The performance of our proposed method can be further improved by combining our provable method with other heuristic-based weakly supervised learning approaches. As and---nd-payyy method, F FedMT can be applied to non-III setting by combining with advanced FL schemes, including Li et al. 2020b
2021
Karimireddy et al. (020).
'The laheling strategies detailed in Appendix C.2

---

Under review as a conference paper at ICLR 2023
REFERENCES
Accountability Act. Health insurance portability and accountability act of 1996. Public law, 104:191,
1996.
Sina Alemohammad, Zichao Wang, Randall Balestriero, and Richard Baraniuk. The recurrent neural tangent kernel. arXiv preprint ar06v: 10246, 2020.
Sanjeev Arora, Simon $ Du, Wei Hu, Zhiyuan Li, Russ Salakhutdinov, and Ruosong Wang. On exact computation with an infinitely wide neural net. Advances in Neural Information Processing Systems, 32, 2019.
Tariq Bdair, Nassir Navab, and Shadi Albarqoumi. Fedperl: Semi--uperrvied peer learning for skin lesion classification. In International Conference on Medical Image Computing and Computer- Assisted Intervention, pp. 336-344- 346. Springer, 2021.
Junjie Chen, Li Niu, Liu Liu, and Liqing Zhang. Weak-shot fine-grained classification via similarity transfer. Advances in Neural Information Processing Systems, 34:7306-7318, 2021a.
Xinyang Chen, Sinan Wang, Bo Fu, Mingsheng Long, and Jianmin Wang. Catastrophic forgetting meets negative transfer: Batch spectral shrinkage for safe transfer learning. Advances in Neural Information Processing Systems, 32, 2019.
Yinbo Chen, Zhuang Liu, Huijuan Xu, Trevor Darrell, and Xiaolong Wang. Meta-baseline: Exploring simple meta-learning for few-shot learning. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 9062-9071, 2021b.
Rubana H Chowdhury, Mamun BI Reaz, Mohd Alauddin Bin Mohd Ali, Ashrif AA Bakar, Kalaivani Chellappan, and Tae G Chang. Surface electromyography signal processing and classification technigues. Sensors, 13(9):22431-12466 2013.
Liam Collins, Hamed Hassani, Aryan Mokhtari, and Sanjay Shakkottai. Exploiting shared represen- tations for personalized federated learning. In International Conference on Machine Learning, pp. 2089-2099. PMLR, 2021.
Simon Du, Kangcheng Hou, Russ R Salakhutdinov, Barnabas Poezos, Ruosong Wang, and Keyulu Xu. Graph neural tangent kernel: Fusing graph neural networks with graph kernels. Advances in neural information processing systems, 32, 2019.
Christopher G Goetz, Barbara C Tilley. Stephanie R Shaftman, Glenn T Stebbins, Stanley Fahn, Pablo Martinez- Martinez-Martin, Werner Poewe, Cristina Sampaio, Matthew B Stern, Richard Dodel, et al. Movement disorder
revision of the unified parkinson's 's disease rating scale (MDS-UPDRS): scale presentation and clinimetric testing results. Movement disorders: official journal of the Movement Disorder Society, 23(15):2129 2170, 2008
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 770-778, 2016.
Anqi Hu, Zhengxing Sun, and Qian Li. Weakly supervised Ane-grained recognition based on combined learning for small data and coarse label. In Proceedings of the 2022 International Conference on Multimedia Retrieval, pp. 194-201, 2022.
Baihe Huang, Xiaoxiao Li, Zhao Song, and Xin Yang. FL-NTK: A neural tangent kernel-based framework for federated learning analysis. International Conference on Machine Learning, pp- 4423-4434 PMLR, 2021.
Arthur Jacot, Franck Gabriel, and Clément Hongler. Neural tangent kernel: Convergence and generalization in neural networks. Advances in neural information processing systems, 31, 2018.
Wonyong Jeong, Jaehong Yoon, Eunho Yang, and Sung Ju Hwang. Federated semi-supervised learning with inter-client consistency & disjoint learning. In International Conference on Learning Representations, 2021.
10

---

Under review as a conference paper at ICLR 2023
Yihan Jiang, Jakub Konečný. Keith Rush, and Sreeram Kannan. Improving federated learning personalization via model agnostic meta learning. arXiv preprint S.kiv:1909.12450 2019.
Peter Kairouz, H Brendan McMahan, Brendan Avent, Aurélien Bellet, Mehdi Bennis, Arjun Nitin Bhagoji, Kallista Bonawitz, Zachary Charles, Graham Cormode, Rachel Cummings, et al. Ad- vances and open problems in federated learning. Foundations and Trends@ in Machine Learning, 14(1-2):1-210, 2021.
Sai Praneeth Karimireddy, Satyen Kale, Mehryar Mohri, Sashank Reddi, Sebastian Stich, and Ananda Theertha Suresh. Scaffold: Stochastic controlled averaging for federated learning. In International Conference Machine Learning, pp- 5132-5134,, PMLR, 2020.
Ahmed Khaled, Konstantin Mishchenko, and Peter Richtárik. Tighter theory for local sgd on identical and heterogeneous data. In International Conference on Artificial Intelligence and Statistics, pp- 4519-4529. PMLR, 2020.
Alex Krizhevsky, Geoffrey Hinton, al. Learning multiple layers of features from tiny images. Technical report, University of Toronto, 2009.
Jachoon Lee, Lechao Xiao, Samuel Schoenholz, Yasaman Bahri, Roman Novak, Jascha Sohl- Dickstein, and Jeffrey Pennington. Wide neural networks of any depth evolve as linear models under gradient descent. Advances in neural information processing systems, 32, 2019.
California State Legislature. California consumer privacy act (ccpa). https://oas /aa::://aa..o.o ca. gov/
privacy/ ccpa, 2018.
Christopher J Lettieri, Ganesh R Veerappan, Joseph M Parker, Teri J Franks, Dennis Hayden, William D Travis, and Andrew Shorr. Discordance between general and pulmonary pathologists in the diagnosis of interstitial lung disease. Respiratory medicine, 99(1 (9(11):1425-144 2005.
Tian Li, Anit Kumar Sahu, Manzil Zaheer, Maziar Sanjabi, Ameet Talwalkar, and Virginia Smith. Federated optimization in heterogeneous networks. Proceedings of Machine Learning and Systems, 2:429 4500 2020a.
Tian Li, Anit Kumar Sahu, Manzil Zaheer, Maziar Sanjabi, Ameet Talwalkar, and Virginia Smith. Federated optimization in heterogeneous networks. Proceedings of Machine Learning and Systems, 2:429-450, 2020b.
Xiang Li, Kaixuan Huang, Wenhao Yang. Shusen Wang, and Zhihua Zhang. On the convergence of fedavg non-iid data. arXiv preprini arXiv:1907.021880 2019.
Xiaoxiao Li, Meirui Jiang, Xiaofei Zhang, Michael Kamp, and Qi Dou. FedBN; Federated learning non-iid features via local batch normalization. In International Conference Learning Representations, 2021.
Tongliang Liu and Dacheng Tao. Classification with noisy labels by importance reweighting. IEEE Transactions on pattern analysis and machine intelligence, 38(3):447-461, 2015.
Nan Lu, Zhao Wang, Xiaoxiao Li, Gang Niu, Qi Dou, and Masashi Sugiyama. Unsupervised federated learning is possible. International Conference on Learning Representations, 2021.
Nan Lu, Zhao Wang, Xiaoxiao Li, Gang Niu, Qi Dou, and Masashi Sugiyama. Unsupervised federated learning is possible, 2022.
Robert E McKeown, Joseph R Holbrook, Melissa L Danielson, Steven P Cuffe, Mark L Wolraich, and Susanna N Visser. The impact of case definition on attention ceriinndddfccithypratttiiii disorder prevalence estimates in community- samples of school-aged children. Journal the American Academy of Child Adolescent Psychiatry, 54(1):53 61, 2015.
Brendan McMahan, Eider Moore, Daniel Ramage, Seth Hampson, and Blaise Aguera Arcas. Communication- effiicientiiiitttttiee learning of deep networks from decentralized data. In Artificial intelli- gence and statistics, pp. 1273- 1282. PMLR, 2017.
11

---

Under review as a conference paper at ICLR 2023
Giorgio Patrini, Alessandro Rozza, Aditya Krishna Menon, Richard Nock, and Lizhen Qu Making deep neural networks robust to label noise: A loss correction approach. In Proceedings of the IEEE conference computer vision and pattern recognition, pp. 1944-1952, 2017.
Zengyi Qin, Zhenyu Jiang, Jiansheng Chen, Chunhua Hu, and Yu Ma. sEMG-based tremor severity evaluation for Parkinson's disease using light-weight CNN. IEEE Signal Processing Letters, 26 (4):637-641, 2019.
Zengyi Qin, Jiansheng Chen, Zhenyu Jiang, Xumin Yu, Chunhua Hu, Yu Ma, Suhua Miao, and Rongsong Zhou. Learning fine-grained estimation of physiological states from coarse-grained labels by distribution restoration. Scientific Reports, 10(1):I-10, 2020.
Sashank Reddi, Zachary Charles, Manzil Zaheer, Zachary Garrett, Keith Rush, Jakub Konečný, Sanjiv Kumar, and H Brendan McMahan. Adaptive federated optimization. arXiv preprint arXiv:2003.00295, 2020.
Daniel Rothchild, Ashwinee Panda, Enayat Ullah, Nikita Ivkin, Ion Stoica, Vladimir Braverman, Joseph Gonzalez, and Raman Arora. Fetchsgd: Communication-efiiieent federated learning with sketching. In International Conference on Machine Learning, pp.
PMLR, 2020.
Sebastian Ruder. An overview of gradient descent optimization algorithms. arXiv preprint MXXi::660..04477, 2016.
Clayton Scott. A rate of convergence for mixture proportion estimation, with application to learning from noisy labels. In Artificial Intelligence and Statistics, pp. 838-846, PMLR, 2015.
Hugo Touvron, Alexandre Sablayrolles, Matthijs Douze, Matthieu Cord, and Hervé Jégou. Grafit: Learning fine-gre fine-grained image representations with coarse labels. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 874-884, 2021.
Brendan Van Rooyen and Robert C Williamson. A theory of learning with corrupted labels. J. Mach. Learn. Res., 18(1):8501-855 -85000 2017.
Paul Voigt et al. The EU general data protection regulation (GDPR). Intersoft consulting, 2018.
Qiang Yang. Yang Liu, Tianjian Chen, and Yongxin Tong. Federated machine learningg Concept and applications. ACM Transactions on Intelligent Systems and Technology (TIST), 10(2):1-19, 2019.
Felix Yu, Ankit Singh Rawat, Aditya Menon, and Sanjiv Kumar. Federated learning with only positive labels. International Conference on Machine Learning, pp. 10946-10956. PMLR, 2020.
12


Reviews:
[{"review_id": "tj31SnswWaw", "reviewer": "Reviewer_xV7p", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "In this paper, the authors propose a method that implements the federated learning(FL) with mixed-type labels. In detail, the authors propose a theory-guided and model-agnostic approach that can make use of the underlying correspondence between different label spaces and can be easily combined with various FL methods. The proposed method achieves better results than the baseline methods.", "summary_of_the_review": "In this paper, the authors propose a method that implements the federated learning(FL) with mixed-type labels. In detail, the authors propose a theory-guided and model-agnostic approach that can make use of the underlying correspondence between different label spaces and can be easily combined with various FL methods. The proposed method achieves better results than the baseline methods. The novelty is relatively good, and the proposed method achieved a relatively good performance than the baseline methods.", "strength_and_weaknesses": "Strength:\n1. This paper is the first work to investigate FL’s theory on neural network classification under mixed-type labeled data.\n\n2. This paper is easy to read.\n\n3. The authors propose a theory-guided and model-agnostic approach that can use the underlying correspondence between those label spaces and can be combined with various FL methods.\n\n4. The proposed method achieved relatively good performance than the baseline methods.\n\nWeaknesses:\n1. The authors implement the experiments with the CIFAR100 dataset and ResNet18. I am curious about the experiments with large datasets, such as ImageNet, or lightweight architectures, like MobileNetV2.\n\n2. The authors propose two kinds of projections for servers and clients, as in Eq. (3) and Eq. (4). I am curious about these two projections' effects on the proposed algorithm.\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "jluxj864db", "reviewer": "Reviewer_23rr", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The authors address the problem of federated learning in the scenario where clients have differently defined label sets (assuming a super set of labels and a subset with and without partial overlap), but without requiring cross-client data relabeling. Further they consider also the setting where only a small portion of the data (in the specialized server) is labeled and labels may be noisy, They also provide theoretical results concerning the convergence of the proposed approach.", "summary_of_the_review": "The proposed approach constitutes a first step in the development of federated learning systems with nested or partially overlapping endpoints, while allowing for clients/servers with small sample size and label noise. However, the assumptions (known and fixed Q and T, one common subset of labels for all clients, noise labels only in the server, etc.) and experiments underscore the limitations of the proposed approach. Specifically, it narrows down the applicability of the proposed approach to very specific and arguably rare practical situations.", "strength_and_weaknesses": "The proposed approach is well motivated, justified from a practical perspective and supported by theoretical results. The experiments demonstrate both the performance of the model relative to competing approaches and support the theoretical findings. The main weaknesses are: i) the need for known and fixed Q and T matrices, ii) the assumption that only the specialized server suffers from label noise, iii) that though the proposed approach is proposed as a federated learning approach, the fact that only two sets of labels are considered (one containing the other) reduces the federated aspect to the model to simply applying FedAvg with the corrupted label learning of (Van Rooyen & Williamson, 2017; Patrini et al., 2017), which can be seen by comparing Algorithms 1 and 2. Note though that the latter could be perceived as a strength of the proposed approach except that it clearly separates the members of the federation as i) those with the larger set of labels which may be affected by noise, and ii) those with a smaller but common set of labels that are not only cheaper to obtain but are noise free. These two assumptions greatly limit the applicability of the proposed approach in general federated learning settings.\n\nThe authors make the assumption that there are only two label spaces, however, such an assumption lacks a practical justification, more so considering the original motivation of the approach (from the paper: \"standards for disease diagnosis may be different across clinical centers due to varying levels of expertise or technology available at different sites\"). Though, the authors claim that extension is easy, it is not clear why being easy, is not explored.\n\nThe structure of the proposed framework raises several questions that are not addressed in the paper:\n- why there is a need for a server that both handles the aggregation but also contributes data. This is not standard in FL, arguably unnecessary, and may result in privacy issues or at the very least power imbalance issues (the specialized server has access to the model, and updates from all the clients while at the same time having a local model and access to data).\n- why only assume that the specialized server has noisy labels?\n- from the text it is not clear under which assumptions does Theorem 3.1 holds and how those relate to the practical setting considered by the authors.\n\nFor a benchmark experiment, it is justifiable that T and Q are not only known but easy to obtain, however, in practice, estimating T is in general difficult and Q is complicated for non-nested cases.\n\nThe paper specifies that the PD data from Qin et al. (2020) is synthetic, however, other places in the text and the supplementary material suggest that the data is real. Can the authors please clarify? The reasoning for the question is to better understand how realistic are the assumptions about Q and T in relation to whether the data, endpoints, noise or sub-classes (with overlap) were artificially crafted.\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "Tls7qjowBs7", "reviewer": "Reviewer_B327", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The authors propose to tackle the potential mixed-type label problem in federated learning. Specifically, two types of projections are introduced to both server and clients to achieve the alignment, which can be also generalized to noisy label scenario. The authors also provide theoretical convergence analysis. In the experiments, the authors conduct evaluation on CIFAR-100 and sEMG datasets, the proposed projections can significantly improve the performance compared to other SOTA federated learning baselines.", "summary_of_the_review": "Overall, I think this paper is interesting and solid. It would be better if the authors can provide more discussion of two types of projections and demonstrate the scenario where probability/label projection shows superiority.", "strength_and_weaknesses": "Pros:\n\nThe paper is well-written and well-motivated. The discussed topic is challenging and important to the federated learning in real scenarios. The proposed projections are easy to follow and seem natural for the mixed-type labels. The authors provide theoretical analysis of the convergence of proposed algorithm. The experiments are extensive. The evaluation is conducted on various datasets and the proposed algorithm is compared to various SOTA baselines. The results are promising and convincing.\n\n\nCons:\n\nMy major concern lies in the advantage of two proposed projections. The authors introduce two projections align the prediction of different types of labels to the true label space, which are applied to probability and label respectively. Although the theorical analysis focuses on the label projection and shows better convergence, the results in Table 1 and Table 2 show the superiority of probability projections over label projections. It is difficult to see the necessity of proposed label projections.\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "D8vgZEdeaE-", "reviewer": "Reviewer_N8Hr", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper mainly focuses on a practical scenario of FL where the labels might be marked by different criteria over different centers/nodes. A new learning strategy is proposed for dealing with the mismatches of the labels, and it can be deployed with existing FL methods. Theoretical analysis shows that the proposed FedMT is able to find the global optimal solution for this class of problems at a linear convergence rate. Multiple numerical results further verify the intuition of this method with the experiments on real datasets.", "summary_of_the_review": "In summary, the work indeed brings the mismatch issue in the FL setting, but a thorough discussion on FedMT should be carefully studied, including projection matrix design, convergence analysis, communication efficiency, etc. ", "strength_and_weaknesses": "The strength of this work is mainly about the consideration of a new setting in the FL system. The mismatch among the center do exist in practice. To the best of my knowledge, this would be the first work addressing this issue. The proposed algorithm seems simple and easily implementable. The numerical results are encouraging. \n\nThe main weaknesses of this work are as follows:\n\n1) it is quite confusing about the projection matrices. It seems that this idea is borrowed from the other (centralized one). There are many issues missing, e.g., how to choose these matrices so that they are invertible? it seems that they are given as prior. Then, why do claim the agonistic? \n\n2) The convergence results are built upon the NTK regime. Why is this regime considered? Convergence on the first-order stationarity for nonconvex loss function is widely used and applicable for general problem beyond neural nets.\n\n3) It is claimed that even SGD used in this work can achieve the linear convergence rate and a step size of independent on iteration is used. Then, how about the gradient descent case? How does stochastic gradient estimate error play into the convergence result?\n\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "2: Several of the paper’s claims are incorrect or not well-supported.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "Not applicable"}, "extra_text": {}}]

Author rebuttal:

Decision: Reject