Assisted Learning for Organizations with Limited Imbalanced Data
Authors: Cheng Chen
Jiaying Zhou
Jie Ding
Yi Zhou
Venue: Accepted by TMLR ()

Abstract: In the era of big data, many big organizations are integrating machine learning into their work pipelines to facilitate data analysis. However, the performance of their trained models is often restricted by limited and imbalanced data available to them. In this work, we develop an assisted learning framework for assisting organizations to improve their learning performance. The organizations have sufficient computation resources but are subject to stringent data-sharing and collaboration policies. Their limited imbalanced data often cause biased inference and sub-optimal decision-making. In assisted learning, an organizational learner purchases assistance service from an external service provider and aims to enhance its model performance within only a few assistance rounds. We develop effective stochastic training algorithms for both assisted deep learning and assisted reinforcement learning. Different from existing distributed algorithms that need to frequently transmit gradients or models, our framework allows the learner to only occasionally share information with the service provider, but still obtain a model that achieves near-oracle performance as if all the data were centralized.

Published in Transactions on Machine Learning Research (05/2023)
Assisted Learning for Organizations with Limited Imbalanced Data
Cheng Chen*
u0952128@utah... edu
Deportment of Electrical and Computer Engineering University of Utah Salt Lake City, UT 84112
Jiaying Zhou'
zhou105, 1054 Gumm edu
School of Statistics University of Minnesota Minneapolis, MN 55455
Jie Ding School of Statistics University of Minnesota Minneapolis, MN 55455
dingi& edu
Yi Zhou
yizhouuutaah edu
Department of Electrical and Computer Engineering University of Utah Salt Lake City, UT 84112
Reviewed on OpenReview: https: // openreview. net/ formm? id- SEDWIAcPWA
Abstract
In the era of big data, many big organizations are integrating machine learning into their work pipelines facilitate data analysis. However, the performance of their trained models often restricted by limited and inbalanced data available to them. In this work, we develop assisted learning framework for assisting organizations to improve their learning performance. The organizations have sufficient computation resources but are subject to stringent data-sharing and collaboration policies. Their limited imbalanced data often cause biased inference and sub-optimal decision-making In assisted learning, an organizat ional learner purchases assistance service from an external service provider and aims to enhance its model performance within only few assistance rounds. We develop effective stochastic training algorithms for both assisted deep learning and assisted reinforcement learning. Different from existing distributed algorithms that need to transmit gradients models frequently, our framework allows the learner to only occasionally share information with the service provider, but still, obtain a model that achieves near-oracle performance as if all the data were centralized.
1 Introduction
Over the past decade, machine learning has demonstrated its great success in various engineering and science domains, e.g., robotic control (Kober & Peters 2014 Deisenroth et al. 2013 natural language
processing (Li et al. 2016 Bahdanau et al. 2017), computer vision (Liu et al. 2017 Brunner et al.] (017)).
finance (Lee et al.) 2020 Lussange et al. 2021 Koratamaddi et al.) 2021). Following this trend, many
organizations, c.g., governments, hospitals, schools, and companics, are intograting machine learning models
"Equal contribution

---

Published in Transactions on Machine Learning Research (05/2023)
into their work pipelines to facilitate data analysis and improve decision- making. For example, according to a recent survey (Financesonline) 2021), about 49% of companies worldwide are considering to use machine learning, 51% of organizations claim to be early adopters of machine learning, and the estimated productivity improvement obtained by learning models can be as high as 40%.
Although machine learning techniques have been standardized and are relatively easy for organizations to implement with real-world data, the model performance critically depends on the quality of the training data they hold [Goodfellow et al., 2016). However, these training data may be of limited size or biased toward certain distributions. For example, for local banks collecting financial data from economic activities, the sample size is often limited by the size of the local population, and the data distribution is affected by the local economy. Consequently, machine learning models trained on such limited and imbalanced data may generalize poorly on test data. Therefore, there is a need to develop a modern machine learning framework that can assist organizations in improving their model performance.
A natural solution is to connect the organization to an external consulting agent, who has machine learning expertise and holds large amount of data that is balanced complementary to the organization's 's imbalanced data. For example, for a local small hospital, the consulting agent may be a large medical center at the state or national level. In particular, the organization can purchase some generic assistance service (e.g., information exchange) from the consulting agent to help improve its model performance. However, organizations' interactions with external consulting agents are usually subject to stringent regulations and policies. Two common restrictions are: (i) neither the organization nor the consulting agent wants to share their raw data with the other side; and (ii) the organization often has a limited budget and can purchase very limited services from the agent. Considering these challenges for organizational machine learning users, it is desired to develop a machine learning framework for organizations to significantly improve their model performance by purchasing limited assistance services from external agents without data sharing. This constitutes the overarching goal of this work.
In this work, we develop an assisted learning framework in the "horizontal-splittingg" setting, where the organization and the consulting agent possess different sets of data samples that are utilized for training a common model. In our context, the organization's 's data are assumed to be limited and imbalanced, while the consulting agent's data are large complement the organization's data. Our assisted learning framework is inspired by organizations common characteristics: they may have a limited budget for the rounds of purchasing external assistance, yet they can exchange a large amount of side information per assistance round to maximize the performance gain. We summarize our contributions as follows.
1.1 Our Contributions
We formally define an assisted learning framework with "horizontalspplttinng data. In this framework, an organization (referred to as the learner) is connected to an external service provider (referred to as the provider) for assistance, and their interaction is subject to the following rules: (i) Neither the learner nor the provider can share data with each other; (ii) The learner can interact with the provider for only few assistance rounds due to limited budget, and it aims to maximize the performance gain; and (iii) In each assistance round, both the learner and the provider have access to sufficient computation and communication resources.
We then develop a fully-decentralizeed assisted learning algorithm named Assist.Deep for deep learning tasks. This algorithm enables the learner to effectively interact with the provider without sharing data. Specifically, every assistance round of AssistDeep consists of two phases. In the first phase, the learner performs local training for multiple iterations and sends the generated trajectory of models together with their corresponding local loss values to the provider. In the second phase, the provider utilizes the learner's 's information to evaluate the global loss of the received models and picks the model that achieves the minimum global loss as the initialization. Then, the provider performs local training for multiple iterations and sends the generated trajectory of models together with their corresponding local loss values to the lcarner. Finally, the learner utilizes the provider's information to evaluate the global loss of the received models and outputs the best model that achieves the minimum global loss. Moreover, for reinforcement learning tasks, we further propose
2

---

Published in Transactions on Machine Learning Research (05/2023)
fully-decentrallized assisted learning algorithm named AssistPG, which is based on the policy gradient algorithm and has the same training logic as that of Deep.
Through extensive experiments on deep learning and reinforcement learning tasks, we demonstrate that the learner can achieve a near-oracle performance with DesistDeep and AssistPG as if the model was trained on centralized data (i.e., the learner has full access to the provider's raw data). In particular, as the learner data's 's level of imbalance increases, Deep can help the learner achieve a higher performance gain. Moreover, data are never exchanged in the assisted learning process for both participants.
1.2 Related Work
Assisted learning. The concept of assisted learning was originally proposed by the earlier work (Xian ct al. 2020). However, they consider a very different assisted learning setting, i.e., the 'vertical-plliting setting in which the organization and the consulting agent collect different features from the same cohort. This is in contrast with our setting where they hold the same features but different. data samples. Moreover, their assisted learning algorithm specially developed for regression-type tasks, whereas algorithms apply to both classification and reinforcement learning tasks with general deep models. Consoquently, our algorithm designs and application scenarios are substantially different. from the prior work.
Distributed learning. Distributed learning frameworks such as federated learning (Shokri & Shmatikov 2015 Koneeny et al. 2016 McMahan al., 2017) aim improve the Icarning performance for large number of learners that have limited data and computation ionuuaatiion/ael httpppaaaiioo/ommmnnnaaaiooo /commmmmmiiaaiio resources. These learning frameworks
are well suited for cloud systems and IoT systems (Ray) 2016 Gomathi et al. 2018) that manage numerous smart devices through wireless communication. In conventional distributed optimization, the data is evenly distributed among workers, which collaboratively solves a large-scale problem by frequently exchanging local ion (i.e., gradients or models) via either decentralized networks (Xie et al. 2016 Lian et al.
2017
2018) or centralized networks (Ho et al. 2013: Li et al., 2014 Richtarik & Takave 2016 Zhou et al.
2016
2018). As a comparison, our assist learning framework requires only a few interaction rounds between the learner and provider. This is particularly appealing for organizational learners, who can employ sophisticated optimization process locally while restricting the rounds of assistance.
2 Assisted Deep Learning
In this section, we introduce the assisted learning framework for deep learning lasks. Throughout the paper, L denotes learner who seeks assistance, and denotes a service provider who provides assistance to L.
2.1 Problem Formulation
We consider the case where the learner L aims to train a machine learning model 0 that performs well on its own datasct (LL)) and generalizes well to unseon data. In general, L can train a machine lcarning model by solving the empirical risk minimization problem minøge /(0; D(L),, where is the parameter space. Standard statistical learning theories show that the obtained model can generalize well to intact test samples under suitable constraints of model parsimoniousness (Ding al. 2018). However, when the learner's data D(I.) contains limited number of samples that are highly imbalanced, the learned model will suffer from overfitting or deteriorated generalization capability to the unseen test data.
is the loss on D(L) and θ
To overcome data deficiency, the learner L intends to connect with an external service provider P (e.g., a commercialized data company), who possesses data D(P) that are sufficient or complementary to the learner's data D(L), Tdeally, the learner L, would improve the model by solving the following data-augmented problem, where D(I.P) == D(L) u DPP) denotes the contralized data.
((LP)) = arg min f(O; (())). BEO
(1)
We note that f(0; ((1PP))))) = f(0, D(1)) ((8,((P) If D(P) generated from a distribution that is close the underlying data distribution, it expected that ((L.P) will achieve significantly improved performance on
3

---

Published in Transactions on Machine Learning Research (05/2023)
Algorithm Deei
Imput:
Initialization model 80, learning rate assistance rounds R, number of local iterations (for learner)
and T" (for provider).
for assistance rounds 1, ,...., R do Learner L:
Initialize 8% = f
Local training to generate T-1 TT!
=t=0
Send (((()) ; )EE to provider P
Provider p
Initialize (! arg min
DE mi (0 her ((0:)(
Local training to generate (0,-) (")
T-
Jt=u
Send {0(²), f(ol") her to learner L
Learner L
Output 8" = arg min, mi her ((0:D(1.P)
end
Output:
The best model {0")
Jr=1
unseen data. However, it is unrealistic to centralize the data since the interactions between the learner L and the provider P are often restricted by stringent regulations. More specifically, in the assisted learning framework, we consider the following protocols listed below.
Assisted Learning Protocols
1. No data sharing: Neither the learner L nor the provider P will share data with each other.
2. Limited assistance: The learner L has limited budget for purchasing assistance servicos. The learner desires to maximize the performance gain within only few assistance rounds.
3. Unlimited computation and communication: In each assistance round, both the learner and the provider can perform unlimited computation and exchange unlimited information.
To elaborate on the above protocols, first note that data sharing is often considered sensitive and prohibited in modern distributed learning. Also, assistance service between organizations is usually costly and time- consuming in reality, and organizational learners usually have a limited budget to purchase and manage such service. Morcover, we generally assume that both the organizational learnor and the service provider have sufficient compulation resources, and they can exchange unlimited information each interaction round, e.g., the learner (resp. provider) can send an employee (resp. technician) to deliver large-capacity hard drive the other side.
Conventional distributed learning algorithms cannot be applied to enable assisted learning, as they are developed for distributed systems involving large number of agents having access to very limited computation and communication resources, and they often require frequent information exchange among the agents. Hence, we need to develop a training algorithm specifically for assisted learning that can substantially improve the learner's model performance limited interactions with the service provider. Next, we present a generic assisted learning algorithm for solving deep learning tasks.
2.2 Deep for Assisted Deep Learning
We propose AssistDeep in Algorithm I for assisted deep learning. The learning process consists of R rounds, each consisting of the following interactions between the learner L and the provider P.
4

---

Published in Transactions on Machine Learning Research (05/2023)
(1) First, the learner L initiates a local learning process. It initializes a model ogd and applies any standard deep learning optimizer (e.g., SGD, Adam, etc.) with learning rate n to update it for T iterations using the local datasct D(L). Then, the learner evaluates the local loss f(; ((;)) in subset T of the iterations Lastly, the learner sends this subset of models and their corresponding local loss to the
= 0,1, provider P
(2) Upon receiving the information from the Iearner L, the provider P first evaluates the global loss f(:D(L)))) of the received set of models (0(t), ,t E T and picks the one that achieves the minimum global loss as the initialization model of"h. Note that the global loss can be evaluated because the local loss (s(0 D(D)), E are provided by the learner L, ; and the provider P just needs to evaluate the local loss {s(0) (1) D(P)) ) E on its local data. After that, the provider applies any standard optimizer with learning rate n to update the model for T iterations on the local dataset D(P). Then, the provider evaluates the local loss f(; D(P)) in a subset T' of the iterations i = 0, I..." - 1, and sends the subset of models and their corresponding local loss to the learner L.
(3) Once the learner L receives the information sent by the provider P, it evaluates the global loss P(L)))))) of the received set of models [0/-). . E T'} and picks the one that achieves the minimum global loss as the output model of this assistance round.
The above assisted learning algorithm works for general deep learning tasks. It does not require data sharing between the learner and the provider. In particular, the interaction between the learner and the provider has the following two prominent features.
Discussions.
Both the learner and the provider need to store a number of models sampled from the trajectory of models generated in their local training process and evaluate the corresponding local loss value of these sampled models. Then, these sampled models and their local loss values are sent to the other side. Normally, this information exchange may require a large amount of communication. But as we show later in the experimental studies, it sulfices to sample the training trajectory at low frequency.
After receiving the models and loss values sent by the other side, both the learner and the provider will evaluate the global loss of these models on the centralized data, and then will pick the one that achieves the minimum global loss as the initialization/ /output model. Here, the global loss of these models can be evaluated on one side, as the loss values of the models on the other side's local data are evaluated and sent by the other side.
2.3 Convergence Analysis of Assisted Learning
In this subsection, we show that assisted learning provably converges to a stationary point in smooth nonconvex optimization. For simplicity, consider the full gradient sctting, where the local training of AssistDeep uses full gradient updates. We also make the following standard assumptions.
Assumption 1. We assume that the assisted learning problem in the goal satisfies the following conditions.
1. The global loss f(0; ((LPP) has T--iipschizz gradients. Moreover, info [0; D(L.P) -0; (0: (V): S for all
2. There exists a constant G 0 such that max [ [[;; D( generated by AesistDeep.
Note that in each assistance round r, both the provider and the learner pick the best model via the arg min operation to initialize and finalize their local training, This guarantees that AssistDeep continuously makes optimization progress. Specifically, we let (L),r (()), denote learner L's and provider P's initialization models, respectively, in the round r. Recall that 0" is the output model of learner L (see Algorithm D. Then, the two arg min operations guarantee that the following proposition holds.
Proposition 1. The sequence of global loss {f(("; :D(,P))), achieved by Assist monotonically decreases, i.c.,
[(0", (O,.DL.P)) & s(0f))r :(LPP)) (o(f),))(LLP))) < 1(0 D(L,P)) /(0r-1, (.(L))
5

---

Published in Transactions on Machine Learning Research (05/2023)
Next, we prove that the output model 0" asymptotically converges to stationary point. The full proof is presented in the appendix of this paper.
Theorem 1. Let Assumption 1 hold and run Desi with full gradient updates for R rounds. Choose learning rate 1f - (((LTG*))-a)) Then, minog-sR-1 |
VV (0')l 4 0.
Therefore, with a proper choice of the learning rate n, assisted learning is guaranteed to find stationary point in general nonconvex optimization. We note that it is challenging to establish such a convergence result for AssistDeep, as the model chosen by the arg min operations can be any model in the entire model trajectories. As we show in the experiments later, our Deep can often achieve a similar performance to that of SGD with centralized data.
2.4 Comparison of Federated Learning and the Proposed Assisted Learning
Federated learning is a widely-adopted distributed learning framework (Shokii & Shmatikov 2015 Konecny et al.) 2016; McMahan et al., 2017 Zhao al. 2018 Li et al., 2020 Diao et al. 2021] 2022). It focuses learning global model by averaging local models trained numerous smart devicos, showcasing substantial differences from assisted learning in terms of application scenarios and algorithmss Firstly, federated learning seeks to enhance learning performance for multitude of learners with limited data and computation/ cnnputation/oommmiaattii /commuiicatio resources, making it well-suited for cloud and loT systems. Conversely, assisted learning strives to support a single organization-level learner (e.g., small labs or local clinics) in consistently and antiallyy improving learning performance through a limited number of interactions with an external organization-lewe service provider (e.g., research institutes or hospitals). Both learner and provider possess ample resources but are constrained by strict data-sharing policies. Secondly, existing federated learning algorithms necessitate frequent transmission and exchange of local information (i.e., gradients model parameters). In comparison, assisted learning algorithms mandate only few interaction rounds between the learner and provider. This approach is particularly attractive for organizational learners, as it allows them to utilize a refined optimization process locally while limiting assistance rounds to reduce the cost of acquiring services from external providers.
Moreover, in contrast to assisted learning (specifically, the Assist algorithm), is worth noting that the conventional federated learning framework, such as the FedAvg algorithm (McMahan et al. 2017), exhibits sensitivity to data heterogeneity and imbalance, even though FedAvg can address the same problem (involving two clients) as AesistDecp. To claborate, during each round of FedAvg, the cloud server compiles all the latest local models generated by the clients to derive a global model. This approach is susceptible to both local data heterogeneity and over-fitting during local training. In comparison, in Dessst both the learner and provider transmit their local models and corresponding loss values to the other party for evaluation, and the optimal local model that achieves the lowest global loss is selected as the initialization for the subsequent round. This filtering process is highly nonlinear, as opposed to the linear weighted combination utilized in FedAvg. To validate this, we examine the performance of FedAvg alongside that of our AssistDeep in experiments involving highly imbalanced data (see Appendix A.1). The results indicate that FedAvg converges slowly due to the significant data imbalance, while our AssistDeep demonstrates superior and more robust performance. This can be attributed to FedAvg only lincarly aggregating models generated during the final iterations of local training, whereas Desp conducts screening across all models produced during local training and selects the best one.
3 Assisted Reinforcement Learning
We further extend our assisted learning framework to Reinforcement Learning (RL) scenarios to enhance the model's generalizability. We first introduce some basic setups of RI.
Markov Decision Process (MDP). We consider a standard finite-horizon MDP that is denoted by a tuple M = (S, A,P, n, Po, T), where S the state space, A corresponds to the action space, P Sxxxxx + [0, denotes the underlying state transition kernel that drives the new state given the previous state and action, r S x A + R is the reward function, म S + is the policy, Po denotes the initial state distribution, and T is the episode length. Given policy no parameterized with 0, the goal of RI, also known on-policy
6

---

which is Published in Transactions on Machine Learning Research (05/2023)
learning, is to learn an optimal policy parameter H that maximizes the expected accumulated reward, namely 0* arg max, 9 [ E
3.1 Problem Formulation
We assume that RL learner L has collected small amount of Markovian data D(L) by interacting with cortain environment. It wants to train a policy that generalizes well to other similar environments. However, the data and environment that the learner L can access are limited. In assisted reinforcement learning, the learner aims to enhance its policy's generalizability to unseen environments by querying assistance from a service provider P. For example, autonomous-driving startup companies typically own limited data that are insufficient for training good autonomous driving models that perform well in heterogeneous environments, and they purchase assistance services from big companies (who own massive data) to improve the model performance and generalizability.
Formally, we assume that there is an underlying distribution of transition kernel that models the variability of the environment. Specifically, denote Fgg as an environment with the transition kernel Pa parameterized by 3, which follows an underlying distribution q. Let (0) denote the expected accumulated reward collected from the enviromment Eg following the policy my. The learner L's ultimate goal is learn good policy that applies to the underlying distribution of environment, namely, maxy Ez~g [J)( In practice, the learner L only has training data collected from a limited number of environment instances, say ((T.) { ..... , ".l".,..",,,, ). On the other hand, the service provider may have rich experience interacting with more diverse set of environments, say ((P) { ...... ".")) Consequently, the learner aims to solve the following RL problem with centralized data.
max J gLPP) (0) BEB0)
Σ Jz(0) +
Σ (0).
(2)
B'EB()
3.2 AssistPG for Assisted Reinforcement Learning
Policy gradient (PG) is classic RL algorithm for policy optimization. The PG algorithm estimates the policy gradient (+) via the policy gradient theorem, and applies it to update the policy, Specifically, given one episode 7 with length T that is collected under the current policy wo, the corresponding policy gradient takes the following form, where R(T) = Et= In practice, a mini-batch episodes is used to estimate the policy gradient (Sutton & Barto 2018 approximated as follows.
7f-1r, 1 is the discounted accumulated reward over this episode.
VJ(0) R(T) Σ V log To (a()) (0))
In Algorithm below, we present Assisted Policy Gradient (AssistPG) a policy gradient-type algorithm for solving the assisted RL problem in Equation (2). The main logic of the AssistPG algorithm is the same as that of the AesistDecp for assisted deep learning.
4 Experiments
In this section, we first visualize AssistDeep training to help understand the mechanism assisted learning. Then, we provide extensive experiments of deep learning and reinforcement learning to demonstrate the effectiveness of the proposed assisted learning algorithms.
4.1 Visualization of AssistDeep Training
Regression Example. We apply AssistDeop solve regression problem with simulated data a = 111, -1], [1, 1125 and hyperparameters T 10, r7 0.9" for both the learner and the provider. We run the algorithm for R = 10 assistance rounds. Fig. shows the learning trajectory of or for 0, 1,..,... 9. It
7

---

Published in Transactions on Machine Learning Research (05/2023)
Algorithm AssistPG
Imput:
Initialization model 80, learning rate assistance rounds R, number of local iterations (for learner)
and T" (for provider).
for assistance rounds ,,..., do
Learner L: Initialize 8% = -
Local PG training to generate (0!4)) 1i=0
Send (0(t) Esggas (((())))er to provider
Provider p
Initialize (1 arg max be her (8)
Local PG training to generate (0,-) 1T"0
1i=0
Send (0(²) Egegies (o(0) heT to learner
Learner L
Output 0" = arg max her JAU. (0)
end
Output:
The best model {0
Tr
(a) Assisted solutions
(b) Dracle solution 5%)
0.50
Us mit 6 the actooy Us ocal opt Ps local Giobal
(81.0%)
-0.5
-0.0
-0.55
Non-assisted solution (70.0%)
-1.00
-1.25
--..
Cass
Decision Boundary
Clese
Decillon Boundary
-1.50
Casa
Clase D
Figure 1: Learning trajectory of Assist in a synthetic regres- sion example.
Figure Visualization of Desi in classification: (a) the learner's classifiers after being assisted by the provider at different rounds, and (b) oracle classifier obtained by using centralized data. The test accuracies are shown in the parentheses.
be seen that the beginning, the learner L's learning trajectory moves toward the oracle solution since the directions of two local optima are roughly the same; then, it oscillates in between two opposite directions and converges to the oracle solution.
Classification Example. We apply Deep to solve a simple logistic regression problem for binary classification. We generate two classes of data samples: Class A data contains 50 points drawn from 1),-1,1), 1.522l)) and Class B data contains 50 points drawn from W(11, -1], 1.52 T2), where N and denote Gaussian distribution and identity matrix, respectively. Suppose that Icarner L. observes 90% class A samples and 10% class B samples. Another service provider P observes a similar number of data samples consisting of 10% class A samples and 90% class B samples. The learning process of Assist is illustrated in Fig. (Left), and we also show the oracle solution trained by SGD with centralized data in Fig. 2 (Right).
It can be seen that without any assistance, the learner can only achieve 70% accuracy and the corresponding classifier performs poorly on the samples in class B. In comparison, after a single round of assistance, the 81.9% accuracy. After three rounds assistance, the corresponding classifier is relatively close the oracle classifier, which achieves an accuracy 82.5%. Hence, it be seen that Assist.Deep has the potential to achieve near-oracle performance.
classification performance is significantly improved
8

---

Published in Transactions on Machine Learning Research (05/2023)
4.2 Assisted Deep Learning Experiments
We test the performance of AssistDeep by comparing it with two baselines: SGD (using centralized data D(I.P)) and Learner-SGD (using only the learner's data D(L).. We consider two popular datasets CIFAR-10 (Krizhevsky 2009) and SVHN. We implement all these algorithms using the Adam optimizer with learning rate 0.001 and batch size 256 to train AlexNet and ResNet- on CIFAR-10 10 and SVHN, respectively.
To test Deep, we split the classification datasct into two parts and assign them to the learner and the provider, respectively. Specifically, the provider's 's data consists of half of the total samples of each classification class, and therefore is balanced and of a large size. On the other hand, the learner's data is sampled from the rest half of the data (excluding the provider's data) and is determined by the data imbalance level parameter P == [D(LMIooo) // |/D([MANOO) E [o. 1], where D(IMAJOO) and D(LMINO)) denote the major class and minor class of the learner's local data, respectively, and we fix the major class to include the remaining data samples of the smallest class. Then, all the other classes are minor class of the learner's data and their sizes are given by Yz [[(sAAoo)) Intuitively, 7r mcans that learner's 's data balanced and large, whereas y = 0 mcans that learner's data is extremely imbalanced and small.
We fix the number of assistance rounds to be 10. The total number of local optimization iterations in each assistance round is fixed to be 2000. We assign these iteration budget to the learner and provider in proportion to their local sample sizes. Both the learner and provider record their local training models and local loss values for every = 50 iteration, which is referred to as the sampling period. In addition, we conducted repeated experiments with different random seeds to estimate the standard deviation of the above results. We find that the training loss and the test accuracy of the output model are highly consistent across the repeated experiments. Specifically, the standard deviation of the training loss and test accuracy are 4e-3 and 3c-3, respectively. Therefore, in all the figures corresponding to deep learning experiments, we only plot the curves using the data obtained from one particular experiment, and the curves of all the other repeated experiments are nearly identical.
4.2.1 Effect of Sample Size and Data Imbalance Level
CIFAR-10 Dataset. We first compare these algorithms with balanced learner's data (77. 1) and imbalanced learner's data (7 0, 0.3, 0.7) in training an AlexNet on the CIFAR-10 dataset. In Fig. 3 we plot the training loss (on centralized data D(L.P)) and the test accuracy (on the 10k test data) against the number of assistance rounds. Here, one assistance round on the x-axis is interpreted as 2k local iterations for SGD and Learner-SGD. The training loss of Learner-SGD is not reported as it is trained on D(L) only. It can be seen that AssistDeep achieves a comparable performance to that of SGD with centralized data. In particular, when ? = O, meaning that the learner has limited and extremely imbalanced data, the test. performance of Assist Deep is significantly better than Learner-SGD, demonstrating the effectiveness of querying assistance from the service provider. When 7 = 0.3, 0.7, 1 and the learner has more data, AssistDeep still achieves a near-oracle performance, and its test performance is better than Learner-SGD.
We further test and compare these algorithms with balanced learner's data "L 1 and imbalanced learner's data 74- = 0, 0.3, 0.7 in training a R8 on CIFAR-10. The results on ResNet-18 are shown in Fig. 4 It can be seen that in all the results, Assist achieves a comparable test performance to that of SGD. Also, its test performance is better than Learner SGD, especially when the learner has limited and imbalanced data (yL = 0.3, 0.7). Combining these results with those in Fig. 3 we conclude that Assist.Deep improves the test performance more (compare to Learner-SGD) when the learner's data is more imbalanced and limitod.
SVHN Dataset. In Fig. 5 and 6 repeat the above experiments with a different SVHN dataset using the same hyper- parameter settings. One make the same observations from these results, which show that our proposed AssistDeep works well on diverse types of datasets.
4.2.2 Effect of Sampling Period
On CIFAR-10, we explore whether incrcasing the sampling frequency of the model and loss value can improve the performance of AssistDeep. We consider 72 = 0, and compare AssistDeep with centralized SGD under different sampling periods 20 and 50. The comparison results in training AlexNet and 18 are shown in
9

---

Published in Transactions on Machine Learning Research (05/2023)
CIFAR-10, AlexNet, 2
CIEAR-10, AlexNet, 71-0.3
2.5
80
80
9-30 (haanbon Dese
G-5 (uniin A-AaeasDop
Test Accuracy
60
Teot Accoracy
60
Train Loss
1.5
1.5
40
40
0.5
20
Astetileep
0.5
20
Leamer SGD (Daoeltne)
SGD (Baseline)
Aerictioog
0
3
5
10
3
6
5 Number of Rounde
Namber Rounds
N Number of Rouads
Number of Rounds
CIFAR-10, AlexNet, -07
CIFAR-IO, AlexNet,
SUDD [[asslin))
Ei SUD (Bawiion)) AndetDeep
Aeciae Deeg
Thes Accuracy
60
Too Accuracy
Thain Lose
Thain Loss
2.5
1.5
40
40
SGD (Baxclno)
20
(Beclize)
0.5
Leaina SUD (Baxeloo) +h AsiistDeep
0.5
Lex (Baellax)
Dvep
Number of Rounds
Number Rounds
Number of Rounds
Number Rounds
Figure 3: Comparison of Aesist SGD, and Learner-SGD with 7%2 = 0, 0,3,0.3.000 0.7,1 in training an AlexNet on CIFAR-10.
CIFAR-10, ResNet-18, 2r-0
CIFAR-10, ResNet-18, 70.3
2.5
100
2.5
100
SGD [Bescline DesdeDDap
(Badine) Ao-AAcssDDeppp
80
80
Test Accurscy
Accuracy
Towin Lons
1.5
SGD (lneehmme)
Learner 8GD (Dacelme))
(Haestmr)
0.5
20
Anterleep
0.5
20
Laursan-8CD (Buseline)
Deep
0
10
10
Number Rounds
Number of Rounds
Number of Rounds
Number of Rounds
CIFAR-10, ResNet-18,
707
CIFAR-10, Nes- 72
2.5
100
2.5
100
B-D (Buallina) At-Anaativpp
Deop
Thain Loos
Test Accurary
Train Loss
Test Accmony
40
40
9-5010 (Bowelinn)
Lo SCD (Buscaas)
0.5
20
-aseeerSGD (Baeline)
0.5
20
x. Laasar-SGD (Buseins)
2. Assaallvup
7. AsstDDopp
0
0
5
Number of Rounds
Number of Rounds
Number of Rounds
Number of Rounds
Figure 4: Comparison of Deep, SGD, and Learner-SGD with 7%. = 0.3, 0.7, in training a ResNet-18 on CIFAR-10.
Fig. and g respectively. It can be seen that using a low sampling frequency for AssistDecp already achieves the baseline performance of SGD. It implies that AssistDeep does not require much information exchange between the learner and provider. This helps save computation resources and reduce information leakage.
10

---

Published in Transactions on Machine Learning Research (05/2023)
SVHN, AlexNet, 70
SVHN AlexNet, 7r- 72.0.3
Ho SCID (lnseline)) Aesk
Hb-SDD (Baxcino)) -trrssceiDepp
80
Testt Accuracy
Avcurecy
Towin Leos
1.5
60
1.5
60
1
40
O SUD (Bonelnn))
40
Leanee SGD (loelitr))
SGD (Bacctor)
0.5
20
+ AndettDeep
0.5
20
L-Learner 50D (lteclnn)
AstisDepp
0
Number Rounds
Number Rounds
Number Rounds
Number of Rounds
SVHN, AlexNet, 1/=
SVHN, Ale
72
2.5
100
2.5
100
(aasin))
(Bascioo) Assletibep
Doge
80
80
Tess Accurory
Thst Accuracy
Toin Looe
Thiin Loss
1.5
60
1.5
B-SGD (ESaniinn)
DD-GD [[3sa6in))
LemmerGGBB (Bawllin)) 9 A-Aceellpppp
20g
Leamer-SG (Baseine)
1
3 4
2 13345777 8
2
4
10
0
2 1
6 10 Number of Rounds
53345677
Number of Rounds
Number of Rounds
Number of Rounds
Figure 5: Comparison of DesistDeep, SGD, and Learner-SGD with 7.. 0, 0.3, 0.7, in training an AlexNet SVHN.
SVHN, ResNet-18, yy-
SVHN ResNet-18, y2
2.5
100
2.5
100
[Baacine
(Badinn
Doos
80
80
Accussyy
Train Loss
60
(Boseliac)
AndetDbopp
(Booelioe)
Awdsttlepp
3
5
8
10
3
8
3
6
8
10
5
Number
Rounds
Number Rounds
Number of Rounds
Number of Rounds
SVHN ResNet-18, 7t. 0.7
SVHN ResNet-18, 7x=1
2.5
100
2.5
100
to [lassine)
SEGD (lowelion)
d AndetDDepp
80
80
Test Accuracy
Teot Accuracy
Toein Loos
Thsin Loos
2.5
60
1.5
60
40
40
(Bneelan))
(Uaneline)
0.5
SCD (Badiins) DairiDapp
0.5
(Iasellne)
2
2
10
Number
Figure 6: Comparison of AssistDeep, SGD, and LearnerSGGD with 7%. 0, 0.3, 0.7,1 training a ResNet-18 on SVHN.
4.3 Assisted Reinforcement Learning Experiments
We demonstrate the effectiveness of AssistPG via two RL applications: CartPole (Brtoo et al.) 1983) and LunarLander (Brockman et al. 2016). In the CartPole problem, a controller aims stabilize a pole attached cart by applying left or right force the cart (see the first figure in Fig. 9h, and we show that AssistPG can help the controller stabilize the pole with different pole lengths. For the LunarLander problem, lander
11

---

Published in Transactions on Machine Learning Research (05/2023)
CIEAR-10, AlexNet,22
CIFAR-10, AlexNet, yx=1
2.5
80
2,5
80
-0-0i1 (loesline)
5-5GD [Bedine)
Ancetlhep with Prr
de AosstDapp with Perkel22
- Anceetlhepp with Prrodi:9
Tees Acmrory
60
E-oostDap with Perkot-5
Test Accuncy
60
Triin Loos
Trai Lass
1.5
40
40
20
SCD (Bosiin))
20
NeixDeup with Polld=30
Parted-21
NeirDDey Poind-1o
8
AmieiDaop Perted
4 6
2 3 4 5 Number Rounds
10
2
0
2 4
Number of Rounds
Number of Rounde
Number of Rounds
Figure 7: Comparison of AssistDeep and SGD with y 0, under different sampling periods in training AlexNet on CIFAR-10.
CIFAR-I0, ResNet-18, 72
CIFAR-10, ResNet-18, 72_
2.5
3-5GD (Boneline)
100
100
AsdettDepp Period-20)
F..AassDeesp Period-00
Thst. Accuracy
Dost Accuracy
80
1.5
Train Loss
1.5
60
Toin
40
O SGD [Lassinno]
0.5
20
d- Anest with Preseid0
Avasc Aopp with Perwindd
0.5
20
AmselDopp with Peridd:00
Anselinpp with Peried 20
0
Number of Rounds
Number of Rounds
Number of Roumdb
Numbet of Rounds
Figure 8: Comparison of Deep and SGD with YL = 0, under different sampling periods in training a ResNet-18 on CIFAR-10.
initializes its landing from the top left of the sky and aims to land on a landing pad by controlling its engine (see the first figure in Fig. 10p. We show that AssistPG can help land the lander with different engine powers.
100
PG (Baseling)
(Baseine)
800
Learner PG (Baseline)
800
Leamer-Be PG (Baseine)
Test Roward
Test Roward
Discount
60
400
left
right
20
Assin1PG PG (Baseline)
200
200
Figure 9: Comparison of AssistPG, PG, and Learner-PG in the CartPole game.
50
PG (Baseline)
200
PG (Baseline)
Learner PG (Bazeline)
left
right
-50
Test Reewwrd
-300
Train Discouutt
down
landing pad
100
-150
-300
-00
AssistPG
-300
(Baseline)
-250
Learner-P (Baseline)
-000
550
-500
Rounds
Rounds
Figure 10: Comparison of AssistPG, PG, and Learner-PG in the LunarLander game.
12

---

Published in Transactions on Machine Learning Research (05/2023)
We assume that the learner and the provider query episode data by interacting with diverse environments. Specifically, for the Cart problem, we parameterize the environment using the pole length. Both the learner and the provider train their control policies by playing 5 Cartpole games with the pole length randomly generated from Uniform(4,5) (for the learner) and Uniform(O, 1) (for the provider). For the LunarLander problem, we parameterize the environment using the engine power. Both the learner and provider train their control policies by playing LunarLander games with the engine power randomly generated from Uniform 10, 15) (for the learner) and Uniform(35, 40) (for the provider). Moreover, we consider testing environments that are uniform ("Test 1*) and non- non-uniform (*Test II"), respectively. For the CartPole problem, Test T environments randomly generate the pole length from Uniform(O, 5), and Test II environments randomly generate the pole length from Beta(1,5) 5) with probability 0.2 and Uniform (0, 5) with probability 0.8. For the LunarLande problem, Test I environments randomly generate the engine power from Uniform(10, (10,40), and Test environments randomly generate the engine power as 30r + 10, where r ~ Beta(5, 1).
sets of
We test AssistPG on both RL problems and compare its performance with two baselines: the standard PG (using centralized episode data) and the Learner-PG (using only the learner's episode data). All these algorithms are implemented with the learning rate 5 X 10 and episode batch size 32. We model the policy using a three-layer foed-forward neural network with 4 and 32 hidden neurons for CartPole and LunarLander, respectively. Moreover, for AssistPG, we fix the total number of assistance rounds to be 10 and 5 for CartPole and LunarLander, respectively. The total number of local PG iterations in each assistance round is fixed to 20 for both problems. We also set the sampling period to be four, namely, the learner and the provider record their local model and discounted training reward for every four local PG iterations. Fig. and I0 plot the discounted training rewards (collected in local enviromments only), Test cumulative rewards, and Test II cumulative rewards against the assistance round obtained by all these algorithms, for solving the CartPole and LunarLander problems, respectively. Here, one assistant round is interpreted 20 local PG ilerations for algorithms other than AssistPG.
Fig. indicates that AssistPG outperforms Learner-PG, when the testing environments include diverse lengths of poles. Also, AssistPG can achieve comparable performance to that of the PG with centralized data. Moreover, Fig. [0] indicates that AssistPG can swiftly adapt scenarios out of their comfort zone (namely the training environments) in only few rounds. These experiments demonstrate that assisted learning framework can help the learner significantly improve the quality of the policy for handling complex RI. problems.
4.4 Visualization of LunarLander Experiment
engine power = 10
engine power 20
engine power 30
engine power 40
AssistPG Leamor-RG
AssistPG Leamor-PG
Assis Leamer-PG
Assin
Loamer-IG
Round
Round
Round
Round
AccistRS Leatnee-PG
As PG
AssisiP Leamen
Leamen-PG
Round 5
Round 5
Round
Round
Figure 11: Landing traces of LunarLander with engine power 10, 20, 30, 40 trained by AssistPG and Learner-PG.
In this section, we visualize the landing trace of the LunarLander trained by AssistPG and Leaner-Pa in different test environments.
13

---

Published in Transactions on Machine Learning Research (05/2023)
Test
Test II
AssitiP
AssisARG
AssistPG Learner-PG
AssistPG Learnen-P
Leamen-P PG
AssistPG
AssstPG Leamer--G
Ass877G
Assistt
Figure 12: Landing traces with engine power Uniform 40) and ~ 30 Beta + 10 trained by AssistPG and Learner-PG.
Specifically, we consider a fixed map and set the engine power of the lander to be 10, 20, 30, and 40, respectively. In each setting, we train the lander using both AssistPG and Learner-PG for R = rounds. After each round of training, we let the lander play an episode using the trained model and plot the corresponding landing trace. These traces are plotted in Fig. [T] From these figures, it can be seen that the lander with engine power 20-40 trained by AssistPG can successfully land on the landpad after 5 rounds of assisted learning. As comparison, the lander trained by Learner-PG cannot even land after 5 rounds of training. This demonstrates the advantage of AssistPG. On the other hand, when the lander has small engine power 10, it is challenging for both algorithms to land the lander properly, as the engine cannot provide sufficient acceleration. Moreover, after 5 rounds of training (using both AssistPG and Learner-PG), we test the lander in both the test environment ("Test I") and II ("Test IT"), and plot. the landing traces in Fig. 12 Here, for each test, we consider fixed map and randomly generate different engine powers from Uniform 10, 40) (for Test I) and 30- (5ta(5, 1) + (for Test IT).
From both figures, can be seen that the lander trained by the AssistPG lands smoothly in all test. environments under diverse engine powers than that trained by the Learner-PG. The video version for the CartPole and LunarLander games can be accessed from the anonymous link https::/ww// htctp:://ww.drrpboooooo com/ sh/o22jswj361 361 141 141kh/ BAA.A//DDDDAA....AAAAAAALLLLLA..
In the CartPole game, four videos record the performance of AssistPG and Learner-PC against the first five rounds with pole lengths equaling 1, 2, 3, and 4, respectively. Another two videos record 10 plays in the test environment and II, respectively.
all the plays, both AssistPG and Learner-PC. use the model trained from the fifth round. In the LunarLander game, four videos record the performance of AssistPG and Learner-PG against the first five rounds with engine power equaling 10, 20, 30, and 40, respectively. Another two videos record 10 plays in the test environment and II, respectively. In all the plays, both AssisUPG and Learner-PG use the model trained from the fifth round. The videos show that with assistance from the provider, the learner can quickly generalize its model to more diverse environments.
5 Conclusion
This work develops a learning framework for assisting organizational learners to improve their learning performance with limited imbalanced data. In particular, the proposed AssistDeep and AssistPG allow the provider to assist the learner's training process and significantly improve its model quality within only a few assistance rounds. We demonstrate the effectiveness of both assisted learning algorithms through experimental studies. In the future, we expect that this learning framework can be integrated with other learning frameworks such as meta-learning and multi-task learning. limitation of this study is that it only considers pair of learner and provider. An interesting future direction to emulate the current assisted learning framework allow multiple learners service providers.
14

---

Published in Transactions on Machine Learning Research (05/2023)
Acknowledgments
The work of Cheng Chen and Zhou was supported part by U.S. National Science Foundation under the Grant. Nos. CCF-2106216, DMSS-134423 and CAREER-2237800. The work Jiaying Zhou was supported in part by Army Research Office under grant number W9L1NF-20-L-0222 The work of Jie Ding was supported in part by U.S. National Science Foundation under the Grant. Nos. ECCS-203860)) DMS-2134148, and CNS-2220286.
References
Martin Abadi, Andy Chu, Ian Goodfellow, H. Brendan McMahan, llya Mironov, Kunal Talwar, and Li Zhang. Deep learning with differential privacy. Tn Proc. Conference Computer and Communications Securily, pp. 308 318, 2016.
Damitry Bahdanau, Philemon Brakel, Kelvin Xu, Anirudh Goyal, Ryan Lowe, Joelle Pineau, Aaron Courville, and Yoshua Bengio. An actor-critic algorithm for sequence prediction. In Proc. International Conference on Learning Represenlations, 2017.
Andrew G Barto, Richard S Sutton, and Charles W Anderson. Neuronlike adaptive elements that can solve difficult. learning control problems. IEFE transactions systems, man, and cybernetics, (5):834-844) 846, 1983.
Greg Brockman, Vicki Cheung, Ludwig Pettersson, Jonas Schneider, John Schulman, Jie Tang, and Wojeiech Zaremba. Openai gym. arXiv preprint arxi::1600 10:.1606.000 01 540,, 2016.
Gino Brunner, Oliver Richter, Yuyi Wang, and Roger Wattenhofer. Teaching a machine to read maps with deep reinforcement learning. In Proc. Association for the Advancement of Artificial Intelligence (A. (AAAI, 11 2017.
Luke N Darlow, Elliot J Crowley, Antreas Antoniou, and Amos J Storkey. Cinic-10 is not imagenet or cifar-10. arXiv preprinl arXiv:1810 (11::181003355,, 2018.
M. P. Deisenroth, G. Neumann, and J. Peters. A Survey Policy Search for Robotics. 2013.
Enmao Diao, Jie Ding, and Vahid Tarokh. HeteroFL: Computation and communication efficient federated learning for heterogeneous clients. Proc. ICLR, 2021.
Enmao Diao, Jie Ding, and Vahid Tarokh. SemiFFL: Semi-supervised federated learning for unlabeled clients
with alternate training. In Alice H. Oh, Alekh Agarwal, Danielle Belgrave, and Kyunghyun Cho (eds.), Advances in Neural Information Processing Systems, 2022. URL https://openrevieek htw.pppaareeeeeeeeeeeeeorrrraooa net/forum?id=
[GAjC_Faute
Jie Ding and Bangjun Ding. "To Tell You the Truth" by Interval-Private Data. IEEE BigData, 2020.
Jie Ding, Vahid Tarokh, and Yuhong Yang. Model selection techniques an overview. IEEE Signal Process.
Mog., 35(6))16 34, 2018.
Cynthia Dwork and Kobbi Nissim. preacccpreeervinggg datamining on vertically partitioned databases. In Proc. CRYPTO, pp. 528 544. Springer, 2004.
Alexandre Evfimievski, Johannes Gehrke, and Ramakrishnan Srikant. Limiting privacy breaches privacy preserving data mining. In Proc. SIGMOD/PODSO3, pp. 211 222, 2003.
Financesonline. Market share & data analysis.
http:://finaccesolnn.cco
machine- machine-learning
2021. Accessed: 2021-01-18.
RM Gomathi, G Hari Satya Krishna, E Brumancia, and Y Mistica Dhas. A survey on iot technologies, evolution and architecture. In Proc. ICCCSP, pp. 1-5. IEEE, 2018.
Ian Goodfellow, Yoshua Bengio, and Aaron Courville. Deep Learning. MIT Press, Cambridge, MA, USA 2016.
15

---

Published in Transactions on Machine Learning Research (05/2023)
Geoffrey Hinton, Oriol Vinyals, and Jeff Dean. Distilling the knowledge in neural network. arXie preprint or
2015.
Qirong Ho, James Cipar, Henggang Cui, Seunghak Lee, Jin Kyu Kim, Phillip B. Gibbons, Garth Gibson, Greg Ganger, and Eric P Xing. More effective distributed ML via stale synchronous parallel parameter server. In Proc. NeurIPS, pp. 1223-1231. 2013.
Shiva Prasad Kasiviswanathan, Homin K Lee, Kobbi Nissim, Sofya Raskhodnikova, and Adam Smith. What can we learn privately? SIAM J. Comput., 40(3):793--8) 826, 2011.
Jens Kober and Jan Peters. Reinforcement Learning in Robotics: A Survey, pp. 9-67. Springer, 2014.
Jakub Konecny, H Brendan McMahan, Felix X Yu, Peter Richtárik, Ananda Theertha Suresh, and Dave Bacon. Federated learning: Strategies for improving communication efficiency. arXiv preprint arXi::1610.00 05492, 2016.
Prahlad Koratamaddi, Karan Wadhwani, Mridul Gupta, and Dr. Sriram G. Sanjeevi. A multi-agent reinforcement learning approach for stock portfolio allocation. In ACM IKDD CODS, pp. 410, 2021.
A. Krizhevsky. Learning multiple layers of features from tiny images. Technical report, 2009.
Jinho Lee, Rachyun Kim, Seok-Woo Yi, and Jaewoo Kang. Maps: Multi-agent reinforcement learning-base based portfolio management system. In Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence, IJCAI-20, pp. 4520 4526, 7 2020.
Jiwei Li, Will Monroc, Alan Ritter, Dan Jurafsky, Michel Galley, and Jianfeng Gao. Deep reinforcement learning for dialogue generation. In Proc. Conference on Empirical Methods in Natural Language Processing, pp. 1102-1202, 2016.
Mu Li, David G. Andersen, Jun Woo Park, Alexander Smola, Amr Ahmed, Vanja Josifovski, James Long, Eugene Shekita, and Bor-Yiing Yiing Su. Scaling distributed machine learning with the parameter server. Proc. OSDI, pp. 583-598, 2014.
Xiang Li, Kaixuan Huang, Wenhao Yang, Shusen Wang, and Zhihua Zhang. On the convergence of fedavg non-iid data. In Proc. ICLR, 2020.
Xiangru Lian, Ce Zhang, Huan Zhang, Cho-Jui Hsieh, Wei Zhang, and Ji Liu. Can decentralized algorithms outperform centralized algorithms? a case study for decentralized parallel stochastic gradient descent. In Proc. NewrIPS, 2017.
Xiangru Lian, Wei Zhang, Ce Zhang, and Ji Liu. Asynchronous docentralized parallel stochastic gradient 2018.
descent. In Proc. ICML, volume 80, pp.
F. Liu, Li, L. Zhang, C. Zhou, R. Ye, Y. Wang, and J. Lu. 3denn-dqn-run: A deep reinforcement learning framework for semantic parsing of large-scale 3d point clouds. In Proc. International Conference on Computer Vision (ICCV), pp. 5679-5688, 2017.
Johann Lussange, Ivan Lazarevich, Sacha Bourgeois- Stefano Palminteri, and Boris Gutkin. Modelling stock markets by multi-agent. reinforcement learning. Compulational Economics, 57, 01 2021.
Brendan McMahan, Eider Moore, Daniel Ramage, Seth Hampson, and Blaise Aguera y Arcas. Communication- efficient learning of deep networks from decentralized data. In Proc. AISTATS, pp. 1273 1282. PMLR, 2017.
Partha Pratim Ray. A survey of iot cloud platforms. Future Computing and Informatics Journal, 1(1-2): 35-46, 2016.
P. Richtarik and M. Takave, Distributed Coordinate Descent Method for Learning with Big Data. Mach. Learn. Res., 2016.
16

---

Published in Transactions on Machine Learning Research (05/2023)
Reza Shokri and Vitaly Shmatikov. Privacy-preserving deep learning. In Proc. CCS, pp. 1310-1321. ACM, 2015.
Richard S Sutton and Andrew G Barto. Reinforcement learning: An introduction. MIT press, 2018.
Xun Xian, Xinran Wang, Jie Ding, and Reza Ghanadan. Assisted learning: A framework for multi-organization learning. Proc. NewrIPS, 2020.
Pengtao Xie, Jin Kyu Kim, Yi Zhou, Qirong Ho, Abhimanu Kumar, Yaoliang Yu, and Eric Xing. Lighter- communication distributed machine learning via sufficient factor broadcasting. In Proc. UAT, pp. 795-804, 2016.
Yue Zhao, Meng Li, Liangzhen Lai, Naveen Suda, Damon Civin, and Vikas Chandra. Federated learning with non-iid data. ArXi::180600055 00582, 2018.
Zhou, Y.L. Yu, Dai, Y.B. Liang, and E.P. Xing. On convergence of model parallel proximal gradient algorithm for stale synchronous parallel system. In Proc. AISTATS, 2016.
Yi Zhou, Yingbin Liang, Yaoliang Yu, Wei Dai, and Eric P. Xing. Distributed proximal gradient algorithm for partially asynchronous computer clusters. J. Mach. Learn. Res,, 19(19):1 1-30, 2018.
17

---

Published in Transactions on Machine Learning Research (05/2023)
A Appendix
A.1 Federated Learning (FL)
Federated learning is an emerging distributed learning framework Shokri & Shmatikov 2015 Koneeny
et al. 2016 McMahan et al., 2017 Zhao et al. 2018 Li et al.) 2020; Diao et al. 2021 2022) that aims to learn a global model using the average of local models trained by numerous smart devices with possibly haiiaanccc//heeergenoooss data. The existing foderated learning algorithms require frequent transmissions of local model parameters. This is different from our solution designed for the organizational learning scenarios, where each learner is an organization that often has unconstrained communication and computation resources, but is restricted to interacting with external service providers. Our solution aims to help the learner improve learning performance within ten rounds, while federated learning needs many more rounds.
A.1.1 Comparison between FedAvg and AssistDeep
Experimental Setup. In this subsection, we compare the standard Fed algorithm @MeMahan et al., for federated learning with centralized SGD (bascline), Learner-SGD (baselino) and our Assist algorithm. We use the same datasets (CIFAR-10 and SVHN) and models (AlexNet and ResNet-18) as in Section 4.2 and keep all settings and hyperparameters unchanged. We implement all algorithms using the SGD optimizer with learning rate of 0.01 for AlexNet and a. learning rate of 0.1 for ResNet-18. The provider's data are uniformly sampled from all classes and are therefore balanced. The learner's data are also sampled with the same size as the provider's, ss but are taken from single class and are therefore extely imbalanced. For FedAvg, we treat the learner and provider as two federated learning clients/ /agents and use the same local data and local SGD iteration budgets as Ascis
CIFAR-10, AleeNet, Intulanced Leararr's Hata
CIFAR- ResNet-18, 18, Inbolanced Leurner's 's Data
SVII, AlreNet, Imbilancal Learner's Data
SVHN, Res Net-81,, Imhalanced I Learse''s Data
So-BiD SCD (Baalias)
e-sab (Bankar)
100
Tost Acsuracy
80
PilAv (Beerline)
Test. Accuracy
60
100
e Andetllepp
d RedAvg (Bartar)
-e La SCD (Baciind
Test. Accurayy
90
80
60
40-
$5
20
Leammn-SiD [Foueltor) A Boldby (Beacine)
9OD (RoneLoc)
AesistDerp
Tedt
60-
40-
80
20
Deeezeee SCD (Buxlius)
BodAAg (Bascino))
[Bandine)
20
Dwop
0
6 Numberr of Rounds
10
Numaber Rounds
Number of Rounds
Number Rounds
Figure 13: Comparison of AssistDeep, FedAvg, SGD, and Learner-SGD with imbalanced learner's data in training an AlexNet (left) and ResNet-18 (right) on CIFAR-10.
Figure 14: Comparison of AssistDeep, FedAvg, SGD, and Learner-SGD with imbalanced learner's data in training an AlexNet (left) and ResNet-18 (right) on SVHN.
Comparison Results for CIFAR-10 Dataset. We first compare these algorithms with extremely imbalanced learner's data in training an AlexNet and ResNet-18 the CIFAR-10 dataset, respectively. In Fig. 13 we plot the test accuracy against the number of assistance rounds for AlexNet (left) and Nes-Net-18 (right), respectively. For SGD, Learner-SGD and Deep, almost the same results can be observed as those shown in Fig. 3 (top left) and the same conclusions can be drawn as those made in subsection 4.2.1 (when the learner has extremely imbalanced data). Additionally, can be observed that our Assist Deep algorithm converges much faster and achieves a much better test performance than FedAvg for both AlexNot and ResNet-18. Specifically, with AlexNet, our Assist achieves about 60% test accuracy while FedAvg achieves about 27% accuracy; with ResNet-18, AssistDeep achieves about 71% test accuracy while FedAvg achieves about 40% (averaged) accuracy. This is because FedAvg simply averages the last local models generated by the two clients, and one of them is trained on highly imbalanced learner's data, which causes a significant. bias. These results demonstrate that our proposed Aesist algorithm is more robust to data imbalance compared to standard federated learning.
Comparison Results for SVHN Dataset. In Fig. 14 we repeat the above experiments with the SVHN dataset using AlexNet. (left) and R8 (right), respectively. From these results, one can make the
18

---

Published in Transactions on Machine Learning Research (05/2023)
same observations the previous CIFAR-10 results, which show that our proposed AssistDeep significantly outperforms FedAvg on diverse lypes of datasets. Specifically, with AlexNet, our DeeistDeep achieves about 84% test accuracy while FedAvg achieves about 57% accuracy; with ResNet-18, AssistDeep achieves about 87% test accuracy while FodAvg achieves about 74% accuracy.
Remark 1. Based on the experimental settings above, we can quantify the communication load of both AssistDeep and FedAvg as follows. In AssistDeep, as described in Algorithm | and Section 4.2 the number of learning rounds R = 10, the sampling period T= 50 iterations, and the numbers of local training iterations for the learner and provider are 7' and T" respectively, where T + T" = 2000. Suppose the model consists of n parameters, then the total communication load is nR(T + T = 400n for AssistDoep to converge. Note that Deep also communicates the losses between the learner and provider, but it occupies little communication so we just neglect them. As a comparison, the FedAvg algorithm in federated learning transmits two models per round but usually requires hundreds of rounds to converge, so its total communication load is of similar scale to that of AssistDeep. This proves that our assisted learning algorithm does not strictly rely on unrestricted setrriccteduuiimmiteed /unlimitedd commmication resources.
A.2
Ds Assisted Learning with Knowledge Distillation
A.2.1 Knowledge Distillation
Knowledge distillation (KD) is a machine learning technique that trains one classifier, called the student, using the outputs (also known as soft labels, which is a vector of probability scores assigned to each class) of another classifier, called the teacher (Hinton et al. 2015). It has been found to be often more efficient to train classifiers using soft labels rather than ground truth labels. The basic idea behind KD is to train a student model to match the output logits of a teacher model.
When defining the logit label of the teacher's model, a higher temperature t represents the softer distribution of output classes. Suppose the output has e classes. The i-th entry the teacher's logit label is defined as
exp(2r,(xx/1)
(x,, t) E exp (22, (x)
where 27, represents the pre-softmax layer associated with the i-th class, Similarly, we define the i-th entry in the student's 's logit. label as
exp(2s, (x, B)/t)
(x, B,t) Σ exp(=xs (x,
where B is the training parameters in student's model, and =s,( (8) is the pre-softmax layer associated with the i-th class given training model parameters B. We use the following loss function (Hinton et al. 2015 for knowledge distillation:
E(y, x) aa Σ (T,a(x log (x,,, - a1- Σ 1(v (x,
(3)
i=1
where a is a hyper-parameter that docides the weighting of soft labels and hard labels, and y the ground truth label. Here, the first. term in the above loss corresponds to the distillation loss between the teacher's soft label and the student's soft label, while the second term corresponds to the student's 's cross-entropy loss.
We adapt KD to the assisted learning framework and call it AssistKD. In AssistKD, knowledge distillation occurs in two stages: 1) When a learner learns its local models, it will distill every model generated in the local trajectory of models to the shared model architecture. To save computation, we use cach distilled model as a warm start for distilling the next in the trajectory. 2) When a learner picks the model that yields the lowest global loss, the learner will distill the chosen model to its local model architecture initialize the training of its local models
A.2.2 Experiments
We conduct an experiment where the learner and the provider train a ResNet-18 as their local training model. They communicate with each other by transmitting smaller-scale CNN model (shared between learner and
19

---

Published in Transactions on Machine Learning Research (05/2023)
CIFAR-10, Imbalanced Leamnr's Data
90
80
Test Accuracy
70-
60
50
40
30
SGD (Baseline)
Learner-SGD (Baseline)
20
AssistKD
10
4 5
6 7
9 10
Number of Rounds
Figure 15: Comparison of AssistKD (CNN as the commuicction model), SGD (using model ResNet-18), and Learner-SGD (using model ResNet-18) 18) on CIFAR-10.
provider). In this example, the shared model consists of six convolutional layers and three fully connected layers. It only has half of the trainable parameters compared with the trainable parameters in ResNet-18.
We test the performance of AssistKD on CIFAR-10 by comparing with two baselines: SGD (using centralized data D(I.P)) and Learner-SGD (using only the learner's data D(U)). We implement all these algorithms using the SGD optimizer with learning rate of 0.1 and batch size of 256 train CNN and ResNet-18 CIFAR-10. We assign different SGD iterations to the aforementioned two-stage knowledge distillation. In particular, we let the SGD iterations in the first stage be 10. In the second stage, we choose the SGD iterations that traverse the local training data for 60 times, i.e., 60 epochs. We choose hyperparameter t = and a = 0.9 in (3).
We compared these algorithms with imbalanced learner data, In particular, the provider's data are uniformly sampled from all classes and are therefore balanced. The size of the learner's data only 20% of the provider's sample size. Moreover, half of the learner data is taken from a single class, and the other half is uniformly sampled from other classcs. We fix the number of assistance rounds to be 10. The total number of local SGD iteralions in each assistance round is fixed to be 2000. We assign the SGD iteration budget to the learner and provider in proportion to their data samples' sizes. Both the learner and provider record their local training models and local loss values for every = 50 SGD iteration, which is the sampling period.
The lest performance shown in Fig. 15 From the plot, the performance of Assist approaches that of the standard SGD which trains a larger model (ResNet-18) on the centralized data. Morcover, the AssistKD significantly outperforms the Learner-SGD which trains on the learner's data alone. This demonstrates that our AssistKD enables the learner and provider to transmit models of different sizes/scales while keeping similar learning performance to the original centralized learning.
A.3 Proof of Theorem 1
For brevity, we denote the loss functions f(:D()) F(ID(P)), ((:D(LP)) as fin Sp. SL,,, respectively. We let || Il denote the Euclidean norm.
Consider any assistance round Recall that the learner L initializes with the output model obtained from the previous round, namely 0(.)rr = 0r-1, In the local training, learner L performs T local gradient descent. steps and generates the trajectory (0(4) yr Then, provider P picks the best model from this trajectory that achieves the minimum global loss, and we denote this model as (((),, for certain E (O, It clear that ( & (1. for all t. By smoothness of the global loss and the gradient descent update
20

---

Published in Transactions on Machine Learning Research (05/2023)
rule, we have that
fl.p(0((11)) s fi.p(0 (0())) u ( + 110 [10 1
L 2
- - + (-nV.n..0 (0 +
| (1 I|2
Since fi.r((f S (1.)(0(11), the above inequality further implies that
v ( (O. (())))))))))))))))))) S Ln ( LG 2
(4)
Next, consider the local training of the provider P. Recall that provider P iniiiiaizzss with the best model sent by the learner, namely 06th"" oftrr In the local training, the provider P performs T" local gradient ceeccntt steps and generates the trajectory 10t Me According to the smoothness of the global loss, we have that
f S fL.P((( + (0( o0 (OL ((())))))) L |10 fL.r(0 (-
6 1
(O ())))) + Ln
| fr (0 )r))
2
Lnj²
-Lrr(0
(O + 2
| (0 'r)1
- 1
(V ) (.. (o(L))
(0
LG
S $i.p(000 - n/ V.. )12
2 2
where the last inequality utilizes Inequality 4 and the boundedness of the gradient. Denote 0 as the best model from this trajectory that achieves the minimum global loss. The above Inequality (5) and Proposition further imply that
fi,r(0)) = (1.r(0(f)r) (for some k E To,...,?'})
S (o
$ (0 Phr)) 7 VL.
LG
S f //V (L.
12 LG2?
S (1 - V (0.
12 // -
+ 2nll fL. (1 fT.
( (1) PV (1) LG
2 fi. fi. - - +
where Inequality (i) uses the fact that
2n/ (f
(L)) (
(o a
< 2nGGL - 0 || 2r2
Σ Coi
2n
Telescoping the above inequality over r -,... and rearranging them, we obtain that
(L.((0011 fL.r(0)) - info ft.r(() + InLT
R
n
21

---

Published in Transactions on Machine Learning Research (05/2023)
The result follows by choosing the step size and noting that
/ ( (fi. - inf (t.((0)33LLL
1 R-1 R y=0
min 0SrER-1
[
Remark 2. If the learning rate is small, Inequality (4) shows that the gradient (. should not be well aligned with the gradient of the global loss VLL.((0 Intuitively, this because 0f o(h,r the best. model chosen from the training trajectory (0("),), that achieves the minimum global loss, and therefore the subsequent gradient update (0( must be badly correlated with (L. so that the global loss would increase the next iteration.
A.4 Assisted Learning with Differential Privacy (DP)
In recent years, data privacy has been formulated under different frameworks. general direction is database privacy, such as the differential privacy (Dwork & Nissim 2004), where private data are collected and managed by a trusted third party. Another general direction local data privacy, to obfuscate data that have to be shared with potential adversaries. Examples include the local differential privacy (Evfimievski et al. 2003 Kasiviswanathan et al. 2011) that randomizes the raw data and the interval privacy (Ding & Ding, 2020
that reports random intervals containing the raw data. We note that a privacy framework usually concerns the protection of raw data or their identities, which is conceptually different from a learning framework. The integration of data privacy and learning has been recently studied. For example, (Abadi et al., 2016) developed a differentially-privaai SGD for deep learning, where the main idea is to perturb the gradient each minibatch step.
Experimental Setup. We demonstrate that our AssistDeep be implemented in the differential privacy (DP) framework to enhance the information privacy of both the learner and provider. We replace the local SGD training of both the learner and provider with the differentially private SGD proposed in (Abadi et al., 2016). We consider the (c, O)DP with = 10-5 and 6 = 1,5, 10 at each SGD step. The noise added to the gradients follow the Gaussian distribution (0, 2 -2 log 8-1). We implement this differentially private AssistDeep with imbalanced and limited learner's data (7 = 0.3), and all other experimental hyperparametens and settings remain the same Section 4.2 Using the strong composition formula (Kasiviswanal han el. al., 2011), the privacy after T' SGD steps and batch fraction satisfies (E','')DD,, with é' 2 log(1/0), 8'= -.8. We calculated that the learner satisfies (15.5, 0.001)-DP, and the provider satisfies (5.2, 0.001)-DP (after rounding).
CIFAR-10, AlexNet, 7. 00.3
CIFAR-10, ResNet- =0.3
SVHN, AlexNet, 7 =03
SVHN, ResNet-18, y -0.3
80
80
100
100
Test Accuracy
60
That Accuracy
60
Text Accuracy
Text Accunccy
80
60
40
2. Amtor with coo]
AmsinDeap with om]
Doso wilh om]
40
AssintDepp
20
de Awenst Doep with 5=
20
AesisDDepp with c-5
Deept with em AuistDDap
AssistDepp
Doup -=10 Doep (without Notse)
AesinDeep with AuidDDep (wilhout Naine)
20
20
AwiitDDes with em20 AssistDerp (without Noise)
Ancet (wethost Noter)
0
3
0
1 2 3
9
2 3 5 Number of Rounds
2 4 6 8 10 Number of Rounds
Number of Rounds
Number Rounds
Figure 16: Comparison of original AssistDeep (without noise) and AssistDeep algorithms with different levels of DP noises (e = 1,,5, 10) under 7t 0.3.
Comparison Results. Fig. shows the results of training both AlexNet and ResNet-18 on CIFAR-10 and SVHN datasets, respectively. We that for more restricted privacy level (namely small e), more noise
22

---

Published in Transactions on Machine Learning Research (05/2023)
is added and the performance of AssistDeep degrades. However, when the is large enough, the performance of differentially private Dssp is very similar to that of the original AssistDeep. Therefore, the results demonstrate that our AesistDeep can achieve reasonable levels of differential privacy without significantly degrading the performance.
A.5 Effect of Checkpoints Transferred from Learner to Provider
Experimental Setup. To evaluate the importance of the checkpoints passed from the learner to the provider in our algorithm design, we modified the original Assist algorithm to let the provider train on its local data using weights initialized from the model generated in the learner's last local training iteration (instead of the checkpoint model that achieves the minimum global loss). We name this algorithm Revised-Assisi
Then under imbalanced and limited learner's data (ry 0.3) and different. numbers of learner's local iterations (T 2000, 6000, 12000), conducted experiments to train the AlexNet CIFAR-10 dataset for both Desi and Revised-Assist to compare their performance, And all other experimental hyperparameters and settings remain the same as Section 4.2
Comparison Results. For each experiment, we averaged the test accuracies of the last four rounds to approximate the model's fully trained performance, and put the results into Table [ From all the results with different T values (2000, 6000, 12000), can be seen that AssistDeep. Specifically, under T 2000, averaged test accuracies of Assist and Revised-Assist are 66.0% and 65.3% respectively; under T 6000, averaged test accuracies of AssistDeep and Revised- AssistDeep are 65.6% and 64.8% respectively; under T = 12000, averaged test. accuracies of Assist Deep and Revised-Assist Deep are 66.2% and 64.1% respectively. These results demonstrate that in the algorithm design of our AesistDeep, it necessary for the provider to initializo its weights from the chockpoints passed by the learner to do the local training.
performs worse than our
Table 1: Comparison of Assist and Revised-AsssitDeeep in training an AlexNet on CIFAR-10
Test Accuracy
A Algorithm Name AssistDeep
T 2000
T 6000
12000
66.0%
65.6%
66.2%
Revised-Assist
65.3%
64.8%
64.1%
A.6 Provider-SGD: Centralized Model Trained on the Provider's Data Only
Experimental Setup. To compare the performance of our AesistDeep to that of the same model trained
on the provider's 's data only (named Provider-SGD), we conduct experiments on ProviderSGGD by training
AlexNet on CIFAR-10 and SVHN datasets, respectively. Then we compare the performance of Provider-SGD with that of SGD, Learner-SGD and our AesistDeep under both imbalanced (7ı. = 0.5) and balanced (Yı. =) learner's data. All other experimental hyperparameters and settings remain the same Section
Table 2: Comparison of SGD, Learner-GDD, Provider-SGD, and Assist in training AlexNet CIFAR-10 and SVHN
Test Accuracy
Algorithm Name
CIFAR-10, 74 = 0.5
CIFAR-10 10, 71 = 1
SVHN, 74 = 0.5
SVHN, 7L =
SGD
66.6%
68.1%
86.8%
87.5%
Learner-SGD
58.4%
63.2%
82.4%
83.9%
Provider-SGD
64.0%
64.0%
85.6%
85.6%
AssistDeep 66.0%
68.1%
86.8%
87.3%
Comparison Results. For each experiment, we averaged the test accuracies of the last nine rounds approximate the model's fully trained performance, and put the results into Table Both the CIFAR-10 and
23

---

Published in Transactions on Machine Learning Research (05/2023)
SVHN results show that Provider-SGD performs worse than Desi and SGD, but better than Learner- SGD. Specifically, for CIFAR-10, Provider-SGD achieves about 64.0% test accuracy while our Assist achieves about 66.0% accuracy under imbalanced learner's data and about 68.1% accuracy under balanced learner's data; for SVHN, Provider-SGD achieves about 85.6% test accuracy while our AssistDeep achieves about 86.8% accuracy under imbalanced learner's data and about 87.3% accuracy under balanced learner's data. Thus from all the results, one can see that our AssistDeep performs better than Provider-SGD. This proves that under both imbalanced and balanced learner's data, our AssistDeep outperforms Provider-SGD since the former can leverage both the learner's data and provider's data to improve the model's generalization performance while the latter leverages only the provider's ' data.
A.7 Resampled-Learne--S50 Learne-SSGr Senner--GDDao with Balanced Learner's Data
To address the issue of the learner's data imbalance, we resample the data so that each batch will sample data points from each class with equal probability. Specifically, for the learner's local data, we copy the samples of each minor class to augment them and align them with the samples of the major class. In this case, the learner has the data balance. Then we implemented the Learner-SGD with such a resampling strategy and named it
Experimental Setup. Under imbalanced and limited learner's data (Yy. = 0.3), we compare the Resampled- Learner-SGD with the original Learner-SGD and our AssistDeep in training AlexNet and ResNet-18 on CIFAR-10 10 and SVHN datasets, respectively. And all other experimental hyperparameters and settings remain
the same as Section 4.2
CIFAR-10, AlexNet, %
0.3
CIFAR-10, ResNet-18, % m03
SVHN, A AlexNet, % =0.3
SVHN, ResNet- 18, % =0.3
80
80
100
Test Aceorayy
Test Accrocy
60
Test Accuracy
Tost Aceoracy
80
60
40
40
40
40
20
Leorner- SGD Baaoengli-LLmmmee-SCCt Dosi
20
Learner SGD Ranangiiei- AssistDepp
Letzuee SQD
Learner SGD
20
20
R--BRonnlaalb-aarrreere Saa--e----- AssinDDrp
0
2. 3 6 7 10
4 5 Number Rounds
8
9 10
1
3
4 5
10
2 4 6 8 9 10 Number of Rounds
Number of Rounds
Number Rounds
Figure 17: Comparison of original Learner-GDD, Resampled-Leamer-SGD] and AssistDeep under Y2 = 0.3.
Comparison Results. From all the results shown in Fig. L7 it can be seen that the performance of Rosampled-Learne-SGDD is very similar to that of Learner-SGD but worse than Assist which proves the advantage of our AssistDeep over the Resampled-Learee Lea regardless of models and training data.
A.8 Effect of Number of Local Optimization Iterations
To explore the elfect of the total number of local optimization iterations in the training process, we compare the performance of AssistDeep under the different total numbers of local iterations in each assistance round. Note that as we described in Section 4.2 we assign the iteration budget of the total number of local optimization iterations to the learner (T iterations) and provider (T' iterations) in proportion to their local data sizes. Also note that in our paper, the total number of iterations is fixed as 2000 by default, namely T + T" 2000.
Experimental Setup. Specifically, under imbalanced and limited learner's data (vz. = 0.3), we compare AssistDeep's performance under several different values of the total number of iterations, namely T + T" 500, 1000, 2000, 4000. We conduct those experiments in training AlexNet and ResNet-18 on CIFAR-10 and SVHN datasets, respectively. And all other experimental hyperparameters and settings remain the same as
Section 4.2
Comparison Results. From all the results shown in Fig. 18 one can see that under different values of the total number of local optimization iterations (i.e., T TT), AssistDeep achieves very similar performance, indicating that our Deep is not sensitive to this hyper-parameter.. Regardless of models and training
24

---

Published in Transactions on Machine Learning Research (05/2023)
CIFAR-10 R10, AletNet, % =0.3
C-FAR-10, ResNet-18, % =0.3
SVHN, AeexNet, % 7.33
SVHN, ResNet-18, 7 -0.3
80
80
100
100
80
80
Test Accuraty
Test Accurary
60
Ted. Asturacy
Test Accuracy
60
40
40
Aneidd Dwep (500)
Dhery (500)
Aesistleepp (500)
40
AssistDesp (020)
20
G-AscatDwe (decanlt: 2000)
A. AeiatDop [1000)
20
3 Decet (defanit: 2000)
AsststDepp (1000)
20
AssiniDepp (1300)
AssinDDep (deloult. 2000)
20
Aesiet
AssistDeap (difaull: 2000
DoesDDeo (000)
Aois (4020)
ArrietDepp (9000)
Assistiepp (0000)
10
2
2
Number of Rounda
Number Rounds
Number Rounds
Number of Rounds
Figure 18: Comparison of AssistDeep algorithms with different numbers of local optimization iterations (500, 1000, 2000, 4000) under 7% = 0.3.
data, the performance of Assist almost keeps unchanged under the values of the total number of local iterations greater or equal to 2000, which is the reason that we set this hyper-parameter as 2000 in our paper.
A.9 Explore the Transferability and Efficiency Aesist When the Learner and Provider Hold Data from Different Categories or Domains
Most of our experiments assumed that learner and provider P hold data from the same or very similar category and domain. In the real world, however, sometimes it is hard for the learner and provider to procure data from the same category and domain. Hence, it is necessary to conduct some experiments to prove the transferability and efficiency of our proposed Assisted Learning when the learner and provider have data from different calegories or domains. To implement this, we conduct two groups of experiments: 1. learner's and provider's is data come from different categories (but with the same domains), which means the major class data of learner's 's data does not appear in the provider's data; 2. learner's and provider's data come from different domains (but with the same categories), which means learner's and provider's data come from two different datasets. The details are presented as follows.
SVHN, AlexNet, =0.1
CIFAR-10, 10, 18 7 =0.1
CIFAR-10 CONAR-OOLLIIICII Alecke 11
CIFAR-U 10 (L), CDNCC ID (P), Beske IB
100
60
Test Accuracy
80
Text Accurscy
Test Acouracy
Tes Accurscy
80
60
60
40
40
40
20
Pe-SCI ((aneeline)
H-Jeemer-SGD (Baeeline)
20
8-500 (Baealine)
Lemnne-SCD (Baseinn))
20
-- SUD (Boseline) Lamm--GD (Baseline)) AecestDrepp
20
SOD (Bsselnne)
SGD (Baatinn))
S-AnintDeep
DointDepp
AeeatDoop
10
2 3 Number of Rounds
5
Number of Rounds
Number of Roands
Numbet of Rounds
Figure 19: Comparison of Assist SGD, and Learner-SGD under imbalanced learner's data in training an AlexNet on SVHN (left) and ResNet-18 on CIFAR-10 (right) when learner and provider P hold data from different categorics.
Figure 20: Comparison of AssistDeep, SGD, and Learner-GGD under imbalanced learner's data in training an AlexNet and a ResNet-18 respectively when learner L holds data from CIFAR-10 and provider P holds data from CINIC-10.
A.9.1 Group 1: Learner and Provider Hold Data with Different Categories
Experimental Setup. From SVHN and CIFAR-10 datasets respectively, suppose that we use the original data sampling method for both the learner and provider to get the data, but let the provider remove the data from the learner's major class. And all other settings and ssper-parameters keep the same as those described in Section 4.2 Then by training an AlexNet on SVHN and a ResNet--18 on CIFAR-10 respectively, we conduct experiments to compare the performance of our AesistDeep with that of SGD and Learner-SGD under limited imbalanced learner's data. (7 = 0.1).
25

---

Published in Transactions on Machine Learning Research (05/2023)
Comparison Results. From the results shown in Fig. 19 it can be seen that the performance of AssistDeep is much better than Learner-GGD and a little worse than SGD, which gives the same conclusion as that in Section 4.2 The results prove that our proposed Assisted Learning algorithm (AssistDeep) can still work efficiently even if the learner's and provider's data come from different categories, which further proves the excellent transferability of our proposed algorithm for data with different categories.
A.9.2 Group 2: Learner and Provider Hold Data from Different Domains
Experimental Sctup. With the original data sampling method, suppose that the learner samples data from the CIFAR-10 dalaset and the provider samples data from the CINIC-10 dataset (Darlow et al. 2018). while all other settings and hyper- parameersssssrss keep the same as those described in Section 4.2 Note that in this case, we evaluate and get the test performance on the learner's test data only. Then by training an AlexNet and a ResNet-18 18 respectively, we conduct experiments to compare the performance of our AssistDeep with that of SGD and LearnerSGGD under limited imbalanced learner's data (7z 0.1).
Comparison Results. From the results shown in Fig. 20 it can be seen that the performance of DesistDeep is much better than Learner-SGD and a little worse than SGD, which gives the same conclusion as that in Section 4.2 The results prove that our proposed Assisted Learning algorithm (AssistDeeep) can still work efficiently even if the learner's and provider's data come from different domains, which further proves the excellent transferability of our proposed algorithm for data with different domains.
26


Reviews:
[{"review_id": "YfJjxK7kXH", "reviewer": "Reviewer_emPx", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n\n1. The motivation is clear and the research question is well-defined.\n2. The paper is well written and easy to follow.\n\nWeakness:\n\n1. My biggest concern is that the data used in this paper is in the ‘horizontal-splitting’ setting, which means the data in learner’s hand and provider’s hand is strictly i.i.d. However, I think it is very hard that the data is in the same domain and shares the same category. To prove the transferability of the proposed Assisted Learning, it is necessary to conduct experiments when the data in learner’s hand and provider’s hand is not from the same domain or has different categories.\n2. Assisted Learning is very close to Federate Learning. Though the authors claim that Assisted Learning is within a few rounds. I would like to see Assisted Learning as a special case of Federate Learning. To show the necessity of Assisted Learning, ablation experiments of Federate Learning are needed. Specifically, how is the performance when Federate Learning methods are also conducted within a few rounds?\n3. The experiments are not sufficient. The proposed method is very simple but experiments are only conducted on toy datasets(CIFA10) and outdated neural networks(AlexNet, ResNet18). More experiments on larger datasets(ImageNet) and modern neural networks(Vision Transformer) are encouraged.\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The article discusses the scenario where the provider holds much data and the learner has limited imbalanced data. The author propose ‘Assisted Learning Protocols’ that the learner can purchase assistance services (without data sharing) from the provider to enhance their model performance within a few rounds.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "1. The experimental setting, i.e., W1 in the above review.\n2. Difference between assisted learning and federated learning should be compared.\n3. More detailed and complex experiments (W3).", "broader_impact_concerns": "No", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "jiVVRcxXGm", "reviewer": "Reviewer_h9EE", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "### Pros:\n\nThis paper explores a new field of leveraging assisted learning for the scenario where multiple organizations hold their own data (including different subjects) but share the same feature space.\n\nThis setup is very similar to that of horizontal federated learning. While as mentioned in [1], which proposed (vertical) assisted learning, the difference between assisted learning and federated learning is that\n- \"Conceptually, **the objective of Federated Learning** is to exploit resources of massive edge devices **for achieving a global objective**. At the same time, **the general goal of Assisted Learning** is to facilitate multiple participants (possibly with rich resources) to autonomously **assist each other’s private learning tasks**.\" \n\n[1] Xian, Xun, et al. \"Assisted learning: A framework for multi-organization learning.\" Advances in neural information processing systems 33 (2020): 14580-14591.\n\n### Cons:\n\nHowever, I do have some concerns over the claims/assumptions/executions of this paper.\n\nFirst and foremost, given that the crucial fact distinguishing assisted learning from federated learning lies in assisting each other's private learning tasks,  I am particularly concerned about the specific formulation of this paper that **sets the data hold by a single organization as imbalanced (be supposed to follow a private data distribution) but enforces the model of this organization to learn a global distribution via assisted learning**. This setup is in contradiction with the objective of assisting each organization's private learning tasks but actually follows the spirit of federated learning to achieve a global objective (each organization learns the global data distribution).\n\nSecond, this paper claims no data sharing across organizations (e.g., Section 1.1), but the proposed algorithm (e.g., Algorithm 1) requires the communication of a series of model parameters and losses, which can implicitly violate the no-data-sharing assumption. Thus it is indispensable to include a data-privacy guarantee here.\n\nLast but not the least, this paper assumes that \"In each assistance round, both the learner and the provider have access to sufficient computation and communication resources\". In my view, the assumption of unrestricted commucations is unrealistic in practice because real-world applications probably have limited commucation bandwidths, which have motivated extensive studies in distributed learning. Therefore I expect this paper can quantify the necessary communications needed for horizontal assisted learning.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "No", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper attempts to establish the framework of assisted learning in a context similar to horizontal federated learning, in which the feature space is shared but each participant has its own data.", "claims_and_evidence_yn": "No", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "I hope the authors can address all the critical concerns mentioned above.", "broader_impact_concerns": "N/A", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "hNCbDIfiVO", "reviewer": "Reviewer_gXtW", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n- The proposed method is simple and straightforward, making it easy to apply in practice.\n- The writing is clear and well structured.\n\nWeaknesses:\n- The algorithm conflicts with the goal of preserving data privacy.  In particular, communicating multiple checkpoints to the provider and then back to the client can leak information about the local training data. In prior work on assisted learning, there is usually also the assumption that the model and training object stay private as well between client and provider (http://assisted-learning.org/) hence why Xian et al., 2020 communicated residuals instead of model weights.\n- There is no theoretical analysis to accompany the proposed algorithm to either guarantee convergence or give bounds on convergence.\n- The evaluation is insufficient.  There are a few questions that need to be answered:\n  1. How important it is for the provider to train on it's local data using weights initialized with the checkpoints passed from the client?  How does the performance of AssistDeep compare to using a model just trained on the provider's data (e.g. ProviderSGD)?\n  2. There also does not seem to be any adjustments to the client's training procedure to account for data imbalance (e.g. weighted loss). How do results look if this is incorporated for the LearnerSGD baseline? \n  3. How do results depend on the relative size of the client and provider's data?\n- AssistDeep requires specifying T and T' for the training duration of the client and provider models as well as standard hyperparameters for the model.  It is unclear how to tune hyperparameters in this setting.\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "No", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper studies the assisted learning problem with horizontal-splitting of data and proposes an algorithm called AssistDeep for the setup studied.  Assisted learning addresses the problem of a client with limited data trying to learn a high performing model with the assistance of a provider with access to more data subject to two key constraints: (1). both client and provider cannot share data and (2) learning must be completed within a small number of iterations due to resource constraints.  In the horizontal-splitting setting, the client and provider have data with the same features but the provider has a much more representative dataset than the client.  AssistDeep proceeds in three stages for each round of assisted learning:\n1. The client trains a model on its local data and sends a subset of the model checkpoints and evaluation metrics observed during the learning process to the provider.\n2. The provider evaluates the subset of checkpoints received on a combination of the client's data and its own data then selects the best one for further training.  Once training is complete, it sends back a subset of the checkpoints and evaluation metrics on provider's local data to the client.\n3. The client selects the checkpoint from the provider that performs the best on the joint data.\n\nExperiments on CIFAR10 and SVHN for the AssistDeep and CartPole and LunarLander for AssistPG show the proposed algorithms can achieve test performance comparable to an oracle trained on both client and provider data under different values of imbalance for the client data.", "claims_and_evidence_yn": "No", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "- Critical: Please investigate how the algorithm can be modified to use residuals so that privacy can be better preserved.\n- Critical: Please provide theoretical analysis even in restricted settings for convergence guarantees.\n- Critical: Please run additional experiments to answer the 3 questions above.\n- Critical: Please discuss how sensitive DeepAssist is to T and T' and how you would go about tuning hyperparameters for the model on both client and provider.  ", "broader_impact_concerns": "None.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: The authors have done a good job in revising their paper and addressing the review comments. Most of the concerns raised by the reviewers have been addressed in some way. However, it seems that people still have problems with the relationship (or distinction) between assisted learning and federated learning. I hope the authors can make a systematic discussion on this topic in their future revision. With a better positioning, this work will demonstrate clearer value to the community.
Decision: Accept (poster)