Federated Joint Learning of Robot Networks in Stroke Rehabilitation
Xinyu Jiang1 , Yibei Guo1 , Mengsha Hu1 , Ruoming Jin2 , Hai Phan3 , Jay Alberts4 , Rui Liu1

Abstract— Advanced by rich perception and precise execution, robots possess immense potential to provide professional and customized rehabilitation exercises for patients withmobility impairments caused by strokes. Autonomous robotic
rehabilitation significantly reduces human workloads in the longand tedious rehabilitation process. However, training a rehabilitation robot is challenging due to the data scarcity issue. Thischallenge arises from privacy concerns (e.g., the risk of leaking
private disease and identity information of patients) during
clinical data access and usage. Data from various patients andhospitals cannot be shared for adequate robot training, further
compromising rehabilitation safety and limiting implementation
scopes. To address this challenge, this work developed anovel federated joint learning (FJL) method to jointly train
robots across hospitals. FJL also adopted a long short-term
memory network (LSTM)-Transformer learning mechanism toeffectively explore the complex tempo-spatial relations among
patient mobility conditions and robotic rehabilitation motions.
To validate FJL’s effectiveness in training a robot network,a clinic-simulation combined experiment was designed. Realrehabilitation exercise data from 200 patients with stroke
diseases (upper limb hemiplegia, Parkinson’s syndrome, andback pain syndrome) were adopted. Inversely driven by clinicaldata, 300,000 robotic rehabilitation guidances were simulated.
FJL proved to be effective in joint rehabilitation learning,
performing 20% - 30% better than baseline methods.

I. INTRODUCTION
Stroke is a global healthcare problem contributing toindividual disability and death [1]; rehabilitation typicallyaims to train patients in compensatory strategies with proximal (e.g., shoulder abduction, arm flexion) and distal (e.g.,hand open, finger extension) movements to facilitate patient
recovery on strength, speed, endurance, and precision ofmultijoint movements [1], [2]. While training a human expert
for professional rehabilitation is expensive and lengthy, forexample, an attending physician-level expert will need anaverage professional education and training time of 8-11
years and 0.2-0.5 million dollars [3].
Powered by sensor and control technologies, robots canprecisely and durably provide patient training exercises toensure quality rehabilitation exercise, significantly reducing
human workload and economic/time costs; most importantly,
powered by the latest learning algorithms, a robot withexpert-level skills can be trained within several days [4].
1 Xinyu Jiang, Yibei Guo, Menghua Hu, and Rui Liu are with Collegeof Aeronautics and Engineering, Kent State University, Kent, Ohio 44242,USA.
2 Ruoming Jin is with the Department of Computer Science, Kent State
University, Kent, Ohio 44242, USA.
3 Hai Phan is with the New Jersey Institute of Technology, Departmentof Data Science, Newark, New Jersey 07102, USA.
4 Jay Alberts is with the Cleveland Clinic, Concussion Center, 9500
Euclid Ave, Cleveland, OH 44195, USA.

Fig. 1: Federated joint learning framework overview.
Therefore, it is promising to rely on robots for durable, reliable, and economical rehabilitation for movement disorderstroke diseases, such as hemorrhagic stroke and hemiplegicstroke [5].
However, training a professional and safe rehabilitation
robot is challenging due to clinical data scarcity. Besides
treatment-relevant information (e.g., stroke types and motor
impairments), clinical rehabilitation data also includes irrelevant but private information (e.g., patients’ identity, physiological characteristics, and other illnesses) [6]. Restrained byconcerns of leaking patient information, clinical data cannot
be accessed across hospitals; small-amount local data inadequately train a rehabilitation robot, further undermining its
performance and safety and impeding widespread implementations of robotic rehabilitation [7]. Besides, patients varyin physical characteristics and motor impairments, adding
challenges for robots to provide customized rehabilitation[8].
Therefore, to address the data scarcity issue, in thisresearch, a novel joint training method – Federated Joint
Learning (FJL) was developed to collaboratively train robots
crossing hospitals. Particularly, our work in this paper mainly
has three contributions:• A federated joint learning network was developed tonetwork robots crossing hospitals and enable them tomutually learn rehabilitation skills from each otherwithout directly accessing original patient data.
• A LSTM-Transformer learning framework was developed to efficiently extract representative motion plansfrom complex spatiotemporal motions of patient joints
with differences in body characteristics and motor impairment degree.
• A novel relational loss was designed to refine the robot
pose estimation result and improve the accuracy of thepose estimation model.
II. O UR A PPROACH : FJL N ETWORKThe FJL framework shown in Figure 1, consists of threemodules - 1) AFJL enables joint learning for robot rehabil-itation while concealing sensitive patient data during sharedlearning. 2) The Robot Joint Pose Estimation module accurately infers robot poses from patient joint inputs, using theLSTM-Transformer. 3) A relational-based refinement module
optimizes the robot pose estimation network parameters.
Robot Federated Joint Learning. The objective task is todefine an objective training function Fi that maps the modelparameters set θi ∈ Rd to a training loss respect to private
i-th robot data Ri .
mm
XXmin G (θ) :=Fi (θi ) + λA (∥θi − θj ∥)(1)θ i=1i<j2where A ∥θi − θj ∥ is an attention including function thatdescribed in [9] to measure the difference between θi and θjin a non-linear manner, and λ is a normalized parameter.
At last, the gradient that gets from clients is aggregatedand updated by using the following formula:nθupdate = θ − η ·1X
δCn i=1 i(2)Robot Joint Pose Estimation. The proposed LSTMTransformer-based robot joint pose estimation module consists of three parts. The model computation proceduref (Xt−P +1:t ) can be described as:
Xconcat = concat (Xt−p+1:t )Xlstm , hlstm = LSTM Xconcat , hi−1lstm(3)Ŷt = TransformerBlock (Xlstm )

(4)
(5)

At last, an MSE loss function is applied to train the robot
joint estimation model:NN21 X1 X
2(f (Xt−P +1:t ) − Yt ) =Ŷt − Yt
N i=1
N i=1
(6)
where Yt is the t-th robot joint ground truth and N is thetotalN -timestamp. X is the robot Joint settings as the input
including the joint’s force, positions, velocity in space, andintrinsic positions.
Relational Loss. As defined in [10], to refine the pose
estimation accuracy and improve metrics results throughminimizing the correlation between MSE loss and PCK
metrics, the only way is to improve the negative correlation
between loss function and PCK metrics:

LM SE =Os (L (y, ŷ; θ) , M (y, ŷ)) = ρS (L (y, ŷ; θ) , M (y, ŷ)) (7)where θ is the optimized parameters in the network.
III. E XPERIMENT S ETTINGS AND R ESULT A NALYSIS
Clinical Data Driven Simulation. The experiment utilized the KiMoRe dataset [11], encompassing rehabilitation
exercises for 200 stroke patients across three types. Using
Coppeliasim [12], a simulation scenario mimicking robotic
stroke rehabilitation was created. Human motions from thedataset were used to guide simulated robotic arm movements,

Fig. 2: Settings for robot-guided rehabilitation.
TABLE I: The comparison of PCK value of each model withrelational loss and MSE loss
VIO Algorithms

MSE Loss

Relational Loss (Ours)

LSTM

0.674

0.751

Transformer

0.565

0.630

LSTM-Encoder-Decoder

0.541

0.743

LSTM-Transformer

0.706

0.754

generating robot data used for training the FJL model.
This trained model’s accuracy was validated to ensure its
effectiveness in stroke rehabilitation (Figure 2).
Result Analysis. The final results are presented in Table
I. This table highlights that the proposed LSTM-Transformer
with relational loss exhibits the best performance, signifyingthat the LSTM-Transformer is adept at facilitating accuratepatient rehabilitation. From the results presented in Table
I, the superior accuracy and impressive generalization capabilities of the LSTM-Transformer network proved good
performance in robot pose estimation. The experiment result
indicates the performance of the LSTM-Transformer is 20%30% higher than the baseline.
IV. C ONCLUSION AND F UTURE W ORKThis study tackled challenges in federated joint learning
for rehabilitating robots by introducing FJT, a tailored architecture enabling robot training without direct patient dataaccess. FJT safeguards patient data, enhances robot training, and improves spatial-temporal rehabilitation learningfor upper limb hemiplegia patients. Future work will focus
on developing collaborative mechanisms for synchronizedrehabilitation across patients’ upper and lower limbs.
R EFERENCES[1] B. H. Dobkin, “Strategies for stroke rehabilitation,” The Lancet Neurology, vol. 3, no. 9, pp. 528–536, 2004.
[2] P. Langhorne, J. Bernhardt, and G. Kwakkel, “Stroke rehabilitation,”
The Lancet, vol. 377, no. 9778, pp. 1693–1702, 2011.
[3] G. Carpino, A. Pezzola, M. Urbano, E. Guglielmelli et al., “Assessingeffectiveness and costs in robot-mediated lower limbs rehabilitation: ameta-analysis and state of the art,” Journal of healthcare engineering,
vol. 2018, 2018.
[4] A. I. Károly, P. Galambos, J. Kuti, and I. J. Rudas, “Deep learning
in robotics: Survey on model structures and training strategies,” IEEE
Transactions on Systems, Man, and Cybernetics: Systems, vol. 51,
no. 1, pp. 266–279, 2020.

[5] W. H. Chang and Y.-H. Kim, “Robot-assisted therapy in stroke
rehabilitation,” Journal of stroke, vol. 15, no. 3, p. 174, 2013.
[6] C. Véliz, “Not the doctor’s business: Privacy, personal responsibility
and data rights in medical settings,” Bioethics, vol. 34, no. 7, pp. 712–718, 2020.
[7] A. Al Kuwaiti, K. Nazer, A. Al-Reedy, S. Al-Shehri, A. Al-Muhanna,
A. V. Subbarayalu, D. Al Muhanna, and F. A. Al-Muhanna, “Areview of the role of artificial intelligence in healthcare,” Journal ofPersonalized Medicine, vol. 13, no. 6, p. 951, 2023.
[8] S. Hussain, P. K. Jamwal, P. Van Vliet, and M. H. Ghayesh, “Stateof-the-art robotic devices for wrist rehabilitation: Design and controlaspects,” IEEE Transactions on human-machine systems, vol. 50, no. 5,
pp. 361–372, 2020.
[9] Y. Huang, L. Chu, Z. Zhou, L. Wang, J. Liu, J. Pei, and Y. Zhang,“Personalized cross-silo federated learning on non-iid data,” in Proceedings of the AAAI conference on artificial intelligence, vol. 35,
no. 9, 2021, pp. 7865–7873.
[10] T. Huang, Z. Li, H. Lu, Y. Shan, S. Yang, Y. Feng, F. Wang,
S. You, and C. Xu, “Relational surrogate loss learning,” arXiv preprint
arXiv:2202.13197, 2022.
[11] M. Capecci, M. G. Ceravolo, F. Ferracuti, S. Iarlori, A. Monteriu,
L. Romeo, and F. Verdini, “The kimore dataset: Kinematic assessmentof movement and clinical scores for remote monitoring of physical rehabilitation,” IEEE Transactions on Neural Systems and Rehabilitation
Engineering, vol. 27, no. 7, pp. 1436–1448, 2019.
[12] E. Rohmer, S. P. Singh, and M. Freese, “V-rep: A versatile andscalable robot simulation framework,” in 2013 IEEE/RSJ international
conference on intelligent robots and systems. IEEE, 2013, pp. 1321–1326.