Continual Learning from Simulated Interactions via Multitask Prospective Rehearsal for Bionic Limb Behavior Modeling
Authors: Sharmita Dey
Benjamin Paassen
Sarath Ravindran Nair
Sabri Boughorbel
Arndt F. Schilling
Venue: Accepted by TMLR ()

Abstract: Lower limb amputations and neuromuscular impairments severely restrict mobility, necessitating advancements beyond conventional prosthetics. While motorized bionic limbs show promise, their effectiveness depends on replicating the dynamic coordination of human movement across diverse environments. In this paper, we introduce a model for human behavior in the context of bionic prosthesis control. Our approach leverages human locomotion demonstrations to learn the synergistic coupling of the lower limbs, enabling the prediction of the kinematic behavior of a missing limb during tasks such as walking, climbing inclines, and stairs. We propose a multitasking, continually adaptive model that anticipates and refines movements over time. At the core of our method is a technique which we call the multitask prospective rehearsal, that anticipates and synthesizes future movements based on the previous prediction and employs a corrective mechanism for subsequent predictions. Our evolving architecture merges lightweight, task-specific modules on a shared backbone, ensuring both specificity and scalability. We validate our model through experiments on real-world human gait datasets, including transtibial amputees, across a wide range of locomotion tasks. Results demonstrate that our approach consistently outperforms baseline models, particularly in scenarios with distributional shifts, adversarial perturbations, and noise.

Published in Transactions on Machine Learning Research (01/2025)
Continual Learning from Simulated Interactions via Multitask Prospective Rehearsal for Bionic Limb Behavior Modeling
Sharmita Dey
sharmita Searmita.dey@heeee dey@hest. ethe.
ETH Zurich University of Goetlingen
shormita dey@medd. uni anii-gotttingnn.. de
Benjamin Paassen Bielefeld University
Sarath Ravindran Nair European Neuroscience Institute, Goettingen (ENI-G)
Sabri Boughorbel
shoughorbcll@hbbku.uu
Qalar Compuling Institute
Arndt F. Schilling
University Medical Center, Goettingen (UMG)
Reviewed on OpenReview: https: openreview. net/ porm? id= Bmy82p2eez
Abstract
Lower limb amputations and neuromuscular impairments severely restrict mobility, necessi- tating advancements beyond conventional prosthetics. While motorized bionic limbs show promise, their effectiveness depends replicating the dynamic coordination of human movement across diverse environments. In this paper, we introduce model for human be- havior in the context of bionic prosthesis control. Our approach leverages human locomotion demonstrations to learn the synergistic coupling the lower limbs, enabling the prediction of the kinematic behavior
missing limb during tasks such as walking, climbing inclines, and stairs. We propose multitasking, continually adaptive model that anticipates and refines movements over time. At the of our method is technique called mulbitask prospective rehearsal, that anticipates and synthesizes future movements based the previous prediction and employs a corrective mechanism for subsequent predictions. Our evolving architecture merges lightweight, task-specific modules on shared backbone, ensuring both specificity and scalability. We validate our model through experiments on real-world human gait. datasots, including transtibial ampulees, across a wide range of locomotion tasks. Results demoust rate that our approach consistently outperforms baseline models, particularly in scenarios with distributional shifts, adversarial perturbations, and noise.
Keywords: Human Behavior Modeling, Bionics, Multitask Rehearsal, World Model, Human- machine Interaction
1
Introduction
Lower limb amputation and neuromuscular disorders detrimentally affect natural locomotion, compromising individuals' quality of life (Windrich et al.) 2016). Those alllicted often depend on assistive technologies like prosthetics or orthoses to regain daily mobility (Windrich et al. 2016). However, conventional passive prosthetics often fall short in replicating natural gait during diverse activitics, from basic transitions like standing from a seated position more dynamic actions like running or navigating slopes (Windrich al. 2016). The advent of powered prosthetics, equipped with integrated motors, promises more naturalistic

---

Published in Transactions on Machine Learning Research (01/2025)
gait. Nonetheless, this advancement requires model capable of effectively approximating the complexities of human gait synergy to accurately estimate the necessary motor commands. While finite-state machines are adequate for rudimentary scenarios (Lawson 2014 Chen et al. 2015), their construction becomes infeasible for finer gait phase resolution and multiple locomotion tasks (Lawson ct al. 2013 An emerging solution lies in training models on human demonstrations to intuitively infer target limb motion (Dey el al. 2019 2020a
2021b). This methodology not only replicates gait's inherent fluidity but also facilitates scamless transitions across varying gaits, eliminating the need for rigid rules or heuristics. Furthermore, such learning-basse based gait models can be robustified by training a large and diverse set of human demonstrations Dey et al. (2020b).
To achieve personalization, these models could incorporate references that exhibit anthropomorphic features
similar to those of the physically impaired subject Dey & Nair (0244..
Pretraining
Multitask Prospective Rehearsal
Refining
Muti Galt Predictor,
Multi Gait Prodictor.
2,
Goid Predkaor, f
Prospective Model, gg
R112
Prospective Model g
Nue
Aast
125
Figure 1: Overview of the multitask prospective rehearsal-based training: In the pretraining phase, the multi gait predictor model, f, is trained to forecast the desired joint profile y; from the history of sensor states [X,,...X,), while the prospective model g aims to predict the subsequent sensor state X,+1 from X, and the target joint profiles y/- For prospective rehearsal, the multi gait model predictions fk are fed along with the current sensor state X to the prospective model to project the next sensor state X Multi gait predictor, f is then refined with this projected state, X = X..t| -
against the actual future output
Vk++1 E Deal, to compensate the effect of its prediction errors. Recent studies attest to the efficacy of learning- based models in gait and prosthetic behavior estimation (Dhir et al. 2018 Zabre-Gonzalez et al. 2021 Fang et al.) 2020 Dey & Schilling, 2022a). However, most of these works have locused on either a single locomotion task or jointly training multiple modes in the context of multi-locomotion scenarios. Furthermore, prior research has approached multitask learning from human demonstration as a standard supervised learning task, which assumes independence across time. This oversight can cascade predictive errors, deviating significantly from the original training distribution (Kumar et al. 2022 Ross & Bagnell, 2010). Some approaches (Ross et al.) 2011 Judah et al., 2014 Kelly et al. 2019) have addressed the issuc of state distribution shift by using trained model to collect additional data and label them as the model encounters new states, refining the model ileratively. However, these studies require having access to a simulator, environment, or deployment of models in the real world for data collection to estimate the expert action for the induced state. This may not be feasible or highly costly for multiple safety-critical scenarios, e.g., human-centered robotics that might entail risks of imbalance and fall. Moreover, these approaches have largely focused on single-task settings rather than continual multitask adaptation. The multifaceted dynamics of human gait, which can shift based on varying terrains, speeds, fatigue levels, or specific locomotion tasks, underscore the necessity for model that fluidly adapts to these evolving conditions.
To achieve this objective, we present a multitask, continually adaptive gait synergy approximation model tailored for various locomotion tasks, capable of adapting to evolving gait patterns and refining itself by integrating the impact of prediction errors. Central to our method is what we call the multitask prospective rehearsal (Figure [] that prepares the model to anticipate and handle potential trajectories that may arise from its predictions, creating a scamless connection between continual adaptation and the integration of prediction errors for model refinement. Unlike conventional models that often rely on retrospective analysis of movement, our method is designed to prospectively imagine and synthesize potential future locomotion patterns. By synthesizing future states, the model is, in essence, creating its own new 'unscen' data to practice on, which could lead to better generalization when encountering actual new data. The model refines its parameters incrementally as it updates itself with this enhanced data, negating the need for a simulator or new training data.
We conduct a wide range of experiments in continual and joint learning settings with different model architectures and backbones on three world gait datasets, including our own patient. (transtibial amputee)
2

---

Published in Transactions on Machine Learning Research (01/2025)
dataset. We establish that our model outperforms many bascline models with multiple benchmarks. Our study is the first to approach multi-gait adaptation in bionic prostheses as an error-aware multitask continual adaptation problem.
2 Related Work
Prosthetic behavior models. Conventional prosthetic behavior models primarily rely on state machines
that deterministically generate control commands based on finite states within a gait cycle. These models
segment the continuous gait cycle into discrete states (Lawson 2014 Chen et al. 2015 Culver et al. 2023).
which fails to accurately replicate the smooth and continuous movement patterns typical of human walking.
To better model the fluidity of the gait cycle, learning-based regression methods (Dey et al. 2020b 2021a
Dhir et a.. 2018 Fang et al. 2020 Yabre-Gouzalez et al.] 2021 Dey & Schilling, 2022b) have been utilized that directly predict the joint motion commands from the sensor states of residual body movement. However, these models generally adopt straightforward reactive strategy, learning the correlation between the residual states of the body and the limb's movements. Hence, these models often oversimplify the complex nature of human motor abilities, which are inherently adaptable to various tasks, diverse environments, and evolving gait patterns.
Multitask adaptive learning. A principal challenge in multitask adaptive learning is the phenomenon of cat catastrophic forgetting (McCloskey & Cohen 1989 Robins 1995 French 1999), where newly acquired knowledge can cause the loss of previously learned information. To combat this, continual learning strategies,
such regularization-beseee based methods, architectural strategies, and experience replay (Erkpaatra et al.
2017: Zenke et al.) 2017 Smith et al., (023) have been developed. Regularization- based methods like Elastic Weight ht Consolidation (EWC) (Kirkpatrick et al. 2017) and Synaptic Intelligence (ST) (Zenke et al.) 2017
introduce regularization terms to guide the model towards a parameter space optimized for low error across both previous and new tasks. Architectural methods such as Progressive Neural Networks (PNNs) Ruusu
et al. 2016) expand the model's structure by introducing task-specific parameters to sustain performance
across tasks. Rehearsal replay methods, including Experience Replay (ER) (Robins 1995 Luo & Li 2020;
et al. 2023) and Gradient Episodic Memory (GEM) (Lopez-Paz & Ranzato 2017), maintain sample of previous lasks to be revisited during new task training, reinforcing the model's 's exposure to the old dala distribution. Rehearsal approaches have empirically proved to be the most effective among several approaches developed (Merlin et al. 2022). However, these techniques have a retrospective nature and are meant for treating data samples independently. ly. In the context of bionic prosthetics, where predictive models interact. with a dynamic environment, anticipating the impact of predictions on subsequent data inputs is critical.
Model-based reinforcement learning (RL). Model-based RL focuses developing a representation of the environment by gathering data from interactions driven by specific policy. This constructed model of the environment facilitates level of planning, enabling the estimation of potential outcomes for future Racanière et al.
states (Mnih et al.) 2013 Silver et al. 2016
2017 Nagabandi et al. 2018 Chua et al. 2018). Typically, these methods rely on a reward signal and online interactions with the environment. With the
advent of offline reinforcement (Zhou et al.
2020 and model-based imitation learning (TI.) Englert et al. 2013 Kidambi et al.) (2021). access the reward signals or online interactions can be bypassed Hu et al.) 2022). Both mosel-based RL and II, by internalizing environmental dynamics, enable an agent to forecast outcomes and make informed decisions. Our approach is inspired by model-based RL; however, we use it as a rehearsal strategy within a multitask continual learning setting to mitigate forgetting, adapt to evolving tasks, and limit compounding errors, all without needing access to a reward signal or online interactions.
2021 Yu et al.
3 Method
Let = X..I..... E Rd be a time scries of sensor states (3D body and joint angles, angular velocities, and linear accelerations) during locomotion task L (such as level ground walking, ascending ramp, or climbing stairs) and let y: = yr....... Vt,N, E R be the corresponding desired kinematic profiles (knee or ankle joint profiles) as demonstrated by a human. Our goal is a model f which robustly predicts the correct kinematic profiles y from input features x, even under prediction errors in previous time steps, and adapts to new tasks while preserving performance on older tasks. To realize such a model, we integrate principles
3

---

Published in Transactions on Machine Learning Research (01/2025)
from multitask adaptive learning with prospective rehearsal approach inspired by model-based RL. Our model builds upon three core features, which we describe in turn: 1) multitask learning via synthesis of shared learning mechanisms with task-specific adaptability, 2) continual adaptation with task rehearsal in the facet of an evolving architecture to avoid catastrophic forgetting, and 3) a prospective model, that imagines and informs about potential deviations to provide robustiesss against prediction errors.
Task
Task 2
Task 3
Xproin 1s
X
Xpain
G
Cg
81
81
82
8a
Training
Xtram
10 rrats
9 grraks
geekeursan
Gorot 12
5
1/
y2
yt
y
Oweer
Mation veriables
Imag irpute
Imaginec iiputs
Prospective
rehearsal
mation wemeerr
f
83
82
91
81
Figure 2: Summary of the overall approach. For each new task i, the data is split into a train (Xtrain "errumnstraar sfrain) and validation (Xpal, yfaat) set. A task-specific layer fe is added to the shared backbone fa and trained to predict the target joint profile yf frain Concurrently, a prospective model for the task anticipates potential inputs based on the current joint profiles. The rehcarsal data Xrehearsal for all encountered tasks is generatod using the prospective model imagined inputs. As new tasks are introduced, both the shared backbone ʃs and previously established task-specific layers are updated with this rehearsal data to facilitate continual learning.
Multitask learning via generic and task-specific modules. A core challenge in multilask learning is lo exploit commonalities between the tasks while maintaining flexibility to account for the particularities of every single task. Our approach is to compose the gait model f of two functions: a shared backbone f across tasks, followed by task-specific layers fc. The overall prediction for an input sample (x.,IT...Xx,,) for task at time is (x., = (x
This duality enables the model to distill common features across all tasks and concurrently adapt to the unique aspects of individual tasks. We employ temporal convolutions (Oord et al. 2016 Fang et al.) 2020) to model the shared backbone due to their elfectiveness in handling temporal sequences. To tailor the model to specific tasks, we implement task-specific layers using a two-layer feod-forward network, providing lightweight and efficient customization for cach task.
Continual learning via evolving architecture and rehearsal. In (lask) incremental al adaptation, the model trained incrementally with one task a time. Thus, the model does not require data from all the tasks during training. When new task t is encountered, a task-specific prediction layer is created and both fo and f are trained using task-specific data. However, this method is prone to catastrophic forgetting French 1999: McCloskey & Cohen 1989), that is, the model forgets the previously trained tasks when it learns a new task. To deal with the forgetting problem, we include samples from prior tasks in the training (i.c., a 'rehcarsal' (van de Ven et al., 2020 However, beyond prior rehearsal approaches, we do not merely copy training data from prior tasks. Instead, we augment it, incorporating prediction errors from the current. model ff using the following scheme.
Prospective model. A principal challenge in scenario is that prediction errors at time step k may negatively impact prediction performance at time step + 1, such that samples are not identically and independently distributed across time. In more detail, for the human demonstrations, we know that the sensor state Xt, and desired kinematic profile yt, is followed by the sensor state Xt,++-- However, if we predict a slightly diffferntt kinematic profile Pr. = ((x,x=T..xx..t) # Yt,k: we will also observe a different sensor state & # x2,4+1 the next time step, pushing us away from the training data distribution. Such deviations can accumulate over time, which is well-known phenomenon in imitation learning (Kumar et al., 2022
Ross & Bagnel] 2010). In general, imitation learning helpful metaphor for our situation, we also
4

---

Published in Transactions on Machine Learning Research (01/2025)
wish to mimic the demonstration of human experts in setting where independence over time does not hold. However, past theory on imitation learning has usually assumed constant bounds on the error in every time step. Such constant bounds are not realistic in our scenario with a continuous space, Unfortunately, if we slightly relax the assumption of constant bounds to Lipschitz bounds, the deviation between desired time series and predicted time series can grow exponentially, as we show in the following, simple theorem.
Theorem 1. Let II,...,IIN E X be a time series from space X, equipped with metric d. Further, let f:X- + Y for some set y, and let g : xXXy xxy+X +X For any xf E X, we define the time series x'....; ,yy via the recursive equation r'2-1 = g f fx//)).
Finally, assume that for all k E (1,...,N- 1}, for all X, and for some C R, the following Lipschitz condition holds: d(g(z, ((x)), S C d(z,2z). Then, a) we obtain the exponentiol bound (L) S d(s',, and b) this bound is tight, i.e. there exist at least one combination of time series 211..,,, some f and 9, well some oi, such that the bound holds exactly.
Proof. We obtain a) via induction: The base case is x1x) S d((x), which is trivially fulfilled. Now, assume that the claim holds for k. Then, for ++1, we obtain d(z'+11 #x+1) d d(glzk. f ((x()), < C C-d(a'.. due to the definition of 4/++1 and the Lipschitz assumption made in the theorem. According to the induction hypothesis, the right-hand-side is bounded by C- cok-i-dazi, = C* -d(z(,z)), which concludes the induction.
b) First, note that it is sufficient to show one specific example to prove the tightnoss of the bound as we only claim that there exists at least one example for which the bound holds exactly. Then, assuming the setup of the example as given in the proof, we obtain the following induction. The base case is d(rl,,n) S C d(s(,,1), which is trivially fulfilled. Now, assume the claim holds for k. Then, for k + 1, we obtain 11=1)1:;)41)) utilizing the definitions of x1 =... =...= IN 0, = |x y), and C C==11 1 k. According to the induction hypothesis, the right- hand-side is equal to C -(=))_(((,, x1) = C d(z', æ1), which concludes the induction. Accordingly, for this example, the equality is always fulfilled exactly, implying that the bound is tight. ]
|at + x- 1/- - = (1+ | = C- 12 - 0 = C- - -xxl = C' d(s),,
In this theorem, f is our prediction model mapping sensor state X to target joint profile yk, and g describes the dynamics of our system, mapping the current sensor state Xx and joint profile yı to the next sensor state X&+1- The theorem states that, even if our predictive model f is Lipschitz-bounded (i.e., it exactly matches the training data and degrades gracefully beyond the training data), the deviation can grow exponentially over time.
This observation motivates the key innovation of our work, namely prospective rehearsal. More precisely, we train a prospective model g to mimic the dynamics of the system with g(x,, 3k) x Xk+1 and imagine the subsequent inputs resulting from the predictions of f. We then add data tuples (x1+1) Mk+1) with Xf+1 = g( Xx. (xxx-T...x)) to our training data. These additional training data tuples take the prediction errors of f into account and thus help f to counteract its own prediction errors. In our scenario, Uk+1 is the right target for the input x because yk+1 represents target joint profile in a prosthetic limb. This target profile should remain the same in the same gait. phase, even if X slightly deviates from Xx+1- In a general scenario, other targets might be required. Theoretically speaking, our aim is to achieve a constant bound d(g(xx, [x. (xx--T...... Xx )1, xx+1) < e for small e > 0, such that classical imitation learning theory applics 2022 Ross & Bagnell 2010) and an exponential deviation over time is avoided. We do so by
(Kumar et al.
minimizing d( (gg Xx. 5k) Xx+1) explicitly in Eq. equation
Summary of proposed approach. Our overall approach is shown in Algorithm I and Figures D and 2
We add new tasks one by one and split the training data for task t into training and a validation set. We train newly evolved task-specific layer ft and the shared backbone f to minimize the distance between [t[S" (x. and yt.k, and, at the same time, we train the shared backbone fe and the old task layers for to minimize the distance between fo (Xx k-1, &e,,)), x and yv, for the data in the rehearsal buffer. The rehearsal buffer contains both the original validation samples Xe, and the prospoctive samples * for a subsample Ve of the time steps le in the validation set. The prospective samples are generated via task-specific, prospective model ge as &e k = ge (xe, J [5 (XH, T-1 Xy.
5

---

Published in Transactions on Machine Learning Research (01/2025)
Algorithm 1: Multitask Prospective Rehearsal-based Adaptive Model
Initialize the shared backbone f".
for tasks & from 1,...,/ I. do
Let the data for task t be (t.),, ............. (X..N NN, Use the first M, steps as training data, the final No - M, steps as validation data for some Mt < No-
Add a task-specific layer ft- Train f and fi,..., ft by minimizing the loss:
M
f 1ye. fel5 (xx: (x...... x
Σ
Σ
Σ l
xv xV + llve - Sv (&e & &
Train the prospective model gr for task by minimizing the loss:
Mc-1
bg
Σ
|x ((xx,,,,..+)))
Update the rehearsal data for all tasks. for lask l' from ......6 do Vy + subsample time steps & from My,..., Ny - 1.
for time slep & E Vir do &y. = gr (-1 fo (xe x---A------ .... xk))=
VP & (& Me
4 Experiments
Datasets and metrics. We use three real-world human gait experiments' datasets. The first two, ENABL3S (Hu et al. 2018) and Embry (Embry et al. 2018), publicly available from popular gait labs. ENABL3S comprises approximately 5.2 million training samples and 1.3 million test samples collected from ten subjects performing various locomotion activities, including level-ground walking, stair ascent and descent, and ambulating on an inclined walkway. From the Embry dataset, we selected data representing nine very distinct combinations of walking speeds (ranging from 0.8 m/s to 1.2 m/s) and inclines (ranging from -10 degrecs to +10 degrees). This selection resulted in approximately 512,000 training samples and 104,000 test samples from ten subjects, encompassing a diverse range of movements. The third dataset is a novel collection from our lab, focusing gait patterns of patients (transtibial amputeos), recorded with 200 Hz infrared motion capture system using twelve cameras, and includes activities like walking, stair ascent, and descent, with around 27K training and 11K testing samples. Ethical clearance for these experiments was obtained from our institutional review board. This dataset, along with the others, includes motion-related variables such as 3D angles of body joints and segments, velocities, and accelerations. To evaluate the accuracy of the desired joint profiles, Ye', we calculate the coefficient of determination (R²), between the ground truth trajectories and model-basod joint profiles.
Ablations. We assess the contribution of each component overall performance. We explore both joint and continual learning paradigms, comparing models with shared final layers against those with task-specific final layers Dey et al. (2024). Further, we consider both conventional and prospective rehearsal with cach of the shared and task-specific final layer variants. In joint training, the training data is either augmented with the multitask prospective rehearsal from a validation set or simply with the original samples from the validation set itself (conventional rehcarsal). all conditions, we used the same amount of training data maintain fairness. Augmentation with prospective rehearsal improved the performance of models with both shared and task-specific final layers. Further, the results provide evidence that task-specific final layers help

---

Published in Transactions on Machine Learning Research (01/2025)
Table 1: Experimental results for (A) joint multitask learning using different prediction head architectures and replays, (B) task incremental learning using different prediction head architectures and replays, (C) different continual learning strategies SI (Synaptic Intelligence), EWC (Elastic weight. consolidation), PNN (Progressive Neural Networks), GEM (Gradient episodic memory), ER (Experience Replay), and proposed prospective replay, (D) data augmentation with Gaussian noise vs. prospective replay buffer. (IW=Level walking, RA=Ramp ascent, RD=Ramp descent, SA-Stair ascent, SD-Stair descent, Spd=Speed in m/s,
Tne=Inclination in )
Embor
Transtibinl
Datasel
ENABL3S
e6, al. 2018
ampuless
Task
LW
Spd
0.8
0.8
0.8
1.0
1.0
1.0
1.2
1.2
1.2
Inc
-10
0
10
-10
10
-10
10
bead
replay
(A) Joint multi-task learning
9
conventional
0.83
0.87
0.89
0.91
0.86
0.86
0.88
0.91
0.91
0.34
0.94
0.85
0.93
0,83
0.79
0.79
0.80
prospective (ours)
0.89
0.91
0.92
0.93
0.88
0.88
0.92
0.92
0.93
0.96
0.91
0.89
0.96
0.94
0.93
0.72
0.90
T
conventional
0.90
0.94
0.95
0.96
0:93
0.95
0.98
0.94
0.95
0.99
0:96
0.80
0.96
0.98
0.97
0.82
0:92
proqpective (ours)
0.92
0.94
0.96
0.96
0.94
0.96
0.98
0.96
0.96
0.99
0.97
0.90
0.97
0.98
0.98
0.91
0.94
replay
(B)
Yaalk-inerement
learning
conventional
0.50
0.56
0.81
0.89
0.83
0.76
0.87
0.88
0.81
0.90
0.90
0.71
0.50
0,88
0.91
0.83
0.68
prospective (ours)
0,81
0.90
0.59
0.91
0.89
0,90
0.92
0.92
0.94
0.96
0.95
0.89
0.97
0.95
0.91
0.85
0.72
T
conventional
0.58
0.53
0.93
0:94
0:92
0.92
0.98
0.93
0.94
0.98
0.97
0.87
0.97
0.97
0.97
0.91
0:91
prospertive (murs)
0.91
0.95
0.95
0.97
0.94
0.96
0.98
0.95
0.97
0.99
0.97
0.91
0.97
0.98
0..88
0.91
0.92
(C)
Continual learning stratiggess
SI
0.06
0.15
0.51
0.35
0.93
-0.38
0.33
0.59
0.26
0.40
0.83
0.31
0.66
0.97
0.97
0.31
0.92
EWC
0.13
0.31
0.55
0.52
0.92
0.04
0.33
0.65
0.08
0.54
0.92
0.53
0.77
0.96
-0.30
-0.57
0.85
PNN
0.90
0.91
0.94
0:93
0:91
0.85
0.97
0.96
0.58
0:97
0:96
0.87
0.96
0.98
0.68
0.82
0.79
GEM
0.76
0.91
0.92
0:95
0:93
0.88
0.97
0.93
0.90
0:97
0:96
0.844
0.917
0.97
0.82
0.50
0.86
0.94
0.92
0.98
0.97
0.87
0.97
0.97
0.91
prospeesive (owss)
0.91
0.95
0.95
0.97
0.94
0.96
0.98
0.95
0.97
0.99
0.99
0.91
0.91
0.98
0.988
0.91
0.92
(D) Data augmentation
Gaussinn nnlse Inj.
0.58
0.33
0:33
0.34
0.91
0.51
0:97
0:96
0.81
0.97
0.32
0.83
0.89
prospective (owrs)
0.91
0.95
0.95
0.97
0.94
0.96
0.98
0.95
0.97
0.99
0.97
0.91
0.97
0.98
0.98
0.91
0.92
to account for particularities among tasks. A comparison of performance in task-incremental learning yiclds similar results. The results demonstrate that the choice of a combination of prospective rehearsal and an evolving architecture, where task-specific final layers are integrated with shared backbone, yields the best performance across the board (Table ABB). All following experiments are performed with task-specific final layers.
Effect of distribution shift on performance
ENABL35
Embry dataset
inc incineeee 20"
Indinew 1D"
Indinee 1 tran
AR (prospective Intrentoonn
0.32
0.08
0.06
0.08
0.04
0.64
Houregr
Destence
0,212
0.116
0.105
0.55
0.55
conventional rehearsa
0.91
0.97
0.87
0.97
005
prospective reheorsal
6,97
0.97
0.88
0.88
00.4
15 Distance
0.02
DiDtaacce
Figure 3: Comparison of prospective rehearsal with a conventional rehearsal in the face of distribution shift. (Left) Training/tes test data distributions, their Jenson-Shannon (JS) distances, and Rt2 predictions on four different (asks from the Embry dataset show the prospective rehearsal's 's efficacy. (Right) A graph plots the H² difference between methods against JS distance, with a linear trendline indicating that the prospective rehcarsal's benefits increase with greater distribution divergence.
Benchmarking continual learning strategies. In the continual learning paradigm, we compare our approach against regularization strategics like SI and EWC, architectural strategics, PNNs, and state-of- the-art replay-based strategies like GEM and the classic form of experience replay, which we refer to as conventional rehearsal (Table [C). We observe that prospective rehearsal performs best across the board, indicating better task memory retention and generalization. In Table [P, we compare data, augmentation via Gaussian noise injection to the proposed prospective rehearsal, with prospective rehearsal performing best across the board. All results are statistically significant (see appendix).
7

---

Published in Transactions on Machine Learning Research (01/2025)
Resilience to distribution shift. In some cases, we do not observe significant improvement in the performance of the prospective rehearsal over conventional rehearsal. To explain these results, we inspect the difference in Rº between the prospective and conventional rehearsal versus the Jenson-Shannon (JS) distance (Endres& & Schindolin, 2003) between training and test distributions (Figure 3 We observe that the error difference positively correlates with JS distance. In other words, the advantage of the prospective rehearsal becomes more pronounced the more training and test distribution deviate. This in line with our theoretical analysis: The more training and test distribution differ, the more prediction errors of f we expect, which means that proposed prospective rehearsal helps more. Since real-world prosthesis data is prone to noise and variability from various exteroceptive and interoceptive factors (Prahm et al. 2019), being robust to distribution shifts is an important requirement for this domain, and our approach performs well in this
regard.
A
B
C
1.0
0.95
conventional
prospective
0.9
0.90
0.8
accuracy
R²
0.8
0.85
0.6
0.7
0.80
0.4
0.0
0.01
0.02 0.03
0.04 0.05
0.6
Linear
Nonlinear
0.75
0.0
0.001
0.005
0.01
Figure A) Performance comparison of prospective and conventional rehearsal strategics across different strengths, T, of adversarial perturbations. B) Performance of linear and non-linear probing on downstream locomotion task classification. Prospective rehearsal training performs best across the board. (C) Comparison of the performance of continual models with conventional and prospective rehearsal for different levels of input noise. Results are presented for models with task-specific final layers.
Resilience to adversarial perturbations. To further assess the resilience of the prospective rehearsal approach, we compare its outcomes to those without it in the face of adversarial perturbations. These adversarial samples are created using the Fast Gradient Sign Method (FGSM) (Goodfellow et al. 2014] across diverse epsilon thresholds, т. Notably, our prospective rehearsal technique surpasses the conventional bascline across all ovaluated FGSM attack intonsitics (Figure A Furthermore, as the perturbation intonsity escalates, the performance disparity between the prospective and the conventional method widens, reinforcing the superior adaptability of prospective rehearsal under scenarios where test data distribution deviates from the training set. The results are statistically significant.
Probing downstream performance. We evaluate the representation capacity of models trained with prospective rehearsal techniques versus those trained without it, focusing on downstream chal- lenge classifying different types of locomotion. We employ both linear and nonlinear (MLP) probes to assess the representation of the shared layers, denoted as (x from models with task-specific heads aimed at classifying the task t. The findings indicate that prospective rehearsal yields significantly better task classification performance than its conventional counterpart for both linear and nonlinear probes (Figure 4B). This result underscores that the prospective model- based training confers an added benefit of robustness and adaptability compared to conventional rehearsal.
Resilience to noisc. Further, we analyze the robustness of our proposed model against perturbations to the input signals during test. time in the form of multivariale white Gaussian noise, (O,0, Σ E dRdd)). We apply different levels of perturbation L, where is defined as the ratio of the standard deviation of the Gaussian noise to the standard deviation of the test inputs. Figure 4C shows the R² of our proposed rehearsal and a conventional rehearsal for five different tasks on the ENABL3S dataset. While performance generally decreases with higher amounts of noise, the prospective rehearsal yields better R² across the board.
8

---

Published in Transactions on Machine Learning Research (01/202))
100
convertiooal
40
prospective
Dendfy
OR:
60-
Treadwil walk
Treadiill walk
Treadmal wotk, degrees
D Treadeell walk degrees
e Troadesil walk. 12 degrees
Kicking wait
Gressed ground
10
0.09
0:10 A
0.12
0.02
0,03 Aa
0.4
Experiment
A1
Conventional rehearsal
0.095
Prospective rehearsal (ours)
0.089
H Standing
Slow speed ground
Backwards ground walk
arn-around (elociiwtes)
Tarn around (courterr clockutte
Forwards trattig walk
Hoelloon
ground
Figure 5: (Left) Snapshots from the experiments on a subject. Different motion conditions such as walking at different speeds and inclines were tested on an instrumented treadmill with adjustable inclines (A-E)- Furthermore, ground walking on level walkway with different motion maneuvers was also tested (F N). (Right) Stability comparison of foot kinematic profiles yielded using prospective and conventional rehearsals. First Å and second A2 Lyapunov exponents (lower is better), computed using the Eckmann method, indicate that the foot kinematic profiles obtained using proposed method exhibit greater stability compared to those obtained without employing our method. Median Lyapunov exponent of the prospective rehearsal method, (dotted line) is lower compared to conventional rehearsal (solid line).
Gait stability assessment with an exoskeleton. In a pilot study, able-bodied individuals walked with an ankle exoskeleton (Figure 5 left) controlled by joint profiles predicted by the model with the prospective rehearsal approach. Participants reported that the predictions felt timely, accurate, and in syne with their residual body movements. To evaluate the stability of the model-based foot profiles, with and without the prospective rehearsal approach, measured the maximal Lyapunov exponent using the Eckmann method (Eckmann et al. 1986). The Lyapunov exponent measures the rate of divergence of nearby trajectories in dynamical system, with lower values indicating higher stability. Our results showed that both the first and second Lyapunov exponents were lower when using our proposed method, indicating more stable gait. trajectories (Figure right).
5 Conclusion
Our research introduces novel framework for bionic prostheses' application, adept at handling multiple locomotion lasks, adapting progressively, foreseeing movements, and refining. Central to our method, is novel prospective rehearsal training scheme that, through empirical studies on diverse datasets, reliably outperforms standard tochniques, particularly under challenging conditions adversarics, distribution shifts, noise, and task transfer. Notably, the prospective rehearsal approach is data-centric, hence, model-agnostic, and can be applicable across various model architectures. While these results are promising, a limitation of our study is the absence of comprehensive clinical trials, largely due to the rigorous and extensive ethical approval process required for such rescarch. Recognizing this, our future work is set to expand upon these findings through more extensive trials, including clinical trials and in-home studies, to validate and refine our model in everyday settings. This adaptable, robust system signals a significant leap forward for prosthetic gait prediction in dynamic settings, offering improved quality of life for individuals with lower limb impairments.

---

Published in Transactions on Machine Learning Research (01/2025)
References
Guoxing Chen, Zuojun Liu, Lingling Chen, and Peng Yang. Control of powered knee joint prosthesis based finitestaate machine. In Proceedings of the 2015 Chinese Intelligent Automation Conference, pp. 395 403. Springer, 2015.
Kurtland Chua, Roberto Calandra, Rowan Mcstrrster, and Sergey Levine. Deep reinforcement learning in a handful of trials using probabilistic dynamics models. Advances in neural information processing systems, 31, 2018.
Steve C Culver, Léo G Vailati, David C Morgenroth, and Michael Goldfarb. A new approach to a powered knee prosthesis: Layering powered assistance onto strictly passive prosthesis behavior. Wearable Technologies, 4: e21, 2023.
Sharmita Dey and Sarath R Nair. Enhancing joint motion prediction for individuals with limb loss through model reprogramming, Xii preprind N4w:2443306660 2024.
Sharmita Dey and Arndt Schilling. Data-driven gait- gait-predictive model for anticipatory prosthesis control. In 2022 International Conference on Rehabilitation Robotics (ICORR), pp. 1-6. IEFE, 2022a.
Sharmita Dey and Arndt F Schilling. A function approximator model for robust online foot angle trajectory prediction using single imu sensor: Implication for controlling active prosthetic feet. IEEE Transactions Industrial Informalies, 19(2):1467-144 1475, 2022b.
Sharmita Dey, Mahdy Eslamy, Takashi Yoshida, Michael Ernst, Thomas Schmalz, and ArndtF' Schilling, A support vector regression approach for contimuous prediction of ankle angle and moment during walking: An implication for developing a control strategy for active ankle prostheses. 2019 IEEE 16th International Conference on Rehabilitation Robotics (ICORR), pp. 727-733. IEEE, 2019.
Sharmita Dey, Takashi Yoshida, Robert H Foerster, Michael Ernst, Thomas Schmalz, and Arndt F Schilling. Continuous prediction of joint angular positions and moments: A potential control strategy for active knee-ankle prostheses. IEEE Transactions Medical Robolics and Bionics, 2(3):347 355, 2020a.
Sharmita Dey, Takashi Yoshida, and Arndt Schilling. Feasibility of training random forest model with incomplete user-specific data for devising a control strategy for active biomimetic ankle. Frontiers in Bioengineering and Biotechnology, 8:855, 2020b.
Sharmita Dey, Sabri Boughorbel, and Arnd Schilling. Learning a shared model for motorized prosthetic joints to predict ankle-joint motion. arXio preprinl urxi::2111.0741 2021a.
Sharmita Dey, Takashi Yoshida, Robert H Foerster, Michael Ernst, Thomas Schmalz, Rodrigo M Carnier, and Arndt F Schilling. A hybrid approach for dynamically training torque prediction model for devising a machnn interface control strategy. arXiv preprint arXic:2110.033: 10.03085, 2021b.
Sharmita Doy, David Fan, Robin Schmid, Amushri Dixit, Kyohei Otsu, Thomas Touma, Arndt F Schilling, and Ali-Akbar Aghaa-Mhaammdii. Prepare: Predictive proprioception for agile failure event detection robotic exploration of extreme terrains. In 2022 JEEE/RS. International Conference on Intelligent Robots and Systems (IROS), pp. 4338 4343. IEEE, 2022.
Sharmita Dey, Benjamin Paassen, Sarath Ravindran Nair, Sabri Boughorbel, and Arndt F Schilling, Continual imitation learning for prosthetic limbs. arXiv preprint ar arXin:2405.01 0111,, 2024.
Neil Dhir, Houman Dallali, Evandro Maicon Ficanha, Guilherme Aramizo Ribeiro, and Mo Rastgaar. Locomotion envelopes for adaptive control of powered ankle prosthescs. In 2018 IEEE International Conference on Robolics and Automation (ICRA), pp. 1488 1495. IEEE, 2018.
Natalia Dlaz-Rodríguez, Vincenzo Lomonaco, David Filliat, and Davide Maltoni. Don't 't forget, there is more than forgetting: metries for continual learning. aixiv preprint ar Xiv 13166, 2018.
10

---

Published in Transactions on Machine Learning Research (01/2025)
J-P Eckmann, S Oliffson Kamphorst, David Ruelle, and S Ciliberto. Liapunov exponents from time series. Physical Review A, 34(6):4971, 1986.
Kyle R Embry, Dario J Villarreal, Rebecca L Macaluso, and Robert D Gregg. Modeling the kinematics of human locomotion over continuously varying speeds and inclines. IEEE transactions on neural systems and rehabilitation engineering, 26(12):2342 2350, 2018.
Dominik Maria Endres and Johannes E Schindelin. A new metric for probability distributions. Transactions Information theory, 49(7):1858-1888)) 2003.
IEEE
Peter Englert, Alexandros Paraschos, Mare Peter Deisenroth, and Jan Peters. Probabilistic model-based imitation learning. Adaptive Behavior, 21(5):388-403 2013.
Bin Fang, Quan Zhou, Fuchun Sun, Jianhua Shan, Ming Wang, Cheng Xiang, and Qin Zhang. Gait neural network for human-exxokelleto interaction. Frontiers in Neurorobotics, 14, 2020.
Robert M French. Catastrophic forgetting in connectionist networks. Trends in cognilive sciences, 3(4): 128 135, 1999.
Ian J Goodfellow, Jonathon Shlens, and Christian Szegody. Explaining and harnessing adversarial examples. arXiv preprint arXiv:442.66572, 2014.
Anthony Hu, Gianluca Corrado, Nicolas Griffiths, Zachary Murez, Corina Gurau, Hudson Yeo, Alex Kendall, Roberto Cipolla, and Jamie Shotton. Model-batt based imitation learning for urban driving, Advances in Newral Information Processing Systems, 35:20703-210 -07716, 2022.
Blair Hu, Elliott Rouse, and Levi Hargrove. Benchmark datasets for bilateral lower-limb neuromechanical signals from wearable sensors during unassisted locomotion in bodied individuals. Frontiers in Robotics and AT, 5:14, 2018.
Kshitij Judah, Alan Paul Fern, Thomas G Dietterich, and Prasad Tadepalli. Active Imitation learning: formal and practical reductions to iid learning. J. Mach. Learn. Res., 15())33925-3963, 2014.
Michael Kelly, Chelsea Sidrane, Katherine Driggs-Camplnn and Mykel J Kochenderfer. Hg-dagger: Interactive imitation learning with human experts. In 2019 International Conference on Robotics and Automation
(ICRA), pp. 8077 8083. IEEE, 2019.
Rahul Kidambi, Jonathan Chang, and Wen Sun. Mobile: Model-based imitation learning from observation alone. Advances in Neural Information Processing Systems, 34:28598 28811, 2021.
James Kirkpatrick, Razvan Pascanu, Neil Rabinowitz, Joel Veness, Guillaume Desjardins, Andrei A Rusu, Kieran Milan, John Quan, Tiago Ramalho, Agnieszka Grabska-BBrwinske, et al. Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences, 114(13):3521 3526, 2017.
Alexander Krawezyk and Alexander Gepperth. An analysis of best-practicn strategics for replay and rehcarsal continual learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pullern Recognition, pp. 4196 0264, 2024.
Aviral Kumar, Jocy Hong, Anikait Singh, and Sergey Levine. Should run offline reinforcement learning behavioral cloning? In International Conference on Learning Representations, 2022. URL https;
Brian E Lawson, Amanda H Shultz, and Michael Goldfarb. Evaluation of a coordinated control system for a pair of powered transfemoral prostheses. In 2013 IEEE International Conference on Robotics and Automation, pp. 3888 3893. IEEE, 2013.
Brian Edward Lawson Control methodologies for powered prosthetic interventions in unilateral and bilateral transfemoral amputees. PhD thesis, Vanderbilt University, 2014.
11

---

Published in Transactions on Machine Learning Research (01/2025)
Xingyu Li, Bo Tang, and Haifeng Li. Adaer: An adaptive experience replay approach for continual lifelong learning. Neurocompuling, pp. 127204, 2023.
David Lopez-Paz and Marc' Aurelio Ranzato. Gradient episodic memory for continual learning. Advances in neural information processing systems, 30, 2017.
Jicliang Luo and Hui Li. Dynamic experience replay. In Conference on robot learning, pp. 1191-1200. PMLR, 2020.
Michael McCloskey and Neal J Cohen. Catastrophic interference in connectionist networks: The sequential learning problem. In Psychology of learning and molivation, volume 24, pp. 109-165. Elsevier, 1989.
Gabriele Merlin, Vincenzo Lomonaco, Andrea Cossu, Antonio Carta, and Davide Bacciu. Practical recom- mendations for replay-based continual learning methods. In International Conference on Image Analysis and Processing, pp. 548-559. Springer, 2022.
Volodymyr Mnih, Koray Kavukcuoglu, David Silver, Alex Graves, Joannis Antonoglou, Daan Wierstra, and Martin Riedmiller. Playing atari with deep reinforcement learning. arXiv preprint arXiv:1312 1602.662, 2013.
Benjamin Morrell, Kyohei Otsu, Ali Agha, David D Fan, Sung-Kyun Kim, Muhammad Fadhil Ginting, Xianmei Lei, Jeffrey Edlund, Seyed Pakoorian, Amanda Bouman, et al. An addendum to nebula: Towards extending team costar's solution to larger scale environments. IEEE Transactions on Field Robotics, 2024.
Anusha Nagabandi, Gregory Kahn, Ronald S Fearing, and Sergey Levine. Neural network dynamics for model-based deep reinforcement learning with model-free fine-tuning. In 2018 IEEF international conference robotics and automation (ICRA), pp. 7559-7566. IEEE, 2018.
Aaron van den Oord, Sander Dieleman, Heiga Zen, Karen Simonyan, Oriol Vinyals, Alex Graves, Nal Kalchbremner, Andrew Senior, and Koray Kavukcuoglu. Wavenet: A generative model for raw audio. arxiv preprint Arxiv:1609. 03499, 2016.
Cosima Prahm, Alexander Schulz, Benjamin Paaßen, Johannes Schoisswohl, Eugenijus Kaniusas, Georg Dorffner, Barbara Hammer, and Oskar Aszmann. Counteracting electrode shifts in upper-limb prosthesis control via transfer learning. IEEE Transactions Newral Systems and Rehabilitation Engineering. 27(5): 956-962, 2019. doi: 2011109/TSRR.2199992
Christopher A Rábago, Jonathan B Dingwell, and Jason M Wilken. Reliability and minimum detectable change of temporal-spatial, kinematic, and dynamic stability measures during perturbed gait. PloS one, 10 (11):e014200) 42083, 2015.
Sébastien Racanière, Théophane Weber, David Reichert, Lars Buesing, Arthur Guez, Danilo Jimenez Rezende, Adrià Puigdomènech Badia, Oriol Vinyals, Nicolas Hoess, Yujia Li, et al. Imagination-augmented agents for deep reinforcement learning. Advances in neural information processing systems, 30, 2017.
Anthony Robins. Catastrophic forgetting, rehearsal and pseudorehearsal. Connection Science, 7(2):123 146, 1995.
Stephane Ross and Drew Bagnell. Efficient. reductions for imitation learning. In Proceedings of the Thörteenth International Conference on Artificial Intelligence and Statistics, pp. 661 668, 2010. URL https:// proceedings mlr. wwwpss/eeereeeeeeeeeeee
html
Stéphane Ross, Geoffrey Gordon, and Drew Bagnell. A reduction of imitation learning and structured prediction to no-regret online learning. In Proceedings of the fourteenth international conference on artificial intelligence and statistics, pp. 627-635. JMLR Workshop and Conference Proccedings, 2011.
Andrei Rusu, Neil C Rabinowitz, Guillaume Desjardins, Hubert Soyer, James Kirkpatrick, Koray Kavukcuoglu, Razvan Pascanu, and Raia Hadsell. Progressive neural networks. arXiv preprint Niv::606.0440 2016.
12

---

Published in Transactions on Machine Learning Research (01/2025)
Robin Schmid, Deegan Atha, Frederik Schöller, Sharmita Dey, Seyed Fakoorian, Kyohei Otsu, Barry Ridge, Marko Bjelonic, Lorenz Wellhausen, Marco Hutter, et al. Self-supervised traversability prediction by learning to reconstruct safe terrain. In 2022 LEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 12419-12425. IEEE, 2022.
David Silver, Aja Huang, Chris J Maddison, Arthur Guez, Laurent Sifre, George Van Den Driessche, Julian Schritlwieser, Toannis Antonoglou, Veda Panneershelvam, Mare Lanclot, et al. Mastering the game of go with deep neural networks and tree search. nature, 529(7587):484 489, 2016.
James Seale Smith, Junjiao Tian, Shaunak Halbe, Yen-Chang Hsu, and Zsolt Kira. A closer look at rehearsal- free continual learning. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pp. 2409 2419, 2023.
Gido M van de Ven, Hava Siegelmann, and Andreas S Tolias. Brain-inspired replay for continual learning with artificial neural networks. Nature communiicalinss, 11(1):1 14, 2020.
Michael Windrich, Martin Grimmer, Oliver Christ, Stephan Rinderknecht, and Philipp Beckerle. Active lower limb prosthetics: a systematic review design issues and solutions. Biomedical engineering online, 15(3): 140, 2016.
Tianhe Yu, Garrett Thomas, Lantao Yu, Stefano Ermon, James Y Zou, Sergey Levine, Chelsea Finn, and Tengyu Ma. Mopo: Model-based offline policy optimization. Advances in Neural Information Processing Systems, 33:14129-14142, 2020.
Erika V Zabre-Gonzalez, Lara Riem, Philip A Voglewede, Barbara Silver-Thorn, Sara R Koehler-McNicholas, and Scott A Beardsley, Continuous myoelectric prediction of future ankle angle and moment across ambulation conditions and their transitions. Frontiers in Newroscience, 15:709422, 2021.
Friedemann Zenke, Ben Poole, and Surya Ganguli. Continual learning through synaptic intelligence. Tn International Conference Machine Learning, pp. 3987 3995. PMLR, 2017.
Wenxuan Zhou, Sujay Bajracharya, and David Held. Plas: Latent action space for offline reinforcement learning. In Conference on Robot Learning, pp. 1719-1735. PMLR, 2021.
13

---

Published in Transactions on Machine Learning Research (01/2025)
A Additional Results
A.1 Model architecture
In Table 2 we show the results of using different architectures for the shared backbone f All models were designed to have approximalely the same number of parameters. Since TON lends to perform best across datasets, we opt for TCN as the backbone architecture in practice.
Dataset
ENABLIS
Einbey
Transtihial amputees
et. al. 2018
Spd
0.8
1.0
1.2
Task
1.W
RA
RD
SA
LXV
SA
SD
Inc
-10
10
-10
10
-10
10
Linear
0.68
0.89
0.80
0.30
0.33
0.04
0.15
0.51
0.04
0.12
0.55
0.13
0.11
0.55
0.30
0.64
0.68
MLP
0.67
0.91
0.91
0.91
0.86
-0.06
0.14
0.32
0.06
0.13
0.55
0.11
0.12
0.56
0.42
0.56
0.70
LSTM
0.94
0.94
0.96
0.93
0.97
0.95
0.83
0.96
0.82
0.95
0.97
0.41
0.50
Transformer
0.9
0.94
0.95
0.96
0.93
0.88
0.97
0.96
0.86
0.98
0.97
0.88
0.97
0.98
0.98
0.84
0.92
TCN
0.91
0.95
0.95
0.97
0.94
0.96
0.98
0.95
0.97
0.99
0.97
0.91
0.97
0.98
0.98
0.91
0.92
Table 2: Comparison of various shared module architectures on different locomotion conditions from three datasets.
A.2 Model predictions
In Figure T (left), We show the model-based motion trajectories of a transtibial amputee subject's hypothetical ankle, as demonstrated by our multitask model using prospective rehcarsal. The aligmment between the predicted and biological able-bodied limb motions suggests the model's elfectiveness in approximating gait synergy across various lower limb joints and segments, aiming to replicate normal locomotion for impaired patients. Figure 6
(right) presents analogous results for an able-bodied subject, using their own limb motion the larget. The close match between model-based and natural limb motions further underscores the model's potential in generating movement signals for bionic limbs, offering hope for enhancing mobility in individuals with lower-extremity disabilities.
In Figure 7 we show the example predictions from baseline models as compared to our approach. The joint angular positions predicted by the baseline models (shared final layer model and EWC regularization) showed frequent misaliguments with sharp disagreements between the predicted and actual trajectories. In a safety-critical scenario such as prosthesis control for locomotion assistance, such misalignments can be very costly as they can lead imbalance. On the other hand, predictions from our proposed model show a higher degree of alignment with the ground truth (actual) trajectories, which is necessary to ensure smoother locomotion using& prosthetic device controlled by these predictions.
A.3 Resilience distribution shift
We show the effectiveness of our prospective rehearsal-based approach in dealing with distribution shifts during test time from the lens of model predictions. Figure B illustrates the predictions of models trained with conventional rehearsal and our prospective rehearsal for two scenarios one with low and the other with high distribution shift during test time. As anticipated, both models exhibit comparable performance when the test and training distributions are similar (see the bottom of Figure S However, in the presence of a distribution shift at test time, the model trained with conventional rehearsal shows a decline in performance, while the prospective rehearsal-trained model sustains its predictive accuracy (see the top of Figure S)
A.4 Stability against forgetting
We explore the ability to preserve the knowledge of previously learned tasks after learning new ones. Figure gA shows how the NRMSE values (lower better) for each task develop when training subsequent tasks, both for a shared final layer (top) and for task-specific final layer (bottom). Both architectures preserve the memory of the previous tasks, but the performance on old tasks slightly worsens for a shared final layer, whereas it remains the same for task-specific final layers.
14

---

Published in Transactions on Machine Learning Research (01/2025)
Walking
Normative Predicted
Level ground walking
Actual Predicted
1
0
Ramp ascent
1
normalized e antlee
Going upstairs
normalized O
0
Ramp descent
1
0
Stair ascent
Going downstairs
0
Stair descent
200
400
600
800
1000 2000 3000 4000 5000 6000 7000 8000 samples
samples
Figure 6: Model-basod kinematic profiles from our proposed rehearsal-based models. (Left) Motion trajectories of a transtibial amputee subject's hypothetical ankle, predicted by our model. The normative (target) trajectories are computed from an able- bodied individual with anthropometric features and walking speed similar to that of the amputee subject. (Right) Predicted motion trajectories of an able-bodied subject from ENABL3S dataset for different locomotion modes. The target trajectories in this case the subject's own limb motion. In both cases, the predicted tra trajectoiees highly aligned with the larget limb molion.
Level ground wallong
Level ground walcin
10
shared call-pooffc
Proppetiie
0.0
Ramp ascent
Ramp eScent
10-
1.-
Noralizzdd joint positinn
0.0
0.0
Serro descent
Ramp descent
0.-
0.0
10-
05.
0.5
0.0
104
0.5
055
0 1000 2000
3000
4000
5000
1000
3000
4000
5000
samples
samples
Figure 7: Baseline prediction examples. (Left) Examples joint position trajectorics for different locomotion tasks predicted by shared final layer model vs. task-specific final layer model. (Right) Examples of joint position trajectories predicted by task-specific final layer model with EWC regularization task-specific final layer model with prospective rehearsal. The black curve shows the ground truth trajectories.
15

---

Published in Transactions on Machine Learning Research (01/2025)
inlleww 10
indiirew 30'
1.0
1.3
scodd=i.2ns Incine--"
Alia Conventional iehernal Frosgastrve
06
10p 2ng
semp MS
Figure 8: (Left) Train and test distributions for two tasks from Embry et. al. dataset for a representative subject. (Right) Predictions of model trained with conventional and prospective rehearsal.
A
Embry datase!
ENABL3S
B
0.3-
N80SE
0.2
0.9
0.1
0.8
0.0
0.8
0.4
0.4
conventional
0.6
0.5
0.2
prospective
0.8
0.6
1.0
1.0
1.2
R56
RD
5A
50
shared
task-specific
GBM
PHN
Gaussan suy prospectiv reheteraal
(3c/7)
head
lacks
Figure 9: (A) Plot depicting NRMSE values for test tasks from Embry and ENABL3S datasets post-training, with the X-axis for learned tasks, color for test tasks, and shading indicating standard error across subjects. (B) (Left) Results statistical significance tests comparing the performance of conventional and proposed prospective rehearsal for shared and task-specific final layers. (Right) Comparison performance of the proposod prospoctive rehcarsal with -tata-tf-tho-art continual lcarning strategies. Statistical significance was assessed using the Wilcoxon-signed rank test. with Bonferroni correction, : > le 4,*** :p le 3,
p 1e-2-
Train task
Walk
Ramp
Remp down
Stair
Stuir down
Walk
0.054
0.14
0.16
0.177
0.18
Ramp
0.454
0.055
0.147
0.134
0.176
Test task
Ramp down
0.517
0.517
0.055
0.165
0.156
Stair
0.516
0.516
0.516
0.054
0.193
Stair down
0.506
0.506
0.502
0.505
0.066
Table 3: RMSE values of a model with task-specific prediction heads trained without replay on different tasks after it is trained for particular task.
We further show through ablation studies that, both the rehearsal and architectural choice are necessary for preventing the forgetting problem (see Tables 5 A model trained without replay (Tab, 3 excels only its current task, but suffers from catastrophic forgetting (Tab. and 5) reduce forgetting. Furthermore, using task-specific prediction heads outperforms a shared head further mitigaling forgetting. Thus, both replay and task-specific heads help address forgetting.
previous tasks. In contrast, models with replay
In addition, systematically analyze the model performance as is trained for multiple tasks, we also computed two metrics (similar to DJaz-RRodrigue, al. (2018)) to measure how well the model performs on previously trained tasks after training it with tasks.
16

---

Published in Transactions on Machine Learning Research (01/2025)
Train task
Walk
Ramp up
Ramp down
Stair up
Stair down
Walk
0.054
0.062
0.067
0.071
0.078
Ramp up 0.15
0.067
0.074
0.075
Test task
Ramp down
0.175
0.19
0.064
0.083
0.083
Stair up 0.213
0.184
0.215
0.071
0.083
Stair down
0.235
0.231
0:204
0.228
0.086
Table 4: RMSE values of model with task-shared prediction heads trained with replay on ciffeeent tasks after it is trained for particular task.
Train
Walk
Ramp up
Ramp down
Stair up
Stair down
Walk 0.053
0.057
0.058
0.059
0.062
Ramp up 0.454
0.057
0.055
0.056
0.056
Test task
Ramp down
0.502
0.501
0.051
0.056
0.057
Stair up 0.553
0.551
0.551
0.053
0.054
Stair down
0.53
0.53
0.53
0.53
0.066
Table 5: RMSE values of a model with task-specific prediction heads trained with replay on different tasks after is trained for a particular task.
The backward transfer is computed as
t-1 BWT(() - =c())
(1)
where is the latest task the model is trained for, ex(5) is the prediction trained with the task i. BWT(() computes the change in performance of the model on previously trained tasks, j = 1.it - after it is trained for a new task = 2. Larger values of backward transfer are preferred.
task j once the model
The forgetting ratio for task t is computed as
FR(t) (T(1) (1(1) (t(
(2)
where T' represents the final task on which the model was trained. FR(t) gives the relative change in performance of the model on task, t = 1..7 -1 1, after it was trained with all available tasks. Smaller values of forgetting ralio are preferred.
Train task, t
Walk Ramp
Ramp down
Stair
Stair down
replay
-0.043
-0.065
-0.079
-0.098
task-shared with replay
-0.004
-0.006
-0.013
-0.014
task-specific with replay
-0.002
-0.001
-0.002
-0.003
Table 6: Backward knowledge transfer, (t) (transfer of knowledge to previously learned tasks) when the model is trained with a new task, t. Higher the better.
Evaluation task, t
Walk
Ramp up
Ramp down
Stair
Stair down
replay
2.33
2.29
1.99
2.64
task-shared with replay
0.42
0.25
0.27
0.16
task-specific with replay
0.16
0.1
-0.02
Table 7: Forgetting ratios, FR(t), of the model on a task, after training it with all available tasks. The lower the better.
Tables and show the backward transfer and forgetting ratios of models trained without replay, and the ones trained with replay with different architectures. These results further illustrate that the models with
17

---

Published in Transactions on Machine Learning Research (01/2025)
task-specific prediction heads and replay perform best in the face of continual task learning and dealing with calastrophic forgetting.
A.5 Statistical significance of results
A Wilcoxon-signed rank test revealed that the proposed rehearsal significantly outperforms conventional rehcarsal, both for shared well as task-specific final layers (Figure DB). Further, task-specifie final layers signilicantly oulperform a shared final layer, irrespective of rehearsal strategy. This shows that both the evolving architecture as well as the prospective rehearsal strongly contribute to the improvement in performance using the proposed method. Our experiments also show that the proposed rehearsal significantly outperforms many state-of-the-art continual learning stralegies and data augmentation with Gaussian noise.
B Real-world Deployment of Models: Exoskeleton/Hardwarr Experiments
B.1
Experimental setup
The experimental setup for the real-world deployment of the proposed models is depicted in Figure 10 Utilizing Inertial Measurement Units (LMUs) to monitor the user's residual body motion, our model could infer and predict the ankle-joint positions real-time. These predictions were then employed as control inputs for the ankle exoskeleton. The ankle exoskeleton, configured in position control mode, received foot angular position predictions generated by the gait-prodictive model to regulate its movements. The gait-predictive model, trained and stored on computer, produced joint position predictions corresponding to the movement of the residual body. These predictions were transmitted through CAN communication system to actuate the ankle exoskeleton motor.
Wireless IMU Sensor
PC
Predictive model
DC in
Main Controller
Motor
IMU base-
station
Figure 10: Communication pipeline of the ankle exoskeleton is shown. The ankle exoskeleton is equipped with a DC power inlet and CAN communication inlet to interact with the main controller of the ankle exoskeleton. First, the IMU sensor signals are received by the computer (PC) via the IMU base station. The IMU signals are processed in real-lime, and the trained model stored on the PC generates predictions control the ankle exoskeleton. The predictions are sent to actuate the ankle exoskeleton via the CAN communication protocol to the main controller of the ankle exoskeleton, which in turn sends commands to actuate the device's motor.
Besides the quantitative stability comparison of gait reported in the main paper, we also analyzed data on trunk posture during two experimental scenarios: one with the subjects walking using the model-predicted ankle positions with the exoskeleton, and another where they walked without any assistance. We additionally looked at the variability trunk kinematics, which another critical indicator of balancod gait (Rábago et al., 2015). Figure [1 shows the lateral trunk angles in the case where a subject walked freely (blue) and when the subject walked with the assisted powered exoskeleton, whose ankle-joint positions were predicted from
18

---

Published in Transactions on Machine Learning Research (01/2025)
2008
2500
sempree
Figure 11: Lateral trunk angles when a subject walked freely (top) and when the subject walked with the assisted powered exoskeleton, whose ankle-joint positions were predicted from our model (bottom).
our model (red). Our findings revealed that the lateral trunk angles during assisted and unassisted walking were comparable, suggesting that the exoskeleton, guided by our model's predictions, did not necessitate any compensatory movements from the subjects. This similarity implies that our model can effectively predict control signals that facilitate stable and comfortable walking experience.
C Baselines
C.1 Continual learning strategies
We compared our method to other continual learning strategics such as synaptic intelligence (SI) (Zenke ct al. 2017), clastic weight consolidation (EWC) (Kirkpatrick et al. 2017), experience replay (ER), and gradient episodic memory (GEM) (Lopez-Pa Ranzato 2017).
SI is regularization-basse continual learning strategy aimed at overcoming catastrophic forgetting, which is the tendency of neural networks to completely forget previously learned information upon learning new data. ST achieves this by measuring the importance of synaptic parameters to past tasks and selectively constraining the update of these important parameters when new tasks are learned. This method estimates a "synaptic importance" score for each paramcter of the neural network, based on how much a change in that parameter would affect the total loss across all tasks learned so far. When new data is encountered, SI allows the network to update primarily those parameters that less important for the tasks learned previously, thereby preserving the performance those tasks.
EWC is another regularization- based continual learning strategy that addresses the problem of catastrophic forgetting in noural networks. EWC works by calculating the importance of cach parameter to the old tasks and then applying a regularization term that penalizes changes to the most crucial parameters. The 'importance" quantified using the Fisher Information Matrix, which captures how sensitive the model's predictions are changes in each parameter. When the model is trained on new task, parameters important for past tasks are kept relatively stable, while less important parameters method is inspired by principles of neuroplasticity in the human brain, where certain synaptic pathways are consolidated and become less plastic after learning, preserving essential knowledge.
more free to change. This
ER combats forgetting by intermittently retraining the model on a random subset previously seen examples. This approach mimics the way humans recall and reinforce knowledge over time by revisiting past experiences. In practice, ER maintains memory buffer that stores a collection of past data samples. During the training of new lasks, the learning algorithm interleaves the new data with samples drawn from this memory buffer. By doing so, the model is regularly reminded of the old tasks while it learns the new ones, which helps to maintain its performance across all tasks. The process of experience replay is simple yet effective way to preserve old knowledge in the neural network without compromising the learning of new information.
GEM is a strategy designed to mitigate forgetting by leveraging experiences stored in a memory. GEM maintains a subsct of the data from previously encounterod tasks an episodic memory and uses this stored data to guide the learning process for new tasks. When a new task is introduced, GEM compares the gradients of the loss with respect to the current task and the gradients concerning the tasks stored in
19

---

Published in Transactions on Machine Learning Research (01/2025)
memory. It then adjusts the update rule to ensure that the loss of the episodic memory does not increase, effectively preventing the network from unlearning the previous tasks. This is achieved through constrained optimization problem that allows the model to learn the new task while not getting worse on the tasks stored in memory.
PNN is an architectural approach to overcoming the challenges catastrophic forgetting in machine learning, particularly when dealing with sequential or multitask learning scenarios. By architecturally encapsulating knowledge gained from previous tasks, PNNs enable the integration of new information without overwriting what has been previously learned. This is achieved through the use of separate neural network columns for each task, where each new column is connected to all previous ones via lateral connections. These connections allow the networks to access previously learned features, making PNNs highly effective for tasks requiring the retention and transfer of knowledge across different domains.
C.2 Adversarial perturbations
The Fast Gradient Sign Method (FGSM) attack used for crafting our adversarial perturbations is method for generating adversarial examples based on the gradients of the neural network. perturbs an original input by adding noise that is determined by the sign of the input's 's gradient with respect to the neural network's 's parameters. This method designed to be quick and effective, typically requiring only one step to create an adversarial example.
To elaborate, the FGSM lakes the original input x, the target label y, and the model parameters 0, and then it computes the gradient of the loss with respect to æ. The noise is then generated by taking the sign of this gradient and multiplying it by a small factor, т, which controls the intensity of the perturbation. The adversarial example is created by simply adding this perturbation to the original input:
= sign(V sign(V 2x,,,,
where V, x, v) the gradient of the model's loss with respect to the input, 0 represents the model's parameters, and is small constant. FGSM can be used to reveal the vulnerabilities of a neural network and is common benchmark for evaluating the robustness of machine learning models against adversarial attacks.
D Training and Hyperparameters
Training Hyperparametters
Value
Optimizer
SGD
Learning rate
lo-4
Momentum
0.9
Early stopping patience
10
Batch size
100
Training steps
130k (ENABL3) 13k (Embry) / 1.3k (Amputees)
Max. rehearsal buffer size
3k
Tusk balancing
yes
Table 8: Hyperparameters used to train our model
Rehearsal size To ensure balanced comparison, we maintained an identical rehearsal size for the conventional rehearsal, prospective rehearsal, and Gaussian noise augmentation approaches. This measure was taken to avoid attributing any performance improvement to data volume inflation, which could result from augmentation. By sampling an equivalent quantity of data for both conventional and prospective rehearsal, we aimed to neutralize the potential impact of having more or less data on performance outcomes.
Rehearsal sampling stratogy We employ a taskebalanced reservoir sampling strategy for rehearsal, which ensures that samples from each task are maintained in balanced manner over time. Specifically, this approach allows us to continuously update the rehearsal buffer with representative samples from all tasks,
20

---

Published in Transactions on Machine Learning Research (01/2025)
without favoring recent tasks over older ones. During each minibatch update, balanced subset of samples from all tasks is used to ensure that the model's performance does not degrade earlier tasks. This method effectively mitigates the issue of bias towards more recent tasks. Moreover, our strategy aligns with well-cstablished practices in continual learning. where task- balanced replay buffers are commonly used to prevent calastrophic forgetting Merlin et al. (2022): Krawezyk & Gepperth (2024)).
The hyperparameters employed for training the model detailed in Table
E
Broader Impact
The strategies introducod in this paper have broad implications across human-contered assistive technolo- gies, intelligent interfaces, and autonomous systems. Beyond prosthetics, this framework improve the traversability of autonomous agents, such legged robots Dey et al. (2022); Morrell et al. (2024) and self-driving vehicles Schmid et al. (2022), by enabling them to predict and adapt dynamic environments. Furthermore, the ability to anticipate and account for the prediction errors is valuable for rehabilitation robotics and personalized healtheare solutions, offering more responsive and robust systems for patient care. Additionally, these modeling techniques may be adapted for industrial robotics to enhance safety human-robot collaboration and for leleoperation scenarios that demand precise movement coordination. Taken together, this research represents step toward resilient, human-centrie Al technologies with wide-reaching socictal benefits.
21


Reviews:
[{"review_id": "Kp7eQOHriZ", "reviewer": "Reviewer_EQtz", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths：\n- The introduction of “multitask prospective rehearsal” is a key innovation, enabling real-time gait prediction without the need for retraining on new data.\n- Real-world deployment with an ankle exoskeleton validates the model’s practical impact, suggesting it could significantly improve quality of life for individuals with mobility impairments.\n- The paper includes ablation studies, stability analysis, and comparisons with various learning strategies, providing a strong validation of the model’s capabilities.\n\nWeakness:\n- I found that despite claims of practical applicability, the model has not been subjected to comprehensive clinical testing with real-world patients outside of lab conditions. This limitation calls into question the reliability and generalizability of the results. Without evidence from prolonged use in uncontrolled environments (such as daily life scenarios), it is unclear if the model could genuinely improve quality of life or withstand the demands of real-world wear and tear.\n- The model relies on high-resolution motion capture and wearable sensor data, which are generally unavailable or impractical for typical clinical or personal use settings. The requirement for continuous, high-quality data acquisition adds significant complexity and cost, potentially limiting its practical deployment. Additionally, the model’s robustness to data artifacts, sensor misalignment, or low-quality sensor inputs is inadequately addressed, a notable oversight given real-world variability in sensor data.\n- Despite claims that prospective rehearsal mitigates forgetting, the paper offers insufficient experimental evidence on how well the model truly retains knowledge over prolonged periods with changing tasks. Catastrophic forgetting remains a well-documented challenge in multitask learning, and without rigorous, long-term evaluations, the risk of task-specific knowledge erosion remains unaddressed.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper presents a multitask, continually adaptive model for human gait prediction, targeting the control of bionic prosthetics, particularly for transtibial amputees. The model utilizes “multitask prospective rehearsal,” a novel approach that anticipates future movements, corrects errors incrementally, and adapts across varied gait patterns and locomotion tasks. Experiments demonstrate the model’s superior performance over baseline models, especially under conditions of distributional shifts and adversarial perturbations.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "- please provide why high-resolution motion capture and wearable sensor data are available  for typical clinical or personal use settings\n- please clarify why simpler architectures will not achieve similar or sufficient accuracy for practical purposes, given that the proposed architecture is so complex", "broader_impact_concerns": "no", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "O0IIa3eesL", "reviewer": "Reviewer_Eak7", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "**Strengths**: \n\n1. The authors introduce a novel multitask prospective rehearsal pipeline, allowing it to anticipate and adjust for future trajectories, improving adaptation over time for locomotion predictions.\n\n2. The proposed method synthesizes future states, effectively generating new ‘unseen’ data for training, which helps enhance generalization capabilities when faced with new conditions.\n\n3. This is the first study to tackle multi-gait adaptation in bionic prostheses using an error-aware multitask continual adaptation framework, offering new insights in the field.\n\n**Weaknesses**:\n\n1. **Catastrophic Forgetting in Continual Learning**: the proposed method to task-specific adaptation involves developing a new task-specific prediction layer for each new task while training both the shared backbone and task-specific layers using only the new task data. This is inherently prone to catastrophic forgetting, where learning new tasks causes the model to forget previously learned tasks. However, even with the rehearsal approach (including data from previous tasks), the model may still lose performance on earlier tasks due to insufficient rehearsal samples or overfitting to new tasks. The evolving architecture could also make it difficult to balance learning between new and old tasks, especially as the number of tasks increases. I suggest that the authors conduct ablation studies to analyze the performance drop on each task, which will help to better understand this issue.\n\n2. **Rehearsal Approach and Data Augmentation**: the method augments rehearsal data using prediction errors generated by the current model. However, this method may introduce biased training samples that might not accurately reflect the true task distribution. By relying on augmented samples based on the model’s own predictions, the model could end up reinforcing its existing errors. This could lead to a divergence between the model’s predictions and actual human demonstrations, especially if the initial errors are significant or poorly handled. The ablation studies should be designed to address concerns about the accumulation of errors from the previous model's augmentation.\n\n3. **Impact of Prediction Errors on Temporal Dynamics**: the approach recognizes that prediction errors at one time step can affect subsequent time steps, causing deviations from the true data distribution. This is a fundamental problem in time-series prediction and imitation learning, where small errors can compound over time. While this issue is well-documented, the method does not provide a robust solution for mitigating the accumulation of errors over time. It mentions relaxing error bounds to Lipschitz bounds, but this can lead to exponential growth in deviations between the predicted and desired time series. Without addressing how to control or correct these deviations effectively, the method may struggle with long-term stability. Meanwhile, Theorem 1 is not clearly defined, and its proof lacks rigor. A proper theoretical proof should include a thorough mathematical derivation that clearly connects the assumptions to the conclusion.\n\n4. **Assumptions of Lipschitz Bounds in Prediction Errors**: the discussion suggests that constant error bounds are unrealistic in continuous spaces and instead considers Lipschitz bounds for the errors. However, this may lead to exponential divergence between predicted and actual outcomes over time. The assumption of Lipschitz continuity implies that small changes in input can lead to proportional changes in output, but in a time-series context, this means that errors can amplify exponentially. Without a clear strategy for managing this, the model’s predictions may become increasingly inaccurate, limiting its applicability to longer sequences.\n\n5. **Potential Scalability Issues with Task-Specific Layers**: the proposed framework developed new task-specific layers for each task, which may lead to scalability issues as the number of tasks grows. If the number of tasks increases significantly, the model size and complexity could grow, leading to higher computational and memory demands. This could make the model less practical for real-world applications where new tasks are frequently introduced. Thus, I suggest conducting an ablation study that demonstrates the performance of each task over time, rather than just focusing on the overall performance across the entire dataset.\n\n6. **Unclear Strategy for Task Sampling in Rehearsal**: the authors mentions including samples from previous tasks during training for rehearsal but does not clarify how the task samples are selected or balanced in the training process. The effectiveness of rehearsal in preventing catastrophic forgetting depends on how representative the rehearsal samples are of prior tasks. Without a clear sampling strategy, the rehearsal process might be biased towards more recent tasks, reducing its ability to preserve performance on earlier tasks.\n\n7. **No Comparison with The Existing Work**: there is no comparison with relevant existing work, such as \"Ensemble diverse hypotheses and knowledge distillation for unsupervised cross-subject adaptation\" (Information Fusion, 2023). Given that the authors conducted all experiments on widely used open-source datasets like ENABL3S, it is important to include comparisons with other studies that address the same topic.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The authors present a multitask, continually adaptive gait synergy approximation model that adapts to various locomotion tasks and evolving gait patterns by integrating prediction errors into its learning process. The key insight of the proposed work is the multitask prospective rehearsal technique, which allows the model to anticipate and adjust for potential future trajectories, enhancing the connection between continual adaptation and error integration. The model generates synthetic future states, generating 'unseen' data from previous task model to improve its generalization capability. The proposed work is evaluated on extensive experiments across continual and joint learning settings on three real-world gait datasets, including a unique transtibial amputee dataset. This study is the first to address multi-gait adaptation in bionic prostheses using an error-aware multitask continual adaptation framework.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "1.Conduct Ablation Studies to Address Concerns on Catastrophic Forgetting (Weakness 1)\n\n2. Design Ablation Studies to Address Concerns on the Bias in Rehearsal Data Augmentation (Weakness 2)\n\n3. Provide a Robust Solution and Ablation Studies for Error Accumulation in Temporal Dynamics (Weakness 3)\n\n4. Clarify and Manage Assumptions of Lipschitz Bounds (Weakness 4)\n\n5. Address Scalability Issues with Task-Specific Layers (Weakness 5)\n\n6. Clarify the Strategy for Task Sampling in Rehearsal (Weakness 6)\n\n7. Include Comparisons with Existing Work (Weakness 7)", "broader_impact_concerns": "I am concerned about the practical value of the proposed framework, as it does not seem to address key challenges like imbalance or fall risks. Moreover, its performance on public datasets is not competitive compared to existing methods.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "CbHR6TiVFr", "reviewer": "Reviewer_t3mt", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "[Strengths]   \n1. The paper tackles an important and practical problem in human behavior modeling for bionic prosthesis control, which has implications for machine learning applications in healthcare and real-world prediction problems.\n2. The proposed method, multitask prospective rehearsal, introduces a reasonable approach for continual learning, enhancing experience replay with prospective data generation.\n3. The experimental results clearly demonstrate the effectiveness of the approach, showing strong performance across multiple benchmarks.\n\n[Weaknesses]    \n1. The proof of Theory 1b is only provided for a very specific case ($x_1 = x_2 = … = x_N = 0,  g(x,y) = x+y$). This does not constitute a strict proof. The authors should either generalize the proof or reframe Theory 1b to be more specific and avoid overly general terms.  \n2. It is unclear why the prediction problem in the paper was not trained using a roll-out strategy or multi-step prediction, similar to model-predictive control or standard time-series prediction methods. It is better to try a roll-out strategy or provide reasons for not using it.  For example, in Algorithm 1, the loss functions Lf and Lg do not incorporate a roll-out mechanism for multi-step prediction, which is commonly used to reduce prediction errors.  As an example  (where $\\hat{x}$ and $\\hat{y}$ is predicted value. )  \n$$\nL_g^{rollout} = \\sum_{k=1}^{M_t-1} \\left\\| x_{t, k+1} - g_t \\left( x_{t, k}, y_{t, k} \\right) \\right\\|^2 + \\sum_{i=1}^{m} \\sum_{k=1}^{M_t-1} \\left\\| {x_{t, k+i+1}}   - g_t \\left( \\hat{x_{t, k+i}}, \\hat{y_{t, k+i}} \\right)   \\right\\|^2 $$\n\n3. Some relevant works are missing from the discussion. The replay strategy with augmentation in this paper is more akin to generative replay (see [1, 2]) than traditional experience replay. Additionally, selective sampling strategies (see [3]) have been proposed to enhance performance in rehearsal-based methods. These works should be discussed.\n\n[1] Shin, Hanul, et al. \"Continual learning with deep generative replay.\" Advances in neural information processing systems, 2017.  \n[2] Van de Ven, et al. \"Brain-inspired replay for continual learning with artificial neural networks.\" Nature communications, 2020.  \n[3] Yin, Peng, et al. \"Bioslam: A bioinspired lifelong memory system for general place recognition.\" IEEE Transactions on Robotics, 2023  .", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The paper studies the human behavior modeling in bionic prosthesis control, focusing on a multitask, continually adaptive framework.   The authors propose a \"multitask prospective rehearsal\" approach to handle multitask learning and continual adaptation while mitigating distribution shifts between training and testing samples. The method relies on a rehearsal strategy similar to experience replay or generative experience replay in continual learning. The model features both a shared backbone and task-specific modules for different tasks. A key contribution is the rehearsal buffer, which contains both real and prospective samples generated by a task-specific model. Experimental results demonstrate that the proposed method outperforms previous baselines, particularly in dealing with distribution shifts, noise, and adversarial attacks.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "1.\tProvide a more general proof or rewrite Theory 1b to better reflect the specific scenario discussed.  \n2.\tExplain why the prediction problem does not use a roll-out strategy for future m-step predictions to reduce error, as is common in time-series prediction tasks.  It is better to try a roll-out strategy or provide reasons for not using it. \n3.\tDiscuss related works on generative replay and selective sampling strategies to provide a more comprehensive background. \n  \nPlease refer to the details in the \"Weaknesses\" section for specifics.", "broader_impact_concerns": "It would be beneficial for the authors to include a Broader Impact Statement discussing the ethical implications of applying machine learning models in medical prosthetics, with particular focus on patient safety and data privacy.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: The paper presents a multitask, continually adaptive model for human gait prediction in bionic prosthesis control, introducing the novel concept of multitask prospective rehearsal. While the reviewers raised concerns about limited novelty, practical validation, and competitiveness on public datasets, the authors addressed all major technical issues and provided additional experimental evidence to support their claims. The work is technically sound, aligns with TMLR’s standards for correctness over innovation, and addresses a relevant problem in a niche area. Given the reviewers’ consensus that the paper is of interest to the community and technically valid, I recommend acceptance.
Decision: Accept (poster)