Exo-Plore: Exploring Exoskeleton Control Space through Human-aligned Simulation
Authors: Geonho Leem
Jaedong Lee
Jehee Lee
Seungmoon Song
Jungdam Won
Venue: ICLR 2026 Poster (2026)
Keywords: Deep reinforcement learning; Musculoskeletal simulation; Pathological gait generalization; Sim-to-real matching
Primary area: applications to robotics, autonomy, planning
TL;DR: A Deep RL-based musculoskeletal simulation framework that optimizes exoskeleton parameters to reduce human metabolic cost.

Abstract: Exoskeletons show great promise for enhancing mobility, but providing appropriate assistance remains challenging due to the complexity of human adaptation to external forces. Current state-of-the-art approaches for optimizing exoskeleton controllers require extensive human experiments in which participants must walk for hours, creating a paradox: those who could benefit most from exoskeleton assistance, such as individuals with mobility impairments, are rarely able to participate in such demanding procedures. We present Exo-plore, a simulation framework that combines neuromechanical simulation with deep reinforcement learning to optimize hip exoskeleton assistance without requiring real human experiments. Exo-plore can (1) generate realistic gait data that captures human adaptation to assistive forces, (2) produce reliable optimization results despite the stochastic nature of human gait, and (3) generalize to pathological gaits, showing strong linear relationships between pathology severity and optimal assistance.

PXORE EXPLORING EXOSKELETON CONTROL SPACE THROUGH HUMAN-ALIGNED SIMULATION
Geonho Leem', Jaedong Lee", Jehee Lee", Scungmoon Song", Jungdam Won'! 'Seoul National University, "Holiday Robotics, NNrthheastern University
ABSTRACT
Exoskeletons show great promise for enhancing mobility, but providing appro- priate assistance remains challenging due to the complexity of human adaptation external forces. Current state-of-the-art approaches for optimizing exoskeleton controllers require extensive human experiments in which participants must walk for hours, creating paradox: those who could benefit most from exoskeleton assistance, such as individuals with mobility impairments, are rarely able to par- ticipate such demanding procedures. We present Exo-plore, simulation frame- work that combines neuromechanical simulation with deep reinforcement learning to optimize hip exoskeleton assistance without requiring real human experiments. Exo-plore can (1) generate realistic gait data that captures human adaptation assistive forces, (2) produce reliable optimization results despite the stochastic nature of human gait, and (3) generalize to pathological gaits, showing strong linear relationships between pathology severity and optimal assistance".
INTRODUCTION
Human mobility is fundamental to meeting basic life needs. While exoskeleton research shows promise for enhancing mobility (Molinaro et al. 2024 Slade al., 2022]. providing appropriate assistance, especially for users with atypical gait patterns, remains challenge (Kim et al.) [024] Kang et al. 2025). Users alter their movement patterns and redistribute muscle coordination in re- sponse assistance (Song & Collins 202I), which can erode the benefits predicted by fixed-gait assumptions (Dembia et al. 2017). To personalize assistance, researchers have employed human- in-the-loop optimization (HILO) techniques, which constrain the exoskeleton controller design to small set of tunable parameters (e.g.. peak torque timing, magnitude, and duration) and systemati- cally explore this parameter space through human experiments (Zhang et al., 2017). While HILO has proven effective for able-bodied young and older adults (Slade et al. [022]] Lakmazaheri et al., 2024]. it requires demanding experiments where participants must walk for hours while wearing an exoskeleton. This situation presents paradox: those who would benefit most from exoskele- ton assistance, such as individuals with mobility impairments, are rarely able to participate in these extensive optimization procedures.
Neuromechanical simulations (Song et al. 2021) that couple musculoskeletal dynamics with control policies predict how humans adapt assistance have been proposed as a promising complement to exhaustive HILO (Luo et al. 2024 Firouzi et al. 2025 Tan et al. 2025). Yet policy design re- mains bottleneck for reliable prediction. Biologically ly inspired controllers optimized with task cost functions can generate human-like gait in certain scenarios, but application and validation for assis- tive devices and pathological gaits remain limited (Song & Geyer, 2017 Geijtenbeek 2019 Bersani et al. 2023). Deep reinforcement learning (Deep RI.) offers an alternative that is less constrained by hand-specified control assumptions and potentially more generalizable, but most implementa- tions that produce human-like motions rely on tracking reference
data to manage large observation and action spaces (Lee et al., 2019 Song et al.] 2021 Park et al., 2025). ) Pure prediction is inherently challenging; practical approaches likely require both fitting controllers to existing data and predicting responses in extended scenarios Park et al. (2022). However, there is no unified framework (i) fit controllers to observed human adaptations and (ii) predict responses in unobserved assistive conditions, particularly for exoskeleton assistance in gait pathology.
"Correspondngg author (jun (junedam@1 anu..a...k... "ooee and videos available at https://daebanggtta github. 1o/exo-plore

---

In this paper, we present Exo-plore, swuurommchaniall-smmaaaaa framework that discovers hip oskeleton control parameters by coupling a Deep RL gait data generator with a stochasticity- aware surrogate optimizer. The gait data generator is tuned to fit human experimental trends by match- ing assistive moment/power scaling across assistance settings and walking speeds. This is achieved through reward designed align with empirical adaptation patterns, which combines metabolic energy minimization with hypothesis. As a result, the generator adapts across hip-assistance levels and musculoskeletal im- pairments, producing plausible assisted gaits in both healthy and pathological settings that align with empirical trends. We then train a surrogate network on abundant simulation data from the gait data generator to obtain smooth, differentiable CoT (cost of transport) landscapes to stabilize op- timization under RL and simulation stochasticity. Using Exo-plore, we (i) reproduce assisted and unassisted gait trends consistent with human experiments, including assistive moment/power scal- ing and realistic metabolic reduction rates; (ii) identify speed-dependel optimal exoskeleton control parameters for able-bodied gait (optimal torque delay decreases speed increases); and (iii) gener- alize to pathological gaits, revealing strong linear relations between pathology severity and optimal assistance in four of five conditions.
term based on resistance minimization
2 RELATED WORK
Human-in-the-lopp optimization (HILO) for exoskeletons. Exoskeletons have demonstrated the ability to provide various benefits, including improved walking speed and reduced metabolic energy consumption (Molinaro et al. 2024: Slade et al.) 2022 Lakmazaheri et al. 2024). However, their effectiveness varies significantly depending on control parameter settings and users' adapta- tion to assistance (Poggensee & Collins 2021). HILO methods have been widely used to address both challenges by iteratively optimizing controllers based on real human responses and by evalu- ating candidate parameters using metrics such as metabolic cost of transport (CoT) after adapta- tion (Zhang et al., 2017 Slade et al.] 2024] Wiitte et al.] 2020 Bryan et al.) 2021 Song & Collins) 2021]. In addition to CoT optimization, several studies have also optimized parameters based on user preference or perceived comfort (Ingraham et al.) 2022: Lee et al. 2023 However, because experiments involved human participants, the number of iterations that can be performed to update parameters is often limited to fewer than 30 iterations, even when using auxiliary techniques such as online simulation (Gordon et al., 2022). Furthermore, individuals who might benefit most from exoskeleton assistance (e.g.. mobility-impaire individuals) often show limited tolerance to this op- timization process (Welker et al.] 2021 Lakmazaheri et al. 2024), making the application of HII.O often infeasible for these groups.
Neuromechanical simulation. for studying human movement, assistive devices, and neuromuscular impairments by coupling multibody musculoskeletal models with Hill-type muscle actuators and a control policy that drives muscle-driven dynamics (Song et al. 2021). Despite advances in modeling and simulation fidelity, predictive capability hinges on the control model, which remains only partially understood and diffi- cult to specify. Existing approaches either (i) treat control black box to reproduce target motions, via trajectory optimization (De Groote & Falisse 2021) or Deep RI. (Song et al.] 2021) or (ii) rely on hand-crafted, biologically inspired controllers tuned for habitual gait (Song & Geyer 2015) Bersani et al. (003).. These strategies can yield plausible motions but often face limits in general- ization and computational efficiency, especially when predicting adaptations to novel exoskeleton assistance or pathological conditions.
Neuromechanical simulations provide physics-based digital twins
Deep RL-based Musculoskeletal Control. Deep RL has shown promising results in neurome- chanical simulation, but raises open questions about physiological fidelity and generalization (Lee) ct al. 2019). Recent studies use Deep RL within neuromechanical simulations explore potential surgical effects, simulate gait impairments arising from altered muscle function, and synthesize di- verse movements (Lee et al. 2019 Park et al. 2022 2023. 2025). However, the sim-to-real gap remains largely unvalidated for muscle activations, metabolic energy, and adaptability to external assistance, which are critical for translational impact. While Luo et al. (2024) applied Deep RL to exoskeleton control, their formulation depends on imitation policies, limiting adaptation to unseen conditions. As a result, the simulation results do not align with natural human movement, and their correlation with real experimental outcomes has not been validated. In contrast, Exo- Exo-plore fits to established experimental trends and predicts responses in unseen conditions, enabling estimation of biomechanical metrics essential for exoskeleton applications.
2

---

3 METHOD: THE EXO-PLORE FRAMEWORK
Figure presents overview of our Exo-plore framework, which operates musculoskcletal character equipped with a hip exoskeleton (Lim et al. 2019a). The exoskeleton controller generates hip assistive torque Texa (t) as follows:
Texo (t) = K. ult At)
where u(t) = sin(0,) (6t) ) control signal encoding relative motion of the two legs. 8,, O denote the low-pass filtered right and left hip joint angles, respectively. The sine function bounds the joint angle signal to the range [-1, IJ. & and Al are control parameters, where the gain x scales the torque magnitude while the delay At introduces temporal lag for the output (i.e., delayed- feedback control). Since the assistive torque is proportional to the relative motion between the two hip joints, effectively acts stiffness parameter. This formulation therefore resembles simplified impedance-control mechanism.
The objective of Exo-plore is to find the optimal control parameters (K, At) that minimize the sim- ulated character's metabolic cost-of-transpor during walking. This is achieved through two com- ponents: the gait data generator, which produces gait data that align with real human experiments, and the exoskeleton optimizer, which efficiently explores the control parameter space to identify the optimal parameters.
Deep RL
Matching
Gait lmsparncy)
Musculoskeletal Sim.
Muselt parameters wcalace)
Controller
Rollouts (Smpling)
Surrogate Netwock
Controller
Optimal exeskeleton parameters ogen, deay)
Gendien--rsse Opimizztion
Gait Data Generator
Exoskeleton optimizer
Figure 1: The Exo-plore framework. The generator produces gait trajectories under hip exoskele- ton assistance using a musculoskeletal character with 164 museles. Through our sim-to-real match- ing approach, these trajectories replicate the adaptation patterns and metabolic responses observed in real human experiments. The generated data are then used train a surrogate network, which enables efficient and customizable optimization of exoskeleton control parameters.
3.1 GAIT DATA GENERATOR
The human controller in the gait data generator consists of three modules: pose network (PoseNet), which computes cositional-derivatii (PD) target joint positions qd: PD controller that generates joint torques TPD reduce the error between the target and current joint positions; then compensate the exoskeleton assistance Texa from TPD to compute the torque to be generated by the muscles TMON; and a muscle coordination network (MCN), which maps the computed target joint torques TMON to muscle activations These activations are then applied to the underlying musculoskeletal dynamics, and the states the musculoskeletal character and the exoskeleton are updated accord- ingly. The PoseNet and MCN are jointly trained during the iterative simulation process using Deep RL and supervised learning, respectively, while a stable PD is used for our PD control (Tan et al. 2011) The detailed architecture of the human controller is provided in Appendix A
To train the PoseNet, we follow the formulation of Generative GaitNet (Park et al. 2022 but we extend the state, action, and reward to incorporate the exoskeleton and improve motion naturalness. Here, describe only the crucial components, including
extensions, while other technical de- tails are provided in Appendix B The state is defined as s = (Sakeleton: Smusele: Seso). where Sskeleton and Smusele represent the physical states of the musculoskeletal character, and 8exo denotes the ex- oskeleton state. The total reward is defined as
Total
Wgait
Tgait Warm -Tamm + Wenergy Tenargy + WHEI THEI
3

---

where "gait encourages the character to follow the given target gait, and is defined as the product of four components as "gait = Tsep Tvel Thead Tssaay where Taep encourages the specified step length, Tvel encourages the specified walking speed, and Thead and Tsway stabilize the head and body sway, respectively. To prevent unnatural arm movements, an imitation reward Tamn is introduced. The energy reward, Femergy 1 kenergy MEF., regularizes energy expenditure during walking, where MEE denotes the metabolic energy expenditure, computed as follows:
d MEE(a; a. B)= Σ m2 dt
where m, and ag are the mass and activation of the i-th muscle, respectively, and a, 3 are tunable design parameters. This allows flexible adjustment between effort-like (a-dominant) and fatigue- like (B-dominant) behaviors, in line with previous studies (Ackermann & Van den Bogert 2010)) These are crucial parameters that must be properly adjusted to ensure that the simulation results align with those from real human experiments. The detailed design of a and B3 is presented in Section 3.2
To account for human adaptation during exoskeletal assistance, we introduce novel reward THEI that explicitly models physical human- ceoselexoo interaction (HEI). In our framework, THEI plays a critical role in bridging the sim-to-real gap under exoskeleton assistance. This reward is also further discussed in Section 3.2
The MCN is trained in a supervised manner to learn the mapping from desired joint torque Ta to muscle activations a. The total loss is computed as
LMCN
TMCN
-(a)l Wreg |a|? + WINR LIMR-
(4)
The first term guides the generated torque T(a) from the output muscle activations match the desired torque TMCN. while the second term penalizes excessively large activations. Please refer to (Lee al. 2019) for more technical details. Large muscles such as the gluteus medius and soleus are represented by multiple line muscles. Controlling each line muscle independently could lead to physiologically implausible results. To prevent this, we introduce the final term, the intra-muscular regularizer (IMR), which enforces coherent activations among the line muscles belonging to the same anatomical musele group. Specifically, the IMR penalizes deviations from the mean group activation, which is computed as follows:
N
LIMR Σ
E
-0,1 > 0.1) (a, -a,)²² a
1 aj.
(5)
9=1 JEG
jEC,
where ay denotes the activation of the j-th musele in musele group g. and ag is its corresponding mean group activation. The number of muscle groups and the set of indices of the line muscles in muscle group are denoted by Ng and Gg. respectively. For details on LIMR, see Appendix C
3.2 SIM-TO-REAL MATCHING IN GAIT DATA GENERATOR
The simulation results generated by the Gait data generator need align with real human experimen- tal results to ensure that the control parameters optimized in simulation are applicable to real- scenarios, a process often referred to as bridging the sim-to-real gap. To this end, we focus on repro- ducing two core human behaviors observed in several biomechanics studies: the principle of energy minimization and the active adaptation to exoskeleton assistance.
First, many studies have observed that human walking tends toward an energetically optimal gait (Umberger & Martin, 2007]] Zarrugh et al. 1974). This means that humans adjust their step size and step frequency at each walking speed to minimize the metabolic cost-of-transport (CoT), which is the metabolic energy expenditure per unit travel distance. As reported in previous stud- ies (Maxwell Donelan et al., (200T), measured CoT values across different walking speeds form an upward-opening parabola, and humans tend to select the minimum point their preferred walking speed (PWS) under nominal conditions. Based on this, we determine the MEE parameters (a, 39) in Equation 3 so that the resulting Walking speed CoT curve and the corresponding PWS best match those reported in real experiments. For this, we develop two algorithms (Algorithm I and 2). The first algorithm assumes feasible values of (a, B) (Line 2). trains gait data generator to compute the PWS (Line 4), and selects the optimal values of (a, 33 that best match real human PWS (Line 5-8). The second algorithm performs rollouts of gait trajectories across feasible parameters (Line
4

---

2-3), calculates the CoT for each, and determines the PWS that yields the minimum CoT (Line 4-7).
We define two procedures used algorithm and
trainGenerator. This procedure trains a human controller conditioned on a given metabolic en- engy model Fm:
G = trainCenerator((nl))
(6)
During this training process, the simulated musculoskeletal character does not an exoskeleton, as we evaluate PWS in the absence of external assistance. Consequently, PoseNet does not use the exoskelcton-related imput THEI-
rollout. This procedure generates gait trajectory from the trained gait data generator G for given gait parameters (L, f):
Teye rollout(G(L,
To ensure stable trajectory generation, we discard the first five gait cycles after the simulation starts and use the subsequent ten cycles to construct the trajectory.
Based those algorithms, the optimal parameters for our framework were determined (a, (3) (1.5, 1.0). A detailed technical discussion is provided in the Appendix D
Algorithm 1 Adjusting the MEE to make Exo- plore to predict human PWS.
Algorithm (evalPWS) evaluating PWS for gait
data generator
Notation: muscle mass, activation mi 0j
Notation:
gait step length L, gait step frequency
Require: Human PWS Ureal
f. gait cycle trajectory Teyc
Ensure: MEE parameter (a", 3*)
Require: Data generator G, metabolic energy ex- penditure Em
1: emin + +oc, (a', 8²) + Ø
2: for (a, B) candidate do
Ensure: Usim (PWS of G)
3: Em + MEE(a, B)
CoT +o,, Usim < Ø
4: G; + trainGeneratoo((Enn
2: for (L, f) E available do
5: Vsim + evalpMs(G,, Em)
3: Teye + collout(G(L, f))
6:
Ureal Usim
4: CoT = + dem
7: if e Emin then
5: if CoT < CoT then
8: Emin t e, (a", 8*) + (a, 3)
6: CoT f CoT, Teim + (Lxf)
9: end if
7: end if
10:
end for
8: end for
Second, experimental studies suggest that humans do not simply adapt to the assistance torque Nor- mand al. 2023 Livolsi et al. (2025). For example, at the same walking speed, increasing the delay parameter (At from 0.15 s to 0.25 results in a 1.5-fold increase in RMS assistive torque (Figure 6 The RMS assistive torque proportional the hip joint angle (Fquation T}. This indicates that hu- mans actively modify their kinematics, such as increasing step size, to further utilize the assistance.
To reproduce this adaptive behavior, we develop the hypothesis, which reflects the human tendency to avoid mechani-
interaction (HEI) reward
based on a ciiistance-mnimmizati cal power loss. This is inspired by the behavioral principle of loss aversion, which states that people are more sensitive to losses than to equivalent profits (Kahneman) 1979 Tversky & Kahneman
1991 Brown et al. 2024). Our HEI reward is formulated as follows:
1
THEI 1+
Σ KE(L.R)
min
where Pk denotes the mechanical power applied by the exoskeleton on either the left or right side, and is the gain parameter of the exoskeleton controller. Equation 8 Stakes effect when the exoskele- ton applies resistive power to the human (P& < 0), causing ThEI to drop below 1.
5

---

3.3 EXOSKELETON OPTIMIZER
The goal of the exoskeleton optimizer is identify exoskeleton control parameters that minimize COT, meaning the parameters that provide the most effective assistance to humans wearing the ex- oskeleton. Because the mapping between the control parameters and their resulting CoT often highly non-linear, previous methods (HILO) typically relies on gradient-free and sampling- -ased optimization methods such as CMA-ES or Bayesian optimization (BO) (Slade et al. 2024 Gordon al. )022). These methods are particularly effective in HILOO settings, where data collection is limited and expensive. For instance, BО often employs a Gaussian process (GP) as surrogate model to approximate the CoT and guide the search more efffccinntly. However, in simulation- based framework, where data can be generated as needed with low cost, the assumptions underlying such sample-efficient methods no longer apply. While GP offer excellent performance in low-data regimes, their poor scalability in both time and memory complexity makes them unsuitable for our framework (see Appendix E). To take advantage of the data-rich simulation environment, we stead use neural network surrogate based on multilayer perceptron (MLP). Neural networks scale efficiently with data and support fast, gradient- gradient-based optimization, making them a better for Exo-plore.
Surrogate networks trained on naive uniform grid-sampled parameter set may exhibit arti- facts due to the regularity of the sampling pat- terns. In particular, grid-based sampling may lead to aliasing effects and lacks the stochastic variation. To mitigate these issues, we instead use Latin hypercube sampling (LHS) (Forrester eet al.) 2008). which stratifies each parameter range and draws one sample per interval. This stratification reduces aliasing and ensures uni- form coverage without the clustering of random sampling. Please refer to Appendix F for abla- tion studies.
Algorithm Optimizing exoskeleton control pa- rameters
Require: Simulation parameter space X, Data generator G, Target gait parameter g
Ensure: Optimal control parameter c*
1: B + 2
2: for x; E LHS(X)) do
3: Teyn f- ro rollout(G(L, S
4:
+ L S Trye dFm
5: B B ( y)}
6: end for
Algorihmmooutlins the overall process of opti- mizing the control parameters: For the simula- tion parameter space X' used in training the gait data generator, we perform three steps: sam- pling candidate parameters and their evaluation through simulation (Line 2-5), training surrogate network (Line 6), and performing gradient- optimization to minimize CoT given a target gait (Line 7).
7: fo(c) + fit(B)
8: c* + arg min fo(c; g)
9:
gradient-based optimization
To train the surrogate network, we minimize the following loss:
Suunogan:- = Cs,, (ū. 8) + Agrad [ V ALI Σ l00/ + AL2 Σ w
(9)
where ý is the predicted CoT value from the surrogate network, y is the ground truth value ob- tained from simulation. We used Huber loss (Ls,) ) (Hastie et al.] 2009) to reduce the effect of outliers that may arise from the instability and stochasticity of the simulation. The gradient penalty (Vağ) encourages smoothness of the surrogate landscape by constraining the norm of the output gradient with respect to input (Appendix G In addition, the LI and 1.2 weight penalties act as standard regularizers. This formulation enables the trained surrogate network to generalize well in predicting CoT values for unseen control parameters while ensuring stable gradient behavior. For gradient-based optimization, we use sequential least-squares quadratic programming (SLSQP) and trust-region algorithms. Please refer to Appendix for the optimization objective.
4 RESULTS
4.1 SIMULATION SETUP
Simulation model. We the musculoskeletal model from prior work (Lee et al., 2019: Park jet al. 2022), which consists of 23 bones and 164 muscles. The bones are simulated as rigid bodies, and the muscles modeled as Hill-type muscle actuators (Hill 1938). The simulation is performed

---

vary widely with setup. For discussion, please refer in the DART physies engine (Lee et al. 2018). In this model, tendons represented rigid rather than compliant to reduce the computational burden induced by Deep RL. To further reduce computational cost, we apply muscle actuation to the lower body only, while torque-base brquee-ased actuation used for the upper body. For more technical details about simulated model, see Appendix [
Generalization to pathological gait. We model five pathological gait conditions (calcaneal gait, foot drop, equinus gait, crouch gait, and waddling gait) by implementing specific alterations to mus- cle strength and contracture parameters. We trained separate Exo-plore models for each condition, rather than employing unified mixture-of-exper approach (Park al. 2022). To enhance gener- alization, we randomized simulation parameters during human controller training. Technical details are provided in Appendix J
4.2 GAIT GENERATION UNASSISTED
We first validate whether our gait data generator can produce results that align with real human experiments under the unassisted setup (i.e., K 0). This serves as a useful sanity check, since several human experiments without assistance have been previously reported.
dorsiflexion
Knee flexion (*)
Hip flexion (*)
Hip abduction (*)
Hip internal retation (*)
Sinulation
Human exp.
100
20
-00
-40-
100
0
100
90
(00
50
100
50
100
Gait cycle (%)
Figure 2: Comparison of joint kinematics gait cycle, with simulated results shown in red and human experimental data in gray.
Figure shows comparison of joint kinematics between our simulated results and human experi- ments (Boo al., 2025), where the joint angles the ankle, knee, and hip are reported. To obtain the simulated results, we trained the gait data generator with multiple random seeds and averaged the resulting joint angles over one gait cycle. Overall, joint kinematics qualitatively match with human data, except for hip internal rotation. We suspect that this discrepancy occurred due to overfitting of the Deep RL policy to the specific training environment, and thus it could be improved by mod- ifying training conditions, such changing the terrain of the environment. The kinematics of the pathological gaits are shown in Appendix K
Biezo Femoria
Gustreramiia Lacend
Clatsonemiia
Misimes
Acbotbi
Vostud Lazenlis
Viste Medalts
Rocta Feroais
Thiklis Amsorio
Preneees Borve
Penovay: Lorge
Sennerdoouus
Gat Gakoode PN
Figure 3: Comparison of simulated and experimental muscle activations.
Figure compares simulated and experimental muscle activations (Boo et al.) 2025) over one gait cycle. The results show that similar activation patterns to those observed in humans can be gen- crated, even though we did not explicitly incorporate human muscle activation patterns into the training process. Note that activation timing and patterns more informative than absolute values because muscle activations in human data (i.e., EMG signals) are normalized to each muscle's 's max- imum voluntary contraction, which to Appendix [[] addition, we performed a detailed analysis of the ground reaction forces (GRF). Additional information can be found in the Appendix M

---

Additionally, demonstrate the importance of choosing appropriate metabolic energy expen- diture (MEE, Equation 3 parameters in Fig- ure E The leftmost figure shows the Walk- ing speed CoTl curves obtained with differ- ent parameter choices. The curve with the op- timal parameters chosen by Algorithm [ best matches the human experiment results (Brown- ing et al., 2006) in terms of overall trend and PWS. The middle and rightmost figures show the predicted gait parameters (step length and step frequency) as walking speed increases. The prediction results from our gait data gener- ator match reasonably well with human exper- iments (Schwartz et al. 2008), except for the increased step frequency at low walking speeds (< km/ hh). This discrepancy might be because the Deep RL policy often had difficulty producing gaits with short step lengths due to the simplified foot model (box-shaped and rigid), which often caused unwanted collisions.
10
MEE mAa
Blama Sinulation
Septrr 000
MEE = mat
Meuiolic coot trattgoo (CoT)
Step Sequesso (1t))
0.5
30 Walking spoed (lmchi
45
6.0
Wilking speed Banch
Figure 4: Walking speed vs (CoT, Step length, Step frequency)
4.3 GAIT GENERATION ASSISTED
Hlip flexion ungle (*)
Hip flexion velocity (%))
Exo assist momeni (Nm)
Exo ussist power (W)
Human subject
-20
10
Walking sood hi
100
20
100
50
100
20
100
Simulation
1.5
Gait eyele (%))
Figure 5: Comparison of gait kinematics and dynamics under exoskeleton assistance between human experimental results (top) and simulated results (bottom).
Figure 5 compares simulation results with human experiments (Lim et al. 2019b) under oskeleton assistance with the control parameters (k, At) = (8 Nm, 0.25 Two kinematic results (hip flexion angle and its velocity) and dynamic results (assistive moment and power) ported, and the overall trends match reasonably across different walking speeds, demonstrating the effectiveness of our framework. Although overall trends are consistent across all four results, gaps in absolute values may result from discrepancies in musculoskeletal conditions between the simulated character and human participants, as well as from potential overfitting of the Deep RL policy the fixed environment.
We incorporated the human-exxskeleton interaction (HEI) reward to better model human adapta- tion exoskeleton assistance. To validate its effectiveness, we conducted ablation study and comparison with an alternative design choice like Fassist or no ThEi (Appendix [). Figure 6 reports RMS assistive moment, mean assistive power, and mean resistive power across different walking speeds. In the human experiment (Lim et al.) 2019b), the results resemble an increasing line, downward-opening parabola, and an upward-opentnn formulation based on resistance minimization, no THE means no HEI reward is incorporated, and Tassist means that assistance maximization is used. The results with our HEI reward show similar trends to the human experiments and also yield the highest Pearson correlation with them. For ex- ample, at 4 km/ hh human experiments show a 1.8 1.88-fold increase in assistive power when the delay increases from 0.05 to 0.25 With THEi the result showed similar 1.73-fold increase (correla- tion = 0.83), whereas no THE and r rassist showed weaker or inconsistent scaling (0.67- and 1.04-fold; correlation 0.69 and 0.23, respectively). Figure 7 shows the maximum metabolic reduction rate
parabola, respectively. THE our original
8

---

achieved through exoskeleton assistance. Our HEI reward achieved the closest alignment with hu- man experimental results (Lim et al. 2019b). whereas the others produced lower metabolic reduction rate values, indicating that the simulated characters did not realize the same level of benefit observed in human participants. Evaluated metabolic reduction rate the simulation slightly higher than in human experimental data. Note that simulation metabolic reduction rate is computed relative the unassisted condition (ı 0), whereas human experiments measure it relative to walking without an exoskeleton, partly explaining the discrepancy. Additional details provided in Appendix O
RMS assist mement (Nm)
Mcan assist power (W)
Moan resist power (W)
Mudimum 10M50000 redection
dday (At)
rewards
to fwe Buman dep
Walking speed (km/h)
Figure 6: Comparison of assistive moment, assistive power, and resistive power across walking speeds, showing that our HEI reward (Thε) repro- duces the trends observed in human experiments and achieves the highest correlation with empirical data.
Figure 7: ison of maximum metabolic rate.
Compar-
reduction
4.4 OPTIMIZING EXOSKELETON CONTROL PARAMETERS
We demonstrate the applicability of Exo-plore by discovering and analyzing optimal exoskeleton control parameters for both able-bodied and disabled individuals.
Able-bodied Individuals. To optimize exoskeleton control parameters, we train a surrogate net- work to learn the mapping between CoT (i.e., metabolic energy consumption per unit travel distance) and the control parameters (ı: gain, At: time delay). Figure aa-c) a-c) show the raw training data ob- tained from simulation and the learned mappings with two gradient penalty settings (Agp = 0.1 and Arp 0.01), respectively. While the raw data contain many local minima from simulation stochas- ticity and Deep RL training, the surrogate network provides a smoother landscape that enables faster and more stable optimization while still capturing essential trends. A trade-off exists between small and large gradient penalties: smaller penalties allow more aggressive optimization but risk overfit- ting artifacts, whereas larger penalties reduce this risk but may limit sensitivity. Figure d) shows the optimal control parameters obtained from the two surrogate networks. The optimized delay (At) decreases monotonically as speed increases.
Raw gait data
Surrogate (Agp 0.11
Surmngat Mgg 0.01)
20
0.50
sanu -01
Exosk eeleton giir, X (Nim)
Optimal gain, (Nim)
0.00
delay, ())
Normalirad Car
dalyy Ag
daks, 0.01
L
t0
0.5
1.0
1.0
10
0.00
0.00
0.25
0.50
0.00
0.25
0L50
0.00
0.25
0.50
Deley (x)
Delay 6x)
Deley (x)
Walking speed (lamit)
Figure 8: CoT landscape from raw simulation data (a), surrogate network trained with Agp = 0.1 (b), surrogate network trained with Agp 0.01 (c), and optimized exoskeleton control parameters
(K, left axis; At, right axis) across walking speeds (d).
Disabled Individuals. We evaluate our framework on musculoskeletal impairment profiles repre- sentative of pathological gait. Pathology severity is quantified as the degree of deviation in muscle parameters from healthy reference values. We examine five common pathological gait types: equi- nus, waddling, crouch, calcaneus, and foot drop. Figure presents the optimal gains (R) identified by Exo-plore, averaged across walking speeds from 2 3.5 km/h, with values reported for different levels of pathology severity in each gait type. The optimal gains exhibit strong linear correlations with severity across all conditions except foot drop.

---

These correlations be explained by the characteristies of each pathology. In equinus and wad- dling gaits, characterized by tal-walking and lateral trunk sway, respectively, the pathological pat- terns create greater instability compared to normal gait. External forces, such exoskeleton sistance, may amplify this instability, necessitating minimal and carefully controlled intervention. In contrast, crouch and calcaneal gaits, characterized by excessive knee flexion and reduced ankle plantarflexion, respectively, present stable but metabolically demanding kinematic patterns where exoskeleton assistance effectively reduces energy expenditure without compromising stability, Foot drop exhibited irregular trends causing frequent toe- noe-ground collisions, which substantially increased gait variability and prevented stable optimization convergence. Further detailed analysis is provided in Appendix
Eqpainss (r" 0.90)
Waddling (f" -0 0.83)
Clouch ( 0.85)
Calcenens (2f 0.82)
Foot drop (f 0059
Avergga optinal gai.. a (Nm)
20
20
MBM
Severe
Mia
Sevez
Mid
Mild
Sevven
Midd
Sevene
Figure 9: Optimal cxoskeleton gain over five pathological gait types, with values reported for differ- ent levels of pathology severity in each gait type. Each color corresponds to different random seed. Detailed numerical results are provided in Appendix P.4
5 CONCLUSION AND LIMITATIONS
Conclusion. This study developed Exo-plore, a simulation-based framework that optimizes ex- oskeleton controllers through neuromechanical simulation and Deep RL. Through sim-to-real matching, we demonstrated that Exo-plore can serve as an alternative to experimental trials for controller optimization. Furthermore, by extending the framework to pathological gaits, Exo-plore enables the design cxoskeleton controller for mobility-impaice individuals who are not suitable for traditional human-in-the-lowe optimization methods.
Limitations. Despite these promising results, key limitations remain. Most importantly, exper- imental validation with human subjects, including patient populations, is required to establish the validity of cimulation-based predictions and the clinical effectiveness of the optimized controllers. Within our framework, several components warrant evidence of real-world impact, including simpli- fied reward models and the current lack of personalization to subject-specing motor control, and the use of approximate musele dynamics (see Appendix R for details) that may fail to capture subject- specific neuromuseular responses assistance.
ACKNOWLEDGMENTS
This work was supported by the National Research Foundation (NRF) of Korea grant funded by the Korea government (MSTT) RKS-2024-0045064777 This work also supported by the Institute of Information Communications Technology Planning Evaluation (ITTP) grant funded by MSIT under Grant Nos. RS-2025-25442338 (AI Star Fellowship Support Program, Scoul National University), RS-202 1343 (Artificial Intelligence Graduate School Program, Seoul National University), and 211P-2022-RSS-022-1220100 (ITRC: Information Technology Research Center).
REPRODUCIAIILITY STATEMENT
Code and videos available at htttps://dabbangee www.aaanntt.ggihuu........ github. woabboooooooppppeee plore The experi- mental data used in this work are properly cited in their respective figures.
REFERENCES
BC Abbott and XM Aubert. The force exerted by active striated muscle during and after change of length. The Journal of physiology, 117(1):77, 1952.
10

---

Farzad Abdolhosseini, Hung Yu Ling, Zhaoming Xie. Xue Bin Peng, and Michiel Van de Panne. On learning symmetric locomotion. In Proceedings of the 12th ACM SIGGRAPH Conference on Motion, Interaction and Games, pp. 1-10, 2019.
Marko Ackermann and Antonie Van den Bogert. Optimality principles for model-based prediction of human gait. Journal of biomechanics, (3(6):1055-106) 2010.
Caitlin Banks, Mihir Pai, Theresa McGuirk, Benjamin Fregly, and Carolynn Patten. Methodological choices in muscle synergy analysis impact differentiation of physiological char- acteristics following stroke. Frontiers in computational neuroscience, 11:78, 2017.
Alex Bersani, Giorgio Davico, and Marco Viceconti. Modeling human suboptimal control: a review. Journal of Applied Biomechanics, 39(5):294-300 2023.
Junyo Boo, Dongwook Seo, Minseung Kim, and Seungbum Koo. Comprehensive human locomo- tion and electromyography dataset: 2o.t1 Scientifie Data, 12(1):1023, 2025.
Alexander I. Brown, Taisuke Imai, Ferdinand M Vieider, and Colin F Camerer. Meta-analysis of empirical estimates of loss aversion. Journal of Economic Literature, 62(2):485-516, 2024.
Raymond C. Browning, Emily A. Baker, Jessica A. Herron, and Rodger Kram. Effects of obesity and sex the energetic cost and preferred speed of walking. Journal of Applied Physiology, 100 (2):390-398, 2006. doi: 10.1
Gwendolyn M Bryan, Patrick W Franks, Seungmoon Song, Alexandra S Voloshina, Ricardo Reyes, Meghan P O'Donovan, Karen N Gregorezyk, and Steven H Collins. Optimized hip hip-knee ankle exoskeleton assistance at a range of walking speeds. Journal of neuroengineering and rehabilita- fion, 18(1):152, 2021.
John W Chow and Warren G Darling. The maximum shortening velocity of muscle should be scaled with activation. Journal of Applied Physiology, 86(3):1025-1031, 1999.
Friedl De Groote and Antoine Falisse. Perspective on musculoskeletal modelling and predictive simulations of human movement to assess the neuromechanics of gait. Proceedings of the Royal Society B, 288(1996))20202433. 2021.
Christopher Dembia, Amy Silder, Thomas K Uchida, Jennifer Hicks, and Scott Delp. Simu- lating ideal assistive devices to reduce the metabolic cost of walking with heavy loads. PloS one, 12(7))e01 80320, 2017.
Vahid Firouzi, Andre Seyfarth, Seungmoon Song, Oskar von Stryk, and Maziar Ahmad Sharbafi. Biomechanical models in the lower-limb exoskeletons development: A review. Journal of Neu- roEngineering and Rehabilitation, 22, 2025.
Alexander Forrester, Andras Sobester, and Andy Keane. Engineering design via surrogate mod- elling: practical guide. John Wiley & Sons, 2008.
Jacob R Gardner, Geoff Pleiss, David Bindel, Kilian Q Weinberger, and Andrew Gordon Wilson Gpytorch: Blackbox matrix-matrix gaussian process inference with gpu acceleration. In Advances in Neural Information Processing Systems, 2018.
Thomas Geijtenbeek. Scone: Open source software for predictive simulation of biological motion. Journal of Open Source Software, H(38):1411, 2019.
Daniel FN Gordon, Christopher McGreavy, Andreas Christou, and Sethu Vijayakumar. Human-in- the-loop optimization of exoskeleton assistance via online simulation of metabolic cost. IEEE Transactions on Robotics,
2022.
Ishaan Guirajani, Faruk Ahmed, Martin Arjovsky, Vincent Dumoulin, and Aaron C Courville. Im- proved training of wasserstein gans. Advances in neural information processing systems, 30, 2017.
Geoffrey G Handsfield, Craig H Meyer, Joseph M Hart, Mark F Abel, and Silvia Blemker. Re- lationships of 35 lower limb muscles to height and body mass quantified using mri Journal of biomechanics, 47(3):631 1-638, 2014.
11

---

Trevor Hastie, Robert Tibshirani, Jerome Friedman, et al. The elements of statistical learning, 2009.
Archibald Vivian Hill. The heat shortening and the dynamic constants of muscle. Proceedings of the Royal Society of London. Series B- B-Biological Sciences, 126(843):1366 136-1996..11911 1938.
Fabian Horst, Djordje Slijepcevic, Marvin Simak, and Wolfgang Schöllhorn. Gutenberg gait database, ground reaction force database of level overground walking in healthy individuals. Scientific data, 8(1):232, 2021.
Kimberly A Ingraham, C David Remy, and Elliott J Rouse. The role of user preference in the customized control robotic exoskeletons. Science robotics, 7(64))eab33487, 2022.
Daniel Kahneman. Prospect theory: An analysis of decisions under risk. Econometrica, 47:278,
1979.
Inseung Kang. Dean D Molinaro, Dongho Park, Dawit Lee, Pratik Kunapuli, Kinsey R Herrin, and Aaron Young. Online adaptation framework enables personalization of exoskeleton assistance during locomotion in patients affected by stroke. IEEE Transactions on Robotics, 2025.
Jinsoo Kim, Franchino Porciuncula, Hee Doo Yang, Nicholas Wendel, Teresa Baker, Andrew Chin, Terry D Ellis, and Conor Walsh. Soft robotic apparel to avert freezing of gait in parkinson's disease. Nature medicine, 30(1): 2024.
Ava Lakmazaheri, Seungmoon Song, Brian B Vuong, Blake Biskner, Deborah M Kado, and Steven H Collins. Optimizing exoskeleton assistance to improve walking speed and energy econ- omy for older adults. Journal of NeuroEngineering and Rehabilitation, 21(1):1, 2024.
Jehee Lee. Representing rotations and orientations in geometric computing. IEEE Computer Graph- ics and Applications, 28(2):75-83, 2008.
Jeongseok Lee, Michael X. Grey, Sehoon Ha, Tobias Kunz, Sumit Jain, Yuting Ye, Siddhartha S. Srinivasa. Mike Stilman, and C. Karen Liu. DART: Dynamic animation and robotics toolkit. The Journal of Open Source Software, 3(22):500, Feb 2018. doi: 105) URL
21100/ joss 00500
Seungwhan Lee, Moonseok Park, Kyoungmin Lee, and Jehee Lec. Scalable musclc-actuated human simulation and control. ACM Transactions On Graphics (TOG), 38(4:11-13, 2019.
Ung Hee Lee, Varun S Shetty, Patrick W Franks, Jie Tan, Georgios Evangelopoulos, Schoon Ha, and Elliott Rouse. User preference optimization for control of ankle exoskeletons using sample efficient active learning. Science Robotics, 8883)eedd33705, 2023.
Bokman Lim, Junwon Jang, Jusuk Lee, Byungjune Choi, Younbaek Lee, and Youngbo Shim. De- layed output feedback control for gait assistance and resistance using a robotic exoskeleton. IEEE Robotics and Automation Letters, 4(4):3521-3528, 2019a.
Bokman Lim, Jusuk Lee, Junwon Jang, Kyungrock Kim, Young Jin Park, Keehong Seo, and Youngbo Shim. Delayed output feedback control for gait assistance with robotic hip exoskele- 2019b.
ton. IEEE Transactions Robotics,
Chiara Livolsi, Roberto Conti, Tommaso Ciapetti, Eleonora Guanziroli, Thor Fridriksson, Ásgeir Alexandersson, Emilio Trigili, Francesco Giovacchini, Raffacle Molino Lova, Alberto Esquenazi, et al. Bilateral hip exoskeleton assistance enables faster walking individuals with chronic stroke-related gait impairments. Scientific Reports, 15(1):2017, 2025.
Shuzhen Luo, Menghan Jiang, Sainan Zhang, Junxi Zhu, Shuangyue Yu, Israel Dominguez Silva, Tian Wang, Elliott Rouse, Bolei Zhou, Hyunwoo Yuk, al. Experiment-free exoskeleton assis- tance via learning in simulation. Nature, 630(8016):35333355 2024.
J Maxwell Donelan, Rodger Kram, and Kuo Arthur D. Mechanical and metabolic determinants of the preferred step width in human walking. Proceedings of the Royal Society of London. Series B: Biological Sciences, (68(1480):19)) 1985- 2001.
12

---

Dean D Molinaro, Keaton L Scherpereel, Ethan B Schonhaut, Georgios Evangelopoulos, Max K Shepherd, and Aaron Young. agno-sgi exoskeleton control via biological joint moment estimation. Nature, 635(8038):337- 2024.
Kiisa C Nishikawa, Jenna A Monroy, Theodore E Uyeno, Sang Hoon Yeo, Dinesh K Pai, and Stan L Lindstedt. titin 'winding filament'? a new twist on muscle contraction. Proceedings of the royal society B: Biological sciences, 279(1730):981
2012.
Michael A Normand, Jeonghwan Lee, Hao Su, and James S Sulzer. The effect of hip exoskele- ton weight on kinematics, kinetics, and electromyography during human walking. Journal of biomechanics, 152:111552, 2023.
Carmichael F Ong. Thomas Geijtenbeek, Jennifer I. Hicks, and Scott I. Delp. Predicting gait adap- tations due to ankle plantarflexor muscle weakness and contracture using physics-based museu- loskeletal simulations. PLoS computational biology, 15(10)xc 2019.
Jungnam Park, Sehee Min, Phil Sik Chang, Jaedong Lee, Moon Seok Park, and Jehee Lee. Genera- tive gaitnet. In ACM SIGGRAPH 2022 Conference Proceedings, pp. 1-9, 2022.
Jungnam Park, Moon Scok Park, Jehee Lee, and Jungdam Won. Bidirectional gaitnet: A bidirec- tional prediction model of human gait and anatomical conditions. In ACM SIGGRAPH 2023 Conference Proceedings, pp. 1-9, 2023.
Jungnam Park, Euikyun Jung, Jehee Lee, and Jungdam Won. Magnet: Muscle activation generation networks for diverse human movement, In Proceedings of the Special Interest Group on Computer Graphics and Interactive Techniques Conference Conference Papers, pp- 1-11, 2025.
Katherine L Poggensee and Steven H Collins. How adaptation, training, and customization con- tribute to benefits from exoskeleton assistance. Science Robolics, 6(58):eabf1078 2021.
Apoorva Rajagopal, Christopher L. Dembia, Matthew S DeMers, Denny D Delp. Jennifer L Hicks, and Scott L Delp. Full-body musculoskeletal model for muscle-driven simulation of human gait. IEEE transactions on biomedical engineering, 63(10))2068-200) 2079, 2016.
Boris Roszek and Peter A Huijing. Stimulation frequency history alters length-force characteristics of fully recruited rat muscle. Journal of electromyography and kinesiology, 7(3):161-177, 1997.
Michael H Schwartz, Adam Rozumalski, and Joyce P Trost. The effect of walking speed the gait of typically developing children. Journal of biomechanics, 11(8))1633-1663)) 2008.
Mohammad S Shourijeh and Benjamin J Fregly. Muscle synergies modify optimization estimates of joint stiffness during walking. Journal of biomechanical engineering, 142(1):011011, 2020.
Patrick Slade, Mykel Kochenderfer, Scott L Delp, and Steven H Collins. Personalizing exoskeleton assistance while walking the real world. Nature, 610(7931):277-22 2022.
Patrick Slade, Christopher Atkeson, Maxwell Donelan, Han Houdijk, Kimberly A Ingraham, Myunghee Kim, Kyoungchul Kong, Katherine L Poggensee, Robert Riener, Martin Steinert, et al. On
optimization of human interaction. Nature, 633(8031):779--88
2024.
Seungmoon Song and Steven H Collins. Optimizing exoskeleton assistance for faster self-selected walking. IEEE Transactions on Neural Systems and Rehabilitation Engineering, 29:786-795, 2021.
Seungmoon Song and Hartmut Geyer. A neural circuitry that emphasizes spinal feedback generates diverse behaviours of human locomotion. The Journal of physiology, 5933116:3493-3511 2015.
Seungmoon Song and Hartmut Geyer. Evaluation of neuromechanical walking control model using disturbance experiments. Frontiers in computational neuroscience, 11:15, 2017.
Seungmoon Song, Lukasz Kidziński, Xue Bin Peng, Carmichael Ong, Jennifer Hicks, Sergey Levine, Christopher G Atkeson, and Scott L Delp. Deep reinforcement learning for modeling human locomotion control in neuromechanical simulation. Journal of neuroengineering and re- habilitation, 18(1):126, 2021.
13

---

Chun Kwang Tan, Cheryl Wang, Shirui Lyu, Balint K. Hodossy, Pierre Schumacher, Elizabeth B Wilson, Vittorio Caggiano, Vikash Kumar, Dario Farina, Letizia Gionfrida, et al. Myoassist 0.1: Myosuite for dexterity and agility in bionic humans. In 2025 International Conference On Rehabilitatinn Robotics (ICORR), pp. 437-442. IEEF, 2025.
Jie Tan, Karen Liu, and Greg Turk. Stable proportional-derivatiee controllers. IEEE Computer Graphics and Applications, 31(4):34 44, 2011.
Lena H Ting and Stacie A Chvatal. Decomposing muscle activity in motor tasks. Motor control:
theories, experiments, and applications, pp. 102-138, 2010.
Matthew C C'Teehh, Philippe Saltiel, and Emilio Bizzi. The construction of movement by the spinal cord. Nature neuroscience, 122):122-167 1999.
Amos Tversky and Daniel Kahneman. Loss aversion in riskless choice: A reference-dependent model. The quarterly journal of economics, 103(4):009--101, 1991.
Brian R Umberger and Philip Martin. Mechanical power and efficiency of level walking with 2007.
different stride rates. Journal of Experimental Biology,
Cara Gonzalez Welker, Alexandra S Voloshina, Vincent L Chiu, and Steven H Collins. Shortcomings of human-in caaum-i--the-loge the-loop optimization of an ankle-foot prosthesis emulator: a case series. Royal Society open science, 8(5):202020, 2021.
Kirby A Witte, Pieter Fiers, Alison Sheets-Singer, and Steven H Collins. Improving the energy economy of human running with powered and unpowered ankle exoskeleton assistance. Science Robotics, 5(40)=eaay9108, 2020.
Jungdam Won and Jehee Lee. Learning body shape variation in physics- characters. ACM Transactions Graphics (TOG), 38(6):1-12, 2019.
MY Zaruuh,, FN Todd, and HJ Ralston. Optimization of energy expenditure during level walking. European journal of applied physiology and occupational physiology, 33(4):293-306 1974.
Juanjuan Zhang, Pieter Fiers, Kirby A Witte, Rachel W Jackson, Katherine L Poggensee, Christo- pher G Atkeson, and Steven H Collins. couman-in-the-laee optimization of exoskeleton assistance during walking. Science, 25666344 2017.
14

---

A ARCHITECTURE OF HUMAN CONTROLLER
qd
7PD 7MCN
Muscle coordination network
State S
Pase network
Stable
Muscle octivation
a
controller
Human Controller
Figure A.I: The human controller computes muscle activations from the musculoskeletal simulation state and applies them back to the simulation.
B DETAILS OF THE RL FORMULATION FOR POSENET
B. STATE REPRESENTATION
Kinematics (R:19): All simulated rigid bodies are represented in a root-centered local coordinate frame, including their relative positions, orientations, and linear and angular velocities. Phase-based mirroring is applied to account for gait symmetry (Abdolhosseinj
et 2019).
Root height: R
Root linear velocities: R
Body part positions: R
Body part rotations: R
Body part linear velocities: R
Body part angular velocities: 1969
Gait features (IR"):
Target foot position: R
Foot contact state: IR2
Target gait velocity: R!
Current/Taget/Reeerennee gait phase: R
Muscle-related terms (R:5): Passive muscle forces, lower and upper torque bounds per joint, and dynamic muscle-specific parameters like muscle length.
Muscle weakness and contracture: R
Passive forces: IR 18
Torque bounds: 36
Exoskeleton assist torque history: (R54); Recent history of assistive torques applied at the hip joints (left/right), representing the temporal profile of exoskeletal assistance the human.
B.2 ACTION REPRESENTATION
As in Park al. (2022), employ the residual pose action formulation. The action consists of pose and phase modulation:
A (Ad, AM), where AM denotes the pose displacement, and A represents the phase increment that either ad- vances or delays gait phase with respect to the reference gait motion.
(10)
The gait (d) and target pose(M for the PD controller is then updated according to
Φr+1 Φr + Ad,
(11)
Mr Met + A..
where Miet denotes the reference motion (obtained from Park et al. (2022) and ① denotes the operation between pose and displacement, which includes operations in SO(3) to properly update 3DD rotations. For more technical details, refer Lo (0088
15

---

B.3 REWARD DESIGN
We use composite reward consisting of four terms:
Itotal Wgaat: Tgaai. Wenergy TTnnrgyy Warm Tarm + WHEI THEI
(12)
Since we previously mentioned Teuergy and THEI we now discuss the remaining one. There are four terms in the "gait Tstep" Tvvl" hhead Tsway- Specifically, rsep and Tvels encourage the agent to maintain a target step size and walking velocity:
(| hdesirea (tasere
ssep exp
(13)
Tvel
exp
(1lv "deesirell//wwl
Here, denotes the current foot position, and v denotes the average forward walking velocity of the simulated character. Both values compared against their respective targets to guide the agent toward producing the desired gait pattern.
Humans generally maintain head stability during walking, as key sensory systems like vision and balance are located there. Thead encourages consistent head motion by penalizing abrupt changes in head angle, linear acceleration, and angular acceleration.
Thead = fr fo fus
fr kalive + (1 kalive) exp (Ohead/ ('uead)
Opead > Xr, else 1,
(14)
fo Kalive + (1 kialive) exp (ani// (oeaad) )²
if alio > Au else 1,
fw = kalive + (1 kialive) exp (aang// (Obead)
if Gang
>
Aw else 1.
Here, Ohead denotes the head orientation with respect to the frame aligned to the progression direction and the global up direction. djn and Gang represent linear and angular accelerations the head, respectively.
To prevent the emergence of unrealistic gait patterns due to model inaccuracies or excessive energy minimization, a sway reward discourages abnormal deviations in musculoskeletal body orientations, promoting kinematics that remain close to typical human walking behavior.
(-(1
[ [19=//ll -82 -6 Tonay
)²)
I'swayb & exp
if 9gb qol S 8y. otherwise,
(15)
II Tsway.b: BEB
B = (pelvis, spine, foot].
Tsway
where Q denotes the angle of body b, Tb is the angle derived from normative gait data, and & represents the deviation threshold for body b. The total sway reward Reway is computed as the product body-wise sway terms & across joints.
To promote natural arm movements during gait, we introduce an upper-arm imitation reward. This reward measures the deviation between the simulated and reference upper-arm joint angles, encour- aging the musculoskeletal model to align its arm posture with the reference motion. The reward is computed as:
T Tarm
exp
Σ (1 dall/
(16)
jeam
where a, denotes the joint angle of joint j.j in the reference motion Mrets qj represents the current joint angle.
16

---

B.4 TRAINING CURVES
Itotal
Tenergy
[HEI
Farm
2
gait
rstep
rvel
Thead
Tsway
0
steps (x (x108)
Figure B.2: Training curves of each reward component. Shaded regions indicate 120 over n runs.
C INTRAMUSCULAR REGULARIZATION ENHANCES PHYSIOLOGICAL PLAUSIBILITY OF MUSCLE ACTIVATION
In this section, we investigate how LIMR affects both the training dynamics and the actual activations computed by the MCN. As shown in Figure C.3 during the training process with the intramuscular regularizer (IMR), the IMR-specific loss decreased steadily while the activation discrepancies across line segments of the same anatomical muscle also diminished. In contrast, training without the IMR often led to large differences between line-muscle activations where human experimental data does show (Ong al. 2019). Although the baseline MCN loss includes an activation regularizer that prevents over-reliance on individual muscles, this alone was insufficient to avoid highly imbal- anced segment activations. Such imbalances are physiologically implausible, since humans cannot selectively control individual line segments of single muscle. Therefore, the IMR is essential for producing activation patterns that are both stable and physiologically plausible.
Gluteus Maximus
Gluteus Medius
Soleus
w/
(b)
Hazaan subject
Within-grow group musele activation difference (log)
WD/
without EnA
(ColerdySSimmntin
100
100
100
with CINK
Training spoch (log)
100
Gait eyele
100
16o
Figure C.3: LIMR training epochs (a), Line-wise activation profiles (colored) across the gait cycle, in comparison with human data (b)
17

---

D DETERMINING METABOLIC ENERGY EXPENDITURE PARAMETERS
We selected the MEE exponent parameter pair (a, B) = (1.5, 1.0). For each pair, we trained multiple data generators with different random seeds and computed the resulting PWS using Algorithm 2 Migure[D.4]shows the result which revealed that a had significant influence on the simulated preferred walking speed (PWS), whereas 5 had relatively little effect. There- fore, we chose closest to the human reference value (Ureal 1.25 m/s). For 3, we selected value of 1.0 to ensure that metabolic cost scaled linearly with
Proferred walking specd sin) (mis)
(Musele activition exponent)
MEC
MEE =
MEE
MEE a
Vrral Chesen (a. B)
1.5, which produced PWS
mét ear
mEt-nei
[ (Muselc mass exponent)
musele mass, allowing the total cost to remain consistent regardless of whether mass was con- sidered per-musele or globally.
Figure D.4: Selection of MFF exponents param- eter pair (ox, 3) based on resulting PWS.
E LIMITATIONS OF GAUSSIAN PROCESSES AS A SURROGATE MODEL
E.1 GAUSSIAN PROCESSES CONSUME EXCESSIVE MEMORY
We conduct a feasibility comparison between Gaussian processes (GP) and neural network (NN) by analyzing peak runtime memory consumption. For fair comparison, employed GPU-based GP implementation (Gardner et al. 2018). All experiments were conducted on an RTX 3070 GPU. Since GP require ( memory, datasets larger than 20k samples exceeded the GPU VRAM limit (8GB).
dataset size
GP (MB)
NN (MB)
20,000
Out-of-memory
538
10,000
2232
360
500
258
226
Table E.I: Comparison of peak runtime memory usage between GP and NN.
E.2 DATASET SIZE REQUIRED FOR PLAUSIBLE RESULTS EXCEEDS GP's COMPUTATIONAL.
LIMIT
Figure E.5 illustrates the relationship between optimized exoskeleton parameters and the size of the training dataset when using a neural network as a surrogate. With 5k and 10k sample sizes, the result- ing optimization landscapes are noisy and contain unrealistic local minima, making them unsuitable for robust parameter selection. Increasing the dataset size to 20k samples improves the smoothness and plausibility of the parameter predictions; however, the memory requirements quickly become prohibitive for GP (see Appendix E.Ip. In contrast, neural network surrogates can efficiently uti- lize large datasets with relatively low computational resources, enabling stable and physiologically plausible optimization results.
Sample size: 5k
Sample size: 10k
Sample size: 20k
Sample size: 60kk
Oplinal gain, K (Nm)
gain,
Optimal dolay, At (5)
dalay, At
0.25
10
n25
10
0.25
10
0.35
Walking speed (kmm/h)
Figure E.5: Exoskeleton controller optimization results across different dataset sizes (5k, 10k, 20k, and 60k parameter samples).
18

---

F HUBER LOSS AND LATIN HYPERCUBE SAMPLING ARE REQUIRED FOR SMOOTH SURROGATE NETWORK LANDSCAPE
Figure F.6 shows the surrogate network landscape across walking speeds. Our proposed combination of Huber loss and Latin Hypercube Sampling (LHS) produces the smoothest landscape compared to other approaches. The aise-based surrogate network exhibited excessive sensitivity to noisy gait data, resulting in jagged cost landscapes, especially at higher exoskeleton gains (k), where external assistance amplifies gait variability. Similarly, the surrogate network trained on uniformly sampled data displayed aliasing artifacts due poor sample coverage, despite using the same sample size
(NV = 60k).
Raw gait
Preposed: Huber LHS
Casel: MSE
Case2: Uniform sampling
12
12
12-
Eroskoeen snin K (Nmm)
Normalized Col
1.1
11
111
10
0.9
0.9
0.9
0.9
Walking speed (km/h)
Figure F6: Landscape of CoT from gait data generator across walking speeds and exoskeleton gains (x) normalized to the exoskeleton-off condition (first panel). Surrogate predictions under different training setups: Our proposed configuration (second panel) yields a smoother landscape than altermative approaches (third and forth panels).
G MOTIVATION OF THE GRADIENT PENALTY LOSS OF THE EXOSKELETON OPTIMIZER
This penalty is motivated by the Lipschitz condition:
[l f(x2)) S ||21 - x2ll
V0 X,
(1((1)) f(x2)]
||>] - x2ll
(17)
[1((x)) f(x2)) [V)()) 1.
lim 22-121
[ - .22
small gradient norm implics local Lipschitz continuity, and uniform bound on the gradient norm ensures global Lipschitz continuity (Gulrajani al. (017)). Enforcing this property prevents abrupt shifts in predicted CoT values across the parameter space and yields stable gradients that can be exploited by downstream optimization.
H OPTIMIZING EXOSKELETON WITH SURROGATE NETWORK
Let {gn JMM denote the set of gait parameters. K2 and At, are the optimal parameters corresponding to each In. and the optimization performed with the following objective:
M
C(Kn, Aln) =
Mgni Kiy Aln)
(18)
M-1 (kn411
A1
Σ |k + d
Σ
Knp Attt1 Atrl).
Here, ò represents the perrogate-predcctee metabolic cost of transport. However, surrogate predic- tions can exhibit minor fluctuations across the parameter space due to the inherent complexity of the energy landscape or enrors introduced during model training. These fluctuations can lead to abrupt variations in the exoskeleton control parameters when optimizing solely for COT. To miti-
19

---

gate this issue, introduce two additional regularization terms. The first regularizer discourages extreme parameter magnitudes, while the second promotes smooth variation of solutions across speeds. Together, these regularizers ensure that the optimized parameters minimize metabolic cost while remaining realistic and consistent across varying gait parameters. All hyperparameters used in the exoskeleton optimizer provided in Appendix
SIMULATION MODEL
1.1 SIMULATED CHARACTER
Key muscle parameters, including isometric force and tendon slack length, were adjusted to match the OpenSim model by Rajagopal at. (20T6), al.T6))) and muscle mass distributions followed the estimates Handsfiell et al. 2014). The simulated subject's 's body mass and height were set to match average values reported in the target exoskeleton study (Lim et al. 2019a). Anthropometric data are provided
in Table L.2
Item
Unit
Value
Height
cm
169
Mass
kg
72.9
Table 1.2: Physical properties of the musculoskeletal character.
I.2 EXOSKELETON
We adopted Samsung Electronics' gait enhancing and motivating system (GEMS)
(Lim et al.
2019b See Table for mechanical properties.
Item
Unit
Value
Body segment
kg
1.7
Thigh segment
kg 0.6
Total mass"
kg 2.3
Table I.3: Physical properties of the exoskeleton.
*The total mass differs slightly from er al. (2019a) because we imposed minimum mass of 50 on each exoskeleton element to prevent numerical instability caused by nearly singular dynamics when solving Newton's equations.
In the human controller of the data generator, the stable PD controller computes the final desired joint torque as:
( (2) (1)
MCN = TPD T2 Vjj E Thip
(19)
where Tex denotes the torque applied by the exoskeleton, TPD denotes computed torque from the PD-target in the PD-controller and TMNN serves as the input to the MCN for computing muscle activations. This formulation ensures that the MCN receives only the net torque required for joint motion, effectively reflecting dynamics under exoskeletal assistance.
20

---

J SIMULATION PARAMETERS
Parameters for the simulation displayed in T.bleJ..4 Gait parameters and exoskeleton control pa- rameters are uniformly sampled, while muscle parameters are sampled in an adaptive manner (Won) & Leee 2019). Adaptive sampling is training technique in reinforcement learning that adjusts the parameter sampling rate during RL. training based on the learning difficulty of each parameter. Specifically, the muscle parameter sampling interval is divided into bins. For each bin, we evaluate the average episode length of the human controller while training. If some bins have lower average episode lengths than other bins, we increase the sampling rate for those bins.
Category
Name
Unit
Range
Muscle weakness
(maximum isometric force)
Triceps surae (Calcaneous)
N.A.
0, 0.4
Tibialis anterior (Footdrop)
N.A.
0.5
Musele
Gluteal muscles (Waddling)
N.A.
[0, 0.4
parameter
Muscle contracture
(optimal muscle fiber length)
Triceps surae (Equinus)
N.A.
[0.73, 1]
Psoas (Crouch)
N.A.
[0.55, 1
Gait parameter
Step length
[13.4, 93.8]
Step frequency
spm
[1.27, 2.55]
Exoskeleton control
Gain (x)
Nm
[0. 21]
parameter
Delay (At)
sec
[0, 0.5]
Table J.4: Simulation parameter name and its range.
21

---

K GENERATED PATHOLOGICAL GAIT KINEMATICS
Anile dossifexion
drg Eyunns)
Krze Blexion (*) (Cosess]
Tonoo seay (cm) (Maddlad)
Caloacoome
Choudh
Wobiling
20
Fost dhop
nocmal
poctool
Equiine
40
00
0
100
G0 Cycle
200
100
al contact
Miletnec
Foot
Initid contact
Foot of
Righn midssanee
Left midstanee
Normal
Noenal
Calonnouu
1.
Reduced pusboff angl
Foot deop.
1
Workling
Choocs
The strill procsses Equiaue
/
TocRaaking mibrance
Excessivve lanee fleskoo
Latural trunk
Figure K.7: Comparison of simulation results between normal muscle conditions and each patho- logical condition.
Exo-plore trained with pathological muscle conditions using parameters specified in Ap- pendix []] and the resulting gait patterns are presented in Figure|K. Rendered images with descrip- tive annotations are provided to facilitate understanding of each pathological condition. Close-up images of the feet include ground reaction force visualization to illustrate contact states during the gait cycle.
We compare pathological gaits with unimpaired gait to clearly show deviation patterns.
For pathological gait cases, the walking speed was fixed at 3.5 km/h to ensure consistent comparison across severities. In contrast, for unimpaired gait, we used the preferred walking speed (PWS) identified by Algorithm 2 and selected gait parameters that minimized the CoT at that speed.
Name
Cause
Description
Equinus
Contracture in Triceps surae
Walking the toes due to persistent plantarflex- ion of the ankle. The heel does not touch the ground during gait.
Waddling
Weakness in Gluteus
Swaying of the trunk from side to side while walk- ing due to weakness of the gluteal muscles, which fail to stabilize the pelvis. This gait is also referred as waddling or "duck-like" gait.
Crouch
Contracture in Psoas
Walking in a flexed, crouched posture, which maintains the hips in persistent flexion, often ac- companied by knee flexion.
Calcaneous
Weakness in Triceps surae
Walking with the heel contacting the ground first due to weakness of the triceps surae muscles, lead- ing to excessive dorsiflexion of the ankle.
Footdrop
Weakness in Tibialis anterior
Inability to dorsiflex the ankle during foot-off, re- sulting the toes dragging on the ground. Pa- tients often compensate with high-stepping gait.
Table K.5: Types of pathological gait patterns, their causes, and descriptions.
22

---

L OVERACTIVATION OF THE RECTUS FEMORIS IN THE SIMULATION
Figure shows the muscle activation of rectus femoris (RF) have the largest sim-to-real gap. Sim- ulated RF is activated much more than humans typically utilize it, especially around the foot-off phase. This increased activation of the RF in the gait data generator is related to the superior motor performance of the human controller, mentioned earlier. Humans tend control the RF together with the vastus lateralis and vastus medialis, as these muscles are located close to each other and work in coordination, rather than using independently. Consequently, human activation data shows that these three muscles are activated in similar patterns. However, the RF, being biarticular mus- cle, can move two joints and more metabolically efficient when activated alone. This provides an incentive for the controller to actively utilize it. To investigate this, we trained the human con- troller while artificially deactivating the RF, and while kinematics remained nearly unchanged, we observed slightly higher (~~ 4%) CoT (See Figure L.8 Therefore, address the excessive use of the RF, an additional regularizer that reflects the limitations of human motor control, such as synergy
control (Tresch et al. 1999 Ting & Chvatal 2010 Banks et al. 2017: Shourijeh & Fregly 2020).
should be applied.
Normalized Cost of Tranpport
1.20
m= 11 CoT=1.038
1.05
n=11
CoT-
0.90
(Proposed) With Rectus femoris
Deactivated
Reclus femoris
Figure L.8: Inclusion of rectus femoris in the model reduces normalized cost of transport compared to the deactivated rectus femoris condition.
M
GROUND REACTION FORCE OF UNASSISTED GAIT
Vertical
Medial-Lateral
Prottirr-oottrrior
0.08
Human exp.
1.2
0.3
Simulation
BW %%)
0.00
0.6
0.0
-0.08
0
50
100
50
100
50
100
Stance (%)
Figure M.9: Comparison of simulated and experimental ground reaction forces (GRF).
Figure M.9 compares the simulated and experimental ground reaction forces (GRF) Horst et al. 2021) over single stance cycle, with all forces normalized by body weight (BW) for consistency with the reference human data. Overall, the simulated GRF profiles exhibit characteristic patterns similar to those observed in human locomotion. The primary discrepancy appears in the vertical component, where the simulation produces a sharper peak than the experimental measurements. This difference likely arises from structural modeling: while the human foot features arched and compliant architecture that distributes load more gradually, our simulation employs a box-shaped rigid element, resulting in more abrupt transfer of force during stance.
23

---

N ALTERNATIVE HEI REWARDS
Assist Maximization: Encourages the agent to align with assistive power by promoting
positive power contributions:
THEI
Passist
Σ
max
(20)
Z
#E(L,R)
No HEI Reward: With THEi 0, the human is modeled as minimizing metabolic en- ergy expenditure without incorporating any additional considerations related to exoskeletal assistance.
24

---

0 HEI MAKES HIGHER METABOLIC REDUCTION RATE THAN THE ALTERNATIVE MODELS
Figure demonstrates that our HEI reward accurately predicts the metabolic reduction rate (MRR) achieved through exoskeleton assistance. This finding may appear counterintuitive, the HEI ward does not explicitly regulate metabolic energy expenditure. Furthermore, one might expect the HEI reward impede metabolic reduction by exoskeleton, given its potential competition with T Tenergy, which directly minimizes metabolic energy consumption. To investigate this apparent para- dox, we analyze the relationship between gait parameters and their metabolic benefits follows:
Benefit CoT 0) min min(CoT(n: 0))
(21)
Figure O. 10 reveals that the maximum normalized Benefit remains consistent across different HEI reward configurations. While the mean values are nearly identical across reward conditions, the gait parameters at which these maxima occur differ significantly. In the absence HEI, the max- imum Benefit coincides with the highest cost of transport (CoT) values. This observation aligns with expectations, as Tenergy primarily aims to reduce MEE, leading the human controller to learn exoskeleton utilization strategies parameters corresponding maximum CoT.
In contrast, when our proposed HEI reward is employed, the maximum benefit occurs large gait parameters (Step frequency=2.5 Hz, Step length=0.6 m, walking speed=5,2 km /h) than former. This phenomenon can be attributed to the enhanced effectiveness of the HEI reward higher walking speeds, where exoskeleton assistance power is relatively elevated. Consequently, the human con- troller learns to exploit exoskeleton assistance more effectively under these conditions, resulting in higher measured MRR values despite the lower baseline COT. This finding is consistent with human experimental studies (Lim et al. 2019b). which report maximum MRR at relatively high walking speeds of 4 km/h.
THEI
Cassist
no FHEI
2.5
0.80
0,80
0.80
Coll(
0.40
0.40
0.40
0.20
1.20
L.50
1.80
2.10
2.40
1.50
1.80
2.10
2-40
1.50
1,50
2.10
2.40
(max]
0.32, (mean) 0.07
(mxx)
0.17, (mean) 0.07
(wDX))
0.34.
(mean)
0.07
0.90
0,80
0,800
Step longth (m)
0.60
0.60
Benelit lit
0.1
0.20
0,20
0.20
0.01
1.50
1,50
2.10
2,40
1.50
1.80
2.10
1,50
1,80
2.10
2,40
(max)
22.5%
(mxx) 12.4%
(max)
11.1%
0.60
0.60
0.60
MRR
10
0.20
0,20
11200
1.50
2.10
1.50
1.50
2.40
Step frequency (Hz)
Figure 0.10: Comparison of the simulated cost of transport (top), Benefit (middle) and metabolic reduction rate (bottom) between our proposed HEI reward(r and the alternatives (Tassists no THEI).
25

---

P OPTIMAL EXOSKELETON GAIN TRENDS IN PATHOLOGICAL GAIT
P.I. PATHHOLOGY--PPECIICC TRENDS IN OPTIMAL GAIN
Although the absolute scale of the optimal gain differed between pathological gait data generator, the gains consistently follow severity-dependent trends. This suggests that pathology-specife gain patterns can serve as distinguishing characteristics, since the gain trend is less sensitive to surrogate hyperparameters than the absolute gain value. Based on these trends, we identified two distinct groups reflecting fundamentally different assistance requirements:
Group A (Benefit from hip exoskeleton). Crouch and calcaneus gaits show increasing optimal gains with severity. Crouch gait, caused by psoas contracture, maintains stable but energetically expensive postures with sustained knee and hip flexion. Similarly, calcaneus gait from triceps surae weakness results in meel-walking relying heavily on hip flexion compensate for the inability to plantarflex the ankle. Both conditions feature stable but metabolically demanding gait patterns where exoskeleton assistance effectively reduces energy expenditure without compromising stabil- ity.
Group B (Limited benefit from hip exoskeleton). Equinus and waddling gaits exhibit decreasing optimal gains with severity. Equinus, resulting from triceps surae contracture, causes inherently unstable tae-walking where external forces exacerbate balance issues. Waddling gait, stemming from gluteus weakness, involves lateral trunk sway for pelvic stabilization, where compensatory strategy that external hip assistance can disrupt. These pathologies prioritize maintaining precarious stability over energy optimization, making them less amenable to hip exoskeleton intervention.
Footdrop presents unique with no clear severity correlation. While hip exoskeleton assistance was expected to benefit the compensatory high-stepping pattern, the frequent toe-ground collisions during toe-off introduced substantial gait variability, preventing stable optimization convergence. Detailed discussion is provided in Appendix P.2
This classification reveals fundamental principle: pathologies with stable but energy-intensive compensation patterns (Group A) benefit from increasing assistance, while those with unstable com- pensation mechanisms (Group B) require minimal intervention to preserve their delicate balance strategies. Thus, effective exoskeleton prescription depends not on pathology severity alone, but on the underlying stability-energy trade-off inherent to each condition's 's biomechanical adaptation.
P.2
THE CAUSE OF UNCLEAR TREND OF FOOT DROP IS EXCESSIVE GAIT VARIABILITY OF FOOT DROP
Foot drop did not follow a clear trend due its inherently unstable gait dynamics. This pathology, characterized by insufficient toe clearance from tibialis anterior weakness, leads to frequent toe- ground collisions that substantially increase gait variability. As shown in Figure P.I.T severe foot drop exhibited coefficient of variation (CV) in CoT nearly four times higher than that of other pathologies. While most conditions maintained consistent CV values across severity levels, foot drop's ss variability increased dramatically with severity. This elevated stochasticity in the simulated gait, arising from unpredictable toe-catching events, obscures any systematic relationship between pathology severity and optimal exoskeleton gain.
Coefficient varistion (%)
4.0
3.2
n-10
2.4
1.5
D=S
TO-10
7IF
0.8
S
0.0
Foot drop (severe)
Foet drop (mild)
Caleaneous (sevore)
Equimus (severe)
Crouch (sevore)
Widdding (severs)
Calcueneus (mild)
Equius (mild)
Crauch (mild)
Wadsllngg (mild)
Figure P.I1:: The foot drop pathology exhibits the highest coefficient of variation among all pathol- ogy types.
26

---

P.3 EFFECT OF HEI REWARD ON OPTIMAL GAIN TRENDS
The HEI reward (THEI) proved essential for capturing clinically relevant exoskeleton optimization outcomes. For example, calcaneus gait, which should benefit from assistance due to its high hip muscle demands, only showed the expected positive correlation (Group A) when THE included. Without this reward term, calcaneus incorrectly appeared in Group B with decreasing gains, despite maintaining strong linearity. This demonstrates that THET is necessary component for revealing the true biomechanical characteristics of pathological gait, ensuring that optimization captures the actual assistance needs.
Calcaneuus 0.82)
Calcancus (wei)
Averagei optimall gains K (Nm)
20
15
15
10
Mild
Severe
Mild
Severe
Figure P.12: Our proposed HEI reward reverses the average optimal gain trend with respect to pathology severity.
P.4 R VALUES FOR LINEAR FITS OF OPTIMAL GAIN ACROSS PATHOLOGY SEVERITIES
In the Table P. we examined the linear relationship between optimal exoskeleton gain and pathol- ogy severity by calculating Pearson's 's Ra values across different random seeds.
Category
Seed
R²
Category
Seed
R²
seed] 0.972
seed1 0.944
seed2 0.914
seed2 0.589
Equinus
Calcaneus
seed3 0.787
seed3 0,885
seed4
0,941
seed4
0.845
seed1 0.574
seed1 0.509
seed2 0.930
seed2 0.711
Waddling
Foot drop
seed3 0,991
seed3 0,808
seed4
0.825
seed4
0.321
seed1 0,949
seed1 0.819
seed2 0.611
Calcaneus (no THEI)
seed2
0.845
Crouch
seed3 0.875
seed3 0.923
seed4
0.956
seed4
0.708
Table P.6: Pearson's R² values for linearity between optimal exoskeleton gain and pathology sever- ity.
27

---

Q ANTHROPOMORPHIC CONSISTENCY BETWEEN REAL AND SIMULATED DATA
To validate the consistency between simulation and experiment, we confirmed that the anthropomet- ric parameters of our simulated character match those of subjects in the real datasets. As shown in Table the model's height and weight fall within the natural variability of the populations in all referenced datasets, indicating that these datasets appropriate for in comparisons.
Dataset
Height (cm)
Weight (kg)
n=
Simulation
169
72.9
Kinematics (Boo al.
2025
173.2 + 5.6
74.1
12.5
120
GEMS (Lim et al.
2019b)
173 = 8.2
68.4 6.3
5
GRF Horst ct al.
2021)
176 9
70.7 = 12.0
350
Table Q.7: Comparison between the anthropometric parameters of the simulated character and real human datasets.
R LIMITATIONS OF HILL TYPE MUSCLE DYNAMICS.
The Hill muscle model, which was originally proposed based on experiments on frog muscle in 1938 (Hill 1938). has since been used as the mathematical description of muscle in almost all musculoskeictal simulation studies. this section, we discuss the limitations of the simulation that arise from the known shortcomings of the Hill muscle model.
The contractile force of a Hill-type muscle is formulated as follows:
f= a f(()) (((0)-f())) +
where fi. fu. and fp represent the force-length, force-velocity, and passive force-length rela- tionships, respectively. The variables and a denote the muscle fibre length and activation level. Hill's original experiments imply that these three factors activatiin length, and shortening contty-co independently to muscle force generation. Despite its wide adoption, several important limitations of the Hill model have been identified.
Inability to reproduce history-dependent behavior. Real museles exhibit clear history- dependent behavior. When muscle is stretched, the resulting contractile force exceeds the force produced when the muscle is activated directly the same length. This phenomenon known as force enhancement following stretch. Conversely, active shortening results in force depression. Because the Hill muscle model stateless, cannot reproduce these history-dependent proper- ties (Abbott & Aubert 1952, Nishikawa et al., 2012).
This limitation becomes particularly problematic in the simulation of fast running. During high- speed locomotion, muscles such the hamstrings and calf muscles activated while being stretched at ground contact. In real muscles, this stretch-induce, force enhancement allows them sustain substantially higher forces lower energetic cost, enabling rapid and efficient running. In contrast, Hill-based simulations underestimate the available force and overestimate metabolic expenditure, thereby producing significant sim-to-real gap.
Lack of activatioon-dependent changes in muscle parameters. Experimental studies have shown that key muscle parameters, such as optimal fibre length (Roszek & Huijing 1997) and maximum shortening velocity (Chow & Darling 1999,, depend on activation level the muscle. However, in the Hill muscle model these quantities typically treated fixed constants.
28

---

S QUANTITATIVE COMPARISON BETWEEN SIMULATED AND EXPERIMENTAL DATA
To evaluate the similarity between the simulated and experimental time-series data, we report three metrics: the normalized root-mea--square error (NRMSE), the Pearson correlation coefficient (R), and normalized dynamic time warping (NDTW) distance.
The NRMSE quantifies the overall magnitude error between the two signals and is normalized by the range of the experimental data, allowing comparisons across different variables. Lower NRMSE val- ues indicate better amplitude agreement. The measures the similarity in waveform shape between the signals, independent of scale. Values closer indicate stronger similarity in the temporal pattern. The NDTW distance assesses temporal alignment between the two signals while allow- ing local time stretching or compression. The distance normalized by the amplitude range and warping-pat length, so smaller values indicate closer temporal correspondence.
S.1 UNASSISTED GAIT
Joint
NRMSE
R
NDTW
Ankle Dorsiflexion
0.563
0.892
0.200
Knee Flexion
0.185
0.966
0.108
Hip Flexion
0.210
0.965
0.068
Hip Abduction
0.244
0.777
0.067
Hip Internal Rotation
1,436
-0.443
1.019
Table S.8: Similarity metries between simulated and real joint kinematics.
Muscle
NRMSE
R
NDTW
Bicep Femoris Longus
0.799
0.835
0.266
Bicep Femoris Short
0.533
-0,216
0.344
Gastroenemius Lateral Head
0.531
0.775
0.129
Gastroenemius Medial Head
0.348
0.605
0.057
Soleus
0.419
0.575
0.086
Gluteus Maximus
1.296
0.207
1.254
Gluteus Medius
0.283
0.574
0.025
Vastus Lateralis
0.271
0.634
0.061
Vastus Medialis
0.366
0.649
0.151
Rectus Femoris
5.565
-0.430
2.040
Tibialis Anterior
1.780
0.516
0.395
Peroneus Brevis
0.321
0.823
0.097
Peroneus Longus
0,223
0.931
0.072
Semitendinosus
0.596
0.353
0.117
Table S.9: Similarity metrics between simulated and real muscle activations.
GRF Axis
NRMSE
R
NDTW
Vertical
0.216
0.563
0.057
Medial-Lateral
0.159
0.807
0.039
Prottirr
0.113
0.923
0.028
Table S.10: Similarity metrics between simulated and real ground reaction forces.
29

---

S.2 ASSISTED GAIT
Walking velocity (km/h)
Hip flexion angle
Hip flexion velocity
Exo assist moment
Exo assist
Metric
power
2.0
0,267
0.247
0.210
0.294
3.0
0,241
0.204
0.158
0.290
NRMSE
4.0
0,227
0.187
0.154
0.311
5.0
0,242
0.173
0.111
0.289
2.0
0.896
0.645
0.916
0.328
3.0
0,922
0.775
0,962
0.754
R
4.0
0,887
0.856
0,932
0.507
5.0
0.867
0.927
0,969
0.737
2.0
0,120
0.067
0.089
0.113
3.0
0,106
0.068
0.058
0.105
NDTW
4.0
0.074
0.072
0.051
0.129
5.0
0.066
0.078
0.033
0.102
Table S.11: Similarity metrics between simulated and real kinematics and dynamics.
Data Type
7HEI
Tassist
"HEI
RMS Assist Moment
0.964
0.745
0.726
Mean Assist Power
0.462
-0.180
-0,082
Mean Resist Power
0.919
0.623
0.589
Table S. 121 Correlation (r) between simulated and real Gumaan-exoskelere interaction.
T HYPERPARAMETERS
T.I HUMAN CONTROLLER
Reward parameters
Wgait
1.0
Wenergy
0.35
Warm
1.0
WHE!
0.1
step
1.581
Ovel
3.162
abead
4.0
sway A
0.816
aarm
1.0
Ay
0.018
Ap
0.0105
0.045
Speivis
10"
8
spine
3
12"
k Kenergy
0.2
Raliee
0.1
PPO parameters for PDN
Batch size
8192
Ir
5e-5
log klaseff
1.0
?
0.99
0.99
eclip
0.01
kharget
0.01
MCN learning parameters
Nopoch
5
br
1c-4
WFreg
0.01
WIMR
0.1
T.2
EXOSKELETON OPTIMIZER
8p
1.0
Agrad
5e-2
AL Irint
5e-4
A
5e-4
Epoch
le4
Hidden
[256, 256]
0.1
Irent
5e-4
30

---

U NOMENCLATURE
U.1 HUMAN CONTROLLER AND EXOSKELETON
Texo
Hip assistive torque generated by the exoskeleton controller.
K
Gain scaling the magnitude of exoskeleton torque.
At
Time delay of the delayed-outpe
feedback controller.
u(l)
Control signal encoding relative hip motion.
Or, 01
Low-pass filtered right and left hip joint angles.
s
Full simulation state.
Sskeleton Smuscle
State of the musculoskeletal skeleton (kinematics and dynamics). Muscle-related state (forces, bounds, and dynamic parameters).
Sexo
Exoskeleton state, including recent assistive torques.
qd
PD target joint positions produced by PoseNet. Muscle activations applied to the musculoskelctal model. Joint torque resulting from muscle activations computed by the simulation model.
a
r(a)
TPD
Output torque of the PD controller (before subtracting exoskeleton torque).
TMCN
Target joint torque fed into the MCN to compute muscle activations.
U.2 RL REWARD
Tintal
Total reinforcement learning reward for the PoseNet.
gait
Gait tracking reward composed of step, velocity, head, and sway terms.
Tstpp
Reward encouraging target step length.
Tvel
Reward encouraging target walking speed.
Thead
Reward encouraging stable head orientation and motion.
Tsway
Reward penalizing abnormal body sway.
Tarm
Upper-arm imitation reward encouraging natural arm posture.
Tenergy
Reward regularizing metabolic energy expenditure.
THEI
exoskello interaction reward based on resistance minimization.
h
Current foot position.
hidesired
Target foot position corresponding to the desired step length.
Average forward walking velocity of the character.
Udesired
Target forward walking velocity.
Ostep, vel
Scale parameter for Tstep. vel
0head
Head orientation with respect the progression and global up directions.
Olin
Linear acceleration of the head.
Oang
Angular acceleration of the head.
Kalive
Baseline term preventing full suppression of the head reward. Thresholds for head angle, linear acceleration, and angular acceleration.
Xr, Auy Au
Tbead
Scale parameter used in the head-stability reward.
Orientation angle of body segment b.
Gh
Reference body angle from normative gait data. Allowed deviation threshold for segment b.
8n
Set of body segments considered in the sway reward. Sway reward term for body segment b.
T
ij
Reference joint angle of upper-arm joint j in the reference motion. Current joint angle of upper-arm joint
Oj
Farm
Scale parameter for arm imitation error.
Ps
Mechanical power applicd by the exoskeleton on side k E {L, R).
U.3 RL ACTION
31

---

Phase increment controlling advancement delay of the gait phase.
AM
Residual pose displacement applied to the reference pose.
Gait phase at time step
M,
Target pose for the PD) controller at time step 1. Reference gait motion from which residuals applied.
Miet
+
Operator combining a reference pose with a residual displacement.
U.4
MCN LOSS AND INTRAMUSCULAN REGULARIZZTION
LMON
Total loss for training the muscle coordination network. Intramuscular regularization loss enforcing coherent muscle activations. Weight for activation regularization in the MCN loss.
LIMR
Wreg
WIMR Ng
Weight for the intramuscular regularizer term.
Number of anatomical muscle groups.
Gg
Index set of line muscles belonging to group
0g
Mean activation over all line muscles in group g.
U.5
METABOLIC ENERGY
MEE
Metabolic energy expenditure.
m, / a4
Mass Activation associated with the i-th muscle.
a, B
Exponent parameters governing effort- and like behavior in the MEE model.
CoT
Metabolic cost of transport (energy per unit distance).
L f Teye Benefit
Step length in the PWS evaluation routine.
Step frequency in the PWS evaluation routine.
Time interval corresponding to one gait cycle.
Improvement in CoT due to assistance
U.6 EXOSKELETON OPTIMIZER
X
Simulation parameter space. Sampled parameter point from X.
B
Buffer of parameter-Colf pairs collected from simulation.
c
Exoskeleton control parameter vector.
Optimal exoskeleton control parameter identified by the optimizer.
fc(c)
Surrogate network predicting CoT from control parameters
Laurrogate
Training loss for the surrogate network.
9
Surrogate prediction of COT.
y
Ground-truth CoT obtained from simulation.
Lsn
Huber loss.
V.Ú
Gradient of surrogate output with respect its input parameters. Coefficient for the gradient penalty term.
Agrad
Apt. AL2
Coefficients for L1 and L2 regularization on network weights.
wi
Individual weight parameter of the surrogate network.
Discrete gait parameter (e.g., walking speed) index n. Optimal exoskeleton gain corresponding gait parameter In-
Kn
At,
Optimal time delay corresponding to gait parameter 9..
Ar Xa
Regularization weights in the final exoskeleton optimization objective. Number of discrete gait parameter settings used in the optimization.
M
U.7 OTHER HYPERPAR AMETERS
Batch size
Number of samples per PPO update batch.
32

---

Ir
Learning rate (PPO MCN, depending context).
log a
Log standard deviation of the action distribution.
X
Discount factor in PPO.
GAE parameter for advantage estimation.
Cclip
PPO clipping parameter.
klooeff
KL penalty coefficient in PPO.
klaaget
Target KL divergence in PPO.
Nepuch
Number of epochs for MCN training. Weight of the IMR term the MCN loss.
WIMR
Epoch
Number of training epochs for the surrogate network.
Hidden
Hidden-layer dimensions of the surrogate network MLP.
Ummit/ Tend
Initial Final learning rate of the optimizer schedule.
33


Reviews:
[{"review_id": "Qka08ziugB", "reviewer": "Reviewer_9K5t", "summary": "The paper introduces a neuromechanical simulation framework that couples a Deep RL–driven gait generator with a stochasticity‑aware surrogate optimizer to select hip‑exoskeleton control parameters, specifically a gain without human-in-the-loop experiments. Core ingredients are: (i) a gait data generator (PoseNet + PD control + a Muscle Coordination Network) trained with a composite reward that blends energy minimization and a human-exoskeleton interaction (HEI) term; and (ii) a surrogate network trained on simulated datasets.", "questions": "- How sensitive are optimized (κ,Δt) to the HEI scaling in Eq. (6)? Would normalizing by peak positive device power or by COM mechanical work be more robust across gains?\n\n- Did you test parameterizations where Δt is specified in phase (as a fraction of gait cycle) rather than time, to reduce speed dependence and does the optimizer still recover the same pattern seen in Fig. 8d?\n\nFor more questions, please see the weakness section", "limitations": "", "rating": 4, "confidence": 3, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "- Clear problem framing and a convincing argument for simulation‑first controller selection when HILO is impractical. (Intro; Fig. 1.)\n- Well engineered pipeline (PoseNet/MCN/IMR; LHS + Huber + gradient penalty) with thoughtful ablations and appendices documenting design choices. (Method; Appendix B, D-G).\n- Clinically relevant analysis of pathological gaits", "weaknesses": "- The paper’s scope seems more suitable to a robotics or rehabilitation venue than here. While applications to a particular domain, such as, rehabilitation and robotics are a perfect fit for this venue, the core of this work lies primarily in the exoskeleton control and simulation framework rather than in a methodological advance in/applying deep learning itself.\n\n- Fitting (α,β) to match the human CoT-speed parabola incorporates experimental behavior into the simulator (Algorithm 1–2). This aids realism but complicates claims of prediction vs post‑hoc alignment. \n\n- The HEI term is explicitly engineered to reproduce delay/power scaling observed in data (Fig. 6–7). This is practical, but it risks a form of “trend imitation.” Evidence that other experimentally reported adaptations (not used in shaping) also emerge would clarify generality.\n\n- The surrogate approach is compared mainly to GP‑capacity limits (Appendix D). Modern scalable GP approximations (e.g., inducing points) or kernel ridge with random features would be useful baselines. \n\n- I missed comparison to other controllers ( e.g., the 2‑parameter delayed‑feedback controller to a phase‑based profile with 3–5 parameters)", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "WmumQzIXAx", "reviewer": "Reviewer_MxFv", "summary": "A neuro-mechanical simulation framework is presented which generates realistic gaits for not just able-bodied humans but also for certain pathological conditions. These gait simulations are generated using deep reinforcement learning with a novel reward term that captures the human-exoskeleton interaction. Further, a surrogate network trained from simulation data gathered from the gait generator, estimates a differentiable cost of transport. A gradient based optimizer then consumes the outputs of the surrogate network to compute optimal exoskeleton control parameters given certain gait parameters.", "questions": "1) It's not clear how accurate the gait patterns are for pathological conditions as they are compared with normal gaits (figure J5) and not with real human data with the same pathological condition, is the issue here the lack of biomechanical data? \n\n2) In figure 2, comparison of joint kinematics in simulation - How does the anthropomorphic data such as height and weight of the model (169 cm, 72.9 kg) compare with the real human data?\n\n3) In the videos, the arm motion does not look like natural human motion at all, how much impact does that have on lower-body kinetics? \n\n4) Lastly, I’m curious about the ground reaction forces in a gait cycle. Do they match biomechanics data? These might also be a great indication for how close these simulations are compared to reality. The contact forces would depend on how the contact dynamics is modeled, which might also explain some observations in the gait kinematics.", "limitations": "", "rating": 8, "confidence": 5, "soundness": 4, "presentation": 4, "contribution": 4, "strengths": "The paper is well written and organized. The problem is well motivated by the authors who position it well among existing literature. Bridging the gap between neuromuscular simulation and reality is a highly relevant open problem in robotics. \n\nThe paper also presents interesting formulations - The novel reward term r_{hei} for training the generative GaitNet, intra-muscular regularizer in the loss term to learn Muscle Coordination Network, and the Surrogate Network which estimates cost of transport from simulated data are technically sound ideas and the results and ablation studies around these terms show their benefit very clearly. \n\nThe paper presents convincing and thorough results across the board - the gait kinematics with and without assistance, optimal exoskeleton gain values for disabled individuals, The Cost of transport landscape are explained thoroughly. The ablation studies around the requirement for the r_{hei} reward term are also extremely useful for the reader to understand the necessity of adding this term to model human-exoskeleton interaction. \n\nIn my opinion one of the bigger strengths of the paper are a few insightful discussions presented in the paper that pose very valid questions about the gap in simulation and reality with neuromuscular systems and potentially inspire future research - A few examples are -  \nIn figure 4, RL policy has difficulty producing short step lengths due to the rigid body dynamics and unwanted collisions. \nIn appendix K and figure K.6, the over activation of Rectus femoris because of reduced cost of transport compared to real humans. \nIn appendix N, The results indicate that effective exoskeleton prescription depends on not just pathological severity but also the underlying trade off between stability and energy specific to each biomechanical adaptation is a great insight.", "weaknesses": "As the authors themselves acknowledge, while the results here show promise in closing the sim-2-real gap in neuro-mechanical simulation, the ultimate test is how well this actually works in real world testing, which is still missing. So, a question still remains as to how effective the proposed methodology is? But this does not take away the good work and its contributions in this paper.\n\nSmall observations - \nIn equation 16, it's not clear what M and n stand here. There are a lot of variables described in this paper, it would be great to have a table that summarizes every variable. \n\nIn appendix A, It's not clear where this M_{ref} is obtained from. Also, the policy architecture optimized using PPO is not mentioned anywhere in the paper. It might actually be useful to include learning curves and details of the PPO training  in the appendix for the benefit of anyone trying to reproduce this paper.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "39ognjVaWZ", "reviewer": "Reviewer_xhMH", "summary": "The paper presents a very advanced simulative biomechanical model, called Exo-plore, to study lower limb gait kinematics, kinetics, and muscular activity. Exo-plore allows to simulate also wearable assistive devices such as a hip assistive exoskeleton integrated with the biomechanical model. This allows to study more advanced control strategies for wearable systems. The simulation framework is benchmarked against actual human experimental data showing promising result. The work is novel, well presented, and relevant.\nI would greatly appreciate a better quantification of the differences between simulated and experimental results and a more insightful evaluation of such discrepancies. I honestly believe those difference could elucidate not only the simulative environment limitations but also show some potentially interesting biomechanical features of motor control.", "questions": "1. Is metabolic rate efficiency i.e., energy efficiency, the 'holy grail' of gait? Most of the existing research focuses on developing lower-limb exoskeletons that lower metabolic cost. However, humans do not always plan for most efficient behavior. It might be worth discussing this aspect in the paper. I also recommend having a look at this interesting (and recent paper): https://www.nature.com/articles/s44172-023-00091-2\n\n2. Equation (1) is clear but I struggle to understand why you apply a torque proportional to the difference in sine of the hip angles? Why not applying a torque proportional to the actual difference between the angles? What's the need of the sine function? Moreover, 'k' has the dimensions of a stiffness, thus making your controller equivalent to an impedance control system. Is that correct? If so, I would mention it.\nLast but not least, delayed-feedback control can become unstable. Have you considered this aspect?\n\n3. While I understand the need to have an overall goal for the optimizer, why do most papers focus on metabolic cost minimization? Is this the actual human underlying goal during walking/gait? I think this is a non-trivial question. I don't expect you to answer quantitatively to this point - but it might be beneficial to specify somewhere that the metabolic cost minimization is only one of the possible goals of walking/gait, but other goals might exist.\n\n4. In Figure 4, the step frequency and step length results are also interesting. I invite the authors to compute any form of error, similarity or distance metric between experimental and simulated results. I agree the similarity is good, but I also wonder what the discrepancies are telling us. For example, we can observe that the simulations present the best fit at 4 km/h, while they tend to diverge at the extremes (either higher or lower). Why is that? Is this a bias of the training or is this an indicator that some other aspects of the musculoskeletal system need to be included? (Also, but a minor point, the font of the plot labels (in most figures) is very small - can you slightly increase it?)\n\n5. In page 7, line 377, you write (k,\\Delta_t) = (8 Nm, 0.25s). But wouldn't 'k' represent physically a stiffness i.e., its unit of measurement shouldn't it be Nm/rad?\n\n6. You write that the overall trends in Figure 5 match reasonably well between simulation and human subjects, and this is true especially for the exo assist moment. However, if we look at the hip flexion angle we observe that the simulation does not capture the fact that at the increase of the walking speed the hip flexion angle presents a left-shift. Why isn't the simulation capturing this? Is this something the simulation can or cannot capture? What is the source of this dissimilarity? \nMoreover, the overall exo assist power is underestimated (in the simulation) by almost a factor of 4 or even 5 at times. Why is this happening? Furthermore, why did you not compute any form of similarity metric between experimental and simulated results?\n\n7. Lines 388-389, there are two small typos: (i) \"an downward-opening\" should be \"a downward-opening\", (ii) \"a upward-opening\" should be \"an upward-opening\".\n\n8. Line 978-979. Can you clarify why do you need to add a minimum mass constraint to stabilize the simulation? This aspect is unclear.\n\n9. Table I.4. Please see my previous comment on the unit of measurement of ‘k’. Shouldn’t that be a stiffness? If not, please explain why.\n\n10. Line 1121-1122. I believe there is a typo in the title of section M. Shouldn’t it be “Reduction” and not “Recution”?", "limitations": "", "rating": 10, "confidence": 4, "soundness": 4, "presentation": 4, "contribution": 3, "strengths": "1. The Exo-plore simulation-based framework represents a notable step forward in the development of biomechanical simulation environments that reduce the sim2real gap. The framework demonstrated promising results when benchmarked against experimental data despite the complexity of the musculoskeletal system.\n\n2. The possibility to simulate assistive devices in combination with biomechanical models is also another promising result. Fine-tuning exoskeletal control parameters is often hard, and Exo-plore might help developing faster and more reliable controllers.\n\n2. I find convincing the definition of the total reward, and I especially appreciate the level of detail and reasoning behind it. Often times, papers neglect the proper presentation of the rationale behind the reward function. However, reward shaping is fundamental and has a very-well known impact on the performance of learning algorithms.\n\n3. The idea of the resistance-minimization hypothesis is compelling and somehow novel in this context. I found this a strength of the proposed work, which differentiates it from the state-of-the-art.\n\n4, The paper is clearly presented, and it provides a significant step forward in the development of realistic lower-limb musculoskeletal models capable to reproduce multiple gait patterns and different assistive scenarios.", "weaknesses": "1. Authors could improve the literature on human-in-the-loop optimization for exoskeletons. The group of Prof. Elliott Rouse at the U. Michigan has done extensive work on the topic on both prostheses and exoskeletons, but I could only find 1 work from his group in your references. Consider extending your literature review and include also this relevant portion of literature.\n\n2. In neuromechanical simulations, authors should briefly mention the fact that Hill-type muscle actuators are an approximation of the actual muscle behavior. More specifically, Hill never designed the muscle model to be used for complex musculoskeletal models but only to investigate the force-speed relationship in human individual muscles. As a consequence, many other features of muscle dynamics are not captured by it. Nonetheless, most musculoskeletal models employ Hill-type muscle models ''carelessly'' believing that it gives a complete representation of human behavior. This aspect can lead users of such models to incorrect insights on the actual human musculoskeletal system.\nPlease consider mentioning this aspect in your digression.\n\n3. It would be nice if you could provide some measure of similarity (or distance) between the simulated and experimental behaviors - both at the joint and muscular level, as well as for the other results presented in the paper. For join trajectories, I agree that the hip internal rotation is the one presenting the largest difference, while the other joint angular trajectories are closer to the behavior of human subjects. Nonetheless, some differences might be physiologically relevant. For example, in Figure 2 - Knee Flexion panel, you can observe that the simulation generates: (i) an overall smaller knee flexion peak during swing, and (ii) a practically negligible knee flexion during stance. Why is that? The knee flexion peak during swing is fundamental in normal walking to reduce the risk of stumbling and potentially falling, while the knee flexion peak during stance is important for shock absorption purposes. Is there any insight on why the model is not able to capture these features? Analogous observations could be done for the ankle dorsiflexion or hip abduction. I would greatly appreciate if you could comment more extensively on these results. (Similar comments are reported in the Questions also for Figure 5).", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "IF4m4kx2FL", "reviewer": "Reviewer_BgQH", "summary": "This paper proposes Exo-plore, a simulation pipeline to optimize exoskeleton controller parameters without real human experiments. In Exo-plore, the authors train a gait generator to generate gait trajectories with exoskeleton assistance. Then they train a surrogate model with the generated gait data under a set of controller parameters sampled by Latin hypercube. The final control parameter is selected by optimizing over the surrogate model. The experimental results demonstrate the qualitative fidelity of the gait generator in both joint and muscle levels compared with real human experiments, and the optimization results in both healthy and disable conditions.", "questions": "1. In Algorithm 1, how many candidates $(\\alpha, \\beta)$ are evaluated during experiment?  What is the training time of a single round of gaite generator training?\n\n2. In Algorithm 2, how many available $(L, f)$ are evaluated?\n\n3.  How to set the hyperparameters of the defined reward and loss functions in practice?\n\n4. Do the body parameters of the used musculoskeletal model match the human with real experimental data (e.g. height and weight)?\n\n5. The whole optimization problem is a 2d black box optimization problem, where GP-based Bayesian optimization can usually converge within 100 samples even under noisy function evaluation. Why did you generate over 60k data to train a surrogate model in Appendix D.2? I think this amount of data is sufficient to delineate a 2d function landscape without any further surrogate fitting.", "limitations": "", "rating": 2, "confidence": 4, "soundness": 3, "presentation": 2, "contribution": 2, "strengths": "1. This paper aims at conducting exoskeleton control optimization in pure simulation with biomechanical fidelity by using a musculoskeletal-based controller, which is well motivated and will benefit safer and more efficient assisted control optimization.\n\n2. The experimental results is comprehensive with video demonstration, which demonstrate the qualitative fidelity of the gait generator and the optimized controller parameters. The limitations are also mentioned.", "weaknesses": "1. From the text description and Figure 1, the whole optimization pipeline seems not a closed-loop. During the training of the gait generator, the model is fit to a certain control parameter by maximizing $r_{HEI}$. This control parameter is not optimized. When training the surrogate model, different control parameters are fed to the gait generator to get gait data. I think the data collection pipeline is unreasonable to me, because control parameters sampled by Latin hypercube are not in the training distribution of gait generator, which may lead to lower $R_{HEI}$ and unknown gait generating performance. \n\n2. Algorithm 1 seems very time-consuming, which requires whole training round of gait generator for each reward parameters.\n\n3. The reward and loss terms in e.q. (2), (3), (4), (7) have many hyperparameters. It may be hard to determine these hyperparameters without real human experimental data. It may not be practical to apply the proposed method in pure simulation.\n\n4. According to Figure 8 (b)-(d), the trained surrogate model seems sensitive to the weight of gradient penalty loss, where the optimized control parameters have distinct discrepancy. The \"smooth landscape\" shown by the fitted surrogate model might also be accessed by Gaussian process with much fewer data under proper kernel function.\n\n5. (Minor) The writing can be further improved to be clearer. e,g. (1) In Algorithm 1 and 3, There is no formal definition of \"trainDataGenerator\" and \"EvalCoT\" in the main text. (2) In line 390: there is no formal introduction of $r_{assist}$.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)