## 2 Design and Methods  
The workflow for this work is demonstrated in Figure 3. The 2022 version of SolidWorks software was used to redesign the leg assembly and to establish feasibility through motion analysis. MATLAB 2022a was used to create a MATLAB/Simscape (Simscape) model that reflects the proposed redesign of a leg composed of three links (upper leg, lower leg, and foot). Reinforcement learning was implemented to train the new leg assembly to follow the mocap data. These results were compared to SolidWorks and mocap data for validation purposes. Figure 2: Original RoboREHAB leg assembly design. The original design of the RoboREHAB leg assembly has the upper-leg section connected to the lower-leg section as a fixed knee joint. The human knee joint is not so much a fixed joint, but rather, a saddle joint that the femur and the tibia form, as seen in Figure 4. The saddle joint allows the proximal end of the tibia to roll and travel along the distal end of the femur. By this logic, the interaction between the ends of the femur and tibia can be considered a small segment with two rollers in contact at the top and bottom of the segment. To emulate this movement, the redesign of the leg assembly involved the addition of a link between the upper- and lower-leg sections, along with two rollers being placed in the assembly to mimic the rolling joint of the knee. This is the modification that needs to be made to the preexisting leg assembly. The inclusion of Figure 4 is not intended for determining which muscle contributes the most to the motion of the leg assembly, but more so for use as a visual reference of the mechanical formation of the knee joint. Figures 5 and 6 present the reinforcement-learning (RL) Simscape model and the Simscape block diagram for the redesigned leg assembly, respectively. Figure 5 is the Simscape model that was used to conduct reinforcement-learning training using the redesigned leg assembly. Figure 6 is the block diagram in the Simscape model which assembles the  
Figure 4: Anatomy of human knee joint [20]. Figure 3: Workflow for this study. redesigned leg and allows for the joints to actuate based on the joint angles prescribed by the reinforcement-learning agent. Figure 7 depicts the variables which are controlled by the RL networks for the redesigned and original leg assemblies. In Figure 7, the variable q represents the joint angle for a specific joint, q1 corresponds to the hip joint, q2 to the joint above the knee segment, and q3 to the joint below the knee segment. The additional m subscript denotes joint angle calculated from the motion-capture data. The Simscape model consists of the redesigned leg model with three reference points at the hip, the midpoint of the second segment, and the ankle. Adjacent to the leg model are the mocap datapoints. The Simscape model features six different signals as seen in Figure 5: kinematic measurements of the leg model (meas) and mocap data (mocap meas), the action requested by the RL block (action), an observation of the state and performance of the redesigned leg assembly (observation), the reward generated by the performance of the leg assembly (reward), and a signal to end the current episode if the threshold distance between the leg model and mocap joints is met (isdone). The reinforcement learner interprets the observations and the reward that it received and performs a new action based on this feedback to maximize the reward it receives. The reward function governing the gait training is illustrated in Equation (1); the variables controlled by the RL agent for the redesigned and original RoboREHAB leg assemblies are depicted in Figure 7.  
\[\text{Reward}=(x_{k}-x_{k,m})-(y_{k}-y_{k,m})-(x_{a}-x_{a,m})-(y_{a}-y_{a,m}) \tag{1}\]  
Figure 5: MATLAB/Simscape model of leg assembly. Figure 6: Leg subsystem for RoboREHAB leg assembly modified with rollers at knee. The logic flow for the algorithm is compiled below:  
1. Initialize angles of the leg segments and load any previous experience if any exists. 2. For \(t=0\): \(T_{s}\): \(T_{f}\) sec. 3.