# Analysis of Cost Functions for Reinforcement Learning of Reaching Tasks in Humanoid Robots
## 5 Execution of Trajectories Learnt in Simulation on the Real Robot  
Since all solutions computed by RL resulted in stable and safe reaching motions (see Section 4.1), it was possible to use them directly to generate the reaching motions on the real robot. For evaluation, we used the solution computed with weights set to \(w_{pos}=20\), \(w_{ori}=4\), \(w_{stab}=20\), \(w_{traj}=0.0\), and \(w_{acc}=0\).  
Figure 12b shows the comparison between the reaching trajectories executed in simulation and on the real robot. Clearly, the reality gap is small and there is not considerable deviation between the two trajectories. Notably, the execution of the computed reaching motion on the real robot resulted in a similarly stable ZMP trajectory, as shown in Figure 12a. Figures 13 and 14 show the task execution, illustrating the performance in both simulation and on the real robot. Sequences of the execution of the initial demonstration and the optimal motion in both scenarios is shown, confirming the efficacy of the proposed learning approach in achieving precise reaching motions.  
Low discrepancy between the motions in simulation and on the real robot is probably due to the reaching motion that includes only the upper body joint angles. Additionally, no contacts occurred in our experiments. Considering lower body degrees of freedom and tasks that include contacts with the environment, the solution computed in simulation might need to be adapted to ensure its applicability on the real robot. Such an adaptation process can utilize the motion computed in simulation as an initial approximation for the adaptation on the real robot. This should result in faster learning compared to starting from the human-demonstrated trajectories.  
The results showing the reaching accuracy and stability performance are given also in Table 4, where the errors for the final reaching position and orientation of the executions in simulation and on the real robot are presented. Additionally, Table 4 shows the average distance between the measured center of pressure and the center of the support polygon while executing the reaching trajectory. There is an insignificant difference between the performance of the simulated robot and the real robot in terms of reachability. The stability cost shows similar distances from the center of the support polygon in both the simulated and real robot reaching motion. The ZMP trajectories plotted in Figure 12a even exhibit less oscillations when the reaching motion is performed by the real robot.  
Figure 12: Simulation results and the execution on the real robot showing the stability performance and reaching behavior resulting from the initially reproduced motion and the reaching motion computed by RL, with cost function weights set to \(w_{pos}=20\), \(w_{ori}=4\), \(w_{stab}=20\), \(w_{traj}=0.0\), and \(w_{acc}=0\). (**a**) Simulated and real robot ZMP trajectories associated with the initial reaching motion and with the motion toward the new reaching pose computed by RL, viewed in the support polygon of the two feet (left) and as separate \(x\) and \(y\) components of the ZMP over time (right). (**b**) Spatial path of the initial reaching motion and the motions toward the new reaching pose computed by RL, executed in simulation and on the real robot.