# Reinforcement Learning-Based Algorithm to Avoid Obstacles by the Anthropomorphic Robotic Arm
## 3 Results  
### Simulation Results  
In Table 1, simulation results characterized by the parameters presented in the previous section for all 50 points are presented. The average position error \(e\) (the distance between the goal point and the TCP of the robot) is equal to 3.84 mm. Standard deviation \(\sigma_{e}\) for error \(e\) is equal to 1.83 mm. The maximum error \(e\) that occurred during testing was over 8 mm in only a single case. The results mainly oscillate around the average error \(e\). For individual axes, the greatest error occurred for the \(y\) axis; it was almost 2.6 mm. The errors for the remaining two axes were below 2 mm. Figures 8 and 9 show the motion trajectories of the robot's TCP in 2D space recorded during simulation tests. Each figure contains two groups of waveforms: with an obstacle in the workspace of the robot--(a), and without an obstacle--(b). These two panels differ in the presence of obstacles, but are made for the same start position of the TCP of the robot and the same target position. The robot's trajectories are generated by two agents; therefore, they have two colors; the points marked in purple are actions taken by Agent#1 (labeled as _a_obs), and the points marked in cyan are actions taken by Agent#2 (labeled as _a_dtr_). The presented trajectories include cases in which the starting position of the robot is on both sides of the obstacle on the \(y\) axis. This means that the \(y\) coordinate of the start position has a positive and negative sign. During the tests, an obstacle occurred in the workspace after several steps of the entire episode. The obstacle was not in the workspace from the beginning of the episode; it is supposed to simulate the sudden appearance of an obstacle and test the algorithm's reaction to such a change in the environment. \begin{table}
\begin{tabular}{c c c c c} \hline \hline
**Avg. \(e\pm\sigma_{e}\) [mm]** & **Max \(e\) [mm]** & **Avg. \(\sigma_{x}\pm\sigma_{e_{x}}\) [mm]** & **Avg. \(\sigma_{y}\pm\sigma_{e_{y}}\) [mm]** & **Avg. \(\sigma_{z}\pm\sigma_{e_{x}}\) [mm]** \\ \hline
3.84 \(\pm\) 1.83 & 8.39 & 1.67 \(\pm\) 1.32 & 2.58 \(\pm\) 1.73 & 1.43 \(\pm\) 1.12 \\ \hline \hline \end{tabular}
\end{table} 
Table 1: Results of simulation tests for the task of TCP positioning of the robot at a given point and avoiding an obstacle. Figure 8 shows the trajectory for the goal position \(g_{1}\). The \(y\) coordinate of the starting position has a negative sign. As it can be seen in Figure 8a, at the beginning of the move, the actions are performed by Agent#2 (cyan--_a_dtr_). After detecting the obstacle, the system switches the agents and the actions are performed by Agent#1 (purple--_a_dtr_); after a few steps, this agent makes moves to avoid the obstacle that has appeared in the workspace. The agent moves away from the obstacle and then starts moving towards the goal position. It can be seen that during the turning maneuver, Agent#2 takes some steps because the distance between the TCP of the robot and the obstacle has exceeded the threshold distance \(d_{obs}\). After moving away from the obstacle, Agent#2 takes control again, which finally positions the TCP in the goal position. When looking at the \(z\)-axis graphs, it can be seen that the algorithm does not keep a straight line. At the beginning of the movement, when Agent#1 takes control, it goes up in the \(z\)-axis, even though the goal point is below the current position of the robot tip. In the case of no obstacle in the workspace (Figure 8b), the task of the agent is to move as quickly as possible towards the goal position. In the figure, some deviation from the straight line can be seen, especially on the \(x\) axis. Figure 9 shows the trajectory for the goal position \(g_{2}\). The \(y\) coordinate of the starting position has a positive sign. As in the previous case, the first steps are taken by Agent#2; after detecting an obstacle, Agent#1 takes control (Figure 9a).