# Reinforcement Learning-Based Algorithm to Avoid Obstacles by the Anthropomorphic Robotic Arm
## 3 Results
### Research on the Application of Reinforcement Learning Algorithms to Control a Physical Robot  
Similar tests as in the simulation were also carried out on a Mitsubishi robot in laboratory conditions. This robot is shown in Figure 2a and described in Section 2.3.1. In the case of tests carried out in laboratory conditions, different types of objects are used as obstacles, unlike in simulation. In Figure 10, the robot with the mounted head is shown bypassing different obstacles located in its workspace. During tests, robot was bypassing a cuboid block, a cylinder block, multiple obstacles, and a human. Simulation tests and preliminary laboratory tests showed that the size and shape of the obstacle do not affect its detection and avoidance. Therefore, the obstacle dimensions are different than the size and shape of the obstacle used in the simulation. In order to compare the simulation outcomes with the tests performed in the laboratory, the algorithm was tested for the same 50 start gripper positions and 50 target positions. This comparison was only carried out with a cuboid block as an obstacle (Figure 10a). In Table 2, results obtained during laboratory investigation are presented. Data presented in this table can be compared with data recorded in simulation, given in Table 1. Comparing Table 1 with Table 2, it can be noticed that the results are very similar. The average error \(e\) is equal to 3.91 mm, which differs by 0.07 mm compared to simulation tests. Standard deviation \(\sigma_{e}\) for error \(e\) is higher than in the case of simulation tests and is equal to 2.51 mm. Maximum error \(e\) is also over 8 mm. For individual axes, the greatest error occurs for the \(y\) axis; it is 2.69 mm. The errors for the remaining two axes are below 2 mm. Figure 9: The trajectory on which the TCP of the robot moved in simulation for goal point \(g_{2}\): (**a**) with an obstacle; (**b**) without an obstacle. Figures 11 and 12 show the example trajectories of the TCP of the robot during tests carried out in laboratory conditions while the robot was bypassing the cuboid obstacle. The method of presentation and the colors of the trajectory fragments performed by Agent#1 and Agent#2 are the same as in the case of figures introduced in the section on simulation tests. In the case of the designed head, the obstacle could not be detected in all three planes, as in the case of the simulation. Therefore, in the figures in Section 3.2, the obstacles are only plotted in the \(x\)-\(y\) plane as colored points obtained from sensors. Each laser sensor output signal has a different color; i.e., different colors indicate that the obstacle was detected by another sensor. From this, the shape of the obstacle can be deduced and laser signals used for the detection of an obstacle by the head. Figure 11 shows the trajectory for the target point \(g_{3}\). The \(y\) coordinate of the start position has a negative sign. As can be seen in Figure 11a, at the beginning of the movement, the actions are performed by Agent#2 because the obstacle is inserted into the workspace when the robot is moving, and not at the very beginning of the episode. The TCP of the robot was too close to the obstacle after performing the turning maneuver; therefore, as can be seen during the simulation tests, Agent#1 moves along the obstacle to avoid it. The goal position is very close to the obstacle; therefore, all steps to reach it are taken by Agent#1. In the case when the obstacle is not in the workspace (Figure 11b), the graphs show that the agent for both the \(x\) and \(z\)-axis has a large deviation from the straight line. Figure 12 shows the trajectory for the target point \(g_{4}\). The \(y\) coordinate of the starting position has a positive sign. In this case, oscillations appear, during the obstacle avoidance maneuver, but when the agent passes the obstacle, it begins to move straight to the target without oscillations. When there is no obstacle in the workspace, the deviation from the straight line again occurs for the \(x\)-axis.