# A Novel Hybrid Path Planning Method Based on Q-Learning and Neural Network for Robot Arm
## 3 Simulation and Experiment Results
### Experiment Results  
After implementing the hybrid method in the simulation world and confirming the safety during its operation, we can implement it using a real robot arm. In this regard, the experimental setup is presented at first. Figure 7 shows the setup used to test the presented method. In this setup, a camera installed at the top of the workspace covers this area. \begin{table}
\begin{tabular}{c c c} \hline \hline \multicolumn{2}{c}{**Angles:**} \\ \hline \([[18^{\circ},47^{\circ},49^{\circ},0^{\circ},-6^{\circ},198^{\circ}],\)\([6^{\circ},47^{\circ},51^{\circ},0^{\circ},-8^{\circ},186^{\circ}],\)\([-7^{\circ},46^{\circ},51^{\circ},0^{\circ},-8^{\circ},171^{\circ}],\)\([-20^{\circ},46^{\circ},49^{\circ},0^{\circ},-6^{\circ},158^{\circ}],\)\([-29^{\circ},46^{\circ},43^{\circ},0^{\circ},-1^{\circ},149^{\circ}],\)\([-34^{\circ},46^{\circ},38^{\circ},0^{\circ},3^{\circ},144^{\circ}],\)\([-32^{\circ},47^{\circ},28^{\circ},0^{\circ},12^{\circ},146^{\circ}],\)\([-28^{\circ},51^{\circ},13^{\circ},0^{\circ},23^{\circ},150^{\circ}],\)\([-25^{\circ},54^{\circ},4^{\circ},0^{\circ},29^{\circ},153^{  
Moreover, the robot arm is fixed on the next left side of the gridded workspace, just like the construction robot in a factory near a conveyor belt in a production line. Additionally, a computer is connected to both the camera and robot arm. All the algorithms that exist in this method, including the KNN algorithm (Image Processing), Q-learning algorithm (RL), and neural network, are performed by the computer. Like the previous section, the image is processed in this part, then the start cell, obstacle cell, and target cell are obtained. Next, using the Q-learning algorithm, the actions are determined. After that, the joints angles are obtained by using the trained neural network. Finally, the found joints angles are implemented in the real robot. Figure 8 shows the real-world test process. As shown in the images, the end-effector moves from the start cell and follows the path; eventually, it reaches the target while avoiding the obstacle. Taking the simulation and real-world test results into account, we prove that the proposed hybrid method is a unique path planning approach that has several advantages and resolves the shortcomings of previous path planning methods. This hybrid method has two different active and passive phases to reduce complexity and improve slowness. Unlike the previous methods, which directly tried to find joints angles, this hybrid method  
Figure 8: Real-world test. **(1)** Initial position of end-effector before starting the process. **(2)** The robot starts moving from the start point in yellow. **(3–5)** The robot track the found path while avoiding the obstacle. **(6)** The robot reaches the target in green. Figure 7: Experimental setup: camera, robot arm, workspace, yellow start, red obstacle, green target, computer. first uses simple cells and actions to find optimal actions. Second, this method uses a trained neural network to obtain required joints angles concerning the actions in each cell of the path found in the active phase. The combination of these two active and passive phases is the new approach proposed in this article, which significantly increases the speed of robots in a production line and improves the complexity of path planning. As a result, it reduces computational costs and increases the speed of manufacturing, which causes factories to take advantage economically and perform efficiently. To understand the speed of the proposed hybrid method, Figure 9 shows the running time of codes, including active phase (Q-Learning) and passive phase (Neural Network), as well as total running time, which is actually the sum of them. These times have been plotted in different numbers of cells of the gridded 2D workspace (i.e., different resolutions). As we highlight in the previous section the neural network completed its training, and here is the time of using trained weights in the joint angles finding phase.