# A Novel Hybrid Path Planning Method Based on Q-Learning and Neural Network for Robot Arm
## 2 Method Description
### Angle-Finding by Trained Neural Network (Passive)  
The second component, angle-finding, provides joints angles of each selected action in a particular cell by a trained neural network. In other words, after finding the appropriate actions in the action-finding section, it is time to obtain the joints angles for each selected action in each cell to be understandable for the robot arm. This component of our hybrid path planning is called the passive phase because we use a trained neural network, which is a static process in contrast to a dynamic process like Q-learning. In other words, the neural network does not need to be trained every time; instead, it is trained only one time, but that trained neural network is used every time during path planning. Indeed, during this part of our hybrid path planning, the robot uses the passive section of memory without needing to do any learning or training processes, which are time-consuming. Before definition the type of neural network used in this paper, it is better to discuss other angle-finding methods because it can help to understand the advantages of using neural networks. One of the most common approaches in the classic methods to find joints angles is inverse kinematics (IK). The idea behind IK is to calculate the joints angles of the robot arm for a given position of the end-effector. To accomplish motion with this approach, one would first compute a path for the end-effector from the initial position to the target position. Then, with IK, the joint configurations needed for each intermediate time step along the trajectory are calculated. However, using IK in each step makes the method extremely slow. Considering the article's aims, which are improving slowness and improving complexity, the IK is not a suitable one due to its slowness. Another way of angle-finding is to use table lookup. In this method, a table containing states in its rows, actions in its columns, and angles in its heights is used. Nonetheless, this method is still inefficient because, in high resolution of moving (i.e., more cells), a large table should be provided and saved. It is blatantly apparent that using a large table can make the method slow, which slowness is out of the aim of this article. Given the disadvantages of IK and table lookup methods presented in previous paragraphs, we use a neural network in the angle-finding phase. In this paper, we train a neural network to use in the angle-finding section. The proposed neural network is trained by using a data set. A data set usually is collected by a real robot in experimental environments. Yet, in the cases that a robot arm there exists in the simulator's library, one can use simulator software to collect data. However, it should be noted that for high resolution of the grid, in which the distance between cells is small, using the real robot in experimental environments is recommended because the error between the simulator and the real robot is no longer ignorable. In this paper, we collected the data by means of the RoboDK simulator because our robot arm exists in the RoboDK library, and also our resolution of the grid is relatively low. Since required joints angles of moving toward each direction are dependent on the current state of the end-effector, in each step, that current cell must be taken into account in the neural network. Therefore, the indexes of the cell's row and column and the selected action should be the inputs of the neural network, and then the joints angles that provide the position of the next cell corresponding to the action are its outputs. It is worth mention that since the input must be in numerical form, the actions including up, down, left, and right are rewritten numerically as 1, 2, 3, and 4, respectively. Moreover, the number of outputs is dependent on the degree of freedom of the robot arm. In a robot arm which have six joints, the number of outputs is equal to six. The neural network structure we propose has one hidden layer, which has thirty neurons.