# A Novel Hybrid Path Planning Method Based on Q-Learning and Neural Network for Robot Arm
## 1 Introduction
In this method, the end-effector positions can be determined by a polynomial from the start point to the final point. Then, the joints angles can be obtained by using inverse kinematic. [PERSON] et al. presented a real-time path planning method to avoid collision with obstacles for a fixed robot arm that combines the benefits of both global and local techniques. This method is based on the idea of configuring the robot arm to adjust its joints on a selected polynomial to solve the inverse kinematics problem [10]. [PERSON] et al. proposed a path planning method to avoid collision with obstacles, based on non-uniform rational B-splines and a smooth polynomial [11]. However, these methods have many problems, including high computational costs and time-consuming processes because to find the joints angles, the inverse kinematic should be used in each step, which makes the algorithm too slow. All in all, there are many problems in the methods mentioned above, including failing to find a path, finding a non-optimal path, complexity, time-consuming, requiring re-planning, cubic graphs, irregular paths, and slowness. Since slowness and complexity can negatively affect the performance of robot arms and thus production, finding a new path planning method to address these two limitations is essential. Solving these limitations can significantly reduce the computational cost of robot arms' path planning. More than this, it can also increase the speed of production, which has many economic benefits for factories. Indeed, the faster production, the more efficient the costs are spent by the company for the production process. For these reasons, in this paper, we present a novel hybrid path planning method based on Q-learning and a neural network to improve the slowness and complexity. In the proposed approach, we divide the path planning process into two separate phases, active and passive. In the first part, using the Q-learning algorithm and inspired by the \"windy grid world\" problem, a sequence of simple actions is found in a griddedworkspace so that the robot arm begins moving from the start cell to reach the target cell while avoiding collision with an obstacle. In other words, the robot arm learns how to get the target and avoid collision with the obstacle in a simplified agent-environment interaction. A simplified RL problem reduces the complexity and slowness due to using a simple environment, states, and actions. The states are the cells in the gridded workspace, and the actions are moving up, down, left, and right. Although this simplification makes the process fast, the results of this process, such as up, down, left, and right, cannot be used in the robot arm, so the second part is used to find useable actions. In the second part, using a trained neural network, the required angles of the found actions are obtained. It is worth mentioning that it does not need to run a time-consuming neural network process because its weights were already trained; thus, it saves computational time. The first part, action-finding, is called the active phase because, in this part of the hybrid path planning, a learning code is run, which takes time to learn. In other words, during this part of our hybrid path planning, the robot uses an active section of memory and computer brain that needs to do a learning process, which is a time-consuming process. On the other hand, the second part, angle-finding, is called the passive phase because, in this part of path planning, the neural network does not need to be trained every time; instead, it is trained only one time and is used forever. In other words, during this part of our hybrid path planning, the robot uses a passive section of memory without needing to do any learning or training processes, which are time-consuming. The combination of these two active and passive phases reduces the complexity and improves slowness.