# A Novel Hybrid Path Planning Method Based on Q-Learning and Neural Network for Robot Arm
## 1 Introduction  
Currently, the use of robot arms has dramatically increased in various applications, including manufacturing, logistics, medicine, home, education, defense, and factories. due to their plentiful benefits such as high quality, fast production, less waste, and excellent safety (removing employees from hazardous working conditions, and handling heavy parts). Because of their crucial role, path planning, including target reaching and obstacle avoidance, is essential. Therefore, this article tries to address the issues of path planning methods to improve the efficiency of their operation. In this regard, since all robot arms' activities are dependent on their end-effector, target reaching (i.e., getting the end-effector to the desired position) is a crucial task for them. Additionally, obstacle avoidance is another equally vital task for these robots because robot arms must avoid self-collisions and any objects within their workspace. For the past decades, scholars have presented many algorithms in robot arm path planning to improve those tasks. Artificial Potential Field (APF), Probabilistic Road Maps (PRM), Rapidly exploring Random Tree (RRT), Reinforcement Learning based (RL-based) are some of the well-known examples. In this regard, some of the authors used the APF method for path planning. [PERSON] et al. presented a whole-arm path planning algorithm for a 6-DOF industrial robot based on the artificial potential field method [1]. [PERSON] et al. proposed a path planning method and gave an idea for collision avoidance by a three-dimensional artificialpotential field for a robot arm [2]. However, this method can become trapped in the local minima of the potential field and fail to find a path or find a non-optimal path [3]. Some other authors used the PRM method for path planning. [PERSON] et al. presented a path planning algorithm for robots based on PRM [4]. Nevertheless, this approach is time-consuming and requires re-planning when the initial position changes [3]. Some others presented an RRT algorithm for path planning. [PERSON] et al. proposed a dynamic path planning method for robotic manipulator autonomous obstacle avoidance based on an improved RRT algorithm [5]. [PERSON] et al. proposed an improved RRT algorithm by introducing the random point generation mechanism based on random probability. Nonetheless, this method has many problems, including cubic graphs, irregular paths, and non-optimal paths [6]. That is because the structural nature of these graphs hinders the probability of finding an optimal path. Additionally, like the PRM, this method is time-consuming and requires re-planning when the initial position changes [3]. Since the reinforcement learning methods have the advantage of high computational efficiency, some authors used RL-based methods to address path planning limitations. [PERSON] et al. presented a new obstacle avoidance algorithm based on an existing deep reinforcement learning framework called deep deterministic policy gradient (DDPG) [7]. [PERSON] et al. proposed a Soft Actor-Critic-based path planning algorithm to overcome RL-based path planning difficulties [8]. [PERSON] et al. proposed a modular method to efficiently learn and transfer visuomotor policies from simulation to the real world. They combine domain randomization and adaptation. In fact, in that paper, they could generate trajectories to control the arm with a random initial configuration to reach a target arbitrarily placed in the operation area, regardless of obstacle avoidance [9]. [PERSON] et al. used approximate regions instead of accurate measurements to define new state space and joint actions in the Q-learning method [3]. Nevertheless, some other researchers prefer not to use RL-based methods for solving the robot arm path planning problem because it has to take each joint's motion into account, which makes them slow and complex. Other authors presented some unconventional methods, for example, polynomial path planning method.