# Avoiding Obstacles via Missile Real-Time Inference by Reinforcement Learning
###### Abstract  
In the contemporary battlefield where complexity has increased, the enhancement of the role and ability of missiles has become crucial. Thus, missile guidance systems are required to be further developed in a more intelligent and autonomous way to deal with complicated environments. In this paper, we propose novel missile guidance laws using reinforcement learning, which can autonomously avoid obstacles and terrains in complicated environments with limited prior information and even without the need of off-line trajectory or waypoint generation. The proposed guidance laws are focused on two mission scenarios: the first is with planar obstacles, which is used  
of trajectories is good for collision avoidance or not, and activates the chosen maneuver when the collision comes closer.  
Recently, reinforcement learning (RL) has attracted a lot of attention in the optimization and design of guidance in various fields. [PERSON] and [PERSON] [15; 16] used RL for fixed-wing UAVs to implement collision avoidance tasks. They showed that they reduced the probability of UAVs' collision with many simulation experiments. Furthermore, there are some prior studies for missile guidance via reinforcement learning. [PERSON] [17] argued that RL working under a stochastic environment could make the logic more robust. They presented an RL-based missile guidance law and its framework for a missile homing phase via Q learning. They also presented a framework for interceptor guidance law design, which is able to infer guidance commands with only line-of-sight angles via Proximal policy optimization (PPO) in [18]. However, they dealt with a small and limited environment. [PERSON] [19] expanded the environment further to cover a whole planar environment and set the fair comparison condition. They presented an RL-based missile guidance law for a wide range of environments and showed some advantageous features.  
In practice, some missiles have the ability to avoid anti-missile systems and obstacles. Harpoons, for example, guide themselves to a sea-skimming maneuver to hide from radar detection. In the terminal phase, the missile kicks into the high for pop-up maneuver and prevents the mission failure due to CIWS (close-in weapon system) counteraction. It is also able to avoid the known obstacles such as friendly ships or islets by following the predefined waypoints. Such capabilities of missile guidance could raise the mission success rate, since it makes missile defense systems difficult to properly counter.  
Several algorithms for obstacle avoidance have already been suggested in the literature to guide the missile to the target in complicated environments containing mountainous areas, islets, and ships. They were achieved by following the predefined trajectory, which requires a complete map of the operation field. Their approaches obviously limit the operation environment and requires too much prior information.  
In this paper, we propose novel missile guidance laws using reinforcement learning, which can autonomously avoid obstacles and terrains in complicated environments with limited prior information and even without the need of off-line trajectory or waypoint generation. Our guidance laws are operating in real-time inference with less computational burdens and are also able to determine the probability of mission failure, which provides the missile some time to quit the mission safely when a mission is predicted as a failure.  
This paper is organized as follows. Section 2 explains some basic missile dynamics and discusses environment modeling in which missile guidance laws are trained and operated. In Section 3, we present details of neural network architecture, reward function design, and training methodology. In Section 4, some numerical simulations are provided and the performance of proposed guidance laws is evaluated. Concluding remarks are given in Section 5.