# Automated Enemy Avoidance of Unmanned Aerial Vehicles Based on Reinforcement Learning
## 3 Problem Definition
### Problem Description  
For the convenience of the research, we define the enemy avoidance problem in a fixed region. There are two UAVs flying toward each other in the region, namely the decision making UAV \(f\) and the enemy UAV \(e\). The fixed region is where the two UAVs may collide with each other. The decision making UAV \(f\) enters the region from the left side, while the enemy UAV \(e\) enters the region from the right side. Both UAVs are flying toward their own goal points. Let \(G_{f}\) and \(G_{e}\) denote the goal point of the decision making UAV \(f\) and that of the enemy UAV \(e\), respectively. The goal point \(G_{f}\) for the decision making UAV \(f\) is located near the right edge of the region, while the goal point \(G_{e}\) for the enemy UAV \(e\) is located near the left edge of the region. Both goal points are on the middle line of the region, as shown in Figure 1.  
In such a region, there is no hovering requirement or taking off and landing requirements, therefore the fixed-wing UAV can be easily applied in such problem settings. The mission of the decision making UAV \(f\) is to reach the goal point \(G_{f}\) safely and with as little cost as possible. However, the decision making UAV \(f\) and the enemy UAV \(e\) are flying in opposite directions along the middle line of the region toward their own goal points, which poses the decision making UAV \(f\) the danger of collision with the enemy UAV \(e\). Therefore, the decision making UAV \(f\) needs to avoid the enemy UAV \(e\) during its flight towards the goal point \(G_{f}\). Ways to avoid collision with the enemy UAV include changing the heading angle or the velocity, and attacking the enemy UAV. Though changing the heading angle can let the decision making UAV avoid flying directly into the enemy UAV if the the enemy UAV happens to be in the heading direction of the decision making UAV, inappropriate heading angle change may make the decision making UAV fly too far away from the goal point \(G_{f}\). Similarly, changing the velocity at an appropriate time can also avoid collision with the enemy UAV, such as accelerating to pass the enemy UAV before the collision or decelerating to wait for the enemy UAV to pass. Successfully attacking the enemy UAV can also provide good insurance for the decision making UAV to fulfill its mission. However, the attacking action may fail in destroying the enemy UAV, and the decision making UAV suffers certain losses when using attacking action. Therefore, the decision making UAV should not use the attacking action too often. To achieve the mission requirements, the decision making UAV cannot use just one single avoiding action, but should arrange all the actions in an appropriate sequence. On the other hand, the enemy UAV simply flies toward the goal point \(G_{e}\) with constant velocity and heading angle if the decision making UAV does not collide with it or attack it successfully. The scenario is supposed to end as soon as the decision making UAV has collided with or successfully attacked the enemy UAV, or the decision making UAV has reached the goal point \(G_{f}\) successfully or has been out of the region. To arrange an appropriate action sequence is the process of decision making, which is also the main focus of this work. In decision making, such an action sequence is called the policy. Based on the action chosen by the on-board agent at each decision making step, the decision making UAV can adjust its flying attitude or attack the enemy UAV. The agent makes decisions based on both its own information from its on-board sensor system and the enemy UAV's information from the ground station. The action executed by the decision making UAV makes the UAV changes its state in the environment. On the other hand, the enemy UAV also updates its states in the environment and senses its own states from the environment with its on-board sensor system. The ground station captures all UAVs' information, and then transmits the information to the decision making UAV.