## 4 Implementation of Autonomous Task Planning
### Reinforcement Learning Reward Function  
A well-designed reward function can motivate UAV swarms to adopt more effective action strategies, thereby achieving autonomous task planning. The reward for the mission UAV swarm consists of two parts.  
Within time \(T\), after the mission UAV swarm hits all targets, it is judged that the mission swarm wins the round, and after the round ends, the swarm gains the shared sparse reward \(R_{b1}\). If the mission UAV swarm does not destroy all targets within time \(T\), the round ends and the swarm gains a shared reward of 0. This is the reward for the first part.  
The second part refers to the mission UAV obtaining dense rewards through the relative motion with the ground target, which are divided into collision avoidance reward \(R_{b1}\), search reward \(R_{b2}\), and flight altitude reward \(R_{b3}\) in simulation. For the swarm SEAD mission, the core command and control points are patrolling, detection, locking, striking, and safe collision avoidance. The collision avoidance reward is \(R_{b1}\), which is primarily for the safe progression of the swarm and also to promote dispersion, thereby increasing search efficiency. Detection, locking, and striking are combined into Search Reward \(R_{b2}\); \(R_{b2}\) is a time-dependent quantity. To encourage the swarm to adopt a more aggressive strategy, these three actions are linked in sequence. The reward is only obtained after completing the three consecutive operations. Set the low-altitude patrolling reward as Flight Altitude Reward \(R_{b3}\).  
The reward \(R_{b}\) of intelligent UAV swarms consists of three parts,  
\[R_{b}=w_{1}R_{b1}+w_{2}R_{b2}+w_{3}R_{b3} \tag{6}\]  
Collision Avoidance Reward \(R_{b1}\). In order to avoid the internal collision of the swarm in complex confrontation, set the collision avoidance reward \(R_{b1}\). The formula is as follows:  
\[R_{b1}=\begin{cases}1-\frac{d_{A-A}}{d_{r}},d_{A-A}<d_{safe}\\ 0,d_{A-A}\geq d_{safe}\end{cases} \tag{7}\]  
The safe distance \(d_{safe}\), is set here. When the relative distance \(d_{A-A}\) between the mission UAV and the adjacent UAV is less than \(d_{safe}\), \(d_{safe}=200\), it will gain a continuous negative reward, and when the relative distance between the mission UAV is greater than \(d_{safe}\), it will gain a reward of 0. \(R_{b1}\) is set up to guide mission drones to keep a safe distance between them.  
Search Reward \(R_{b2}\). In the scenario setting, the UAV swarm uses radar to search the ground. When the relative distance between the target and the UAV \(d_{t}{}^{A-D}\) is less than the UAV's radar detection range \(d_{find}\), \(d_{find}=1500\), the search and positioning process starts, and the UAV starts to calculate the reward. If the target remains found for five time steps, the positioning is judged to be effective, and the UAV successfully searches for the target, obtaining the cumulative reward. If the target escapes, the location fails, restart the search process, and the reward is empty and recalculated.  
\[R_{b2}=\begin{cases}\sum\frac{(d_{find}-d_{t}^{A-D})}{d_{find}},d_{t}^{A-D} \leq d_{find}\\ 0,d_{t}^{A-D}>d_{find}\end{cases} \tag{8}\]  
Flight Altitude Reward. The reward \(R_{b3}\) for fixed height patrol is limited by safe height. Because the detection distance of UAVs in a preset mission is limited, when this item is not added, a UAV will reduce its flight height to the limit in exchange for the maximum ground detection coverage. Therefore, this item is added to limit the flying height \(h_{A-G}\) of UAV,s \(h_{A-G}=800\), so as to encourage UAVs to keep patrolling and searching more actively.  
\[R_{k3}=\begin{cases}1-\frac{h_{A-G}}{h_{safe}},h_{A-G}<h_{safe}\\ 0,h_{A-G}\geq h_{safe}\end{cases} \tag{9}\]