## 3 Meta-PPO Anti-Jamming Intelligent Decision Algorithm
### Meta-Learner  
As shown in Figure 6, in the proposed Meta-PPO algorithm, the meta-learner is responsible for collecting and summarizing the learned experiences from all tasks. After each learning process from the basic learner, the meta-learner will integrate those experiences and update its parameters accordingly. In the end, the meta-learner, having integrated the learning experiences from all tasks, delivers the initial model network values to the basic  
Figure 5: Decision model of the basic learner based on the Meta-PPO algorithm. learner, allowing the basic learner to achieve good accuracy after a few iterations in dealing with new tasks. As shown in Figure 3, the memory module of the proposed algorithm includes two parts: an initial parameter value and the loss function's gradient value. The initial parameter value represents the initial parameter of a certain task. During the basic learner's learning process, the meta-learner extracts the initial parameter value of the most relevant task from the memory module and feeds it to the basic learner as the initial parameter value. After the basic learner completes its learning process, it feeds the loss function's gradient value back to the meta-learner. The meta-learner then distributes the initial parameter values to all the basic learners. After the basic learners complete their learning processes, they feed the loss function's gradient values back to the meta-learner. Finally, the meta-learner collects the function's gradient values from all the tasks and then uses these gradient values to update the initial parameter values stored in the memory module in a timely manner. Therefore, the latest initial parameter values from the most relevant task experience can be provided timely for each task. The intelligent anti-jamming model of the Meta-PPO algorithm is shown in Figure 7. The specific learning process of the meta-learner is as follows:  
Assume there is a model \(f_{\theta}()\) affected by \(\theta\) and the distribution of tasks \(p(T)\). First, initialize the parameters \(\theta\) with some random values. Next, draw a batch of tasks \(T_{i}\) (\(T_{i}\sim p(T)\)) from the distribution of tasks. Then, for each task, draw \(k\) trajectories and construct the training and test sets: \(D_{i}^{train}\sim T_{i}\). Execute gradient descent through Equation (12) and find the optimal parameters \(\theta_{i}^{\prime}\) to minimize the loss on the training set \(D_{i}^{train}\):  
\[\theta_{i}^{\prime}=\theta-\alpha\
abla_{\theta}L_{T_{i}}(f_{\theta}) \tag{12}\]  
Before drawing the next batch of tasks, perform a meta-update. Minimize the loss by calculating the loss gradient relative to the optimal parameter \(\theta_{i}^{\prime}\) through Equation (13) to update our randomly initialized parameter \(\theta\):  
\[\theta\leftarrow\theta-\beta\
abla_{\theta}\sum_{T_{i}\sim p(\tau)}L_{T_{i}}( f_{\theta\prime}) \tag{13}\]  
Figure 6: Overall decision-making process based on the Meta-PPO algorithm. The references cited in this article are presented in Table 1, which lists the network environments and core algorithms of all the referenced literature.