## 4 Experiments and Results
### Results in the Offline Stage  
In this section, we will evaluate the reconstruction accuracy of the opponent model and the distributions in the generated models. In the following experiments, we assume that the agent adopts the optimal response policy \(\pi_{i}\) to play against the opponent policy \(\tau_{i}\) and uses the opponent model \(x_{i}\) to reconstruct the opponent's actions.  
Figure 4 illustrates the accuracy of the opponent model in reconstructing the opponent's actions. We observe that both \(x_{1}\) and \(x_{2}\) can achieve an accuracy of approximately 80%, while \(x_{3}\) and \(x_{4}\) can only reach 44% and 52%. This is because \(\tau_{1}\) and \(\tau_{2}\) have fewer random actions to pursue the prey. In contrast, \(\tau_{3}\) and \(\tau_{4}\) have greater randomness, and  
Figure 3: Predator–Prey: an adversarial multi-agent environment used in our evaluation.  
they take all possible actions with a slight frequency difference, making VAE inaccurate in reconstructing the predators' actions.  
Figure 5 illustrates the performance model \(P(U|\tau,\pi)\) (line 5 in Algorithm 1). Four opponent policies \(\{\tau_{i}\in\mathcal{T}\}_{i=0}^{i=4}\) and four response policies \(\{\pi_{i}\in\Pi\}_{i=0}^{i=4}\) play against each other to fit sixteen Gaussian distributions. Each distribution reflects the corresponding episodic reward information. We observe that the optimal response policy receives the highest average episodic reward, all-around 10, with minor variance. In addition, the distribution is likely to be very similar to each other. For example, when the agent uses \(\pi_{1}\), the distributions of interacting with \(\tau_{3}\) and \(\tau_{4}\) are almost the same. When the agent only uses the performance model to detect the opponent's policy, such as BPR [39] and BPR+ [40], the high similarity of distributions may cause low detection accuracy.  
Figure 6 illustrates the reconstruction accuracy model \(\Omega(W|\tau,\pi,x)\) (line 9 in Algorithm 1). It also consists of sixteen Gaussian distributions, like in the performance model \(P\). Each distribution describes how accurately the corresponding opponent model \(x\) reconstructs the opponent's actions. We observe that the optimal response policy has the highest accuracy, with \(\pi_{1}\) and \(\pi_{2}\) achieving an accuracy of around 80%, while \(\pi_{3}\) and \(\pi_{4}\) only reach approximately 50%. The difference between distributions in \(\Omega\) is more significant than that in \(P\). For example, when the agent adopts \(\pi_{1}\) to play against \(\tau_{3}\) and \(\tau_{4}\), the distributions in \(P\) are very similar, but they are much more different in \(\Omega\). In addition, \(\Omega\) is more suitable for situations where the opponent's actions are available, and it can be jointly used with \(P\) during online detection.  
Figure 7 shows the planning similarity model \(\Theta(Q|\tau,\pi,x)\) (line 17 in Algorithm 1). We measure the similarity in terms of the mean square error between continuous observations. The smaller the error, the greater the similarity. We observe that the optimal response policy has the minimum mean squared error, with \(\pi_{1}\) and \(\pi_{2}\) reaching around 0. However, the minimum mean squared error for \(\pi_{3}\) and \(\pi_{4}\) is around 200 due to the lower accuracy of action reconstruction. In addition, \(\Theta\) is more suitable for situations where neither the opponent's observations nor the actions are available, and it can also be used in combination with \(P\) during the online phase.  
Figure 4: The accuracy of the opponent model in reconstructing the opponent’s actions.  
Figure 5: The performance model \(P\).  
Figure 6: The reconstruction accuracy model \(\Omega\).  
Figure 7: The planning similarity model \(\Theta\).