## 5 Conclusions and Future Works  
This paper focuses on accurate opponent policy detection and optimal response policy reuse in partially observable settings. The most significant contribution of the proposed algorithms, which include Bayes-Lab and Bayes-Lab (A), is that they can efficiently track the non-stationary opponents without their local information. Our model incorporates VAE into the BPR framework and follows the CTDE paradigm. It can reconstruct the opponent's actions only based on its own local observations. Moreover, the reconstructed actions and episodic rewards are jointly used to update the belief about the opponent's types. As a result, the agent can reuse the best response policies accurately. Extensive simulations in a predator-prey scenario with continuous observations and multiple opponents demonstrate that Bayes-Lab and Bayes-Lab (A) can adapt to a wide range of different policy switching intervals and achieve higher detection accuracy and accumulative rewards than other state-of-the-art algorithms. Many end applications that require quick online decisions may benefit from the proposed networks, such as human-computer interaction and service robots.  
The main limitations of Bayes-Lab and Bayes-Lab (A) are as follows: First, we assume that the opponent's behaviors significantly affect the agent's local observations. However, the opponent's behaviors may have little correlation to the agent's local observations. In this case, it is difficult for the opponent model to learn the policy features of the opponent from the local observations. Second, we only consider the opponent switching policies at the end of an episode. However, the opponent may change more frequently, such as switching policies within an episode. Third, we assume that the opponent adopts its policy from a library that is fully known to the agent. However, the opponent may use a new strategy unknown to the agent. In future work, we would like to investigate how to build the model from different supervision signals rather than just depending on the opponent's local information, and how to deal with opponents with abrupt policy changes, such as switching policies within an episode, is also worth investigating. Furthermore, we are also interested in accurately detecting the unknown opponent's policy and learning a new optimal response policy by efficiently extracting available knowledge from the learned policies.  
Conceptualization, Y.W.; software, Y.W. and K.F.; validation, H.C. and K.F.; writing--original draft preparation, Y.W.; writing--review and editing, H.C. and Z.Z.; visualization, Q.L.; supervision, J.H. All authors have read and agreed to the published version of the manuscript.  
This research was funded by the National Natural Science Foundation of China, grant number No. 61906202.  
Not applicable.  
The authors thank the support from the National University of Defense Technology.  
The authors declare no conflict of interest.  
\begin{table}
\begin{tabular}{c c c c c} \hline \hline
**Algorithm** & **Switch Cycle 4** & **Switch Cycle 8** & **Switch Cycle 10** & **Switch Cycle 12** \\ \hline BPR+ & \(-103.9\pm 73.6\) & \(190.9\pm 112.3\) & \(376.1\pm 104.5\) & \(563.4\pm 106.0\) \\ Deep-BPR+ & \(-33.4\pm 89.4\) & \(326.5\pm 88.8\) & \(524.2\pm 96.8\) & \(728.8\pm 93.1\) \\ Bayes-Tomop & \(-171.6\pm 108.1\) & \(-16.1\pm 216.9\) & \(146.8\pm 237.9\) & \(302.7\pm 262.6\) \\ Bayes-Lab & \(23.9\pm 81.0\) & \(422.3\pm 82.5\) & \(604.3\pm 96.4\) & \(793.1\pm 90.5\) \\ Bayes-Lab (A) & \(\mathbf{131.8\pm 58.2}\) & \(\mathbf{494.1\pm 74.8}\) & \(\mathbf{691.6\pm 75.9}\) & \(\mathbf{875.7\pm 74.6}\) \\ \hline \hline \end{tabular}
\end{table}
Table 1: The effect of the switching interval on the accumulative rewards.