## 4 Experiments and Results
### Results in Online Stage  
In this section, we will evaluate the performance of Bayes-Lab by comparing it with the state-of-the-art algorithms, including BPR+ [40], Deep BPR+ [43], and Bayes-Tomop [44]. In the following experiments, we use Bayes-Lab(A) to represent the first detecting mode (line 10 in Algorithm 2), which combines \(\Omega\) with \(P\) to update \(\beta(\tau)\), while using Bayes-Lab to define the second detecting mode (line 17 in Algorithm 2), which incorporates \(\Theta\) with \(P\) to detect the opponent's type. In addition, we assume that the agent cannot obtain the opponent's observations. We evaluated the performance of the proposed algorithm comprehensively through indicators such as episodic rewards, accumulated rewards, and detection accuracy. The following results were averaged over 500 runs to enable statistical analysis. There are 100 episodes in each run, and the opponents switch policies every 10 episodes. The order of policy switching is (\(\tau_{2}\), \(\tau_{3}\), \(\tau_{1}\), \(\tau_{3}\), \(\tau_{4}\), \(\tau_{1}\), \(\tau_{4}\), \(\tau_{1}\), \(\tau_{3}\), \(\tau_{1}\)). Each episode terminates after 100 time steps. Figure 8 illustrates the episodic rewards for 100 episodes of interactions. The optimal reward will be around 10 within 100 time steps, as the agent receives a reward of 0.1 for each step without colliding with the opponents. We observe that the policy switch will lead to a rapid decrease in the reward, and the performance of the algorithms can be evaluated by observing how quickly the episodic rewards recover to 10. Bayes-Lab (A) recaptures the optimal reward most quickly during each policy switch. Bayes-Lab has a similar performance to Bayes-Lab (A) for the most part, but it has significantly lower rewards in episodes 50-60 and 70-80, where the policy switches from \(\tau_{4}\) to \(\tau_{1}\). By observing \(\Omega\) (Figure 6) and \(\Theta\) (Figure 7), we notice that the difference in the distributions in \(\Omega\) is more significant when the response policy is \(\pi_{4}\). Therefore, \(\Omega\) is better suited to cope with the situation where the \(\tau_{4}\) switches to \(\tau_{1}\). Moreover, Deep-BPR+ performs worse than Bayes-Lab (A) and Bayes-Lab because it overly depends on the opponent's information and can only obtain the opponent's behavior data within the agent's vision range in our setting. BPR+ is slower to respond than Deep BPR+ because it detects the opponent based only on \(P\) (Figure 5). In other words, the distributions in \(P\) are relatively similar, resulting in poor performance for BPR+. Bayes-Tomop has the bottom performance of all algorithms as it takes additional steps to determine whether the opponent is using BPR, which is time-consuming. Figure 9 illustrates the accumulative rewards for 100 episodes of interactions. We observe that the rewards decrease with each policy switch but increase again when detection and reuse are completed. Bayes-Lab (A) achieves the highest accumulative rewards because it detects and reuses the most quickly. Bayes-Lab has a lower performance than Bayes  
Figure 8: The episodic rewards for 100 episodes of interactions. Lab (A), but it is still better than the other algorithms because \(\Theta\) can compensate for the effects of the incomplete opponent information. Deep-BPR+ archives significantly higher rewards than BPR+, proving that detecting the opponent's type by using its behaviors is more effective. Bayes-Tomop has the lowest rewards, demonstrating that it is inefficient in identifying whether the opponent is using BPR. Moreover, it assumes that the optimal rewards are consistent, but the agent may receive different optimal rewards in our setting, making detection more difficult. Figure 10 shows the detection accuracy against non-stationary opponents. We observe that Bayes-Lab (A) and Bayes-Lab achieve relatively higher detection accuracies than other algorithms. We note that Bayes-Lab (A) and Bayes-Lab detect the opponent's type only based on the agent's observations.