## 4 Experiments and Results  
### Experimental Setup  
In this section, we use a predator-prey environment, which is a well-known multi-agent scenario used in many studies [37, 57, 58, 59, 60]. As shown in Figure 3, the environment consists of one prey, three predators, and two immovable obstacles. The predators try to collide with the prey, while the prey must avoid collisions. All players choose actions from the action set \(\{up,right,down,left,standby\}\), and any action that tries to move the players out of the environment is ignored. At each time step, if the prey avoids collisions with the predators, it will receive a reward of 0.1. As the number of collisions increases to 1, 2, and 3, the reward will become \(-1\), \(-5\), and \(-10\), respectively. In this environment, we control the prey, and the predators sample their policies from several fixed policies. This environment is partially observable, where the prey can only see the predators and obstacles within a limited range of vision, while the predators can see all players and obstacles without limitations. The state observations are represented as a fourteen-dimensional vector which consists of the position of all players and obstacles and the speed of the prey. Each episode terminates after 100 time steps, and the position of all players and obstacles will be regenerated randomly. We design four fixed policies for the predators. (1) Vertical priority pursuit (\(\tau_{1}\)): the predators first reduce the vertical distance to the prey by moving up or down, and then move left or right when the vertical distance is small enough. (2) Horizontal priority pursuit (\(\tau_{2}\)): the predators first take a left or right movement to reduce the horizontal distance to the prey, and then take an up or down movement when the horizontal distance is small enough. (3) Clockwise pursuit (\(\tau_{3}\)): the predators pursue the prey in an action sequence of right, down, left, and up. (4) Counterclockwise pursuit (\(\tau_{4}\)): the predators pursue the prey in a left, down, right, and up action sequence. These four policies constitute the opponent policy library \(\mathcal{T}\). Then, the prey uses A2C to learn the optimal response policy for each opponent's policy and construct the optimal response policy library \(\Pi\). By designing the opponent's fixed policy and learning the optimal response policy in the offline stage, both the agent and the opponent will directly choose the corresponding actions according to the existing policies during online interactions. Specifically, the opponent randomly selects a fixed policy in \(\mathcal{T}\) and executes the corresponding action. The agent needs to identify which fixed policy the opponent has chosen and select the corresponding response policy from \(\Pi\), which is used to select the optimal action. In addition, when identifying the opponent's fixed policy, the agent needs to reconstruct the opponent's actions, which are used in two ways: First, they are used to compare with the real opponent's actions to calculate the reconstruction accuracy. Second, the agent will estimate the opponent's position by assuming that the opponent uses these reconstructed actions in a parallel environment, and then calculate the planning similarity. In other words, in the online stage, the reconstructed actions are only used as additional information to assist the agent in identifying the opponent's policy, and then the agent reuses the optimal response policy to execute the optimal actions. There are significant differences between Bayes-Lab and Deep-BPR+ [43] in the experimental environment. Deep-BPR+ uses the grid-world environments, which have the following characteristics: (1) Both the observations and actions are discrete. (2) The opponent does not consider the agent's behaviors, and it can only act according to a fixed trajectory. (3) They are globally observable: the controlled agent has access to the opponent's observations and actions. (4) There is only a single opponent in the environment.