# Inferring Agents' Goals from Observing Successful Traces
## 4 Experiments
For each environment, the agent's goals and its decision model are different. The first environment is a grid of size 16 \(\times\) 16 that we have named \"GridWorld\", the second environment is a game in which the agent has to move some blocks, we named it \"CubeWorld\", and the last environment is a dice game named \"421 World\". GridWorld is then, a grid of size 16 \(\times\) 16 with some randomly placed walls between its cells. In this environment, the goal of the agent is reaching a dead-end, a dead-end being a cell surrounded by exactly three walls. We assume that for each generated grid there exists a dead-end and that this dead-end is reachable by the agent from its different starting positions in the grid. In GridWorld the agent can perform five different actions, it can move one cell top, down, left, right, or stay at its current position. Walls are impassable obstacles for the agent, a wall between two cells will prevent the agent to move from a cell to the other, the agent cannot get out of the grid either, even if there are no walls along the edge of the grid. GridWorld is a deterministic environment since the agent's actions cannot fail, meaning that for each move that is not prohibited in its current position, the agent will necessarily arrive at the corresponding position. The successful traces are generated by implementing the agent as follows. First, the agent uses the \(A^{*}\) algorithm to compute a path that reaches a dead-end from its starting position. Then the agent begins its journey. Each state and action is recorded in the trace, knowing that only one action is performed by the agent at each time step. Using the formalization described earlier to design the trace of the agent's journey, we will use the following variables to build our atomic set \(\mathcal{L}\): a variable _pos_ related to the agent's position in the environment, the variables \(upW\), \(rightW\), \(downW\), and \(leftW\) respectively related to the presence of walls at the top, right, down, and left of the agent. For the experiments in GridWorld, we generated eight different grids. Given that, depending on the inlet position, there are four possible configurations for dead ends, we assume that each of these configurations appears at least once among all the grids. We generated multiple traces for each instance of a grid, depending on the agent's starting position. Figure 2 is an illustration of GridWorld, the yellow ball being the agent and the green cell a goal position (a dead-end). The second environment, CubeWorld, is a game with six blocks numbered from 1 to 6 and three columns. The agent must place block 1 between block 2 and 3 with block 2, which must be at the column's base, and block 3 above block 1. The agent wins the game as soon as this configuration is reached on one of the columns. Knowing that a block cannot be moved if another block is above this one. The agent can stack as many blocks as wanted on a column. We implemented a naive algorithm to allow the agent to solve this problem  
Figure 1: Illustration of C4.5 learning setup with our model. Figure 2: GridWorld example. and thus obtain many solutions. Figure 3 is an illustration of CubeWorld. Each block movement caused by the agent corresponds to a time step. To design the agent's trace in this environment, we use a set of variables describing the result of each action performed by the agent. For each of the six blocks, we have a variable corresponding to the column on which the block is situated \(blockPos\), and another variable indicates the height on which it is located \(blockH\), which corresponds to coordinates on a plan. The third and last environment, 421 World, is a game of chance with four dices (six-sided dices), and the goal of the agent is to get a combination with at least one \"4\", one \"2\", and one \"1\". The agent can choose to keep the value of some dices; for example, if it gets a \"4\", it can keep it and roll only the three remaining dices.