# Inferring Agents' Goals from Observing Successful Traces
## 5 Inferring the Agent's Model and Environment Rules from Data  
In the previous sections, the input \(\Sigma\), the set of successful traces, is simply reduced to two different sets of states \(S_{negative}\) and \(S_{positive}\). By doing so, we are not taking advantage of the information included in \(\Sigma\) about the order in which the states are explored and the actions that the agent performs at each step. By ignoring these aspects, we have the advantage of being able to infer possible goals without more assumptions about the agent than the ones induced by successful traces how successful traces are defined, i.e., as soon as its goal is reached, the agent stops (with a success action) and this goal depends only on the current state of the environment. However, if the dynamics of the environment are known, it seems relevant to derive some information from what the agent has chosen to do, compared to what the agent could have done. It is only possible if we are given some insight into how the agent chooses its actions and takes decisions. For instance, if we consider a rational omniscient agent's case, we know that the observed agent has complete knowledge about its environment and its rules and uses a planner to compute the shortest plans to reach its goal. In such a case, by knowing the environment dynamics, we can use it to generate more negative examples, for each successful trace \(T\in\Sigma\) of size \(K\), all the negative examples that are reachable from the state \(i\) in \(K-i\) steps can be generated and added to the set \(S_{negative}\). We can compute this using \(K_{i}\) times the function \(next\) with every possible action to get all the reachable states. Using the same process with weaker assumptions, we consider now the case of a deterministic environment for which it is assumed that the agent is smart enough to reach its goal with only one move when its goal is reachable in one move (it cannot fail since the environment is deterministic). This amounts to considering that the agent can plan at least one step ahead, i.e., to understand its actions' direct consequences and choose its actions accordingly. In such a case, when also knowing the environment and its dynamics, many negative examples can be generated. Indeed, if the action model ruling the effect of the agent's actions is known, we can deduce from each state of the world the states that can be reached by the agent at the next time step. Next, if we know that at the time step \(t\) the agent did not reach its goal, it means that all the states that were reachable by the agent at the time step \(t-1\) are not final ones. Furthermore, all these intermediate states can thus be regarded as negative examples, allowing us to refine the results of LFST by using more learning examples. In such a case, we modify LFST by this pre-processing that is applied to each \(T\in\Sigma\) to compute additional negative examples. This process improves learning by expanding the initial input. It allows us to learn precise goals when using fewer original examples. Figure 15: Comparison of the average runtime in CubeWorld (in seconds). Figure 16 shows us the advantage of having these kinds of assumptions about the agent and environment. For this graph, the same experimental protocol as GridWorld is used but this time with a grid of size 4 \(\times\) 4, which means that we have 192 positive examples and 776 negative ones that are used as input to LFST when no assumptions about the agent and environment are made. Then, we make two assumptions about the agent: the agent action model is known, and the agent can plan at least one step ahead so if its goal is reachable at step \(t-1\), at step \(t\) it will reach it. These are not strong assumptions, and they allow us to generate more data from the 192 itineraries that have been followed by the agent throughout the simulation. In total, we get 2074 negative examples thanks to this data generation, which is almost three times more than what we have without making assumptions about the agent.