Understanding What May Have Happened in Dynamic, Partially Observable Environments

UNDERSTANDING WHAT MAY HAVE HAPPENED IN DYNAMIC, PARTIALLY OBSERVABLE ENVIRONMENTS Matthew Molineaux, PhD George Mason University, 2017 Dissertation Director: Dr. Gheorghe Tecuci In this work, we address the problem of understanding what may have happened in a goal-based deliberative agent’s environment after the occurrence of exogenous actions and events. Such an agent observes, periodically, information about the state of the world, but this information is incomplete, and reasons for state changes are not observed. We propose methods a goal-based agent can use to construct internal, causal explanations of its observations based on a model of its environment. These explanations comprise a series of inferred actions and events that have occurred and continue to occur in its world, as well as assumptions about the initial state of the world. We show that an agent can more accurately predict future events and states by reference to these explanations, and thereby more reliably achieve its goals. This dissertation presents the following novel contributions: (1) a formalization of the problems of achieving goals, understanding what has happened, and updating an agent’s model in a partially observable, dynamic world with partially known dynamics; (2) a complete agent (DHAgent) that achieves goals in such environments more reliably than existing agents; (3) a novel algorithm (DiscoverHistory) and technique (DiscoverHistory search) for rapidly and accurately iteratively constructing causal explanations of what may have happened in these environments; (4) an examination of formal properties of these techniques; (5) a novel method (EML), capable of inferring improved models of an environment based on a small number of training scenarios; (6) experiments supporting performance claims about the novel methods described; and (7) an analysis of the efficiency of two DiscoverHistory algorithm implementations. Chapter 1: Introduction Autonomous agents are proliferating quickly as computers become embedded in every aspect of daily life. Agents are embodied in the real world as autonomous vehicles, interact with humans in military simulations and games, and are used for complex predictions in agentbased models. The environments in which they act grow more and more complex as we rely on them more heavily. In the future, robots will be used in domains such as anti-submarine warfare, where the world is very hard to perceive and enemies may be present at any time. Simultaneously, as more robots are deployed, interest in greater autonomy increases. Where simple reactive or reflex agents were once acceptable, more complex deliberative goal-based agents are now desired to work in these complex environments. In particular, these agents must be able to achieve a large range of goals, specified at execution time, without constant human oversight. These goals must be specified with respect to objects and relationships in worlds with many objects and relationships. Some deliberative goal-based agents developed by the artificial intelligence community are already capable of handling large domains with many objects and relationships, but they do so by making strong assumptions about the nature of the environment. In fact, most goal-based agent research makes a variety of strong assumptions that do not hold in more realistic environments. In this work, we specify a novel agent and techniques capable of operating in larger worlds than comparable research systems, while relaxing the following assumptions: 1. the static environment assumption, which states that the environment is affected only by the agent, and by no other actors or natural processes, 2. the full observability assumption, which states that all information that impacts the agent’s performance can be observed, and