Probing Emergent Semantics in Predictive Agents via Question Answering
Recent work has demonstrated how predictive modeling can endow agents with rich knowledge of their surroundings, improving their ability to act in complex environments.We propose question-answering as a general paradigm to decode and understand the representations that such agents develop, applying our method to two recent approaches to predictive modeling – action-conditional CPC and SimCore.After training agents with these predictive objectives in a visually-rich, 3D environment with an assortment of objects, colors, shapes, and spatial configurations, we probe their internal state representations with a host of synthetic questions, without backpropagating gradients from the question-answering decoder into the agent.The performance of different agents when probed in this way reveals that they learn to encode detailed, and seemingly compositional, information about objects, properties and spatial relations from their physical environment.Our approach is intuitive, i.e. humans can easily interpret the responses of the model as opposed to inspecting continuous vectors, and model-agnostic, i.e. applicable to any modeling approach.By revealing the implicit knowledge of objects, quantities, properties and relations acquired by agents as they learn, question-conditional agent probing can stimulate the design and development of stronger predictive learning objectives.
We use question-answering to evaluate how much knowledge about the environment can agents learn by self-supervised prediction.Proposes QA as a tool to investigate what agents learn about in the world, arguing this as an intuitive method for humans which allows for arbitrary complexity.The authors propose a framework to assess representations built by predictive models that contain sufficient information to answer questions about the environment they are trained on, showing those by SimCore contained sufficient information for the LSTM to answer questions accurately.