Learning Action-Effect Dynamics for Hypothetical Vision-Language Reasoning Task (2022)

Abstract: ‘Actions’ play a vital role in how humans interact with the world. Thus, autonomous agents that would assist us in everyday tasks also require the capability to perform ‘Reasoning about Actions & Change’ (RAC). This has been an important research direction in Artificial Intelligence (AI) in general, but the study of RAC with visual and linguistic inputs is relatively recent. The CLEVR_HYP (Sampat et. al., 2021) is one such testbed for hypothetical vision-language reasoning with actions as the key focus. In this work, we propose a novel learning strategy that can improve reasoning about the effects of actions. We implement an encoder-decoder architecture to learn the representation of actions as vectors. We combine the aforementioned encoder-decoder architecture with existing modality parsers and a scene graph question answering model to evaluate our proposed system on the CLEVR_HYP dataset. We conduct thorough experiments to demonstrate the effectiveness of our proposed approach and discuss its advantages over previous baselines in terms of performance, data efficiency, and generalization capability.

## introduction humans interact with their environment to accomplish desired goals. object manipulation (i.e., performing "actions" over the objects) is a fundamental concept that makes this interaction possible. in other words, actions in their simplest form have the power to change the state of a world and hence play a vital role in enabling humans to perform dayto-day tasks. as we are developing autonomous agents that can assist us in everyday tasks, they would also require to interact with complex environments. hence, the development of autonomous agents that can perform actions to effectively manipulate objects and understand corresponding effects is of great importance. as a result, reasoning 1 dataset setup scripts and code for baselines are available at https://github.com/shailaja183/arl about action and change (rac) has been a longstanding research problem, since the rise of ai. the work of @xcite was the earliest to emphasize the importance of reasoning about actions. they developed an advice taker system that can do deductive reasoning about scenarios such as "going to the airport from home" requires "walking to the car" and "driving the car to airport". since then, many real-life use cases have been identified which require ai models to understand interactions among the current states of the world, actions being performed over various objects, and most likely following states @xcite . while rac has been more popular among knowledge representation and logic communities, it has recently piqued the interest of researchers in nlp and vision domains. a recent survey by @xcite compiled a comprehensive list of works that explore neural network's ability to reason about actions and changes, provided a dataset of linguistic and/or visual inputs. in a recent tweet, prof. yann lecunn also emphasized the importance of this research direction. he mentions that "while we progress towards human-level ai, i believe we need to find new concepts that would (i) allow machines to learn to predict how one can influence the world through taking actions, (ii) learn hierarchical representations that allow long-term predictions in abstract spaces, (iii) enable agents to predict the effects of sequences of actions so as to be able to reason & plan -all of this in ways that are compatible with gradient-based learning" @xcite . in this work, we aim to better tackle the hypothetical action reasoning task of clevr_hyp dataset @xcite . an example from this dataset is shown in figure 1 ). the key objective is to understand changes caused over a visual scene by an action described in the natural language and answer a reasoning question. in figure 2 , we describe two possible action-effect learning strategies (ls1 and ls2) through a toy example to convey our intuition behind this work. ls1 uses visual features (i.e. features from the image of an apple) and a representation of actions (i.e. text "rotten") through sentence embeddings to imagine the effects (i.e. how a rotten apple would look like). this can be an intuitive choice to model the clevr_hyp task using pre-trained vision-language models. in our hypothesis, ls1 does not improve the model's understanding of what effects the actions will produce. thus, we propose an alternative strategy, ls2 2 . specifically, we let the model observe the difference between pairs of states before and after the action is performed (i.e. decayed portion of the apple that distinguishes a good apple from the rotten one), then associate those visual differences with the corresponding linguistic action descriptions (i.e. text "rotten"). ls2 is likely to better capture action-effect dynamics, as action representations are learned explicitly. to empirically test the above hypothesis, we develop a model (which is described in section 3) and evaluate it on the clevr_hyp dataset. we hope that our exciting results would enable the development of ai agents that can better collaborate with humans in the physical world and encourage further investigations in this research area. in summary, our key contributions are as follows; ## clevr_hyp in this section, we briefly summarize important aspects of the clevr_hyp dataset @xcite and related terminologies used in the subsequent sections. ## proposed model: action representation learner (arl) in this section, we describe the architecture of our proposed model action representation learner (arl). in our point of view, the most critical component of a model that attempts to solve clevr_hyp is the one where mapping between visual changes and actions are learned. in figure 2 , we graphically demonstrated our intuition behind how we can do so. our hypothesis is that a model can learn better action representations by observing difference between a pair of states (before and after the action is performed) and then associate those visual differences with given linguistic description of actions. in this regard, we attempt to create a 3-stage model shown in figure 5 , which we believe would better capture the causal structure of this task. detailed description of each individual component is provided below and can be visualized in figure 6 . ## results and analysis in this section, we discuss the performance of our model quantitatively and qualitatively. additionally, we discuss our findings from three ablations for our model. ## conclusion in the vision and language domain, several tasks are proposed that require an understanding of the causal structure of the world. in this work, we propose an effective way of learning action representations and implement a 3-stage model for the what-if vision-language reasoning task clevr_hyp. we provide insights on the learned action representations and validate the effectiveness of our proposed method through ablations. finally, we demonstrate that our proposed method outperforms existing baselines while being data-efficient and showing some degree of generalization capability. by extending our approach to a larger set of actions, we aim to develop ai agents which are equipped with action-effect reasoning capability and can better collaborate with humans in the physical world.