In this task, you are given the abstract of a research paper. Your task is to generate a summary of this abstract. Your summary should not be very short, but it's better if it's not more than 30 words.

In this work, we focus on the problem of grounding language by training an agent
 to follow a set of natural language instructions and navigate to a target object
 in a 2D grid environment. The agent receives visual information through raw
 pixels and a natural language instruction telling what task needs to be achieved.
 Other than these two sources of information, our model does not have any prior
 information of both the visual and textual modalities and is end-to-end trainable.
 We develop an attention mechanism for multi-modal fusion of visual and textual
 modalities that allows the agent to learn to complete the navigation tasks and also
 achieve language grounding. Our experimental results show that our attention
 mechanism outperforms the existing multi-modal fusion mechanisms proposed in
 order to solve the above mentioned navigation task. We demonstrate through the
 visualization of attention weights that our model learns to correlate attributes of
 the object referred in the instruction with visual representations and also show
 that the learnt textual representations are semantically meaningful as they follow
 vector arithmetic and are also consistent enough to induce translation between instructions
 in different natural languages. We also show that our model generalizes
 effectively to unseen scenarios and exhibit zero-shot generalization capabilities.
 In order to simulate the above described challenges, we introduce a new 2D environment
 for an agent to jointly learn visual and textual modalities
Attention based architecture for language grounding via reinforcement learning in a new customizable 2D grid environment  

The ability to look multiple times through a series of pose-adjusted glimpses is fundamental to human vision. This critical faculty allows us to understand highly complex visual scenes. Short term memory plays an integral role in aggregating the information obtained from these glimpses and informing our interpretation of the scene. Computational models have attempted to address glimpsing and visual attention but have failed to incorporate the notion of memory. We introduce a novel, biologically inspired visual working memory architecture that we term the Hebb-Rosenblatt memory. We subsequently introduce a fully differentiable Short Term Attentive Working Memory model (STAWM) which uses transformational attention to learn a memory over each image it sees. The state of our Hebb-Rosenblatt memory is embedded in STAWM as the weights space of a layer. By projecting different queries through this layer we can obtain goal-oriented latent representations for tasks including classification and visual reconstruction. Our model obtains highly competitive classification performance on MNIST and CIFAR-10. As demonstrated through the CelebA dataset, to perform reconstruction the model learns to make a sequence of updates to a canvas which constitute a parts-based representation. Classification with the self supervised representation obtained from MNIST is shown to be in line with the state of the art models (none of which use a visual attention mechanism). Finally, we show that STAWM can be trained under the dual constraints of classification and reconstruction to provide an interpretable visual sketchpad which helps open the `black-box' of deep learning.
A biologically inspired working memory that can be integrated in recurrent visual attention models for state of the art performance

The capability of making interpretable and self-explanatory decisions is essential for developing responsible machine learning systems. In this work, we study the learning to explain the problem in the scope of inductive logic programming (ILP). We propose Neural Logic Inductive Learning (NLIL), an efficient differentiable ILP framework that learns first-order logic rules that can explain the patterns in the data. In experiments, compared with the state-of-the-art models, we find NLIL is able to search for rules that are x10 times longer while remaining x3 times faster. We also show that NLIL can scale to large image datasets, i.e. Visual Genome, with 1M entities.

An efficient differentiable ILP model that learns first-order logic rules that can explain the data.