  In this paper we propose a new framework - MoViLan (Modular Vision and
Language) for execution of visually grounded natural language instructions for
day to day indoor household tasks. While several data-driven, end-to-end
learning frameworks have been proposed for targeted navigation tasks based on
the vision and language modalities, performance on recent benchmark data sets
revealed the gap in developing comprehensive techniques for long horizon,
compositional tasks (involving manipulation and navigation) with diverse object
categories, realistic instructions and visual scenarios with non-reversible
state changes. We propose a modular approach to deal with the combined
navigation and object interaction problem without the need for strictly aligned
vision and language training data (e.g., in the form of expert demonstrated
trajectories). Such an approach is a significant departure from the traditional
end-to-end techniques in this space and allows for a more tractable training
process with separate vision and language data sets. Specifically, we propose a
novel geometry-aware mapping technique for cluttered indoor environments, and a
language understanding model generalized for household instruction following.
We demonstrate a significant increase in success rates for long-horizon,
compositional tasks over the baseline on the recently released benchmark data
set-ALFRED.
