Before revision:  Our model is pretrained on the Conceptual Captions and Visual Question Answering datasets, and finetuned on VisDial with a VisDial-specific input representation and the masked language modeling and next sentence prediction objectives (as in BERT) .
After revision:  Our model is pretrained on the Conceptual Captions and Visual Question Answering datasets, and finetuned on VisDial .