Before revision:  Evaluation on four V L benchmarks shows that Weakly-supervised VisualBERT achieves similar performance with a model pre-trained with paired data. Besides, pre-training on more image-only data further improves a model that already has access to aligned data , suggesting the possibility of utilizing billions of raw images available to enhance & V&L models.
After revision:  Evaluation on four V L benchmarks shows that Weakly-supervised VisualBERT achieves similar performance with a model pre-trained with aligned data , suggesting the possibility of utilizing billions of raw images available to enhance & V&L models.