Before revision:  Besides, pre-training on more image-only data further improves a model that already has access to aligned data , suggesting the possibility of utilizing billions of raw images available to enhance & V&L models.
After revision:  Besides, pre-training on more image-only data further improves a model that already has access to aligned data , suggesting the possibility of utilizing billions of raw images available to enhance L benchmarks. Our work challenges the widely held notion that aligned data is necessary for V& V&L models.