Before revision:  Much weaker results are found in control experiments, attesting the selectivity of the probe. All examined models greatly under-perform humans in retrieval, highlighting substantial room for future progress.
After revision:  Visually grounded language models slightly outperform text-only language models in instance retrieval, but greatly under-perform humans in retrieval, highlighting substantial room for future progress.