Cross-Probe BERT for Efficient and Effective Cross-Modal Search
Authors: TAN YU, Hongliang Fei, Ping Li
Abstract: Inspired by the great success of BERT in NLP tasks, many text-vision BERT models emerged recently. Benefited from cross-modal attention,  text-vision BERT models have achieved excellent performance in many language-vision tasks including text-image retrieval.   Nevertheless,  cross-modal attentions used in text-vision BERT models require too expensive computation cost when solving text-vision retrieval, which is impractical for large-scale search. In this work, we develop a novel architecture, Cross-Probe BERT.  It relies on devised text and vision probes, and cross-modal attentions are conducted on text and vision probes.  It takes lightweight computation cost, and meanwhile effectively exploits cross-modal attention.  Systematic experiments conducted on two public benchmarks demonstrate state-of-the-art effectiveness and efficiency of the proposed method.