GeAR: Generation Augmented Retrieval
Authors: name: Haoyu Liu, name: Shaohan Huang, name: Jianfeng Liu, name: Yuefeng Zhan, name: Hao Sun, name: Weiwei Deng, name: Feng Sun, name: Furu Wei, name: Qi Zhang

Document retrieval techniques form the foundation for the development of
large-scale information systems. The prevailing methodology is to construct a
bi-encoder and compute the semantic similarity. However, such scalar similarity
is difficult to reflect enough information and impedes our comprehension of the
retrieval results. In addition, this computational process mainly emphasizes
the global semantics and ignores the fine-grained semantic relationship between
the query and the complex text in the document. In this paper, we propose a new
method called Generation Augmented Retrieval
(GeAR) that incorporates well-designed fusion and decoding modules.
This enables GeAR to generate the relevant text from documents based on the
fused representation of the query and the document, thus learning to "focus on"
the fine-grained information. Also when used as a retriever, GeAR does not add
any computational burden over bi-encoders. To support the training of the new
framework, we have introduced a pipeline to efficiently synthesize high-quality
data by utilizing large language models. GeAR exhibits competitive retrieval
and localization performance across diverse scenarios and datasets. Moreover,
the qualitative analysis and the results generated by GeAR provide novel
insights into the interpretation of retrieval results. The code, data, and
models will be released after completing technical review to facilitate future
research.