L ist T 5: Listwise Reranking with Fusion-in-Decoder Improves Zero-shot Retrieval (2024)

Abstract: We propose ListT5, a novel reranking approach based on Fusion-in-Decoder (FiD) that handles multiple candidate passages at both train and inference time. We also introduce an efficient inference framework for listwise ranking based on m-ary tournament sort with output caching. We evaluate and compare our model on the BEIR benchmark for zero-shot retrieval task, demonstrating that ListT5 (1) outperforms the state-of-the-art RankT5 baseline with a notable +1.3 gain in the average NDCG@10 score, (2) has an efficiency comparable to pointwise ranking models and surpasses the efficiency of previous listwise ranking models, and (3) overcomes the lost-in-the-middle problem of previous listwise rerankers. Our code, model checkpoints, and the evaluation framework will be fully open-sourced.

## introduction recent advancements on neural information retrievers have made significant progress in their semantic search capabilities. however, they still struggle in zero-shot or out-domain tasks where statistical retrievers such as bm25 @xcite often outperform, a crucial challenge since the setting is closely related to real-world scenarios. until now, the field of zero-shot reranking has been generally driven by cross-encoder models @xcite such as monot5 @xcite or rankt5 @xcite . these models rely on pointwise reranking of each passage, and thus lacks the ability to compare between passages relatively at inference time. this could lead to a suboptimal solution in the task of reranking, where discrimination and ordering between passages are crucial. recently, listwise reranking models @xcite , which evaluates multiple passages together, is gaining attention for its effectiveness in zero-shot retrieval. listwise rerankers can condition on and compare multiple passages to calibrate the relevance scores better, and can reduce the inaccuracy of zero-shot predictions arising from domain shift, as theoretically supported by @xcite , and empirically evidenced by a line of works such as @xcite ; @xcite . however, existing approaches that attempt listwise reranking have limitations. duot5 @xcite implements pairwise reranking, a form of listwise reranking as pair, which incurs quadratic time complexity to the number of candidate passages. on the other hand, listwise reranking using large language models (llms) @xcite @xcite sacrifice efficiency in another aspect due to large parametric model size, and also face the "lost in the middle" problem @xcite . in this work, we present listt5 (fig. 1 ), a novel listwise approach for zero-shot reranking that overcomes the aforementioned limitations. listt5 jointly considers the relevancy of multiple candi-date passages at both training and inference time based on fusion-in-decoder (fid) @xcite architecture. it outputs a sorted list of input passages in the increasing order of relevance, and employs a novel tournament tree structure at inference time for efficient sorting. with these, listt5 provides the following contributions: ## methods our method listt5 for listwise reranking has two components: (1) the basic operating unit that takes a fixed number of m passages and ranks the top-r based on fid, and (2) an extension of this basic unit that takes full n passages and ranks the top-k based on tournament sort. we describe the basic unit in sec. 3.1 and its extension in sec. 3.2. ## conclusion we proposed listt5, a fid-based listwise reranking model that jointly considers multiple passages as input and generates an ordered list of passages, in increasing order of relevance. listt5 outperforms others in beir, with a remarkable +1.3 gain of average ndcg@10 performance. listt5 is efficient, leveraging tournament sort to its advantage. finally, we show that listt5 overcomes the lost-in-the-middle problem in llms, and are more robust regardless of the initial ordering of the passage.