A Two-Stage Adaptation of Large Language Models for Text Ranking (2024)

Abstract: Text ranking is a critical task in information retrieval. Recent advances in pre-trained language models (PLMs), especially large language models (LLMs), present new opportunities for applying them to text ranking. While supervised fine-tuning (SFT) with ranking data has been widely explored to better align PLMs with text ranking goals, previous studies have focused primarily on encoder-only and encoder-decoder PLMs. Research on leveraging decoder-only LLMs for text ranking remains scarce. An exception to this is RankLLaMA, which uses direct SFT to explore LLaMA’s potential for text ranking. In this work, we propose a two-stage progressive paradigm to better adapt LLMs to text ranking. First, we conduct continual pre-training (CPT) of LLMs on a large weakly-supervised corpus. Second, we perform SFT, and propose an improved optimization strategy building upon RankLLaMA. Our experimental results on multiple benchmarks show that our approach outperforms previous methods in both in-domain and out-domain scenarios.

## introduction text ranking is to order a set of candidate documents by their relevance to a given query. this process is often the second step in information retrieval, following the initial collection of candidate documents from a large corpus by a fast retriever 1 . early work relied primarily on the handcrafted numerical features based on query-document pairs @xcite . recent advances in plms such as bert, along with large-scale annotated datasets like ms marco @xcite , have significantly improved model performance in text ranking. llms, such as llama @xcite and gpt4 @xcite , have brought a paradigm shift in natural language processing through their impressive performance on various tasks. this has driven growing interest in applying llms to text ranking @xcite . recent works have explored prompt learning, as well as pointwise @xcite , pairwise @xcite and listwise @xcite text ranking schemas to enable out-of-the-box llms to perform unsupervised ranking. with the help of llms, substantial improvements over the bert-style plm counterparts have been achieved. however, a misalignment persists between the llm pre-training and the text ranking, as shown in figure 1 . several studies address this through sft of encoder-only @xcite and encoder-decoder @xcite models. yet rare work has targeted decoder-only llms. rankllama @xcite ) might be the only exceptional work exploring sft on decoder-only llms. while rankllama shows some gains, its achievements still lag behind those of previous sft studies, and the observation could be even more serious when tested on the out-domain scenario. in this work, we propose a two-stage training framework to adapt decoder-only llms to text ranking progressively: (1) cpt followed by (2) sft. given the broad definition of text relevance, e.g., reasoning and semantic similarity, we first exploit a cpt stage to teach llms various cases of relevance. this helps the second-stage sft more readily and accurately align llms with text ranking objectives. during cpt, we construct a largescale weakly-supervised text-pair dataset, and then perform the next-token prediction task (ntp) @xcite on it. for sft, we introduce a new optimization objective different from rankllama to better explore the potential of llms. we demonstrate the efficacy and generalizability of our two-stage adaptation with extensive experiments on in-domain and out-domain datasets. we test our method on major decoder-only llms and various model scales, covering bloom 560m-7b @xcite , llama-7b @xcite , baichuan-7b @xcite , and qwen-7b @xcite . the experimental results show that our method substantially improves over its baselines, highlighting the benefits of our progressive paradigm for text ranking. we also perform in-depth analysis into how our two-stage adaptation bridges the gap between llms and the text ranking task foot_0 . ## related work text ranking has been an active area of research for decades @xcite . one widely-adopted strategy is to rank candidate documents based on the relevance score between a query and the document @xcite . this strategy is referred to as the pointwise approach. additionally, pairwise @xcite and listwise @xcite approaches, which consider the relative ordering of multiple documents in response to a query, have also gained great attention due to their strong performance. in our work, we utilize the pointwise approach for efficient inference and the listwise approach for effective training, making the best use of the both. the precise calculation of the query-document relevance scores is the key to the pointwise inference, which has been generally dominated by supervised techniques @xcite . initially, manually-crafted sparse features were employed to estimate these scores @xcite . subsequently, neural network models marked a turning point, showcasing their substantial promise @xcite . more recently, the progress of plms has led to remarkable advances in score calculation by using a pre-training and fine-tuning framework for text ranking @xcite @xcite @xcite . as plms evolved into decoder-only llms, early work explored out-of-the-box strategies for text ranking to leverage the inherent strong reasoning capabilities of llms @xcite @xcite . for instance, one could simply ask llms to determine the relevance of a query-document pair, yielding a rudimentary solution @xcite . subsequent works @xcite @xcite , propose the use of query generation likelihood based on a candidate document as a measure of relevance. we follow this line of work for the relevance score definition. nevertheless, these out-of-the-box strategies often overlook the potential misalignment between llms and the specific requirements of text ranking tasks, which is a major issue that our work aims to address. task-specific llm training can help bridge this gap @xcite @xcite , as exemplified by rankl-lama @xcite , which uses the last token as the ranking basis and trains with ranking losses. while dominant llms typically feature a decoder-only architecture, previous research on adapting encoder(-decoder) plms to text ranking remains highly pertinent @xcite @xcite . the underlying principles and core ideas of these studies underpin our approach, and their benefits apply across diverse model architectures. ## conclusion in our work, we proposed a novel two-stage training paradigm to adapt llms to text ranking tasks. specifically, we first performed cpt on a largescale weakly-supervised corpus to initially align llms with ranking objectives. this is then followed by sft on high-quality data along with fullquery generation optimization and auxiliary objec-tives. through this two-stage training paradigm, we achieved improved ranking performance on various llms in both in-domain and out-domain experimental settings. the significant gains exhibited by our approach highlight its effectiveness in enhancing the capabilities of llms for text ranking.