FORTIFY : Generative Model Fine-tuning with ORPO for R e T rieval Expansion of I n F ormal N ois Y Text (2025)

Abstract: Despite recent advancements in neural retrieval, representing text fragments or phrases with proper contextualized embeddings is still challenging. Particularly in video retrieval, where documents are text extracted through OCR from the frames or ASR from audio tracks, the textual content is rarely complete sentences but only a bag of phrases. In this work, we propose FORTIFY, a generative model fine-tuning approach for noisy document rewriting and summarization, to improve the downstream retrieval effectiveness. By experimenting on MultiVENT 2.0, an informational video retrieval benchmark, we show Llama fine-tuned with FORTIFY provides an effective document expansion, leading to a 30% improvement over prompting an out-of-box Llama model on nDCG@10. Zero-shot transferring the model tailored for MultiVENT 2.0 to two out-of-distribution datasets still demonstrates competitive retrieval effectiveness to other document preprocessing alternatives.

## introduction in typical ad hoc retrieval, documents are usually assumed to be well-formed and informative, such as news articles, blog posts, or social media threads @xcite @xcite . while some may be more structured and readable than others, they generally convey information in a way that is easily understandable to human readers. since neural retrieval models, such as dense passage retrieval (dpr) @xcite and colbert @xcite , leverage pretrained language models @xcite trained on natural language to encode documents, they typically achieve strong performance on such tasks. however, in many real-world settings, documents contain noisy or fragmented text, which does not resemble typical human communications. while this is relatively rare in traditional ad hoc retrieval, it is much more common when text is extracted from other modalities, such as automated speech recognition (asr) from audio, or optical character recognition (ocr) from images or videos. because this textual content is automatically generated, it may contain recognition errors, misidentifications, and incorrect reading order @xcite , often resulting in disjointed sentence fragments or even incomplete words. as a result, neural retrieval models struggle to represent these texts effectively, leading to weaker retrieval performance. to address this challenge, we propose a document expansion and rewriting approach using a generative model to transform fragmented text into coherent passages. we first explore a zero-shot prompting approach and demonstrate the innate ability of generative models like llama3 @xcite to reconstruct text. while this method is promising, generating meaningful summaries from unordered, disjointed tokens remains a significant challenge. to further instill retrieval-driven preferences into the generative model, we finetune it using odds ratio preference optimization (orpo) @xcite , a technique that does not require an explicit reference model or reward function. we name this method fortify, or fine-tuning with orpo for retrieval expansion of informal noisy text. we evaluate our approach on multiple video and cross-language retrieval benchmarks, and demonstrate that expanding raw documents with generated summaries leads to significant and robust performance improvements. additionally, we find that fortified summaries further boost retrieval effectiveness. to our knowledge, this is the first work to apply preference optimization to document expansion for retrieval. our contributions are threefold: ## related work text retrieval recently developed neural retrieval models leverage pretrained language models to encode documents into one @xcite @xcite or multiple @xcite contextualized embeddings to achieve better @xcite and more robust retrieval effectiveness, even in multilingual retrieval @xcite . however, because of their pretraining data @xcite , they are not welltuned for retrieving informal or even fragmented text @xcite @xcite . while recent work, such as raptor @xcite , tries to preprocess text through layers of summarization, these models still anticipate well-formed text as the input. particularly in video retrieval, text is extracted from different modalities and thus may be ill-formed. neural text retrieval models suffer when dealing with this kind of text. video retrieval traditional benchmarks for video retrieval @xcite @xcite generally involve generic web images or three to five-second video clips paired with web-scraped or automatically generated captions. methods typically compute visual features from these images or from sampled video frames that can be mapped to these natural language captions @xcite @xcite . however, there has been a shift away from these tasks to harder tasks requiring multimodal understanding, like audio and overlaid text, and longer videos @xcite . this has lead to a rise in multimodal models that jointly incorporate modalities @xcite @xcite . however, these approaches are not robust to these challenging benchmarks, with one significant factor being the fusion of noisy outputs from ocr and asr compounding errors and decreasing performance. ## methods in this section, we describe our initial document expansion approach without fine-tuning (no-finetune -nft), along with fortify, a novel method for optimizing machine-generated document expansion for information retrieval. given a noisy document d, nft involves zeroshot prompting a generative model for one or more summaries d1,...,n from d, focusing on maximizing the inclusion of synonyms and keywords to enhance retrieval performance. these summaries are then used to augment the original document, producing an expanded version in the form d + d1 + . . . + dn , where + denotes concatenation. fortify further refines this expansion by optimizing machine-generated summaries based on their relevance to corresponding queries. given a retrieval method, nft summaries are scored against the corresponding queries, and training pairs are constructed by pairing the highest-scoring summary with several lower-scoring alternatives. this enables a retrieval-driven preference optimization. ## results and analysis for multivent 2.0 (the dataset on which for-tify is trained), presented at the left part of regardless of the source of text (ocr or asr), expanding with generated summaries is more effective than using machine translation, which is an alternative document processing method (with similar hardware requirements) since the extracted text is not necessarily in the query language. such improvements are consistent across multiple settings, indicating that the summaries are useful for a wide range of retrieval models, including statistical models like bm25. however, since fortify is trained to tailor the expansion for retrieval using colbert, documents expanded with fortify summaries are more advantageous for colbert, resulting in improvement in both ndcg@10 and r@1000 over zero-shot prompting, though ndcg@10 is not statistically significant. however, the differences in r@1000 are significant, indicating that the for-tify-expanded documents include more related terms to the expansion but are not more accurate than what zero-shot prompting the generative model can provide. when using bm25 and dpr to encode and index the fortify-expanded documents, since they are not the predefined customer of the summarization model, the resulting retrieval metrics are only similar or slightly lower than nft summaries, which also indicates that fortify can effectively tailor the document expansion to the expressed preferences of the downstream retrieval model during fine-tuning. interestingly, although dpr significantly underperforms with respect to colbert, the improvement due to expansion with generative summaries is much larger for dpr than for colbert, which validates our initial intuition that it is possible to leverage the linguistic ability of a generative model to provide additional context and language structure for the downstream neural retrieval model to consume. since dpr encodes the entire piece of text as a single dense vector, providing it with better-structured documents is more advantageous for dpr than colbert, which is capable of falling back to term matching through dense token embeddings. without such expansion, dpr is even less effective than bm25 as shown in the no expansion condition in ## conclusion and future work in this paper, we proposed a generative model finetuning approach fortify for document expansion. fortify tailors a generative model to a specific kind of noisy document and a downstream retrieval model through orpo, a preference optimization approach. we showed that models finetuned with fortify provide more effective expansion summaries than an out-of-the-box llama model. the resulting fortified llama model also demonstrates robustness to documents and retrieval models beyond the ones predefined during orpo fine-tuning. beyond the success of fortify on noisy text, we would like to explore it on other general ad hoc retrieval tasks to tailor the retrieval to a specific domain, corpus, or even user. given the flexibility of preference optimization, we believe fortify can be adapted to arbitrary retrieval model preference.