A Survey of Model Architectures in Information Retrieval
Authors: Zhichao Xu
Fengran Mo
Zhiqi Huang
Crystina Zhang
Puxuan Yu
Bei Wang Phillips
Jimmy Lin
Vivek Srikumar
Venue: Accepted by TMLR ()

Abstract: The period from 2019 to the present has represented one of the biggest paradigm shifts in information retrieval (IR) and natural language processing (NLP), culminating in the emergence of powerful large language models (LLMs) from 2022 onward. Methods leveraging pretrained encoder-only models (e.g., BERT) and decoder-only generative LLMs have outperformed many previous approaches, particularly excelling in zero-shot scenarios and complex reasoning tasks. Our survey study investigates the evolution of model architectures in IR, focusing on two key aspects: backbone models for feature extraction and end-to-end system architectures for relevance estimation. The review intentionally separates architectural considerations from training methodologies, in order to provide a focused analysis of structural innovations in IR systems. We trace the development from traditional term-based methods to modern neural approaches, particularly discussing the impact of transformer-based models and subsequent large language models (LLMs). We conclude with a forward-looking discussion of emerging challenges and future directions, including architectural optimizations for performance and scalability, handling of multimodal, multilingual data, and adaptation to novel application domains such as autonomous search agents that might be the next-generation paradigm of IR.

Published in Transactions on Machine Learning Research (02/2026)
A Survey of Model Architectures in Information Retrieval
Zhichao Xu
zhichao.ou@ulaah.ct eduu
Kahlert School of Computing University of Utah
Fengran Mo
Cengran..oouumontre
University of Montreal
Zhiqi Huang Capital One
ahag.huangg@capitallo..cmm
Crystina Zhang University of Waterloo
Singgorgsina..znnngvvutetteo
Puxuan Yu Snouflake Inc.
puaruan.yu.ssoouffae.ccmm
Bei Wang
beiwang@sci. utah. edu
Kahlert School of Computing Scientific Computing and Imaging (SCI) Institute University of Utah
Jimmy Lin University of Waterloo
jimmylin @uwaterloo.co
Vivek Srikumar
suivek@es. utah. edu
Kahlert School of Computing University of Utah
Reviewed OpenReview: https: openreview. net/ forum? id= dd=aAIbbbRRrr
Abstract
The period from 2019 to the present marks of the most significant paradigm shifts in information retrieval (TR) and natural language processing (NLP), culminating in the emergence of powerful large language models (LLMs) from 2022 onward. Methods based on pretrained encoder-only architectures (e.g., BERT) as well as decoder-only generative LLMs have outperformed many earlier approaches, demonstrating particularly strong performance in zero-shot scenarios and complex reasoning tasks. This survey examines the evolution of model architectures in IR, with focus on two key aspects: backbone models for fea- ture extraction and end-to-end system architectures for relevance estimation. To maintain analytical clarity, deliberately separate architectural design from training methodolo- gies, enabling a focused examination of structural innovations in IR systems. We trace the progression from traditional term-based retrieval models to modern neural approaches, highlighting the transformative impact of transformer- based architectures and subsequent LLM developments. The survey concludes with a forward-looking discussion of open chal- lenges and emerging research directions, including architectural optimization for efficiency and scalability, robust handling of multimodal and multilingual data, and adaptation to novel application domains such autonomous search agents, which may represent the next paradigm in IR.

---

Published in Transactions on Machine Learning Research (02/2026)
1
Introduction
Boolean Model
Background & Terminology $2
Vector Space Model
Traditional IR Models $3
Probabilistic Model
Statistic Language Model
Machine Learning Bearning-bbased Model
Learning-to Ra-R Model Architectures $4
Neural LTR Model
A Survey of IR Model Architectures
Representation-baese Neural Ranking Model
Pre-BERT Neural Ranking Models $5
interaction-baser Neural Ranking Nodel
Hybrid Model
Cross pross-encoder Reranking Model
encoddr Retrieval Model
IR Model Architecture with Pre-trained Transformers 56
Advanced Interaction and Hybrid Model
Orthogonal Improvements and Analysis
Large Language Models for Information Retrieval 57
LLM as Retriever
LLM as Reranker
Generative Retrieval
Architecture for Multimodal and Multilingual Data
Architectures for Diverse Scenarios and Constraints $8
Perforraace-Effiiccenny Tradeoff
Calibration and Confidence Estimation
Energing Directions and Challenges 59
Comain-Specifal Applications
Figure 1: High-level overview of the survey structure and scope.
2

---

Published in Transactions on Machine Learning Research (02/2026)
Information Retrieval (IR) aims to retrieve relevant information sources to satisfy users' information needs. In the past decades, IR has become indispensable for efficiently and effectively accessing vast amounts of information across various applications. Beyond its traditional role, IR now also plays a critical role in assist- ing large language models (LLMs) to generate grounded and factual responses under the generative AI cra. Research in IR primarily centers on two key aspects: (1) extracting beller query and document fealure rep- resentations, and (2) developing more accurate relevance estimators. Extracting better query and document feature representations focuses on modeling textual content so that queries and documents can be compared in meaningful space, ranging from early cerm-frequency vectors (e.g., TF-IDF (Sparck Jones, 1972) and BM25 (Robertson & Zaragoza, 2009)) to modern contextual embeddings derived from pre-trained language models. Developing more accurate relevance estimators then builds on these representations to assess how well a document satisfies an information need, using scoring functions or learned ranking models such as BM25, neural interaction models, or later learning-to- to-k frameworks that combine multiple signals. The approaches for extracting query and document features have evolved from traditional term-based mothods, such as boolean logic (Radecki, 1979; Kraft & Buell, 1983) and vector space models (Salton el al., 1975), to modern solutions such dense retrieval based on pre-trained language models (Lee et al., 2019; Karpukhin et al., 2020; Logeswaran et al., 2019; Lin et al., 2022, inter alia).
Relevance estimators have evolved alongside advances in feature representations. Early approaches, including probabilistic and statistical language models, computed relevance with simple similarity functions based on term-based features. bearning-to-rant (LTR) techniques later emerged, incorporating machine learning models like support vector machines (Cortes & Vapnik, 1995), boosting methods (Kearns & Valiant, 1994; Freund & Schapire, 1995) as well as multi-layer neural networks for relevance estimation (Li, 2011). The success of LTR methods can be largely attributed to their extensive derived from both statistical properties of text terms and user behavior data collected from web browsing traffic (Qin& & Liu, 2013). In the 2010s, a vast literature explored neural rerankers in different architectures to capture the semantic similarity between queries and documents (Pang et al., 2016; Guo et al., 201 6a; Xiong et al., 2017; Dai et al., 2018, inter alia). Then pre-trained transformers, represented by BERT (Devlin et al., 2019) and its variants (Liu et. al., 2019; Sun et. al., 2019; Lan et al., 2020; Beltagy et al., 2020), quickly revolutionized the model design, leading to an era where retrieval and ranking models adopt simpler architectures for relevance estimation, such as dot product operations and MLP prediction heads, which operate on learned neural representations (MacAvaney et al., 2019b; Lee et al., 2019; Karpukhin et al., 2020; Nogueira et al., 2020; Lin et al., 2022; Formal et al., 2021b;a).
of manually engincered features,
Recent advancements in LLMs have revolutionized applicd machine learning (ML) communities, including IR. One intriguing property of modern instructionnffolowinng LLMs (e.g., ChatGPT OpenAT (2022)) is that they can be used for fcature extraction and relevance estimation, achieving strong performance without extensive training (Ni et al., 2022a; Neelakantan et al., 2022; BehnamGhader et al., 2024; Sun et al., 2023; Qin et al., 2024b, inter alia). The rise of these models builds upon rich foundation of neural architectures, including the classical Transformer architecture with multi-head attention (MHA, Vaswani et al., 2017), Recurrent Neural Networks (RNN, Elman, 1990), attention mechanisms (Bahdanau et al., 2014), and pre- trained static word representation atins like Word2Vec (Mikolov el al., 2013) and GloVe (Pennington et al., 2014) (Collobert et al., 2011; Le & Mikolov, 2014, inter alia).
This work reviews the evolution of model architoctures in IR (with an overview in Figure 1). Here, the mcan- ing of model architecture is twofold: it describes (1) backbone models for extracting query and document feature representations, and (2) end-to-end system architectures that process raw inputs, perform feature extraction, and estimate relevance. Different from prior works and surveys (Lin et al., 2022; Zhu et al., 2023), we intentionally separate our discussion of model architectures from training methodologies and deployment best practices to deliver a focused architectural analysis. This analysis centers on the core components of Al infrastructure in the LLM era. The shift towards neural architectures, particularly Transformer based models, has fundamentally transformed IR by enabling rich, contextualized representations and improved capacity for handling complex queries. While this evolution enhanced retrieval performance, it also presents new challenges, especially with the development of LLMs. These challenges include the need for architoc- tural innovations to optimize performance and scalability, handle multimodal and multilingual data, and understand complex instructions. Moreover, as IR systems are increasingly integrated into diverse applica-
3

---

Published in Transactions on Machine Learning Research (02/2026)
tions -fomm-from robotics (Xie et al., 2024b), protein structure discovery (Jumper et al., 2021) to autonomous agents (Wu et. al., 2023a; Chen et al., 2025; Hu et al., 2025; OpenAT, 2025; Wu et al., 2025b, inter alia) that are capable of reasoning and search the field must evolve beyond traditional search paradigms. We conclude this survey by examining these challenges and discussing their implications for the future of IR model architectures research.
2 Background and Terminology
We focus on the classical ad hoc retricval task, which forms the foundation for many modern IR applications. In this section, we define the core lask, introduce key system architectures and evaluation paradigms, and clarify the scope of our architectural review.
Task Definition and Evaluation. Given query Q, the task is find ranked list of k documents, denoted as (D1, D2,..., DuJ. that exhibit the highest relevance to Q. This is achieved either by retrieving top- & documents from a large collection C (IC) > k), which typically comprises millions or billions of documents, or by reranking the top-k candidates returned by a retriever. System performance is measured using standard, list IR metrics. Common metrics include:
Mean Reciprocal Rank (MRR): Measures the rank of the first relevant document. is particu- larly useful for tasks where finding correct answer is the primary goal (e.g., question answering).
Recall@k: Measures the fraction of all relevant documents that are found within the top-k results. It emphasizes the system's ability find all relevant items.
Normalized Discounted Cumulative Gain (nDCG@k), A sophisticated metric that evaluates the quality of the ranking over the top-k documents. It gives higher scores for ranking highly relevant documents at the top of the list and uses a logarithmic discount to penalize relevant documents that appear lower in the ranking. It is the de facto standard for evaluating ranked lists with graded relevance judgments.
The Multi-Stage "Retrieve-then Rerank" Architecture. Modern large-scale IR systems almost uni- versally operate on a multi-stage pipeline, commonly known as the "retrieve-then-reraakk"" architecture, This design balances the tradeoff between efficiency and effectiveness.
1. Retrieval (or First-Stage Ranking): In the first stage, computationally efficient but less precise model scans the entire collection (potentially billions of documents) to quickly identify an initial set of several hundred or thousand candidate documents. These models, often called retrievers, must be extremely fast. Examples include traditional models like BM25 (Section 3) or modern bi-encoder models (Section 6).
Reranking (or Sccond-Stage Ranking): In the socond stage, a more powerful but computation- ally expensive model, known reranker, is applied only to the small candidate set returned by the retriever. This model can afford to perform deep, fine-graines analysis of the interaction between the query and each candidate document to produce more accurate final ranking. Examples include Learning-to-Raut models (Section 4) and modern cross-encoder transformers (Section 6).
This two-stage process is a central architectural pattern IR, and much of the evolution discussed in this survey can be understood developing more advanced models for each of these stages.
Query and Document. A query expresses an information need and serves as input to the ad hoc retrieval system. We denote document as the atomic unit for retrieval and ranking. Our discussions are primarily based on text-based documents, but document can also refer a webpage or an email, depending on the actual IR application of interest.
4

---

Published in Transactions on Machine Learning Research (02/2026)
Disentangling Model Architecture from Training Strategies. Similar to other applied ML domains, the performance of an IR system is product of its model architecture, its training methodology (e-g., loss functions, data augmentation, optimization algorithms), and deployment best practices (e.g., indexing, quan- tization, parallclization, aggorithm-hardeee these aspects to provide a focused analysis on the evolution of model architecture. This focus allows for a clearer narrative on how the core components for representation learning and relevance estimation have changed over time, from term-based logic to deep neural networks. We refer readers to dedicated surveys for in-depth reviews of training strategies and other related topics (Schützee et al., 2008; Lin et al., 2022; Song et al., 2023).
co-design). In this survey, we intentionally seck to disentangle
3 Traditional IR Models
In this section, we briefly review traditional Information Retrieval (IR) models prior to neural methods, with focus on the Boolean model, vector space model, probabilistic model, and statistical language model. These models, which serve as the foundation for later developments in IR (Sections 4 to 7), are built upon the basic unit "term" their representations (Nie, 2010).
Boolean Model. In the Boolean Model, document D is represented by a set of terms it contains, i.e., D tt., taj, and a query Q is represented as a similar boolean expression of terms. A document is considered relevant to query only if logical implication D + Q holds, i.c., the document representation logically implies the query expression. This basic model can be extended by incorporating term weighting, allowing both queries and documents to be represented as sets of weighted termss Consequently, the logical implication D + Q is also weighted. Common approaches for this include using a fuzzy set extension of Boolean logic (Radecki, 1979; Kraft Buell, 1983) and the p-norm (Salton et al., 1983).
Vector Space Model. In Vector Space Models (VSMs, Salton et al., 1975), the queries and documents are represented by vectors, e.g., =< q1. 12.92..... In > and D =< d1, dh.d2.....dd >. The vector space is and each element (q: di, S S n) in the
defined by a vocabulary of terms V =< 1, 12, vectors represents the weight of the corresponding term in the query the document. The weights qi di could be binary, representing presence or absence. Given the vector representations, the relevance score is estimated by a similarity function between the query Q and the document. DD The weights & or dę can be determined by more sophisticated schema (Salton & Buckley, 1988), such as TF-IDF (Sparck Jones, 1972) and BM25 (Robertson & Zaragoza, 2009; Robertson el al., 1995). This allows for more abundant features that can improve the capacity and accuracy of the models. Besides, given the vector representations of query Q and document D, the most commonly used is cosine similarity, defined as:
.....,,,, 11,
sim (2 = Q X DD
where Q
D is the dot product and [Q], [D] denotes the length of the vector.
Probabilistic Model. In the probabilistic model, the relevance score of document D to query Q depends on set of events {a:}f representing the occurrence of term i; in this document. The simplest probabilistic model is the binary independence retrieval model (Robertson & Jones, 1976), which assumes terms are independent so only = 1 and x, = 0 exist in the representation. Given a set of sample documents whose relovance is judged, the estimation of the relovance score can be derived as
log Ti(T n1 -
Score(Q,
Σ (x.m1)ED (R - r:)(m+ rz)
where and R are the total number of sampled judged documents and relevant samples, and ny and ra denote the number of samples and relevant samples containing t, respectively.
In contrast, line of statistical retrieval functions such as TF-IDE (Sparck Jones, 1972) move beyond binary term indicators by incorporating term frequency (TF) and inverse document. frequency (IDF), allowing
5

---

Published in Transactions on Machine Learning Research (02/2026)
more nuanced term weighting while still assuming term independence. algorithm (Robertson et al., 1995):
We illustrate the famous BM25
Ss. Js.(k + 1)
(M25(Q,2)
E (FF( GEQnD
fi+ k1 (1-b+b- b. D
avgdl
where Si is the frequency of term & in document D, (D] the length of the document, avgdl is the average document length in the collection, and kn and b are hyperparameters typically set between [1.2, 2.0] and [0.5, 0.8], respectively. The inverse document frequency term is computed as IDt(t;) = log N-n mc+0.5 N is the total number of documents in the collection and n; is the number of documents containing term 4,
where
The smoothing mechanisms (Baeza- Yates & Ribeiro-Neto, 1999) are necessary to deal with zero occurrences of t. Except for the binary independence retrieval model, more sophisticated probabilistic models have been proposed in the literature (Wong & Yao, 1989; Fuhr, 1992), such as the inter-dependency between terms (Van Rijsbergen, 1979).
Statistical Language Model. The general idea of statistical language model is to estimate the relevance score of document D to query Q via P(D) Q) (Ponte & Croft, 1998). Based on Bayes' Rule, P )) can be derived as directly proportional to ()D))) For simplification, most studies assune a uniform distribution for P(D). The main focus is on modeling P()[D) as a ranking function. By treating the query as a set of independent terms Q - {4]==1, we have P(2ID) = II P(4[DD). The probability P(4[D) is determined using a statistical language model 0D that represents the document. The relevance is then estimated by logllikel:oood: Score( D) log P(Q)0p) = En,e log (()) where the estimation of the language model Op is usually achieved by maximum likelihood.
The statistical language models for IR (Miller et al., 1999; Berger & Lafferty, 1999; Song & Croft, 1999; Hiemstra & Kraaij, 1999) also encounter the problem of zero occurrences of query term 4, i.c., the probability P(Q(0p) becomes zero if a query term f; does not appear in the document. This is too restrictive for IR, as a document can still be relevant even it contains only some of the query terms. To address this zero-pra cerroprobability issue, smoothing techniques are applied, assigning small probabilities to terms that do not appear in the document. The principle behind smoothing that any text used to model language caplures only a limited subset its linguistic patterns (or terms, in this case). The commonly used smoothing methods (Zhai & Lafferty, 2004; Zhai et al., 2008) include Jelinek-Mercer smoothing (Jelinek, 1980), Dirichlet smoothing (MacKay & Peto, 1995), etc.
These foundational models, while useful, share common characteristic: they rely on heuristic- scoring functions derived from statistical properties of terms. Althongh their parameters can be tuned (e.g., kn and b in BM25), the functional form of the model is fixed. A natural evolution is to frame ranking as a supervised machine learning task, where model learns to combine various signals of relevance automatically from labeled data. This approach allows for the systematic combination of not only scores from traditional models like BM25 but also multitude of other features describing the query, the document, and their interaction. This paradigm shift from hand-crafted formulas to learned functions is the core idea behind Learning-to- to- Ra-R models, which we discuss next.
4 Learning-te -oaiingtto-Rakk Model Architectures
Different from traditional IR models that rely heuristic-based scoring formulas (Section 3), Learning- to-Rank (LTR) frames ranking as a supervised machine learning problem (Liu, 2009). The core idea is to train a model that can optimally combine a wide array of signals, or "features", to predict the relevance of documents to a query. For each query-document pair (Q,, D,), a k-dimensional feature vector x; E g& is extracted, and a relevance label y; (e.g., from human judgments) is provided. The goal is to learn a ranking model parameterized by that minimizes an empirical loss I(-) on labeled training set V
L =
((fo(x)).1).
(x..)))
6

---

Published in Transactions on Machine Learning Research (02/2026)
Table 1: A list of learning-to- -o-rinnkkt--akk works and their model architectures.
Name
Backbone Architecture
Loss Function
MART (Friedman, 2001)
Boosting
Pointwise
RANkBoOST (Freund et al., 2003)
Boosting
Pairwise
RANKNET (Burges et al., 2005)
Neural Network
Pairwise
RANKSVM (Joachims, 2006)
SVM
Pairwise
AAMBDARANK (Burges et al., 2006)
Neural Network
Pairwise
LISTNET (Cao et al., 2007)
Neural Network
Listwise
SOFTRANK (Taylor et al., 2008)
Neural Network
Listwise
LISTMLE (Xia et al., 2008)
Linear
Listwise
LAMBDAMART (Burges, 2010)
GBDT
Listwise
APPROXNDOG (Qin et al., 2010)
Linear
Listwise
DLCM (Ai et al., 2018a)
Neural Network
Listwise
GSF (Ai et al., 2019)
Neural Network
Listwise
APPROXNDCG (Bruch ct al., 2019)
Neural Network
Listwise
SETRANK (Pang et al., 2020)
Self Atlention Blocks
Listwise
LTR methods are typically categorized into three main approaches based on their input and loss function: pointwise, pairwise, and listwisc.
Feature Engincering in LTR. A cornerstone of traditional LTR is the meticulous engineering of the feature vector x,. These features are designed capture diverse aspects of relevance and can be grouped into several categorics: (1) Query-based features, such the number of terms in the query; (2) Document- based features, which are query-independenl, such as document length, PageRank (Brin & Page, 1998), or the number of incoming URL links; and (3) Query-document interaction features, which form the largest and most critical group. This category includes scores from traditional IR models like BM25 and language models, counts matching terms, proximity features measuring how close query terms are in the document, and various Tr-IDe-related statistics. The power of LTR lies in its ability to learn complex, non- non-linear combinations of these diverse signals, moving beyond what single hand-tuned formula could achieve.
4.1 Pointwise, Pairwise and Listwise Approaches with ML Models
The pointwise approach is the simplest, trcating each document independently. It frames the problem as a regression or classification task, where the model f(x;) aims to predict the exact relevance label yı. While straightforward, this approach ignores the crucial fact that ranking about the relative order of documents, not their absolute scores (Burges, 2010).
The pairwise approach addresses this by focusing the relative order of document pairs. Given two documents D; and 'Dj for the same query, the goal is to predict which one is more relevant. This transforms ranking into a binary classification problem. Seminal pairwise models include RANKSVM (Joachims, 2006), which adapts the Support Vector Machine framework to maximize the number correctly ordered pairs, and RANKNET (Burges et al., 2005), which uses a neural network and probabilistic cost function based on pairwise logistic loss. While more alignod with the nature of ranking than the pointwisc approach, pairwise methods still do not directly optimize the list-based evaluation metries (e.g., nDCG, MRR) that are standard in IR evaluation.
The listwisc approach directly tackles this issue by defining the loss function over an entire list of documents for a given query. These methods aim to directly optimize ranking metrics. A pivotal line of work began with LAMBDARANK (Burges et al., 2006), which observed that for gradient-based optimization, one only needs the gradients of the loss function. in an IR metric (like nDCG) when two documents in a ranked list are swapped. This technique was then combined with Multiple Additive Regression Trees (MART), Gradient Boosted Decision Tree (GBDT)
introduced "Lambda gradients", which are derived from the change

---

Published in Transactions on Machine Learning Research (02/2026)
relevance score
relevance score customized deep network
customized
customized deep network
deep network
word embeddings
word embeddings
word embeddings
word embeddings
where is whitemarsh island
the strategy of island
where is whitemarsh island
the strategy of island ...
(a) Representation- neural reranker
(b) Interaction- neural reranker
Figure 2: Illustration on neural ranking models. Brown boxes indicate uncontextualized word embeddings
(e.g., Word2vee).
algorithm (Friedman, 2001), to create LAMBDAMART (Wu et al., 2010). Due to its strong performance and robustness, LAMBDAMART became a dominant, industry standard for many years (Ke et al., 2017).
4.2 Neural LTR Models
While LAMBDAMART represents peak for GBDT-based LTR, early works also explored neural networks for the ranking function fa. RANKNET and LAMBDARANK both parameterized the LTR model with neural networks. More recent works such as GSF (Ai et al., 2019) and APPROXNDCG (Bruch et al., 2019) have continued this trend, using multiple fully connected layers and designing differentiable approximations of TR metrics. Other architectures like DLCM (Ai et al., 2018a), based on RNNs, and SETRANK (Pang et al., 2020), using sell-attention, explore ways to model the entire document list jointly. A rigorous benchmark by Qin et al. (2021) compared the performance of these modern neural ranking models against strong GBDT-basee based basclines. A summary of LTR models and their backbone architectures is provided in Table 1.
4.3 Orthogonal Directions
Beyond core model architectures, LTR research has explored many other important directions. A significant portion of the literature focuses on loss functions and feature transformations (Qin et al., 2021; Bruch et al., 2019; Burges, 2010). Other critical areas include developing methods for unbiased relevance estimation from biased user foedback (c.g., clicks) (Joachims et al., 2017; Ai et al., 2018b; Wang et al., 2018; Hu et al., 2019) and jointly optimizing for both effectiveness and fairness in ranking systems (Singh & Joachims, 2018; Biega et al., 2018; Yang et al., 2023b;a;c). We omit detailed discussions here and refer readers to the original papers and prior surveys (Liu, 2009; Li, 2011).
Despite their success, traditional LTR models have fundamental ceiling. Their reliance on handcrafted features prevents them from operating in an end-to-end manner and, more importantly, limits their ability to bridge the lexical gap the difference between the words in query and the semantically related words in a relevant document. Their understanding is based on pre-defined statistical signals, not the underlying meaning of the text. This limitation created a clear need for new class of models capable of learning semantic representations directly from raw text, setting the stage for the rise of neural ranking (Section 5).
5 Neural Ranking Models
Neural ranking models emerged to directly address the key limitations of LTR (Section 4). By learning se- mantic representations directly from raw text, they could automatically bridge the lexical gap gap-for for instance, recognizing that query for "computer" is conceptually related to a document about a "PC" without rely- ing on term overlap. This end-to-end approach simultaneously eliminated the laborious process of manual
8

---

Published in Transactions on Machine Learning Research (02/2026)
feature engineering, shifting the paradigm from engineering statistical signals to learning semantic patterns from data."
Depending on how queries interact with documents during network processing, neural ranking models can be roughly divided into representation-based models and interaction-based models (Guo et al., 2016a). This division reflects a fundamental tradeoff between efficiency and matching depth. Representation-Dasee models pre-encode documents into vectors offline, enabling highly efficient retrieval suitable for first-pass ranking. In contrast, pearactton-basedd models process the query and document together, allowing for deeper and more precise matching at higher computational cost, making them ideal for reranking smaller set of candidates.
5.1 RepresentationnBasse Models
This genre of models can be regarded as an extension of Vector Space Models (Section 3), which independently encode queries and documents into a shared latent vector space, where relevance is determined through simple comparison functions such as cosine similarity or dot product, as illustrated in Figure 2a. This approach maintains clear separation between query and document processing, with no interaction occurring during the encoding procedure. The core architectural challenge is designing an encoder network that transforms a variable-length sequence of term embeddings into single, fixed-size semantic vector.
The Deep Structured Semantic Model (DSSM, Huang et al., 2013; Gao et al., 2014) is an early example. It utilizes word hashing (a technique to manage large vocabularies by grouping words into a smaller number of hash buckets) and multilayer perceptrons (MLPs) to independently encode term vectors of queries and documents, enabling the computation of ranking scores based on the cosine similarity of their embeddings. Later works modify DSSM's encoder network better capture richer semantic and contextual information. Convolutional DSSM(C-DSSM, Shen et al., 2014a) leverages convolutional neural network (CNN) archi- tecture. Specifically, it applies 1D convolutions over the sequence of word embeddings, allowing the model to learn representations for n-grams and local phrases. A max-pooling layer then selects the most salient local features to form the final document vector. Another variant of DSSM replaccs MLPs with recurrent layers such as Long Short-Term Memory (LSTM) network (Hochreiter & Schmidhuber, 1997; Palangi et al., 2016; Wan al., 2016; Cohen & Croft, 2016) or tree-structured networks (Tai et al., 2015). The LSTM processes the text sequentially, and its recurrent nature allows it to capture word order and long-range dependencies across the entire text, with the final hidden state often used as the comprehensive representation for the query or document. Based on the assumption of documents' hierarchical structure, Yang et al. (2016b); Song et al. (2018); Zhu et al. (2019) use attention (Bahdanau et al., 2014) to model token, phrase and sentence representations for enhanced document/ passage representations.
Tn line with these works, the NLP community has also extensively investigated tions. Le Mikolov (2014) proposed Paragraph Vectors (Doc2VEc), an unsupervised algorithm that Icarns fixed-length feature representations from variable-length pieces of texts, which is based on single-hidden-ben neural network. Kim (2014) studied CNN for sentence representations, while Wieting et al. (2016) proposed to use LSTM network. Arora et al. (2017) reported a weighted average of pre-trained word embeddings fine-tuned with unsupervised random walk algorithm can outperform more complex neural networks on the sentence similarity task. However, weighted averaging word embeddings ignores the word order, which fails to capture the rich, contextual information in longer, more complex documents.
representa-
To summarize, representation baaed models excel in scenarios requiring global semantic understanding and offer significant computational advantages through their ability to pre-compute document representations offline (Guo et al., 2020). However, these approaches also face inherent limitations due to their reliance on fixed-size embedding vectors, which can struggle to capture all relevant information from the original text and may not effectively handle precise lexical matching requirements. These limitations are the focus of interaction-based models.
1 For structural clarity, this section focuses on neural information retrieval, covering neural network basod retricval models developed prior to the advent of pre-trained transformers. For more comprehensive treatment, we refer readers the dedicated surveys (Onal al., 2018 8; Mitra et al., 2018; Xu al., 2018).
9

---

Published in Transactions on Machine Learning Research (02/2026)
5.2 Interaction Based Models
Different from representation-bant based models, interaction- based models (Figure 2b) process queries and docu- ments jointly through neural networks. Instead of compressing each text into a single vector, they first build detailed, low-level interaction representation between the query and the document, and then use neural networks to learn hierarchical matching patterns from this representation. The model's output is typically scalar relevance score of the input query-document pair. Various network architectures have been proposed under this paradigm. MATCHPYRAMID (Pang et al., 2016) employs CNN over the interaction matrix be- tween query and document terms. The interaction matrix is treated as an image, allowing the CNN with its 2D filters to capture local matching patterns (e.g., phrases or bigrams matching) through convolution and pooling operations (Hu et al., 2014).
Building upon the concept of ineracctionfocuseer models, Guo et al. (2016a) highlighted the importance of exact term matches in neural ranking models and proposed the Deep Relevance Matching Model (DRMM). Rather than a single interaction matrix, DRMM creates matching histogram for each query term. This histogram discretizes the similarity scores against all document terms into bins, effectively capturing the distribution of matching signals (c.g., how many terms in the document are an exact match, a strong semantic match, or a weak match to a given query term). An MLP then learns the relevance contribution from these histogram features.
Kernel-Based Neural Ranking Model (K-NRM, Xiong et al., 2017) further advances interaction-based ap- proaches. It employs radial basis function (RBF) kernels to transform the query-document interaction matrix into more informative feature representation. Each kernel corresponds to a certain similarity level (e.g., "exact match", "strong match", "weak match"). The model uses these kernels to produce "soft-TF" counts for each query term countinng how many document words match the query term at each similarity level. These soft-match features are then aggregated and fed into a simple feed-forward network to compute the final relevance score. This kernel-based mechanism enables models to capture nuanced matching features, enhancing their ability to model complex query-document interactions. CONv-KNRM (Dai et al., 2018) later extends it to convolutional kernels to capture n-gram level soft matches, further improving matching granularity.
In line with these works, the interaction matrix-based approach have been explored for short text match- ing (Lu & Li, 2013; Yin et al., 2016; Yang et al., 2016a) as well as long document ranking (Mitra et al., 2017: Hui et al., 2017: 2018, inter alia). Mulsee-erseeeetve CNN approaches compare sentences via diverse pooling functions and filter widths to capture multiple perspectives between texts He et al. (2015). ANMM (Yang et al., 2016a), as an example of attention-based methods, computes passage terms' attention weights over query terms using a query attention network and achieves performance improvement compared to CNN- based bascline (Severyn & Moschitti, 2015). Term adjacency and positional information represent another important dimension of interaction modeling. Models such as MATCHPYRAMID, PACRR, and CONVKNRM capture term adjacency patterns and position-dependent interactions (Pang et al., 2016; Hui et al., 2017: Dai et al., 2018).
The interaction functions in these models can be categorized as either non- non-parametric (using traditional similarity measures like cosine similarity, dot product, or binary indicators) or parametric (learning similar- ity functions from data through neural networks) (Dong et al., 2022b). While interaction-focused models require one forward pass through the entire model for each potentially relevant document, making them com- putationally more expensive than representation-focusee approaches, they typically achieve superior ranking quality due to their ability to capture fino-grainod matching signals. We list some representative works in Table 2 and direct readers to these works for architectural details.
5.3 Hybrid Models
Recognizing the complementary strengths of representation-focuee and Interaction-focusee architectures, rescarchers have proposed hybrid models that combine the efficiency of representation- basod mothods with the effectiveness of interaction- based approaches. These models represent third category in neural ranking architectures, alongside the two primary approaches.
10

---

Published in Transactions on Machine Learning Research (02/2026)
Table 2: list neural ranking models and their model architectures.
Name
Architecture
Backbone
Embeddings
DSSM (Huang et al., 2013)
Representation based
MLP
Semantic Hashing
CDSSM (Shen et al., 2014a)
Representation-base based
CNN
Semantic Hashing
CLSM (Shen et al., 2014b)
Representation- based
CNN
Semantic Hashing
ARC-I (Hu al., 2014)
Representation- based
CNN
Word2Vec
Tai et al. (2015)
Representation- baase
Tree-structured LSTM
GloVe
LSTM-RNN (Palangi et al., 2016)
Representation-bun based
LSTM
Randomly Initialized
MV-LSTM (Wan et al., 2016)
Representation based
Bi-LSTM
Word2Vec
DESM Nalisnick et al. (2016)
Representation- based
MLP
Randomly Initialized
Li (2013)
Representation- based
MLP
Randomly Initialized
ARC-II (Hlu et al., 2014)
Interaction-
CNN
Word2Vec
MATCHPYRAMID (Pang et al., 2016)
Interaction-bel based
CNN
Randomll Initialized
DRMM (Guo et al., 2016a)
Interaction-basse
MLP
Word2
ABCNN (Yin et al., 2016)
Interaction-bast based
CNN Attention
Word2Vec
ANMM (Yang et al., 2016a)
Interaction-ban based
Attention
Word2Vec
DESM (Nalisnick al., 2016)
Interaction-basee
MLP
Word2Vec
K-NRM (Xiong et al., 2017)
Interaction-baser
MLP + RBF kernels
Word2
CONv-KNRM (Dai et al., 2018)
Interaction-
CNN
Word2Vec
PACRR (Hui al., 2017)
Interaction-based
CNN RNN
Word2Vee
Co-PACRR (Hui et al., 2018)
Interaction-bascd
CNN
Word2Vec
TK (Hofstätter et al., 2020c)
Interaction- based
Transformer + Kernel
GloVe
TKL (Hofstätter al., 2020a)
Interaction-basee
Transformer + Kernel
GloVe
NDRM (Mitra et al., 2021)
Interaction-ban
Conformer + Kernel
BERT
The most notable example is DUET (Mitra et al., 2017), which employs two separate deep neural networks operating parallel. One network performs local interaction-based matching similar to interaction-focusee models, while the other learns distributed representations for query and document separately, similar representation-focusee, approaches. The term interaction matrix between query and document feeds into the exact matching layers, while term embeddings of the input sequence enter the semantic matching layers. The outpuls from both networks are then combined using fully connected network to produce the final ranking score.
Different from the metric learning theme of representation-basse models, a line of works formulates the ranking problem as a classification problem (commonly referred as Extreme Label Classification, or XMC), where the input the query, and the output is a probability distribution over the corpus, where each document is unique "class" "label". Instead of optimizing the similarity between query and document representations, XMC models aim to predict the correct subset of relevant document IDs (Prabhu & Varma, 2014; Jain et al., 2016; Liu et al., 7017; Jain et al., 2019). In the inference time, XMC methods use tree-based hierarchies or cluster-based sampling to quickly narrow down the search path to the likely labels without scanning every candidates (Prabhu al., 2018; You et al., 2019). ArTENTIONXML (You et al., 2019) uses an attention mechanism to focus on specific parts of the input text that are most relevant the label's semantic mcaning, and thus can be considered hybrid model. We refer readers to (Dasgupta al., 2023) for comprehensive review of XMC methods.
This hybrid architecture demonstrates that combining distributed representations with traditional local reprosentations is favorable, with the combined approach significantly outperforming either neural network individually. More recent hybrid approaches have focused on reducing computational costs while maintaining effectiveness, with some models incorporating cached token-level representations to enable faster query- document interactions when document representations are pro-computed (Wrzalik Krochel, 2021). The success of hybrid models has established that interaction-basee and representation- based approaches can be effectively combined for further improvements in ranking performance (Liu et al., 2018).
11

---

Published in Transactions on Machine Learning Research (02/2026)
5.4 Orthogonal Directions
In addition to the development of network architecture, pre-trained embeddings (Salakhutdinov & Hinton, 2009; Mikolov et al., 2013; Pennington et al., 2014; Le & Mikolov, 2014) provide semantic- basadd term rep- resentations to enable neural ranking models to focus on learning relevance matching patterns, improving training convergence and retrieval performance on both representation-bate based and interaction- based mod- els (Levy et al., 2015). Both GloVe (Pennington et al., 2014) and Word2Vec (Mikolov et al., 2013) learn dense vector representations for each vocabulary term from large-scale text corpora. By initializing the embedding layer with these pre-trained vectors, models start with strong semantic foundation, which proved crucial for performance, especially on smaller training datasets (Guo et al., 2016b). Interaction-based models with cross-lingual word embeddings (Joulin et al., 2018) for cross-lingual reranking have also been explored (Yu & Allan, 2020). Table 2 shows a list of neural ranking models and backbonn architectures. Researchers have explored different backbone neural network architectures in this era, inchuding Convolutional Neural Network (CNN, LeCun et al., 1989), Long Short Term Memory (LSTM, Hochreiter & Schmidhuber, 1997) and kernel methods (Vert et al., 2004; Chang et al., 2010; Xiong et al., 2017).
Notably, line of research explores integrating kernel methods with the Transformer architecture ( (Vaswani et al., 2017). The main distinction between this line of rescarch and the models discussed in Section 6 is that the transformer modules here are not pre-trained on large-scale corpora like Wikipedia and C4 (Devlin et al., 2019; Raffel et al., 2020). We consider this line of research as an intersection between neural ranking models (Section 5) and retrieval with pre-trained transformers (Section 6). TK (Hofstätter et al., 2020c) uses a shallow transformer neural network (up to 3 layors) to encode the query Q and document DD scparately. After encoding, the contextualized representations are input to an interaction module inspired by K-NRM, where RBF kernels are used to create soft-match features from the contextualized embeddings. This fusion of transformer encoder with a kernel-based interaction mechanism allowed the model to achieve better performance-fflciinccy tradeoff compared to BERT-based reranker (Nogueira et al., 2019b). The main bottleneck of applying transformer architectures to long document reranking is O(n²) time complexity, where n denotes the document length. TKL (Hofstätter et al., 2020b) further improves upon TK with a local attention mechanism and leads to performance improvement on long document ranking.
The neural ranking models described above, particularly later developments like TK and TKL, demon- strated the potential of the transformer's attention mechanism for modeling relevance. However, the truc paradigm shift occurred when the IR community moved from using these architect ures trained from scratch to leveraging massive, pre-trained transformer models like BERT (Devlin et al., 2019) and its variants (Liu et al., 2019; Sun et al., 2019; Lan et al., 2020; Beltagy et al., 2020). This marked a fundamental change in approach: instead of designing novel, task-specific network backbones (e.g., CNNs, LSTMs) on top of static word embeddings, research shifted to fine-tuning a single, powerful, and deeply contextualized architecture for IR tasks. This new foundation did not eliminate the core architectural tradeoffis but rather recast them in more powerful form, leading to the development of cross-encoder rerankers and bi-encoder retrievers, which we explore next.
6 IR with Pre- Trained Transformers
BERT (Devlin et al., 2019) revolutionized research in both natural language processing (NLP) and informa- tion retrieval (IR). Its success is largely attributed to two key factors: (1) the Multi-Head Attention (MHA) architecture (Vaswani et al., 2017), which enables high-dimensional, contextualized loken representations; and (2) large-scale pre-training, which equips BERT with the ability to capture rich semantics and world knowledge. The expressive power of BERT has been extensively analyzed in prior work, e.g., (Rogers et al., 2020; Tenney et al., 2019; Clark et al., 2019).
High-level architectural families. Before discussing specific IR modeling architectures, it is useful to understand the architectural families of transformers, they have distinct implications for IR tasks:
Encoder-only models (c.g., BERT (Devlin et al., 2019), ROBERTA (Liu et al., 2019)) use a bidirectional self-attention mechanism, allowing each token's representation to be informed by the
12

---

Published in Transactions on Machine Learning Research (02/2026)
relevance score
relevance score
pooling (e.g. first-token, mean)
pooling (e.g. CLS, neural-rerankers, MaxSimi
Query Encoder
Document Encoder
Reranker
where whitemarsh island
the strategy of island (w are token-level scalar
where is whitemarsh island
the strategy of island
(b) Learned dense retrieval. weights)
(a) Transformer-based reranker
relevance score
relevance score
pooling
token interaction operation
(e.8. MaxSim)
[Wy. Wq. Wg.... Wq [W. We. Wil..., We....
regularizer legg. FIOPS, top-k)
post-processing op. (e.g. clustering, routing)
lexical encoder (e.g, BINARY, MLP, MLM)
Query Encoder
Document Encoder
Query Encoder
Document Encoder
where is whitemarsh island
the strategy of island
where is whitemarsh island
the strategy of island
(c) Learned sparse retriever Figure 3: Illustration on transformer-basee retricval and reranking models. Ycllow boxes indicate pretrained Transformers (e.g., BERT). Query lext, embeddings, and associated weights are color-coded in orange, whereas document representations are color-coded in blue.
(d) Multi-vector representations
entire input sequence (both left and right context). This deep contextual understanding makes them naturally suited for cerseettation-ocussee tasks. In IR, they have been the workhorses for building powerful bi-encoder retrievers and cross-encoder rerankers.
Decoder-only models (e.g., proprietary GPT series (Radford et al., 2019; Brown et al., 2020; OpenAl et al., 2024), weintt LLAMA (Touvron et al., 2023), MISTRAL (Jiang et al., 2023a)) use unidirectional (causal) self-attention mechanism, where each token can only attend to previous tokens in the sequence. This architecture is optimized for next-token prediction and, by extension, text generation. Thcir application to reprosentation tasks like retrieval is less dircct and often requires architectural adaptations lo create meaningful summary vectors from their unidirectional hidden states.
Encoder-decoder models (e.g., T5 (Raffel et al., 2020), BART (Lewis et al., 2020a)) combine both architectures. The encoder processes the input sequence bidiroctionally to create rich repre- sentation, which then conditions the decoder to generate an output sequence autoregressively. This beoquennceto-sequenee design makes them highly versatile. In IR, they can be framed rerankers (generating "relevant" or "irrelevant" token), document identifiers.
generative retrievers that directly generate
This section discusses IR architectures based on pre-trained transformers, with a focus on BERT-type encoder models, which is to be distinct from encoder-decoder models and decoder-only models covered
13

---

Published in Transactions on Machine Learning Research (02/2026)
in Section 7. We structure our review around the fundamental architectural tradeoff between interaction depth and computational elficiency. These constraints necessitate two primary paradigms:
Cross-Encodee (Deep Interaction): A single model processes the concatenated query and doe- ument as one sequence, allowing every query token to interact deeply with every document token. This provides state-of-the-art ranking quality but is computationally expensive, making it suitable only for reranking.
2. Bi-Encoder (Separable Pre-computation): Separate encoders process the query and document independently to create fixed-size vectors. Since document vectors be pre-computed offline, this architecture enables extremely fast similarity search suitable for first-stage relrieval.
We first discuss the crucial training strategies used to optimize these two architectures. We then detail the cross-cncoder models that perform doep, full interaction, followed by the separable bi-encoder architectures that prioritize efficiency, exploring their dense, sparse, and multi-vector variants. Finally, we discuss advanced hybrid models and orthogonal improvements such continual training and interpretability.
6.1 Training Strategies for Transformer-Basee IR
Although we aim to disentangle model architectures from training strategies, the co-evolvement of these two areas is defining pattern of this era. The architectural dichotomy (cross-encoder vs. bi-encoder) has impaciful influence the training methodology, exlending the loss function calegories discussed in Section 4 into the deep learming paradigm.
Contrastive and Listwise Objectives. The application of contrastive learning builds on the principle of the InfoNCE loss (Oord et al., 2018), which derived from Noise- Contrastive Estimation (Gutmann & Hyvärinen, 2012). The general goal is to learn model that distinguishes "positive" sample from set of "negative" samples.
The InfoNCE framework is primarily used to optimize bi-encoders in the dense retrieval setting. In this context, the relevance score f is a simple similarity function (e.g., dot product) between the query and document vectors, f(Q, = sim vp). For query Qi a positive document D and a set of negative documents DT, the bi-encoder trained to minimize the negative log probability of correctly classifying the positive document:
LInfANCE = [|
1
log (EE
Σ
exp folQi, D,))
exp DJ) + Σ
exp So (2 Dj)
ED
where S the training set and DJ is the set of sampled negative documents.
For cross-encoders, which compute relevance often simplified to a standard Negative Log- LikeLikeld (NLL) loss over the final softmax probabilities of the candidates, minimizing the distance between the predicted distribution and the ground truth relevance distribution for the entire list.
concatenated list, the objective is also listwise but
Hard Negative Mining. A critical challenge in training bi-encoders is generating sufficiently difficult negative examples, random sampling typically yields easy negatives that do not challenge the model effectively. This is where Hard Negative Mining (HNM) becomes essential. HNM strategies ensure that the model is exposed to challenging cases where positive and negative document features are difficult to distinguish. Key strategies include:
In-Batch Negatives (IBN): Leveraging other queries' positive documents within the same mini- batch negative examples for the current query. IBN provides balance of efficiency and difficulty.
Lexical Negatives: Using documents highly ranked by a traditional sparse model (e.g., BM25) but not labeled relevant.
14

---

Published in Transactions on Machine Learning Research (02/2026)
3. Iterative Hard Negative Mining: Employing an existing dense retriever to periodically mine difficult. negatives from the collection (i.e., documents that the current model mistakenly ranks highly). Seminal methods like ANCE (Xiong et al., 2020) and ADORE (Zhan et al., 2021) use this itcrative approach to continually feed the model better training data.
Knowledge Distillation. To further narrow the effectiveness gap between efficient bi-encoders and power- ful cross-encoders, the community widely adopted Knowledge Distillation (KD) (Buciluă et al., 2006; Hinton et al., 2015). KD technique for training a smaller, efficient "student" model by transferring knowledge from larger, more capable "teacher" model (Hinton al., 2015; Gou et al., 2021). In IR, KD is used to create fast rerankers or retrievers that approximale the performance of slower, larger models, which is critical for production systems (Hofstätter et al., 2020a; 2021a; Chen et al., 2021; Kim et al., 2023; Xu et al., 2025c; Zhang et al., 2025b; Vera et al., 2025).
Let ft denote the teacher model (typically a cross-encoder) and f. denote the student model (typically bi- encoder or a smaller cross-encoder). For given query Q and a list of candidate texts DD = (Di, D2,..., ) we first compute relevance scores (logits) from both models:
Z, lf((Q, D1). f(l. fe()....ff.(.....
2, (.) [.....)..K(..2).......... (.Q..
....
The student model fa is trained to mimic the teacher's output distribution over the candidate texts by minimizing the Kullback-Leibler (KL) divergence between the two softened probability distributions:
LKD = DKL softmax (7) softmax
2, (T)
where 7' is the temperature hyperparameter. A higher temperature creates a softer probability distribution, which can help in transferring more nuanced information from the teacher. KD thus provides essential bridge, allowing the elficiency of bi-encoders to approach the effectiveness ceiling set by cross-encoders.
6.2 Deep Interaction Models: The Cross-Encoder for Reranking
The most effective application of transformers in IR involves full, deep interaction between query and doe- ument tokens. This architecture, known cross-encoder (Humeau et al., 2020), takes the concatenated sequence of (Q, D) as single imput. Nogucira et al. (2019b) first employed this approach in their MONOBERT model for reranking candidate passages from a first-slage retriever. The model outpuls a relevance score s via a linear layer on top of the final BERT representation, typically from a linear layer using the [CLS] token's ' representation (Figure 3a).
Conceptually similar to pre- Transformer interaction-basee neural ranking models, this schema has proven effective across various pre-trained encoders (Zhang et al., 2021b), well as other transformer architectures (Section 7). However, this cross-encoder schema faces two primary challenges: (1) the fixed context length of models like BERT (e.g., 512 tokens) makes processing long documents difficult, and (2) relying on a single token's 's fixed-dimensional representation (e.g., 768-dimensional representation for BERT) may limit the model's expressive power.
Handling Long Documents. Corresponding mitigations for these two challenges have been extensively investigated in literature. Chunk-and-aggregate approaches represent a practical solution to handle long documents that exceed BERT's input constraints by decomposing the ranking problem into passage-level streettggsstteggyg scoring followed by aggregation (Gao & Callan, 2022). The fundamental strategy involves splitting documents into fixed-length passages or sentences, applying BERT-based cross-encoders to score each query-passage pair independently, then combining these scores to produce a final document-level relevance score. Early work in this direction explored sentence-level aggregation, where BERT scores computed at the sentence level were shown to be effective for document ranking (MacAvancy et al., 2020; Yilmaz et al., 2019). The BERT-Maxh (Dai & Callan, 2019a) approach became particularly influential, where documents are split into fixed-length passages and the maximum passage score serves the document. score.
15

---

Published in Transactions on Machine Learning Research (02/2026)
Two primary aggregation strategies have emerged: (1) score-pooling and (2) representation aggregation. Score-pooling methods apply simple operations like maximum, sum, or first passage scores to combine passage-level relevance scores (Dai & Callan, 2019a). In contrast, representation aggregation methods address both the long-document problem and the single-vector expressiveness limitation. Instead of collapsing cach passage's signal into a single scalar soore, these approaches gather the rich, low-dimensional [CLS] token representations from each passage. This collection of vectors forms more comprehensive and expressive document-level feature set, which is then processed by additional neural networks (MacAvaney et al., 2020). Notable systems like PARADE (Li et al., 2023a) employ CNNs and transformers for aggregation, while CEDR (MacAvaney et al., 2019b) pioneered joint approaches that combine BERT outputs with existing neural IR models through averaging.
While auunk-and-aggregatt approaches successfully handle long documents, they fundamentally limit query- document interactions to the passage level, creating information bottlenecks where passage scores or low- dimensional representations constrain the model's ability to capture wocument-wide relevance patterns. Hof- stätter et al. (2021b) argue that this tradeoff between scalability and interaction richness remains defining characteristic of this approach.
6.3 Efficient Pre- Tre-Computation Models: The Bi-Encoder Architecture
While cross-encoders offer state-of-the-art effectiveness, their computational cost roqu-req a full trans- former pass for every (2, D) pair makes them infeasible for retrieval over large collections. This limitation motivated the development of bi-encoder architectures, which are conceptually similar to representation- based neural ranking models (Section 5).2
A bi-encoder uses a backbone network (typically a transformer) encode the query Q and document D separately. The resulting dense vector representations are then used to compute relevance score with simple similarity function like dot product cosine similarity (Xiong et al., 2020; Karpukhin et al., 2020; Gao et al., 2021b). The key advantage of this separation is efficiency: the entire document collection can be encoded into a vector index offline. At query time, retrieval becomes fast approximate nearest neighbor (ANN) search problem (Johnson et al., 2019; Malkov & Yashunin, 2018) or search with an inverted index data structure (Zobel et al., 1998), avoiding costly neural network inference.
A notable insight from Lin (2021) is that the bi-encoder framework provides a unifying lens for understanding diverse retrieval approaches. Dense retrieval models, learned sparse retrieval models, and traditional bag- of-words approaches such BM25 can all be interpreted as parametric variations of this architecture. They differ primarily along two axes: (1) the representational basis (dense semantic embeddings versus sparse lexical vectors), and (2) whether those representations are learned from data or manually engineered. Existing methods based on this bi-encoder architecture vary primarily in their representation format (dense versus sparse), pooling strategies, and training methodologies.
6.3.1 Learned Dense Retrieval
Dense retrieval models typically adopt a standardized dual-encoder architecture built on pre-trained trans- former models, most commonly BERT in this context. The standard formulation uses separate BERT encoders for queries and documents, with layer-normalized linear projection applied to the token represen- tation: Encoder(-) Linear Linear(BERT(.)) The encoder weights can be separate shared between qucry and docunent sides.
The core architectural principle involves encoding queries and documents into low-dimensional dense vectors, typically 768 dimensions matching BERT's hidden size. Rather than using all hidden representations, most models compress the sequence information using a reduction function, usually the token representation or mean pooling of the final transformer layer outputs. This creates single dense vector representation per text sequence that captures semantic information beyond simple lexical matching.
2The term "bi-encoder" also known two- wowtower architecture or embedding model. We "bi-encoder" contrast. with "ccoes-,, which takes concatenated imput.
16

---

Published in Transactions on Machine Learning Research (02/2026)
Relevance scoring in dense retrieval is performed through simple similarity functions, most commonly dot product or cosine similarity between query and document vectors. This design enables efficient. ANN search over pre-computed document representations (Johnson et al., 2019), making dense retrieval practical for large-scale collections while maintaining the semantic understanding capabilities of transformer models. The success of this architecture stemss from ability to learn semantic representations that address the vocabulary mismatch problem inherent traditional sparse retrieval methods (Lee et al., 2019; Karpukhin et al., 2020; Xiong al., 2020; Reimers & Gurevych, 2019). Dense retrieval models have demonstrated notable effectiveness improvements over BM25 baselines across various tasks including open-domain question answering and web search.
6.3.2 Learned Sparse Retrieval
Learned sparse retrieval (LSR, Figure 3c) employs the same bi-encoder architecture as dense retrieval but produces fundamentally different representations." While sharing the transformer backbone, sparse retrieval models encode queries and documents into high-dimensional sparse vectors whose dimensionality typically matches the vocabulary size of the underlying pre-trained model, often containing tens of thousands of dimen- sions. Each dimension corresponds to a specific vocabulary term, creating an interpretable representation where non-zero weights indicate term importance (Formal et al., 2021b;a; Nguyen et al., 2023).
Three key architectural constraints distinguish sparse encoders from their dense counterparts. First, sparsity is enforeed through explicit regularization techniques, ensuring most term weights remain zero to maintain efficiency (Formal et al., 2021b; Xu et al., 2025b; 2026). Second, all weights must. be non-negative to main- tain compatibility with traditional inverted index software designed for lexical search systems like Lucene, Third, the high-dimensional vocabulary-aligned vectors enable integration with existing inverted index in- frastructure and optimization algorithms (Turtle & Flood, 1995; Broder et al., 2003; Bruch et al., 2024). Notably, inference-free learned sparse retrieval methods such as SPLADE-Doc (Formal et al., 2021a; Shen et al., 2025) eliminates requirement of specialized accelerators such as GPUs, making them highly efficient for inference on multi-core CPU machines.
At conceptual level, learned sparse retrieval be viewed as sophisticated evolution of traditional term weighting schemes, learning context-aware token importance scores from data rather than relying on heuristic formulas (Zamani et al., 2018; Dai & Callan, 2019b; Mallia et al., 2021; Yu et al., 2024a; Xu et al., 2025b; 2026). This approach inherits desirable properties from bag-of-words models such as exact term matching while leveraging the somantic understanding capabilities of pretrained transformers. Notable implementations include SPLADE (Formal et al., 2021a), DREPIMPACT (Mallia et al., 2021), and UNICOIL (Lin & Ma, 2021), which demonstrate that transformer- based sparse representations can achieve effectiveness comparable to dense retrieval while maintaining the efficiency benefits of inverted indexes.
6.4 Bridging the Gap: Advanced Interaction and Hybrid Models
The standard bi-encoder's lack of term-level interaction is performance bottleneck compared to cross- encoders. Several lines of research aim bridge this gap by introducing more granular representations or by combining different retrieval paradigms.
6.4.1 Multi-Vector Representations
To re-introduce query-document interaction without the full cost of a cross-encoder, multi-vector models represent queries and documents using multiple vectors. POLY-ENCODER (Humcau et al., 2020) computes a fixed number of vectors per query and aggregates them with softmax attention over document vectors. ME- BERT (Luan et al., 2021) represents documents with m vectors and uses the maximum similarity between any query and document vector to estimate relevance.
We focus learned sparse retrieval within the bi-encoder formulation, thereby excluding approaches based on Icarned document expansion (c.g., Nogucira et al. (2019a); Zbib et al. (2019)) and document term reweighting (c.g., DEEFCT (Dai & Callan, 2019h)). For unified treatment of learned sparse retrieval that encompasses these lines of work, Mallia et (2021); Basnet, et al. (2024).
17

---

Published in Transactions on Machine Learning Research (02/2026)
In line with this idea, COLBERT (Khattab & Zaharia, 2020; Santhanam et al., 2022; Hofstätter et al., 2022) represents each token in the query and document as context contextualized vector. It then performs "late interaction" step where each query vector is compared against all document vectors via a MaxSim operator, and the final score is the sum of these maximum similaritics. This late interaction scheme (Figure 3d) allows COLBERT for end-to-end training to achieve strong performance while still achieving efficient retrieval through a dedicated index structure. On the other hand, also leads to drastically increased index size, which has been the focus in later studies (Santhanam et al., 2022; Hofstätter al., 2022, inter alia).
We should also note that multi-vector retrieval can be viewed the learned fcature representation is a matrix of size n x h, with n vectors of hidden dimension h. This matrix can be conceptually flattened into a single dense vector, showing its connection to the vanilla single- vector retrieval. The key difference lies not in the representation itself but in the richer relevance estimation strategy: instead of applying a simple linear relevance like dot product, models like COLBERT aggregate fino-grained token-level interactions compute a relevance score.
special case of dense retrieval where
6.4.2 Hybrid Retrieval
Another direction combines the strengths of different retrieval systems. A simple yet effective approach is ranklist fusion (e.g., Reciprocal Rank Fusion, Cormack et al., 2009), which merges ranked lists from sparse (e.g., BM25) and dense retrievers post-retrieval without architectural changes. More integrated models combine signals at a deeper level. COIL (Gao et al., 2021a) enhances traditional bag-of-words retrieval with semantic embeddings from a BERT encoder. UNICOIL (Lin & Ma, 2021) simplifies this by reducing the semantic embedding to single dimension, effectively learning a term weight akin to LSR models like SPLADE (Formal et al., 2021a;b). A few works fall into the intersection of learned sparse retrieval and multi- vector representations. For example, SLIM (Li et al., 2023c) first maps cach contextualized token vector to a sparse, high-dimensional lexical space before performing late interaction between these sparse token embeddings. SPLATE (Formal et al., 2024) takes an alternative approach to first encode contextualized token vectors, then map these token vectors to a sparse vocabulary space with a partially learned SPLADE module. Both models achieve performance improvement compared to learned sparse retrieval basclines such SPLADE.
6.5 Orthogonal Improvements and Analysis
Beyond architectural innovations, performance can be enhanced through improvements to the underlying models and a doeper analysis of their behavior. We show a list of models and their corresponding architectures in Table 3, a majorily of which use BERT (Devlin et al., 2019) as the backbone, with exceptions using DISTI.BERT (Sanh et al., 2019), ROBERTA (Liu et al., 2019), and T5 (Raffel et al., 2020; Sanh et al., 2022; Mo et al., 2023; Chung et al., 2024).
Continual Training and Adaptation. Instead of modifying the retrieval architecture, this line of work enhances the backbone language model itself through domain adaptation or continued pro-training, a proven stralegy in NLP (Howard & Ruder, 2018; Gururangan el al., 2020). For instance, Lee et al. (2019) pre- train BERT with an Inverse- Cloze Task (Taylor, 1953) for better text representations. CONDENSER (Gao & Callan, 2021) proposes a dedicated pre-training architecture to "condense" text representations into the [CLS] token. COCO-DR (Yu et al., 2022c) extends CONDENSER by using a technique named Distribu- tionally Robust. Optimization to mitigate distribution shifts in dense retrieval. A line of works have also explored other pre-training objectives such masked auto-encoders (Xiao et al., 2022; Wu et al., 2023b) and bag-of-words prediction (Ma et al., 2024a). Recent works (Wang et al., 2023a; Nussbaum et al., 2025; Yu et al., 2024b, änter alia) have employed a "middle-stage" training on large-scale unlabeled text pairs, wet pretrained encoder models and supervised fine-tuning on labeled text pairs, and have demonstrated the cor- responding performance improvement compared to the traditional two-stage pipeline. We refer readers to the original papers for details.
Interpretability and Explainability. A few works have attempted to interpret what transformer- based models learn, i.e. mechanistie interpretability (Elhage et al., 2021; Saphra & Wiegreffe, 2024). MacAvaney
18

---

Published in Transactions on Machine Learning Research (02/2026)
Table 3: Summary of IR model architecture for passage retrieval and passage ranking based on pre-trained transformers. Dense Retrieval and LSR denote learned dense retrieval and learned sparse retrieval, respec- tively. DEEPCT (Dai & Callan, 2019b) is trained without labeled training set. Contrastive Learning and in-batch negatives means listwise loss function is used. SENTENCEBERT (Reimers & Gurevych, 2019) is originally designed for the symmetrrccl sentence similarity tasks, but is quickly expanded to asymmetrical
retrieval tasks.
Name
Model
Architecture Backbone LM
Training strategy
MONOBERT (Nogueira al., 2019b)
Reranking
Cross-enender
BERT
Classffcation
CEDR (MacAvaney et al., 2019b)
Reranking
Cross-encoder
BERT
Contrastive Learning
BERT-MAXP (Dai & Callan, 2019a)
Reranking
Cross-eneuder
BERT
Pairwise Loss
Gao et al. (2020)
Reranking
Cross-eneoer
BERT
Distillation
TART-FOLL (Asal et al., 2023)
Reranking
Cross-encoder
FLAN-T5- TN-E
Instruction Tuning Unsupervised
ODQA (Lee al., 2019)
Dense Retrieval
Bl-encoder
BERT
SENTENCEBERT [Reimers & Curevych, 2019)
Deuse Retrieval
Bi-enooder
BERT
Triplet
DPR (Karpukhin al., 2020)
Dense Retrieval
Bi-enooder
BERT
Contrastive Learning
ANCE (Xiong et al., 2020)
Dense Retrieval
Bi-enooder
ROBERTA
Contrastive Learning
REPBERT (Zhan al., 2020)
Dense Retrieval
Bi-encoder
BERT
In-batch negatives
MARGIN MSE (Hofstätter et al., 2020a)
Dense Retrieval
Bi-encoder
DISTIBERT
Distillation
TAS-B (Hofstätter ct al., 2021a)
Dense Retrieval
Bi-enooder
BERT
Distillation
ROCKETQA (Qu et al., 2020)
Dense Retrieval
Bi-encoder
ERNIE
Contrastive Learning
RoCKETQA-V2 A-12 (Ren et al., 2021)
Dense Retrieval
Bi-enooder
ERNIE
Distillation
GTR (Ni et al., 2022b)
Donse Retrieval
Bi-encoder
ENCI5
Contrastive Learning
TART-DUAL (Asal et al., 2023)
Dense Retrieval
Bi-encoder
CONTRIEVER
Instruction Tuning
E5 (Wang al., 2022a)
Dense Retrieval
Bi-enooder
BERT
Contrustive Learning
GTE (Li al,, 2023d)
Dense Retrieval
Bi-encoder
BERT
Contrastive Learning
POLY-ENCODER (Humeau al,, 2020)
Multi-vector
Misc
BERT
In-batch Negatives
ME-BERT (Luan et al., 2021)
Multi-vector
Bi-enooder
BERT
Contrastive Learning
COLBERT (Khattab Zaharia, 2020)
Multi-vector
Bi-encoder
BERT
Pairwise Loss
COTT. (Gao al., 2021a)
Multi-vector
Bi-encoder
BERT
Contrastive Learning
COLBERT-V2 (Santhanam et al., 2022)
Multi-vector
Bi-enooder
BERT
Distillation
COLBERTER (Hofstätter et al., 2022)
Multi-vector
Bi-encoder
BERT
Distillation
DEEPCT (Dai & Callan, 2019b)
LSR
Bi-encoder
BERT
Unsupervised
SPARTTEMM (Bai ct al., 2020)
LSR
Bi-cncoder
BERT
Contrastive Learning
SPLADE (Formal et al., 2021b)
LSR
Bl-enooder
BERT
Contrastive Learning
SPLADE-V2 (Formal al,, 2021a)
LSR
Bi-encoder
BERT
Distillation
DEEPIMPACT (Mallia al., 2021)
LSR
Bi-encoder
BERT
Contrastive Learning
UNICOLL Lin
(2021)
LSR
Bi-enooder
BERT
Contrustive Learning
SPARSEEMBED (Kong et ol., 2023)
LSR
Bi-enooder
BERT
Coutrustive Learning
SLIM (Li et al., 2023c)
LSR + Multi- vector
Bi-encoder
BERT
Contrastive Learning
SLIM (Li ct al., 2023c)
LSR
Multi-vector
Bi-encoder
BERT
Distillation
SPLATE (Formal et al, 2024)
LSR
Multi-vector
Bi-encoder
BERT
Distillation
et al. (2022) showed that neural models rely less on exact match signals and instead encode rich semantic information. Ram al. (2023a) connected dense and sparse retrieval by projecting dense retricver's intermediate representations into the vocabulary space. Separately, other work focuses designing systems that provide model-agnostic explanations (Rahimi et al., 2021; Yu et al., 2022b; Xu et al., 2024c) to satisfy desiderata like faithfulness (Jacovi & Goldberg, 2020; Xu et al., 2023). As IR systerns become integral to other applied ML domains, we believe it is important to study and design interpretable, truthful, and trustworthy TR models.
The architectural innovations discussed in this section highlight a mature research field dedicated to harness- ing pre-trained transformers for information retrieval. The central theme has been the architectural tradeoff between interaction depth and computational cost, giving rise to spectrum of models from highly effective cross-cncoder rerankers to efficient bi-cncoder retrievers. By developing advanced representations - whether dense, sparse, or multi-vector and hybridizing these approaches, the community has pushed the boundaries of the classic "retrieve-then then-rank" paradigm. However, these models still primarily function as specialized components for representation and scoring. The next wave of innovation emerged from models capable not only of understanding text but also of generating it, marking the beginning of the era of Large Language Models.
19

---

Published in Transactions on Machine Learning Research (02/2026)
7 Large Language Models for IR
The natural cvolution from pro-trained encoders is the recent ascendance of Large Language Models (LLMs).* While building on the same transformer principles discussed in Section 6, the sheer scale and generative capabilities of modern inttructionfolloviiin LLMs are reshaping the architectural landscape of IR. These models are not just larger backbones for feature extraction; their proficiency in language understanding, generation, and instruction-following allows them to take on entirely new roles. Trained to align with human preferences (OpenAT et al., 2023; Gemini Team Google et al., 2023; Bai et al., 2022), LLMs can perform complex tasks such as reasoning (Wei et al., 2022; OpenAl et al., 2024; Guo al., 2025), tool usage (Schick et al., 2023; Patil et al., 2024b; Qin et al., 2024a; Patil et al., 2024a) and planning (Song et al., 2023; Huang et al., 2024a). In this section, we review how these powerful models - spanning decoder-only and encoder-decoder architectures- are being adapted for IR tasks, moving beyond established paradigms into new frontiers of retrieval, reranking, and direct generation.
7.1 LLM as Retriever
A straightforward yet highly effective application of LLMs to serve as the backbone for bi-encoder retrieval models. We categorize these developments into backbone scaling, architectural adaptation, and unified modeling. We show shortlist of works in Table 5.
Scaling Bi-Encoders. The dramatic increase in parameter count and training data provides LLMs with richer world knowledge and more nuanced understanding of semantics compared to their smaller BERT- sized predecessors. This directly translates to performance improvements. Neelakantan al. (2022) fine- tuned a scries of off-the-shelf GPT models towards text and code represcntation. They empirically verified that the bi-encoder retriever's performance can benefit from increased backbone language model capacity. Muennighoff (2022); Ma et al. (2024b) empirically verified the effectiveness of scaling the size of dense retriev- ers with open-weight models such as GPT-J (Wang Komatsuzaki, 2021) and LLAMA-2 (Touvron et al., 2023). Today, common text retrieval benchmarks like BEIR (Thakur et al., 2021) and MTEB (Muennighoff al., 2023) are dominated by proprietary and open-source T.LM-based embedding models (Wang et al., 2023a; Li et al., 2023d; Lee et al., 2025; Zhang et al., 2025b; Muennighoff et al., 2025, inter alia).
Decoder-Only Adaptation. A parallel line of research focuses on adapting the unidirectional architecture of decoder-only LLMs (c.g., LLAMA) to better suit the needs of bidirectional text representation. Standard decoder-only models are optimized for next-token prediction, which may not be ideal for creating a single summary vector for a whole text. To address this, LLM2VEC (BehnamGhades et al., 2024) enables bidirec- tional attention and further trains LLAMA-2 (Touvron et al., 2023) with specific unsupervised and supervisod adaptive lasks. Similarly, NV-EMBED (Lee et al., 2025) introduces a new latent atlention mechanism to produce improved representations, leading to improved performance on the MTEB benchmark compared to directly enabling bi-directional attention.
Unified Modeling. GRETLM (Muennighoff al., 2025) fine-tunes MISTRAL family models with both dense retrieval task and text generation task with different attention mechanisms and demonstrate the potential of unifying retrieval and generation with one single foundation model.
7.2 LLM as Reranker
The reranking task has also been significantly advanced by LLMs, which introduce new capabilities beyond the cross-encoder architecture discussed in Section 6. This evolution can be categorized into two main architectural approaches: fine-tuning and zero-shot prompting. We show a shortlist of works that use LLM as rerankers in Table
"The term LLagge Language Model" lacks precise, universally accepted definition in the literature. In this survey, the term to refer to models with over one billion parameters that pre-trained with text generation objective, such as text infilling (egg,, T5) or causal language modeling (e-g., the GPT series), and are often optionally post-trained for instruction following and human preference alignment.
20

---

Published in Transactions on Machine Learning Research (02/2026)
Fine- Rerankers. First, LLMs can be fine-tuned powerful rerankers. Extending earlier work BERRTttye models, researchers have applied similar techniques to larger encoder-decoder models like T5 (Raffel et al., 2020) and decoder-only models like LLAMA (Touvron et al., 2023).
Pointwise and Pairwise: Nogueira et al. (2020) fine-tuned T5 with a classification loss, treating reranking as a binary relevance decision. RANKT5 (Zhuang et al., 2023a) took a more direct approach by fine-tuning T5 to output a numerical relevance score, optimizing the model with established ranking losses like RANKNET (Burges, 2010). Further, Zhuang et al. (2023a) also investigated the impact of language model architectures (T' encoder-decoder versus T5 encoder), loss functions (pointwise, pairwise, listwise), and pooling strategies to ranking performance.
Listwise: Instead of scoring documents individually, LISTT5 (Yoon et al., 2024) adopts a Fusion- in-Decoder architecture (Izacard & Grave, 2021) process and rank an entire list of candidate documents in single forward pass. More specifically, the architecture consists of an encoder and decoder, where the encoder takes query and multiple passages as input in parallel, and the decoder outputs a sorted list of input passages in the decreasing order of relevance, achieving better reranking efficiency compared to pointwise methods. Yang et al. (2025b) fine-tuned QwQ-32B (Qwen Team, 2025) for listwise reranking where multiple passages are concatenated together as input and achieved better performance than pointwise reranking, leveraging the reasoning and long context capability of the strong base model.
The long-context capabilities of modern LLMs have also enabled a new reranking paradigm. RANKLLAMA (Ma et al., 2024b) demonstrated superior point reranking performance compared to BERT and T5-basod rerankers for long document rcranking where the input is truncated at 4,096 tokens.
Long-Context Reranking:
Zero-Shot / Few-Shot Prompting. Second, the instruction-following ability of modern LLMs has un- locked zero-shot and few-shot reranking via prompting. This paradigm requires no task-specific fine-tuning. Instead, the LLM is prompted with query and a list of candidate documents and asked to identify the most relevant ones (Ma et al., 2023; Zhang et al., 2023c; Pradoep et al., 2023b;c; Sun et al., 2023). This listwise approach is a natural fit for the long context windows of models like GPT-4. To mitigate the high computational cost and context length limitations processing full documents, Liu et al. (2024b) proposed using passage embeddings as compact document representations for the LLM, training a specialized reranker that operates on these embeddings to improve efficiency. As this line of research primarily involves prompt engineering rather than architectural changes, we refer readers to recent survey (Zhu et al., 2023) for further details.
7.3 Generative Retrieval
Perhaps the most radical architectural shift enabled by LLMs is generative retrieval. Traditional IR. systems follow the "retrieve-then-reranke paradigm (Schütze et al., 2008; Asai et al., 2024a; Xu et al., 2025c). Gener- ative retrieval fundamentally challenges this by reframing retrieval as a sequence-to-squeene task. Instead of searching an index, an autoregressive language model is trained to directly generate the unique document identifiers (DocIDs) of relovant documents in response to a query.
Evolution of the Paradigm. The foundational work in generative retrieval emerged from the entity linking domain with Generation of ENtity REtrieval (GENRE, De Cao et al., 2021). Rather than treating entity retrieval as classification problem over atomic labels with dense representations, GENRE reframed it generative task where an encoder-decoder model produces entity names autoregressively, token-by- token, conditioned on the input context. Building on GENRE's success, DSI introduced generative retrieval to the broader document retrieval domain (Tay et al., 2022). The core innovation was fully parameterizing pipelines within a single neural model, where all corpus information is
traditional "retrieve-then-reraee encoded the model parameters rather than external indices (Tay et al., 2022; Pradeep et al., 2023a). DST operates through two fundamental et al., 2024): the indexing task (Learn to Index) and the retrieval task (Learn to Retrieve).
tasks that can be trained jointly or sequentially (He
21

---

Lislwise Loss Published in Transactions on Machine Learning Research (02/2026)
Table 4: Taxonomy of identifier types in Generative Retrieval. The choice of identifier is a key architectural distinction.
Identifier Type
Description
Atomic Identifiers
Unique inlegers assigned documents, Simple but lacks semantic generalization, Natural language strings (Litles, URLs). Leverages pre-trained kuowledge but can be ambiguous.
String Identifeers
Semantic Identifiers
Structured IDs derived from clustering embeddings. Enables semantic generaliza- tion in the ID space.
Table 5: Summary of IR model architecture utilizing large language models retrieval backbone.
Name
Architecture
Backbone LM
Training strategy
SPTT-EXT (Neelakantan et al., 2022)
LLM Encoder
GPT-3
Listwise Loss
SGPT-BE (Muennighoff, 2022)
LLM Encoder
GPT-J GPT-NEOX
Listwise Loss
GTR (Ni el al., 2022b)
LLM Encoder
T5
REPLLAMA [Ms et sl., 20245)
LLM Eneoder
LLAMA
Listwise Loss
E5- E5-MISTRAL. ( (Wang et al,, 2023a)
L.M Encoder
MISTRAL
Syuthetic Data + Listwise Loss
LLARA (Li et ul., 20236)
LLM Encoder
LLAMA
Adaptation + Contrastive TTraining
MAMBARETRIRVERR (Zhang el al., 2024)
LLM Encoder
MAMBA
Listwise Loss
LLM2VDC "BelnammCaade
al., 2024)
LLM Encoder
LLAMA & MISTRAL
Adaptation + Contrastive Pre-training
GRIT-LM (Muennighofff et 2025]
LLM
MISTRAL & MIXTRAL 8x7B
Generative /mmbedding Joint Training
Gonerative/Embeedddng
NVEXBED (Lee al., 2025)
LLM Encoder
MISTRAL
Adaptation Synthetic Data Listwise Loss
CTE-QWEN2-INSTRUCY (Li et al., 2023d)
LLM Encoder
QWEN
Adaptation + Synthetic Data Listwise Loss
QWRN3: RETREEVER (Zhang al., 2025h)
I.M Encoder
QWEN3
Syuthetic Data + Listwise Loss
Taxonomy of Identifiers. A critical design choice in DST involves document identifier representation (summarized in Table 4).
Atomic and String Identifiers: Early work explored atomic identifiers (unique integers) and simple string identifiers (titles, URLs) (Chen et al., 2023a). DST be implemented in two variants: dassificationn-aseed approaches that classification layer to output atomic document identifiers, and generative approaches that autoregressively generate identifier strings (Mchta et al., 2023). More sophisticated methods have introduced n-gram-based identifiers (Bevilacqua el. al., 2022).
Semantic Identifiers: Semantically structured identifiers created through clustering algorithms proved most effective (Zhu et al., 2023). This often involves hierarchical representations using tech- niques like residual quantization (Zeng al., 2024), where the model learns to associate document content with corresponding semantically meaningful document identifiers (Kishore et al., 2023).
Inference and Constraints. The technical implementation of generative retrieval systems centers on eqquence-to-squeene modeling. A critical technical requirement ensuring only valid document identifiers are generated during inference. This typically achieved through constrained beam search over a prefix tree (tric) constructed from all valid document identifiers (Tang et al., 2023). Alternative approaches include constrained greedy search and FMM-indxx structures (Tang et al., 2023). The T5 model backbone (Raffel et al., 2020) serves as the foundation for most DSI implementations, trained with cross-entropy loss on both indexing and retricval objectives.
Challenges. Generative retricval faces significant challenges in dynamic environments. The tight coupling between index and retrieval modules makes updating the corpus computationally expensive, typically re- quiring full model retraining (Mehta et al., 2023). Scalability poses a major challenge; most research has focused on relatively small collections, as the memory and computational requirements grow substantially corpus size increases. Generative retrieval is an active and rapidly evolving research area; we direct interested readers to dedicated survey (L. et. al., 2025d) for comprehensive review.
22

---

Published in Transactions on Machine Learning Research (02/2026)
Table 6: Summary of IR model architecture utilizing large language models for reranking. Nogueira dos Santos et al. (2020) and Zhuang al. (2021) revisit the statistic language model problem with modern transformer-basee
models, including BART (Lewis et al., 2020a), T5 (Raffel al., 2020), and GPT- 2 (Radford et al., 2019). We use Seq2Seq LLM to refer to encoder-decoder architecture language models such T5 and BART, and Casual LLM to refer to modern LLMs with causal language model architecture like GPT family models.
Name
Architecture
Backbone LM
Training / Prompting Strategy
Fine-Tune LLM for Revonkinng
MoNOTS (Nogueira al,, 2020)
Seq2Seq LM
T5
Classificatio
Nogueica dos Santoe et al. (2020)
Seq2Seq LLM
BART
Unlikelibood
QLM-15 (2imang
20121)
Seq2Seq LLM
Language Modeling
DUOTb (Pradeep ol., 2021)
Seq2Seq LLM
T5
Pairwise Loss
RANKTS (Ziuanng et al., 20133a]
Seq2Seq LLM Encoder Prediction Layer
T5
Listwise
LISTT5 (Yoon ni., 2024)
Pusion-in-derode
T5
Listwise Loss
SGPT-CE (Meenigholf, 2022)
Causal LLM
GPT-J & GPT-Nzo
Listwise Loes
RANKLLAMA (Ma al., 2012bb
Encodee + Prediction Layer
RANKMAMBA (Nu, 202()
Causal LLM Encoder Prediction Layer
MAMBA
Listwise Loss
RANKVICUNA (Pradeep et al., 20136))
Causal LLM
VICUNA
Listwise
RANKZEPHYR (Pradeup
2023k)
Causal LLM
ZEPRYR
Listwise
Zhang et al. (2023c)
Causal LLM
CODE LNSMA-INSRRUCT
Listwise
[20245)
Cansal
QWEN3-RRee RERANKRR (Zhang ot ol., 2025b)
Causal LLM
QWTN3
Syuthetic Data Pairwisn Lass
Prompt LLM for Heranding
Zimang (2023b)
Cousal LLM
Multiple
Pointwise Prompting
Zhuang et al. (2024a)
Causal LLM
FLAN-PALM-S
Pointwise Prompting
al., 2022)
Smq25eq
Prompting
PRP (Qin ot ol., 20241)
Seq28oq TM
FLAN-UT2
Pairwise Prompting
Yan et al. (2024)
Seq2Seq LM
FLAN-UL
Pairwise Prompting
Ziuang al. (20241)
Seq2Soq LM
FLAN-T5
Pairwise Setwise Prompting
LRL (Ms et al., 2023)
Casual LLM
GPT-3
Listwise Prompling
RANKCPT-3355 (Stn 2023)
GPT-3.5
Prompting
RANEGPT-4 (Sun al, 2023)
Causal LLM
GPT-4
Listwise Prompting
7.4 Broader Ecosystem and Concluding Remarks
Beyond core architectural changes, LLMs are influencing the entire IR ecosystem. Their advanced generative and understanding capabilities are being harnessed for crucial supporting tasks:
Data Synthesis: Modern IR systems require extensive labeled data for training, which expensive to create. A promising line of work is to use LLMs to synthesize high-quality training data (c.g.. queries, relevant passages, and hard negatives) (Bonifacio et al., 2022; Boytsov el. al., 2024; Dai et al., 2023; Lee et al., 2024; Mo et al., 2024a;c; Zhang et al., 2025b).
Evaluation: From an evaluation perspective, LLMs' language understanding has led to research on using them as proxies for human relevance judges, which could dramatically accelerate the evaluation cycle (Faggioli et al., 2023; 2024; Clarke & Dietz, 2024).
We also point readers to a comprehensive survey on conversational information retrieval (Mo et al., 2024b), another area being reshaped by LLMs.
In conclusion, the adoption of LLMs in IR represents more than simple increase in model scale. While they certainly serve more powerful feature extractors within existing bi-encoder and cross-encoded frameworks, their unique generative and instruction-followin abilities are forging entirely new architectural paradigms like generative retrieval and zero-shot listwise reranking. However, the advancement of IR architecture is not driven solely by the pursuit superior semantic matching capabilities. The practical deployment of these systems ranging from lightweight encoders to massive LLMs necessitates architectures that can withstand rigorous efficiency constraints, handle diverse data modalities, and ensure reliability. We examine these cross-cutting architectural adaptations in the following section.
23

---

Published in Transactions on Machine Learning Research (02/2026)
8 Architectures for Diverse Scenarios and Constraints
The evolution of IR models described in previous sections from vector space models LLMs primarily traces the pursuit. of better semantic matching for English text. However, deploying these models in real- world environments requires navigating complex scenarios and constraints beyond pure textual relevance. These include handling diverse data modalitics and languages, balancing the inherent tradcoff between accuracy and latency, and ensuring model reliability through calibration. In this section, we review how IR architectures are adapted to meet these specific requirements. Across these settings, architectural choices such as representation granularity and modularity, serve as the primary mechanisms to balance task-specific constraints with scalable retrieval.
8.1
Architectures for Multimodal and Multilingual Data
8.1.1
Multilingual and Cross- Lingull Architectures
Problem Definitions and Retricval Scttings. Although often used interchangeably, Multilingual Infor- mation Retrieval (MLIR) and Cross-Lingual Information Retrieval (CLIR) correspond to distinct retrieval scenarios. CLIR refers to the setting in which a user issues a query in a source language and retrieves documents written in a different target language (Oard & Dorr, 1998; Fluhr et al., 1999). This distinc- tion has direct architectural implications: CLIR requires explicit cross-language alignment at query time, whereas ML.R emphasizes building shared or interoperable representations during indexing. The rentral architectural challenge in CLIR is bridging the lexical and semantic gap between the query and document language spaces (Goworek et al., 2025). In contrast, MLIR is a broader paradigm in which system indexes and searches over document collections containing multiple languages, potentially serving queries in any of them (Oard& & Dorr, 1996; Oard et al., 1999; Fluhr et al., 1999). Architecturally, CLIR can be viewed as a special case of MLIR that explicitly requires cross-language alignment at retrieval time.
Early Translation- Architectures. The architectural foundations of cross- lingull retrieval date back to early work on the Vector Space Model (Salton et al., 1975), where cross- language retrieval framed as synonymy problem. Early systems relied on bilingual thesauri to map query terms into a shared conceptual space prior to retrieval, treating translation as a distinct pre-processing step (Salton, 1969; 1970). Architecturally, these systems isolated linguistic complexity into a separate translation component, leaving the retrieval engine itself unchanged and monolingual. When high-quality thesauri were available, such architcctures could approach monolingual retricval performance; however, they were brittle due limited vocabulary coverage, poor handling of polysemy,, and an inability model multi-word expressions contextual meaning.
Statistical and Representation- Cross-Lingual Models. The availability of large parallel cor- pora in the 1990s, such as the Canadian Hansards", enabled shift from static dictionarics to statistically grounded architectures. A major departure from direct translation was introduced by Cross-Language La- tent Semantic Indexing (CI.-I.ST), which projected documents and queries from different languages into shared, language-independent latent space using Singular Value Decomposition over parallel data (Dumais et al., 1997). In parallel, Statistical Machine Translation (SMT) became a dominant architectural component in CLIR systems, with retrieval pipelines adopting either query translation or document translation using probabilistic alignment models such as the IBM Models (Brown et al., 1993). Both CL-LSI and SMT-basod pipelines preserved monolingual retrieval backends, differing primarily in whether cross-lingual alignment was achieved through latent semantic projection or probabilistic translation. These systems established the translation-base based retrieval paradigm, where the retrieval engine itself remained monolingual and linguistic complexity was isolated within the translation module.
Multilingual Transformers and End-to-End Retrieval. The introduction of multilingual pre-trained transformers, including mBERT and XLM-R, marked a fundamental architectural shift away from explicit
hhttp::/een.xiiippedaa.oo
24

---

Published in Transactions on Machine Learning Research (02/2026)
translation toward shared semantic representation learning (Devlin et al., 2019; Conneau et al., 2020a; MacA- vaney et al., 2019a). Trained on large-scale multilingual corpora, these models enabled a single encoder to represent queries and documents across languages, substantially reducing dependence on parallel data (Con- neau et al., 2020b; Feng et al., 2022; Shi et al., 2020; Goswami et al., 2021). However, general-purpose multilingual encoders often underperform in retrieval settings due to insulficient cross-lingual alignment in- duced during pretraining (Zhang et al., 2022; Elmahdy et al., 2024), which has been the focus of subscquent research works.
Per-Language Modules. To improve the performance of multilingual pre-trained transformers on low- resource languages, a line of works proposed to add language-specific adapters to enable zero-shot or few-shot cross-lingual transfer. Early explorations in NLP community focus on classical NLP tasks such as dependency parsing and named entity recognition (Üstūn et al., 2020; Pfeiffer et al., 2020; Artetxe et al., 2020; Pfciffer et al., 2021). In the context of IR, Litschko et al. (2022) compared the performance of adapter-based approaches and Sparse-Fines Sharse-Fine-Tuning Masks (Ansell et al., 2022) to NMT-based approaches and reported their efficacy in both performance and training efficiency.
Alignment-Focusedd and Modern Architoctures. Tackling the same challenge of per-language mod- ules approaches, specialized multilingual retrieval models such as LaREQA, InfoXLM, and LaBSE enforce tighter alignment between semantically equivalent gross-lingual pairs using parallel corpora and contrastive objectives (Roy ct al., 2020; Chi et al., 2021; Feng et al., 2022; Huang et al., 2024b). Notably, these approaches typically preserve the standard transformer backbone, focusing architectural innovation on rep- resentation learning objectives rather than structural redesign. Recent work further refines end-to-end mul- tilingual retrieval through improved data curation, supervision, and knowledge distillation, continuing the shift away from multi-stage translation pipclines toward unified modcls that directly match meaning across languages (Zhang et al., 2023d; Yang et al., 2024).
8.1.2 Multimodal Architectures
From Shallow Fusion to Deep Joint Embeddings. Early multimodal retrieval architectures relied on modality-specifie feature extraction followed by shallow fusion, with text typically represented using bag- of-words models and images encoded via hand-crafted descriptors such as scale-invariant feature transform (SIFT) (Lowe, 1999). To enable cross-modal comparison, projection- based methods such as canonical cor- relation analysis (CCA) (Hotelling, 1992) mapped heterogeneous features into a shared embedding space, establishing the foundational principle that multimodal retrieval requires learning semantic correspondences across modalities rather than treating them independently (Rasiwasia et al., 2010; Sharma el al., 2012; Gong et al., 2014; Ranjan et al., 2015). The transition to deep learning replaced manual feature engineering with end-to-end reprosentation learning, exemplified by carly visual-semantic embedding models such as DE- VISE (Frome et al., 2013; Wang et al., 2016). Sentence encoders incorporating syntactic structure, including Dependency Tree Recursive Neural Networks (DT-RNNss), further improved alignment by modeling rela- tional semantics (Socher et al., 2014). Fragment- level architectures extended this paradigm by decomposing images and sentences into finer-grained units and aligning them with structured max- max-margin objectives, en- abling both global and local cross-moda reasoning (Karpathy et al., 2014). Adversarial frameworks such invariant
ACMR subsequently refined joint embeddings by introducing nonlinear projections and regularization (Wang et al., 2017). We refer to Wang et al. (2024b) for comprchensive survey.
Dual-Encoder Contrastive Models and Controlled Interaction. Large-scale vision language datasets enabled paradigm shift toward bi-encoder architectures trained with contrastive objectives. CLIP and ALIGN independently encoded images and text and aligned them through contrastive learning on hun- dreds of millions to billions of image-text pairs, establishing scalable and efficient retricval backbones (Rad- ford et al., 2021; Jia et al., 2021; Wei et al., 2025). This architectural separation facilitated efficient indexing and retrieval and rapidly generalized to additional modalities including video, audio, depth, and sensor data (Girdhar et al., 2023; Chen et al., 2023b; Kong et al., 2025). Subsequent refinements explored the Srrriency-expressifie tradeoff within this paradigm, most notably through late-interaction architectures such as COI,ERR,, which replaced single-vector embeddings with multi-vector representations to enable
25

---

Published in Transactions on Machine Learning Research (02/2026)
token-level matching while preserving much of the efficiency of bi-encoders (Khattab & Zaharia, 2020; Faysse et al., 2025; Wan et al., 2025). Together, these models established dominant architectural family centered on modality-separated encoding with limited but scalable interaction.
Rich Fusion, Hierarchical Interaction, and Unified Architectures. Beyond bi-encoders, multi- modal retricval architectures increasingly incorporated sophisticated mechanisms to support complex rea- soning, particularly in video-text retrieval (Chen et al., 2020a). Transformer- based and hierarchical models decomposed aligmment into global-to-local or multi-granular stages, combining cross-modal attention with temporal and semantic structure (Gabeur et al., 2020; Liu et al., 2021; Zhang et al., 2023b; Gorti et al., 2022). Hybrid designs integrated CLIP-style encoders with pross-modal fusion or temporal alignment mod- ules, balancing pretrained representations with task-specific interaction (Portillo-Quinter et al., 2021; Fang et al., 2021). These architectures substantially increase computational cost, often restricting their use to reranking small candidate sets.
Recent architectures further differentiate between multi-encoder designs that preserve modality-specifie en- coders and singlo-encoder models employing full cross-attention at higher computational cost (Li ct al., 2019; 2022b; Zhai el al., 2023; Kim et al., 2025). At the same time, multimodal large language models (MLLMs) and multi-agent retrieval frameworks unify retrieval, reasoning, and generation within a single system, while codaaity-prrservingg and any-to-any architectures emphasize flexible interaction without collapsing modality structure (Liu et al., 2023; Xie et al., 2024a; Liu et al., 2025b; Xu et al., 2025a; Ju et al., 2025).
Remarks. From structural perspective, multimodal retrieval architectures can be broadly grouped into: (1) shallow fusion and projection-based models, (2) deep joint-embedding architectures with global fragment-level alignment, (3) contrastively trained dual-encoder models emphasizing scalability, (4) late- interaction hybrids balancing efficiency and expressivity, and (5) cross-attention and MLLM-centric ar- chitectures enabling deep fusion and unified multimodal reasoning. This progression reflects a recurring architectural tension between interaction richness and computational efficiency.
8.2 Performance- Efficiiicc Tradeoffs and the Role of Architectural Choices
Retrieval systems face fundamental tradcoff between effectiveness (c.g., recall, MRR, nDCG) and resource efficiency (latency, throughput, memory footprint, and index/update cost). Architectural choices how queries and documents are represented, how similarity is computed, and how candidates are staged and scored drive where method falls on this tradeoff curve. Modern neural retrievers demonstrated that learned dense embeddings can substantially improve effectiveness over classic lexical baselines, but achieving that gain requires extra compute and index engineering compared to lightweight lexical methods.
A common high-performance design is the bi-encoder (single-vector) architecture (Sections 6 and 7): queries and documents are encoded independently into compact vectors and approximate nearest-neighbor search (ANN) retrieves candidates quickly. This architecture attractive for strict latency budgets and large corpora because it enables highly optimized ANN indexes (c.g., HNSW (Malkov & Yashunin, 2018)) that deliver millisecond-scale queries at large scale; however, single-vector representations can miss fine-grained token-level matches that matter for some queries.
To improve effectiveness, rescarchers have pursued richer interaction patterns. Late-interaction or multi- vector models (e.g., COI.BRRT and COLPAli (Khattab & Zaharia, 2020; Faysse et al., 2025)) keep token- granular signals and aggregate local token similaritics, which raises offectiveness but incrcases index size and per-query compute; compression and residual quantization techniques can reduce the space/latency penalty but do not eliminate it entirely. Similarly, learned sparse/ sparse/lexical /lxxical models (e.g., SPLADE (Formal et al., 2021a;b)) produce high-dimensional but sparse representations that recover lexical signals and inter- pretability while trading off somewhat higher compute or indexing complexity versus classical BM25. These architectural variants illustrate the spectrum: more expressive interactions often translates to better quality, typically at higher memory and latency cost (unless mitigated by compression).
A pragmatic pattern in production is staged (two- or multi-stage) retrieval: a fast, coarse first-stage (BM25 compact dense ANN) produces small candidate set, and a more expensive cross-encoder interaction-
26

---

Published in Transactions on Machine Learning Research (02/2026)
based reranker refines the top results (Huang et al., 2020; Su et al., 2025). This cascade yields most of the accuracy of expensive models while preserving throughput, but it requires careful budgeting (how many candidates to pass) and engineering (batching, caching, and efficient GPU/CPU placement). Indexing and compression techniques (o.g., product quantization, pruning, residual quantization), along with hybrid lexical dense pipelines, are commonly employed to navigate the tradeoff curve as operational constraints evolve.
Architectural Design Hcuristics. Adopt singlo-vector dual-encoder with efficient ANN scarch when low latency, low cost, and frequent updales are the primary constraints. Favor multi-vector or interaction- intensive models when top-tier ranking quality is required and computational budget permits. To balance these objectives, employ a two-stage pipeline comprising coarse retrieval followed by specialized reranker. In large-scale deployments, the iitetiveness-effiiinnee operating point is chiefty adjusted through index compression techniques, hybrid lexical-dense signals, and careful calibration of candidate set size.
8.3 Calibration and Confidence Estimation
Calibration in IR aims to align model scores with true probabilities of relevance, such that confidence faithfully reflects correctness. Unlike traditional deterministic rankers that output single relevance scores, calibrated IR models explicitly represent uncertainty, often as a distribution over scores whose mcan encodes relevance belief and whose variance captures uncertainty (Cohen et al., 2021). This distinction archi- tecturally significant: uncertainty-aware scoring exposes information that is otherwise hidden in standard ranking pipelines. The importance of calibration is formalized by the Probability Ranking Principle (PRP), which guarantees optimal ranking only when relevance probabilities are well calibrated and reported with certainty (Penha & Hauff, 2021). However, modern neural rankers frequently violate these assumptions, motivating calibration-awarr architectural design.
Neural Architectures and Uncertainty Modeling. Calibration properties in IR are strongly depen- dent on architectural choices, with empirical studies showing mixed calibration behavior across neural model familics (Guo et al., 2017; Minderer et al., 2021). Transformer-base based rankers, including BERT variants, are often poorly calibrated, with calibration quality varying by model scale and design (Dan & Roth, 2021; Li et al., 2022a). Introducing stochasticity at the architectural level tree-ttreee -through stochastic inference approx- imate Bayesian formulations - consistently improves calibration compared to deterministic counterparts, while adding only modest computational overhead (Penha & Hauff, 2021; Cohen et al., 2021). These find- ings position stochastic and Bayesian architectures as principled mechanisms for embedding uncertainly directly into relevance estimation.
Calibration in Multi-Comp Colpi-Component Retrieval Systems. Calibration challenges are amplified in multi- stage retrieval architectured ures, such the "retrieve-lhen-rerankk pipeline, where hard top-k retrieval steps break differentiability and preclude end-to-end calibration. Architectural interventions, including differen- tiable sampling mechanisms based on Gumbel approximations, restore gradient flow and enable joint calibra- tion of retriever and reader components (Dhuliawala et al., 2022). Jointly calibrated systems produce more reliable confidence estimates than calibrating individual modules in isolation, underscoring how calibration requirements can directly shape architectural design in complex IR pipelines.
Modularity and Calibration-Aware Computation. Beyond end-to-end design, modular architectures enable calibration to be treated as attachable component rather than intrinsic model property. Uni- versal post-hoc calibrators can be applied across heterogeneous retrieval architectures without modifying their internals, offering scalable and architecture-agnostic improvements (Zhang et al., 2021a). When un- certainty is explicitly modeled, calibration becomes operational rather than merely diagnostic: confidence estimates can guide adaptive computation, such as selective reranking or deferred inference for ambiguous querics, improving both efficiency and robustness (Cohen al., 2021; Yoon et al., 2025). Recent evidence further suggests that architectural specialization c.g., modeling relevance across multiple criteria instead of single scalar score inherently reduce calibration error, reinforcing calibration as first-class archi-
27

---

Published in Transactions on Machine Learning Research (02/2026)
tectural consideration in IR system design (Penha & Hauff, 2021; Javdan et al., 2025). To summarize, how to incorporate calibration into modern retrieval systems is still an open question in the IR community.
8.4 Domain-Specific Applications
General Web Search. General-purpose web search has historically been dominated by the "retrieve- then-rerank" paradigm, built around inverted indexes and multi-stage cascade ranking ensure efficiency at web scale (Shen et al., 2014a; Mitra et al., 2016; Qin et al., 2022; Zhang et al., 2025a). These architectures decompose retrieval into heterogeneous components for query understanding, candidate generation, ranking, and re-ranking, each optimized independently (Wang & Na, 2023). As web content diversified, search engines incorporated additional signals such as anchor text, hyperlinks, and layout-based features to enrich document representations beyond plain text (Oliveira & Teixeira Lopes, 2023). While highly effective, these pipelines impose rigid, predefined information flows that limit adaptability. The recent generative retrieval (Section 7.3) thus explores replacing this modular pipeline with model-centric architectures, where a single LLM performs indexing, retrieval, and ranking end-to-end.
Domeif-Specific and Thematic Retrieval. Domain-specific search engines target locused corpora such as scientific literature, medicinal chemistry databases, and job postings, leveraging domain knowledge to im- prove relevance beyond general web search. These applications share architectural challenges arising from specialized terminology, underspecified expert queries, and narrowly scoped user intents (Wang & Na, 2023; Kang et al., 2024). Generic pretrained models often fail to capture domain-specifie semantics without archi- tectural support for structured knowledge. Traditional solutions rely on query expansion, lexical analysis, and large task-specific training sets, but suffer from scalability and complexity limitations (Xu et al., 2025f). In response, modern architectures increasingly integrate entity- and relation-centric representations to better align retricval with domain semantics (Dong et al., 2022a). As a result, domain-spocific IR architectures emphasize knowledge-aware representations and domain adaptation over purely generic pretrained models.
Medical and Legal Information Retrieval. Medical and legal search systems represent high-stakes in- stantiations of domain-specific retrieval, imposing strieter architectural constraints. Medical IR must operate over long temporal records that combine unstructured clinical notes with structured diagnoses, laboratory values, and medications, motivating architectures that integrate information extraction, faceted search, and structured database querying (Sonntag & Profitlich, 2018). These systems must additionally handle spe- cialized medical terminology, clinical documentation standards, and stringent privacy requirements. Legal IR faces analogous challenges due to highly specialized language, complex document structures, and limited labeled relevance data (Althammer et al., 2020). Despite recent efforts to adapt pretrained language models such as BERT, neural approaches often fail to consistently outperform strong lexical basclines like BM25, reinforcing the continued relevance of hybrid and lexically grounded architectures in legally specialized do- mains (de Araujo et al., 2014; Althammer et al., 2020).
E-commerce Search. E-commerce scarch systems operate over structured product catalogs rather than unstructured web pages, fundamentally shaping their architectures (Wang & Na, 2023; Ren et al., 2024). The dominant approach employs embedding- based retrieval using the bi-encoder architecture combined with approximate nearest neighbor search (Nigam et al., 2019; Lin et al., 2024). These systems must address severe vocabulary mismatch between customer queries and product descriptions, motivating query rewriting and semantic bridging components, often implemented using LLMs trained with contrastive instruction-based objectives (Peng et al., 2024b).
Ads Ranking. Ads ranking is another important industrial IR application secnario. Modern advertis- ing recommendation systems face the fundamental challenge of processing massive candidate sets while maintaining strict latency requirements and ranking quality. The traditional approach employs multi-stage cascading architectures that docompose the Ad ranking problem into sequential stages: retrieval, pro-ranking (or lightweight ranking), ranking (or heavyweight ranking), and auction (Gallagher et al., 2019; Wang et al., 2023b; Zheng al., 2024a; Yang et al., 2025d), where the research focus is similar to that of ad hoc rieval....
28

---

Published in Transactions on Machine Learning Research (02/2026)
In the context of Ads ranking, the XMC formulation described in Section 5 has been shown to improve ranking quality of tail items/A Ads (Jain et al., 2019; Dahiya et al., 2021; Yu et al., 2022a; Dahiya et al., 2023; Gupta et al., 2024b), which is often critical for revenue maximization. However, it also comes with higher engincering complexity and often roquires retraining of the models when new items/Ada are added.
The recent breakthrough of LLMs has catalyzed a paradigm shift toward generative recommendation frame- works that can directly generate personalized item sequences from user interaction histories. These ap- proaches index items with meaningful IDs using vector quantization algorithms and generate items from the entire item set for recommendation, conceptualizing online advertising systems a unified generative process that climinates inherent goal conflicts between different pipeline stages (Rajput et al., 2023; Zheng et al., 2023; Zhai et al., 2024). Given space constraints, we refer readers to the cited works for a more in-depth treatment of the field.
Pross-Applicationn IR. Across applications, several architectural trends are reshaping IR system design. Retrieval-augmeenee generation (RAG) architectures combine dense vector search with LLMs to support search, reasoning, and synthesis over structured and unstructured data (Asai al., 2024c). Modular and multi-agent retrieval frameworks further decouple retrieval functions into interoperable components, enabling dynamic adaplation ion to application-speciff requirements (Chang et al., 2025; Zhang el. al., 2025a). Finally, contextual personalization and the convergence of search and recommendation architectures reflect broader shift toward unified representations of users, documents, and intent (Zamani & Croft, 2018).
Taken together, the diversity of application-driven architectures underscores that modern IR systems are no longer optimized solely for static relevance ranking, but must incrcasingly adapt to new roles, constraints, and integration patterns that give rise to emerging architectural directions and open challenges.
9 Emerging Directions and Challenges
IR systems have become crucial across diverse domains, from retrieval-angmmented language modeling (Khan- delwal et al., 2020; Borgeaud al., 2022) applications in agents (Wu al., 2023a; Wang al,, 2024a; 2025a), code generation (Wang al., 2025b; Zhang et al., 2023a), robotics (Anwar et al., 2024), medicine (Jeong et al., 2024), and protein research (Jumper et al., 2021), inter alia. These developments present new challenges for IR. research. Drawing from the evolution of IR architectures (Sections 3 to 8), we examine emerging trends, open problems, and potential rescarch dircctions. Our discussion is structured around three key areas: advancing the core components of IR models, adapting to the new paradigm of retrieval for AT, and tackling the pragmatic challenges of real-world deployment.
9.1 Advancing the Core Components of IR Models
At the heart of any IR system are the models that extract features and estimate relevance. As IR moves toward more compute-intensive practices, we identify key areas for improving these fundamental components.
More Powerful and Efficient Foundation Models. Scaling has been winning recipe for modern neural networks (Kaplan et al., 2020; Hoffmann et al., 2022; Dehghani et al., 2023; Fang et al., 2024; Shao et al., 2024, inter alia). However, for IR to leverage this trend sustainably, several challenges in model design and training must be addressed:
Data and training efficiency. Current transformer-based IR models demand extensive training data (Fang et al., 2024), making them impractical for many real-world applications. Developing architectures that can learn eveel from limited data in few-shot/zero-ssol setting remains crucial. Additionally, models should support parallel processing and low-precision training to reduce costs and accelerate convergence (Pool et al., 2021; Fishman et al., 2024; Liu et al., 2024a).
Inference optimization. Real-time applications like conversational scarch (Mo et al., 2024b) and agent-based systems (Yao et al., 2023) require efficient handling of variable-length queries, necessitating advanced compression and optimization techniques for both retriever backbones and
29

---

Published in Transactions on Machine Learning Research (02/2026)
index structures (Dettmers & Zettlemoyer, 2023; Kumar et al., 2024; Xu et al., 2024b; Warner et al., 2025; Bruch et al., 2024; Xu et al., 2025b, inter alia).
Better lite foundation models. IR models need to process queries in real time, and using compact-sized foundation models is often a practical solution. Warner et al. (2025) presented an interesting study on wide-and-shallow versus deep-and-naree narrow architecture the context of training "modern" BERT model. Recent works (Günther et al., 2024; Fu et al., 2023; Portes al., 2023) studied training efficient BERT model that supports longer context length, while Nussbaum & Duderstadt (2025) investigated the efficacy of mixture-of-exper BERT-style encoder model. The best recipe for lite foundational models for IR applications remains an open question.
Transformer alternatives. While transformers have dominated recent. IR research, their quadratic complexity attention computation remains a significant bottleneck. Recent advances linear RNNs (Peng et al., 2023; 2024a; Qin al., 2024c), state space models (Gu & Dao, 2024; Dao & Gu, 2024), and linear attention (Katharopoulos et al., 2020; Yang et al., 2025c) offer alternatives with thcoretical lincar complexity. Although preliminary studies (Xu, 2024; Zhang et al., 2024; Xu et al., 2025e;c) show limited gains compared lo optimized transformers, developing elficient alternative architectures for transformers could revolutionize large-scale IR.
Ultimately, strong foundation models have proven crucial for IR performance (Neclakantan et al., 2022; Ma et al., 2024b, inter alia). As IR applications expand, developing foundation models that balance computa- tional efficiency with robust performance across tasks and modalities emerges as a key research priority.
Flexible and Scalable Relevance Estimators. As discussed in Section 6, cross-encoders provide plex non-linear relevance estimation but are computationally expensive. In contrast, bi-encoder architectures used in dense and sparse retrieval rely on linear similarity functions (e.g., inner product) lo enable fast re- trieval through nearest neighbor search and inverted indexing. Balancing complex relevance matching and scalable retricval remains challenging. COLBERT (Khattab & Zaharia, 2020) addresses this by using docu- ment atpnesentation matrices with MaxSim operations, while recent work (Killingback el al., 2025) explores Hypernetworks (Ha et al., 2022) to generate query-specific neural networks for relevance estimation. The design of flexible yet scalable relevance estimators remains an active research direction.
9.2
The Shifting Paradigm: From Search for Humans Retrieval for AI
The integration of TR systems into other research domains presents new challenges. We discuss key implica- tions for future IR modeling research the primary "user" of retrieval shifts from humans Al models.
The End "User" of Retricval. While traditional IR systems focus on providing scarch results to humans, retrieval is increasingly used to support ML models, particularly LLMs, in tasks such as generation (Gao et al., 2023), reasoning (Yao et al., 2024; Islam et al., 2024), tool usage (Schick et al., 2023; Patil et al., 2024b; Qin et al., 2024a; Patil et al., 2024a) and planning (Song et al., 2023). This shifting paradigm raises fundamental questions about task formulation, evaluation, and system optimization:
Current IR rescarch is grounded in human information-sseekngg behavior (Wilson, 2000; Marchionini, 2006; White & Roth, 2009, inter alia). When the end user becomes another ML model, we must reconsider how to define and assess relevance. For example, document might be irrelevant to human but contain a key factual nugget that an LLM needs to answer question. This suggests a need for flexible, data-efficient models that are adaptable to various downstream tasks.
Traditional IR metrics, which are designed for human-centria evaluation, may not align with down- stream task performance in retricval-augmented systems (Lewis al., 2020b; Petroni al., 2021; Asai et al., 2024c). Future IR models should support end-to-end system optimization rather than focusing solely ranking metrics (OpenAT, 2025; Huang et al., 2025).
30

---

Published in Transactions on Machine Learning Research (02/2026)
Retrieval Augmented Generation. Retrieval Augmented Generation (RAG) refers to architectures that combine an external retrieval module with generative model, enabling the system to access and condition on external knowledge before producing output. Early RAG formulations separate retrieval and generation as distinct components: a dense or sparse retriever extracts relevant passages for a given input, and a generator (usually an LLM) conditions on those retrieved results to produce lext. (Lewis el. al., 2020b; Ram et al., 2023b; Mallen et al., 2023; Asai et al., 2024c; Chen et al., 2020b; Xu Cohen, 2023). This architectural decoupling allows retrieval to be optimized independently from generation, which is useful for modularity and scalability, A typical RAG pipeline comprises:
Corpus embedding and indexing (vector stores or sparse indexes) for fast retrieval;
Retriever model (dense vectors or hybrid techniques) that returns top-k relevant itemss;
Fusion mechanism that combines retrieved content with the query (e.g., concatenation or cross- attention) before passing to the generator;
Generator that produces the final answer based on fused context.
This simple architecture improves factual accuracy and reduces hallucination by grounding generation on external context. However, it also introduces key engineering and modeling considerations:
Retrrever-Geenerttor Coupling: The architectural choice of where and how to combine retrieval results affect quality and efficiency. Early approaches rely simple concatenation of text passages into the generator's context window (Lewis et al., 2020b; Ram et al., 2023b), which do not alter the architecture of the generator. More advanced designs use cross-attention layers or rerankers to improve precision (Izacard & Grave, 2021; Park et al., 2023; An et al., 2025b).
Multi-Modal and Structured Retrieval: Recent work explores RAG variants that extend be- yond plain text. For example, GraphRAG and related graph-enhanced frameworks integrate struc- tured knowledge sources, enabling retrieval of relational paths not easily captured by vector similar- ity alone (Edge et al., 2025; Li et al., 2025a). These architectures introduce additional components like graph encoders and reasoning layers, allowing the system to extract multi-hop or entity-centric context before generation.
Newer RAG pipelines incorporate reasoning capabilities into the generator, to dynamically decide whether to retrieval or to generate answer (Trivedi et al., 2023; Jiang et al., 2023b; Asai et al., 2024b).
Adaptive Retrieval:
These architectural choices have direct implications: strong retrieval can reduce generation errors but may increase pipeline complexity and latency; graph-aware models can support structured reasoning at the cost of larger indexes and additional components; adaptive retrioval stratogics can improve offectiveness but require careful task planning and retrieval coordination.
Overall, RAG architectures represent a spectrum from simple retrieval-plus- generrttnnn pipelines to complex multi-component systems that integrale heterogeneous knowledge representations and adaptive retrieval planning to improve both relevance and reasoning capability. We refer readers to (Gao et al., 2023; Li et al., 2025c; Mci et al., 2025) for more comprehensive reviews of this topic.
The Rise of Autonomous Search Agents. Complex tasks often require retrieving long-tail tail knowledge using lengthy, complex queries (Soudani ct al., 2024; Su et al., 2024), demanding retricval models capable of instruction following (Weller et al., 2025a; Ravfogel el al., 2024) and reasoning (Su el al., 2024; Shao el. al., 2025; Liang et al., 2025). This has spurred the development of autonomous search agents. Existing attempts can be divided into two main directions. One line of work focuses on training more capable retrievers and rerankers. This involves creating new data pipelines synthesize instruction-follviii focused training data (Oh et al., 2024; Weller et al., 2024; Shao et. al., 2025, inter alia) and leveraging
and reasoning-
31

---

Published in Transactions on Machine Learning Research (02/2026)
strong backbone language models such as the proprictary GPT-4 family models (OpenAI et al., 2023) and open-weight QWEN (Yang et al., 2025a) to imbue the retriever with reasoning capabilities.
system as a static tool and focuses on improving a large reasoning model's (LRM) ability to use the retrieval tool to improve reasoning and handle in in-context mem- ory (Jin et al., 2025; Yu et al., 2025; Gao et al., 2025; Xu et al., 2025d, inter alia). In this setup, the LRM decides when, where, and how to conduct a search, and the results subsequently influence its further reasoning and decision-making (Nakano et al., 2021; He et al,, 2025). This line of work, commonly referred to Deep Research Agents, represents paradigm shift from retrieval as the end goal to retrieval tools for LLMs. Formally, we use the definition from Huang et al. (2025): deep research agents refer to AT agents powered by LLMs, integrating dynamic reasoning, adaptive planning, multi-iteration external data retrieval and tool use, and comprehensive analytical report generation for informational research tasks.
Another line of work treats the
From the retrieval perspective, the retrieval usage of deep rescarch agents can be broadly categorizod into two lypes: (1) API-based search engines which interact with structured data sources, such as search engine APIs scientific database APIs; (2) browser- based search engines, which simulate human-like interactions with web pages and enable real-time extraction of dynamic or unstructured content leveraging LLMs' long- context, code understanding and multimodality capabilities.
Both formulations have strengths and weaknesses. API-based retrieval, in line with traditional TR literature, is a fast, efficient, structured and scalable method to enable deep research agents' access to external knowl- edge (Schütze et al., 2008; Singh et al., 2025b). Browser-based retrioval (Nakano et al., 2021; OpenAI, 2025) offers the advantage of simulating real-time, interactive information-seeking behavior- conceptually similar to that of human users. However, it incurs additional latency and token consumption, and introduces further complexity arising from the web-browsing environment. Given the current landscape of retrieval integration within deep research agents, a central open question is how to design a hybrid retrieval architecture that combines the strengths of both paradigms to achieve a superior performance-eftiiingg tradeoff.
From the model training perspective, recent efforts have abstracted away the details of retrieval models, treating retrieval as static tools and instead focusing on improving the capabilities of LLM-based search agents. For example, many recent efforts aim to train LRMs to use search tools more effectively via rein- forcement learning specialized fine-tuning (Li et al., 2025b; Jin et al., 2025; Li et al., 2025c; Chen et al., 2025; Wu et al., 2025b;a, inter alia). This approach is central to agentic frameworks that orchestrate tool use for complex task completion (Wu et al., 2023a; Shinn et al., 2023; Asai et al., 2024b). Despite this exciting progress, key limitations remain. To enable retrievers' reasoning capability often requires strong backbone models (e.g.; 7B scale), which can be infeasible for production systems. Even larger models (e-g.. 32B scale) augmented with retricval and trained via expensive reinforcement lcarning (Jin et al., 2025; Chen et al., 2025) still sometimes underperform simpler baselines with query decomposition and chain-of-thought prompting (Khot et al., 2023; Trivedi et al., 2023). A key open question is how to endow retrievers with strong reasoning capabilities using lightweight, scalable models. Another challenge lies in the joint opti- mization of retrievers and language models within a unified, reasoning-awan framework. Lastly, the human factors of applying such autonomous search agents remain to be studied. We refer readers to (Singh et al., 2025a; Liang et al., 2025; Xi et al., 2025; Lin et al., 2025) for more comprehensive reviews of this topic.
9.3 Retrieval Beyond Simple Relevance: Instruction and Reasoning- Retrieval
Instruction-Folloving Retrieval. Instruction- retrieval extends the standard query-document formulation by conditioning the retriever on natural-language instruction. Architecturally, this is typi- cally implemented by jointly encoding the instruction and the query in shared bi-encoder, for example through simple concatenation or lightweight attention layers, that the query representation adapts to task intent (Weller et al., 2025a; Oh et al., 2024). Training objectives often use contrastive lcarning with codeuuctoonconddttooneee posilives and hard negatives, allowing the model to capture fine-grained task seman- tics without large cross-encoder computations (Weller et al., 2024). Instruction-awan aware retrievers benefit from curated or synthetic datascts pairing instructions, queries, and relevant passages, which improves robustness to paraphrasing and query phrasing variations. These design choices preserve scalability while improving performance on instruction-heavy tasks. Hybrid pipelines may further combine instruction-conniiionn re-
32

---

Published in Transactions on Machine Learning Research (02/2026)
trieval with downstream rerankers fusion mechanisms, ensuring that retrieved evidence aligns with the specific needs of generation or reasoning tasks (Shao et al., 2024; Weller et al., 2025b).
Rcasoning-Aware Retrieval. Reasoning-aware retrieval aims to retrieve evidence that supports multi- step inference and complex reasoning (Su et al., 2024). One line of work uses graph-based architectures, where the retriever constructs and encodes relational graphs or multi-hop chains of passages. These compo- graph construction, encoding, path selection, and ranking allow the retriever to return connected evidence that reflects reasoning dependencies (Liu al., 2025a; Edge al., 2025). While effective for compositional queries, these approaches increase system complexity, lateney, and index size.
A different line of work, exemplified by REASONIR (Shao et al., 2025) and RANKR1 (Zhuang et al., 2025), improves reasoning capability through data curation and task-aligned training rather than altering the un- derlying "retrieve-then-rerank pipeline. By generating multi-step reasoning datasets or instruction-guided examples, these methods teach standard retrievers and rerankers to prioritize evidence that is useful for down- stream reasoning, achicving stronger performance without introducing graph- based components. From an evaluation perspective, reasoning aware retrieval highlights the need for task centeren metrics that measure downstream reasoning success, emphasizing the tight connection between architectural or training choices and reasoning effectiveness.
9.4 Deployment, Robustness, and Trustworthiness
As modern IR systems become more powerful and intograted into high-stakes applications, ensuring their practical deployability and reliability is paramount.
Tradeoff at Scale. Traditional retrieval systems face significant chal- lenges when scaling to web-scale document corpus, and deploying such systems requires a blend of science and engineering expertise (Dean, 2009; Huang et al., 2020; Li et al., 2021). In recent years, reurreeal-uugneenteed generation, conversational search, and agentic systems with memory have been widely adopted for informa- tion access (Guu et al., 2020; Lewis et al., 2020b; Nayak, 2019; OpenAI, 2024; Google, 2024, inter alia). These applications often require multiple rounds of retrieval and operate on dynamic corpuses, urging for efficient and effective retrieval. Mainstream inference optimization frameworks such as vLLM (Kwon et al., 2023) and SGLang (Zheng et al., 2024b) have provided support for embedding models. From the modeling perspective, an open question is to design and pre-train models explicitly for retricval purposes (Warner et al., 2025; Nussbaum et. al., 2025; Günther et al., 2024).
The
Ensuring Robustness in Noisy and Adversarial World. We discuss few challenges in IR models' deployment in noisy environments, especially when used in retrieval-augmented generation systems. We should note that while these challenges have been studied by prior works, it remains an open question on how to mitigate these challenges from the perspective of IR modeling and architectures.
Robustness to Al-generated content. With the advent of LLMs, the amount of Al-generated content is also increasing. Dai et al. (2024) showed that neural retrievers are biased towards Al- generated documents. Xu et al. (2024a) showed that similar problems persist in text-image retrieval models. Future IR modeling research should also consider the robustness of models to Al-generated content.
Robustness to adversarial attacks Recent works ad hoc retrieval and RAG LLM safety have discussed BERT-based models' brittleness to adversarial attacks (Wang et al., 2022b), as well as the threat of corpus poisoning where injected harmful documents lead to unsafe RAG system outputs (Zhong et al., 2023; Xiang et al., 2024a; Deng et al., 2024, inter alia). This topic is also relevant to the safety of LIM agents using tools (Deng et al., 2025; Tian et al., 2023; Xiang et al., 2024b), noting the importance of IR models being robust to adversarial attacks for downstream applications.
33

---

Published in Transactions on Machine Learning Research (02/2026)
Robustness to bias and toxicity. As noted by recent work (An et al., 2025a), documents that contain biases and toxic materials can potentially jailbreak aligned LLMs. This observation highlights the importance for IR models to be robust to bias and toxic content.
Robustness to imperfect retrieval results. Different works have pointed out that existing RAG systems show performance degradation when the retrieval results contain irrelevant docu- ments (Yoran et al., 2024; Chang et al., 2025; Yu et al., 2024c, inter alia). Therefore, the RAG paradigm demands more precise results from the retrieval models.
Robustness to dusti--dist input. Given the fact that modern neural retrieval models are trained with data-driven approaches, perhaps it is not surprising to find their performance may vary with different linguistic properties of the queries and documents, i.e, out-of-distribution input from the training data. Several works have reported cross-encoder rerankers' performance drops on out-of-domain datasets (Mokrii et al., 2021; Thakur et al., 2021). In the context of retrieval- augmented generation, Cao et al. (2025) conducted rigorous benchmarking and found that for- malily, readability, politeness and grammalical correct fundamental aspects of real-world user- LLM queries can lead to significant performance variances of retrievers and RAGG systems. This observation highlights the importance of retrieval models' robustness to out-of-distribution (OOD) input (Gupta al., 2024a).
We refer readers for more detailed discussions on IR models' robustness to dedicated surveys (Asai et al., 2024c; Liu et al., 2025c; Zhou et al., 2024). Addressing these robustness issues at the model architecture level is critical and underexplored direction for future research.
10 Conclusions and Closing Thoughts
The journey of IR model architectures, as we have charted, is a story of escalating abstraction and semantic depth. Beginning with the foundational principles of term-based matching in Boolean, vector space, and probabilistic models, the field systematically evolved. Learning-to-Rank introduced the power of machine learning to combine diverse statistical features, but it was the advent of neural networks that marked the first major leap toward semantic understanding. These neural ranking models, with their ability to learn reprosentations directly from text, began to bridge the lexical gap that had long constrained traditional methods. The arrival of pre-trained transformers like BERT then catalyzed paradigm shift, providing a powerful, universal foundation for both highly-effeccive cross-encoder rerankers and efficient bi-encoder retrievers. Most recently, the ascent of LLMs has not only scaled up these cxisting architectures but has also introduced entirely new paradigms, such as generative retrieval and zero-shot listwise reranking, fundamen- tally reshaping what a retrieval system can do.
Throughout this evolution, core architectural tension has persisted: the tradcoff between effectiveness and efficiency. This is the fundamental conflict between deep, fine- the-grained interaction models (like interaction- based neural rankers and cross-encoders) that offer high accuracy, and scalable representation-batt (like vector space models, dense retrievers) that enable fast, pre-oomputable search over massive collections. The enduring weetreeve-then-eraee like multi-vector models (e-g., COIBERT) and hybrid sparse-dense systems represent icphed attempts to find a better balance on this spectrum, pushing the Parcto fronticr of what is possible.
models
pipeline is a direct architectural answer to this tradeoff. Innovations
Today, we stand at another inflection point. The primary "user" of IR. is shifting from human at search bar to an Al model within a larger system. IR is no longer just tool for finding documents; it is becoming critical cognitive component for retrieval-augmenntd generation, autonomous agents, and complex reasoning systems. This shift, as outlined in Section 9, forces us to re-evaluate our core assumptions. Relevance is no longer solely about satisfying human information needs but about providing the precise factual or contextual information an AI needs to complete a downstream task. This demands new model architoctures that are not only powerful and efficient but also instruction-aware, contextually flexible, and seamlessly integrable into end-to-end differentiable systems.
34

---

Published in Transactions on Machine Learning Research (02/2026)
Looking ahead, the future of IR. model architecture will be defined by its ability to meet these new demands. The grand challenges lie in building foundation models that are multimodal, multilingual, and computation- ally sustainable; in designing systems that are robust, trustworthy, and resistant to adversarial manipulation; and in developing autonomous search agents that can reason, plan, and interact with the world's information on our behalf. As IR becomes ever more deeply embedded in the fabric of artificial intelligence, its continued evolution will be crucial, not just for the future of search, but for the future of intelligent systems themselves.
Acknowledgments
This work was partially supported by grants from NSF (DMS 2134223 and IIS 2205418).
References
Qingyao Ai, Keping Bi, Jiafeng Guo, and W. Bruce Croft. Learning a deep listwisc context model for ranking refinement. In Proceedings of the 4 41sl International ACM SIGIR Conference on Research & Development in Information Retrieval, pp. 135 144, 2018a.
Qingyao Ai, Jiaxin Mao, Yiqun Liu, and W. Bruce Croft. Unbiased learning to rank: Theory and practice. In Proceedings of the 27th ACM International Conference on Information and Knowledge Management, CIKM '18, pp. 2305 2306, New York, NY, USA, 2018b. Association for Computing Machinery. doi: 02444//3326220.32777772
Qingyao Ai, Xuanhui Wang, Sebastian Bruch, Nadav Golbandi, Michael Bendersky, and Marc Najork. Tearning groupwise multivariate scoring functions using deep neural networks. In Proceedings of the 2019 ACM SIGIR International Conference on Theory of Information Retrieval, pp. 85 92, 2019.
Sophia Althammer, Sebastian Hofstätter, and Allan Hanbury. Cross-domain retrieval in the legal and patent domains: a reproducability study. In European Conference on Information Retrieval, 2020.
Bang An, Shiyue Zhang, and Mark Dredze. RAG LLMs are not safer: A safety analysis of aetrieeel--uggmetee generation for large language models. In Luis Chiruzzo, Alan Ritter, and Lu Wang (eds.), Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational L Linguistics: Human Language Technologies (Volume 1: Long Papers), pp. 5444-5474, Albuquerque, New Mcxico, April 2025a. Association for Computational Linguistics. URL https: htappss//accaathooog..ooo wwwoooyy..ooooggoooeee org/2025. naacl-long. 281/.
Yuwei An, Yihua Cheng, Seo Jin Park, and Junchen Jiang. HyperRAG: Enhancing quality- tradeoffs in retrieval-augmented generation with reranker kv-cache reuse. arXiv preprint arx::250440222 2025b.
Alan Ansell, Edoardo Ponti, Anna Korhonen, and Ivan Vulié. Composable sparse fine-tuning for cross- lingual transfer. In Smaranda Muresan, Preslav Nakov, and Aline Villavicencio (eds.), Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 1796, Dublin, Ireland, May 2022. Association for Computational Linguistics. doi: 10. 11653/v1/220 acl-long.125. URL https: httpp/////ccaathhlgggg/o org/2022. acl-long 125/.
Abrar Anwar, John Welsh, Joydeep Biswas, Soha Pouya, and Yan Chang, Remembr: Building and reasoning over long-horizon spatio-temporal memory for robot navigation. arXiv preprint arXiv:2400. 1e00009....60 13682, 2024.
Sanjeev Arora, Yingyu Liang, and Tengyu Ma. A simple but tough-to-beat baseline for sentence embeddings. In International Conference on Learning Representations, 2017. URL http:://ppparrrree.nee//// id=SyK00v5xx.
Mikel Artetxe, Sebastian Ruder, and Dani Yogatama. On the cross-lingual transferability of monolingual representations. In Dan Jurafsky, Joyce Chai, Natalie Schluter, and Joel Tetreault (eds.), Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pp. 4623-4637, Online, July 2020. Association for Computational Linguisties. doi: 10.18653 10.86633vv/222/2 v1/2020..aa acl-main 421. URL https: / //aclanthology.oo wwwp:///oooggooo//0000000 acl- main. 421/.
35

---

Published in Transactions on Machine Learning Research (02/2026)
Akari Asai, Timo Schick, Patrick Lewis, Xilun Chen, Gautier Izacard, Sebastian Riedel, Hannanch Hajishirzi, and Wen-tau Yih. Task-aware retrieval with instructions. In Anna Rogers, Jordan Boyd-Graber, and Naoaki Okazaki (eds.), Findings of the Association for Computational Linguistics: ACL 2023, pp. 3650 3675, Toronto, Canada, July 2023. Association for Computational Linguistics. doi: 10.18653/v1/202. findings-acl2255. URL https: https://aclanthont http://accaanthoggyyy..... wrg. findings-acl.222 225/.
Akari Asai, Jacqueline He, Rulin Shao, Weijia Shi, Amanpreet Singh, Joseph Chee Chang, Kyle Lo, Luca Soldaini, Sergey Feldman, Mike D'arey, al. OpenScholar: Synthesizing scientific literature with retrieval- auginented LMs. arxiv preprint ar 2rxx:2411.14199 2024a.
Akari Asai, Zeqiu Wu, Yizhong Wang, Avirup Sil, and Hannanch Hajishirzi. Self-RAG: Learning to re- trieve, generate, and critique through self-reftection. In The 12th International Conference on Learning Representations, 2024b. URL https://openrevier. http:/wwwwww..eeeeeeeeeeeeeeeeee wet. htt.://wwww...e.ooooo///a////a/a /ttt/foorn..dessy5o0oo...
Akari Asai, Zexuan Zhong, Danqi Chen, Pang Wei Koh, Luke Zettlemoyer, Hannanch Hajishirzi, and Wen-tau Yih. Reliable, adaptable, and attributable language models with retrieval. ar preprint arXiv:2403.0311 2024c.
Ricardo Baeza- Yates and Berthier Ribeiro-Neto. Modern Information Retrieval. ACM Press; Addison- Wesley, 1st edition, 1999.
Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio. Neural machine translation by jointly learning to align and translate. arXiv preprint vrXio:1409.0473 2014.
Yang Bai, Xiaoguang Li, Gang Wang, Chaoliang Zhang, Lifeng Shang, Jun Xu, Zhaowei Wang, Fangshan Wang, and Qun Liu. SparTerm: Learning term-based sparse representation for fast text retrieval. arXiv preprint ar 2010 00766,, 2020.
Yuntao Bai, Saurav Kadavath, Sandipan Kundu, Amanda Askell, Jackson Kernion, Andy Jones, Anna Chen, Anna Goldie, Azalia Mirhoseini, Cameron McKinnon, et al. Constitutional Al: Harmlessness from AI feedback. arXiv preprint ar arXiv:2212.0680 080773 2022.
Soyuj Basnet, Jerry Gou, Antonio Mallia, and Torsten Suel. Deeperimpact: Optimizing sparse learned index structures. arXie preprint 4rXtt:2405.1700) 2024.
Parishad BehnamGhader, Vaibhav Adlakha, Marius Mosbach, Damitry Bahdanau, Nicolas Chapados, and Siva Reddy. LLM2Vee: Large language models secretly powerful text encoders. In 1st Conference Language Modeling, 2024. URL
Ix Beltagy, Matthew E. Peters, and Arman Cohan. Longformer: The long-document transformer. arXiv preprint trXfv:2004.05150, 2020.
Adam Berger and John Lafferty. Information retrieval as statistical translation. In Proceedings of the 22nd Annual International ACM SIGIR Conference Research and Development in Information Retrieval, pp. 222 229, 1999.
Michele Bevilacqua, Giuseppe Otlaviano, Patrick Lewis, Scott Yih, Sebastian Riedel, and Fabio Petroni. Au- toregressive search engines: Generating substrings as document identifiers. In Alice H. Oh, Alekh Agarwal, Danielle Belgrave, and Kyunghyun Cho (eds.), Advances in Neural Information Processing Systems, 2022.
URL https: / /pwnreveeeeeeeeeeeeeeeeeeeeeeeee nett:forummidd=24222aaa
Asia J. Biega, Krishna P. Gummadi, and Gerhard Weikum. Equity of attention: Amortizing individual fairness in rankings. In The 41st International ACM SIGIR Conference on Research & Development in Information Retrieval, pp. 405 414, 2018.
Luiz Bonifacio, Hugo Abonizio, Marzich Fadacc and Rodrigo Nogucira. InPars: Unsupervised datasct generation for information retrieval. In Proceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 2387-2392, 2022.
36

---

Published in Transactions on Machine Learning Research (02/2026)
Sebastian Borgeaud, Arthur Mensch, Jordan Hoffmann, Trevor Cai, Eliza Rutherford, Katie Millican, George Bm Van Den Driessche, Jean-Baptiste Lespiau, Bogdan Damoc, Aidan Clark, et al. Improving language models by retrieving from trillions of tokens. In International Conference on Machine Learning, pp. 2206-2202222 2206-2240. PMLR, 2022.
Leonid Boytsov, Preksha Patel, Vivek Sourabh, Riddhi Nisar, Sayani Kundu, Ramya Ramanathan, and Eric Nyberg. InPars-Light: Cost-effective unsupervised training of efficient rankers. Transaclions Machine Learning Research, 2024.
Sergey Brin and Lawrence Page. The anatomy of large-scale hypertextual web search engine. Compuler Networks, 30:107-117, 1998. URL http://www-db: wta.bbrttafor....o balb/bbbckrrubgoooglee .hmml.
Andrei Z. Broder, David Carmel, Michael Herscovici, Aya Soffer, and Jason Zien. Efficient query evaluation using two-level retrieval process. In Proceedings of the 12th International Conference on Information and Knowledge Management, pp. 426-434 2003.
Peter F. Brown, Stephen A. Della Pietra, Vincent J. Della Pietra, and Robert L. Mercer. The mathematics statistical machine translation: Parameter estimation. Computational linguistics, 19(2):263-311 1993.
Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D. Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. Language models are few-shot learners. Advances in neural information processing systems, 33:1877 1901, 2020.
Sebastian Bruch, Masrour Zoghi, Michael Bendersky, and Marc Najork. Revisiting approximate metric optimization in the age of deep neural networks. In Proceedings of the 42nd International ACM SIGIR Conference Research und Development. in Information Retrieval, pp. 1241-1244, 2019.
Sebastian Bruch, Franco Maria Nardini, Cosimo Rulli, and Rossano Venturini. Efficient inverted indexes for approximate retrieval over learned sparse representations. In Proceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 152-162, 2024.
Cristian Buciluä, Rich Caruana, and Alexandru Niculescu-Mizil. Model compression. In Proceedings of the 12th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 535-541, 2006.
Chris Burges, Tal Shaked, Erin Renshaw, Ari Lazier, Matt Deeds, Nicole Hamilton, and Greg Hullender. Learning to rank using gradient descent, In Proceedings of the 22nd International Conference on Machine Learning, pp. 89-96, 2005.
Christopher Burges, Robert Ragno, and Quoe Le. Learning to rank with nonsmooth cost functions. Advances in neural information processing systems, 19, 2006.
Christopher J.C. Burges. From ranknet to lambdarank lambdamart: An overview. Learning, 11(23-581): 81, 2010.
Tianyu Cao, Neel Bhandari, Akhila Yerukola, Akari Asai, and Maarten Sap. Out of style: RAG's fragility linguistic variation. orXiv preprint arXiv:2504. 08231, 2025.
Zhe Cao, Tao Qin, Tic-Yan Liu, Ming-Fus Feng Tsai, and Hang Li. Learning to rank: from pairwise approach to listwise approach. In Proceedings of the 24th International Conference on Machine Learning, pp. 129-136, 2007.
Chia-Yuan Chang, Zhimeng Jiang, Vineeth Rakesh, Menghai Pan, Chin-Chin Michael Yeh, Guanchu Wang, Mingzhi Hu, Zhichao Xu, Yan Zheng, Mahashweta Das, and Na Zou. MAIN-RAG: Multi-agent filtering retrieval-augmented generation. In Wanxiang Che, Joyce Nabende, Ekaterina Shutova, and Moham- mad Taher Pilehvar (eds.), Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 2607-2622, Vienna, Austria, July 2025. Association for Com- putational Linguistics. doi: 10.18653/v1 wtt03//l1/222..a// 22222 131. URL 1o..oong.1///
hat.ollooyyorg//22.... org/2025.
37

---

Published in Transactions on Machine Learning Research (02/2026)
Yin- Wen Chang, Cho-Jui Hsich, Kai-Wei Chang, Michael Ringgaard, and Chih-Jen Lin. Training and testing low-degree polynomial data mappings via linear svm. Journal of Machine Learning Research, 11 (48):1471 11911471-449,,,, 2010. URL http://jmlr
Jiangui Chen, Ruqing Zhang, Jiafeng Guo, M. de Rijke, Wei Chen, Yixing Fan, and Xueqi Cheng. Contin- ual learning for generative retrieval over dynamic corpora. Proceedings of the 32nd ACM International Conference on Information and Knowledge Management, 2023a.
Mingyang Chen, Tianpeng Li, Haoze Sun, Yijie Zhou, Chenzheng Zhu, Haofen Wang, Jeff Z. Pan, Wen Zhang, Huajun Chen, Fan Yang, et al Learning to reason with scarch for LLMs via reinforcement learning. arxiv preprinl arXiv:2503. 19470, 2025.
Shizhe Chen, Yida Zhao, Qin Jin, and Qi Wu. Fine-grained video-text retrieval with hierarchical graph reasoning. 2020 IEEF/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 10635- 10644, 2020a.
Sihan Chen, Handong Li, Qunbo Wang, Zijia Zhao, Ming-Ting Sun, Xinxin Zhu, and J. Liu. VAST: A sesion-addiosubtiill--e amni-modality foundation model and dataset. ArXiv preprint arXiv:2305. 18500, 2023b.
Wei-Fan Chen, Shahbaz Syed, Benno Stein, Matthias Hagen, and Martin Potthast. Abstractive snippet generation. In Proceedings of the Web Conference 2020, WWW '20, pp. 1309 1319, New York, NY, USA, 2020b. Association for Computing Machinery. ISBN 9781450370233, doi: 10.1145 01111453336423.33366 https://dol org/or/.//// 10. 1145/3366423.... 33802066.
URL
Xuanang Chen, Ben He, Kai Hui, Le Sun, and Yingfei Sun. Simplified TinyBERT: Knowledge distillation for document retrieval. In Europeon Conference on Information Retrieval, pp. 241-248 Springer, 2021.
Zewen Chi, Li Dong, Furu Wei, Nan Yang, Saksham Singhal, Wenhui Wang, Xia Song, Xian-Ling Mao, Heyan Huang, and Ming Zhou. InfoXIMM An information-theoreiicc framework for cross-lingual language model pre-training, In Kristina Toutanova, Anna Rumshisky, Luke Zettlemoyer, Dilek Hakkani-Tur, Iz Beltagy, Steven Bethard, Ryan Cotterell, Tanmoy Chakraborty, and Yichao Zhou (eds.), Proceedings of the 2021 Conference of the North American Chapter of the Association for Compulational Linguislics: Human Language Technologies, pp. 3576 3588, Online, June 2021. Association for Computational Linguistics. org/2021 naacl-main. maa...../......aa..m........a.
doi: 10. 10.18653/v 11/2211.1122222 www.//2..aacclllnee maaaclmmain...m 280. URL https: bettps://aclanthot
Hyung Won Chung, Le Hou, Shayne Longpre, Barret Zoph, Yi Tay, William Fedus, Yunxuan Li, Xuezhi Wang, Mostafa. Dehghani, Siddhartha Brahma, et al. Scaling instruction-finetueed language models. Journal of Machine Learning Research, 25(70):1 53, 2024.
Kevin Clark, Urvashi Khandelwal, Omer Levy, and Christopher D. Manning, What does BERT look at? An Analysis of BERT's Attention. In Tal Linzen, Grzegorz Chrupala, Yonatan Belinkov, and Dieuwke Hupkes (eds.), Proceedings of the 2019 ACI. Workshop BlackbozNL.P: Analyzing and Interpreting Neural Networks for NLP, pp. 276 286, Florence, Italy, August 2019. Association for Computational Linguistics. URL https://aclanthooogy)
doi: 10. 111115//1//1191411/
Charles I. A. Clarke and Laura Dietz. LLM-based relevance assessment still can't replace human relevance assessment. arXiv preprint ar 212 2024.
Daniel Cohen and W. Bruce Croft. End to end long short term memory networks for non-factoid question answering. In Proceedings of the 2016 ACM International Conference on the Theory of Information Retrieval, ICTIR '16, pp. 1431146 New York, NY, USA, 2016. Association for Computing Machinery. doi:
10.1145/2970338829700
Daniel Cohen, Bhaskar Mitra, Oleg Lesota, Navid Rekabsaz, and Carsten Eickhoff. Not all relevance scores are equal: Efficient uncertainty and calibration modeling for deep retrioval models. Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval, 2021.
38

---

/15533//1/220accmmmaaa Published in Transactions on Machine Learning Research (02/2026)
Ronan Collobert, Jason Weston, Léon Bottou, Michael Karlen, Koray Kavukcuoglu, and Pavel Kuksa. Natural language processing (almost) from scratch. Journal of machine learning research, 12(7), 2011.
Alexis Conneau, Kartikay Khandelwal, Naman Goyal, Vishrav Chaudhary, Guillaume Wenzek, Francisco Guzmán, Edouard Grave, Myle Ott, Luke Zettlemoyer, and Veselin Stoyanov. Unsupervised cross- lingual representation learning at scale. In Proceedings of the 58th Annual Meeting of the Associa- URL
tion for Computational Linguistics, pp. 8440-8451, 2020a. doi: https://aclanthology org/2020/00
Alexis Conneau, Shijie Wu, Haoran Li, Luke Zettlemoyer, and Veselin Stoyanov. Emerging cross-lingual structure in pretrained language models. In Dan Jurafsky, Joyce Chai, Natalie Schluter, and Joel Tetreault (eds.), Proceedings of the 58th Annual Meeting of the Association for Computational Linguisties, pp. 6022 6034, Online, July 2020b. Association for Computational Linguistics. doi: 11111668/1122200.00 acl- http:/0000..mmmmmm535333333 URL https://aclanthol///
org/2022 acl-main 536/.
Gordon V. Cormack, Charles L. A. Clarke, and Stefan Bucttcher. Reciprocal rank fusion outperforms condorcet and individual rank learning methods. In Proceedings of the 32nd International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR '09, pp. 758-759, New York, NY, USA, 2009. Association for Computing Machinery. doi: 10.1145/1571994115771111
Corinna Cortes and Vladimir Vapnik. Support-vector networks. Machine learning, 20(3):273 297, 1995.
Kunal Daliya, Ananye Agarwal, Deepak Saini, Jian Jiao, Amit Singh, Sumeet Agarwal, Purushottam Kar, Manik Varma, et al. Siamesexml: Siamese networks meet extreme classifiers with 100m labels. In Inter- national Conference on Machine Learning, pp. 2330-2340. PMLR, 2021.
Kunal Dahiya, Sachin Yadav, Sushant Sondhi, Deepak Saini, Sonu Mehta, Jian Jiao, Sumeet Agarwal, Purushottam Kar, and Manik Varma. Deep encoders with auxiliary parameters for extreme classification. In Proceedings of the 29th ACM SIGKDD Conference Knowledge Discovery and Data Mining, pp.
358 337,, 2023.
Sunhao Dai, Yuqi Zhou, Liang Pang, Weihao Liu, Xiaolin Hu, Yong Liu, Xiao Zhang, Gang Wang, and Jun Xu. Neural retrievers are biased towards LLM-generated content. In Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pp. 526-537, 2024.
Zhuyun Dai and Jamie Callan. Deeper text understanding for ir with contextual neural language mod- cling. Proceedings of the 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval, 2019a.
Zhuyun Dai and Jamie Callan. Context-aware sentence sentence/passage passage term importance estimation for 1st stage retrieval. arXio preprint arXiv:1910.10699 2019b.
Zhuyun Dai, Chenyan Xiong, Janie Callan, and Zhiyuan Liu. Convolutional neural networks for soft- matching n-grams in ad-hoc search. In Proceedings of the 11th ACM International Conference on Web Search and Data Mining, WSDM '18, pp. 126 134, New York, NY, USA, 2018. Association for Computing Machinory. doi:
Zhuyun Dai, Vincent Y. Zhao, Ji Ma, Yi Luan, Jianmo Ni, Jing Lu, Anton Bakalov, Kelvin Guu, Keith Hall, and Ming-Wei Chang. Promptagator: Few-shot dense retrieval from 8 examples. In The 1 11th International Conference on Learning Representations, 2023.
Soham Dan and Dan Roth. On the elfects of transformer size on in- and out-of-domain calibration. In Mlarie-Francin Moens, Xuanjing Huang, Lucia Specia, and Scott Wen-tau Yih (eds.), Findings of the As- sociation for Computational Linguistics: EMNLP 2021, pp. 2096-2101, Punta Cana, Dominican Republic, November 2021. Association for Computational Linguistics. doi: 10.18653/v1 v122021 fndings-emnlp. URL http:::/acclaaabloogg.oo//////// findings--nnneie www.inngg-emll111 emllp.
39

---

Published in Transactions on Machine Learning Research (02/2026)
Tri Dao and Albert Gu. Transformers are SSMs: Generalized models and efficient algorithms through structured state space duality. Proceedings of the 4 1st International Conference on Machine Learning, pp. 10041 - 10071, 2024.
Arpan Dasgupta, Siddhant Katyan, Shrutimoy Das, and Pawan Kumar. Review of extreme multilabel classification. arXiv preprint arXiv:2302.055: 05971, 2023.
Denis Andrei de Araujo, Carolina Müller, Rove Chishman, and Sandro José Rigo. Information extraction for legal knowledge representation-a review of approaches and trends. Revista Brasileira de Computação Aplicada, 6(2):2 19, 2014.
Nicola De Cao, G. Izacard, S. Riedel, and F. Petroni. Autoregressive entity retrieval. In ICLR 2021-9th International Conference on Learning Representations, volume 2021. ICLR, 2021.
Jeffrey Dean. Challenges in building large-scale information retrieval systems: invited talk. In Proceedings of the 2nd ACM International Conference on Web Search and Data Mining, 2009. doi: 10.1145/14987777 1498761
Mostafa Dehghani, Josip Djolonga, Basil Mustafa, Piotr Padlewski, Jonathan Heek, Justin Gilmer, An- dreas Peter Steiner, Mathilde Caron, Robert Geirhos, Tbrahim Alabdulmohsin, et al. Scaling vision transformers to 22 billion parameters. In International Conference on Machine Learning, pp. 7480-7512. PMLR, 2023.
Gelei Deng, Yi Liu, Kailong Wang, Yuekang Li, Tianwei Zhang, and Yang Liu. Pandora: Jailbreak GPTs by retricval augmented gencration poisoning. arXiv preprint orXiv:2402.08.4 2024.
Zehang Deng, Yongjian Guo, Changzhou Han, Wanlun Ma, Junwn Xiong, Sheng Wen, and Yang Xiang. AI agents under threat: A survey of key security challenges and future pathways. ACM Computing Surveys, 57(7):1-36, 2025.
Tim Dettmers and Luke Zettlemoyer. The case for 4-bit precision: k-bit inference scaling laws. In Interna- tional Conference on Machine Learning, pp. 7770-7774. PMLR, 2023.
Jacob Devlin, Ming-Wee Wei Chang, Kenton Lee, and Kristina Toutanova. BERT: Pre-training of deep bidirec- tional transformers for language understanding, In Jill Burstein, Christy Doran, and Thamar Solorio (eds.), Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pp. 4171-4186, Min- neapolis, Minnesola, June 2019. Association for Computational Linguistics. doi: 10.18653/v1/ND 1/233111 N11///999111////
URL https: htat::://aclanthooggg
Shehzaad Dhuliawala, Leonard Adolphs, Rajarshi Das, and Mrinmaya Sachan. Calibration of machine reading systems at scale. In Smaranda Muresan, Preslav Nakov, and Aline Villavicencio (eds.), Find- ings of the Association for Computational Linguistics: ACL 2022, pp. 1682 1693, Dublin, Ireland, May 2022. Association for Computational Linguistics. doi: 10.18633/vl /tt.///2..2.//nn//./. findings-acl. 133/.
URL
Qian Dong, Yiding Liu, Suqi Cheng, Shuaiqiang Wang, Zhicong Cheng, Shuzi Niu, and Dawei Yin. Incorpo- rating explicit knowledge in pre-trained language models for passage re-ranking. Proceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval, 2022a.
Qian Dong, Shuzi Niu, Tao Yuan, and Yucheng Li. Disentangled graph recurrent network for document ranking. Data Science and Engineering, 7:30 -3, 2022b.
Susan T. Dumais, Todd A. Letsche, Michael L. Littman, and Thomas K. Landauer. Automatic lrogu-lan retricval using latent semantic indexing. In AAAT Spring Symposium on Cross-Language Text and Speech Retrieval, volume 15, pp. 21, 1997.
Darren Edge, Ha Trinh, Newman Cheng, Joshua Bradley, Alex Chao, Apurva Mody, Steven Truitt, Dasha Metropolitansky, Robert Osazuwa Ness, and Jonathan Larson. From local global: graph RAG approach to query-focused sunmarivation. Xixiv preprint or orxi::2404. 161330 2025.
40

---

Published in Transactions on Machine Learning Research (02/2026)
Nelson Elhage, Neel Nanda, Catherine Olsson, Tom Henighan, Nicholas Joseph, Ben Mann, Amanda Askell, Yuniao Bai, Anna Chen, Tom Conerly, Nova DasSarma, Dawn Drain, Deep Ganguli, Zac Hatfield-Dodds. Danny Hernandez, Andy Jones, Jackson Kernion, Liane Lovitt, Kamal Ndousse, Dario Amodei, Tom Brown, Jack Clark, Jared Kaplan, Sam McCandlish, and Chris Olah. A mathematical framework for transformer circuits. Transformer Circuils Thread, 2021. URL ttapss//trransfomme--rrree pub/ 2021/framework/ winpee///iiiiiieeeeeeee mondex.tml.
Adel Elmahdy, Sheng-Chieh Chieh Lin, and Amin Ahmad. Synergistic approach for simultaneous optimization of monolingual, cross-lingual, and multilingual information retrieval. arXiv preprint 208:2408.. 10536, 2024.
Jeffrey L. Elman. Finding structure in time. Cognitive Science, 14(2):179-21) 1990. doi: 10.1016/ (3614-0213(90)90002EE
Gugliclmo Faggioli, Laura Dictz, Charles L. A. Clarke, Gianluca Demartini, Matthias Hagen, Clandia Hauff, Noriko Kando, Evangelos Kanoulas, Martin Potthast, Benno Stein, al. Perspectives on large language models for relevance judgment. Tn Proceedings of the 2023 ACM STGIR International Conference on Theory of Information Retrieval, pp. 39 50, 2023.
Guglielmo Faggioli, Laura Dietz, Charles L. A. Clarke, Gianluca Demartini, Matthias Hagen, Clandia Hauff, Noriko Kando, Evangelos Kanoulas, Martin Potthast, Benno Stein, et al. Who determines what relevant? humans or AT? why not both? Communications of the ACM, 67(4):31-34 34, 2024.
Han Fang, Pengfei Xiong, Lubui Xu, and Yu Chen. CLIP2Video: Mastering vidco-text retricval via image CLIP. ArXio preprint arXiv:2166. 11097, 2021.
Yan Fang, Jinglao Zhan, Qingyao Ai, Jiaxin Mao, Weihang Su, Jia Chen, and Yiqun Liu. Scaling laws for dense retrieval. In Proceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 1339-1349, 2024.
Manuel Faysse, Hugues Sibille, Tony Wu, Bilel Omrani, Gautier Viaud, Céline Hudelot, and Pierre Colombo. ColPali: Efficient document retrieval with vision language models. ar preprint orXiv:2407.01443 2025.
Fangxiaoyu Feng, Yinfci Yang, Daniel Cer, Naveen Arivazhagan, and Wei Wang. Languago-agnostic BERT sentence embedding. In Smaranda Muresan, Preslav Nakov, and Aline Villavicencio (eds.), Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 878 891, Dublin, Ireland, May 2022. Association for Computational Linguistics. doi: (0.18633/v1/2000 acl-long.62. URL https://aclantholloon
arg/2022.aco acl- 1all-oog.660
Maxim Fishman, Brian Chmicl, Ron Banner, and Daniel Soudry. Scaling fp8 training trillion-tkken LLMs. arXiv preprinl or 20x: 12517, 2024.
Christian Fluhr, Robert E. Frederking, Doug Oard, Akitoshi Okumura, Kai Ishikawa, and Kenji Satoh. Mul- tilingual (or cross-lingual) information retrieval. Proceedings of the Multilingual Information Management: Current Levels and Future Abilities, pp. 10-13, 1999.
Thibault Formal, Carlos Lassance, Benjamin Piwowarski, and Stéphane Clinchant. SPLADE v2: Sparse lexical and expansion model for information rotrieval. arXiv preprint arXiv:2109.10088 2021a.
Thibault Formal, Benjamin Piwowarski, and Stéphanc Clinchant. SPLADE: Sparse lexical and expansion model for 1st stage ranking. In Proceedings of the 44th Intervational ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 2288-2292, New York, NY, USA, 2021b. Association for Computing Machinery. doi: 0111145/444835.3444
Thibault Formal, Stéphane Clinchant, Hervé Déjean, and Carlos Lassance. SPLATE: Sparse late interaction retrieval. In Proceedings of the 4 77h International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 2635-26400 2024. doi:
Yoav Freund and Robert E Schapire. desicion-theorett generalization of on-line learning and an application to boosting. Tn European Conference on Compulational Learning Theory, pp- 23 37. Springer, 1995.
41

---

Published in Transactions on Machine Learning Research (02/2026)
Yoav Freund, Raj Iyer, Robert E. Schapire, and Yoram Singer. An efficient boosting algorithm for combining preferences. Journal of machine learning research,
2003.
Jerome H. Friedman. Greedy function approximation: a gradient boosting machine. Annals of statistics, pp. 1189-1232, 2001.
Andrea Frome, Greg S Corrado, Jon Shlens, Samy Bengio, Jeff Dean, Marc' Aurelio Ranzato, and Tomas Mikolov. DeViSE: A deep visual-semantic embedding model. In C.J. Burges, L. Bottou, M. Welling, Z. Ghahramani, and K.Q. Weinberger (eds.), Advances in Neural Information Processing Systems, vol- ume 26. Curran Associates, Inc., 2013.
Dan Fu, Simran Arora, Jessica Grogan, Isys Johnson, Evan Sabri Eyuboglu, Armin Thomas, Benjamin Spector, Michael Poli, Atri Rudra, and Christopher Ré. Monarch Mixer: A simple sub-quadratic GEMM- based archilecture. Aduances in Newral Information Processing Systems, 36:77546 77603, 2023.
Norbert Fuhr. Probabilistic models in information retrieval. The compuler journal, 35(3):243 255, 1992.
Valentin Gabeur, Chen Sun, Alahari Karteek, and Cordelia Schmid. Multi-modal transformer for video retrieval. arXiv preprint or Xiv: 2007.10638 2020.
Luke Gallagher, Ruey-Cheng Chen, Roi Blanco, and J. Shane Culpepper. Joint optimization of cascade ranking models. Proceedings of the 12th ACM International Conference on Web Search and Data Mining, 2019.
Jianfeng Gao, Patrick Pantel, Michael Gamon, Xiaodong He, and Li Deng. Modeling interestinguess with deep neural networks. In Proceedings of the 2014 Conference on Empérical Methods in Natural Language Processing (EMNLP), pp. 2-13, 2014.
Jiaxuan Gao, Wei Fu, Minyang Xie, Shusheng Xu, Chuyi He, Zhiyu Mei, Banghua Zhu, and Yi Wu. Beyond ten turns: Unlocking mor-horizon agentic search with largo-scale asynchronous rl, 2025. URL https: //aaxiiv.rrg.///e..............e abrrg//bb/2508... 07977
Luyu Gao and Jamie Callan. Condenser: pre-training archilecture for dense retrieval. In Marie-Francine Moens, Xuanjing Huang, Lucia Specia, and Scott Wen-tau Yih (eds.), Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pp. 981-993, Online and Punta Cana, Dominican Republic, November 2021. Association for Computational Linguistics. doi: 10. 18653/v11 11t1/222.eeeeeee mmnnl--mann. 2021.. mmmlp-main.78 75/.
http::://acaatttolll 75. URL https: http///aaaann/t// aclanthologyyy....o org/2021.
Luyu Gao and Jamie Callan. Long document re-ranking with modular re-ranker. Proceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieual, 2022.
Luyu Gao, Zhuyun Dai, and Jamie Callan, Understanding BERT rankers under distillation. In Proceedings of the 2020 ACM SIGIR on International Conference on Theory of Information Retrieval, pp. 149-152, 2020.
Luyu Gao, Zhuyun Dai, and Jamie Callan. COIT: Revisit exact lexical match in information retrieval with contextualized inverted list. In Kristina Toutanova, Anna Rumshisky, Luke Zettlemoyer, Dilek Hakkani- Tur, Iz Beltagy, Steven Bethard, Ryan Cottcrell, Tanmoy Chakraborty, and Yichao Zhou (cds.), Proceedings of the 2021 Conference of the North American Chapter of the Association for Compulational Linguistics: Human Language Technologies, pp. -040- Online, June 2021a. Association for Com- putational Linguistics. doi: naacl maal.mainn..m main.241/.
URL https://aclanthology wrgp//21............
Tianyu Gao, Xingcheng Yao, and Danqi Chen. SimCSE: Simple contrastive learning of sentence embeddings. In Frani-Fra Moens, Xuanjing Huang, Lucia Specia, and Scott Wen-tau Yih (eds.), Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pp. 6894-6910, Online and Punta Cana, Dominican Republic, November 2021b. Association for Computational Linguistics. doi: 10. 10118853vv/2200 http://111222..mmpmmmm....mmmm emmllp-minn552. URL https: http:///aaaaantoy..../.. www.://...o.....o..e..... emnlp-man matppmma.................. 552/.
42

---

Published in Transactions on Machine Learning Research (02/2026)
Yunfan Gao, Yun Xiong, Xinyu Gao, Kangxiang Jia, Jinliu Pan, Yuxi Bi, Yi Dai, Jiawei Sun, and Haofen Wang. Retrieval-augmentel generation for large language models: A survey. Xiv preprinl arXiv:2312. 2023.
Gemini Team Google, Rohan Anil, Sebastian Borgeaud, Jean-Baptiss Alayrac, Jiahui Yu, Radu Soricut, Johan Schalkwyk, Andrew M. Dai, Anja Hauth, Katic Millican, et al. Gemini: A family of highly capable multimodal models. Xiv preprinl or Xiv:23312. 11805, 2023.
Rohit Girdhar, Alaacldin El-Nouby, Zhuang Liu, Mannat Singh, Kalyan Vasudev Alwala, Armand Joulin, and Ishan Misra. ImageBind one embedding space bind them all. 2023 IEEE/CVE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 15180-15190, 2023.
Yunchao Gong, Qifa Ke, Michael Isard, and Svetlana Lazebnik. A multi-view embedding space for modeling internet images, tags, and their semantics. International journal of computer vision, 106(2):210-233, 2014.
Google. Grounding with google search. python, 2024.
Satya Krishna Gorti, Noël Vouitsis, Junwei Ma, Keyvan Golestan, Maksims Volkovs, Animesh Garg, and Guangwei Yu. X-Pool: Cross-modal language-video attention for text-video retrieval. 2022 IEEE/CVP Conference on Computer Vision and Paltern Recognition (CVPR), pp. 4996-5005, 2022.
Koustava Goswami, Sourav Dutta, Haytham Assem, Theodorus Fransen, and John McCrae. Cross- lingual sentence embedding using multi-task learning. In Maric-Francine Moens, Xuanjing Huang, Lu- cia Specia, and Scott Wen-tau Yih (eds.), Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pp. 9099-911991 Online and Punta Cana, Dominican Republic, Novem- ber 2021. Association for Computational Linguistics. doi: 10.18653/v1, wwwp///.eenllmm.....eee main.716.... URL https://aclantholle. org/2021. emnlp-main 716/.
Jianping Gou, Baosheng Yu, Stephen J. Maybank, and Dacheng Tao. Knowledge distillation: A survey.
International journal of computer vision, 129(6):1789 1819, 2021.
Roksana Goworek, Olivia Macmillan-coott, and Eda B. Özyigit. Bridging language gaps: Advances in
cross-lingual information retrieval with multilingual LLMs. arXiv preprint 3rXiv:2510.00900 2025.
Albert Gu and Tri Dao. Mamba: Linear-time sequence modeling with selective state spaces. In 1st Conference on Language Modeling, 2024.
Michael Günther, Jackmin Ong, Isabelle Mohr, Alaeddine Abdessalem, Tanguy Abel, Mohammad Kalim Akram, Susana Guzman, Georgios Mastrapas, Saba Sturua, Bo Wang, Maximilian Werk, Nan Wang, and Han Xiao. Jina Embeddings 2: 8192-token general-purpose text embeddings for long documents. arXiv preprint M.Xi::3310119222 2024.
Chuan Guo, Geoff Pleiss, Yu Sun, and Kilian Q. Weinberger. On calibration of modern neural networks. arXiv preprint ar Xiv: 7777770.4550 2017.
Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, et al. DeepSeek-Rl: Incentivizing reasoning capability in LLMs via reinforcement learning. arx preprint arXiv:2501. 12948, 2025.
Jiafeng Guo, Yixing Fan, Qingyao Ai, and W. Bruce Croft. deep relevance matching model for ad-hoc retrioval. In Proceedings of the 25th ACM International on Conference on Information and Knowledge Management, CIKM '16, pp. 55-64 64, New York, NY, USA, 2016a. Association for Computing Machinery. doi: 01145/298333.298368
Jiafeng Guo, Yixing Fan, Qingyao Ai, and W. Bruce Croft. Semantic matching by non-linear word trans- portation for information retricval. In Proceedings of the 25th ACM International on Conference on In- formation and Knowledge Management, CIKM '16 pp. 701-710, New York, NY, USA, 2016b. Association for Computing Machinery. doi: 10.1145/2983323.1
43

---

Published in Transactions on Machine Learning Research (02/2026)
Jiafeng Guo, Yixing Fan, Liang Pang, Liu Yang, Qingyao Ai, Hamed Zamani, Chen Wu, W Bruce Croft, and Xueqi Cheng. A deep look into neural ranking models for information retrieval. Information Processing & Management, 57(6))102067, 2020.
Ashim Gupta, Rishanth Rajendhran, Nathan Stringham, Vivek Srikumar, and Ana Marasovic. Whispers of doubt amidst ochoes of triumph in NLP robustness. In Kevin Duh, Helena Gomez, and Steven Bethard (eds.), Proceedings of the 2024 Conference of the North American Chapler of the Associalion for Compula- tional Lònguistics: Human Language Technologies (Volume 1: Long Papers), pp. 5533 5590, Mexico City, Mexico, June 2024a. Association for Computational Linguistics. doi: 10.18653/v1 11t/2222.2aaacloon.
URL https: http:///aaaaatthooooooo org/2024. maaalllong.311.. 310/.
Nilesh Gupta, Fnu Devvrit, Ankit Singh Rawat, Srinadh Bhojanapalli, Prateek Jain, and Inderjit S. Dhillon. Dual-encoders for extreme multi-label classification. In The 12th International Conference on Learning Representations, 2024b. URL
Suchin Gururangan, Ana Marasović, Swabha Swayamdipta, Kyle Lo, Ix Beltagy, Doug Downey, and Noah A. Smith. Don't stop pretraining: Adapt language models to domains and tasks. In Dan Jurafsky, Joyce Chai, Natalic Schluter, and Joel Tetreault (eds.), Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pp- 8342 8360, Online, July 2020. Association for Computational Linguistics. wwwpooooooyyoggggooeeeo/ org/2020. acl mtmmain...0/....
doi: 10.18653/ 00053///12022..clmmiinn
URL
Michael U. Gutmann and Aapo Hyvärinen. Noise-contrastive estimation of unnormalized statistical models, with applications to natural image statistics. The journal of machine learning research, 13(1):307-361, 2012.
Kelvin Guu, Kenton Lee, Zora Tung, Panupong Pasupat, and Mingwei Chang. Retrieval augmented language model pre-training. In International Conference on Machine Learning, pp. 3929-399 -938. PMLR, 2020.
David Ha, Andrew M. Dai, and Quoe V. Le. Hypernetworks. In International Conference on Learning Representations, 2022.
Hua He, Kevin Gimpel, and Jimmy Lin. Multi cuiti-perspecctve sentence similarity modeling with convolu- tional neural networks. In Lluís Mårquez, Chris Callison-Burch, and Jian Su (eds.). Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pp. 1576-1586, Lisbon, Por- tugal, September 2015. Association for Computational Linguistics. doi:
URL
wtwp:oloyy.oogg/1-11111 org/D15-118//.
Yichen He, Guanhua Huang, Peiyuan Feng, Yuan Lin, Yuchen Zhang, Hang Li, et al. PaSa: An LLM agent for comprehensive academic paper search. arXiv preprint 2rx1::2501.101 20, 2025.
Zhankui He, Zhouhang Xie, Harald Steck, Dawen Liang, Rahul Jha, Nathan Kallus, and Julian McAuley. teindex-thennddapt:: Improving large language models for conversational recommendation. Proceedings of the 18th ACM International Conference on Web Search and Dala Mining, 2024.
Djoerd Hiemstra and Wessel Kraaij. Twenty-one al TREC-7: Ad-hoc and cross-language track. In 7th Terl REtrieval Conference, TREC-7 1998, pp. 227-238. National Institute of Standards and Technology, 1999.
Geoffrey Hinton, Oriol Vinyals, and Jeff Dean. Distilling the knowledge in a neural network. stat, 1050:9, 2015.
S. Hochreiter and J. Schmidhuber. Long short-term memory. Neural Computation MIT-Prest 1997.
Jordan Hoffmann, Sebastian Borgeaud, Arthur Mensch, Elena Buchatskaya, Trevor Cai, Eliza Rutherford, Diego de Las Casas, Lisa Anne Hendricks, Johannes Welbl, Aidan Clark, et al. Training compute-optimal large language models. ar preprint arXiv:2203.15555 2022.
Scbastian Hofstätter, Sophia Althammer, Michael Schröder, Mete Sertkan, and Allan Hanbury. Im- proving efficient neural ranking models with cross-architecture knowledge distillation. arXio preprint arXiv:201000666 2020a.
44

---

Published in Transactions on Machine Learning Research (02/2026)
Sebastian Hofstätter, Hamed Zamani, Bhaskar Mitra, Nick Craswell, and Allan Hanbury. Local self-attention over long text for efficient document retrieval. In Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 2021 2024, 2020b.
Sebastian Hofstätter, Markus Zlabinger, and Allan Hanbury. Interpretable & sime-bbudgee-consttaainddddd con- textualization for re-ranking. In ECAI 2020, pp. 513-520. IOS Press, 2020c.
Scbastian Hofstätter, Sheng-Chich Lin, Jheng-Hong Yang, Jimmy Lin, and Allan Hanbury. Efficiently teaching an effective dense retriever with balanced topic aware sampling, In Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 113- 122, 2021a.
Sebastian Hofstätter, Bhaskar Mitra, Hamed Zamani, Nick Craswell, and Allan Hanbury. Intra-document cascading: Learning to select passages for neural document ranking. Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval, 2021b.
Scbastian Hofstätter, Omar Khattab, Sophia Althammer, Mete Sertkan, and Allan Hanbury. Introducing neural bag of whole-words with ColBERTer: Contextualized late interactions using enhanced reduction. Proceedings of the 31st ACM International Conference on Information & Knowledge Management, pp-
737 747, 2022.
Harold Hotelling. Relations between two sets of variates. In Breakthroughs in Slatislics: Methodology and Distribution, pp. 162 190. Springer, 1992.
Jeremy Howard and Sebastian Ruder. Universal language model fine-tuning for text classification. arXiv preprinl ar Xix::1801.001 06146, 2018.
Baotian Hu, Zhengdong Lu, Hang Li, and Qingcai Chen. Convolutional neural network architectures for matching natural language sentences. In Proceedings of the 28th International Conference on Newral Information Processing Systems- Volume 2, NIPSS11, 14, pp. 2042-2050, Cambridge, MA, USA, 2014. MIT Press.
Mengkang Hu, Yuhang Zhou, Wendong Fan, Yuzhou Nie, Bowei Xia, Tao Sun, Ziyu Ye, Zhaoxuan Jin, Yingru Li, Qiguang Chen, et al. OWL: Optimized workforce learning for general multi-agent assistance real-world lask automation. arXiv preprint orXiv22505.28855 2025.
Ziniu Hu, Yang Wang, Qu Peng, and Hang Li. Unbiased LambdaMARI: An unbiased pairwise learning-te to rank algorithm. In The World Wide Web Conference, pp. 2830-2836, 2019.
Jui-Ting Huang, Ashish Sharma, Shuying Sun, Li Xia, David Zhang. Philip Pronin, Janani Padmanabhan, Giuseppe Ottaviano, and Linjun Yang. Embedding-based retrieval in facebook search. Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pp. 2553 2561, 2020.
Po-Sen Huang, Xiaodong He, Jianfeng Gao, Li Deng, Alex Acero, and Larry Heck. Learning deep structured semantic models for web search using clickthrough data. In Proceedings of the 22nd ACM International Conference on Information & Knowledge Management, CIKM '13, pp. 2333-2338, New York, NY, USA, 2013. Association for Computing Machinery. doi: 10. N0.1145/2505515.550555
Xu Huang, Weiwen Liu, Xiaolong Chen, Xingmei Wang, Hao Wang, Defu Lian, Yasheng Wang, Ruim- ing Tang, and Enhong Chen. Understanding the planning of LLM agents: survey. arXiv preprint arXiv:2102.022 02716, 2024a.
Yuxuan Huang, Yihang Chen, Haozheng Zhang, Kang Li, Meng Fang, Linyi Yang, Xiaoguang Li, Lifeng Shang, Songeen Xu, Jianye Hao, al. Deep research agents: A systematic examination and roadmap. arXiv preprint or Xic:2506. 1 18096, 2025.
45

---

Published in Transactions on Machine Learning Research (02/2026)
Zhiqi Huang, Puxuan Yu, Shauli Ravfogel, and James Allan. Language concept erasure for invariat dense retrieval. In Yaser Al-Onaizan, Mohit Bansal, and Yun-Nung Chen (eds.), Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pp. 13261 13273, Miami, Florida, USA, November 2024b. Association for Computational Linguistics. doi: 10.18653 101553/vv//0 /2044 http///....//m/mmmm...a camll-mmain... 736.
URL https: http:///caaathhooooooo www.ollooyy.ggg/2222 wttg/2024.mmmpmmmaaiie.. emnlp-main. 7366/
Kai Hui, Andrew Yates, Klaus Berberich, and Gerard de Melo. PACRR: position-awaro neural IR model for relevance matching. In Martha Palmer, Rebecca. Hwa, and Sebastian Riedel (eds.), ) Proceedings of
the 2017 Conference on Empirical Methods in Natural Longuage Processing, pp. 1049 1058, Copenhagen, Denmark, September 2017. Association for Computational Linguistics. doi: 10.18663/v1/D17-1110 URL https://aclanthollo0 org/D17-1110/
Kai Hui, Andrew Yates, Klaus Berberich, and Gerard de Melo. Co-PACRR: A context-aware neural ir model for ad-hoc retrieval. In Proceedings of the 11th ACM International Conference Web Search and Dala Mining, WSDM '18, pp. 279-28772 New York, NY, USA, 2018. Association for Computing Machinery. doi: (0.1145/3159655.3159668
Samuel Humean, Kurt Shuster, Marie-Anne Lachaux, and Jason Weston. Poly-encoders: Architectures and pre-training strategics for fast and accurate multi-sentence scoring. In International Conference on Learning Representations, 2020. URL https: /tttppe//openreveee.. ntw.:/oummee/oommmeeee// wtwrrrmmmddddddedeeeee SkxgnnNFV...
Shayekh Bin Islam, Md Asib Rahman, K. S. M. Tozammel Hossain, Enamul Hoque, Shafiq Joty, and Md Rizwan Parvez. Open-RAG: Enhanced retrieval augmented reasoning with open-source large language models. In Yaser Al-Onaizan, Mohit Bansal, and Yun-Nung Chen (eds.), Findings of the Association for Computational Linguistics: EMNLP 2024, pp. 14231 14244, Miami, Florida, USA, November 2024. Association for Computational Linguistics. doi: 10.18653/v /v 12024 find...iiiigg----o.. com.g.s8mmmmp.8... 831. URL https:// aclanthology org/2024 Sindngg--emll..33
Gautier Izacard and Edouard Grave. Leveraging passage retrieval with generative models for open domain question answering. In Paola Merlo, Jorg Tiedemann, and Reut Tsarfaty (eds.), Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume, pp. 974-880, Online, April 2021. Association for Computational Linguistics. doi: (0.18653/v1/20)) 1aa22021.ccllmaiii 74. URL https////caaaatoooooo/oo org/2021 eacl-main. 74/.
Alon Jacovi and Yoav Goldberg. Towards faithfully interpretable NLP systems: How should we define and evaluate faithfulness? Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pp. 4198-4205, Online, July 2020. Association for Computational Linguistics. doi: 10.18653/v1/202D mma.m.....cm URL
Dan Jurafsky, Joyce Chai, Natalie Schluter, and Joel Tetreault (eds.),
org 2020. acl aallmain..30 ma61.
Himanshu Jain, Yashoteja. Prabhu, and Manik Varma. Extreme multi-label loss functions for recommen- dation, tagging, ranking & other missing label applications. In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 935-944, 2016.
Himanshu Jain, Venkatesh Balasubramanian, Bhanu Chunduri, and Manik Varma. Slice: Scalable linear extreme classifiers trained on 100 million labels for related searches. In Proceedings of the 12th ACM International Conference on Web Search and Data Mining, pp. 128-536, 2019. doi: 10.1145/3289600. 3290979.
Soroush Javdan, Pragash Krishnamoorthy, and Olga Baysal. CREST: Improving interpretability and effec- tiveness of troubleshooting at ericsson through criterion-specific trouble report retrieval. arXie preprint 1rXiv:2511.17417,, 2025.
Frederick Jelinek. Interpolated estimation of Markov source parameters from sparse data. In Proceedings of Workshop on Pattern Recognition in Practice, 1980.
Minbyul Jeong, Jiwoong Sohn, Mujeen Sung, and Jacwoo Kang. Improving medical reasoning through retrieval and self-reflection with retrieval-angmented large language models. Bioinformatics, 40 (Supplement. 1itpplement__1:il111- (ehil19-i129) 2024. doi: 10.1093 httn://bbiiformatis//baaaae
46

---

Published in Transactions on Machine Learning Research (02/2026)
Chao Jia, Yinfei Yang. Ye Xia, Yi-Ting Chen, Zarana Parekh, Hieu Pham, Quoe V. Le, Yun-Hsuan Sung, Zhen Li, and Tom Duerig. Scaling up visual and vision-language representation learning with noisy text supervision. In International Conference on Machine Learning, 2021.
Albert Qiaochu Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chaplot, Diego de Las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lucile Saulnier, L'elio Renard Lavaud, Maric-Anne Lachaux, Pierre Stock, Teven Le Scao, Thibaut Lavril, Thomas Wang, Timothée Lacroix, and William El Sayed. Mistral 7B. arXiv preprint arXit:2310. 06825, 2023a.
Zhengbao Jiang, Frank F. Xu, Luyu Gao, Zhiqing Sun, Qian Liu, Jane Dwivedi-Yis Yu, Yiming Yang, Jamie Callan, and Graham Neubig. Active retrieval augmented generation. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pp. 7969-7992, 2023b.
Bowen Jin, Hansi Zeng, Zhenrui Yue, Jinsung Yoon, Sercan Arik, Dong Wang, Hamed Zamani, and Jiawei Han. Scarch-R1: Training LLMs to reason and leverage search engines with reinforcement learning. arXiv preprint ar Xrx: 099116, 2025.
Thorsten Joachims. Training linear SVMs in linear time. In Proceedings of the 12th ACM SIGKDD Inter- national Conference on Knowledge Discovery and Data Mining, KDD '06, pp. 217-226, New York, NY, USA, 2006. Association for Computing Machinery. doi: 241145/1150002.155000
Thorsten Joachims, Adith Swaminathan, and Tobias Schnabel. Unbiased Marning-to-rant with biased food- back. In Proceedings of the 10th ACM International Conference on Web Search and Dala Mining, pp. 781-789 2017.
Jeff Johnson, Matthijs Douze, and Hervé Jégou. Billion-scale similarity search with gpus. IEEE Transactions on Big Data, 7(3):535-55 547, 2019.
Armand Joulin, Piotr Bojanowski, Tomáš Mikolov, Hervé Jégou, and Édouard Grave. Loss in translation: Learning bilingual word mapping with retrieval criterion. In Proceedings of the 2018 Conference Empirical Methods in Natural Language Processing, pp. 2979 2984, 2018.
Yeong-Joon Ju, Ho-Joong Kim, and Seong- Seong-Whan Lee. MIRe: Enhancing multimodal queries representation via fusion-free modality interaction for multimodal retrieval. In Wanxiang Che, Joyce Nabende, Ekaterina Shutova, and Mohamund Taher Pilehvar (eds.), Findings of the Associalion for Compulational Linguistics: Vienna, Austria, July 2025. Association for Computational Linguistics. doi: ACL 2025, pp. www.fiinnng-ac..22.. URL attps://aclantholl/ 10.18653/v1
or2/2025. f finding--ac1
John Jumper, Richard Evans, Alexander Pritzel, Tim Green, Michael Figurnov, Olaf Ronneberger, Kathryn Tunyasuvunakool, Russ Bates, Augustin Zidek, Anna Potapenko, et al. Highly accurate protein structure prediction with AlphaFold. Nature, 596(7873):583-566)) 189,, 2021.
SeongKu Kang, Shivam Agarwal, Bowen Jin, Dongha Lee, Hwanjo Yu, and Jiawei Han. Improving retrieval in theme-specific applications using a corpus topical taxonomy. Proceedings of the ACM Web Conference 2024. 2024.
Jared Kaplan, Sam McCandlish, Tom Henighan, Tom B. Brown, Benjamin Chess, Rewon Child, Scott Gray, Alec Radford, Jeffrey Wu, and Dario Amodei. Scaling laws for neural language models. arXie preprint arXiv:2001.08.0 2020.
Andrej Karpathy, Armand Joulin, and Li Fei-Fei. Deep fragment embeddings for bidirectional image sentence mapping. Advances in neural information processing systems, 27, 2014.
Vladimir Karpukhin, Barlas Ozuz, Sewon Min, Patrick Lewis, Ledell Yu Wu, Sergey Edunov, Danqi Chen, and Wen tau Yih. Dense passage retricval for open-domain question answering. arXiv preprint arXiv:2004.0490 2020.
Angelos Katharopoulos, Apoorv Vyas, Nikolaos Pappas, and François Fleuret. Transformers are RNNs: Fast autoregressive transformers with linear attention. In International Conference on Machine Learning, pp. PMLR, 2020.
47

---

Published in Transactions on Machine Learning Research (02/2026)
Guolin Ke, Qi Meng, Thomas Finley, Taifeng Wang, Wei Chen, Weidong Ma, Qiwei Ye, and Tie-Yan Liu. Lightgbm: A highly efficient gradient boosting decision tree. Advances in neural informalion processing systems, 30, 2017.
Michael Kearns and Leslie Valiant. Cryptographic limitations on learning boolean formulae and finite tomata. Journal of the ACM (JACM), (11):677, 1994.
Urvashi Khandelwal, Omer Levy, Dan Jurafsky, Luke Zettlemoyer, and Mike Lewis. Generalization through memorization: Nearest neighbor language models. In International Conference on Learning Representa- lions; 2020. URL https: httpa/:////peeeeeee/ee 1
Omar Khatlab and Matei Zaharia. ColBERT: Efficient and effective passage search via contextualized late interaction over BERT, In Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 39-48, 2020.
Tushar Khot, Harsh Trivedi, Matthew Finlayson, Yao Fu, Kyle Richardson, Peter Clark, and Ashish Sabhar- wal. Decomposed prompting: A modular approach for solving complex tasks. In The 11th International Conference on Learning Representations, 2023. URL
nwt.forum madrmmiidiaa.. __n_ggQjzzaay
Julian Killingback, Hansi Zeng, and Hamed Zamani. Hypencoder: Hypernetworks for information retrieval. arXiv preprint ar Xiv:2502. 05364, 2025.
Seungyeon Kim, Ankit Singh Rawat, Manzil Zaheer, Sadeep Jayasumana, Veeranjaneyuhı Sadhanala, Wit- tawat Jitkrittum, Aditya Krishna Menon, Rob Fergus, and Sanjiv Kumar. EmbedDistilll: A geometric knowledge distillation for information retrieval. arXiv preprint ar arXh:2301.12000 2023.
Sungyeon Kim, Xinliang Zhu, Xiaofan Lin, Muhammet Bastan, Douglas Gray, and Suha Kwak. GENIUS: A generative framework for universal multimodal search. 2025 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 19659-19669, 2025.
Yoon Kim. Convolutional neural networks for sentence classification. In Alessandro Moschitti, Bo Pang, and Walter Daelemans (eds.), Proceedings of the 2014 Conference on Empirical Methods in Natural Lan- guage Processing (EMNLP), pp. 1746-1751, Doha, Qatar, October 2014. Association for Computational Linguistics. doi: 10.3115/v1/D14 1181. URL https: //apaa/ho/olll org/D14-1188)
Varsha Kishore, Chao Wan, Justin Lovelace, Yoav Artzi, and Kilian Q. Weinberger. Incdsi: Incrementally updatable document retrieval. In International Conference on Machine Learning, pp. 2023.
PMLR,
Fanheng Kong, Jingyuan Zhang, Yahui Liu, Hongzhi Zhang, Shi Feng, Xiaocui Yang, Daling Wang, Yu Tian, Qi Wang, Fuzheng Zhang, and Guorui Zhou. Modality curation: Building universal embeddings for advanced multimodal information retrieval. arXiv preprint arXix:2505. 19650, 2025.
Weize Kong, Jeffrey M. Dudek, Cheng Li, Mingyang Zhang, and Michael Bendersky. Sparscembed: Learning sparse lexical representations with contextual embeddings for retrieval. In Proceedings of the 46th Inler- national ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 2399 2403, 2023.
Donald H. Kraft and Duncan A. Buell. Fuzzy sets and generalized boolean retrieval systems. International journal of man-machine studies, 19(1):45 56, 1983.
Tanishq Kumar, Zachary Ankner, Benjamin F. Spector, Blake Bordelon, Niklas Muennighoff, Manshoej Paul, Cengiz Pehlevan, Christopher Ré, and Aditi Raghunathan. Scaling laws for precision. orXiv preprint 2xXi::2111.03390 2024.
Woosuk Kwon, Zhuohan Li, Siyuan Zhuang, Ying Sheng, Lianmin Zheng, Cody Hao Yu, Joseph Gonza- lez, Hao Zhang, and Ton Stoica. Efficient memory management for large language model serving with pagedattention. In Proceedings of the 29th Symposium on Operating Systems Principles, pp. 611-626, 2023.
48

---

Omer Levy, Yoav Goldberg, and Ido Dagan. Improving distributional similarity with lessons learned from word embeddings. Transactions of the Association for Computational Linguistics, 3:211-225, 2015. doi: http://aaaatooooooor///11//----/ Published in Transactions on Machine Learning Research (02/2026)
Zhenzhong Lan, Mingda Chen, Sebastian Goodman, Kevin Gimpel, Piyush Sharma, and Radu Soricut. AL- BERT: A lite BERT for self-supervised learning of language representations. In International Conference on Learning Representations, 2020. URL http::///peaarrieeeeeeee///omaar htt.://e....ooorriiicceeeAAA///.
Quoc Le and Tomas Mikolov. Distributed representations of sentences and documents. Proceedings of the 31st International Conference on Machine Learning, PMLR, 32( -196, 22-24 Jun 2014. URL
https://proceed .tt..mllrrprssr wwt.ppesss//eeeeeeeeeeee html.
Yann LeCun, Bernhard Boser, John S. Denker, Donnie Henderson, Richard E. Howard, Wayne Hubbard, and Lawrence D. Jackel. Backpropagation applied to handwritten zip code recognition. Neurol compulation, 1(4):541-551: 1989.
Chankyu Lee, Rajarshi Roy, Mengyao Xu, Jonathan Raiman, Mohammad Shoeybi, Bryan Catanzaro, and Wei Ping. NV-embed: Improved techniques for training LLMs as generalist embedding models. In The 13th International Conference on Learning Represenlations, 2025. URL 1gsyLSaDRe.
forum?id=
Jinhyuk Lee, Zhuyun Dai, Xiaoqi Ren, Blair Chen, Daniel Cer, Jeremy R. Cole, Kai Hui, Michael Boratko, Rajvi Kapadia, Wen Ding, et al. Gocko: Versatile text emboddings distilled from large language models. arXiv preprint 2rXim:2433.2022 2024.
Kenton Lee, Ming- Wei Chang, and Kristina Toutanova. Latent retrieval for weakly supervised open domain question answering. arXiv preprint ar Xix:11906. 100:000000 2019.
10.1162/tacl_a 00134. URL
Mike Lewis, Yinhan Liu, Naman Goyal, Marjan Ghazvininejad, Abdelrahman Mohamed, Omer Levy, Veselin Stoyanov, and Luke Zettlemoyer. BART: Denoising scquennceto--sqquunn pro-training for natural language generation, translation, and comprehension. In Dan Jurafsky, Joyce Chai, Natalie Schluter, and Joel Tetreault (eds.), Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pp. 7871-7880, Online, July 2020a. Association for Computational Linguistics. doi: 10. 10.18653/v1/2020 acl-main. 703. URL https: haap::ss///cllahholoooooo org/2020 acl-main 703/.
Patrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich Küttler, Mike Lewis, Wen-tau Yih, Tim Rocktäschel, et al. Retrieval-augmenti generation for knowledge- intensive nlp tasks. Advances in neural information processing systems, 33:9459 9474, 2020b.
Canjia Li, Andrew Yates, Scan MacAvancy, Ben Hc, and Yingfci Sun. PARADE: Passage representation aggregation fordocument reranking. ACM Trunsuclions on Information Systems, 42(2), September 2023a. doi: 10.1145/36000 /360088..
Chaofan Li, Zheng Liu, Shitao Xiao, and Yingxia Shao. Making large language models better foundation for dense retrieval. arXio preprint arXiv:2312. 15503, 2023b.
Dongfang Li, Baotian Hu, and Qingcai Chen. Calibration meets explanation: A simple and effective approach for model confidence estimates. In Yoav Goldberg, Zornitsa Kozareva, and Yue Zhang (eds.), Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pp. 2775-2784, Abu Dhabi, United Arab Emirates, December 2022a. Association for Computational Linguistics. doi: 10.18653/v1/ 2022.emnly 178. URL https:
et00022.emnlpp. ennnp-main.n.... main. 178/.
Gen Li, Nan Duan, Yuejian Fang, Daxin Jiang, and Ming Zhou. Unicoder- -orooder--L::.. A universal encoder for vision and language by cross-modal pro-training. In AAAI Conference on Artificial Intelligence, 2019.
Hang Li. Learning for ranking aggregation. In Learning to Rank for Information Retrieval and Natural Language Processing, pp. 33 35. Springer, 2011.
49

---

Published in Transactions on Machine Learning Research (02/2026)
Junnan Li, Dongxu Li, Caiming Xiong, and Steven Hoi. BLIP: Bootstrapping language-image pre-training for unified vision-language understanding and generation. In International Conference on Machine Learning, pp. 12888 12900.. PMLR, 2022b.
Minghan Li, Sheng- Chich Lin, Xueguang Ma, and Jimmy Lin. SLIM: Sparsified late interaction for multi- vector retrieval with inverted indexes. In Proceedings of the 46th International ACM STGIR Conference on Research and Development in Information Retrieval, SIGIR '23, pp. 1954 1959, New York, NY, USA, 2023c. Association for Computing Machinery. doi:
Mufei Li, Siqi Miao, and Pan Li. Simple is effective: The roles of graphs and large language models in sawwledge-graph-basee retrieval-augmenntr generation. In The 13th International Conference on Learning Representations, 2025a. URL https: http:///pppe/eeeee/ee/ httt://orummiidJvaaZZooo0o
Sen Li, Fuyu Lv, Taiwei Jin, Guli Lin, Keping Yang, Xiaoyi Zeng, Xiao-Ming Wu, and Qianli Ma. Embedding- based product retrieval in taobao search. In Proceedings of the 27th ACM SIGKDD Conference on Knowl- edge Discovery & Data Mining, pp. 3181-3189, 2021.
Xiaoxi Li, Guanting Dong, Jiajie Jin, Yuyao Zhang, Yujia. Zhou, Yutao Zhu, Peitian Zhang, and Zhicheng Dou. Search-ol: Agentic search- eaaanc-e large reasoning models. arXiv preprint arXiv:2501.0536 2025b.
Xiaoxi Li, Jiajie Jin, Guanting Dong, Hongjin Qian, Yutao Zhu, Yongkang Wu, Ji-Rong Wen, and Zhicheng Dou. Webthinker: Empowering large reasoning models with deep research capability. arXiv preprinl arXiv:2504. 21776, 2025c.
Xiaoxi Li, Jiajie Jin, Yujia Zhou, Yuyao Zhang, Peitian Zhang, Yutao Zhu, and Zhicheng Dou. From matching to generation: A survey on generative information retrieval. ACM Transaclions Information Systems, 43(3), May 2025d. doi: 10.1145/3722552.
Yangning Li, Weizhi Zhang, Yuyao Yang, Wei-Chieh Huang, Yaozu Wu, Junyu Luo, Yuanchen Bei, Henry Peng Zou, Xiao Luo, Yusheng Zhao, et al. Towards agentic RAG with deep reasoning: A sur- vey of RAG-reasoning systems in LLMs. arXiv preprint arXiv:2507.09/77 2025e.
Zehan Li, Xin Zhang, Yanzhao Zhang, Dingkun Long, Pengjun Xie, and Meishan Zhang. Towards general text embeddings with mulli-stage contrastive learning. ar preprinl ar urXic:2308. 032281, 2023d.
Jintao Liang, Gang Su, Huifeng Lin, You Wu, Rui Zhao, and Ziyue Li. Reasoning RAG via system 1 system 2: A survey reasoning agentic retrieval-augmeente generation for industry challenges. ar preprint arXiv:2506. 10408, 2025.
Jimmy Lin. A proposed conceptual framework for representational approach to information retrieval. ACM SIGIR Forum, 55:1 29, 2021.
Jimmy Lin and Xueguang Ma. A few brief notes on DeepImpact, COIL, and a conceptual framework for information retrieval techniques. arXiv preprint XrXin:2006. 14807, 2021.
Jimmy Lin, Rodrigo Nogueira, and Andrew Yates. Pretrained transformers for text ranking: BERT and beyond. Springer Nature, 2022.
Juexin Lin, Sachin Yadav, Feng Liu, Nicholas Rossi, Praveen Reddy Suram, Satya Chembolu, Prijith Chan- dran, Hrushikesh Mohapatra, Tony Lee, Alessandro Magnani, and Ciya Liao. Enhancing relevance of embedding. based retrieval at Walmart. Proceedings of the 33rd ACM International Conference on Infor- mation and Knowledge Management, 2024.
Minhua Lin, Zongyu Wu, Zhichao Xu, Hui Liu, Xianfeng Tang, Qi Hc, Charu Aggarwal, Hui Liu, Xi- ang Zhang, and Suhang Wang. A comprehensive survey on reinforcement Cearning-based agentic search: Foundations, roles, optimizations, evaluations, and applications. Xr preprint arXi:22100.100 16724,, 2025.
50

---

Published in Transactions on Machine Learning Research (02/2026)
Robert Litschko, Ivan Vulié, and Goran Glavaš. Parameter-effiiiet neural reranking for cross-lingual and multilingual retrieval. In Nicoletta Calzolari, Chu-Ren Huang, Hansaem Kim, James Pustejovsky, Leo Wanner, Key-Sun Choi, Pum-Mo Ryu, Hsin-Hsi Chen, Lucia Donatelli, Heng Ji, Sadao Kurohashi, Pa- trizia Paggio, Nianwen Xue, Scokhwan Kim, Younggyun Hahm, Zhong He, Tony Kyungil Lee, Enrico Santus, Francis Bond, and Seung- Hoon Na (eds.), Proceedings of the 29th International Conference on Computational Linguistics, pp. 1071-1082, Gycongju, Republic of Korca, October 2022. International Committee on Computational Linguisties. URL httpp://accaai http///aaaaaaoooyocco.o..
Aixin Liu, Bei Feng, Bing Xue, Bingxuan Wang, Bochao Wu, Chengda Lu, Chenggang Zhao, Chengqi Deng, Chenyu Zhang, Chong Ruan, et al. DeepSeek-v3 technical report. arXiv preprinl ar arXiv:24 12, 19437, 2024a.
Hao Liu, Zhengren Wang, Xi Chen, Zhiyu Li, Feiyu Xiong, Qinhan Yu, and Wentao Zhang. HopRAG: Multi- hop reasoning for logic-aware retzeeeal-uggmennted generation. In Wanxiang Che, Joyce Nabende, Ekaterina Shutova, and Mohammad Taher Pilehvar (eds.), Findings of the Association for Computational Linguistios: ACL 2025, pp. 1897 1913, Vienna, Austria, July 2025a. Association for Computational Linguistics. doi: 10.18653/v1/2000 http:///////222....nnnnn-aaee acl.99.. URL https://aclanthone htwpe/aatttoooooooooo//ooooaaeee aaw.ggs-acl.977 97/.
org/2025. we.005.finninne.
Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. Visual instruction tuning. XrXiv preprinl
WwXiv:2304.08450 2023.
Jingzhou Liu, Wei-Cheng Chang, Yuexin Wu, and Yiming Yang. Deep learning for extreme multi-label text classification. In Proceedings of the 40th International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 115 124, 2017.
Pei Liu, Xin Liu, Ruoyu Yao, Junming Liu, Siyuan Meng, Ding Wang, and Jun Ma. Hm-RAG: Hierarchi- cal multi-agent multimodal retrieval augmented generation. Proceedings of the 33rd ACM International Conference on Multimedia, 2025b.
Qi Liu, Bo Wang, Nan Wang, and Jiaxin Mao. Leveraging passage embeddings for efficient listwise reranking with large language models. In Proceedings of the ACM on Web Conference 2025, 2024b. doi: 10.1145/ 3696410.3714554
Song Liu, Haoqi Fan, Shengsheng Qian, Yiru Chen, Wenkui Ding, and Zhongyuan Wang. HiT: Hierarchical transformer with momentum contrast for video-text retrieval. 2021 IEEE/CVE International Conference on Computer Vision (ICCV), pp. 11895-11905, 2021.
Tie Yan Liu. Learning to rank for information retrieval. Foundations and Trends in Information Retrieval, 3(3):225-33) March 2009. doi: 10.116/1550000016
Yinhan Liu, Myle Out, Naman Goyal, Jinglei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. RoBERTa: A robustly optimized BERT pretraining approach. arviv preprint ar Xiv: 11692, 2019.
Yu-An Liu, Ruqing Zhang, Jiafeng Guo, and Maarten de Rijke. Robust information retrieval. In Proceedings of the 18th ACM International Conference on Web Search and Dala Mining, pp. 1008-1011, 2025c.
Zhenghao Liu, Chenyan Xiong, Maosong Sun, and Zhiyuan Liu. Entity-duet neural ranking: Understanding the role of knowledge graph semantics in neural information retrieval. Tn Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 2395-2405, 2018.
Lajanugen Logeswaran, Ming- Wei Chang, Kenton Lee, Kristina Toutanova, Jacob Devlin, and Honglak Lee. Zero-shot entity linking by reading entity descriptions. In Anna Korhonen, David Traum, and Lluis Mårquez (eds.), Proceedings of the 57th Annual Meeting of the Association for Computational Länguistics, pp. 3449 3460, Florence, Italy, July 2019. Association for Computational Linguistics. doi: 10.18653/v /vll P19- 1335. URL https:
David G. Lowe. Object recognition from local scale-invariant features. In Proceedings of the 7th IEEE International Conference on Computer Vision, volume 2, pp. 1150-1157, 1157, 1999.
51

---

Published in Transactions on Machine Learning Research (02/2026)
Zhengdong Lu and Hang Li. A deep architecture for matching short texts. Advances in neural information processing systems, 26, 2013.
Yi Luan, Jacob Fisenstein, Kristina Toutanova, and Michael Collins. Sparse, dense, and attentional rep- resentations for text retrieval. Trunsactions of the Association for Computational Linguistics, 9:329 345, 2021.
Guangyuan Ma, Xing Wu, Zijia Lin, and Songlin Hu. Drop your decoder: Pre-training with bag-of-wort prediction for dense passage retrieval. In Proceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 1818 1827, 2024a.
Xueguang Ma, Xinyu Zhang, Ronak Pradeep, and Jimmy Lin. Zero-shot listwise document reranking with a large language model. arXiv preprint aixi 2805.02156 2023.
Xueguang Ma, Liang Wang, Nan Yang, Furn Wei, and Jimmy Lin. Fine-tuning LLaMA for multi-stage text retrieval. In Proceedings of the 4 47th International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 2421-22 2421-2425, 2024b.
Sean MacAvaney, Luca Soldaini, and Nazli Goharian. Teaching a new dog old tricks: Resurrecting multilin- gual retricval using zero-shot learning. Advances in Information Retrieval, 12036:246 254, 2019a.
Scan MacAvancy, Andrew Yates, Arman Cohan, and Nazli Goharian. CEDR: Contextualized embeddings for document ranking. In Proceedings of the 42nd International ACM SIGIR Conference on Research and Development in Information Retricual, pp. 1101-1104, 2019b.
Sean MacAvancy, Franco Maria Nardini, R. Perego, Nicola Tonellotto, Nazli Goharian, and Ophir Frieder.
Efficient document re-ranking for transformers by precomputing term representations. Proceedings of the 43rd International ACM SIGIR Conference on Reseorch and Development in Information Retrieval, 2020.
Sean MacAvaney, Sergey Feldman, Nazli Goharian, Doug Downey, and Arman Cohan. ABNIRML: Analyzing the behavior of neural IR models. Transactions of the Association for Computational Linguisties, 10:224- 239, 2022. doi: 10.1162/tacl a 00457. URL
wwwp:/yyy.rgg.2..... tacl-1.133
David J. C. MacKay and Linda C. Bauman Peto. A hierarchical Dirichlet language model. Natural language engineering, 1(3):289-300 308, 1995.
Yu A. Malkov and Dmitry A. Yashunin. Efficient and robust approximate nearest neighbor search using hierarchical navigable small world graphs. IEEE transactions pattern analysis and machine intelligence, 42(4))824 836, 2018.
Alex Mallen, Akari Asai, Victor Zhong, Rajarshi Das, Daniel Khashabi, and Hannaneh Hajishirzi. When not to trust language models: Investigating effectiveness of parametric and non porameemet memories. In Anna Rogers, Jordan Boyd-Graber, and Naoaki Okazaki (eds.), Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 9802-9822 Toronto, Canada, July 2023. Association for Computational Linguistics. doi: 10.18653 vl /2ll222222cllooong// attps://aclanthen
URL
org/2023 sel-long 546/.
Antonio Mallia, Omar Khattab, Torsten Suel, and Nicola Tonellotto. Learning passage impacts for inverted indexes. In Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 1723 1727, 2021.
Gary Marchionini. Exploratory search: from finding to understanding, Communicaatonss of the ACM, 49(4): 41-46, April 2006. doi:
Sanket Vaibhav Mehta, Jai Gupta, Yi Tay, Mostafa Dehghani, Vinh Q Tran, Jinfeng Rao, Mare Najork, Emma Strubell, and Donald Metzler. +SI++: Updating transformer memory with new documents. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pp. 8198-8213, 2023.
52

---

Published in Transactions on Machine Learning Research (02/2026)
Lang Mei, Siyu Mo, Zhihan Yang, and Chong Chen. A survey of Multimodal Retrieval-Angmeented Genera- tion. arXiv preprint ar arXiv:2504.08748, 2025.
Tomas Mikolov, Kai Chen, Greg Corrado, and Jeffrey Dean. Efficient estimation of word representations in vector space, arXiv preprint arX 2013.
David R. H. Miller, Tim Leek, and Richard M. Schwartz. A hidden markov model information retrieval system. In Proceedings of the 22nd Annual International ACM SIGIR Conference on Research and De- velopment in Information Retrieval, pp. 214-221, 1999.
Matthias Minderer, Josip Djolonga, Rob Romijnders, Frances Ann Hubis, Xiaohua Zhai, Neil Houlsby, Dustin Tran, and Mario Lucic. Revisiting the calibration of modern neural networks. arXiv preprint 00X.:92006.07908, 2021.
Bhaskar Mitra, Eric Nalisnick, Nick Craswell, and Rich Caruana. A dual embedding space model for document ranking. arXiv preprinl or Xiv: 01 137, 2016.
Bhaskar Mitra, Fernando Diaz, and Nick Craswell. Learning to match using local and distributed represen- tations of text for web search. In Proceedings of the 26th International Conference on World Wide Web, WWW '17, pp. 1291-1299, Republic and Canton of Geneva, CHE, 2017. International World Wide Web Conferences Steering Committee. doi: 10. 10.1145/300381223333333
Bhaskar Mitra, Nick Craswell, et al. An introduction to neural information retrieval. Foundations and Trends in Information Retrieval, 13(1):1-126, 2018.
Bhaskar Mitra, Sebastian Hofstätter, Hamed Zamani, and Nick Craswell. Improving transformer-kernel ranking model using conformer and query term independence. In Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 1697-1702, 2021.
Fengran Mo, Kelong Mao, Yutao Zhu, Yihong Wu, Kaiyu Huang, and Jian- Nie. ConvGQR: Generative query reformulation for conversational search. In Proceedings of the 61st Arauall Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 4998 5012, 2023.
Fengran Mo, Abbas Ghaddar, Kelong Mao, Mehdi Rezagholizadeh, Boxing Chen, Qun Liu, and Jian- Yun Nie. CHIQ: Contextual history enhancement for improving query rewriting in conversational search, arXiv preprint ar Xiv:24406.. 05013, 2024a.
Fengran Mo, Kelong Mao, Ziliang Zhao, Hongjin Qian, Haonan Chen, Yiruo Cheng, Xiaoxi Li, Yutao Zhu, Zhicheng Dou, and Jian-Yur Yun Nie. A survey of conversational search. arXiv preprint arXiv:24100 10. 15576, 2024b.
Fengran Mo, Bole Yi, Kelong Mao, Chen Qu, Kaiyu Huang, and Jian- Yun Nie. ConvSDG: Session data generation for conversational search. In Companion Proceedings of the ACM on Web Conference 2024, pp. 1634-1642, 2024c.
Turii Mokrii, Lconid Boytsov, and Pavel Braslavski. A systematic evaluation of transfer lcarning and pscudo- labeling with BERT-based ranking models. In Proceedings of the 44th International ACM SIGIR Confer- ence on Research and Development in Information Retrieval, SIGIR '21, pp. 2081-2085, New York, NY, USA, 2021. Association for Computing Machinery. doi: 10.1145534400 05555334443553-445 34633093.
Niklas Mucnnighoff. SGPT: GPT sentence embeddings for semantic scarch. arXiv preprint or 4720:2200.008904 2022.
Niklas Muennighoff, Nouamane Tazi, Loic Magne, and Nils Reimers. MTEB: Massive text embedding benchmark. In Andreas Vlachos and Isabelle Augenstein (eds.), Proceedings of the 17th Conference of the European Chapter of the Association for Computational Länguistics, pp. 2014-2037, Dubrovnik, Croatia, May 2023. Association for Computational Linguistics. doi: 10.18653/vl 1222222..222/2 main.145 URL https: / //aclanthology www.eeo..rr/222..... macl-main.... 148/.
53

---

Published in Transactions on Machine Learning Research (02/2026)
Niklas Muennighoff, Hongjin Su, Liang Wang, Nan Yang, Furu Wei, Tao Yu, Amanpreet Singh, and Douwe Kiela. Generative representational instruction tuning. In The 13th International Conference on Learning Representations, 2025. URL https://openreviews net http:r/www.....ooooodddoooosssa.
Relichir Nakano, Jacob Hilton, Suchir Balaji, Jeff Wu, Long Ouyang, Christina Kim, Christopher Hesse, Shantanu Jain, Vineet Kosaraju, William Saunders, et al. WebGPT: Browser-assisted question-answering with human leedback. arXiv prepriul or arXiv:21122 12..122.0333. 2021.
Eric Nalismick, Bhaskar Mitra, Nick Craswell, and Rich Caruana. Improving document ranking with dual word embeddings. In Proceedings of the 25th International Conference Companion on World Wide Web, WWW '16 Companion, pp. 53-84, Republic and Canton of Geneva, CHE, 2016. International World Wide Web Conferences Steering Committee. doi:
28889361.
Pandu Nayak. Undorstanding searches better than ever before. https://blog woggle//ooducts//eseeeeeeeee
search- -anguugee understanding-eeettet bert/, 2019.
Arvind Neelakantan, Tao Xu, Raul Puri, Alec Radford, Jesse Michael Han, Jerry Tworek, Qiming Yuan, Nikolas Tezak, Jong Wook Kim, Chris Hallacy, et al. Text and code embeddings by contrastive pre-training. arXiv preprint ar 2r1: 10005, 2022.
Thong Nguyen, Sean MacAvaney, and Andrew Yates. A unified framework for learned sparse retrieval. In European Conference on Information Retrieval, pp. 101 116. Springer, 2023.
Jianmo Ni, Gustavo Hernandez Abrego, Noah Constant, Ji Ma, Keith Hall, Daniel Cer, and Yinfei Yang. Sentenco-T5: Scalable sentence encoders from pro-trained text-to-text models. In Smaranda Muresan, Preslav Nakov, and Aline Villavicencio (eds.), Findings of the Association for Compulational Linguistics: ACL 2022, pp. 1864-18744 Dublin, Ireland, May 2022a. Association for Computational Linguistics. doi: 10.18653/V /v1 aww.....ninnns-al.... 146. URL fttps://accanthollo.
org/2022. findings-aci acc.dnnnssa...16/..
Jianmo Ni, Chen Qu, Jing Lu, Zhuyun Dai, Gustavo Hernandez Abrego, Ji Ma, Vincent Zhao, Yi Luan, Keith Hall, Ming- Wei Chang, and Yinfei Yang. Large dual encoders are generalizable retrievers. In Yoav Goldberg, Zornitsa Kozareva, and Yue Zhang (eds.), Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pp. 9844-985,, Abu Dhabi, United Arab Emirates, December 2022b. Association for Computational Linguistics. doi: / //aclanthology. wrtp/022.2222222... ammlp-main.ce 669/.
URL https:
Jian-Yun Nic. Cross-language information retrieval. Morgan & Claypool Publishers, 2010.
Priyank Nigam, Yiwei Song, Vijai Mohan, Vihan Lakshman, Weitian Ding, Ankit Shingavi, Choon Hui Too, Hao Gu, and Bing Yin. Semantic product search. Proceedings of the 25th ACM SIGKDD International Conference Knowledge Discovery & Data Mining, 2019.
Rodrigo Nogueira, Jimmy Lin, and A. Epistemic. From doc2query to doc TTTTTTTQur.. Online preprind, 6(2), 2019a.
Rodrigo Nogueira, Wei Yang, Kyunghyun Cho, and Jimmy Lin. Multi-stage document ranking with BERT. arXiv preprint 19100.41.1444 2019b.
Rodrigo Nogucira, Zhiying Jiang, and Jimmy Lin. Document ranking with pretrained sequenco-to--sqquenc model. arXiv preprint 2XXi:2003.066133 2020.
Cicero Nogucira dos Santos, Xiaofei Ma, Ramesh Nallapati, Zhiheng Huang, and Bing Xiang. Beyond [CLS] through ranking by generation. In Bonnie Webber, Trevor Cohn, Yulan He, and Yang Liu (eds.), Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pp. 1722 1727, Online, November 2020. Association for Computational Linguistics. doi: 10.18653/v1/200 emnlp- main. URL https://aclanthen wwwp//00....0000oo./ mmmlp-main... 134/.
Zach Nussbaum and Brandon Duderstadt. Training sparse mixture of experts text embedding models, arXiv preprint 27X.v22502.07972 2025.
54

---

Published in Transactions on Machine Learning Research (02/2026)
Zach Nussbaum, John Xavier Morris, Andriy Mulyar, and Brandon Duderstadt. Nomic Embed: Training reproducible long context text embedder. Transactions on Machine Learning Research, 2025. URL https:///ppnrreview. http://nnnnttoommmmddtaaaaaaoaa
Douglas Oard, Carol Peters, Miguel Ruiz, Robert Frederking, Judith Klavans, and Paraic Sheridan. Mul- tilingual Information Discovery and AccesS (MIDAS), D-Lib Magazine, 5(10), 1999. doi: 10.1045/ october99-oard.
Douglas W. Oard and Bonnie J. Dorr. A survey of multilingual text retrieval. Technical Report UMTACS- TR-96-19 CS-TR-3615, University of Maryland Institute for Advanced Computer Studies, 1996.
Douglas W. Oard and Bonnic J. Dorr. Evaluating cross-language text filtering effectivencss. In Cross- Language Information Retrieval, pp. 151-161. Springer, 1998.
Hanseok Oh, Hyunji Lee, Seonghyeon Ye, Haebin Shin, Hansol Jang, Changwook Jun, and Minjoon Seo. INSTRUCTIR: A benchmark for instruction following of information retrieval models. orXiv preprint arXiv:2402.14388 2024.
Bruno Oliveira and Carla Teixeira Lopes. From 10 blue links pages to full search engine results pages analysis of the temporal evolution of SERP features. In Proceedings of the 2023 Conference on Human Information Interaction and Retrieval, pp. 338-345. ACM, March 2023. doi: 10.1145/3576840.33
Kezban Dilek Onal, Ye Zhang, Ismail Sengor Altingovde, Md Mustafizur Rahman, Pinar Karagoz, Alex Braylan, Brandon Dang, Heng-Lu Chang, Henna Kim, Quinten McNamara, et al. Neural information retrieval: At the end of the early years. Information Retrieval Journal, 21:111-182, 2018.
Aaron van den Oord, Yazhe Li, and Oriol Vinyals. Representation learning with contrastive predictive coding. arXiv preprint 0rXi::1877.09745 2018.
OpenAl. Introducing ChatGPT. https: httpg///pppeaa..ccmmmnnee///tttt 2022.
OpenAl. Introducing deep research. http::///gaaaa..co//////////rrrr
OpenAL. Introducing ChatGPT search. 2024.
2025.
OpenAL, Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Tlge Akkaya, et al. GPT-4 technical report. arXiv preprint arXiv:2303.08714 2023.
OpenAI, Aaron Hurst, Adam Lerer, Adam P. Goucher, Adam Perelman, Aditya Ramesh, et al. GPT-4o system card. arXiv preprint arXiv:2410.21276, 2024.
Hamid Palangi, Li Deng, Yelong Shen, Jianfeng Gao, Xiaodong He, Jianshu Chen, Xinying Song, and Rabab Ward. Deep sentence embedding using long short-term memory networks: analysis and application to information retrioval. IEEE Transactions on Audio, Speech and Language Processing, 24(4):694-707, 2016. doi: TT0T0//TTS...01.225555
Liang Pang, Yanyan Lan, Jiafeng Guo, Jun Xu, Shengxian Wan, and Xueqi Cheng. Text matching as image recognition. In Proceedings of the 30th AAAI Conference Artificial Intelligence, AAAT16, pp. 2793 2799. AAAI Press, 2016.
Liang Pang, Jun Xu, Qingyao Ai, Yanyan Lan, Xueqi Cheng, and Ji-Rong Wen. SetRank: Learning a permutation-invarian ranking model for information retrieval. In Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 499-508, 2020.
Eunhwan Park, Sung-Min Lee, Dearyong Sco, Sconhoon Kim, Inho Kang, and Scung- Na. RINK: Rchder-in evidence reranker for table-and-text open domain question answering authors. In Pro- ceedings of the AAAT Conference Artificial Intelligence, volume 37, pp.
2023.
55

---

Published in Transactions on Machine Learning Research (02/2026)
Shishir G. Patil, Huanzhi Mao, Charlie Cheng-Jie Ji, Fanjia Yan, Vishmu Suresh, Ion Stoica, and Joseph E. Gonzalez. The berkeley function calling leaderboard (BFCT)): From tool use to agentic evaluation of large language models. In Advances in Neural Information Processing Systems, 2024a.
Shishir G. Patil, Tianjun Zhang, Xin Wang, and Joseph E. Gonzalez. Gorilla: Large Language Model connected with massive APIs. In The 38th Annual Conference on Neural Information Processing Systems, http:/orrmmiddddtttte
2024b. URL https://
Bo Peng, Eric Alcaide, Quentin Anthony, Alon Albalak, Samuel Arcadinho, Stella Biderman, Huanqi Cao, Xin Cheng, Michael Chung, Leon Derczynski, Xingjian Du, Matteo Grella, Kranthi Gv, Xuzheng He, Haowen Hou, Przemyslaw Kazienko, Jan Kocon, Jiaming Kong, Bartlomiej Koptyra, Hayden Lau, Jiaju Lin, Krishna Sri Ipsit Mantri, Ferdinand Mom, Atsushi Saito, Guangyu Song, Xiangru Tang, Johan Wind, Stanistaw Woćniak, Zhenyuan Zhang, Qinghua Zhou,, Jian Zhu, and Rui-Jie Zhu. RWKV; Reinventing RNNs for the transformer era. In Houda Bouamor, Juan Pino, and Kalika Bali (eds.), Findings of the Association for Computational Linguistics: EMNLP 2023, pp. 14048 14077, Singapore, 2023. Association for Computational Linguistics. doi: 10.18653/v1 findiiinininggnnnl...... URL https: haapa////llaa org/2023. f findingsn-eiiinn w.w.e..mnnl........
Bo Peng, Daniel Goldstein, Quentin Anthony, Alon Albalak, Fric Alcaide, Stella Biderman, Eugene Cheah, Xingjian Du, Teddy Ferdinan, Haowen Hou, et al. Eagle and finch: RWKV with matrix-valued states and dynamic recurrence. arXiv preprinl Nxc:2444.00889 2024a.
Wenjun Peng, Guiyang Li, Yue Jiang, Zilong Wang, Dan Ou, Xiaoyi Zeng, Derong Xu, Tong Xu, and Enhong Chen. Large language model based long-tail query rewriting in Taobao search. In Companion Proceedings of the ACM Web Conference 2024, WWW '24, pp. 20-28, New York, NY, USA, 2024b. Association for Computing Machinery. doi: 10. 01145/5559335.3666
Gustavo Penha and Claudia Hauff. On the calibration and uncertainty of neural learning to rank models for conversational search. In Paola Merlo, Jorg Tiedemann, and Reut Tsarfaty (eds.), Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume, pp. 160 170, Online, April 2021. Association for Computational Linguistics. doi: 10.18653/v1, w10////1122221../ meelleeealmmiin 12. URL https://aclantaai
org/2021. eacl- 12/.
Jeffrey Pennington, Richard Socher, and Christopher Manning. GloVe: Global vectors for word repre- sentation. In Alessandro Moschitti, Bo Pang, and Walter Daelemans (ods.), Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), pp. 1532-1543. Doha, Qatar, October 2014. Association for Computational Linguistics. doi: 10.3115/11/D14-112.. URL https://aclantthologg ogg.or/y/////111///
Fabio Petroni, Aleksandra Piktus, Angela Fan, Patrick Lewis, Majid Yazdani, Nicola De Cao, James Thorne, Yacine Jernite, Vladimir Karpukhin, Jean Maillard, et al. KILT: a beuchmark for knowledge intensive language tasks. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pp. 2523 2544, 2021.
Jonas PfeifTer, Ivan Vulić, Iryna Gurevych, and Sebastian ian Ruder. MAD-X: An Adapter- Baade Framework for Multi-Task, Lingual Transfer. In Bonnie Webber, Trevor Cohn, Yulan He, and Yang Liu (eds.); Proceedings of the 2020 Conference on Empérical Methods in Natural Language Processing (EMNLP), pp. 7654 7673, Online, November 2020. Association for Compulational Linguistics. doi: 10.18653/v1/200 emnlp-main.617. URL https: ht/////aaaaaahhhcec htwphhhooogooooo/2/000 org/2020....... emnlp- main..677
Jonas Pfeiffer, Aishwarya Kamath, Andreas Rüikklé, Kyunghyun Cho, and Iryna Gurevych. AdapterFusion: Non-destructtive task composition for transfer learning. In Paola Merlo, Jorg Tiedemann, and Reut Tsarfaly (eds.), Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume, pp. 487-503, Online, April 2021. Association for Computational Linguistics. doi: 10. 10.18653/v1 1122221 1tt./22eeacclaaaiiiee eacl-main.39. URL https: tttps://aclanthine //tp/aaaaaaatoooyo....... org/2021. eacl-main .39/
Jay M. Ponte and W. Bruce Croft. A language modeling approach to information retrieval. In Proceedings of the 21st Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 275 281, 1998.
56

---

Published in Transactions on Machine Learning Research (02/2026)
Jeff Pool, Abhishek Sawarkar, and Jay Rodge. Accelerating inference with sparsity using the NVIDIA Ampere architecture and NVIDIA TensorRT. NVIDIA Technical Blog, July 2021. URL. https://developer: andeere-rrd-ttsorr//.
nvidia. htt.daa.com/boog/ccceeeeetnnggg htwp://gcccclloocommmmmeeeeeeeee stta--ttttsststyusssiiiae
Jacob Portes, Alexander Trott, Sam Havens, Daniel King, Abhinav Venigalla, Moin Nadeem, Nikhil Sardana, Daya Khudia, and Jonathan Frankle. MosaicBERT: A bidirectional encoder optimized for fast pretraining. Advances in Newral Information Processing Systems, 36:3106 3130, 2023.
Jes'us Andr'es Portillo-Quintero, José carlos Ortiz- Btaii-Bayliss, and Hugo Terashima-Mar A straightforward framework for video retrieval using CLIP. In Mericon Conference on Paltern Recognation, 2021.
Yashoteja Prabhu and Manik Varma. MastXML: A fast, accurate and stable tree-classifier for extreme multi-label learning. In Proceedings of the 20th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 263-272, 2014.
Yashoteja Prabhu, Anil Kag, Shrutendra Harsola, Rahul Agrawal, and Manik Varma. Parabel: Partitioned label trees for extreme classification with application to dynamic search advertising, In Proceedings of the 2018 World Wide Web Conference, pp. 993-1002, 2018.
Ronak Pradeep, Rodrigo Nogueira, and Jimmy Lin. The Expando-Mono-DDe design pattern for text ranking with pretrained sequence-to-soquunee models. arXiv preprint orXiv:2101 (X1::2001.05667, 2021.
Ronak Pradeep, Kai Hui, Jai Gupta, Adam Lelkes, Honglei Zhuang, Jimmy Lin, Donald Metzler, and Vinh Tran. How does generative retrieval scale to millions of passages? In Houda Bouamor, Juan Pino, and Kalika Bali (ods.), Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pp. 1305 1321, Singapore, December 2023a. Association for Compulational Linguistics. doi: (0.18653/v1/2000 eeeee-.mmllppaee main.83. URL https: tttp:///aaaaatthhco wwrgee...ggg/e.... org/2023. munlp-main.89
Ronak Pradeep, Sahel Sharifymoghaddamm, and Jimmy Lin. RankVicuna: Zero-shot listwise document reranking with open-source large language models. arXiv preprint ar Xiv:2909.15088, 2023b.
Ronak Pradeep, Sahel Sharifymoghadttam, and Jimmy Lin. RankZephyr: Effective and robust zero-shot listwise reranking is a breeze! arXiv preprint arxiv: 2312 2312.02724, 2023c.
Jiarui Qin, Jiachen Zhu, Bo Chen, Zhirong Liu, Weiwen Liu, Ruiming Tang, Rui Zhang, Yong Yu, and Weinan Zhang, RankPlow: Joint optimization of multi-stage cascade ranking systems as flows. Proceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval, 2022.
Tao Qin and Tie- Yan Liu. Introducing LETOR 4.0 datasets. arXiv preprint wxXiv:336.25999 2013.
Tao Qin, Tie-Ya Liu, and Hang Li. A general approximation framework for direct optimization of informa- tion retrieval measures. Information retrieval, 13:375 3975, 2010.
Yujia Qin, Shihao Liang Yining Ye, Kunlun Zhu, Lan Yan, Yaxi Lu, Yankai Lin, Xin Cong, Xiangru Tang, Bill Qian, Sihan Zhao, Lauren Hong. Runchu Tian, Ruobing Xie, Jie Zhou, Mark Gerstein, dahai li Zhiyuan Liu, and Maosong Sun. TooILLM: Facilitating large language models to master 16000+ real- world APIs. In The 12th International Conference on Learning Representations, 2024a. URL https: //openreview net/ htt.:/orruumiddddggg000....
Zhen Qin, Le Yan, Honglei Zhuang, Yi Tay, Rama Kumar Pasumarthi, Xuanhui Wang, Michael Bender- sky, and Mare Najork. Are neural rankers still outperformed by gradient boosted decision trees? In International Conference on Learning Representations, 2021.
Zhen Qin, Rolf Jagerman, Kai Hui, Honglei Zhuang, Junru Wu, Le Yan, Jiaming Shen, Tianqi Liu, Jialu Liu, Donald Metzler, Xuanhui Wang, and Michael Bendersky. Large language models are elfective text rankers with pairwise ranking prompting. In Kevin Duh, Helena Gomez, and Steven Bethard (eds.), Findings of the Association for Computational Linguistics: NAACL 2024, pp. 1504-1518, Mexico City, Mexico, June 2024b. Association for Computational Linguistics. doi: 10.18653/v1 v2/22/4.1222///// htwp://......in.s...m.eea lttps://accanthee www.thoooy..rrr/22222222 findings-naacl.999 .97/
findings-naacl.9r na7c..97. URL
57

---

Published in Transactions on Machine Learning Research (02/2026)
Zhen Qin, Songlin Yang, and Yiran Zhong. Hierarchically gated recurrent neural network for sequence modeling. Advances in Neural Information Processing Systems, 36, 2024c.
Yingqi Qu, Yuchen Ding, Jing Liu, Kai Liu, Ruiyang Ren, Xin Zhao, Daxiang Dong, Hua Wu, and Haifeng Wang. RocketQA: An optimized training approach to dense passage retrieval for open-domain question answering. In North American Chapter of the Association for Computational Linguistics, 2020.
Qwen Team. QwQ-32B: Embracing the power of reinforcement learning. https://qwenln, github. io/ (togg/quq-2b///// March 2025.
Tadeusz Radecki. Fuzzy set thcoretical approach to document retrieval. Information Processing 8 Manage- menl, 15(5):247-258 1979.
Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, and Ilya Sutskever. Language models are unsupervised multitask learners. https://cdn www.ai.enninc.... wonm/eetter-aagugeeeeeeeeee language-models: .dd, 2019.
Alec Radford, Jong Wook Kim, Chris Hallacy Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and Πly Sutskever. Learning transferable visual models from natural language supervision. In International Conference on Machine Learning, 2021.
Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J. Liu. Exploring the limits of transfer learning with unified text-to-text transformer. Journal of Machine Learning Research, 21(14)):1-67, 2020. URL httpttp::// http://jmlr. http://iinrrrrrppprrrseeeee http/pppp//v//22002/2/ html.
Razich Rahimi, Youngwoo Kim, Hamed Zamani, and James Allan. Explaining documents' relevance search queries. arXiv preprint Xrx.v:2111.000 013114, 2021.
Shashank Rajput, Nikhil Mehta, Anima Singh, Raghunandan H. Keshavan, Trung Hieu Vu, Lukasz Heldt, Lichan Hong, Yi Tay, Vinh Q Tran, Jonah Samost, Maciej Kula, Ed H. Chi, and Maheswaran Sathi- amoorthy. Recommender systems with generative retricval. arXiv preprint ar Xrxx:220000000 050655, 2023.
Ori Ram, Liat Bezalel, Adi Zicher, Yonatan Belinkov, Jonathan Berant, and Amir Globerson. What are you token about? Dense retrieval as distributions over the vocabulary. In Anna Rogers, Jor- dan Boyd-Graber, and Naoaki Okazaki (eds.), Proceedings of the 61st Annual Meeting of the Associ- ation for Computational Linguistics (Volume 1: Long Papers), pp. 2481-2498 Toronto, Canada, July 2023a. Association for Computational Linguistics. doi: 10.18653/v1/2022 / //aclanthology- hwwphhhoogg.ooog//0000 www.ooo..........oo http:/222.accl1ooom. long. 140/.
aalp:///o/22....lllooooggg long-onn..14 140. URL https:
Ori Ram, Yoav Levine, Itay Dahmedigos, Dor Muhlgay, Amnon Shashua, Kevin Leyton-Brown, Brown, and Yoav Shoham. In-context retrrivallangmmentel language models. Transactions of the Association for Compula- tional Linguistics, 11:1316-1331, 2023b.
Viresh Ranjan, Nikhil Rasiwasia, and C. V. Jawahar. Multi-label cross-modal retrieval. In Proceedings of the IEEE International Conference on Computer Vision, pp. 4094-4102, 2015.
Nikhil Rasiwasia, Jose Costa Percira, Emanuele Coviello, Gabriel Doyle, Gert R. G. Lanckrict, Roger Levy, and Nuno Vasconcelos. A new approach to cross-modal multimedia retrieval. In Proceedliggs of the 18th
ACM International Conference on Multimedia, pp. 251-260) 2010.
Shauli Ravfogel, Valentina Pyatkin, Amir David Nissan Cohen, Avshalom Manevich, and Yoav Gold- berg. Description-based text similarity. In 1st Conference Language Modeling, 2024. URL https:
Nils Reimers and Iryna Gurevych. Sentenco-BERT: Sentence emboddings using Siamese BERT-networks In Kentaro Inui, Jing Jiang, Vincent Ng, and Xiaojun Wan (eds.). Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint. Conference
58

---

Published in Transactions on Machine Learning Research (02/2026)
Natural Language Processing (EMNLP-IJCNLP), pp. 2982-3992 Hong Kong, China, November 2019. URL https://aclaattho..
Association for Computational Linguistios. doi: org/D19-1410/.
Ruiyang Ren, Yingqi Qu, Jing Liu, Wayne Xin Zhao, QiaoQiao She, Hua Wu, Haifeng Wang, and Ji- Rong Wen. RocketQAv2: A joint training method for dense passage retrieval and passage re-ranking. In Marie-Prancine Moens, Xuanjing Huang, Lucia Specia, and Scott Wen-tau Yih (eds.), Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pp- 2825 2835, Online and Punta Cana, Dominican Republic, November 2021. Association for Computational Linguistics. doi: 10.18653/ v1 211/221.eenllmmmme... main. 224. URL https: https////aaaaaahloaaasee org/2021, emnlp- main. 224/.
Zhaochun Ren, Xiangnan He, Dawei Yin, and M. de Rijke. Information discovery in c-commerce. Foundations and Trends Information Retrieval, 18:417 690, 2024.
Stephen Robertson and Hugo Zaragoza. The probabilistic relevance framework: BM25 and beyond. Foun- dations and Trends in Information Retrieval, 3(4))333-3)) 389, April 2009. doi: (0.1561/15000000
Stephen E. Robertson and K. Sparek Jones. Relevance weighting of search terms. Journal of the American Sociely for Information science, 27(3):129 146, 1976.
Stephen E. Robertson, Steve Walker, Susan Jones, Micheline M. Hlanock-Beeauliee and Mike Gatford. Okapi at TREC-3. In Proceedings of the 3rd Tert RElrieval Conference (TREC-3), NIST Special Publicalion, pp. 109- 126. National Institute of Standards & Technology, 1995.
Anna Rogers, Olga Kovaleva, and Anna Rumshisky. A primer in BERTology: What know about how BERT works. Transactions of the Association for Computational Linguistics, 8:842-866, 2020. doi: 10. 1162/tacl 00349. URL htapa:://aclannhollogyy wwg.g/20..000.m/ org/2020.tac1111.555 tacl-1.54/.
Uma Roy, Noah Constant, Rami Al-Rfou, Aditya Barua, Aaron Phillips, and Yinfei Yang. LAReQA: Language-agnostic answer retrieval from a multilingual pool. In Bonnic Webber, Trevor Cohn, Yulan He, and Yang Liu (eds.), Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pp. doi: 10.18653/v1 wt11222..emmm///p. mainp.47.77777 URL https: https: h/tpa/aaaaaannto...../.. . org/2020 emn po-main..777
Online, November 2020. Association for Computational Linguistics.
Devendra Sachan, Mike Lewis, Mandar Joshi, Armen Aghajanyan, Wen-tau Yih, Joelle Pineau, and Luke Zettlemoyer. Improving passage retrieval with zero-shot question generation. In Yoav Goldberg, Zornitsa Kozareva, and Yue Zhang (eds.), Proceedings of the 2022 Conference on Empirical Melhods in Natural Language Processing, pp. 3781-3797, Abu Dhabi, United Arab Emirates, December 2022. Association for 249. URL https: https://accaanthologg. org/
Computational Linguistics. doi: 20155/3/11202.mmm/ppp 2022. emnlp-main 249/.
Ruslan Salakhutdinov and Geoffrey Hinton. Semantic hashing. International Journal of Approrimate Rea- sonvng, 50(7):969 199699977762209 2009.
Gerard Salton. Interactive information retrieval. Technical report, Cornell University, 1969.
Gerard Salton. Automatic processing of foreign language documents. Journal of the American Society for Information Science, 21(3):187-19) 1944, 1970.
Gerard Salton and Christopher Buckley. Term Term-weighting approaches in automatic text retrieval. Information processing & management, 24(5):513-55) 523, 1988.
Gerard Salton, Anita Wong, and Chung-Shu Yang. A vector space model for automatic indexing, Commus nications of the ACM, 18(11):613 0200662000122 1975.
Gerard Salton, Edward A. Fox, and Harry Wu. Extended Boolean information retrieval. Communicaations of the ACM, 26(11):1022 -0000 1983.
Victor Sanh, Lysandre Debut, Julien Chaumond, and Thomas Wolf. DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter. arXio preprint or
2019.
59

---

Published in Transactions on Machine Learning Research (02/2026)
Victor Sanh, Albert Webson, Colin Raffel, Stephen Bach, Lintang Sutawika, Zaid Alyafeai, Antoine Chaffin, Arnaud Stiegler, Arun Raja, Manan Dey, et al. Multitask prompted training enables wero-shot task generalization. In International Conference on Learning Representations, 2022.
Keshav Santhanam, Omar Khattab, Jon Saad-Falcon, Christopher Potts, and Matei Zaharia. ColBERTv2: Effective and efficient retrieval via lightweight late interaction. In Marine Carpuat, Marie- Catherine de Marneffe, and Ivan Vladimir Meza Ruiz (eds.), Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pp. 3715 3734, Seattle, United States, July 2022. Association for Computational Linguistics. doi: URL
10.18653/vl. 20/6533/11202..mme//
wrg/2022 naacl-aaammannnaa 272/.
Naomi Saphra and Sarah Wiegreffe. Mechanistic? arXiv preprint arXiv:2410.09093 2024.
Timo Schick, Jane Dwivedi-Yu, Roberto Dessi, Roberta Raileanu, Maria Lomeli, Eric Hambro, Luke Zeitle- moyer, Nicola Cancedda, and Thomas Scialom. Toolformer: Language models can teach themselves to use tools. Advances in Neural Information Processing Systems, 96:68539-68551. 2023.
Hinrich Schütze, Christopher D. Manning, and Prabhakar Raghavan. Introduction to information retrieval, volume 39. Cambridge University Press Cambridge, 2008.
Aliaksei Severyn and Alessandro Moschitti. Learning to rank short text pairs with convolutional deep neural networks. In Proceedings of the 38th International ACM SIGIR Conference on Research and Development Information Retrieval, pp. 373 382, 2015.
Rulin Shao, Jacqueline He, Akari Asai, Weijia Shi, Tim Dettmers, Sewon Min, Luke Zettlemoyer, and Pang Wei Koh. Scaling retrieval-based language models with a trillion-token datastore. In The 38th Annual Conference on Neurol Information Processing Systems, 2024.
Rulin Shao, Rui Qiao, Varsha Kishore, Niklas Muennighoff, Xi Victoria Lin, Daniela Rus, Bryan Kian Hsiang Low, Sewon Min, Wen tau Yih, Pang Wei Koh, and Luke Zettlemoyer. ReasonIR: Training retricvers for reasoning lasks. In 2nd Conference on Language Modeling, 2025. URL https://openrewe wwtpriewwweeerrrrr id=KkBCNLMbGj.
Abhishek Sharma, Abhishek Kumar, Hal Daume, and David W. Jacobs. Generalized multiview analysis: A discriminative latent space. In 2012 IEEE Conference on Compuler Vision and Pallern Recognition, pp. 2160 2167. IEEE, 2012.
Xinjie Shen, Zhichao Geng, and Yang Yang. Exploring 10 sparsification for inference-free sparse retrievers. In Proceedings of the 48th International ACM SIGIR Conference on Research and Development in In- formation Relrieval, SIGIR '25, pp. 1572-2570 New York, NY, USA, 2025. Association for Computing
Machinery. doi: 01.1145/772632233777
Yelong Shen, Xiaodong He, Jianfeng Gao, Li Deng, and Grégoire Mesnil. Learning semantic representations using convolutional neural networks for web search. In Proceedings of the 23rd International Conference on World Wide Web, WWW '14 Companion, pp. 373-374 New York, NY, USA, 2014a. Association for Computing Machinery. doi: 10.1145/2567948.25777 2577338.
Yelong Shen, Xiaodong He, Jianfeng Gao, Li Deng, and Grégoire Mesnil. A latent semantic model with convolutional-pooling structure for information retrieval. In Proceedings of the 23rd ACM International Conference on Conference on Information and Knowledge Management, CIKM '14, pp. 101 110, New York, NY, USA, 2014b. Association for Computing Machinery. doi: (2114//266129.2220
Peng Shi, He Bai, and Jimmy Lin. Cross-lingual training neural models for document ranking. In Trevor Cohn, Yulan He, and Yang Liu (eds.), Findings of the Association for Computational Linguistios: EMNLP 2020, pp. 7768-2773, Online, November 2020. Association for Computational Linguisties. doi: 00.16633vv/2020 wwn.e.iiinninggenn..222. URL http://aaaaaatlory/..///.
findings- sanes-emnll... 249/.
60

---

Published in Transactions on Machine Learning Research (02/2026)
Noah Shinn, Federico Cassano, Ashwin Gopinath, Karthik Narasimhan, and Shunyu Yao. Reflexion: Lan- guage agents with verbal reinforcement learning. Advances in Newral Informalion Processing Systems, 36: 8634 8652, 2023.
Aditi Singh, Abul Fhtesham, Saket Kumar, and Tala Talaci Khoei. Agentic retrieval-augmented generation: A survey on agentic RAG. arXiv preprint arXiv:2501.0911 2025a.
Amanpreet Singh, Joseph Chee Chang, Dany Haddad, Aakanksha Naik, Jena D. Hwang, Rodney Kinney, Daniel S Weld, Doug Downcy, and Sergey Feldman. A12 scholar QA: Organized literature synthesis with attribution. In Pushkar Mishra, Smaranda Muresan, and Tao Yu (eds.), Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 3: System Demonstrutions), pp. 513- 523, Vienna, Austria, July 2025b. Association for Computational Linguistics. doi: 00.18653/v1/2022 acl-demo.49 49. URL https: https://aclaatthhhceoo .ayy.ozz orggoo2g// acl-demo.49
Ashudoep Singh and Thorsten Joachims. Fairness of exposure rankings. Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pp. 2219-2222 2018
Richard Socher, Andrej Karpathy, Quoc Le, Christopher D. Manning, and Andrew Y. Ng. Grounded compositional semantics for finding and describing images with sentences. Transactions of the association for compulational linguistics, 2:207-218 218, 2014.
Chan Hee Song, Jiaman Wu, Clayton Washington, Brian M. Sadler, Wei-Lun Chao, and Yu Su. LLM- planner: Few-shot grounded planning for embodied agents with large language models. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 2992-9999900 3009, 2023.
Fei Song and W. Bruce Croft. A general language model for information retrieval. In Proceedings of the 8th International Conference on Information and Knowledge Management, pp. 316-321, 1999.
Meina Song, Qing Liu, and E. Haihong. Deep hierarchical attention networks for text matching in infor- mation retrieval. 2018 International Conference on Information Systems and Computer Aided Education (ICISCAE), pp. 476 481, 2018.
Daniel Sonntag and Hans-Jürgen Profitlich. An architecture of open-source tools to combine textual infor- mation extraction, faceted scarch and information visualisation. Artificial intelligence in Medicine, 93: 13-28, 2018.
Heydar Soudani, Evangelos Kanoulas, and Facghch Hasibi. Fine tuning vs. retricval augmented generation for less popular knowledge. In Proceedings of the 2024 Annual International ACM SIGIR Conference on Research and Development in Information Retrieval in the Asia Pacific Region, SIGIR-AP 2024, pp. 12-22, New York, NY, USA, 2024. Association for Computing Machinery. doi: 10.1145/3673791 36698411...
Karen Sparck Jones. A statistical interpretation of term specificity and its application in retrieval. Journal of documentation, 28(1):11 21, 1972.
Hongjin Su, Howard Yen, Mengzhou Xia, Weijia Shi, Niklas Muennighoff, Han-yu Wang, Haisn Liu, Quan Shi, Zachary S. Siegel, Michael Tang, et al. BRIGHT: A realistic and challenging benchmark for reasoning- intensive retrieval. arXiv preprint wxXi:2247.11888 2024.
Yongye Su, Zeya Zhang, Jane Kou, Cheng Ju, Shubhojeet Sarkar, Yamin Wang, Ji Liu, and Shengbo Guo. Modernizing facebook scoped search: Keyword and embedding hybrid retrieval with LLM evaluation. arXiv preprint arXiv:2509.13603 2025.
Weiwei Sun, Lingyong Yan, Xinyu Ma, Shuaiqiang Wang, Pengiie Ren, Zhumin Chen, Dawei Yin, and Zhaochun Ren. Is ChatGPT good at search? investigating large language models as re-ranking agents. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pp. 14918- 14937, 2023.
Yu Sun, Shuohuan Wang. Yukum Li, Shikun Feng, Xuyi Chen, Han Zhang, Xin Tian, Danxiang Zhu, Hao Tian, and Hua Wu. ERNIE: Enhanced representation through knowlodge integration. orXiv preprint urXiv:1904.0922 2019.
61

---

Published in Transactions on Machine Learning Research (02/2026)
Kai Sheng Tai, Richard Socher, and Christopher D. Manning. Improved semantic representations from tree- structured long short-term memory networks. In Chengqing Zong and Michael Strube (eds.), Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pp. 1556-1566, Beijing, China, July 2015. Association for Computational Linguistics. doi: 10.3115/v1/P15-111 //aclanthology. www.:a/oooooooooo/---55555. org/P15-1150/.
URL https:
Yubao Tang, Ruqing Zhang, Jiafeng Guo, and Maarten de Rijke. Recent advances in generative information retrieval. Companion Proceedings of the ACM Web Conference 2024, 2023.
Yi Tay, Vinh Q. Tran, Mostafa Dehghani, Jianmo Ni, Dara Bahri, Harsh Mehla, Zhen Qin, Kai Hui, Zhe Zhao, Jai Gupta, Tal Schuster, William W. Cohen, and Donald Metzler. Transformer memory as a differentiable search index. In Alice H. Oh, Alekh Agarwal, Danielle Belgrave, and Kyunghyun Cho (eds.), Advances in Neural Information Processing Systems, 2022. URL sttps://openreee net/forumiid Vu-BOc1Pfq.
Michael Taylor, John Guiver, Stephen Robertson, and Tom Minka. Softrank: optimizing non-smooth rank metrics. In Proceedings of the 2008 International Conference on Web Search and Data Mining, pp. 77-86, 2008.
Wilson L. Taylor. "Cloze Procedure": A new tool for measuring readability. Journalism & Mass Communi-- cation Quarterly, 30:415 433, 1953.
Ian Tenney, Dipanjan Das, and Ellie Pavlick. BERT rediscovers the classical NLP pipeline. In Anna Korho- nen, David Traun, and Lluís Mårquez (eds.), Proceedings of the 57th Annual Meeling of the Association for Computational Lánguistics, pp. 1593-4601, Florence, Italy, July 2019. Association for Computational Linguistics. doi: 10.18653/v1/P19-1
URL https://aclantthologg wwtpaaanhhooooo.oorr///11-1-111. org/P19-145///
Nandan Thakur, Nils Reimers, Andreas Rücklé, Abhishok Srivastava, and Iryna Gurevych. BEIR: A hetero-
geneous benchmark for zero-shot evaluation of information retrieval models. In 35th Conference on Newral Information Processing Systems Datasets and Benchmarks Track (Round 2), 2021.
Yu Tian, Xiao Yang, Jingyuan Zhang, Yinpeng Dong, and Hang Su. Evil geniuses: Delving into the safety LLM-based agents. arxiv preprint arXw:23311.11855, 2023.
Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bash- lykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, et al. LLaMA 2: Open foundation and fine-tuned chat models. Xix preprint arXiv:2307.09288 2023.
Harsh Trivedi, Niranjan Balasubramanian, Tushar Khot, and Ashish Sabharwal. Interleaving retrieval with chain-of-thought rcasoning for knowledge-intensive multi-step questions. In Anna Rogers, Jor- dan Boyd-Graber and Naoaki Okazaki (eds.), Proceedings of the 61st Annual Meeting of the Asso- ciotion for Computational Linguistics (Volume 1: Long Papers), pp. 10014-10037, Toronto, Canada, July 2023. Association for Computational Linguistics. doi: 10.18653/1112022..m acloo...--. URL https: //aclanthology. hwcellloyyyoogg//// org/2023. wtt.:.....11................. acl-long. 557/.
Howard Turtle and James Flood. Query evaluation: strategies and optimizations. Information Processing & Management, (1(6):831-856) 1995.
Ahmet Üstün, Arianna Bisazza, Gosse Bouna, and Gertjan Noord. UDapter: Language adaptation for truly Universal Dependency parsing. In Bonnie Webber, Trevor Cohn, Yulan He, and Yang Liu (eds.). Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing ( (FMNLP), pp- 2302 2315, Online, November 2020. Association for Computational Linguistics. doi: 10.18653/v1/2000 mailp-main.180. URL https: sttps://aclanttooopg www.:e/..rg......... mmn.pmmain.n80000
C. Van Rijsbergen. Information retrieval: theory and practice. In Proceedings of the Joint IBM/ Uninnuitrrstty of Newcastle Upon Tyne Seminar on Data Base Systems, volume 79, pp. 1-14, 1979.
62

---

Published in Transactions on Machine Learning Research (02/2026)
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, L ukasz Kaiser, and Illia Polosukhin. Attention is all you need. In T. Guyon, U. Von Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Garnett (eds.), Advances in Neural Information Processing Systems, volume 30. Curran Associates, Inc., 2017. URL https: http:/:://ooooeeneeeee wewpeediinnnnnnuiissseeeee www.ips.cc/ppperr... files/ fitess/pperr///////eeeeeaaaa Papeaa..................
Henrique Schechter Vera, Sahil Dua, Biao Zhang, Daniel Salz, Ryan Mullins, Sindhu Raghuram Panyam, al. EmbeediinGemma: Powerful and lightweight text representations. arXiv preprint arXiv:2500.2054, 2025.
Jean-Philippe Vert, Koji Tsuda, and Bernhard Schölkopf. A primer on kernel methods. In Kernel Methods in Computational Biology, pp. 35-70. MIT press, 2004.
David Wan, Han Wang, Elias Stengel-Eskin, Jaemin Cho, and Mohit Bansal. CLaMR: Contextualized late-interaction for multimodal content retrieval. arXiv preprint arXiv:2506.061 06144, 2025.
Shengxian Wan, Yanyan Lan, Jiafeng Guo, Jun Xu, Liang Pang, and Xueqi Cheng. A deep architecture for semantic matching with multiple positional sentence representations. In Proceedings of the 30th AAAI Conference on Artificial Intelligence, AAAI16, pp. 2835-2841. AAAI Press, 2016.
Ben Wang and Aran Komatsuzaki. GPT-J-6B: A 6 Billion Parameter Auloregressive Language Model. https://github... wttph//www.cc/igggoooo///aaaaaaa
jax, May 2021.
Bokun Wang, Yang Yang, Xing Xu, Alan Hanjalic, and Heng Tao Shen. Adversarial cross-modal retrieval.
In Proceedings of the 25th ACM International Conference on Mullimedia, pp. 154 114,162,245 2017.
Haixun Wang and Taesik Na. Rethinking e-commerce search. ACM SIGIR Forum, 57:1-19, 2023.
Jiongxiao Wang, Qiaojing Yan, Yawei Wang, Yijun Tian, Soumya Smruti Mishra, Zhichao Xu, Megha Gandhi, Panpan Xu, and Lin Lee Cheong. Reinforcement learning for self-improving agent with skill library, 2025a. URL attps://arae http///axxvvvoo// wrraivvoorr//aae otgg/bs/////////// 17102.
Kaiye Wang, Qiyue Yin, Wei Wang, Shu Wu, and Liang Wang. A comprehensive survey on cross-modal retrieval. arXin preprint Xrx:v: NNw:1607.0611, 06215, 2016.
Lei Wang, Chen Ma, Xueyang Feng, Zeyu Zhang, Hao Yang, Jingsen Zhang, Zhiyuan Chen, Jiakai Tang, Xu Chen, Yankai Lin, et A survey on large language model based autonomons agents. Frontiers of Computer Science, 18(6)1186345, 2024a.
Liang Wang, Nan Yang, Xiaolong Huang, Binxing Jiao, Linjun Yang, Daxin Jiang, Rangan Majumder, and Furu Wei. Text embeddings by weakly-supervised contrastive pre-training. arXio preprint ar W:Xi:22212005522 2022a.
Liang Wang, Nan Yang, Xiaolong Huang, Linjun Yang, Rangan Majumder, and Furu Wei. Improving text embeddings with large language models. arXiv preprint or arxix22411.0 000:0400.0068, 00368, 2023a.
Tianshi Wang, Fengling Li, Lei Zhu, Jingjing Li, Zheng Zhang, and Heng Tao Shen. Cross-modal retrieval: A systematic review of methods and future directions. arXic preprint Ntii:2308.11260 2024b.
Xuanhui Wang, Nadav Golbandi, Michael Bendersky, Donald Metzler, and Mare Najork. Position bias ostimation for unbiased learning to rank in personal search. In Proceedings of the 11th ACM International Conference on Web Search and Data Mining, pp. 610-618, 2018.
Xuewei Wang, Qiang Jin, Shengyu Huang, Min Zhang, Xi Liu, Zhengli Zhao, Yukun Chen, Zhengyu Zhang, Jiyan Yang, Ellie Wen, Sagar Chordia, Wenlin Chen, and Qin Huang. Towards the better ranking con- sistency: multi-task learning framework for early stage ads ranking. arXiv preprint orXiv:2307.1109) 2023b.
63

---

Published in Transactions on Machine Learning Research (02/2026)
Yumeng Wang, Lijun Lyu, and Avishek Anand. BERT rankers are brittle: A study using adversarial document perturbations. In Proceedings of the 2022 ACM STGIR Intervational Conference on Theory of Information Retrieval, ICTIR '22, pp. 15--20, New York, NY, USA, 2022b. Association for Computing Machinery. doi: 10.1145/ 10.1145/353981... 3539813.3545122.
Zhenduo Wang, Zhichao Xu, Vivek Srikumar, and Qingyao Ai. An in-depth investigation of user response simulation for conversational search. In Proceedings of the ACM on Web Conference 2024, pp. 1407 1418, 2024c.
Zora Zhiruo Wang, Akari Asai, Xinyan Velocity Yu, Frank F. Xu, Yiqing Xie, Graham Neubig, and Daniel Fried. CodeRAG-bench: Can retrieval augment code generation? In Luis Chiruzzo, Alan Ritter, and Lu Wang (eds.), Findings of the Association for Computational Linguistics: NAACL 2025, pp. 3199 3214, Albuquerque, New Mexico, April 2025b. Association for Computational Linguistics. doi: 10.18653/v1/ n77cl.176. findings-naacl.17788 URL
2025. f12diiind
org/2025. findings-naacl. naggss-naall.111 176/.
Benjamin Warner, Antoine Chaffin, Benjamin Clavié, Orion Weller, Oskar Hallström, Said Taghadouini, Alexis Gallagher, Raja Biswas, Faisal Ladhak, Tom Aarsen, Griffin Thomas Adams, Jeremy Howard, and lacopo Poli. Smarter, better, faster, longer: A modern bidirectional encoder for fast, mem- ory efficient, and long context finetuning and inference. In Wanxiang Che, Joyce Nabende, Ekate- rina Shutova, and Mohammad Taher Pilehvar (eds.), Proceedings of the 63rd Annual Meeling of the Association for Computational Línguistics (Volume 1: Long Papers), pp. 2526 2547, Vienna, Aus- tria, July 2025. Association for Computational Linguisties. doi: 10.18653/v112225..c acc.2222....-oo.n1222 127. URL attps://acle hawps///llcattooogo.ooooooo org.//25.222 acl-long 127/.
Cong Wei, Yang Chen, Haonan Chen, Hexiang Hu, Ge Zhang, Jie Fu, Alan Ritter, and Wenhu Chen. UnilR: Training and benchmarking universal multimodal information retrievers. In Computer Vision - ECCV 2024, volume 15145 of Lecture Notes in Computer Science, pp. 387-404. Springer, Cham, 2025. doi: 14.10077998---001--3311------- 23.
Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoe V. Le, Denny Zhou, et al. Chain-of-thought prompting elicits reasoning in large language models. Advances in neural information processing systems, 35:24824 -248377 2022.
Orion Weller, Benjamin Van Durmo, Dawn Lawrie, Ashwin Paranjape, Yuhao Zhang, and Jack Hessel. Promptriever: Instruction-Irained retrievers can be prompted like language models. ar preprinl arXiv:2/09.11136, 2024.
Orion Weller, Benjamin Chang, Sean MacAvaney, Kyle Lo, Arman Cohan, Benjamin Van Durme, Dawn Lawrie, and Luca Soldaini. FollowIR: Evaluating and teaching information retrieval models to follow instructions. In Luis Chiruzzo, Alan Ritter, and Lu Wang (eds.), Proceedings of the 2025 Confer- ence of the Nations of the Americas Chapter of the Association for Computational Linguistics: Hu- man Language Techmologies (Volume 1: Long Papers), pp. 11926-11942, Albuquerque, New Mexico, April 2025a. Association for Computational Linguistics. doi: 10.18653/ve /18533/vl//// / 2222 maac.aalcclo..59 URL https: http://ccaanttholgoo/o
org/2025.maacl-oong 1977
Orion Weller, Kathryn Ricci, Eugene Yang, Andrew Yates, Dawn Lawric, and Benjamin Van Durme. Rank1: Test-time compute for reranking in information retrieval. In 2nd Conference on Language Modeling, 2025b. URL
net/ orum? f wwwpumidddPoOPbbbalee
Ryen W. White and Resa A. Roth. Exploratory search: Beyond the query-response paradign. Morgan & Claypool Publishers, 2009.
John Wieting, Mohit. Bansal, Kevin Gimpel, and Karen Livescu. Towards universal paraphrastic sentence embeddings. arXiv preprint rrXie:1511.08190 2016.
Thomas D. Wilson. Human information behavior. Informing science, 3:49, 2000.
S. K. Michael Wong and Y. Y. Yao. A probability distribution model for information retrieval. Information Processing & Managemend, 25(1):39 53, 1989.
64

---

Published in Transactions on Machine Learning Research (02/2026)
Marco Wrzalik and Dirk Krechel. CoRT: Complementary rankings from transformers. In Proceedings of the 2021 Conference of the North American Chapler of the Associalion for Compulational Linguistics: Human Language Technologies, pp. 4194 4204, 2021.
Jialong Wu, Baixuan Li, Rumnan Fang, Wenbiao Yin, Liwen Zhang, Zhengwei Tao, Dingchu Zhang, Zekun Xi, Yong Jiang, Pengjun Xie, Fei Huang, and Jingren Zhou. WebDancer: Towards autonomous information seeking agency. arv preprint arXiv:2505.220/ 2025a.
Jialong Wu, Wenbiao Yin, Yong Jiang, Zhenglin Wang, Zekun Xi, Runnan Fang, Linhai Zhang, Yulan He, Deyu Zhou, Pengiun Xie, and Fei Huang. WebWalker: Benchmarking LLMs in web traversal. In Wanxiang Che, Joyce Nabende, Ekaterina Shutova, and Mohammad Taher Pilehvar (eds.), Proceedings of the 63md Annual Meeling of the Association for Compulational Linguisties (Volume 1: Long Papers); pp. 10290 10305, Vienna, Austria, July 2025b. Association for Computational Linguistics. URL https: // wwwpantthooooooooo//g/e ara-e/..o//.......oog.g 508/.
Qiang Wu, Christopher J.C. Burges, Krysta M. Svore, and Jianfeng Gao. Adapting boosting for information retrieval measures. Information Retrieval, 13:254-270, 2010.
Qingyun WU, Gagan Bansal, Jieyn Zhang, Yiran Wu, Shaokun Zhang, Erkang Zhu, Beibin Li, Li Jiang, Xi- aoyun Zhang, and Chi Wang. AutoGen: Enabling next-gen LLM applications via multi-agent conversation. arxiv preprint Xrx:22200 08155, 2023a.
Xing Wu, Guangyuan Ma, Meng Lin, Zijia Lin, Zhongyuan Wang, and Songlin Hu Contextual masked aulo-encoder for dense passage retrieval. Proceedings of the AAAI Conference on Artificial Intelligence, 37(4):4738-477 4746, 2023b.
Yunjia Xi, Jianghao Lin, Yongzhao Xiao, Zheli Zhou, Rong Shan, Te Gao, Jiachen Zhu, Weiwen Liu, Yong Yu, and Weinan Zhang. A survey of LLM- deep search agents: Paradigm, optimization, evaluation, and challenges. arxiv preprint arXiv:2588.0 056688 2025.
Fen Xia, Tan Liu, Jue Wang, Wensheng Zhang, and Hang Li. Listwise approach to learning to rank: theory and algorithm. In Proceedings of the 25th International Conference Machine Learning, pp. 1192 1199, 2008
Chong Xiang, Tong Wu, Zexuan Zhong, David Wagner, Danqi Chen, and Prateek Mittal. Certifiably robust RAG against retrieval corruption. arXiv preprint Xrii:24405. 15556, 2024a.
Zhen Xiang, Linzhi Zheng, Yanjie Li, Junyuan Hong, Qinbin Li, Han Xie, Jiawei Zhang, Zidi Xiong, Chulin Xie, Carl Yang, et al. GuardAgent: Safeguard LLM agents by a guard agent via knowledge-enabled reasoning. arXie preprint Xr 091887 2024b.
Shitao Xiao, Zheng Liu, Yingxia Shao, and Zhao Cao. RetroMAE: Pro-training rotrieval-oriented language models via masked auto-encoder. In Yoav Goldberg, Zornitsa Kozareva, and Yue Zhang (eds.), Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pp. 538 338-544, Abu Dhabi, United Arab Emirates, December 2022. Association for Computational Linguistics. doi: 10.18653/vl] 2022. 2022.emnlp- ma....3. URL https: / https://aclanthoon aalaattoonnntoyyo..o org/2022 emnlp- cannp-main.31. 35/.
Jinheng Xie, Weijia Mao, Zechen Bai, David Junhao Zhang, Weihao Wang, Kevin Qinghong Lin, Yuchao Gu, Zhijie Chen, Zhenheng Yang, and Mike Zheng Shou. Show-o: One single transformer to unify multimodal understanding and generation. Xix preprint or Xiv:2400 01044088.2225 12528, 2024a.
Quanting Xie, So Yeon Min, Tianyi Zhang, Kedi Xu, Aarav Bajaj, Rass Salakhutdinov, Matthew Johnson- Roberson, and Yonatan Bisk. Embodied-RACE RAG: General pan-maramtt embodied memory for retrieval and generation. In Language Gamification NewrIPS 2024 Workshop, 2024b. URL httpss//openrreiier net/forum?id=U8p@aaa
Chenyan Xiong, Zhuyun Dai, Jamie Callan, Zhiyuan Liu, and Russell Power. End-to-end neural ad-hoc ranking with kernel pooling. In Proceedings of the 40th International ACM SIGIR Conference on Research and Development in Information Retrievol, SIGIR '17, pp. 55 64, New York, NY, USA, 2017. Association for Computing Machinery. doi: 10.1145/3077 300777730066000000
65

---

Published in Transactions on Machine Learning Research (02/2026)
Lee Xiong, Chenyan Xiong, Ye Li, Kwok-Fung Tang, Jialin Liu, Paul N. Bennett, Junaid Ahmed, and Arnold Overwijk. Approximate nearest neighbor negative contrastive learning for dense text retrieval. arXiv preprint ar arXic:2007.00000 2020.
Jun Xu, Xiangnan He, and Hang Li. Deep learning for matching in search and recommendation. In The 41st International ACM SIGIR Conference Research & Development in Information Retrieval, pp.
1365-1368, 2018.
Mengyao Xu, Wenfei Zhou, Yauhen Babakhin, Gabriel De Souza Pereira Moreira, Ronay Ak, Radek Os- mulski, Bo Liu, Even Oldridge, and Benedikt Schifferer. Saini--meed-NNeootoe A unified multimodal retrieval model for text, image, audio, and video. arXiv preprint arXiv:25 10. 10.03488, 2025a.
Shicheng Xu, Danyang Hou, Liang Pang, Jingcheng Deng, Jun Xu, Huawei Shen, and Xueqi Cheng. In- visible relevance bias: Text-image retrieval models prefer AT-generated images. In Proceedings of the 47th Intervational ACM SIGIR Conference Research and Development in Information Retrieval, SIGIR '24, pp. 208-217, New York, NY, USA, 2024a. Association for Computing Machinery. doi: 10.1145 00.1145/3626772.365777
Zhichao Xu. Context-aware decoding reduces hallucination in query- fuccu-f summarization, 2023. URL
https://aat //axxiv. wwwpaa..rggbbb///..i. 14335.
Zhichao Xu. RankMamba: Benchmarking Mamba's document ranking performance in the era of transform-
ers. arXiv preprint or arXiv:2403. 18276, 2024.
Zhichao Xu and Daniel Cohen. A lightweight constrained generation alternative for query-focused sum- marization. In Proceedings of the 46th International ACM SIGIR Conference on Research and De- velopment in Information Retrieval, SIGIR '23, pp. 1745 1749, New York, NY, USA, 2023. Associa- tion for Computing Machinery. ISBN 9781450394086. //doi.org/// 100....111//3311111 3591936.
doi: 10. 09.1145/3339618.3551133
URL https:
Zhichao Xu and Jiepu Jiang. Multi dimei-dimeniional evaluation of empathetic dialogue responses. In Yaser Al-Onaizan, Mohit Bansal, and Yun-Nung Chen (ods.), Findings of the Association for Computational Linguislies: EMNLP 2024, pp. 2066 2087, Miami, Florida, USA, November 2024. Association for Com- putational Linguistics. doi: 10. v.11633/1122024.fm findings-eemlpp... 2024. f wwwdings-emml...ee.... em3.p.133.
URL https:// .rg// org/
Zhichao Xu, Yi Han, Tao Yang, Anh Tran, and Qingyao Ai. Learning to rank rationales for explainable recommendation, 2022a. URL https://arxivw org/ ab2//20.. 00368..
Zhichao Xu, Anh Tran, Tao Yang, and Qingyao Ai. Reinforcement learning to rank with coarse-grained labels. arXiv preprint arXho:2208.07563 2022b.
Zhichao Xu, Hansi Zeng, Juntao Tan, Zuohui Fu, Yongfeng Zhang, and Qingyao Ai. A reusable model- agnostic framework for faithfully explainable recommendation and system scrutability, ACM Transactions on Information Systems, 42(1):1-29, 2023.
Zhichao Xu, Ashim Gupta, Tao Li, Oliver Bentham, and Vivek Srikumar. Beyond perplexity: Multi- dimensional safety evaluation of LILM compression. In Yaser Al-Onaizan, Mohit Bansal, and Yun-Nung Chen (eds.), Findings of the Association for Computational Linguistics: EMNLP 2024, pp. 15359 15396, Miami, Florida, USA, November 2024b. Association for Computational Linguisties. doi: 10.18653/v1/ 2024. wwwddiinnnnnniinnnn emn1p.901. URL ttttps://aclanntolloo.
htt/2224.findinee. fiidingsseeniin emnlp.901/
Zhichao Xu, Hemank Lamba, Qingyao Ai, Joel Tetreault, and Alex Jaimes. CFE2: Counterfactual editing for search result explanation. In Proceedings of the 2024 ACM SIGIR International Conference on Theory of Information Retricval, pp. 145-155, 2024c.
Zhichao Xu, Aosong Feng, Yijun Tian, Haibo Ding, and Lin Lee Chcong. CSPLADE: Learned sparse retrieval with causal language models. In Kentaro Inui, Sakriani Sakti, Haofen Wang, Derek F. Wong, Pushpak Bhattacharyya, Biplab Banerjee, Asif Ekbba, Tanmoy Chakraborty, and Dhirendra Pratap Singh
66

---

Published in Transactions on Machine Learning Research (02/2026)
(eds.), Proceedings of the 14th International Joint Conference on Natural Language Processing and the 4th Conference of the Asia-Pacific Chapler of the Associalion for Compulational Linguistics, pp. 99- 114, Mumbai, India, December 2025b. The Asian Federation of Natural Language Processing and The Association for Computational Linguistics. URL
.rg.g.org..o 20220025..11 wttp///....nnnpp........o ijcnlp-long.7/
Zhichao Xu, Zhiqi Huang, Shengyao Zhuang, and Vivek Srikumar. Distillation versus contrastive learning: How to train your rerankers. In Kentaro Inui, Sakriani Sakti, Haofen Wang, Derek F. Wong, Pushpak Bhattacharyya, Biplab Banerjee, Asif Ekbal, Tanmoy Chakraborty, and Dhirendra Pratap Singh (eds.), Proceedings of the 14th International Joint Conference on Natural Language Processing and the 4th Confer- ence of the Asia- Saca-Pacificc Chapter of the Association for Computational Linguisties, pp. 564 578, Mumbai, India, December 2025c. The Asian Federation of Natural Language Processing and The Association for Computational Linguistics. URL hatpp:s:/aalaathholoo/ wwwpennhooooooooo/222e org/2025. manding--ijcnl...
Zhichao Xu, Minheng Wang, Yawei Wang, Wenqian Ye, Yuntao Du, Yumpu Ma, and Yijun Tian. Recon: Reasoning with condensation for efficient retrieval-augmented generation, 2025d. URL https://arxxi org/ abg/ 10448.
Zhichao Xu, Jinghua Yan, Ashim Gupta, and Vivek Srikumar. State space models are strong lext rerankers. In Vaibhav Adlakha, Alexandra Chronopoulou, Xiang Lorraine Li, Bodhisattwa Prasad Majumder, Freda Shi, and Giorgos Vernikos (eds.), Proceedings of the 10th Workshop on Representation Learning for NLP (RepL4NLP-2025) pp. 159, Albuquerque, NM, May 2025e. Association for Computational Linguistics. URL http::///acaaahhlloggoeo orgg2025.rep14m1pp rep11m1-1111111
Zhichao Xu, Shengyao Zhuang, Xueguang Ma, Bingsen Chen, Yijun Tian, Fengran Mo, Jie Cao, and Vivek Srikumar. Rethinking on-policy optinization for query augmentation. arXiv preprint orXiw:2510. 17139, 2025f.
Zhichao Xu, Shengyao Zhuang, Crystina Zhang, Xueguang Ma, Yijun Tian, Maitrey Mehta, Jimmy Lin, and Vivek Srikumar. LACONIC: Dense-level effectiveness for scalable sparse retrieval via a two-phase training curriculum. XrXiv preprint orXiv:260100000 2026.
Le Yan, Zhen Qin, Honglei Zhuang, Rolf Jagerman, Xuanhui Wang, Michael Bendersky, and Harrie Oosterhuis. Consolidating ranking and relevance predictions of large language models through post- processing. In Yaser Al-Onaizan, Mohit Bansal, and Yun-Nung Chen (eds.), Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pp. 410-423, Miami, Florida, USA, November 2024. Association for Computational Linguistics. doi: 0055:///11222..mmmpppp https://aclanthhllee .eaei.org//0000 2024 emnlp-main. m5/
URL
An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, et al. Qwen3 technical report. arXiv preprint arXiv:2505.00 093888 2025a.
Eugene Yang, Dawn J. Lawrie, and James Mayfield. Distillation for multilingual information retrieval. Pro- ceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval, 2024.
Eugene Yang, Andrew Yates, Kathryn Ricci, Orion Weller, Vivek Chari, Benjamin Van Durme, and Dawn Lawrie. Rank-K: Test-time reasoning for listwise reranking. arXiv preprint arXiv:2505.1443 2025b.
Liu Yang, Qingvao Ai, Jiafeng Guo, and W. Bruce Croft. aNMM: Ranking short answer texts with attention- based ncural matching model. In Proceedings of the 25th ACM International on Conference on Information and Knowledge Management, CIKM '16, pp. 287 296, New York, NY, USA, 2016a. Association for Com- puting Machinery. doi: 10.1145 11L144/296332322883338
Songlin Yang, Jan Kautz, and Ali Hatamizadch. Gated Delta networks: Improving Mamba2 with Delta rule. In The 13th International Conference on Learning Representations, 2025c. URL /
67

---

Published in Transactions on Machine Learning Research (02/2026)
Tao Yang, Zhichao Xu, and Qingyao Ai. Vertical allocation-based fair exposure amortizing in ranking. In Proceedings of the Anaual International A ACM SIGIR Conference on Research and Development in Information Retrieval in the Asia Pacific Region, pp. 234 244, 2023a.
Tao Yang, Zhichao Xu, Zhenduo Wang, and Qingyao Ai. FARA: Future-aware ranking algorithm for fairness optimization. In Proceedings of the 32nd ACM International Conference on Information and Knowledge Munagement, pp. 2906 2916, 2023b.
Tao Yang. Zhichao Xu, Zhenduo Wang, Anh Tran, and Qingyao Ai. Marghnal-cortaantyyaaaaa fair ranking algorithm. In Proceedings of the 16th ACM International Conference on Web Search and Data Mining, pp. 24-32, 32, 2023c.
Xiao Yang, Peifeng Yin, Abe Engle, Jinfeng Zhuang, and Ling Leng. MTMD: A multi-task multi-domain framework for unified ad lightweight ranking at pinterest. arXiv preprint ar Xiv:255100 1i.:2510.00857, 2025d.
Zichao Yang, Diyi Yang, Chris Dyer, Xiaodong He, Alex Smola, and Eduard Hovy. Hicrarchical attention net- works for document classification. In Kevin Kuight, Ani Nenkova, and Owen Rambow (eds.), Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguis- ties: Human Language: Technologies, pp. 1489, San Diego, California, June 2016b. Association for Computational Linguistics. doi: 10. 10. 10.11853/vll 11111//11/1111111111 /N11--1611.. URL https://aclanthoot
org/N16-1174/.
Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik R. Narasimhan, and Yuan Cao. ReAct: Synergizing reasoning and acting in language models. In The 11th International Conference on Learning Representations, 2023. URL hstps://openreee wwtpeereviee.eee//rrreeeeeeeeeee tett/forummid-w// vluYUL-X.
Zonghai Yao, Adilya Parashar, Huixue Zhou, Won Seok Jang, Feiyun Ouyang, Zhichao Yang, and Hong Yu. MCQG-SReffne: Multiple choice question generation and evaluation with iterative self-critique, correction, and comparison foedback. Xixiv preprint 10.1:24100111100 2024.
Zeynep Akkalyoncu Yilmaz, Wei Yang. Haotian Zhang, and Jimmy Lin. Cross-domain modeling of sentence- level evidence for document retrieval. In Conference on Empirical Methods in Natural Language Processing, 2019.
Wenpeng Yin, Hinrich Schütze, Bing Xiang, and Bowen Zhou ABCNN: Attention based convolutional neural network for modeling sentence pairs. Transactions of the Association for compulational linguisties, 4:259 272, 2016.
Soyoung Yoon, Eunbi Choi, Jiycon Kim, Hyeongu Yun, Yireun Kim, and Seung- wwo Hwang. ListTT: Listwise reranking with indcoder improves zero-shot retrieval. In Lun-Wei Ku, Andre Martins, and Vivek Srikumar (eds.), Proceedings of the 62nd Annual Meeling of the Association for Compulational Linguislics (Volume 1: Long Papers), pp.
Bangkok, Thailand, 2024. Association for Computational Lin- guistics. doi: 10.18653/ /v/ ael.0....looonn....... 125. URL https: /aclanthooo org/2024. acl- long.on.....
Soyoung Yoon, Gyuwan Kim, Gyu-Hwung Cho, and Soung won Hwang. AcuRank: Uncortainty-awarc adaptive compulation for listwise reranking. ar preprinl arXiv:2505.18511 18512, 2025.
Ori Yoran, Tomer Wolfson, Ori Ram, and Jonathan Berant. Making retrieval-angmeented language models robust to irrelevant context. In ICLR 2024 Workshop on Large Language Model (LLM) Agents, 2024.
Ronghui You, Zihan Zhang, Ziye Wang, Suyang Dai, Hiroshi Mamitsuka, and Shanfeng Zhu. XttentionXML: extreme multi-label text classification.
Label free-based attention-aware deep model for Advances in neural information processing systems, 32, 2019.
Hongli Yu, Tinghong Chen, Jiangtao Feng, Jiangjie Chen, Weinan Dai, Qiying Yu, Ya-Qin Zhang, Wei- Ying Ma, Jingjing Liu, Mingxuan Wang, and Hao Zhou. Memagent: Reshaping long- long-context Ilm with multi-conv rl-based memory agent, 2025. URL https: /tt:///axxiii org/ ab5/250. 02259..
Hsiang-Fi Fu Yu, Kai Zhong, Jiong Zhang, Wei-Cheng Chang, and Inderjit S. Dhillon. PECOS: Prediction for enormous and correlated output spaces. Journal of Machine Learning Research, 23(98):1-33 122,, 2022a.
68

---

Published in Transactions on Machine Learning Research (02/2026)
Puxuan Yu and James Allan. A study of neural matching models for cross-lingual IR. In Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 1637 1640, 2020.
Puxuan Yu, Razich Rahimi, and James Allan. Towards explainable search results: a listwise explanation generator. In Proceedings of the 45th International ACM SIGIR Conference on Research and Development in Information. Retrieval, pp. 669 680, 2022b.
Puxuan Yu, Antonio Malii,, and Matthias Petri. Improved learned sparse retrieval with corpus-specifie vocabularies. In European Conference on Information Retrieval, pp. 181-194. Springer, 2024a.
Puxuan Yu, Luke Merrick, Gaurav Nuti, and Daniel Campos. Aretic-Embed 2.0: Multilingual retrieval
without compromise. Xrviv preprint 2Xx::2412.04500 1e..2412.04505 2024b.
Wenhao Yu, Hongming Zhang, Xiaoman Pan, Peixin Cao, Kaixin Ma, Jian Li, Hongwei Wang, and Dong Yu. Chain-of-Nole: Enhancing robustness in retrieval-augmentter language models. In Yaser Al-Onaizan, Mohit Bansal, and Yun-Nung Chen (eds.), Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pp. 14672-14685, Miami, Florida, USA, November 2024c. Association for Computational Linguistics. doi: 10. /tt////1/222.mm///// 2024. wwwelmmmaaeeeeeeeeeeeeeeeee..aee 813/.
URL https: attps://aclaanthology org/
Yue Yu, Chenyan Xiong, Si Sun, Chao Zhang, and Arnold Overwijk. COCO-DR: Combating the distri- bution shift in zero-shot dense retrieval with contrastive and distributionally robust learning. In Yoav Goldberg, Zornitsa Kozareva, and Yue Zhang (eds.), Proceedings of the 2022 Conference on Empiri- cal Methods in Natural Language Processing, pp. 1462-1479, Abu Dhabi, United Arab Emirates, De- cember 2022c. Association for Computational Linguistics. doi: 10.18653/ v1 2002.emmll-mmin.995 URL attps://aclanthol// org/ 2022/ eunlp-main.95/ 95/n.95/..
Hamed Zamani and W. Bruce Croft. Joint modeling and optimization of search and recommendation. In Biennial Conference on Design of Experimental Search & Information Retrieval Systems, 2018
Hamed Zamani, Mostafa Dehghani, W. Bruce Croft, Erik Learned-Miller, and Jaap Kamps. From neural re-ranking to neural ranking: Learning a sparse representation for inverted indexing, In Proceedings of the 27th ACM International Conference on Information and Knowledge Management, pp. 497 506, 2018.
Rabih Zbib, Lingjun Zhao, Damianos Karakos, William Hartmann, Jay DeYoung, Zhongqiang Huang, Zhuolin Jiang, Noah Rivkin, Le Zhang, Richard Schwartz, el al. Neural-networkk lexical translation for cross-lingual ir from text and speech. In Proceedings of the 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 645-654, 2019.
Hansi Zeng, Chen Luo, Bowen Jin, Sheikh Muhammad Sarwar, Tianxin Wei, and Hamed Zamani. Scalable and effective generative information retrieval. In Proceedings of the ACM Web Conference 2024, WWW 24, pp. 1441-1452, New York, NY, USA, 2024. Association for Computing Machinery. doi: 10.1145/ 3555334.3645477.
ChengXiang Zhai and John Lafferty. A study of smoothing methods for language models applied to infor- mation retrieval. ACM Transactions on Information Systems (TOIS), 22(2):179-214, 2004.
ChengXiang Zhai et al. Statistical language modcls for information retrioval a critical review. Foundations and Trends Information Retrieval, 2(3):137 213, 2008
Jiaqi Zhai, Lucy Liao, Xing Liu, Yueming Wang, Rui Li, Xuan Cao, Leon Gao, Zhaojic Gong, Fangda Gu, Michael He, Yin-Hua Lu, and Yu Shi. Actions speak louder than words: prilmion-arrmmteer sequential transducers for generative recommendations arXie preprint arXiv:2402.11155 2024.
Xiaohua Zhai, Basil Mustafa, Alexander Kolesnikov, and Lucas Beyer Sigmoid loss for language image pre- pre-training. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 11975- 11986, 2023.
69

---

Published in Transactions on Machine Learning Research (02/2026)
Jingtao Zhan, Jiaxin Mao, Yiqun Liu, Min Zhang, and Shaoping Ma. RepBERT: Contextualized text embeddings for Ist-stage retrieval. arviv preprint or Xiv:20066 15498, 2020.
Jingtao Zhan, Jiaxin Mao, Yiqui Liu, Jiafeng Guo, Min Zhang, and Shaoping Ma. Optimizing dense retrieval model training with hard negatives. In Proceedings of the 4 44th International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 1503 1512 2021.
Fengji Zhang, Bei Chen, Yue Zhang, Jacky Keung, Jin Liu, Daoguang Zan, Yi Mao, Jian-Guang Lou, and Weizhu Chen. RepoCoder: Repository-level code completion through iterative retrieval and generation. In Houda Bonamor, Juan Pino, and Kalika Bali (eds.), Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pp. 2471 2484, Singapore, December 2023a. Association for .rrg/
Computational Linguistics. doi: 10.1863//v120225 eetee...mmllmmmi.... 151. URL 2023. emnlp-main. 151/.
Hanqi Zhang; Chong Chen, Lang Mei, Qi Liu, and Jiaxin Mao. Mamba retriever: Utilizing Mamba for effec- tive and efficient dense retrieval. In Proceedings of the 33rd ACM International Conference on Information and Knowledge Management, pp. 2268-427, 2024.
Huaiwen Zhang, Yang Yang, Fan Qi, Shengsheng Qian, and Changsheng Xu. Debiased video-text retrieval via soft positive sample calibration. IEEE Transactions on Circuits and Systems for Video Technology, 33:5257 5270, 2023b.
Shujian Zhang, Chengyue Gong, and Eunsol Choi Knowing more about questions can help: Improving calibration in question answering. In Chengqing Zong, Fei Xia, Wenjie Li, and Roberto Navigli (eds.), Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021, pp. 1958-1970, Online, August 2021a. Association for Computational Linguistics. doi: 10.18653 v1 and....indig---a....22 URL https://aclanthir
org/2021 findings-acl.172)
Weinan Zhang, Junwei Liao, Ning Li, Kounianhua Du, and Jianghao Lin. Agentic information retrieval. arXiv preprint arXiv:2410.0977 2025a.
Xinyu Zhang, Andrew Yates, and Jimmy Lin. Comparing score aggregation approaches for document retrieval with pretrained transformers. In Djoerd Hiemstra, Francier Moens, Josiane Mothe, Raffaele Perego, Martin Potthast, and Fabrizio Sebastiani (eds.), Advonces in Information Retrieval, pp. 150 163, Cham, 2021b. Springer International Publishing.
Xinyu Zhang, Kelechi Ogueji, Xueguang Ma, and Jimmy Lin. Toward best practices for training multilingual dense retrieval models. ACM Transactions on Information Systems, 42:1-33, 2022.
Xinyu Zhang, Sebastian Hofstätter, Patrick Lewis, Raphael Tang, and Jimmy Lin. Rank-without-GPTT Building GPT--indeeendent listwise rerankers on open-source large language models. arXiv preprint arXiv:2312. 02969, 2023c.
Xinyu Zhang, Nandan Thakur, Odunayo Ogundepo, Ehsan Kamalloo, David Alfonso-Hermedt Xiaoguang Li, Qun Liu, Mehdi Rezagholizadeh, and Jimmy Lin. MIRACI: A multilingual retrieval dataset covering 18 diverse languages. Transactions of the Association for Computational Linguistics, 11:1114-11 1131, 2023d.
Yanzhao Zhang, Mingxin Li, Dingkun Long, Xin Zhang, Huan Lin, Baosong Yang, Pengjun Xie, An Yang, Dayiheng Liu, Junyang Lin, et al. Qwen3 embedding: Advancing text embodding and reranking through foundation models. Xiv preprint ar Xrxi::2006 15...051711 76, 2025b.
Bowen Zheng, Yupeng Hou, Hongyu Lu, Yu Chen, Wayne Xin Zhao, and Ji rong Wen. Adapting large language models by integrating collaborative semantics for recommendation. 2024 IEFE 40th International Conference on Data Engineering (ICDE), pp. 1435 1448, 2023.
Kai Zheng, Haijun Zhao, Rui Huang, Beichuan Zhang, Na Mou, Yanan Niu, Yang Song, Hongning Wang, and Kun Gai. Full stage learning rank: A unified framework for multi-stage systems. Proceedings of the ACM Web Conference 2024, 2024a.
70

---

Published in Transactions on Machine Learning Research (02/2026)
Lianmin Zheng, Liangsheng Yin, Zhiqiang Xie, Chuyue Livia Sun, Jeff Huang, Cody Hao Yu, Shiyi Cao, Christos Kozyrakis, Ton Stoica, Joseph E. Gonzalez, et al. SGLang: Efficient execution of structured language model programs. Advances in Neural Information Processing Systems, 37:62557 62583, 2024b.
Zexuan Zhong, Ziqing Huang, Alexander Wettig, and Danqi Chen. Poisoning retrieval corpora by injecting adversarial passages. In Houda Bouamor, Juan Pino, and Kalika Bali (ods.). Proceedings of the 2023 Conference on Empiricul Melhods in Natural Language Prucessing, pp. 13764- 13775, Singapore, December 2023. Association for Computational Linguistics. doi: 10. 10.18653//112223..em emmlpp main. //aclanthology.orge org/2023. mmnlp-main... 849/.
URL https:
Yujia Zhou, Yan Liu, Xiaoxi Li, Jiajie Jin, Hongjin Qian, Zheng Liu, Chaozhuo Li, Zhicheng Dou, Tsung- Yi Ho, and Philip S. Yu. Trustworthiness in retrieval-augmentedd generation systems: A survey, arXiv preprint arXiv:2409. 10102, 2024.
Ming Zhu, Aman Ahuja, Wei Wei, and Chandan K. Reddy. A hierarchical attention retrieval model for healthcare question answering. In The World Wide Web Conference, pp. 2472 2482, 2019.
Yutao Zhu, Huaying Yuan, Shuting Wang, Jiongnan Liu, Wenhan Liu, Chenlong Deng, Haonan Chen, Zheng Läu, Zhicheng Dou, and Ji-Rong Wen. Large language models for information retrieval: A survey. arXiv preprint Xiv:22300.. 07107,, 2023.
Honglei Zhuang, Zhen Qin, Rolf Jagerman, Kai Hui, Ji Ma, Jing Lu, Jianmo Ni, Xuanhui Wang, and Michael Bendersky. RankT5: Fine-tuning T5 for text ranking with ranking losses. In Proceedings of the 46th International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR '23, pp. 2308 2313, New York, NY, USA, 2023a. Association for Computing Machinery. doi: 011445/3539618.3555
Honglei Zhuang, Zhen Qin, Kai Hui, Junru Wu, Le Yan, Xuanhui Wang, and Michael Bendersky. Beyond yes and no: Improving zero-shot LLM rankers via scoring fine-grained relevance labels. In Kevin Duh, Helena Gomez, and Steven Bethard (eds.), Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 2: Short Papers), pp. 358-370, Mexico City, Mexico, June 2024a. Association for Computational Linguistics. doi: 10. 10.18653/v1 /2024.2222222 wwwp...aacl-ssor....
URL https: /aacl///cccaaa /aaaaaaaaaanttoooeoeee org/2024
Shengyao Zhuang, Hang Li, and Guido Zuccon. Deep query likelihood model for information retrieval. In Aduances in Information Retrieval: 43vd Furopean Conference TR Research, FCIR 2021, Virtual Event, March 28- A8ril 1, 2021, Proceedings, Part II 43, pp. 463-470. Springer, 2021.
Shengyao Zhuang, Bing Liu, Bevan Koopman, and Guido Zuccon. Open-source large language models are strong zero-shot query likelihood models for document ranking. In Houda Bouamor, Juan Pino, and Kalika Bali (eds.), Findings of the Association for Compulational Länguisties: EMNLP 2023, pp. 8807- 8817, Singapore, December 2023b. Association for Computational Linguistics. doi: 10.18653/v1/2022 findings-emnlp.a. URL https://aclanthhlls htw.e//nnntooogooo/////233333.3 findings-emmlp.590/
Shengyao Zhuang, Honglei Zhuang, Bevan Koopman, and Guido Zuccon. A sctwise approach for effective and highly efficient zero-shot ranking with large language models. In Proceedings of the 47th International
ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR '24, pp. 38 47, New York, NY, USA, 2024b. Association for Computing Machinery. doi:
Shengyao Zhuang, Xueguang Ma, Bevan Koopman, Jimmy Lin, and Guido Zuccon. Rank-R1: Enhancing reasoning in LLM-based document rerankers via reinforcement learning. arXiv preprint arXiv:2503.0603, 2025.
Justin Zobel, Alistair Moffat, and Kotagini Ramamohanarao. Inverted files versus signature files for text indexing. ACM Transactions on Database Systems (TODS), 23(4):553- -400, 1998.
71


Reviews:
[{"review_id": "kyhOE51M2M", "reviewer": "Reviewer_bEJ6", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper presents a comprehensive survey of model architectures for information retrieval ordered by the chronological order of their developments. More specifically, it covers simple term-based retrieval models (e.g. BM25), learning to rank models (e.g. RankNet), pre-BERT neural models (e.g. DSSM), pre-trained transformer-based models (e.g. DPR), LLM based models (e.g. LLM2Vec, RankGPT), and ends with a discussion on upcoming paradigms such as generative retrieval and agentic retrieval. It is generally well written with a nice overview visualized in Figure 1. The paper's key strengths lie in its comprehensive taxonomy that coherently traces the architectural evolution from traditional methods to modern LLM. The paper's key weakness is limited discussion on new paradigms and architectures like agentic search, graph RAG, compared to historical architectures. I have expanded more on the strengths and weaknesses in the responses below.", "claims_and_evidence_yn": "Yes", "claims_explanation": "Yes, the claims are well-supported by the literature. The authors accurately characterize the trade-offs in IR architectures. The paper provides extensive evidence in the form of comparative tables. Table 2 lists neural ranking models with their backbones (CNN, MLP) , and Table 4 lists LLM-based retrievers. These tables effectively support the claim of an architectural evolution.", "audience_yn": "Yes", "audience_explanation": "Since IR is a critical component of modern generative AI for e.g. RAG, grounded response, etc - ML researchers working on LLMs need to understand the architectural options for the retrieval component. The paper also serves as an entry point for researchers looking to understand the evolution of modern IR architectures. By tracing the history from simple term based retrieval to LLM-based ones, it explains why certain design choices (like late interaction) exist.", "requested_changes": "## Major\n- The paper discusses the past retrieval architectures in good detail but the discussion on new paradigms of IR (such as agentic search, graph RAG) is very limited. Since these are the upcoming IR methods - I believe there is more need to systematically organize literature for them.\n- Although the paper makes it explicit that the discussion is on model architectures of information retrieval instead of training methodologies, while reading the paper there are multiple places that I felt that core training details are needed to better ground the discussion on the models. For e,g. Table 3 lists a lot of bi-encoder models which differ primarily in how they are trained, a general overview of the typical training procedure and what components can be changed to get different methods will help in understanding the content better. Similarly for Table 4, 5.\n\n## Minor\n- At certain places the technical terms used is not explained for e.g. \"Attention-based methods like aNMM (Yang et al., 2016) perform soft-attention alignment\" - the exact meaning of \"soft attention alignment\" is hard to parse without explanation.\n- In the introduction paragraph, the key statement \"Research in IR primarily centers on two key aspects: (1) extracting better query and document feature representations, and (2) developing more accurate relevance estimators\", needs to be further expanded. Even with background in IR, it was hard for me to understand what exactly is meant by developing more accurate relevance estimators. The next paragraphs directly go into the methods of (1) and (2) before establishing a clear meaning of (1) and (2).\n- On page 14 - the claim that inverted index is always faster than dense vector search is not necessarily true, depending on the scenario and hardware used dense vector search can be more efficient.\n- Please standardize the use of LaTeX quotes (`` '') throughout the draft. Currently, standard quotation marks (\" \") are used inconsistently", "broader_impact_concerns": "As a survey paper I do not anticipate any significant negative impact of this paper.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "jDF7lp02Mw", "reviewer": "Reviewer_UGkB", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper provides a comprehensive survey of recent advancements in model and network architectures for Information Retrieval (IR). The writing is clear, and the overview of both foundational and recent architectures is extensive. Additionally, the paper raises some pertinent questions for future research.\n\nA. The paper traces the evolution of model architectures for IR:\n1. **Traditional Models**: Section 3 describes Boolean retrieval, TF-IDF, BM25, vector space, and probabilistic models, and critically assesses their limitations in contemporary settings.\n2. **Learning to Rank (LTR)**: Section 4 covers Learning to Rank (LTR) models including RankSVM, LamdbaRank, and LambdaMART. Inclusion of more architectural specifics, particularly for LambdaMART, would strengthen this section.\n3. **Neural Ranking Models**: The survey classifies neural models into representation based and interaction based at high level (Section 5). \n- CNN, RNN/LSTM and Word embedding based models are discussed for representation-based models. This section should include details on pre-transfer attention. For example, [1] uses attention over word and phrase representations in a hierarchical fashion to learn enhanced document representations. \n- MatchPyramid, DRMM, Kernel-based ranking models are discussed for interaction based neural ranking models. This section also includes the differentiation on the basis of parametric / non-parametric model.\n4. **Hybrid Models**: Hybrid model lists models that make use of representations and interaction-based components. Extreme multi-label learning approaches could be included in this category. For example, AttentionXML [2] employs interaction between query-documents. DEXA [3] introduces free parameters (in addition to textual features) to enhance the document representations (as per this paper's terminology).  \n5. **Transformer-Based Architectures**: The modern transformer-based architectures can be used in a variety of ways: \n- Off-the-shelf encoder-based (BERT), decoder-based (LLAMA, GPT, Mistral, NV-Embed) and encoder-decoder (T5, BART) models can be used to compute query-document representations for the IR task. \n- These models can also be trained for a specific task at hand. The paper seems to be missing on Sentence-BERT [4], which is a crucial paper in adapting BERT style models to learn effective sentence representations for IR. Some discussion should be included on ModernBERT [5] and [6] as well.\n- Transformer based architectures can also be used to \"learn\" high dimensional sparse representations (e.g. SPLADE). Such representations can provide competitive performance to dense embeddings while allowing for interpretability. \n- Finally, ColBERT and similar methods deploys multiple vector representations for a single document for improved performance.\n6. **Ranking with LLMs**: LLMs can be used a re-ranker both in zero/few-shot or fine-tuned fashion. A discussion on pairwise and listwise ranking is included in context of LLM as ranker. \n7. **Generative Retrieval Paradigms**: Generative retrieval paradigm tackles the IR problem in one go by directly generating document identifiers (GENRE and DSI). While promising, these approaches are yet to effectively scale to large datasets with millions or billions of documents.\n\nB. The paper also includes a discussion on upcoming challenges and raises some very important questions:\n1. While most IR research targets human end-users, emerging paradigms (e.g., RAG) may introduce unique challenges that current evaluation metrics may not capture.\n2. The survey rightly notes the growing importance of robustness, bias (especially with AI-generated content), toxicity, and data distribution as core evaluation issues moving forward. \n\nReferences:\n\n[1] https://aclanthology.org/N16-1174/\n\n[2] https://arxiv.org/abs/1811.01727\n\n[3] https://dl.acm.org/doi/10.1145/3580305.3599301\n\n[4] https://arxiv.org/pdf/1908.10084\n\n[5] https://arxiv.org/abs/2412.13663\n\n[6] https://openreview.net/forum?id=SyK00v5xx", "claims_and_evidence_yn": "Yes", "claims_explanation": "This is a survey paper which focuses on reviewing existing works and throwing light on future challenges. Appropriate papers are cited when discussing various architectures in the IR community. A small discussion is also included when required which provides an overview of the advancements. Incorporating the discussion points and relevant papers suggested above would strengthen the paper.", "audience_yn": "Yes", "audience_explanation": "Retrieval is one of the key components of several real-world applications/systems including search, document tagging, and RAG. So, a survey paper discussing the advancements in this domain should be helpful to anyone in the IR community. Moreover, the points raised regarding evaluation, robustness, and bias in context of retrieval for AI systems should be interesting for a larger audience as well. For instance, while a human-written and a machine-generated document may achieve similar relevance scores, user preferences can still diverge significantly.", "requested_changes": "1. The paper discusses the architecture in context of accuracy. However, the architecture preferences may change depending on the real-world constraints. A discussion on trade-offs between accuracy and efficiency can be helpful.\n2. Calibration can be a crucial component in several real-world applications including product recommendation. The paper should look into the calibration of confidence estimation techniques in context of model architectures.  \n3. It may be helpful to discuss some applications of IR systems e.g., web search, document/product search [1][2], product recommendation, document tagging, and Ads [3]. This will provide some motivation/context behind progress in IR architectures.\n4. Please see the missing discussion listed in the Summary sections.  \n\n[1] https://www.microsoft.com/en-us/research/wp-content/uploads/2016/02/cikm2014_cdssm_final.pdf\n\n[2] https://arxiv.org/pdf/1907.00937\n\n[3] https://proceedings.mlr.press/v139/dahiya21a.html", "broader_impact_concerns": "NA", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "ffMnNt9KGH", "reviewer": "Reviewer_QkY4", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper presents a survey of Information Retrieval (IR) model architectures for relevance estimation, a central and important topic in the field. The survey follows the advancement of model architecture from 1) classical boolean and statistical lexical models (boolean, VSM, SLM) to 2) feature-engineered learning-to-rank and early neural models to 3) language-model representation-based approaches through to 4) LLM applications in existing (encoders, rerankers) and new (generative retrieval) solutions. \n\n**Strengths**\n- The literature coverage is exceptionally thorough and the summaries of each research strand (at the level of subsection), spanning seminal works to vestigial extensions, are accurate and mostly well-synthesized. \n- It also contextualizes the advancements within the history of adjacent technological advancements (especially feature/representation-learning) well, motivating the transitions from section to section.\n- Section 8.2 on The Shifting Paradigm is particularly well-timed and descriptive.\n\n**Weaknesses**\n- The chronological top-level framing makes it difficult to see conceptual continuities across time. Grouping methods primarily by structural axes (e.g., interaction efficiency vs expressivity, ranking objective, pipeline stage) might better highlight architectural trade-offs and open research gaps, providing greater value to readers seeking to build on existing work. This leads to redundancies/nonlocalities like the many similarly-architected methods across sec 5-6, or discussions (pointwise/pairwise/listwise or decoder-only-architecture) occurring sections before they're most relevant (LLM section) but subsequently less emphasized. \n- The survey's scannability suffers at the sub-section level. Key taxonomic distinctions, such as the contrast between atomic and semantic identifiers for generative retrieval, are often embedded in dense prose.\n- Tables 3, 4, and 5 have some confusing categories, reducing their interpretability and utility. E.g.:\n\t- What is the difference architecturally between LM, LLM, LLM Encoder, etc? \n\t- What is the difference between Listwise Loss, Listwise Prompting, and Listwise?", "claims_and_evidence_yn": "Yes", "claims_explanation": "Yes. The survey is comprehensive and accurately cites relevant literature. However the evidence (survey of architectural methods) could be made clearer and more valuable to readers by addressing the framing and scannability issues mentioned in the weaknesses section.", "audience_yn": "Yes", "audience_explanation": "Yes. IR is well-represented in TMLR publications, and within that, model architecture is a topic of continual interest and publication within the community. This survey would be helpful to researchers looking to review old retrieval architectures or identify gaps for novel research.", "requested_changes": "**Critical to Acceptance**\n- As mentioned in the weaknesses section, the extraction of insights from this survey is inhibited by 1) the splitting of similar architectural features across sections due to chronology and 2) subsections of mostly dense prose. To address both without a section-level rewrite, the authors might consider adding more figures, tables, or even text formatting (bolding, hierarchy) to better surface and taxonomize the core architectural methods.\n\n**Would Strengthen the Work**\n- Move and expand the section on multimodal and multilingual architectures out of the \"Emerging Directions and Challenges\" section and into the body of the survey. Both are frequently addressed with architectural modifications (e.g. CLIP, per-language-modules) and future work would benefit from a more comprehensive review of their methods.\n- Similarly, deeper discussion of instruction following encoders and reasoning rerankers (if these can be considered architectural modifications - prompts/reasoning) would make the survey more up-to-date at the time of publication.\n- Revisit Figure 3. As is, it is hard to tell that the reranker jointly attends to both query and document, or that the solid blocks are vectors and the colored w's are (sparse) terms/weights.\n\n**Misc.**\n- [typo] page 4, **Query and Document** paragraph: \"documnent\"\n- [typo] citations: OpenAI Introducing \"deeo\" research\n- [notation] page 7, end of first paragraph: \"(e.g. NDCG, ERR)\", in section 2 you use \"nDCG\" and \"MRR\" which are more standard.\n- [notation] there's inconsistent use of the \"retrieve-then-rerank\", sometimes it's \"retrieve-then-rank\" or \"index-retrieval-rank\". My vote would be the former because first stage retrieval generally already induces some ranking.\n- [notation] page 20, first paragraph: \"in-pars\" isn't widely accepted as *the name* for such a generic method. \"Synthetic training data\" or \"synthetic triplets\" is more descriptive and less of a anachronism than \"Inquisitive Parrots for Search\"\n- [table] table 1: Neural nets _are_ ML, what's the need for the \"Model\" column if the \"Backbone Architecture\" column carries strictly more info?\n- [table] table 3: \"The \"late interaction\" mechanism ... can be considered a special case of multi-vector architecture\" - isn't it the exemplar? Do you mean \"special case of dense retrieval\" (I've heard that comment/framing made before)", "broader_impact_concerns": "N/A", "additional_comments": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Decision: Accept (poster)