# Towards Lossless Token Pruning in Late-Interaction Retrieval Models

Yuxuan Zong

Sorbonne Université, CNRS, ISIR

Paris, France

yuxuan.zong@isir.upmc.fr

## Abstract

Late interaction neural IR models like ColBERT offer a competitive effectiveness-efficiency trade-off across many benchmarks. However, they require a huge memory space to store the contextual representation for all the document tokens. Some works have proposed using either heuristics or statistical-based techniques to prune tokens from each document. This however doesn't guarantee that the removed tokens have no impact on the retrieval score. Our work uses a principled approach to define how to prune tokens without impacting the score between a document and a query. We introduce three regularization losses, that induce a solution with high pruning ratios, as well as two pruning strategies. We study them experimentally (in and out-domain), showing that we can preserve ColBERT's performance while using only 30% of the tokens.

## CCS Concepts

* Information systems → Document representation; Retrieval models and ranking; Information retrieval.

## Keywords

Information Retrieval, Dense Retrieval, Multi-vector Retrieval, Efficiency-Effectiveness Trade-off,

## ACM Reference Format:

Yuxuan Zong and Benjamin Piwowarski. 2025. Towards Lossless Token Pruning in Late-Interaction Retrieval Models. In *Proceedings of the 48th International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR '25), July 13–18, 2025, Padua, Italy*. ACM, New York, NY, USA, 11 pages. https://doi.org/10.1145/3726302.3730100

## 1 Introduction

Pre-trained contextualized language models based on the Transformer architecture, such as BERT, significantly improved retrieval effectiveness over the previous state-of-the-art methods in information retrieval (IR) like e.g. BM25 [42]. Among these models, dual encoders such as sparse [11, 14] or dense [24] models, have the best efficiency while cross-encoders, e.g. MonoBERT [39], have the best effectiveness.

Another category of models, called late interaction models, like ColBERT [25, 45], offer a good efficiency-effectiveness trade-off, as

<figure><img src="image_1.png" /></figure>

This work is licensed under a Creative Commons Attribution-NoDerivatives 4.0 International License.

SIGIR '25, July 13–18, 2025, Padua, Italy

© 2025 Copyright held by the owner/author(s).

ACM ISBN 979-8-4007-1592-1/2025/07

https://doi.org/10.1145/3726302.3730100

Benjamin Piwowarski

CNRS, Sorbonne Université, ISIR

Paris, France

benjamin.piwowarski@cnrs.fr

<figure><img src="image_2.png" /><figcaption>**Figure 1:** Illustration of the concept of dominance. In this example, $d_3$ is dominated by $d_1$ and $d_2$. $d_3$ can be removed from the document representation without changing a modified ColBERT scoring function (see Section 3.2). Despite its low norm, $d_2$ is kept since it brings new information about the relevance of the document. The hashed area corresponds to the half-space where the inner product of any vector with $d_3$ is negative. The gray area is discussed in Section 3.1.</figcaption></figure>

they are halfway between dual encoders and cross-encoders. Specifically, unlike dual encoders, late interaction models use embeddings at the token level rather than to represent the entire query or document. In ColBERT, the score of a document for one query token is defined as the maximum of the inner product over all the document tokens representations with the query token representation. This token-level granularity allows for finer semantic matching and improved retrieval accuracy. Moreover, compared to the cross-encoder mechanism like MonoBERT [39], this "late interaction" mechanism provides a way to build an index of a whole dataset by storing the document token representations, or *document vectors* in short.

However, storing the vectors of the documents requires a large index space compared to dual encoding approaches. For example, for MS MARCO v1 [2], composed of 8.8M passages, the ColBERT index takes 142 GiB. PLAID [44], which is a highly optimized version of ColBERTv2, requires more than 20 GiB of space. This is in contrast to the 13 GiB needed to store the 768-dimensional vectors of a dense model (without any compression, using FP16). Furthermore, a large index leads to expensive computational costs; as in ColBERT's retrieval pipeline, the retrieval cost scales linearly with the query and document's length.

To reduce the index size, some works have proposed to prune tokens from the document. Some existing works [1, 27, 33] propose to use static methods based on heuristic or statistical approaches,
but they degrade ColBERT performance with high pruning ratios, greater than 50%. Some other works [20, 40] learn to discard some tokens using a gating mechanism, and hence aim at learning sparse (at token level) document representation. These methods remove some of the unimportant tokens, but do not consider how to remove tokens while having a minimum impact on the ColBERT score.

This can be a problem, as illustrated in Figure 1, where the document vector $d_2$, while having the lowest norm, can still match a query token – i.e., the inner product with a query vector located in the green area will be maximized by $d_2$. In the same document, $d_3$ could, however, be pruned without changing the score of the document, since either $d_1$, $d_2$ or $d_4$ would match any query token located in the blue, red or green areas.

In this paper, we first formally define this *lossless pruning strategy* and show that it is equivalent to a linear programming problem. We then propose different training methodologies and approximate pruning strategies that can be used in practice. Finally, through our experiments, we analyze the performance of our proposed approach considering different pruning strategies and regularizations, leading to the removal of 70% tokens and a performance drop of less than 1.5% on in-domain datasets. For the out-of-domain evaluation task, we also end up with a 65% token removal with less than a 3% drop in the performance on BEIR task (1.1% of performance drop in the LoTTE search). The code used for the experiments in this paper can be accessed online<fnref n="1" />.

In the following, we report the research questions that are tackled in this paper, as well as our different contributions.

**RQ1.** *Is it possible to define a late interaction model and a pruning algorithm that guarantees that the score of a document does not change whatever the query?* In Section 3, we formally define the concept of “dominance” whereby a document vector can be removed from a document’s representation without impacting a late interaction scoring function. More precisely, in Section 3.1 we discuss how to modify the ColBERT scoring function to allow for more pruning without impacting its performance, and in Section 3.2, we show how the problem of pruning document tokens with the guarantee of not modifying the score of the document, that is, *lossless pruning*, is equivalent to a Linear Programming problem [38].

**RQ2.** *How can this motivate regularization (during training) and practical pruning strategies (before inference), and what is the effectiveness/efficiency trade-off of our overall methodology for late-interaction models?* In Section 4, we propose three regularizations (so that more document tokens are dominated) as well as two pruning strategies that help to increasing the pruning ratio. In Sections 6.1 and 6.2, we conduct extensive experiments based on these strategies that show that we can achieve a competitive efficiency-effectiveness trade-off.

**RQ3.** *How is our proposed regularization and the pruning strategy perform on out-of-domain datasets?* As regularization might prevent the model from generalizing to out-of-domain datasets, we conduct extensive experiments on a subset of the BEIR and LoTTE datasets in Section 6.3. Although the difference with non-pruned ColBERT is greater than in the in-domain scenario, we still outperform other

pruning approaches and achieve a good efficiency-effectiveness trade-off.

**RQ4.** *Is it better to train from an MLM or from an IR checkpoint?* As the proposed regularizations have an impact on the geometry of space, we question whether starting with an IR checkpoint, i.e. ColBERTv2, is better suited for pruning than starting from an MLM one. In Section 6.4, we show that the impact is limited.

## 2 Related Work

*Multi-vector dense retrieval.* Multi-vector dense retrieval (MVDR) can be seen as a generalization of single-vector dual encoder models. Instead of encoding the full content of a query or document into a single vector of the representation space, MVDR uses fine-grained token-level modeling for scoring. Neural IR models using this type of strategy are called interaction models. The first models were based on non-contextualized token (or word) embeddings, and relied on complex post-processing of the interaction between document and query vectors to accommodate the lack of token contextual encoding [17, 22, 47].

With transformer-based architectures, since token representations are contextualized, MVDR models rely on a much simpler interaction mechanism. Models such as ColBERT models [25, 45], XTR [28], COIL [8, 14], ME-BERT [34], MVR [48], and CITADEL [31] compute query-document relevance by computing, for each query token, the highest inner product of its representation with that of a document token representation – see Eq. (1) in Section 3. Some other works like AlignER [40], MVR [48], ME-BERT [34] propose that a query token could match multiple document tokens in the scoring stage. Although this improves the results, the use of only one token (argmax) is competitive and simplifies the retrieval architecture. We thus stick to the maximum inner product (unary salience) in this work.

*Reducing the index size in MVDR models.* Although MVDR models are more powerful than dual encoders, this, however, comes at a cost during inference. Late interaction methods [40, 45] are based on a two-stage (or more) retrieval, where the first candidate documents are retrieved using vector indices such as Faiss [23]. To avoid having too many vectors in the index, some works have proposed to first cluster them and index only the cluster centroids [37, 44, 45] – which amounts to selecting candidate documents before scoring them by loading the full document representations. To avoid this re-ranking step, some approaches like SLIM and MUVERA [7, 30] propose to directly rely on cluster-based representations, which avoids the complex multistage retrieval and increases the inference speed – but at the cost of decreasing the performance.

A complementary method to reduce index size is to directly prune document tokens before storing them [1, 20, 27, 33, 40]. Some of these methods are based on heuristics, such as pruning based on stopwords [1], IDF value [1, 33], attention score [27, 33], or position [27, 33]. Specifically, for attention score-based methods [33], a token is important if many tokens are similar to it but not the other way around (i.e., tokens whose representation is close to a centroid of clusters of document vectors). These attention-score methods bear some similarity with our model, but are not based on any training and might still discard important tokens. These

<footnote n="1">¹https://github.com/yzong12138/MVDR_pruning</footnote>
static strategies (the Transformer model is kept unchanged) gen-
erally lead to a low efficiency-effectiveness trade-off. For example,
repeated tokens such as “doctor” have the same IDF value, but keep-
ing all of them might not provide supplementary information to
the scoring function.

To further push the efficiency-effectiveness trade-off, some works
propose *dynamic* strategies in which the model learns which to-
kens should be pruned. AligneR [40] employs entropy-regularized
optimization (relaxed top-k optimization), and ColBERTer [20] re-
duces the number of vectors by combining sub-word vectors into
unique whole-word representations and learns to discard them
through a trainable gate vector. Similarly, He et al. [18] propose to
learn if a vector should be “keep” or “drop” using Gumbel-Softmax
sampling. Const-BERT [36] uses a pooling layer that projects docu-
ment vectors to a smaller set of vectors (constant cardinality). Our
work follows this line of research, but we explore more principled
ways to prune tokens, which in theory can lead to lossless pruning
strategies and which work better in practice.

3 Pruning ColBERT – Theory

In this section, we first show how to adapt ColBERT so that more
tokens can be pruned and define the concept of *dominance* as well
as its equivalence with a linear programming problem.

3.1 Adapting ColBERT for Pruning

ColBERT uses an encoder to represent document and query tokens separately, before computing a score based on the inner product between query and document token representations. More formally, given a document **D** and a query **Q**, each document (resp. query) token is represented by a vector **d** ∈ **D** (resp. **q** ∈ **Q**) of dimension **d**. The score of a document **D** for a given query **Q** is given by the following formula:

$$
\text{ColBERT}(\mathbf{Q}, \mathbf{D}) = \sum_{\mathbf{q} \in \mathbf{Q}} \max_{\mathbf{d} \in \mathbf{D}} \mathbf{q} \cdot \mathbf{d} \quad (1)
$$

In this equation, as well as in the remainder of the paper, **D** denotes both a matrix (where rows are the dimension of the latent space and columns the tokens of the document) and a set of vectors in the latent space – given the max operator of the scoring function, the two formulations are in practice equivalent. We also shorten the *phrase document* (*resp. query*) *token representation* by *document* (*resp. query*) *vector*.

To prune tokens, we need to find a subset $D^+ = D \setminus D^- \subseteq D$
such that

$$
\forall \mathbf{q} \in \mathbb{R}^d, \max_{\mathbf{d} \in \mathcal{D}} \mathbf{q} \cdot \mathbf{d} = \max_{\mathbf{d} \in \mathcal{D}^+} \mathbf{q} \cdot \mathbf{d}
$$

In ColBERT, the query and the document vectors have unit norm
(||**q**|| = ||**d**|| = 1), that is, the model implicitly computes the cosine
between the vectors **q** and **d**. In this setting, unless two document
tokens have *exactly* the same representation, there is no way to
remove any token **d**⁻ from **D** without impacting this maximum –
since with **q** = **d**⁻, we have **q** · **d**⁻ > **q** · **d** for any **d** ∈ **D** \ {**d**⁻}.

To allow for more tokens to be removed, we propose to modify
this ColBERT scoring function by the following:

$$
\mathrm{ColBERT}_P(\mathbf{Q}, \mathbf{D}) = \sum_{\mathbf{q} \in \mathcal{Q}} \max_{\mathbf{d} \in \mathcal{D}} [\pi_{\theta}(\mathbf{q}) \cdot \pi_{\theta}(\mathbf{d})]_{+} \quad (2)
$$

where $\pi_\theta$ is a projection operator and $[x]_+ = \text{ReLU}(x) = \max(0, x)$.
To simplify the notation, in the remainder of the paper, we overload
$\mathbf{q}$ and $\mathbf{d}$ as representations after the projection $\pi_\theta$.

Compared to Eq. (1), the first modification is to use a projection
$\pi_{\theta}$ of the document/query tokens. This ensures that their norm be
less or equal to 1, so that some tokens can be pruned on the docu-
ment side. On the query side, this might help the model cope with
more or less important tokens, but this is not strictly required for
pruning. The second modification we make is to only consider pos-
itive inner products – which in practice is already most of the time
true in ColBERT because of the anisotropy of the representation
space [16].

Both of these changes are needed to remove some tokens from
the document without affecting the ColBERT<sub>p</sub> score. This is il-
lustrated in Figure 1, where the document token that reaches the
maximum inner product with a query **q** is indicated by a color code,
and where the inner product with **d**<sub>3</sub> is negative for any query
located within the half-space indicated by the hatched area. In the
other half-space, **d**<sub>1</sub>, **d**<sub>2</sub> or **d**<sub>4</sub> have a higher inner product with any
query vector **q** than **d**<sub>3</sub>. Note that without the ReLU transformation,
**d**<sub>3</sub> could have a maximum inner product (gray zone) below zero,
since the magnitude of the inner product with **d**<sub>3</sub> is less than with
other document vectors in this area.

AligneR [40] and ColBERTer [20] do use another way to reduce
the norm of a document vector, using a gating mechanism by pre-
dicting a scalar in [0, 1] that is multiplied by the document vector at
hand. Our projector $\pi_{\theta}$ reduces not only the vector norm and adds
the ability to discard document vectors. This is important, especially
when the norm is reduced but not to (a numerical) zero. Returning
to Figure 1, according to the pruning scenario based on the norm
(e.g., salience in the notation of AligneR) of the vectors [20, 40],
$d_2$ should be pruned instead of $d_3$. With our approach, $d_3$ can be
pruned without impacting the scoring function.

3.2 Dominance and Linear Programming

*Local and Global Dominance.* Based on Eq. (2), we propose the following definition that allows removing tokens from a document without changing the ColBERT<sub>P</sub> score.

DEFINITION 1 (LOCAL DOMINANCE). A vector $\mathbf{d}^{-} \in \mathbb{R}^{d}$ is domi-
nated by a set of vectors $\mathbf{D}$, which we denote by $\mathbf{d}^{-} < \mathbf{D}$, if and only
if for any $\mathbf{q} \in \mathbb{R}^{d}$,

$$
\text{either } \mathbf{q} \cdot \mathbf{d}^{-} \le 0 \text{ or } \exists \mathbf{d}^{+} \in \mathcal{D} \text{ such that } \mathbf{q} \cdot \mathbf{d}^{+} > \mathbf{q} \cdot \mathbf{d}^{-} \quad (3)
$$

In this case, it is easy to show that for any query vector **q**,

$$
\max_{\mathbf{d} \in \mathcal{D} \cup \{\mathbf{d}^{-}\}} [\mathbf{q} \cdot \mathbf{d}]_{+} = \max_{\mathbf{d} \in \mathcal{D} \setminus \{\mathbf{d}^{-}\}} [\mathbf{q} \cdot \mathbf{d}]_{+}
$$

Based on this definition, we can partition a set **D** of vectors into
**D**⁺ and **D**⁻, by defining **D**⁻ as the set of all the vectors **d** dominated
by **D**. The question remains whether any vector in **D**⁻ is dominated
by **D**⁺. If this is true, we only need to test for *local dominance* to
define the whole set of vectors that can be pruned without impacting
the ColBERT<sub>p</sub> score. The following lemma proves this.

LEMMA 1 (GLOBAL DOMINANCE). Let $D^+$ and $D^-$ be a partition of the set of document vectors $D \subset \mathbb{R}^d$ such that any document vector
$d \in D^-$ is dominated by D. Formally,

$$D^{-} = \{d \in D | d < D\} \text{ and } D^{+} = D \setminus D^{-}$$

Then, for any document $d^- \in D^-$, $d^-$ is dominated by $D^+$.

**PROOF.** To prove this lemma, we choose an arbitrary $q \in \mathbb{R}^d$ and $d_1 \in D^-$, and show that there exists $d^+ \in D^+$ such that $q \cdot d^+ > q \cdot d_1$.

First, for any $d_k \in D^-$, let us denote $D_q(d_k)$ a subset of $D^-$ such that any document vector in $D_q(d_k)$ have a higher inner product with $q$ than $d_k$, i.e.

$$D_q(d_k) = \{d \in D^- | d \cdot q > d_k \cdot q\}$$

If for some $k$, $D_q(d_k) \neq \emptyset$, we can choose $d_{k+1} \in D_q(d_k)$. By construction, $D_q(d_{k+1}) \subset D_q(d_k) \subset D^-$ and $q \cdot d_{k+1} > q \cdot d_k$. As $D^-$ is finite, by induction we can conclude that for some $k^*$ we have $D_q(d_{k^*}) = \emptyset$.

Since $d_{k^*} < D$, this implies that $\exists d^+ \in D \setminus D^- = D^+$ such that $d^+ \cdot q > d_{k^*} \cdot q$. This in turn implies that there exists $d^+ \in D^+$ such that

$$q \cdot d^+ > q \cdot d_{k^*} > \dots > q \cdot d_1$$

Since this holds for any query vector $q$, this implies that $d_1 < D^+$, which concludes the proof. □

**Exact pruning and Linear Programming.** The above lemma implies that we only need to test local dominance to construct $D^+$, i.e. to find the vectors in $D$ that verify Eq. (3). We show that this corresponds to a Linear Programming (LP) problem below by using Farka's lemma [15].

Let us first state the version of Farka's lemma that we use.

**LEMMA 2 (FARKAS' LEMMA).** Let $A \in \mathbb{R}^{d \times n}$ and $b \in \mathbb{R}^m$, then exactly one of the following assertion is true (the vector inequalities mean that they must hold for any component of the vector):

(1) There exists $x \in \mathbb{R}^n$ such that $Ax = b$ and $x \ge 0$.

(2) There exists $y \in \mathbb{R}^d$ such that $A^T y \ge 0$ and $b^T y < 0$.

Setting $A = (d - d_1 \quad \dots \quad d - d_n)$ and $b = -d$, where $d$ is our target document vector, we can use the lemma to show that if

$$\exists x \in \mathbb{R}^n, (d - d_1 \quad \dots \quad d - d_n) x = -d \text{ and } x \ge 0 \quad (4)$$

then for any query vector $q$ either $d^T q \le 0$, or there exists $i$ such that

$$(d - d_i)^T q < 0 \Leftrightarrow d \cdot q < d_i \cdot q$$

which corresponds exactly to the definition of local dominance.

We therefore have shown that the problem of determining whether a document token is dominated is equivalent to a linear programming (LP) problem which can be solved using linear programming solvers [38]. However, LP problems are computationally expensive to solve. The complexity of the simplex algorithm is not guaranteed to be polynomial in the worst case [9], and there is still ongoing work to improve the time complexity of other approaches, such as the methods of interior points [5, 29].

To reduce the computational cost, we describe below simple techniques to determine whether $d$ belongs to $D^+$ or $D^-$, before using LP solvers for the remaining ones.

First, according to the lemma 1 that links local to global dominance, if $d^-$ is dominated, then whatever the query vector $q$, we can find $d^- \in D^+$ that dominates it, that is, for which $q \cdot d^+ > q \cdot d^-$.

This shows that whenever we find a dominated vector $d^-$, we can directly remove it from $D$ for the remaining local dominance tests.

Second, we compute the inner product of each document vector $d$ with the others and look at whether the vector matches itself, that is, whether $d \cdot d \ge d \cdot d'$ for $d \in D$. If this is the case, we know for sure that $d \in D^+$. In practice, between 20 and 30% of “dominating” tokens can be identified by this simple strategy.

# 4 Methodology

In this section, we describe a methodology for training and pruning in practice our ColBERT<sub>P</sub> model, based on the dominance theory we described in the previous section.

## 4.1 Approximate Pruning

The LP-based process described in Section 3.2 is exact but has two main limitations:

* Some vectors might have a very small component that prevents them from being dominated;

* Solving an LP problem is time-consuming.

To tackle the first problem, we propose using dimensionality reduction; for the latter problem, we propose using a very simple heuristic, based on the norm of the vectors, and we experimentally study how well it approximates the LP solution.

### 4.1.1 LP-based Pruning with Reduced Dimension.
Based on the dominance analysis in the last section, a document token $d \in D$ should be pruned if, for any query vector $q$, we can find $d^+ \in D$ such that $d^+ \cdot q > d \cdot q$. Then a requirement for local dominance is that $d$ is in the span of vectors in $D \setminus \{d\}$. However, this rarely happens in practice because of the MLM pre-training that induces a strong prior on the model, that of trying to preserve unique token semantic information.

We hence propose to project the vectors in a lower-dimensional space by using truncated singular value decomposition (SVD). Using a truncated SVD allows for preserving dimensions which are the most common between the vectors, and thus discarding the vector residuals that prevent vectors from belonging to the same subspace.

Formally, given the reduced singular value decomposition of our input document $D = U\Sigma V^T$, with $U \in \mathbb{R}^{d \times m}$, $\Sigma \in \mathbb{R}^{m \times m}$ a diagonal matrix with decreasing and non-negative diagonal components, and $V \in \mathbb{R}^{m \times n}$ with $m \le \min(d, n)$, we can write a document vector $d_i$ as $U\Sigma v_i$ with $v_i$ the column $i$ of $V^T$. For any query $q$, we have

$$q \cdot d_i = q^T U \Sigma v_i = (U^T q)^T (\Sigma v_i)$$

The above equation shows that the local dominance problem can be solved within a space of dimension $m \le d$, by posing $q' = U^T q \in \mathbb{R}^m$ since $q$ is arbitrary. This has an impact on matrix conditioning, which helps stabilize the LP algorithm [3, 4, 12], and also allows to speed up the overall local dominance decisions. Formally, the problem is equivalent to

$$\bar{D} = \Sigma (v_1 \quad \dots \quad v_n)$$

We can push this further by making the decision of local dominance approximate. Denoting $\Sigma_k$ the first $k < m$ columns of $\Sigma$ and $v_i^{(k)}$ the first $k$ components of $v_i$, we know that $U\Sigma_k v_i^{(k)}$ is the best approximation of $d_i$ in a $m$-dimensional subspace (in terms of Frobenius distance). We therefore propose to use $\bar{D}^{(k)} =$
$\Sigma^{(k)} ( v_1^{(k)} \dots v_n^{(k)} )$ instead of $\mathbf{D}$ as the document matrix, which helps to remove the barely used dimensions (and thus increases the number of dominated vectors).

In practice, we use $k$ such that most of the singular values are covered by $\Sigma_k$, that is,

$$ \frac{\mathrm{tr}(\Sigma) - \mathrm{tr}(\Sigma^{(k)})}{\mathrm{tr}(\Sigma)} \geq \theta_{LP} $$

where $\theta_{LP}$ is an arbitrary bound. Note that dimensionality dimension is only used to prune the vectors – i.e. we use the non-pruned vectors $\mathbf{d} \in \mathbf{D}$ to represent a document.

### 4.1.2 Norm Pruning.
A lightweight alternative to LP-based pruning is to rely on norm-based pruning, such as in AlignER [40] or ColBERTer [20]. As having more dominated vectors implies having more low norm vectors, it is natural to question whether using a norm-based strategy for pruning is interesting in our case. Compared to the other methods, the pruning based on the norms is fast and the impact on the score is directly related to the norm of the document vector. In our experiment, we denote the $\theta_N$ as the threshold for this pruning strategy, where it removes all document tokens that have a norm smaller than this threshold.

However, the fact that other document vectors might be closely related – or not – to a document vector with a low norm is not taken into account. This is illustrated in Figure 1, where $\mathbf{d}_3$ is not pruned as it should and $\mathbf{d}_2$ is pruned but might have an impact on the retrieval score.

## 4.2 Regularization

Using an already trained ColBERT model and pruning the document vectors as described above might not work well because the model is not inherently biased toward producing document representations that are prone to high pruning ratios – even if not using the cosine product. As the pruning strategies that rely on the LP problem are not easily differentiable, we propose three different regularizations that can be added to the IR-related loss so that the resulting set of document vectors can be pruned more effectively.

**Nuclear norm regularization.** To force some more document tokens to dominate, we need to ensure that the document vectors are in the span of each other. Given the matrix $\mathbf{D}$, this is linked to the problem of minimizing the nuclear norm of $\mathbf{D}$, which is formally defined as the $L_1$ norm of the singular values of the matrix, i.e., trace $(\Sigma)$ using the notation defined in Section 4.1.1. As $L_1$ pushes some components to zero, this reduces the size of the subspace where the document vectors in $\mathbf{D}$ lie, and also puts pressure on reducing the norm of the document vectors. Both are linked to a higher pruning ratio. Formally, our regularization is defined as

$$ \mathcal{L}_{r}^{(\text{nuc})} = \frac{1}{\min(n, d)} \|\mathbf{D}\|_{\text{nuc}} \quad (5) $$

where $n$ is the number of document vectors and $d$ the dimension of the token representation space.

**Document tokens similarity matrix regularization.** Although the nuclear norm pushes the document vectors to a small-dimensional subspace, it has two problems. First, its computation is time-consuming. Second, it does not really ensure that some document vectors are dominated.

We therefore propose to rely on a simpler measure based on the inner product between $\mathbf{d}$ and the other vectors $\mathbf{d}'$ of $\mathbf{D}$, whereby we want to maximize the inner product $\mathbf{d} \cdot \mathbf{d}'$ between $\mathbf{d}$ and $\mathbf{d}'$. We then modify this inner product to match our needs:

* As we don't want the norm of $\mathbf{d}$ to increase (which would reduce its likelihood to be pruned), we divide the inner product by the norm of $\mathbf{d}$;

* We only consider positive inner products to avoid the norm of $\mathbf{d}'$ to be reduced when maximizing the inner product;

* Finally, we want more pressure on low norm document vectors $\mathbf{d}$;

Putting all together, this leads to the following regularization (we take the opposite since we frame it as a minimization problem):

$$ \mathcal{L}_{r}^{(\text{sim})} = - \frac{1}{n(n-1)} \sum_{\mathbf{d} \in \mathbf{D}} (1 - \|\mathbf{d}\|_2) \sum_{\mathbf{d}' \in \mathbf{D} \setminus \{\mathbf{d}\}} \frac{[\mathbf{d} \cdot \mathbf{d}']_+}{\|\mathbf{d}\|_2 + \varepsilon} \quad (6) $$

where $n$ is the number of document vectors and $\varepsilon$ is a small value to avoid numerical instability when calculating the gradients.<fnref n="2" />

**L1 norm.** Similarly to ColBERTer [20], which uses an L1 norm regularization of sigmoid-based gates (at word level), we propose to use L1-norm regularization that helps to reduce the number of nonzero components in vectors. This in turn allows us to prune vectors, since if $\|\mathbf{d}\| = 0$, the document vector $\mathbf{d}$ is straightforwardly dominated. Our third regularization loss is thus:

$$ \mathcal{L}_{r}^{(L1)} = \frac{1}{n} \sum_{\mathbf{d} \in \mathbf{D}} \|\mathbf{d}\|_{1} \quad (7) $$

## 4.3 IR loss

Our main loss is the one used ColBERTv2 [45] which ensures that we learn an IR scoring function. ColBERTv2 uses both a distillation and an infoNCE loss. We use the hard negatives provided by a cross-encoder.<fnref n="3" /> Based on this collection, we use tuples consisting of one query $\mathbf{q}$, one annotated positive passage $p^+$ and a hard negative $p^-$, as well as a set of easy negative passages $P_{\text{easy}}$ (in-batch negatives). Following previous work [19, 21, 31, 32, 41, 45], we use a KL-divergence loss to distill the teacher's score into the student architecture. Using the following estimation of the probability of $p^+$ being a positive passage,

$$ p(p_+|\mathbf{q}, N) = \frac{e^{s(\mathbf{q}, p_+)}}{e^{s(\mathbf{q}, p_+)} + \sum_{\mathbf{p} \in N} e^{s(\mathbf{q}, N)}} $$

the IR loss is defined as a combination of distillation and cross-entropy:

$$ \mathcal{L}_{IR} = KL(p_{\text{ColBERT}_+}(\cdot|\mathbf{q}, N_h) || p_{\text{teacher}}(\cdot|\mathbf{q}, N_h)) + \log p_{\text{ColBERT}_+}(p_+|\mathbf{q}, N_a) \quad (8) $$

with $N_a$ all the documents (positive, hard, and easy negatives), and $N_h$ the positive and hard negative pair.

<footnote n="2">²We use the value $\varepsilon = 0.01$ in our experiments.</footnote>

<footnote n="3">³We use the same datasets as ColBERTv2</footnote>
## 4.4 Overall

The overall loss of our training is the combination of the regularization and IR losses, defined as follows:

$$ \mathcal{L}_{final} = \mathcal{L}_{IR} + \alpha \mathcal{L}_{r} \qquad (9) $$

where $\alpha$ corresponds to the coefficient of the regularization and $\mathcal{L}_r$ is one of the regularization losses defined in Section 4.2.

# 5 Experiments

In this section, we describe the experiments we develop to validate our pruning strategies and the proposed regularizations.

## 5.1 Experiment Setup

*Datasets.* We train our model on the MS MARCO v1 passage retrieval dataset [2]. This dataset contains 8.8M passages. We evaluate the model on the MS MARCO dev set (6980 queries), as well as the TREC 2019 DL (43 assessed topics) and TREC 2020 DL (54 assessed topics) sets. We also report the zero-shot performance of our training and pruning strategies using a subset of the BEIR [46] benchmark, as well as of the LoTTE collections [45] (datasets were chosen to lower the overall computation budget).

*Pipeline.* We evaluate our model using a reranking-based pipeline. Specifically, we use our own trained SPLADEv2 [11] as a first-stage retrieval and then use our model to rerank the top-k document. We did not use the standard ColBERT pipeline in our experiments for two reasons. First, following Formal et al. [10], the SPLADE model is a very effective first-stage retriever that is competitive with the first-stage used in ColBERT, namely PLAID [44] or EMVB [37]. Second, in our model, as document tokens are not normalized, the centroid interaction used by PLAID and EMVB [37, 44] does not work. The document tokens with small norms are more likely to be in the same cluster, and this increases the number of candidates to be re-ranked, which is problematic. Finally, as we experiment with different ColBERT variations and pruning strategies, building a PLAID [44] or EMVB [37] index would be quite costly. For the evaluation, we use the top 100 passages on the MS MARCO dev set, BEIR, and LoTTE (given the high number of topics), and the top 1000 for the evaluation on the TREC DL 2019 and 2020.

## 5.2 Implementation Details

We conduct experiments<fnref n="4" /> with NVIDIA TESLA V100 gpus. We use an AdamW optimizer with a learning rate of 1e-5, a batch size of 16, and train during 320k steps. We use the ColBERTv2 checkpoint to initialize our model parameters<fnref n="5" /> [45]. We validate the training result on a subset of the MS MARCO development set (1000 topics, distinct from the final dev-small evaluation set) every 8000 steps. To select the checkpoint providing a good trade-off between effectiveness and efficiency, as a validation metric, we use a weighted harmonic mean, i.e. H0.5, H1 and H2, using MRR@10 and an estimation of the pruning ratio.<fnref n="6" /> The training takes around 5 days (L1 and document similarity regularization) or 8 days (nuclear norm regularization) on a single GPU.

For the projection $\pi_{\theta}$, different from previous work [20, 40] which computes a scalar in [0, 1] using a linear layer followed by a sigmoid, we propose to project the last hidden layer of the BERT model (denoted as $\mathbf{d}^{\text{BERT}}$) onto a vector of dimension $d' > d$. We normalize this vector so it is norm 1, before restricting it to the first $d$ dimensions. In practice, we obtain a vector of norm equal to or less than 1, as well as learning to remove the not-important directions of the original document tokens. Formally, we have

$$ \pi_{\theta}(\mathbf{d}^{\text{BERT}}) = \text{normalize} \left( \left( \frac{\mathbf{W}_1}{\mathbf{W}_2} \right) \mathbf{d}^{\text{BERT}} \right)_{:,d} $$

where $\mathbf{W}_1$ is the ColBERT projection matrix (so we can use the ColBERTv2 checkpoint to initialize $\mathbf{W}_1$), and $\mathbf{W}_2$ is a projection matrix (with an output dimension of 32) which is initialized randomly.

*Baseline.* We first consider the following baselines:

* A standard IR baseline, BM25 [42],

* SPLADEv2 [11], our first-stage retriever, and a state-of-the-art sparse model;<fnref n="7" />

* ColBERTv2 [45] end-to-end which uses the full ColBERTv2 pipeline;

* ColBERTv2 [45] re-ranking, which re-reranks the documents retrieved by SPLADEv2;

Our second set of baselines are late-interaction models that prune document vectors – note that all the static methods in the pruning part are based on ColBERTv2 [45]:

* AlignER [40] is a state-of-the-art “learning-to-prune” method that employs an entropy-regularized loss to remove tokens from a document representation. We use only their reported results based on the “base” size of BERT [6] to make the result comparable with ours<fnref n="8" />;

* Const-BERT [36] projects the document vectors into a fixed number of vectors;

* IDF pruning [33] that removes tokens with the lowest (token-based) IDF. We copy their results with 50% pruning ratio in our table;

* First-$p$ [33] that only keeps the first $p$ tokens. We copy their results with 50% pruning ratio in our table;

* Stopwords pruning [1] that removes tokens from a pre-determined list of stopwords;

* Attention-based pruning [33] (see Section 2);

* ColBERTer [20]. It is a DistilBERT-based ColBERT model in which tokens are first aggregated into words before being pruned.

Among all the results reported in these papers, we did select the ones reported in the tables by either using the configuration put forward by the authors (AlignER, ColBERTer, Const-BERT), or selecting the best trade-off between pruning ratio and effectiveness (for IDF, first-$p$, stop-words).

# 6 Results and Analysis

In this section, we perform some quantitative and qualitative analysis based on our experimental results.

<footnote n="4">⁴PyTorch 2.0.1, HuggingFace transformers 4.37, XPM-IR 1.3.1 [49]</footnote>

<footnote n="5">⁵https://huggingface.co/colbert-ir/colbertv2.0</footnote>

<footnote n="6">⁶On 1024 documents, using approximate LP-based pruning with a cumulative singular value proportion of 0.7</footnote>

<footnote n="7">⁷We did not use SPLADEv3 [26] since its training strategy is more complex than the one used in this paper</footnote>

<footnote n="8">⁸AlignER is trained without distillation</footnote>
<figure><img src="image_3.png" /><figcaption>Figure 2: Average TREC DL (2019 and 2020) nDCG@10 for different regularizations and pruning ratios</figcaption></figure>

## 6.1 (In-Domain) Pruning Ratio Analysis

We first examine the impact of varying pruning thresholds on the pruning ratio and evaluation results, based on our Norm (threshold from 0 to .999) and LP-based (threshold between 0.2 and 0.95) pruning strategy. We keep the regularization weight constant for each of our strategies ($\alpha$ = 0.01 for the L1 norm and $\alpha$ = 0.8 for Document Similarity and $\alpha$ = 0.1 for nuclear norm). These weights were set so that they perform similar without any pruning, to measure the effect of the pruning algorithm itself.

In Figure 2, we can observe that, in general, all our models have no clear performance drop until around 30% to 40% of the remaining tokens. Beyond this threshold, we can observe some clear drop in performance. The best performance is achieved using document similarity-based regularization, with LP or norm-based pruning. If we want to prune more, the best models are based on L1-Norm or nuclear norm regularizations, with norm pruning. Comparing the different pruning strategies, we found that $\mathcal{L}_r^{(sim)}$ is the regularization that works best for the LP pruning strategy. It provides a similar pruning ratio to the models with the L1-norm pruning, which means that our designed regularization is close to the idea of our proposed theory based on the LP. In contrast, $\mathcal{L}_r^{(nuc)}$ and $\mathcal{L}_r^{(L1)}$ lead to a larger difference in the pruning ratio between the two pruning strategies. It shows that these two methods' approximations are far from our dominance theory and LP procedure.

## 6.2 In-Domain Retrieval

In table 1, we show our in-domain evaluation result, compared with baselines. Based on the results presented in the previous section, we use $\theta_{LP} = 0.7$ and $\theta_N = 0.5$.

For each trained model, to maintain a good trade-off between the evaluation result and the pruning ratio, we use the checkpoint with the best weighted harmonic value H2 to evaluate our final model, with more emphasis on the pruning ratio.

Comparing our methods with other state-of-the-art methods, we can observe that our best model (similarity loss $\mathcal{L}_r^{(sim)}$ with $\alpha$ = 0.8 and LP pruning with $\theta_{LP}$ = 0.7) is on par with the ColBERTv2 model on the dev-small set (MRR@10 of 39.7 vs. 40.0) and the set

DL19 (nDCG@10 of 73.8 vs. 74.4), while only keeping 40% of the total document tokens. The performance is a bit worse on DL20 (nDCG@10 73.3 vs 75.6) but still well over the SPLADEv2 original ranking (68.7).

Among our models, all regularizations have a direct impact on pruning, since increasing $\alpha$ leads to higher pruning rates. We observe that L1-norm pruning is adapted to nuclear and L1-norm regularization, achieving similar performance with a higher pruning ratio. However, for our similarity-based regularization, LP-based pruning is more advantageous, showing that this regularization is more related to dominance than the others.

Finally, comparing our methods with the other state-of-the-art methods which use the static (e.g. IDF, Attention Score, Stopwords, First *p*) or dynamic (e.g. AligneR) pruning methods, all our regularization outperform their result on both the evaluation effectiveness and the pruning ratio.

## 6.3 Out-of Domain Retrieval

We now turn to the problem of generalization by evaluating our models on out-of-domain datasets (BEIR and LoTTE). We selected the best configurations, that is, the models having the best trade-off between the pruning ratio and the evaluation score, that is (1) L1 norm regularization ($\alpha$ = 0.01) with norm threshold of $\theta_N$ = 0.5; (2) document similarity regularization ($\alpha$ = 0.8) with $\theta_{LP}$ = 0.7. We also include the checkpoint of document similarity regularization ($\alpha$ = 0.1) with $\theta_{LP}$ = 0.7 since this is the best with respect to IR metrics (on MS MARCO dev small).

The results are presented in Tables 4 and 2. Note that AligneR [40] only reports detailed results when not pruning, so we do not include their results here. In general, our models perform slightly worse than the ColBERTv2 re-ranking version (nDCG@10 45.9 vs. 47.0 on BEIR, and Success@5 71.3 vs. 72.0 on LoTTE even for our best performing model). The difference is larger than the in-domain evaluation tasks (-0.8% on dev-small vs. -2.8% on BEIR, and -1.1% on LoTTE on model trained on document similarity regularization with $\alpha$ = 0.8). Increasing the remaining ratio around 60% allows to preserve the performance.

Furthermore, for the same model, on the out-of-domain task, more tokens remain than the in-domain task (e.g. 33% in-domain vs. 39% on BEIR and 29% on LoTTE on a model trained on document similarity regularization with $\alpha$ = 0.8). This was expected since when regularizing we modify the distribution of the tokens: The model mainly learns to distinguish which tokens are important for the in-domain dataset. However, the performance in all cases is still much better than the SPLADEv2 baseline.

## 6.4 IR/MLM Checkpoint Training Analysis

In this section, we report experimental results where we compare whether training from an MLM-based checkpoint (i.e., the DistilBERT checkpoint) is better or worse than starting from an IR model checkpoint (i.e., a DistilBERT-based ColBERTv2 checkpoint). We use DistilBERT [43] checkpoint (HuggingFace) since it is much smaller than a BERT model (2x speed gain) while providing competitive results [11, 21].

We fine-tune DistilBERT using the ColBERTv2 IR loss only (Eq. (8)), resulting in a model we name DistilColBERT. From this
<table><caption>Table 1: The in-domain evaluation results (MRR@10 for dev small, nDCG@10 for TREC DL19 and DL20), and remaining ratio (for pruning models). We highlight the models and corresponding pruning strategy we use in our further experiments (bold). †: NO statistical significant performance drop with ($p \le 0.05$) when comparing with ColBERTv2 re-ranking with the two-tailed Student's t-test.</caption><thead><tr><th></th><th>model</th><th>Remain</th><th>Dev set</th><th>DL 19</th><th>DL 20</th></tr></thead><tbody><tr><td rowspan="4">w/o Pruning</td><td>BM25</td><td>-</td><td>18.7</td><td>50.6</td><td>47.5</td></tr><tr><td>SPLADEv2</td><td>-</td><td>35.8</td><td>70.6</td><td>68.7</td></tr><tr><td>ColBERTv2 end-to-end</td><td>-</td><td>39.7</td><td>74.5</td><td>-</td></tr><tr><td>ColBERTv2 reranking</td><td>-</td><td>40.0</td><td>74.4</td><td>75.6</td></tr><tr><td rowspan="7">w/ Pruning</td><td>First p</td><td>50%</td><td>37.7</td><td>72.3</td><td>-</td></tr><tr><td>IDF</td><td>50%</td><td>32.6</td><td>70.2</td><td>-</td></tr><tr><td>Stopwords</td><td>67%</td><td>-</td><td>70.9</td><td>67.8</td></tr><tr><td>Att. Score</td><td>50%</td><td>36.0</td><td>72.0</td><td>-</td></tr><tr><td>AligneR<sub>base</sub></td><td>40%</td><td>38.1</td><td>-</td><td>-</td></tr><tr><td>ConstBERT</td><td>47%</td><td>39.0</td><td>73.1</td><td>73.3</td></tr><tr><td colspan="6">LP-based pruning with threshold 0.7</td></tr><tr><td rowspan="3">Ours</td><td>α = 0.1</td><td>25%</td><td>37.1</td><td>72.8<sup>†</sup></td><td>70.6</td></tr><tr><td>α = 0.05</td><td>46%</td><td>37.9</td><td>72.6<sup>†</sup></td><td>71.4</td></tr><tr><td>α = 0.01</td><td>83%</td><td>39.2</td><td>73.6<sup>†</sup></td><td>72.1</td></tr><tr><td rowspan="5">Ours</td><td>α = 2</td><td>24%</td><td>38.4</td><td>72.5<sup>†</sup></td><td>72.6</td></tr><tr><td>α = 1.25</td><td>29%</td><td>39.3</td><td>73.7<sup>†</sup></td><td>72.8</td></tr><tr><td>α = 0.8</td><td>32%</td><td>39.7<sup>†</sup></td><td>73.8<sup>†</sup></td><td>73.3</td></tr><tr><td>α = 0.5</td><td>37%</td><td>39.5</td><td>73.9<sup>†</sup></td><td>73.7</td></tr><tr><td>α = 0.1</td><td>52%</td><td>39.9<sup>†</sup></td><td>74.2<sup>†</sup></td><td>73.8</td></tr><tr><td rowspan="3">Our</td><td>α = 0.3</td><td>38%</td><td>38.5</td><td>71.9<sup>†</sup></td><td>73.9<sup>†</sup></td></tr><tr><td>α = 0.2</td><td>40%</td><td>39.2</td><td>73.9<sup>†</sup></td><td>73.1</td></tr><tr><td>α = 0.1</td><td>56%</td><td>39.4</td><td>73.9<sup>†</sup></td><td>73.2</td></tr><tr><td colspan="6">L1-norm based pruning with threshold 0.5</td></tr><tr><td rowspan="3">Ours</td><td>α = 0.1</td><td>9%</td><td>37.5</td><td>73.0<sup>†</sup></td><td>70.6</td></tr><tr><td>α = 0.05</td><td>14%</td><td>37.9</td><td>72.6<sup>†</sup></td><td>71.4</td></tr><tr><td>α = 0.01</td><td>31%</td><td>39.2</td><td>73.6<sup>†</sup></td><td>72.1</td></tr><tr><td rowspan="5">Ours</td><td>α = 2</td><td>25%</td><td>38.7</td><td>72.8<sup>†</sup></td><td>72.6</td></tr><tr><td>α = 1.25</td><td>29%</td><td>39.4</td><td>73.1<sup>†</sup></td><td>72.9</td></tr><tr><td>α = 0.8</td><td>33%</td><td>39.7<sup>†</sup></td><td>73.8<sup>†</sup></td><td>73.2</td></tr><tr><td>α = 0.5</td><td>37%</td><td>39.5</td><td>73.9<sup>†</sup></td><td>73.7</td></tr><tr><td>α = 0.1</td><td>51%</td><td>39.9<sup>†</sup></td><td>74.2<sup>†</sup></td><td>73.8</td></tr><tr><td rowspan="3">Our</td><td>α = 0.3</td><td>18%</td><td>38.5</td><td>71.9<sup>†</sup></td><td>73.9<sup>†</sup></td></tr><tr><td>α = 0.2</td><td>25%</td><td>39.2</td><td>73.9<sup>†</sup></td><td>73.1</td></tr><tr><td>α = 0.1</td><td>34%</td><td>39.4</td><td>73.9<sup>†</sup></td><td>73.2</td></tr></tbody></table>

checkpoint, we fine-tune our model using our proposed regularizations (IR checkpoint), and compare this to a model trained from the start with the different regularizations (MLM checkpoint). Following the previous sections, we select the model with the best harmonic mean (H2) between the pruning ratio and MRR@10.

The results are reported in Table 3 and show that there is no clear best methodology. For the two models tested, the pruning ratio is higher when using the IR checkpoint, but the overall performance (DL19 and DL20) is worse. There is a slight advantage in using an

<table><caption>Table 2: The LoTTE search results in Success@5 and the remaining ratio of our methods for each sub-dataset. †: NO statistical significant performance drop with ($p \le 0.05$) from the result of ColBERTv2 reranking version under the two-tailed Student's t-test.</caption><thead><tr><th></th><th>model</th><th>Wrt.</th><th>Rcr.</th><th>Sci.</th><th>Tch.</th><th>LS.</th><th>Avg.</th></tr></thead><tbody><tr><td rowspan="4">w/o. Pruning</td><td>BM25</td><td>60.3</td><td>56.5</td><td>32.7</td><td>41.8</td><td>63.8</td><td>51.0</td></tr><tr><td>SPLADEv2</td><td>72.1</td><td>67.8</td><td>53.2</td><td>60.1</td><td>80.0</td><td>66.6</td></tr><tr><td>ColBERTv2 end-to-end</td><td>80.1</td><td>72.3</td><td>56.7</td><td>66.1</td><td>84.7</td><td>72.0</td></tr><tr><td>ColBERTv2 reranking</td><td>80.0</td><td>73.0</td><td>56.9</td><td>66.4</td><td>84.3</td><td>72.1</td></tr><tr><td rowspan="2">Ours</td><td>α = 0.01</td><td>77.5</td><td>72.5<sup>†</sup></td><td>57.2<sup>†</sup></td><td>65.4<sup>†</sup></td><td>84.1<sup>†</sup></td><td>71.4</td></tr><tr><td>Remain %</td><td>29%</td><td>29%</td><td>26%</td><td>31%</td><td>26%</td><td>29%</td></tr><tr><td rowspan="3">Ours</td><td>α = 0.8</td><td>77.4</td><td>72.1<sup>†</sup></td><td>56.9<sup>†</sup></td><td>65.9<sup>†</sup></td><td>84.0<sup>†</sup></td><td>71.3</td></tr><tr><td>Remain %</td><td>34%</td><td>35%</td><td>35%</td><td>37%</td><td>34%</td><td>35%</td></tr><tr><td>α = 0.1</td><td>78.9<sup>†</sup></td><td>72.0<sup>†</sup></td><td>57.2<sup>†</sup></td><td>65.4<sup>†</sup></td><td>84.1<sup>†</sup></td><td>71.3</td></tr><tr><td></td><td>Remain %</td><td>57%</td><td>58%</td><td>70%</td><td>59%</td><td>59%</td><td>61%</td></tr></tbody></table>

<table><caption>Table 3: The MLM training and the IR oriented training results. †: NO statistical significant performance drop with ($p \le 0.05$) from the result of Distil-ColBERTv2 reranking version under the two-tailed Student's t-test.</caption><thead><tr><th></th><th>model</th><th>Remain</th><th>DL 19</th><th>DL 20</th></tr></thead><tbody><tr><td>Distil-ColBERTv2 reranking</td><td>-</td><td>73.5</td><td>73.6</td></tr><tr><td>ColBERTer</td><td>40%</td><td>72.7</td><td>73.3</td></tr><tr><td rowspan="2">0.01 L<sub>r</sub><sup>(L1)</sup></td><td>MLM cp.</td><td>35%</td><td>73.2<sup>†</sup></td><td>72.6<sup>†</sup></td></tr><tr><td>IR cp.</td><td>32%</td><td>73.1<sup>†</sup></td><td>72.3<sup>†</sup></td></tr><tr><td rowspan="2">0.8 L<sub>r</sub><sup>(sim)</sup></td><td>MLM cp.</td><td>34%</td><td>73.3<sup>†</sup></td><td>73.0<sup>†</sup></td></tr><tr><td>IR cp.</td><td>32%</td><td>72.4<sup>†</sup></td><td>72.3<sup>†</sup></td></tr></tbody></table>

MLM checkpoint with similarity-based regularization, but more experiments are needed to validate this point.

## 6.5 Interpretability Analysis

To conclude our section of experimental results, in Table 5, we show an example of different pruning for a passage from MS MARCO v1. In general, we observe that the remaining tokens are mostly the important nouns and verbs, which is consistent with human intuition that these tokens play a more important role in the semantic matching. We also note that, for similarity-based regularization, LP-based pruning removes the second occurrence of “Population”, but norm-based pruning does not: this showcases the benefit of using LP-based pruning. Likewise, “city” is discarded with LP-based pruning as its semantic meaning is already covered by “New Dehli”.

## 7 Conclusion, Limitations and Future Work

In this paper, we proposed a principled framework, that of vector dominance, that defines exactly which tokens can be removed from the set of document vectors in a late interaction model like ColBERT, without any impact on the score function. Based on this concept, we
<table><caption>**Table 4:** The BEIR results in nDCG@10 and remaining ratio of our methods for each sub-dataset. †: NO statistical significant performance drop with ($p \le 0.05$) from the result of ColBERTv2 reranking version under the two-tailed Student's t-test.</caption><thead><tr><th></th><th>model</th><th>Remain</th><th>CF</th><th>DB</th><th>FQ</th><th>NF</th><th>NQ</th><th>QU</th><th>SD</th><th>SF</th><th>TC</th><th>TO</th><th>Avg.</th></tr></thead><tbody><tr><td rowspan="4">w/o. Pruning</td><td>BM25</td><td>-</td><td>21.3</td><td>31.3</td><td>23.6</td><td>32.5</td><td>32.9</td><td>78.9</td><td>15.8</td><td>66.5</td><td>65.6</td><td>36.7</td><td>40.5</td></tr><tr><td>SPLADEv2</td><td>-</td><td>21.3</td><td>42.0</td><td>31.6</td><td>32.8</td><td>50.8</td><td>81.1</td><td>14.0</td><td>65.9</td><td>65.5</td><td>25.5</td><td>43.0</td></tr><tr><td>ColBERTv2 end-to-end</td><td>-</td><td>17.6</td><td>44.6</td><td>35.6</td><td>33.8</td><td>56.2</td><td>85.2</td><td>15.4</td><td>69.3</td><td>73.8</td><td>26.3</td><td>45.8</td></tr><tr><td>ColBERTv2 reranking</td><td>-</td><td>20.3</td><td>45.9</td><td>35.7</td><td>34.3</td><td>56.8</td><td>85.7</td><td>14.4</td><td>68.5</td><td>75.5</td><td>33.7</td><td>47.0</td></tr><tr><td rowspan="3">w/. Pruning</td><td>First p</td><td>50%</td><td>15.5</td><td>44.0</td><td>32.4</td><td>32.4</td><td>53.2</td><td>78.7</td><td>15.2</td><td>61.6</td><td>72.1</td><td>26.1</td><td>43.1</td></tr><tr><td>IDF</td><td>50%</td><td>17.0</td><td>45.1</td><td>34.0</td><td>32.6</td><td>55.0</td><td>85.3</td><td>15.5</td><td>64.8</td><td>71.4</td><td>26.2</td><td>44.7</td></tr><tr><td>Att. Score</td><td>50%</td><td>16.7</td><td>44.4</td><td>33.6</td><td>32.6</td><td>54.7</td><td>84.4</td><td>15.5</td><td>64.0</td><td>70.0</td><td>26.2</td><td>44.2</td></tr><tr><td rowspan="2">Ours</td><td>α = 0.01</td><td>-</td><td>18.6</td><td>44.3</td><td>34.3</td><td>33.8<sup>†</sup></td><td>54.0</td><td>84.5</td><td>14.2<sup>†</sup></td><td>67.0<sup>†</sup></td><td>71.6</td><td>31.8</td><td>45.4</td></tr><tr><td>Remain %</td><td>-</td><td>34%</td><td>39%</td><td>30%</td><td>29%</td><td>35%</td><td>76%</td><td>32%</td><td>30%</td><td>32%</td><td>28%</td><td>39%</td></tr><tr><td rowspan="3">Ours</td><td>α = 0.8</td><td>-</td><td>19.6</td><td>42.9</td><td>35.0<sup>†</sup></td><td>33.8<sup>†</sup></td><td>54.7</td><td>82.6</td><td>13.9</td><td>66.9<sup>†</sup></td><td>74.3<sup>†</sup></td><td>33.1<sup>†</sup></td><td>45.7</td></tr><tr><td>Remain %</td><td>-</td><td>36%</td><td>34%</td><td>36%</td><td>38%</td><td>36%</td><td>40%</td><td>36%</td><td>38%</td><td>37%</td><td>40%</td><td>37%</td></tr><tr><td>α = 0.1</td><td>-</td><td>20.2<sup>†</sup></td><td>44.7</td><td>35.0<sup>†</sup></td><td>34.4<sup>†</sup></td><td>55.5</td><td>84.4</td><td>14.0</td><td>67.4<sup>†</sup></td><td>71.1</td><td>32.8<sup>†</sup></td><td>45.9</td></tr><tr><td></td><td>Remain %</td><td>-</td><td>60%</td><td>58%</td><td>57%</td><td>64%</td><td>57%</td><td>58%</td><td>58%</td><td>69%</td><td>65%</td><td>65%</td><td>61%</td></tr></tbody></table>

<table><caption>**Table 5:** A Qualitative analysis of our two proposed pruning strategy based on the best model trained on our designed regularizations. The remaining tokens are in bold.</caption><thead><tr><th>Example passage</th><th>Norm pruning: θ<sub>N</sub> = 0.5</th><th>LP pruning: θ<sub>LP</sub> = 0.7</th></tr></thead><tbody><tr><td>Doc-Sim Regularization</td><td>What is the <b>Population of New Delhi</b>. According to <b>estimated figures</b> from <b>Census</b> of India, <b>Population of New Delhi</b> in <b>2016</b> is 18.6 million. Delhi's is witnessing a <b>huge growth</b> in its population every <b>year</b>. <b>Population of Delhi city</b> is estimated to cross 25 million in <b>2020</b>.</td><td>What is the <b>Population of New Delhi</b>. According to <b>estimated figures</b> from <b>Census</b> of India, <b>Population of New Delhi</b> in <b>2016</b> is 18.6 million. Delhi's is witnessing a <b>huge growth</b> in its population every <b>year</b>. <b>Population of Delhi city</b> is estimated to cross 25 million in <b>2020</b>.</td></tr><tr><td>L1-Norm Regularization</td><td>What is the <b>Population of New Delhi</b>. According to <b>estimated figures</b> from <b>Census</b> of India, <b>Population of New Delhi</b> in <b>2016</b> is 18.6 million. Delhi's is witnessing a <b>huge growth</b> in its population every <b>year</b>. <b>Population of Delhi city</b> is estimated to cross 25 million in <b>2020</b>.</td><td>What is the <b>Population of New Delhi</b>. According to <b>estimated figures</b> from <b>Census</b> of India, <b>Population of New Delhi</b> in <b>2016</b> is 18.6 million. Delhi's is witnessing a <b>huge growth</b> in its population every <b>year</b>. <b>Population of Delhi city</b> is estimated to cross 25 million in <b>2020</b>.</td></tr></tbody></table>

adapted the ColBERT model scoring function and proposed various regularization and pruning strategies. Through our experiments, we found that our methods can remove around 70% of document vectors while preserving good both in-domain and out-of-domain performance. However, our work still has some limitations that we discuss in the following.

First, our current pipeline relies on the SPLADEv2 reranking. Although it provides a good result, the reranking pipeline requires more index storage than the end-to-end scenario. Adapting recent works like [37] to our use case (i.e. handling varying norm document tokens) would allow the model to reach a better space efficiency. Second, although we have provided some techniques to lower the computational cost of the LP procedure, they are still too costly. However, we did not leverage the fact that our LP problem has a lot of symmetries, since for the same document the problems are very similar. An alternative would be to design approximate

pruning where the approximation is controlled – and this could be leveraged during training and pruning. Third, regularization strategies are loosely related to the ratio of dominance, which should be the main target. Designing better regularization functions could substantially increase the performance of our methodology. This could be combined with adapting the Transformer architecture so it is more prone to produce dominated vectors, or by using better pretraining than MLM – better pre-training can lead to substantial performance increase [13, 35].

## Acknowledgments

The authors acknowledge ANR - FRANCE (French National Research Agency) for its financial support of the GUIDANCE project n°ANR-23-IAS1-0003. This work was granted access to the HPC resources of IDRIS (Project No. AD011014315R1 and A17) under the allocation made by GENCI.
References

[1] Antonio Acquavia, Craig Macdonald, and Nicola Tonellotto. 2023. Static Pruning for Multi-Representation Dense Retrieval. In *Proceedings of the ACM Symposium on Document Engineering 2023*. ACM, Limerick Ireland 1–10. https://doi.org/10.1145/3573128.3604896

[2] Payal Bajaj, Daniel Campos, Nick Craswell, Li Deng, Jianfeng Gao, Xiaodong Liu, Rangan Majumder, Andrew McNamara, Bhaskar Mitra, Tri Nguyen, Mir Rosenberg, Xia Song, Alina Stoica, Saurabh Tiwary, and Tong Wang. 2018. MS MARCO: A Human Generated Machine Reading Comprehension Dataset. arXiv:1611.09268 [cs.CL] https://arxiv.org/abs/1611.09268

[3] Dennis Cheung, Felipe Cucker, and Javier Pena. 2008. A Condition Number for Multifold Conic Systems. *SIAM Journal on Optimization* 19 (01 2008), 261–280. https://doi.org/10.1137/060665427

[4] Dennis Cheung, Felipe Cucker, and Javier Peña. 2003. Unifying Condition Numbers for Linear Programming. *Mathematics of Operations Research* 28, 4 (2003), 609–624. http://www.jstor.org/stable/4127002

[5] Michael B. Cohen, Yin Tat Lee, and Zhao Song. 2019. Solving linear programs in the current matrix multiplication time. In *Proceedings of the 51st Annual ACM SIGACT Symposium on Theory of Computing (Phoenix, AZ, USA) (STOC 2019)*. Association for Computing Machinery, New York, NY, USA, 938–942. https://doi.org/10.1145/3313276.3316303

[6] Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. In *Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers)*, Jill Burstein, Christy Doran, and Thamar Solorio (Eds.). Association for Computational Linguistics, Minneapolis, Minnesota, 4171–4186. https://doi.org/10.18653/v1/N19-1423

[7] Laxman Dhulipala, Majid Hadian, Rajesh Jayaram, Jason Lee, and Vahab Mirrokni. 2024. MUVERA: Multi-Vector Retrieval via Fixed Dimensional Encoding. In *Advances in Neural Information Processing Systems*, A. Globerson, L. Mackey, D. Belgrave, A. Fan, U. Paquet, J. Tomczak, and C. Zhang (Eds.), Vol. 37. Curran Associates, Inc., Vancouver, Canada, 101042–101073. https://proceedings.neurips.cc/paper_files/paper/2024/file/b71cfefa46909178603b5bc6c11d3ae-Paper-Conference.pdf

[8] Zhen Fan, Luyu Gao, Rohan Jha, and Jamie Callan. 2023. COILcr: Efficient Semantic Matching in Contextualized Exact Match Retrieval. In *Advances in Information Retrieval: 45th European Conference on Information Retrieval, ECIR 2023*, Dublin, Ireland, April 2–6, 2023, *Proceedings, Part I* (Dublin, Ireland). Springer-Verlag, Berlin, Heidelberg, 298–312. https://doi.org/10.1007/978-3-031-28244-7_19

[9] John Fearnley and Rahul Savani. 2015. The Complexity of the Simplex Method. In *Proceedings of the Forty-Seventh Annual ACM Symposium on Theory of Computing (Portland, Oregon, USA) (STOC '15)*. Association for Computing Machinery, New York, NY, USA, 201–208. https://doi.org/10.1145/2746539.2746558

[10] Thibault Formal, Stéphane Clinchant, Hervé Déjean, and Carlos Lassance. 2024. SPLATE: Sparse Late Interaction Retrieval. arXiv:2404.13950 [cs.IR] https://arxiv.org/abs/2404.13950

[11] Thibault Formal, Carlos Lassance, Benjamin Piwowarski, and Stéphane Clinchant. 2021. SPLADE v2: Sparse Lexical and Expansion Model for Information Retrieval. arXiv:2109.10086 [cs.IR] https://arxiv.org/abs/2109.10086

[12] Robert M. Freund and Jorge R. Vera. 2009. Equivalence of Convex Problem Geometry and Computational Complexity in the Separation Oracle Model. *Mathematics of Operations Research* 34, 4 (Nov. 2009), 869–879. https://doi.org/10.1287/moor.1090.0408

[13] Luyu Gao and Jamie Callan. 2021. Condenser: a Pre-training Architecture for Dense Retrieval. In *Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing*, Marie-Francine Moens, Xuanjing Huang, Lucia Specia, and Scott Wen-tau Yih (Eds.). Association for Computational Linguistics, Online and Punta Cana, Dominican Republic, 981–993. https://doi.org/10.18653/v1/2021.emnlp-main.75

[14] Luyu Gao, Zhuyun Dai, and Jamie Callan. 2021. COIL: Revisit Exact Lexical Match in Information Retrieval with Contextualized Inverted List. In *Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies*, Kristina Toutanova, Anna Rumshisky, Luke Zettlemoyer, Dilek Hakkani-Tur, Iz Beltagy, Steven Bethard, Ryan Cotterell, Tanmoy Chakraborty, and Yichao Zhou (Eds.). Association for Computational Linguistics, Online, 3030–3042. https://doi.org/10.18653/v1/2021.naacl-main.241

[15] Anupam Garg and N. D. Mermin. 1984. Farkas's Lemma and the nature of reality: Statistical implications of quantum correlations. *Foundations of Physics* 14, 1 (Jan. 1984), 1–39. https://doi.org/10.1007/BF00741645

[16] Nathan Godey, Éric Clergerie, and Benoît Sagot. 2024. Anisotropy Is Inherent to Self-Attention in Transformers. In *Proceedings of the 18th Conference of the European Chapter of the Association for Computational Linguistics (Volume 1: Long Papers)*, Yvette Graham and Matthew Purver (Eds.). Association for Computational Linguistics, St. Julian's, Malta, 35–48. https://aclanthology.org/2024.eacl-long.3

[17] Jiafeng Guo, Yixing Fan, Qingyao Ai, and W. Bruce Croft. 2016. A Deep Relevance Matching Model for Ad-hoc Retrieval. In *Proceedings of the 25th ACM International

on Conference on Information and Knowledge Management (Indianapolis, Indiana, USA) (CIKM '16). Association for Computing Machinery, New York, NY, USA, 55–64. https://doi.org/10.1145/2983323.2983769

[18] Shanxiu He, Mutasem Al-Darabsah, Suraj Nair, Jonathan May, Tarun Agarwal, Tao Yang, and Choon Hui Teo. 2025. Token Pruning Optimization for Efficient Multi-vector Dense Retrieval. In *Advances in Information Retrieval: 47th European Conference on Information Retrieval, ECIR 2025, Lucca, Italy, April 6–10, 2025, Proceedings, Part I* (Lucca, Italy). Springer-Verlag, Berlin, Heidelberg, 101–115. https://doi.org/10.1007/978-3-031-88708-6_7

[19] Geoffrey Hinton, Oriol Vinyals, and Jeff Dean. 2015. Distilling the Knowledge in a Neural Network. arXiv:1503.02531 [stat.ML] https://arxiv.org/abs/1503.02531

[20] Sebastian Hofstätter, Omar Khattab, Sophia Althammer, Mete Sertkán, and Al-İlan Hanbury. 2022. Introducing Neural Bag of Whole-Words with ColBERTer: Contextualized Late Interactions using Enhanced Reduction. In *Proceedings of the 31st ACM International Conference on Information & Knowledge Management (Atlanta, GA, USA) (CIKM '22)*. Association for Computing Machinery, New York, NY, USA, 737–747. https://doi.org/10.1145/3511808.3557367

[21] Sebastian Hofstätter, Sheng-Chieh Lin, Jheng-Hong Yang, Jimmy Lin, and Allan Hanbury. 2021. Efficiently Teaching an Effective Dense Retriever with Balanced Topic Aware Sampling. In *Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval* (Virtual Event, Canada) (SIGIR '21). Association for Computing Machinery, New York, NY, USA, 113–122. https://doi.org/10.1145/3404835.3462891

[22] Kai Hui, Andrew Yates, Klaus Berberich, and Gerard de Melo. 2018. Co-PACRR: A Context-Aware Neural IR Model for Ad-hoc Retrieval. In *Proceedings of the Eleventh ACM International Conference on Web Search and Data Mining* (Marina Del Rey, CA, USA) (WSDM '18). Association for Computing Machinery, New York, NY, USA, 279–287. https://doi.org/10.1145/3159652.3159689

[23] Jeff Johnson, Matthijs Douze, and Hervé Jégou. 2021. Billion-Scale Similarity Search with GPUs. *IEEE Transactions on Big Data* 7, 3 (2021), 535–547. https://doi.org/10.1109/TBDATA.2019.2921572

[24] Vladimir Karpukhin, Barlas Oguz, Sewon Min, Patrick Lewis, Ledell Wu, Sergey Edunov, Danqi Chen, and Wen-tau Yih. 2020. Dense Passage Retrieval for Open-Domain Question Answering. In *Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)*, Bonnie Webber, Trevor Cohn, Yulan He, and Yang Liu (Eds.). Association for Computational Linguistics, Online, 6769–6781. https://doi.org/10.18653/v1/2020.emnlp-main.550

[25] Omar Khattab and Matei Zaharia. 2020. ColBERT: Efficient and Effective Passage Search via Contextualized Late Interaction over BERT. In *Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval* (Virtual Event, China) (SIGIR '20). Association for Computing Machinery, New York, NY, USA, 39–48. https://doi.org/10.1145/3397271.3401075

[26] Carlos Lassance, Hervé Déjean, Thibault Formal, and Stéphane Clinchant. 2024. SPLADE-v3: New baselines for SPLADE. arXiv:2403.06789 [cs.IR] https://arxiv.org/abs/2403.06789

[27] Carlos Lassance, Maroua Maachou, Joohee Park, and Stéphane Clinchant. 2022. Learned Token Pruning in Contextualized Late Interaction over BERT (ColBERT). In *Proceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval* (Madrid, Spain) (SIGIR '22). Association for Computing Machinery, New York, NY, USA, 2232–2236. https://doi.org/10.1145/3477495.3531835

[28] Jinhyuk Lee, Zhuyun Dai, Sai Meher Karthik Duddu, Tao Lei, Iftekhar Naim, Ming-Wei Chang, and Vincent Y. Zhao. 2023. Rethinking the role of token retrieval in multi-vector retrieval. In *Proceedings of the 37th International Conference on Neural Information Processing Systems* (New Orleans, LA, USA) (NIPS '23). Curran Associates Inc., Red Hook, NY, USA, Article 677, 22 pages.

[29] Yin Tat Lee, Zhao Song, and Qiuyi Zhang. 2019. Solving Empirical Risk Minimization in the Current Matrix Multiplication Time. arXiv:1905.04447 [cs.DS] https://arxiv.org/abs/1905.04447

[30] Minghan Li, Sheng-Chieh Lin, Xueguang Ma, and Jimmy Lin. 2023. SLIM: Sparsified Late Interaction for Multi-Vector Retrieval with Inverted Indexes. In *Proceedings of the 46th International ACM SIGIR Conference on Research and Development in Information Retrieval* (Taipei, Taiwan) (SIGIR '23). Association for Computing Machinery, New York, NY, USA, 1954–1959. https://doi.org/10.1145/3539618.3591977

[31] Minghan Li, Sheng-Chieh Lin, Barlas Oguz, Asish Ghoshal, Jimmy Lin, Yashar Mehdad, Wen-tau Yih, and Xilun Chen. 2023. CITADEL: Conditional Token Interaction via Dynamic Lexical Routing for Efficient and Effective Multi-Vector Retrieval. In *Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)*, Anna Rogers, Jordan Boyd-Graber, and Naoaki Okazaki (Eds.). Association for Computational Linguistics, Toronto, Canada, 11891–11907. https://doi.org/10.18653/v1/2023.acl-long.663

[32] Sheng-Chieh Lin, Jheng-Hong Yang, and Jimmy Lin. 2021. In-Batch Negatives for Knowledge Distillation with Tightly-Coupled Teachers for Dense Retrieval. In *Proceedings of the 6th Workshop on Representation Learning for NLP (RepL4NLP-2021)*, Anna Rogers, Iacer Calixto, Ivan Vulić, Naomi Saphra, Nora Kassner, Oana-Maria Camburu, Trapit Bansal, and Vered Shwartz (Eds.). Association for Computational Linguistics, Online, 163–173. https://doi.org/10.18653/v1/2021.
repl4nlp-1.17

[33] Qi Liu, Gang Guo, Jiaxin Mao, Zhicheng Dou, Ji-Rong Wen, Hao Jiang, Xinyu Zhang, and Zhao Cao. 2024. An Analysis on Matching Mechanisms and Token Pruning for Late-interaction Models. *ACM Transactions on Information Systems* 42, 5 (Sept. 2024), 1–28. https://doi.org/10.1145/3639818

[34] Yi Luan, Jacob Eisenstein, Kristina Toutanova, and Michael Collins. 2021. Sparse, Dense, and Attentional Representations for Text Retrieval. *Transactions of the Association for Computational Linguistics* 9 (2021), 329–345. https://doi.org/10.1162/tacl_a_00369

[35] Xueguang Ma, Hengxin Fun, Xusen Yin, Antonio Mallia, and Jimmy Lin. 2023. Enhancing Sparse Retrieval via Unsupervised Learning. In *Proceedings of the Annual International ACM SIGIR Conference on Research and Development in Information Retrieval in the Asia Pacific Region* (Beijing, China) (SIGIR-AP '23). Association for Computing Machinery, New York, NY, USA, 150–157. https://doi.org/10.1145/3624918.3625334

[36] Sean MacAvaney, Antonio Mallia, and Nicola Tonellotto. 2025. *Efficient Constant-Space Multi-vector Retrieval*. Lecture Notes in Computer Science, Vol. 15574. Springer Nature Switzerland, Cham, 237–245. https://doi.org/10.1007/978-3-031-88714-7_22

[37] Franco Maria Nardini, Cosimo Rulli, and Rossano Venturini. 2024. Efficient Multi-vector Dense Retrieval with Bit Vectors. In *Advances in Information Retrieval*, Nazli Goharian, Nicola Tonellotto, Yulan He, Aldo Lipani, Graham McDonald, Craig Macdonald, and Iadh Ounis (Eds.). Springer Nature Switzerland, Cham, 3–17.

[38] Jorge Nocedal and Stephen J. Wright. 2006. *Numerical Optimization* (2nd ed ed.). Springer, New York.

[39] Rodrigo Nogueira and Kyunghyun Cho. 2020. Passage Re-ranking with BERT. arXiv:1901.04085 [cs.IR] https://arxiv.org/abs/1901.04085

[40] Yujie Qian, Jinhyuk Lee, Sai Meher Karthik Duddu, Zhuyun Dai, Siddhartha Brahma, Iftekhar Naim, Tao Lei, and Vincent Y. Zhao. 2022. Multi-Vector Retrieval as Sparse Alignment. arXiv:2211.01267 [cs.CL] https://arxiv.org/abs/2211.01267

[41] Ruiyang Ren, Yingqi Qu, Jing Liu, Wayne Xin Zhao, QiaoQiao She, Hua Wu, Haifeng Wang, and Ji-Rong Wen. 2021. RocketQAv2: A Joint Training Method for Dense Passage Retrieval and Passage Re-ranking. In *Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing*, Marie-Francine Moens, Xuanjing Huang, Lucia Specia, and Scott Wen-tau Yih (Eds.). Association for Computational Linguistics, Online and Punta Cana, Dominican Republic, 2825–2835. https://doi.org/10.18653/v1/2021.emnlp-main.224

[42] Stephen E. Robertson, Steve Walker, Susan Jones, Micheline Hancock-Beaulieu, and Mike Gatford. 1994. Okapi at TREC-3. In *Text Retrieval Conference. Proceedings of the Third Text REtrieval Conference (TREC 1994)*, Gaithersburg, USA, 109. https://api.semanticscholar.org/CorpusID:3946054

[43] Victor Sanh, Lysandre Debut, Julien Chaumond, and Thomas Wolf. 2020. DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter. arXiv:1910.01108 [cs.CL] https://arxiv.org/abs/1910.01108

[44] Keshav Santhanam, Omar Khattab, Christopher Potts, and Matei Zaharia. 2022. PLAID: An Efficient Engine for Late Interaction Retrieval. In *Proceedings of the 31st ACM International Conference on Information & Knowledge Management (Atlanta, GA, USA) (CIKM '22)*. Association for Computing Machinery, New York, NY, USA, 1747–1756. https://doi.org/10.1145/3511808.3557325

[45] Keshav Santhanam, Omar Khattab, Jon Saad-Falcon, Christopher Potts, and Matei Zaharia. 2022. ColBERTv2: Effective and Efficient Retrieval via Lightweight Late Interaction. In *Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies*, Marine Carpuat, Marie-Catherine de Marneffe, and Ivan Vladimir Meza Ruiz (Eds.). Association for Computational Linguistics, Seattle, United States, 3715–3734. https://doi.org/10.18653/v1/2022.naacl-main.272

[46] Nandan Thakur, Nils Reimers, Andreas Rücklé, Abhishek Srivastava, and Iryna Gurevych. 2021. BEIR: A Heterogenous Benchmark for Zero-shot Evaluation of Information Retrieval Models. arXiv:2104.08663 [cs.IR] https://arxiv.org/abs/2104.08663

[47] Chenyan Xiong, Zhuyun Dai, Jamie Callan, Zhiyuan Liu, and Russell Power. 2017. End-to-End Neural Ad-hoc Ranking with Kernel Pooling. In *Proceedings of the 40th International ACM SIGIR Conference on Research and Development in Information Retrieval* (Shinjuku, Tokyo, Japan) (SIGIR '17). Association for Computing Machinery, New York, NY, USA, 55–64. https://doi.org/10.1145/3077136.3080809

[48] Shunyu Zhang, Yaobo Liang, Ming Gong, Daxin Jiang, and Nan Duan. 2022. Multi-View Document Representation Learning for Open-Domain Dense Retrieval. In *Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)*, Smaranda Muresan, Preslav Nakov, and Aline Villavicencio (Eds.). Association for Computational Linguistics, Dublin, Ireland, 5990–6000. https://doi.org/10.18653/v1/2022.acl-long.414

[49] Yuxuan Zong and Benjamin Piwowarski. 2023. XpmIR: A Modular Library for Learning to Rank and Neural IR Experiments. In *Proceedings of the 46th International ACM SIGIR Conference on Research and Development in Information Retrieval* (Taipei, Taiwan) (SIGIR '23). Association for Computing Machinery, New York, NY, USA, 3185–3189. https://doi.org/10.1145/3539618.3591818