SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Jinhyuk Lee* Zhuyun Dai Sai Meher Karthik Duddu
Tao Lei Iftekhar Naim Ming-Wei Chang Vincent Y. Zhao
Multi-vector retrieval models such as ColBERT [Khattab and Zaharia, 2020] allow token-level interactions between queries and documents, and hence achieve state of the art on many information retrieval benchmarks. However, their nonlinear scoring function cannot be scaled to millions of documents, necessitating a three-stage process for inference: retrieving initial candidates via token retrieval, accessing all token vectors, and scoring the initial candidate documents. The non-linear scoring function is applied over all token vectors of each candidate document, making the inference process complicated and slow. In this paper, we aim to simplify the multi-vector retrieval by rethinking the role of token retrieval. We present XTR, ConteXtualized Token Retriever, which introduces a simple, yet novel, objective function that encourages the model to retrieve the most important document tokens first. The improvement to token retrieval allows XTR to rank candidates only using the retrieved tokens rather than all tokens in the document, and enables a newly designed scoring stage that is two-to-three orders of magnitude cheaper than that of ColBERT. On the popular BEIR benchmark, XTR advances the state-of-the-art by  $2.8\mathrm{nDCG}@10$  without any distillation. Detailed analysis confirms our decision to revisit the token retrieval stage, as XTR demonstrates much better recall of the token retrieval stage compared to ColBERT.
The performance of a dense retrieval model is largely affected by how it defines expressive representations over queries and documents, and whether it can efficiently retrieve and score a document using these vector representations. For example, dual encoder models [Yih et al., 2011, Lee et al., 2019, Karpukhin et al., 2020, Ni et al., 2021] encode queries and documents into single vectors and compute query-document similarities using dot products. While these models are very efficient for retrieval, their expressivity is limited due to the absence of token-level modeling for scoring. In contrast, multi-vector models such as ColBERT [Khattab and Zaharia, 2020, Santhanam et al., 2022b] are directly designed to capture token-level interactions. By utilizing a (non-linear) scoring function over all query and document token representations, multi-vector models enjoy much better model expressivity and often achieve superior results across various benchmarks [Thakur et al., 2021].
The enhanced model expressivity, however, comes at a great cost of inference complexity. Unlike the case in dual encoders, the non-linear scoring function in multi-vector retrieval models prohibits the use of efficient Maximum Inner Product Search (MIPS) [Ram and Gray, 2012, Shrivastava and Li, 2014, 2015, Shen et al., 2015] for finding the maximum scoring documents. As a result, models such as ColBERT adopt an intricate and resource-intensive inference pipeline, which typically consists
*Correspondence: jinhyuklee@google.com
of three stages: 1) token retrieval: using each query token to retrieve document tokens, with their source documents becoming candidates; 2) gathering: collecting all the token embeddings from each candidate document, including those that are not retrieved in the first stage (most document tokens are not retrieved); and 3) scoring: ranking candidates using a non-linear function based on all the token embeddings per document.
This procedure leads to two major issues. First, compared to the token retrieval stage, gathering all document token embeddings and re-scoring the documents can introduce orders of magnitude additional data loading and floating operation cost, making multi-vector models extremely expensive to deploy. Secondly, while the candidate documents are decided in the token retrieval stage, previous training objectives are designed for the scoring stage. This creates a significant training-inference gap causing multi-vector models achieve sub-optimal (and often poor) recall performance. Clearly, the three-stage pipeline has largely limited the potential of multi-vector models, raising an interesting research question - can the token retrieval stage alone be sufficient for great performance?
We present XTR, ContXexternalized Token Retriever: a simplified and efficient method for multi-vector retrieval, through re-thinking the role of token retrieval. The key insight of XTR is that the token retrieval in multi-vector models should be trained to retrieve the most salient and informative document tokens, so that the score between a query and document can be computed using only the retrieved information, just like how single-vector retrieval models work. By doing so, the gathering step can be completely eliminated, and the cost of scoring is significantly reduced as only a fraction of the tokens need to be considered and the dot products from the token retrieval can be reused. To improve the quality of the token retrieval, XTR proposes a novel, yet simple, training objective, which dramatically improves retrieval accuracy, doubling the chances of a gold token being retrieved in the top- $k$  results. Furthermore, despite the improved token retrieval, some relevant tokens may still be missed (i.e., not retrieved). To address this issue, we propose a simple method, called missing similarity imputation, which accounts for the contribution of the missing tokens to the overall score.
XTR streamlines the inference process, bringing it closer to the straightforward procedure of dual encoders, while maintaining and enhancing the expressive scoring function of multi-vector retrieval models. On the BEIR [Thakur et al., 2021] and LoTTE [Santhanam et al., 2022b] benchmarks, XTR attains state-of-the-art performance, requiring neither distillation nor hard negativc mining. Notably, our model surpasses state-of-the-art dual-encoder GTR [Ni et al., 2021] by  $3.6\mathrm{nDCG}@10$  on BEIR without any additional training data. On the EntityQuestions benchmark [Sciavolino et al., 2021], XTR outperforms the previous state-of-the-art by 4.1 points on top-20 retrieval accuracy. XTR also does not require any secondary pre-training for retrieval and greatly outperforms mContriesver [Izacard et al., 2022] on MIRACL, which contains multilingual retrieval tasks in 18 languages [Zhang et al., 2022b]. Our analysis supports that XTR indeed benefits from retrieving more contextualized tokens in relevant contexts, while making the scoring stage two-to-three orders of magnitude cheaper.
2.1 Multi-vector Retrieval
Single-vector retrieval models, also known as dual encoders, encode an input text sequence as a single dense embedding and define the similarity of a query and a document based on the dot product [Lee et al., 2019, Karpukhin et al., 2020]. Multi-vector retrieval models, on the other hand, make use of multiple dense embeddings for each query and document, typically leveraging all contextualized word representations of the input to gain improved model expressivity.
Consider a query  $Q = \{\mathbf{q}_i\}_{i=1}^n$  and a document  $D = \{\mathbf{d}_j\}_{j=1}^m$  where  $\mathbf{q}_i$  and  $\mathbf{d}_j$  denote the  $d$ -dimensional query token vector and the document token vector, respectively. Multi-vector retrieval models compute the query-document similarity as follows:  $f(Q, D) = \sum_{i=1}^n \sum_{j=1}^m \mathbf{A}_{ij} \mathbf{P}_{ij}$  where  $\mathbf{P}_{ij} = \mathbf{q}_i^\top \mathbf{d}_j$  and  $\mathbf{A} \in \{0, 1\}^{n \times m}$  denotes the alignment matrix with  $\mathbf{A}_{ij}$  being the token-level alignment between the query token vector  $\mathbf{q}_i$  and the document token vector  $\mathbf{d}_j$ . The sum-of-max operator of ColBERT [Khattab and Zaharia, 2020] sets  $\mathbf{A}_{ij} = \mathbb{1}_{[j = \arg\max_{j'} (\mathbf{P}_{ij'})]}$  where the argmax operator is over  $1 \leq j' \leq m$  (i.e., tokens from a single document  $D$ ) and  $\mathbb{1}_{[*]}$  is an indicator function. Then,  $f_{\mathrm{ColBERT}}(Q, D)$  is defined as follows:
$$
f _ {\mathrm {C o l B E R T}} (Q, D) = \frac {1}{n} \sum_ {i = 1} ^ {n} \sum_ {j = 1} ^ {m} \mathbf {A} _ {i j} \mathbf {P} _ {i j} = \frac {1}{n} \sum_ {i = 1} ^ {n} \max  _ {1 \leq j \leq m} \mathbf {q} _ {i} ^ {\top} \mathbf {d} _ {j}. \tag {1}
$$
Figure 1: Overview of XTR. ColBERT has the three-stage inference combining (a) the token retrieval, (b) the gathering and (c) the scoring stages ( $\S 2.2$ ). XTR leverages the token retrieval for both training and inference. XTR efficiently obtains the score of each candidate document by applying  $f_{\mathrm{XTR}}$  (or  $f_{\mathrm{XTR'}}$ ) on the retrieved tokens, completely removing the gathering stage ( $\S 3.2$ ).
Here, we include the normalizer  $n$ , which was not included in the original sum-of-max, as it stabilizes training while not affecting the ranking during inference. After computing the query-document similarity, multi-vector retrieval models are typically trained with a cross-entropy loss over in-batch negatives [Santhanam et al., 2022b, Qian et al., 2022]. Specifically, given a positive document  $D^{+}$  for  $Q$  and a set of mini-batch documents  $D_{1:B} = [D_1,\dots ,D_B]$  where  $D^{+}\in D_{1:B}$ , they minimize the cross-entropy loss defined as:  $\mathcal{L}_{\mathrm{CE}} = -\log \frac{\exp f(Q,D^{+})}{\sum_{b = 1}^{B}\exp f(Q,D_{b})}$ .
2.2 Three-stage inference of Multi-vector Retrieval
Unlike dual encoder models, finding the maximum scoring document—the document that maximizes eq. (1)—cannot be directly handled by MIPS as the scoring function uses a non-linear, sum-of-max operation. Instead, a multi-vector retrieval model typically takes the following steps for the inference. 1) Token Retrieval: for each of the  $n$  query token vectors, it first retrieves  $k^t$  document token vectors, which is simply used to form initial candidate document set by taking the union of source documents of retrieved tokens. The total number of candidate documents is up to  $nk^t$  if each token is coming from a unique document. 2) Gathering: since the scoring function eq. (1) requires the computation over all document tokens, multi-vector models need to load all of the token vectors of the candidate documents. To optimize the loading process, a RAM-based index is often employed. 3) Scoring: to provide final ranks of candidate documents, multi-vector models score all the candidate documents with eq. (1). This stage is also called refinement. Note that the training of typical multi-vector models only takes care of the scoring stage with mini-batch documents. Finally, top-  $k$  documents are returned based on the computed scores. The three-stage inference is illustrated in the top of Figure 1.
3 XTR: Contextualized Token Retriever
Unlike existing multi-vector models that follow the retrieve-gather-score stages, XTR directly scores documents utilizing the tokens retrieved from the token retrieval stage. In this section, we start by showing why the existing cross entropy loss with the sum-of-max scoring function would fail on the first-stage token retrieval. Then, we introduce simple but important modifications for XTR.
2In fact, each candidate document of a T5-based ColBERT is retrieved by 1.48 tokens per on average, meaning that the most of the candidate documents are unique.
Given a positive document  $D^{+}$  and a set of negative documents  $D_{1:r}^{-} = [D_{1}^{-},\dots ,D_{r}^{-}]$  for a query  $Q$ , the first-stage token retrieval needs to retrieve the tokens of  $D^{+}$ , but not the tokens of negative documents. However, the following example shows that the sum-of-max operator used by ColBERT is not specifically designed to retrieve tokens of relevant documents.
Failure case Assume that  $f_{\mathrm{ColBERT}}(Q, D^{+}) = 0.8$  where all the individual max token similarity (i.e.,  $\mathbf{q}_i^\top \mathbf{d}_j^+$  where  $\mathbf{A}_{ij} = 1$ ) is 0.8. On the other hand, assume  $f_{\mathrm{ColBERT}}(Q, D^-) = 0.2$  for all  $D^- \in D_{1:r}^-$  where each  $D^-$  has a highly peaked token similarity greater than 0.8 but others close to zero (i.e., there exists  $\mathbf{q}_i^\top \mathbf{d}_j^- > 0.8$  where  $\mathbf{A}_{ij} = 1$  while other  $\mathbf{q}_i^\top \mathbf{d}_j^- \to 0$ ). Since the sum-of-max operator only cares about the document-level scores, the cross entropy loss would be close to zero during training. However, for each of  $n$  query tokens, if there exists at least one negative document token that has a high token similarity greater than 0.8, the token retrieval with top-  $k' = 1$  would fail to retrieve any tokens of  $D^{+}$ . As a result, multi-vector retrieval model with the sum-of-max operator will not be able to lower the high scores of some negative tokens. Figure 2 shows that the sum-of-max training causes many document tokens to have unreasonably high scores regardless of their actual relevance to the query tokens.
Figure 2: Density histogram of 4,000 token retrieval scores (cosine similarity). Training with  $f_{\mathrm{ColBERT}}$  (T5-ColBERT; §4) causes many document tokens to have extremely high scores regardless of their actual relevance with respect to the input query tokens. XTR mitigates this problem with a better training objective.
3.1 In-Batch Token Retrieval
To train multi-vector retrieval models to directly retrieve tokens of relevant documents, we simulate the token retrieval stage during training. This can be simply achieved by employing a different alignment strategy  $\hat{\mathbf{A}}$ . Specifically, we set the alignment  $\hat{\mathbf{A}}_{ij} = \mathbb{1}_{[j\in \mathrm{top - }k_{j'}(\mathbf{P}_{ij'})]}$  where the top- $k$  operator is applied over  $1\leq j^{\prime}\leq mB$  (i.e., tokens from  $B$  mini-batch documents) returning the indices of  $k$  largest values. During training, we use a hyperparameter  $k_{\mathrm{train}}$  for the top- $k$  operator. Then, we simply modify eq. (1) as follows:
$$
f _ {\mathrm {X T R}} (Q, D) = \frac {1}{Z} \sum_ {i = 1} ^ {n} \max  _ {1 \leq j \leq m} \hat {\mathbf {A}} _ {i j} \mathbf {q} _ {i} ^ {\top} \mathbf {d} _ {j}. \tag {2}
$$
The intuition is that we consider the token similarities within  $D$  only when they are high enough to be retrieved within top- $k_{\text{train}}$  from a mini-batch. Here, we use a normalizer  $Z = |\{i|\exists j, s.t.\hat{\mathbf{A}}_{ij} > 0\}|$ , which is essentially the number of query tokens that retrieved at least one document token of  $D$ . If all  $\hat{\mathbf{A}}_{ij} = 0$ , we clip  $Z$  to a small number and  $f_{\text{XTR}}(Q, D)$  becomes 0. As a result, our model cannot assign a high token similarity to negative documents as it blocks tokens of positive documents to be retrieved. With the previous failure case where  $f_{\text{ColBERT}}$  assigned a high score on  $D^{+}$  even though it cannot be retrieved, our similarity function incurs a high loss as  $f_{\text{XTR}}(Q, D^{+}) = 0$  during training (since tokens of  $D^{+}$  were not retrieved). For training, we use the same cross entropy loss defined in §2.1 with our new scoring function. Note that the training data only contains document-level annotations, but XTR encourages important tokens from positive documents to be retrieved.
3.2 Scoring Documents using Retrieved Tokens
During inference, multi-vector retrieval models first have a set of candidate documents  $\hat{D}_{1:C}$  from the token retrieval stage:
$$
\hat {D} _ {1: C} = \left\{\hat {D} \mid d _ {j} \in \hat {D} \wedge d _ {j} \in \operatorname {t o p} - k ^ {\prime} \left(\mathbf {q} _ {*}\right) \right\}. \tag {3}
$$
Indeed, our derivative analysis in Appendix A shows that the token-level similarity would not change if the document-level scores are already well discriminated.
4We tried different normalizers such as  $n$  and found that  $Z$  works the best while stabilizing the training.
Figure 3: Comparison of  $f_{\mathrm{ColBERT}}$  in eq. (1) and  $f_{\mathrm{XTR'}}$  in eq. (4). Assume that  $D_a$  and  $D_b$  were selected as initial candidate documents from the token retrieval stage.  $f_{\mathrm{ColBERT}}$  loads all token vectors of  $D_a$  and  $D_b$  and exhaustively recomposes pairwise token similarity to obtain the max values (red boxes). On the other hand,  $f_{\mathrm{XTR'}}$  does not load any token vectors and reuses retrieval scores from the first-stage token retrieval. Assume that, with the top-2 token retrieval results, the first query token retrieved each max score of  $D_a$  and  $D_b$ , but the second query token retrieved two tokens only from  $D_a$  but not  $D_b$ . We impute the missing similarity  $m$  for  $D_b$  (denoted as yellow dashed box) by finding its upper bound using the top-2 score (denoted as  $s_2$ ) of the second query token (i.e.,  $m \leq s_2 \leq s_1$ ).
Scoring	Estimated FLOPs/query	Setting
fColBERT	n2k′(2m d + m + 1)	0.36 × 109	M = 3 × 109, n = 16, d = 128,
fXTR'	n2k′(r + 1)	0.09 × 106	k′ = 100, m = 55, r = 2.5
Table 1: FLOPs comparison of ColBERT and XTR for the scoring stage. XTR only adds minimal complexity for scoring each candidate document. The setting is derived from MS MARCO.
Here, top- $k^{\prime}(\mathbf{q}_{*})$  is a union of top- $k^{\prime}$  document tokens (from the entire corpus) based on the inner product scores with each query vector (i.e.,  $\mathbf{q}^\top \mathbf{d}$ ). Given the  $n$  query token vectors, there are  $C$  ( $\leq nk^{\prime}$ ) candidate documents. Previous methods load the entire token vectors of each document and compute eq. (1) for every query and candidate document pair, which takes  $\mathcal{O}(n^{2}k^{\prime}\bar{m}d)$  computation per query ( $\bar{m} =$  average document length). Instead, we propose to score the documents solely using the retrieved token similarity. This significantly reduces the computational cost for the scoring stage since re-using the token retrieval scores removes computing redundant inner products and unnecessary (non-max) inner products. Furthermore, the expensive gathering stage (which requires loading all the document token vectors for computing eq. (1)) can be removed completely. Unlike previous work [Macdonald and Tonellotto, 2021] that leverages token retrieval to sort first-stage candidate documents before the scoring stage, we aim to directly provide the final scores of documents.
Missing similarity imputation During inference, we retrieve  $k^{\prime}$  document tokens for each of  $n$  query tokens. Assume that each document token belongs to a unique document, providing  $C = nk^{\prime}$  candidate documents in total. This leaves us with a single token similarity to score each document in the absence of the gathering stage. However, during training—either with eq. (1) or eq. (2)—each positive document has up to  $n$  (max) token similarities to average, which mostly converges to  $n$  as training proceeds. Hence, during inference, we impute the missing similarity for each query token treating each of candidate documents as if it were positive with  $n$  token similarities.
For every candidate document  $\hat{D}$ , we first define the following scoring function for the inference:
$$
f _ {\mathrm {X T R} ^ {\prime}} (Q, \hat {D}) = \frac {1}{n} \sum_ {i = 1} ^ {n} \max  _ {1 \leq j \leq m} \left[ \hat {\mathbf {A}} _ {i j} \mathbf {q} _ {i} ^ {\top} \mathbf {d} _ {j} + (1 - \hat {\mathbf {A}} _ {i j}) m _ {i} \right]. \tag {4}
$$
This is similar to eq. (2), but introduces  $m_i \in \mathbb{R}$ , which estimates the missing similarity for each  $q_i$ .  $\hat{\mathbf{A}}$  is defined similar to the one described in eq. (2) except that it uses  $k'$  for the top-  $k$  operator. Each  $q_i$  would take the missing similarity  $m_i$  as the maximum value if  $\hat{\mathbf{A}}_{i*} = 0$  and  $m_i \geq 0$ . Importantly,  $f_{\mathrm{XTR'}}$  removes the need of recomputing any  $\mathbf{q}_i^\top \mathbf{d}_j$  since when  $\hat{\mathbf{A}}_{ij} = 1$  we already know the retrieval score from the token retrieval stage, and when  $\hat{\mathbf{A}}_{ij} = 0$  we simply don't need to compute it as  $\hat{\mathbf{A}}_{ij} \mathbf{q}_i^\top \mathbf{d}_j = 0$ . Note that when every  $\hat{\mathbf{A}}_{ij} = 1$ , the equation becomes the sum-of-max operator. On the other hand, when no document tokens of  $\hat{D}$  were retrieved for  $q_i$  (i.e.,  $\hat{\mathbf{A}}_{i*} = 0$ ), we fall back to the imputed score  $m_i$ , which provides an approximated sum-of-max result.
MS	AR	TO	FE	CF	SF	CV	NF	NQ	HQ	FQ	SD	DB	QU	Avg.
One Retriever per Domain
GenQ	40.8	49.3	18.2	66.9	17.5	64.4	61.9	31.9	35.8	53.4	30.8	14.3	32.8	83.0	43.1
PTRretriever	-	58.8	25.6	76.2	23.5	63.8	70.2	33.7	45.6	61.7	43.0	18.3	34.4	87.5	49.4
One Retriever for All
BM25	22.8	31.5	36.7	75.3	21.3	66.5	65.6	32.5	32.9	60.3	23.6	15.8	31.3	78.9	44.0
ColBERT	40.1	23.3	20.2	77.1	18.4	67.1	67.7	30.5	52.4	59.3	31.7	14.5	39.2	85.4	45.1
GTRbase	42.0	51.1	21.5	66.0	24.1	60.0	53.9	30.8	49.5	53.5	34.9	14.9	39.2	88.1	45.2
T5-ColBERTbase	45.6	28.8	31.1	72.4	18.1	70.4	68.3	34.0	52.2	61.7	33.4	14.1	41.6	82.3	46.8
XTRbase	45.0	40.7	31.3	73.7	20.7	71.0	73.6	34.0	53.0	64.7	34.7	14.5	40.9	86.1	49.1
Spladev2	43.3	47.9	27.2	78.6	23.5	69.3	71.0	33.4	52.1	68.4	33.6	15.8	43.5	83.8	49.9
ColBERTv2	-	46.3	26.3	78.5	17.6	69.3	73.8	33.8	56.2	66.7	35.6	15.4	44.6	85.2	49.9
GTRxxl	44.2	54.0	23.3	74.0	26.7	66.2	50.1	34.2	56.8	59.9	46.7	16.1	40.8	89.2	49.1
T5-ColBERTxxl	47.3	33.8	31.0	74.2	19.7	73.1	75.8	35.2	60.5	65.2	43.5	17.1	45.0	86.0	50.8
XTRxxl	46.6	44.2	30.9	77.0	24.5	74.3	78.9	35.3	60.9	66.2	43.8	17.1	44.3	88.1	52.7
LoTTE Search	LoTTE Forum
Writing	Rec.	Sci.	Tech.	Life.	Pooled	Writing	Rec.	Sci.	Tech.	Life.	Pooled
BM25	60.3	56.5	32.7	41.8	63.8	48.3	64.0	55.4	37.1	39.4	60.6	47.2
ColBERT	74.7	68.5	53.6	61.9	80.2	67.3	71.0	65.6	41.8	48.5	73.0	58.2
GTRbase	74.1	65.7	49.8	58.1	82.0	65.0	69.2	62.0	33.7	47.6	72.2	54.9
XTRbase	77.0	69.4	54.9	63.2	82.1	69.0	73.9	68.7	42.2	51.9	74.4	60.1
Spladev2	77.1	69.0	55.4	62.4	82.3	68.9	73.0	67.1	43.7	50.8	74.0	60.1
ColBERTv2	80.1	72.3	56.7	66.1	84.7	71.6	76.3	70.8	46.1	53.6	76.9	63.4
GTRxxl	83.9	78.0	60.0	69.5	87.4	76.0	79.5	73.5	43.1	62.6	81.9	66.9
XTRxxl	83.3	79.3	60.8	73.7	89.1	77.3	83.4	78.4	51.8	64.5	83.9	71.2
$\spadesuit$  : cross-encoder distillation  $\spadesuit$  : model-based hard negatives
Table 2: (top) nDCG@10 on MS MARCO (in-domain) and BEIR (zero-shot). The last column shows the average over 13 BEIR datasets. (bottom) Top-5 retrieval accuracy on LoTTE datasets (zero-shot).
In fact, we can find the upper bound of the missing similarity. For every token retrieval with  $\mathbf{q}_i$ , the missing similarity of the query token for  $\hat{D}$  will be upper bounded by its last top- $k'$  score. Specifically, for each query token  $q_i$ , we have the following top- $k'$  token similarity during inference:  $[\mathbf{q}_i^\top \mathbf{d}_{(1)}, \ldots, \mathbf{q}_i^\top \mathbf{d}_{(k')}]$ . Here, each  $\mathbf{d}_{(*)}$  could come from a different document. Since the missing similarity would have a score less than equal to the score of the last retrieved token, we know that  $m_i \leq \mathbf{q}_i^\top \mathbf{d}_{(k')}$ . With a larger  $k'$ , the upper bound becomes tighter. In our experiments, we show that simply choosing  $m_i = \mathbf{q}_i^\top \mathbf{d}_{(k')}$  works well especially when a model is trained with  $f_{\mathrm{XTR}}$ . While we also tried more complicated imputation methods based on regression, our method was competitive enough despite its simplicity. The imputation process is illustrated in Figure 3.
Table 1 shows the estimated FLOPs of ColBERT and XTR (see Appendix B for more details). Due to the differences in hardware and infrastructure, we mainly compared the theoretical FLOPs. XTR reduces the FLOPs at the scoring stage by  $4000 \times$  making multi-vector retrieval more efficient.