SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Experimental Setting Following Ni et al. [2021], we fine-tune XTR on MS MARCO with a fixed set of hard negatives from RocketQA [Qu et al., 2021]. Then, we test XTR on MS MARCO (MS; in-domain) and zero-shot IR datasets. For the zero-shot evaluation, we use 13 datasets from BEIR [Thakur et al., 2021] (see Appendix C for acronyms), 12 datasets from LoTTE [Santhanam et al., 2022b], and 4 datasets on open-domain QA passage retrieval (EQ: EntityQuestions [Sciavolino et al., 2021], NQ, TQA: TriviaQA, SQD: SQuAD). We also train multilingual XTR (mXTR) and evaluate it on MIRACL [Zhang et al., 2022b], which contains retrieval tasks in 18 languages. The performance gap between T5-ColBERT [Qian et al., 2022] and XTR shows the improvement with our methods on a multi-vector retrieval model. For implementation details and baselines, see Appendix C. For the relationship between hyperparameters (e.g.,  $k_{\text{train}}$  and  $k'$ ), see §5.3.
We found that directly training with  $f_{\mathrm{XTR}}'$  instead of  $f_{\mathrm{XTR}}$  fails to converge, which we leave as future work.
EQ	NQ	TQA	SQD
Top-20	Top-100	Top-20	Top-100	Top-20	Top-100	Top-20	Top-100
BM25*	71.4	80.0	62.9	78.3	76.4	83.2	71.1	81.8
DPRmulti + BM25*	73.3	82.6	82.6	88.6	82.6	86.5	75.1	84.4
ARTMS MARCO*	75.3	81.9	-	-	78.0	84.1	68.4	80.4
GTRbase*	73.3	80.6	78.5	86.5	76.2	83.4	65.9	77.6
GTRxxl*	75.3	82.5	83.5	89.8	81.7	86.6	70.4	80.6
DPRmulti	56.7	70.0	79.5	86.1	78.9	84.8	52.0	67.7
ColBERT	-	-	79.1	-	80.3	-	76.5	-
XTRbase	79.0	85.2	79.3	88.1	80.3	85.5	78.2	85.9
XTRxxl	79.4	85.9	84.9	90.5	83.3	87.1	81.1	87.6
$\spadesuit$  : sparse component  $\spadesuit$  : retrieval pre-training
Table 3: Zero-shot passage retrieval accuracy on open-domain question answering datasets. In-domain performances are underlined and all the other performances are based on the zero-shot evaluation. For EntityQuestions, we report macro-averaged performances over different relations.
ar	bn	en	es	fa	fi	fr	hi	id	ja	ko	ru	sw	te	th	zh	de	yo	Avg.
BM25	48.1	50.8	35.1	31.9	33.3	55.1	18.3	45.8	44.9	36.9	41.9	33.4	38.3	49.4	48.4	18.0	-	-	-
mDPR	49.9	44.3	39.4	47.8	48.0	47.2	43.5	38.3	27.2	43.9	41.9	40.7	29.9	35.6	35.8	51.2	-	-	-
BM25 + mDPR	67.3	65.4	54.9	64.1	59.4	67.2	52.3	61.6	44.3	57.6	60.9	53.2	44.6	60.2	59.9	52.6	-	-	-
Trained on English MS MARCO
mContriever (en)	55.3	54.2	37.9	34.1	42.6	51.2	31.5	40.6	36.8	38.3	46.2	39.9	44.4	48.7	52.4	27.4	32.9	32.9	41.5
mXTRbase (en)	66.1	64.7	49.4	40.5	47.9	62.2	37.5	51.4	46.9	56.8	64.0	49.8	43.0	67.7	69.2	47.2	34.5	40.6	52.2
mXTRxxl (en)	74.1	75.5	56.0	52.4	56.1	75.1	51.4	61.8	52.0	68.7	67.4	61.3	69.7	76.0	76.9	56.9	51.7	60.3	63.5
Trained on English MS MARCO + MIRACL (16 languages)
mContriever	64.6	66.4	41.2	40.3	46.3	61.9	42.9	41.9	44.6	55.6	55.4	48.1	65.3	77.6	69.3	45.9	39.6	41.9	52.7
mXTRbase	73.0	73.9	46.1	42.6	51.0	70.5	39.3	51.3	54.2	62.3	67.7	54.5	69.7	80.7	76.1	51.4	36.1	46.8	58.2
mXTRxxl	77.8	78.4	52.5	48.9	56.0	76.0	52.9	61.5	54.9	73.4	68.5	66.2	79.4	84.3	80.7	58.9	52.8	62.4	65.9
Table 4: nDCG@10 on 18 multilingual retrieval tasks from MIRACL. Each row shows the performance of a single multilingual retrieval model. The last two surprise languages (de and yo) are not included in the training dataset of MIRACL. The last column shows the average over 18 languages.
4.1 In-domain Document Retrieval
MS MARCO The first column of Table 2 (top) shows nDCG@10 on MS MARCO (see Table D.1 for recall@100). XTR outperforms most models and remains competitive with T5-ColBERT. This is encouraging since XTR significantly reduces the cost of the gathering-scoring stage. Note that MS MARCO may fail to reflect the actual improvement of state-of-the-art [Arabzadeh et al., 2022].
4.2 Zero-shot Document Retrieval
BEIR & LoTTE Table 2 (top; except the first columns) shows nDCG@10 on BEIR (see Table D.1 for recall@100). XTR<sub>xxl</sub> achieves the new state-of-the-art performances significantly outperforming both per-domain models and single model state-of-the-art. Simply scaling XTR removes the needs of designing distillation or hard negative mining pipelines [Santhanam et al., 2022b, Formal et al., 2021]. Results on LoTTE (Table 2 bottom) also show that XTR<sub>base</sub> is better than ColBERT and competitive with distillation-based models while XTR<sub>xxl</sub> advances the state-of-the-art.
Passage retrieval for open-domain QA Table 3 shows results on four open-domain QA datasets. While previous work often includes sparse retrievers (e.g., BM25) [Chen et al., 2021] or contrastive pre-training [Ram et al., 2022, Sachan et al., 2022a,b] to achieve better performances on EntityQuestions, XTR simply fine-tuned on MS MARCO achieves the state-of-the-art performance.
4.3 Multilingual Document Retrieval
MIRACL Since XTR does not need any secondary pre-training, we expect it to be better at multilingual retrieval by better utilizing the multilingual language models. We train a multilingual version of XTR with mT5 [Xue et al., 2021] and test it on multilingual retrieval tasks in 18 languages. Table 4 shows that mXTR greatly outperforms mContriesver that uses expensive contrastive pretraining, as well as the hybrid model, BM25 + mDPR.
Figure 4: (top) Gold token retrieval performances of T5-ColBERT and XTR. We plot the probability of each retrieved document token at rank  $k$  coming from the gold document. (bottom) Lexical token retrieval performances of T5-ColBERT and XTR. We plot the probability of each retrieved document token at rank  $k$  being lexically identical to its query token.
Model	Imputation	MRR@10	R@1000
T5-ColBERTbase	None	0.0	0.0
top-k' score	27.7	91.8
XTRbase	None	22.6	88.7
mi=0	36.2	97.3
mi=0.2	36.4	97.3
top-k' score	37.4	98.0
Table 5: Impact of training objectives and imputation methods comparing T5-ColBERT and XTR. For both models, we apply  $f_{\mathrm{XTR}}$  during inference. We report MRR@10 and Recall@1000 on the MS MARCO development set.
Figure 5: Recall@100 of XTR and T5-ColBERT with different  $k'$ . For T5-ColBERT, we use either  $f_{\mathrm{XTR}}'$  or  $f_{\mathrm{ColBERT}}$ .
5.1 Towards Better Token Retrieval
Gold token retrieval If the tokens of gold documents are not retrieved at all, multi-vector retrieval models would fail to retrieve the gold documents. Hence, a better token retrieval would contain these gold tokens more often in their top results. In Figure 4 (top), we show the probability of a token at the rank  $k$  coming from the gold documents of a query. To compute the probability for the rank  $k$ , we simply count the number of an event where a token at rank  $k$  belongs to the gold document and divide it by the number of tokens at rank  $k$ . While this is measuring the precision of the token retrieval, we observed a similar trend for the recall of gold tokens. Compared to T5-ColBERT, XTR retrieves gold tokens with higher probability, even on MS MARCO. This shows that the training objective of XTR encourages it to retrieve tokens from more relevant context.
Lexical token retrieval In Figure 4 (bottom), we show the probability of a token at the rank  $k$  being the same as its query token (e.g., 'insulin' retrieving 'insulin's). T5-ColBERT has very high probability of retrieving the same token across different ranks and datasets. However, it is unclear to what extent the token retrieval stage should behave as sparse retrieval, as it might suffer from the vocabulary mismatch problem. XTR effectively lowers the reliance on the lexical matching while preserving a good amount of lexical precision so that it would achieve a high retrieval accuracy on the entity-centric dataset (\$4.2). In fact, Table 6 in Appendix shows that having lower lexical matching doesn't necessarily mean a lower retrieval quality, but often means better contextualization.
5.2 Efficient Scoring
In Table 5, we show how we can employ the efficient scoring function  $f_{\mathrm{XTR^{\prime}}}$  in XTR with minimal performance losses. We apply  $f_{\mathrm{XTR^{\prime}}}$  on both T5-ColBERT and XTR, and show their performances on MS MARCO. With T5-ColBERT, even if we use the top- $k^{\prime}$  score for the imputation, the performance
Figure 6: MRR@10 of XTR with different  $k_{\mathrm{train}}$  and  $k^{\prime}$ . For T5-ColBERT, we also use  $f_{\mathrm{XTR}'}$  with the top- $k^{\prime}$  score imputation method for the inference.
Figure 7: Effect of training XTR with different batch sizes and  $k_{\text{train}}$ . For each point of the graph, we train XTR<sub>base</sub> with the specified training batch size (128, 256, 320) and  $k_{\text{train}}$  (32, 64, 128, 256) and evaluate on each dataset (MS MARCO and ArguAna). nDCG@10 of each model is reported.
is much worse than the original sum-of-max scoring. With XTR, the performance greatly improves as it has better token retrieval. Figure 5 shows how Recall@100 improves with larger  $k^{\prime}$ 's as it provides more exact upper bound for the missing similarity imputation. Table D.2 shows that even if we use smaller  $k^{\prime}$ , XTR still maintains high performances on BEIR.
5.3 Relationship between Hyperparameters
$k_{\mathrm{train}}$  vs.  $k^{\prime}$  In Figure 6, we show MRR@10 of XTR trained with different  $k_{\mathrm{train}}$  and evaluated with different  $k^{\prime}$  on the MS MARCO development set. While all variants of XTR prefer larger  $k^{\prime}$ , ones trained with smaller  $k_{\mathrm{train}}$  show higher performances than others under small  $k^{\prime}$  settings. XTR with larger  $k_{\mathrm{train}}$  exhibits better performances than ones with smaller  $k_{\mathrm{train}}$  as  $k^{\prime}$  becomes larger.
Training batch size vs.  $k_{\text{train}}$  In Figure 7, we show the relationship between the training batch size and  $k_{\text{train}}$  during training XTR. In this experiment, we use  $k' = 40,000$ . While it is evident that XTR mostly favors large training batch sizes, the optimal top- $k_{\text{train}}$  can be different for different datasets. While most datasets including MS MARCO favored a large enough  $k_{\text{train}}$ , ArguAna prefers smaller  $k_{\text{train}}$ . We hypothesize that this is due to the longer query length in ArguAna, which makes multi-vector models fall short compared to dual-encoders (see GTR vs. T5-ColBERT in Table 2).
5.4 Qualitative Analysis
Table 6 shows a prediction sample from MS MARCO. For T5-ColBERT, all of the top retrieved tokens are exact lexical matches. Surprisingly, none of the retrieved passages are about the query, demonstrating T5-ColBERT's failure to retrieve tokens from the correct context. In contrast, XTR retrieves fewer exact lexically matching tokens, but the contexts of the retrieved tokens are much more related to the query. This example explains the lower lexical token retrieval probability of XTR compared to T5-ColBERT in Figure 4 (bottom), but higher gold token retrieval performance in Figure 4 (top). For more qualitative examples, please see Appendix E.
One of the main limitations of dense retrieval models is that encoding the query and document into a single vector constrains the representational power of the models. Polyencoder [Humeau et al., 2020], MEBERT [Luan et al., 2021], and MVR [Zhang et al., 2022a] propose to use multiple embeddings, instead of one, to represent the query or the document. A more recent approach is token-level multi-vector retrieval, which stores and retrieves with every token embedding. ColBERT [Khattab and Zaharia, 2020] is probably the most renowned model in this family. ALIGNNER (i.e. T5-ColBERT) [Qian et al., 2022] extends ColBERT by scaling up the backbone language model and studying various strategies for aggregating the token-level alignment scores. These token-level retrieval models show strong effectiveness and out-of-domain generalization ability.
Efforts for reducing serving costs of multi-vector models have been mostly focused on the token-level retrieval stage. COIL [Gao et al., 2021] accelerates token-level retrieval by confining retrieval within exact match tokens, sharing the spirit of classic inverted indexing. CITADEL [Li et al., 2022] relaxes COIL with a lexical routing mechanism where a query token vector only retrieves from a subset of
T5-ColBERT token retrieval for "what is the usual pay for stock associates at michael?"
Rank	Token	Context of Token	Relevance
1	usual	routine passport services: the usual waiting time in logan to get your passport is four (4) to eight (8) weeks for routine applications.	No
2	usual	the usual pay days are the 1st and 16th of each month. for annual educational paraprofessionals there is no payroll lag.	No
5	usual	the usual part xiii tax rate is 25% (unless a tax treaty between canada and your home country reduces the rate).	No
50	usual	this is where one can challenge the judgment debtor's claim. one option creditors have is to try and make a deal with the debtor to take less than 25% (the usual amount of a wage levy).	No
100	usual	the usual maximum inventory is 1 talisman, 26 elemental runes, and 26 pure essence. the ingredients must be brought to an opposing altar ... from the runes being crafted.	No
XTR token retrieval for "what is the usual pay for stock associates at michael?"
Rank	Token	Context of Token	Relevance
1	usual	store manager. 1 salary: the usual salary a store manager receives can be anywhere around $52,000 to $115,000 annually.	No
2	usual	1 salary: the usual salary a store manager receives can be anywhere around $52,000 to $115,000 annually. 2 bonuses: publix provide bonuses that could reach up to $40,000.	No
5	average	average salaries for michaels stores stock associate: $9. michaels stores hourly pay trends based on salaries posted anonymously by michaels stores employees.	Yes
50	v	i think the avg starting pay is closer to 30k for asst mgr trainees. it is an hourly position until you are fully trained (40 hours per week).	No
100	average	average macys salaries. the average salary for macys jobs is $32,000. average macys salaries can vary greatly due to company, location, industry, experience and benefits.	No
Table 6: Token retrieval example from MS MARCO. Among the top 100 retrieved tokens,  $100\%$  of T5-ColBERT tokens are lexically identical as the query token usual while only  $8\%$  of XTR tokens are lexically identical. XTR retrieves the relevant passage by retrieving average for usual.
document token vectors routed to the same key. PLAID [Santhanam et al., 2022a] optimizes the speed of ColBERT by pruning weaker candidates in the earlier stages of retrieval and using better vector quantization. ColBERT-v2 [Santhanam et al., 2022b] further adopts residual representations with cluster centroids to improve the efficiency of ColBERT. On the other hand, how to accelerate the scoring stage remains under-explored. To the best of our knowledge, XTR is the first work to simplify the scoring stage and remove the gathering stage in multi-vector retrieval.
Multi-vector retrieval leverages query and document token representations for effective information retrieval. In this paper, we propose XTR that simplifies the existing three-stage inference of multi-vector models by improving the initial token retrieval stage. Specifically, XTR scores documents solely based on the retrieved tokens, which is also optimized during training with in-batch document tokens. As a result, XTR achieves state-of-the-art performances on zero-shot information retrieval benchmarks while greatly reducing the FLOPs of the scoring stage. We further show that our objective function indeed encourages better token retrieval, retrieving more tokens from gold documents, whose contexts are better aligned with the query.
In most of our experiments, XTR was trained on MS MARCO, a large-scale retrieval dataset in English. While our experiments were conducted in a fair setting where most baseline models also utilize MS MARCO, future use cases might need to remove its dependency on MS MARCO due to the license or language-specific issue. We believe that LLM-based retrieval dataset generation [Dai et al., 2022] would be able to mitigate the problem in the future.
We would like to thank the anonymous reviewers for their helpful feedback. We also thank Nicholas Monath, Raphael Hoffmann, Kelvin Guu, Slav Petrov, and others at Google DeepMind for their helpful comments and discussion.
Nandan Thakur, Nils Reimers, Andreas Rücklé, Abhishek Srivastava, and Iryna Gurevych. Beir: A heterogeneous benchmark for zero-shot evaluation of information retrieval models. In Thirty-fifth Conference on Neural Information Processing Systems Datasets and Benchmarks Track (Round 2), 2021.
Kexin Wang, Nandan Thakur, Nils Reimers, and Iryna Gurevych. Gpl: Generative pseudo labeling for unsupervised domain adaptation of dense retrieval. In Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 2345-2360, 2022.
Linting Xue, Noah Constant, Adam Roberts, Mihir Kale, Rami Al-Rfou, Aditya Siddhant, Aditya Barua, and Colin Raffel. mt5: A massively multilingual pre-trained text-to-text transformer. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 483-498, 2021.
Wen-tau Yih, Kristina Toutanova, John C. Platt, and Christopher Meek. Learning discriminative projections for text similarity measures. In Conference on Computational Natural Language Learning, 2011.
Shunyu Zhang, Yaobo Liang, Ming Gong, Daxin Jiang, and Nan Duan. Multi-view document representation learning for open-domain dense retrieval. In Smaranda Muresan, Preslav Nakov, and Aline Villavicencio, editors, Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), ACL 2022, Dublin, Ireland, May 22-27, 2022, pages 5990-6000. Association for Computational Linguistics, 2022a.
Xinyu Zhang, Nandan Thakur, Odunayo Ogundepo, Ehsan Kamalloo, David Alfonso-Hermelo, Xiaoguang Li, Qun Liu, Mehdi Rezagholizadeh, and Jimmy Lin. Making a miracl: Multilingual information retrieval across a continuum of languages. arXiv preprint arXiv:2210.09984, 2022b.
A Derivatives w.r.t. Similarity Scores
Sum-of-max Here, we use a cross-entropy loss  $\mathcal{L}_{\mathrm{CE}}$  with the sum-of-max operator  $f_{\mathrm{ColBERT}}$  and analyze the derivatives with respect to the token similarity scores.
$$
\mathcal {L} _ {\mathrm {C E}} = - \log \frac {\exp f (Q , D ^ {+})}{\sum_ {b = 1} ^ {B} \exp f (Q , D _ {b})} = - f _ {\mathrm {C o l B E R T}} (Q, D ^ {+}) + \log \sum_ {b = 1} ^ {B} \exp f _ {\mathrm {C o l B E R T}} (Q, D _ {b}) \qquad (5)
$$
$$
f _ {\mathrm {C o l B E R T}} (Q, D) = \frac {1}{n} \sum_ {i = 1} ^ {n} \sum_ {j = 1} ^ {m} \mathbf {A} _ {i j} \mathbf {P} _ {i j} = \frac {1}{n} \sum_ {i = 1} ^ {n} \mathbf {P} _ {i \hat {j}} \tag {6}
$$
Here, we denote  $\hat{j}$  as the index of the row-wise maximum value, dependent on each  $i$  (i.e.,  $\mathbf{A}_{ij} = 1$ ). Given the cross-entropy loss with the sum-of-max operator, we compute the gradient with respect to one of the maximum token similarities  $\mathbf{P}_{i\hat{j}}^{+}$  for a positive document  $D^{+} \in D_{1:B}$ :
$$
\begin{array}{l} \frac {\partial \mathcal {L} _ {\mathrm {C E}}}{\partial \mathbf {P} _ {i \hat {j}} ^ {+}} = - \frac {f (Q , D ^ {+})}{\partial \mathbf {P} _ {i \hat {j}} ^ {+}} + \frac {1}{\sum_ {b = 1} ^ {B} \exp f (Q , D _ {b})} \frac {\partial}{\partial \mathbf {P} _ {i \hat {j}} ^ {+}} \sum_ {b = 1} ^ {B} \exp f (Q, D _ {b}) \\ = - \frac {\partial}{\partial \mathbf {P} _ {i \hat {j}} ^ {+}} \frac {1}{n} \sum_ {i = 1} ^ {n} \max _ {1 \leq j \leq m} \mathbf {P} _ {i j} ^ {+} + \frac {1}{\sum_ {b = 1} ^ {B} \exp f (Q , D _ {b})} \sum_ {b = 1} ^ {B} \frac {\partial}{\partial \mathbf {P} _ {i \hat {j}} ^ {+}} \exp f (Q, D _ {b}) \\ = - \frac {1}{n} + \frac {1}{\sum_ {b = 1} ^ {B} \exp f (Q , D _ {b})} \sum_ {b = 1} ^ {B} \exp f (Q, D _ {b}) \frac {\partial f (Q , D _ {b})}{\partial \mathbf {P} _ {i \hat {j}} ^ {+}} \\ = - \frac {1}{n} + \frac {1}{n} \frac {\exp f (Q , D ^ {+})}{\sum_ {b = 1} ^ {B} \exp f (Q , D _ {b})} = - \frac {1}{n} [ 1 - P (D ^ {+} | Q, D _ {1: B}) ]. \\ \end{array}
$$
Similarly, the gradient w.r.t. a maximum token similarity  $\mathbf{P}_{i\hat{j}}^{-}$  for a negative document  $D^{-} \in D_{1:B}$  is computed as follows:
$$
\begin{array}{l} \frac {\partial \mathcal {L} _ {\mathrm {C E}}}{\partial \mathbf {P} _ {i \hat {j}} ^ {-}} = - \frac {f (Q , D ^ {+})}{\partial \mathbf {P} _ {i \hat {j}} ^ {-}} + \frac {1}{\sum_ {b = 1} ^ {B} \exp f (Q , D _ {b})} \frac {\partial}{\partial \mathbf {P} _ {i \hat {j}} ^ {-}} \sum_ {b = 1} ^ {B} \exp f (Q, D _ {b}) \\ = \frac {1}{n} \frac {\exp f (Q , D ^ {-})}{\sum_ {b = 1} ^ {B} \exp f (Q , D _ {b})} = \frac {1}{n} P (D ^ {-} | Q, D _ {1: B}). \\ \end{array}
$$
Hence, the positive token-level score  $\mathbf{P}_{i\hat{j}}^{+}$  will gradually increase until  $P(D^{+}|Q,D_{1:B})\to 1$  and the negative token-level score  $\mathbf{P}_{i\hat{j}}^{-}$  will decrease until  $P(D^{-}|Q,D_{1:B})\to 0$ . This shows that the token-level scores are trained based on the document-level scores, which might stagnate the token-level scores. For instance, even if  $\mathbf{P}_{i\hat{j}}^{-}$  is very high—later causing  $\mathbf{d}_{\hat{j}}^{-}$  to be retrieved instead of ones from positive documents—it will not be penalized as long as  $P(D^{-}|Q,D_{1:B})$  is low enough.
In-batch token retrieval Compared to the sum-of-max operator, our in-batch sum-of-max  $f_{\mathrm{XTR}}$  considers the max values only when they are retrieved over other negative tokens in the mini-batch.
$$
f _ {\mathrm {X T R}} (Q, D _ {1: B}) = \frac {1}{Z} \sum_ {i = 1} ^ {n} \sum_ {j = 1} ^ {m} \mathbf {A} _ {i j} \hat {\mathbf {A}} _ {i j} \mathbf {P} _ {i j} = \frac {1}{Z} \sum_ {i = 1} ^ {n} \mathbf {P} _ {i \bar {j}}
$$
Here, we denote  $\bar{j}$  as the index of the row-wise maximum value that is also within the mini-batch top- $k_{\mathrm{train}}$  given  $q_{i}$  (i.e., satisfies both  $\mathbf{A}_{ij} = 1$  and  $\hat{\mathbf{A}}_{ij} = 1$ ). If there is no such  $\bar{j}$ , we simply use  $\mathbf{P}_{i\bar{j}} = 0$ . We also use a normalizer  $Z$ , which is the number of non-zero  $\mathbf{P}_{i\bar{j}}$ . In this analysis, we assume  $Z > 0$  since if every  $\mathbf{P}_{i\bar{j}}$  is zero, the gradient is undefined.
The gradient w.r.t. the maximum token similarity  $\mathbf{P}_{ij}^{+}$  (non-zero) for a positive document  $D^{+} \in D_{1:B}$  is computed as follows:
$$
\begin{array}{l} \frac {\partial \mathcal {L} _ {\mathrm {C E}}}{\partial \mathbf {P} _ {i \bar {j}} ^ {+}} = - \frac {f (Q , D ^ {+})}{\partial \mathbf {P} _ {i \bar {j}} ^ {+}} + \frac {1}{\sum_ {b = 1} ^ {B} \exp f (Q , D _ {b})} \frac {\partial}{\partial \mathbf {P} _ {i \bar {j}} ^ {+}} \sum_ {b = 1} ^ {B} \exp f (Q, D _ {b}) \\ = - \frac {1}{Z ^ {+}} \left[ 1 - \frac {\exp f (Q , D ^ {+})}{\sum_ {b = 1} ^ {B} \exp f (Q , D _ {b})} \right] \\ = - \frac {1}{Z ^ {+}} \big [ 1 - P (D ^ {+} | Q, D _ {1: B}) \big ]. \\ \end{array}
$$
This is a very similar result compared to the sum-of-max operator except that 1) the gradient is defined only when  $\mathbf{P}_{ij}^{+}$  is non-zero (i.e. retrieved) and 2) it is dependent on  $Z^{+}$ , which means that the gradient will be large whenever there is a small number of retrieved tokens from the positive document. If only a handful of tokens are retrieved for  $D^{+}$ , our objective function increases  $\mathbf{P}_{ij}^{+}$ .
For negative similarity score  $\mathbf{P}_{ij}^{-}$ , we have the following:
$$
\begin{array}{l} \frac {\partial \mathcal {L} _ {\mathrm {C E}}}{\partial \mathbf {P} _ {i \bar {j}} ^ {-}} = - \frac {f (Q , D ^ {+})}{\partial \mathbf {P} _ {i \bar {j}} ^ {-}} + \frac {1}{\sum_ {b = 1} ^ {B} \exp f (Q , D _ {b})} \frac {\partial}{\partial \mathbf {P} _ {i \bar {j}} ^ {-}} \sum_ {b = 1} ^ {B} \exp f (Q, D _ {b}) \\ = - \frac {1}{Z ^ {-}} \left[ - \frac {\exp f (Q , D ^ {-})}{\sum_ {b = 1} ^ {B} \exp f (Q , D _ {b})} \right] \\ = \frac {1}{Z ^ {-}} P (D ^ {-} | Q, D _ {1: B}). \\ \end{array}
$$