Discovering Biases in Information Retrieval Models Using Relevance Thesaurus as Global Explanation (2024)

Abstract: Most of the efforts in interpreting neural relevance models have been on local explanations, which explain the relevance of a document to a query. However, local explanations are not effective in predicting the model’s behavior on unseen texts. We aim at explaining a neural relevance model by providing lexical explanations that can be globally generalized. Specifically, we construct a relevance thesaurus containing semantically relevant query term and document term pairs, which can augment BM25 scoring functions to better approximate the neural model’s predictions. We propose a novel method to build a relevance thesaurus construction. Our method involves training a neural relevance model which can score the relevance for partial segments of query and documents. The trained model is used to identify relevant terms over the vocabulary space. The resulting thesaurus explanation is evaluated based on ranking effectiveness and fidelity to the targeted neural ranking model. Finally, our thesaurus reveals the existence of brand name bias in ranking models, which further supports the utility of our explanation method.

## introduction transformer-based information retrieval (ir) models @xcite that are trained on large datasets like ms marco @xcite are very effective in predicting relevance between a query and document. contextual representations in these models enable semantic matching, such as matching the query term "car" with the document term "vehicle". however, it is challenging for researchers to predict the potential failures of a model, such as when it matches a query term to non-relevant document terms. another potential risk associated with neural retrieval models is an unintended bias toward certain entities or groups @xcite . while it is appropriate for a model to associate the query term "car" with various car brand names (e.g., ford), the model should not exhibit a strong preference for a particular brand, leading to the model favoring that brand over another when all other factors are identical. for the safe deployment of information retrieval (ir) models in real-world scenarios, detailed global understanding of model behavior are essential, such as providing which lexical expressions are considered relevant by the models. to address these challenges and mitigate potential risks, post-hoc explanation methods for black-box machine learning models can be employed. most explanations for ir model explanations are local explanations, focusing on individual model predictions, such as a specific querydocument pair @xcite or a ranked list for a query @xcite . these explanations indicate which terms in the documents contribute to its relevance to the query. however, local explanations have two major limitations that hinder their ability to infer cases where the model may exhibit unexpected behavior. first, the explanations are limited to the terms observed in the given query and document, and bi-ases may exist in queries or documents that were not evaluated or inspected with the explanations. second, attribution to document terms by explainers may be highly dependent on the contexts of those terms, therefore it is unclear whether the attributed document terms in other contexts would match the query. to overcome these limitations, we propose building a global explanation @xcite that provides lexical insights about querydocument terms that are matched by the model in all contexts. we can describe a model's behavior in a compact and interpretable structure that is not limited to a specific instance. our proposed global explanation focuses on identifying relevant pairs of query and document terms that can effectively explain the matching behavior of neural retrieval models. we refer to this format of explanation as a relevance thesaurus, with examples illustrated in constructing a relevance thesaurus is challenging due to the large number of potential term pairs. many local @xcite and global explanation @xcite ) methods build a candidate set of features from data and adjust their scores based on the target model's outputs. however, this approach becomes infeasible when the number of features reaches to billions, as in our study. to overcome this challenge, we propose a novel approach that distill the knowledge of the target model into an intermediate neural model, parm (partial relevance model), which is then used to infer important features. parm is designed to predict a score for a term pair, which is then used to predict the score for the corresponding query-document pair. by training parm with knowledge distillation from the target neural model to be explained, we ensure that the generated relevance thesaurus faithfully explains the target model's behavior. rather than assessing the accuracy of each term pair in the relevance thesaurus individually, the thesaurus is extrinsically evaluated by integrating it into lexical matching models (bm25 @xcite and ql @xcite ), adding interpretable semantic matching to them. the resulting retrieval methods are evaluated based on retrieval effectiveness and fidelity to the target neural retrieval models. the results on multiple datasets show the effectiveness of the acquired relevance thesaurus. to demonstrate the advantages of our relevance thesaurus, we introduce three unexpected findings about the behavior of neural retrieval models trained on ms marco, obtained from our analysis of the relevance thesaurus: (1) the car-brand bias, which suggests that models exhibit biases towards certain car brands; (2) the temporal bias, which indicates that models consider distant future or past years to be more strongly associated with the query term "when" compared to the current year; (3) the postfix-a finding, which reveals that models treat the character "a" appended to a term as equivalent to a quotation mark due to encoding errors. experiments using multiple state-of-the-art neural information retrieval models demonstrate that these behaviors are not limited to the crossencoder ranker which is used to distill the relevance thesaurus but are also the case in multiple other ir models, splade @xcite and contriever @xcite . this highlights the importance of global explanations for retrieval models. ## relevance thesaurus building we define the global explanation of an information retrieval model, based on the definition by @xcite , as follows: model explanation problem. given a blackbox relevance predictor s b that takes a query q and a document d as inputs and predicts a relevance score @xmath0, a global model explainer aims to find a human-interpretable explanation e and an explanation logic ϵ. the explanation logic ϵ is a function that converts the explanation e into a global predictor s e . the global predictor s e predicts a score ŷ for (q, d), which approximates the prediction y. as a black-box predictor s b to be explained, we target the full cross-encoder (ce) document ranking model @xcite , which takes the concatenated sequence q; d into the transformer encoder to predict the relevance. as a format for an explanation e, we use a relevance thesaurus, which is a set of triplets (qt, dt, s), where qt is a query term, dt is a document term, and s is the score assigned to the term pair. to build an interpretable predictor s e , we incorporate the relevance thesaurus (e) into the bm25 scoring function to address vocabulary mismatches between queries and documents. the relevance thesaurus captures semantic relationships between terms that may not be explicitly present in the query or document. incorporating the relevance thesaurus improves the retrieval performance of the sparse retrieval model bm25, and provides an interpretable explanation of the ce model's behavior. many local and global explanation methods explicitly build a candidate set of features (e.g., terms) from data. to determine the appropriateness of each candidate feature as an explanation, these methods initially assign scores to the features. then, the scores are adjusted based on the observed behaviors of the model. this strategy maintains explicit feature candidates and their scores during the optimization process. this can be challenging as the number of features increases, especially in our explanation format where the number of term pairs can scale to billions. to address this challenge, we propose implicitly optimizing the explanation features using an intermediate neural model that scores features, namely term pairs. as an intermediate neural model, we propose parm (partial relevance model), which is designed to score relevance between partial segments of a query and document. unlike the original cross-encoder model and other relevance models that assign a score to an entire query and document, parm can predict meaningful scores for partial queries or documents. this is important because the original ce model cannot accurately assess the contributions of individual tokens when they are isolated from their original contexts. for example, if the query is "who is plato" and the document is a single term "plato", the original ce model would likely predict a score indicating non-relevance, as a document with a single term is unlikely to provide meaningful information. parm, on the other this assumption is useful because it allows us to use any term pair predicted as relevant by parm to be globally indicative of relevance, which is not possible using local explanations. parm is trained end-to-end by distilling predictions from the ce model. however, a challenge arises because parm is expected to predict a score for a query term and document term, while the available signal is only at the query-document level. to supervise term-pair level scores in parm using the ce model, alignments between query and document terms are required, but these are not directly available. the novelty of parm training lies in the ability to infer alignments in an unsupervised way. in the first stage, we train parm to predict scores for two partial query-document segment pairs, using weak alignments, such as attention scores. after the first stage, we use the parm model to infer term-level alignments. in the second stage, we use the generated alignments by parm to further fine-tune the parm model to predict appropriately scaled scores for word-level relevance. these prediction are then used to create the final explanations (e). ## conclusion we explored using a relevance thesaurus as a global explanation for neural ranking models. we proposed an effective approach for constructing the thesaurus by training a partial relevance model (parm). augmenting the acquired thesaurus into bm25 enhanced its ranking effectiveness and fidelity to the targeted neural ranking model across multiple information retrieval datasets. furthermore, the thesaurus uncovered unexpected corpusspecific behaviors and biases of state-of-the-art ranking models, highlighting its value in identifying potential issues and limitations in neural rankers. we expect a few promising research directions on top of our work. the proposed strategy of using thesaurus to explain a model can be further extended to other transformer-based models, including generative language models to discover biases on these models. for ir applications, effectiveness of bm25t could be improved by considering multiple document terms for each query term and incorporating term location information. these enhancements would better mimic neural ranking models' behavior, potentially leading to more efficient and interpretable sparse retrieval models that more closely match the performance of their neural counterparts.