ABNIRML : Analyzing the Behavior of Neural IR Models (2022)

Abstract: Pretrained contextualized language models such as BERT and T5 have established a new state-of-the-art for ad-hoc search. However, it is not yet well understood why these methods are so effective, what makes some variants more effective than others, and what pitfalls they may have. We present a new comprehensive framework for Analyzing the Behavior of Neural IR ModeLs (ABNIRML), which includes new types of diagnostic probes that allow us to test several characteristics—such as writing styles, factuality, sensitivity to paraphrasing and word order—that are not addressed by previous techniques. To demonstrate the value of the framework, we conduct an extensive empirical study that yields insights into the factors that contribute to the neural model’s gains, and identify potential unintended biases the models exhibit. Some of our results confirm conventional wisdom, for example, that recent neural ranking models rely less on exact term overlap with the query, and instead leverage richer linguistic information, evidenced by their higher sensitivity to word and sentence order. Other results are more surprising, such as that some models (e.g., T5 and ColBERT) are biased towards factually correct (rather than simply relevant) texts. Further, some characteristics vary even for the same base language model, and other characteristics can appear due to random variations during model training.1

## introduction pre-trained contextualized language models such as bert @xcite are state-of-the-art for a wide variety of natural language processing tasks @xcite . in information retrieval * currently at the university of glasgow. work done in part during an internship at the allen institute for ai. we propose a new framework aimed at analyzing the behavior of neural ir models (abnirml 2 ), which aims to probe the sensitivity of ranking models on specific textual properties. probes consist of samples comprising a query and two contrastive documents. we propose three strategies for building probes. the ''measure and match'' strategy (akin to the diagnostic datasets proposed by @xcite ) constructs probing samples by controlling one measurement (e.g., term frequency) and varying another (e.g., document length) using samples from an existing ir collection. unlike @xcite , our framework generalizes the idea to any measurable characteristic, rather than relying chiefly on prior proposed ir axioms. a second strategy, ''textual manipulation,'' probes the effect that altering the text of a document text has on its ranking. finally, a ''dataset transfer'' strategy constructs probes from non-ir datasets. the new probes allow us to isolate model characteristics-such as sensitivity to word order, degree of lexical simplicity, or even factuality-that cannot be analyzed using other approaches. using our new framework, we perform the first large-scale analysis of neural ir models. we compare today's leading ranking techniques, including those using bert @xcite and t5 @xcite , methods focused on efficiency like doct5query @xcite and epic @xcite , and dense retrieval models like ance @xcite and colbert @xcite . 3 some of our results establish widely believed, but not-yet-verified, conjectures about neural models. for example, we show that neural models can exploit richer linguistic signals than classical term-matching metrics like bm25: when controlling for term frequency match, the neural models detect document relevance much more accurately than the bm25 baseline. similarly, unlike prior approaches, rankers based on bert and t5 are heavily influenced by word order: shuffling the words in a document consistently lowers the document's score relative to the unmodified version, and neural rankers show a sensitivity to sentence order that is completely absent in classical models. other findings from abnirml are more surprising. for example, we find that the t5 and colbert models we examine prefer answers that are factually correct, implying that they encode and utilize some real-world knowledge. further, although this knowledge may be a result of the model's pre-training process, it is not necessarily utilized as a ranking signal, given that other models that use the same base language model do not have the same preference. our battery of probes also uncover a variety of other findings, including that adding additional text to documents can often exhibit adverse behavior in neural modelsdecreasing the document's score when the added text is relevant, and increasing the score when the added text is irrelevant. 3 although a multitude of other models exist, it is impractical to investigate them all. we instead focus on a representative sample of the recent and successful models and well-known baselines to provide context. in summary, we present a new framework (abnirml) for performing analysis of ad-hoc ranking models. we then demonstrate how the framework can provide insights into ranking model characteristics by providing the most comprehensive analysis of neural ranking models to date. our software implementation of the framework is easily extensible, facilitating the replication of our results and further analyses in future work. ## abnirml in order to characterize the behavior of ranking models we construct several diagnostic probes. each probe aims to evaluate specific properties of ranking models and probe their behavior (if they are heavily influenced by term matching, discourse and coherence, conciseness/verbosity, writing styles, etc.). we formulate three different approaches to construct probes (measure and match, textual manipulation, and dataset transfer). in ad-hoc ranking, a query (expressed in natural language) is submitted by a user to a search engine, and a ranking function provides the user with a list of natural language documents sorted by relevance to the query. more formally, let r(q, d) ∈ r be a ranking function, which maps a given query q and document d (each being a natural-language sequence of terms) to a real-valued ranking score. at query time, documents in a collection d are scored using r(•) for a given query q, and ranked by the scores (conventionally, sorted descending by score). learning-to-rank models optimize a set of parameters for the task of relevance ranking based on training data. ## results and analysis we present results for mmps in contextualized language models can distinguish relevance grades when tf is held constant. from positive scores indicate a preference for the manipulated document text; negative scores prefer the original text. scores marked with * are not statistically significant (see section 3.4). models generally have similar sensitivity to document length, tf, and overlap on trec dl 2019. with the exception of models that use bm25 for scoring (bm25 and dt5q), all the models we explore have similar behaviors when varying length, tf, and overlap. this suggests that although signals like tf are not required for epic, bert, and t5 to rank effectively, they still remain an important signal when available. there are bigger differences between models when exploring the antique dataset, suggesting differences in the models' capacity to generalize. we note that some of the largest differences relate to the relevance measurement, highlighting the differences in label definitions between the two datasets. trained contextualized language models are adversely affected by heavily destructive pre-processing steps. from lexical models handle typographical errors better than trained contextualized language models. in all but one case (ance dl19), bm25, lgbm, and dt5q are negatively affected by typographical errors less than the trained contextualized language models. this is a surprising result, given that contextualized language models should be able to learn common misspellings and treat them similarly to the original words (the transfer sbert model largely ignores typos). this problem is particularly apparent for epic and colbert, which perform matching on the wordpiece level. trained contextualized models behave unexpectedly when additional content is introduced in documents. we find that models that rely heavily on unigram matching (e.g., bm25) and the transfer sbert model respond positively to the addition of doct5query terms. even the doct5query model itself sees an additional boost, suggesting that weighting the expansion terms higher in the document may further improve the effectiveness of this model. however, the contextualized models often respond negatively to these additions. we also find that adding non-relevant sentences to the end of relevant documents often increases the ranking score of contextualized models. this is in contrast with models like bm25, in which the scores of relevant documents decrease with the addition of non-relevant information. from the variable length mmps, we know that this increase in score is likely not due to increasing the length alone. such characteristics may pose a risk to ranking systems based on contextualized models, in which content sources could aim to increase their ranking simply by adding non-relevant content to their documents. paraphrasing text can drastically change ranking scores. in model behaviors vary considerably with succinctness. first, bm25 has a strong (+0.66) preference for the summaries in xsum, a moderate preference for summaries in cnn (+0.37), and no significant preference for daily mail. this suggests different standards among the various datasets-for example, xsum (bbc) must use many of the same terms from the titles in the summaries, and provide long documents (reducing the score) that may not repeat terms from the title much. wmd also appears to be heavily affected by summaries, though in two of the three probes, there is insufficient evidence to claim significance. the preference for summaries in xsum can be seen across all models except t5-b, which very slightly favors the full text. although most contextualized models prefer the full text for cnn and daily mail, vbert prefers summaries for cnn (+0.16) while it prefers full text for daily mail (-0.26). such discrepancies warrant exploration in future work. wmd, t5, and colbert are biased towards factual answers. from our factuality probes, we see that most models have little preference for factual passages. however, wmd, both t5 variants, and colbert are biased towards answers that contain factually correct information. for t5 and colbert, this suggests that these models both learn some real-world information (likely in pre-training), and use this information as a signal when ranking. the larger size of t5-l appears to equip it with more knowledge, particularly about people, nationalities, and organizations. curiously, although colbert exploits this information, the vbert model (which uses the same base language model) does not appear to learn to use this information. for wmd, which doesn't have nearly the modeling capacity of t5 and col-bert, the preference for factual information must be due to the fact that the word embeddings of the question are more similar to the word embeddings from the factual phrase than to those of the non-factual phrase. although the contextualized language models should have the capacity to learn these trends and make similar decisions, this would be subject to such trends being present and distinguishable during fine-tuning. this suggests that using wmd over contextualized word embeddings may also improve the capacity of models to select factual answers. ## related work pretrained contextualized language models are neural networks that are initially trained on language modeling objectives and are later fine-tuned on task-specific objectives @xcite . well-known models include elmo @xcite , bert @xcite , and t5 @xcite . these models can effectively transfer signals to the task of ad-hoc retrieval, either by using the model directly (i.e., vanilla or mono models) @xcite or by using the outputs as features into a larger model @xcite . there has been considerable work in this area; we refer readers to @xcite for a comprehensive survey on these techniques. we shed light on the mechanisms, strengths, and weaknesses of this burgeoning body of work. diagnostic datasets, proposed by @xcite , reformulate traditional ranking axiomsfor example, that documents with a higher term frequency should receive a higher ranking score @xcite -as empirical tests for analyzing ranking models. rennings et al. studied neural ranking architectures that predate the rise of contextualized language models for ranking, and focused on just four axioms. @xcite extended this work by adding five more previously proposed ranking axioms (e.g., term proximity @xcite , and word semantics @xcite ) and evaluating on a distilled bert model. they found that the axioms are inadequate to explain the ranking effectiveness of their model. @xcite examine the extent to which these axioms, when acting in concert, explain ranking model decisions. unlike these prior lines of work, we propose new probes that shed light onto possible sources of effectiveness, and test against current leading neural ranking architectures. although some insights about the effectiveness of contextualized language models for ranking have been gained using existing datasets @xcite and indirectly through various model architectures @xcite @xcite @xcite , they only provide circumstantial evidence. for instance, several works show how contextualized embedding similarity can be effective, but this does not imply that vanilla models utilize these signals for ranking. rather than proposing new ranking models, in this work we analyze the effectiveness of existing models using controlled diagnostic probes, which allows us to gain insights into the particular behaviors and preferences of the ranking models. outside of the work in ir, others have developed techniques for investigating the behavior of contextualized language models in general. although probing techniques @xcite and attention analysis @xcite can be beneficial for understanding model capabilities, these techniques cannot help us characterize and quantify the behaviors of neural ranking models. checklist @xcite and other challenge set techniques @xcite differ conceptually from our goals; we aim to characterize the behaviors to understand the qualities of ranking models, rather than provide additional measures of model quality. ## conclusion we presented a new framework (abnirml) for analyzing ranking models based on three probing strategies. by using probes from each strategy, we demonstrated that a variety of insights can be gained about the behaviors of recently-proposed ranking models, such as those based on bert and t5. our analysis is, to date, the most extensive analysis of the behaviors of neural ranking models, and sheds light on several unexpected model behaviors. for instance, adding non-relevant text can increase a document's ranking score, even though the models are largely not biased towards longer documents. we also see that the same base language model used with a different ranking architecture can yield different behaviors, such as higher sensitivity to shuffling a document's text. we also find that some models learn to utilize real-world knowledge in the ranking process. finally, we observe that some strong biases can appear simply by chance during the training process. this motivates future investigations on approaches for stabilizing training processes and avoiding the introduction of unwanted biases.