M ulti C ontrievers: Analysis of Dense Retrieval Representations (2024)

Abstract: Dense retrievers compress source documents into (possibly lossy) vector representations, yet there is little analysis of what information is lost versus preserved, and how it affects downstream tasks. We conduct the first analysis of the information captured by dense retrievers compared to the language models they are based on (e.g., BERT versus Contriever). We use 25 MultiBert checkpoints as randomized initialisations to train MultiContrievers, a set of 25 contriever models. We test whether specific pieces of information—such as genderand occupation—can be extracted from contriever vectors of wikipedia-like documents. We measure this extractability via information theoretic probing. We then examine the relationship of extractability to performance and gender bias, as well as the sensitivity of these results to many random initialisations and data shuffles. We find that (1) contriever models have significantly increased extractability, but extractability usually correlates poorly with benchmark performance 2) gender bias is present, but is not caused by the contriever representations 3) there is high sensitivity to both random initialisation and to data shuffle, suggesting that future retrieval research should test across a wider spread of both.

## introduction dense retrievers @xcite @xcite are a standard component of retrieval augmented question answering (qa) @xcite , and other retrieval systems such as fact-checking @xcite , argumentation @xcite , and others. despite their ubiquity, we lack an understanding of the information recoverable a) set of male queries who was the first male prime minister of finland? mary somerville was a scottish scientist, writer, and polymath. in 1835 she and caroline herschel were elected as the first female honorary members of the royal astronomical society. from dense retriever representations, and how it affects retrieval system behaviour. this lack of analytical work is surprising. retrievers are widespread, and are used in contexts that require trust: increasing factuality and decreasing hallucination @xcite , and providing trust and transparency @xcite via a source document that has provenance and can be examined. the information a representation retains from a source document constrains these abilities. dense retrievers lossily encode input documents into n-dimensional representations, and by doing so necessarily emphasise some pieces of information over others. a biography of mary somerville will contain many details about her: her profession (astronomy and mathematics), her gender (female), her political influence (women's suffrage), her country of origin (scotland) and others. each of these features are relevant to different kinds of queries. which ones will a given retriever represent most recoverably? some analysis of this type exists for masked language models (mlms) ( §2.2), but there is no such analysis for retrievers, which optimise a contrastive loss. contrastive training is a very different objective than mlm, based on (dis)similarity of paired samples. the choice of pair affects fea-ture suppression -what is recoverable and what is not @xcite . so we extend this previous analytical work into the retrieval domain, by training 25 multicontrievers initialised from multibert checkpoints @xcite . this is the first study that includes variability over a large number of retriever initialisations, with some surprising results from this alone. we use information theoretic probing, also known as minimum description length (mdl) probing @xcite , to measure the information in multicontriever representations. we evaluate the models on 14 retrieval datasets from the beir benchmark @xcite . we test how well retrievers preserve information in a document, like gender and occupation, which we refer to as features. we adapt the existing datasets to better test for knowledge of these, by creating a new manually annotated gender subset of natural questions, nq-gender. we ultimately test if gender information is predictive of gender bias, as it was in previous mlm work ( §2.2).we address the following four research questions: q1 to what extent do retrievers preserve information like gender and occupation in an encoded document? ( §4.1) for both multiberts and multicontrievers, gender is more extractable than occupation, which can cause a model to rely on gender heuristics (a source of gender bias). but there are noticeable differences in the models. both features are more extractable in multicontrievers than multiberts, but there is a lower ratio (less difference) between gender and occupation. this indicates multicontrievers are less likely to rely less on gender heuristics @xcite , but still might. q2 how sensitive is this to random initialisation and data shuffle? ( §4.2) in multiberts, extractability is very sensitive to random initialisation and shuffle, in multicontrievers it is not. multicontrievers have a much smaller variance between the 25 seeds, suggesting a regularising effect. however, multicontriever performance is surprisingly sensitive to both random initialisation and to data shuffle. multicontrievers have a very wide range of performance on beir benchmarks, despite identical loss curves. but it is not easy to select a 'best' model, since the best and worst model is not consistent across datasets -the ranking of each model can change, sometimes drastically. q3 do differences in this information correlate with performance on retrieval benchmarks? ( §4.3) on partitions of examples that ostensibly require gender information (nq-gender), we show that gender extractability is highly correlated with retrieval performance. however, overall retrieval performance on benchmarks like beir is poorly correlated with extractability. this suggests that while some benchmark examples do reward models for preserving gender information, most examples do not require that, so the benchmark as a whole does not require that capability. q4 is gender information in retrievers predictive of their gender bias? ( §4.4) despite the evidence that extractability of gender information is helpful to a model, it is not the cause of gender bias in the nq-gender dataset. when we do a causal analysis by removing gender from multicontriever representations, gender bias persists, suggesting that the source of bias is in the queries or corpus. our contributions are: 1) the first information theoretic analysis of dense retrievers, 2) an analysis of variability in performance and social bias across random retriever seeds, 3) the first causal analysis of sources of social bias in dense retrievers, 4) nq-gender, an annotated subset of natural questions for queries that constrain gender, and 5) a suite of 25 multicontrievers for use in future work, with all training and evaluation code. ## background and related work the below covers dense retrievers, information theoretic probing for extractability, and what extractability can tell us about model behaviour. ## methodology we analyse the relationship between information in different model representations, and their performance & fairness. this requires at minimum a model, a probing dataset (with labels for information we want to probe for), and a performance dataset. we need some of the performance dataset to have gender metadata to calculate performance difference across gender demographics (fig 1 ) also called gender bias, or more precisely, allocational fairness. ## results we address our four research questions: how does extractability change (q1), how sensitive are retrievers to random initialisation (q2), do changes in extractability correlate with performance (q3), and is it predictive of allocational bias (q4). ## discussion, future work, conclusion we trained a suite of 25 multicontrievers, analysed their performance on the beir benchmark, probed them for gender and occupation information, and removed gender information from representations to analyse gender bias. we showed performance to be extremely variable by random seed initialisation, as was the performance ranking of different random initialisations across datasets, despite equal losses during training. best seed performances often exceed the performance of more complex dense retrievers that use explicit supervision. future analysis of retriever loss basins to look for differing generalisation strategies could be valuable @xcite . our results show that a better understanding of initialisations may be more valuable than developing new models. our work also highlights the usefulness of metadata enriched datasets for analysis, and we were limited by what was available. future work could create these datasets and then probe for additional targeted information to learn more about retrievers. this would also enable analysis of demographic biases beyond binary gender. gender and occupation extractability was not predictive of performance except in subsets of queries that require gender information. though both gender and occupation increase in multi-contrievers, the ratio between them decreases, so multicontrievers should be less likely to shortcut based on gender compared to multiberts. we established that the gender bias that we found was not caused by the representations, as it persists when gender is removed. future work should test in a pipeline is best to correct bias, and how various parts interact. this work also shows the utility of information removal (inlp, others) for causality and interpretability, rather than just debiasing. more availability of test sets for shortcutting could increase the scope of these preliminary results. finally, we have analysed only the retriever component of a retrieval system. in an eventual retrieval augmented generation task, the retrieval representation will have to compete with language model priors. the generation will be a composition between unconditionally probable text, and text attested by the retrieved data. future work could investigate the role of information extractability in the full system, and how this bears on vital questions like hallucination in retrieval augmented generation. we have done the first information theoretic analysis of retrieval systems, and the first causal analysis of the reasons for allocational gender bias in retrievers. we release our code and resources for the community to expand and continue this line of enquiry. this is particularly important in the current generative nlp landscape, which is increasingly reliant on retrievers and where understanding of models lags so far behind development.