Measuring and Addressing Indexical Bias in Information Retrieval (2024)

Abstract: Information Retrieval (IR) systems are designed to deliver relevant content, but traditional systems may not optimize rankings for fairness, neutrality, or the balance of ideas. Consequently, IR can often introduce indexical biases, or biases in the positional order of documents. Although indexical bias can demonstrably affect people’s opinion, voting patterns, and other behaviors, these issues remain understudied as the field lacks reliable metrics and procedures for automatically measuring indexical bias. Towards this end, we introduce the PAIR framework, which supports automatic bias audits for ranked documents or entire IR systems. After introducing DUO, the first general-purpose automatic bias metric, we run an extensive evaluation of 8 IR systems on a new corpus of 32k synthetic and 4.7k natural documents, with 4k queries spanning 1.4k controversial issue topics. A human behavioral study validates our approach, showing that our bias metric can help predict when and how indexical bias will shift a reader’s opinion.

## introduction web search, recommendation systems, and personal assistants are all powerful information retrieval (ir) tools that can help people make decisions @xcite . however, skewed results can lead people to make biased @xcite or misinformed conclusions @xcite . for example, undecided voters can be swayed to vote for a candidate who is favored in search results @xcite . this well-known problem is called the search engine manipulation effect (seme; @xcite @xcite @xcite . seme results not from the content of any particular document, but rather from 4. 5. ## foundational bias corpora pair relies on two evaluation corpora, wiki-balance synthetic and wiki-balance natural . the former will critically support the duo computation for automatic, cross-domain evaluation of indexical bias in ir. the latter helps us approximate real-world performance. see source. wiki-balance reflects 1,364 of the most controversial topics from english wikipedia, a comprehensive and reliable knowledge source @xcite . high-level seed topics come from the titles of wikipedia articles that were edited in an oscillatory manner (e.g., bullfighting; beyoncé ; climate change; the israel-palestine conflict). 2 for details on the topic distribution, see for each topic, we prompt gpt 3.5 turbo to generate 10 specific debate questions on that topic. for example, on noam chomsky, we generate queries like is noam chomsky's linguistic theory still relevant? figure 2 contains additional examples of wiki-balance queries. for space they are abbreviated in the figure, but all queries have fully grammatical clauses. we sample a subset of these 138k queries to seed wiki-balance. ## ir bias audits with duo ( §5) and evaluation corpora ( §3), we can audit the indexical bias of leading ir models. the labeled synthetic corpus shows us how duo compares with prior bias metrics. however, synthetic results may not reflect end-model behavior on real document distributions. for more realistic results, we evaluate on the natural corpus, where seme behavioral experiments clearly validate duo as predictive. the results in tables 4 and 5 use both standard relevance metrics and the bias metrics from §4, and are averaged over 3 random seeds. along with concurrent work @xcite , pair is among the first cross-domain audits of indexical biases in open-source ir systems. ## applications and extensions the pair framework is general, and future work can explore its extensions outside of search in other domains where information follows a rank order, such as a chatbot's probability-ranked utterances, a politician's most common phrases, or even the ordered paragraphs of a news article (i.e., framing; @xcite . indexical bias evaluation will become increasingly relevant to mitigate harms in more recent generative methods which rely on retrieval augmented generation for grounding knowledge @xcite @xcite . a natural extension of the duo could also handle issues with more than two sides, simultaneously incorporating multiple axes of semantic variation. our formalization is readily prepared for such an extension. if we generate up to m different viewpoint axes for each issue in wiki-balance synthetic , we could simply increase the dimensionality of the pca projection in §4 such that polarization scores become polarization vectors p @xmath0. this would allow a separate variance utility computation u x v (i, r) = 1 i i @xmath1. depending on the application, one could aggregate across axes x, for example by taking the maximum bias score as in the multi-group attention bias ratio of @xcite . for more discussion of multi-group extensions of indexical bias metrics, see the group relevance framework of @xcite . additionally, there is still much to be learned about the formal and mathematical properties of the duo metric, especially in the context of optimization and reranking. prior works suggest that fluctuations in fairness metrics like duo can lead to unstable training @xcite . this may pose a challenge to the integration of duo with current systems. one other bottleneck that may prevent the widespread adoption of duo in reranking is the expensive computation for normalization. our code includes a stochastic approximation, but the code could be further optimized. ## conclusion from web search to personal assistants, ir systems have the potential to skew users' opinions on a wide range of topics, from media and entertainment to political issues and scientific insights. before one can address the problematic outcomes of such manipulation, one should first expect to measure indexical bias, the root of this psychological effect. pair is the first completely automatic method for evaluating indexical bias in ir systems without the need for manual human annotations. pair is built on two bias evaluation corpora, which support our duo metric. since duo requires no human supervision, it can serve as a scalable evaluation metric and, in future work, as an automatic reranking criterion. we demonstrate the psychological validity of duo using a controlled experiment. after proving its validity, we use duo to run an extensive audit over the biases in current ir technologies, both open and closed-source. together, these contributions provide a basis for future efforts to measure and address indexical bias in ir.