Measuring and Addressing Indexical Bias in Information Retrieval (2024)

Abstract: Information Retrieval (IR) systems are designed to deliver relevant content, but traditional systems may not optimize rankings for fairness, neutrality, or the balance of ideas. Consequently, IR can often introduce indexical biases, or biases in the positional order of documents. Although indexical bias can demonstrably affect people’s opinion, voting patterns, and other behaviors, these issues remain understudied as the field lacks reliable metrics and procedures for automatically measuring indexical bias. Towards this end, we introduce the PAIR framework, which supports automatic bias audits for ranked documents or entire IR systems. After introducing DUO, the first general-purpose automatic bias metric, we run an extensive evaluation of 8 IR systems on a new corpus of 32k synthetic and 4.7k natural documents, with 4k queries spanning 1.4k controversial issue topics. A human behavioral study validates our approach, showing that our bias metric can help predict when and how indexical bias will shift a reader’s opinion. For data and code, see https://github.com/ SALT-NLP/pair

heading: 1 Introduction
text: Web search, recommendation systems, and personal assistants are all powerful Information Retrieval (IR) tools that can help people make decisions (Carroll, 2014; McKay et al., 2020). However, skewed results can lead people to make biased (Novin and Meyers, 2017) or misinformed conclusions (Bar-Ilan et al., 2009; Haas and Unkel, 2017). For example, undecided voters can be swayed to vote for a candidate who is favored in search results (Epstein and Robertson, 2015). This well-known problem is called the search engine manipulation effect (SEME; Allam et al., 2014; Azzopardi, 2021; Draws et al., 2021b; Epstein and Robertson, 2015; Pogacar et al., 2017). SEME results not from the content of any particular document, but rather from
the indexical bias (Mowshowitz and Kawaguchi, 2002) of their rank order,1 since people are more likely to read and trust higher-ranked documents (Schwarz and Morris, 2011).
In general, responsible IR should address indexical bias by providing not only relevant content,
1This is also known as position bias (Biega et al., 2018).
12860
but also a more fair, balanced, and representative distribution of documents (Olteanu et al., 2021). To identify and rerank biased results at scale, practitioners need automatic metrics that operate over diverse unlabeled corpora. This motivates PAIR.
Perspective-Aligned Information Retrieval, or PAIR, is a completely unsupervised method for measuring indexical bias. PAIR introduces the Discounted Uniformity of Perspectives, or DUO bias metric, which measures the variance of perspectives at different ranks within an ordered set of retrieved documents. The DUO critically depends on our WIKI-BALANCE corpus, which allows us to compute beforehand the most polarized semantic axis of debate for each issue, using principle component analysis over the document embeddings. DUO is automatic, unlike prior methods that require human labels. PAIR is also generalizable, as WIKI-BALANCE can be easily expanded beyond the 1.4k distinct issues it already supports.
To validate PAIR, we run a behavioral study which demonstrates how DUO can help predict the Search Engine Manipulation Effect (SEME). DUO is predictive whenever participants click at least one search result link. In these cases, reranking documents to minimize DUO will reduce the SEME. Finally, we leverage PAIR to evaluate 7 traditional open-source IR Systems, as well as one commercial search engine. The synthetic WIKIBALANCE corpus serves as a large and representative stress test, revealing specific weaknesses in leading systems. We complement this test with a more natural evaluation over 4,662 documents retrieved via Google Search on the same issues. In summary, we contribute:
1. The DUO Positional Bias Metric, which is an unsupervised metric that works regardless of the controversial issue of interest.
2. Two diverse, large-scale Bias Evaluation Corpora with 32k highly polarized synthetic documents and 4,662 natural documents.
3. Extensive Bias Audit Evaluations for 8 IR Systems across 15 topical domains.
4. A Human Behavioral Study which validates our approach as predictive of the SEME.

heading: 2 Related Work
text: Classifying Stance, Leaning, and Ideology. Political leaning, ideology, public opinion, and stance can be tagged at scale with supervised classifiers
(Johnson et al., 2017; Luo et al., 2020; Stefanov et al., 2020; Baly et al., 2020) or keywords (Adamic and Glance, 2005). In PAIR, however, we do not assume access to curated lists or supervised bias classifiers. We also opt not to use zero-shot ideology and stance detection, as LLM performance still varies widely in this domain (Ziems et al., 2023). Instead, we take a fully unsupervised approach and use statistics over generative models.
Diversifying and Debiasing IR. Maximum Marginal Relevance (MMR; Carbonell and Goldstein, 1998) is a popular method for diversifying IR results by minimizing documents’ mutual similarity. PAIR embeddings can serve in the MMR similarity metric, but unlike this metric, our DUO metric also incorporates rank order. There are also explicit diversification methods such as IA-Select (Santos et al., 2011), which returns a set with at least one document for each pre-defined category. If such category labels are known in advance, then diversification may be framed as Task-aware Retrieval (Asai et al., 2022) and solved with instruction tuning. Zhao et al. (2024) find such instructiontuning methods insufficient for perspective imbalance. Instead they redefine the document-query similarity score to condition on a pre-defined perspective p. However, all of these explicit methods may be less generally applicable than DUO due to their reliance on document perspective labels, which must be manually-annotated given the limitations above with zero-shot stance detection.
Auditing Bias in IR. Numerous prior studies evaluate bias in commercial systems (Mowshowitz and Kawaguchi, 2002; Kay et al., 2015; Kulshrestha et al., 2017; Chen et al., 2018; Gao and Shah, 2020; Draws et al., 2021a) like Google and Bing (Gezici et al., 2021). They often measure the diversity of intents (Agrawal et al., 2009; Clarke et al., 2008; Sakai et al., 2010), viewpoints (Draws et al., 2023, 2021a), or fairness towards protected groups (Biega et al., 2018; Yang and Stoyanovich, 2017; Zehlike et al., 2022). Apart from the fairness literature (Rekabsaz and Schedl, 2020), most prior work runs only case-studies of black-box proprietary search engines, and they rely on biaskeywords (Klasnja et al., 2022), classification, or manual annotation along a particular axis of interest, which is typically binary (e.g., left-right) and centered on American ideologies (e.g., DemocratRepublican). In comparison, PAIR lets us evaluate open-source IR systems over thousands of distinct
issue topics, which can reveal the relationship between system bias and it’s underlying algorithm and data.

heading: 3 Foundational Bias Corpora
text: PAIR relies on two evaluation corpora, WIKIBALANCESynthetic and WIKI-BALANCENatural. The former will critically support the DUO computation for automatic, cross-domain evaluation of indexical bias in IR. The latter helps us approximate real-world performance. See Table 1, left for a comparison, and right for statistics.
Source. WIKI-BALANCE reflects 1,364 of the most controversial topics from English Wikipedia, a comprehensive and reliable knowledge source (Bruckman, 2022). High-level seed topics come from the titles of Wikipedia articles that were edited in an oscillatory manner (e.g., Bullfighting; Beyoncé ; Climate Change; the Israel-Palestine Conflict).2 For details on the topic distribution, see Table 1 and the discussion in Appendix A.1.
For each topic, we prompt GPT 3.5 Turbo to generate 10 specific debate questions on that topic. For example, on Noam Chomsky, we generate queries like Is Noam Chomsky’s linguistic theory still relevant? Figure 2 contains additional examples of WIKI-BALANCE queries. For space they are abbreviated in the figure, but all queries have fully grammatical clauses. We sample a subset of these 138k queries to seed WIKI-BALANCE.

heading: 3.1 WIKI-BALANCESynthetic
text: LLMs generate WIKI-BALANCESynthetic. For each of 3,996 randomly sampled debate questions (e.g., Should Karl Marx be considered a revolutionary thinker? ), we create 8 synthetic documents. First, we prompt a GPT 3.5 Turbo model with the debate question and: “For each side of the issue, write an article from the perspective of that side.” This induces a single polar axis to divide all subsequent documents. GPT 3.5 Turbo gives regular output: each article begins with a header describing the position; e.g., Perspective 1: Karl Marx as a Revolutionary Thinker; Perspective 2: Karl Marx as a biased ideologist). We use regular expressions to extract them, and continue to prompt the model 3 more
2The subjects of edit wars, NPOV disputes, edit restrictions, or otherwise frequent content revisions, reversions and rollbacks; https://en.wikipedia.org/wiki/ Wikipedia:List_of_controversial_issues
times: Given the question above, write an article from the perspective of the side below. This produces 32k polarized synthetic documents (see Figure 2). In this way, each document is marked as relevant to one out of 3,996 queries. Each query in the WIKI-BALANCESynthetic corpus corresponds to a balanced set of documents, with each half supporting a different perspective.

heading: 3.2 WIKI-BALANCENatural
text: This Natural corpus helps us (1) evaluate the Google Search engine, and (2) evaluate opensource IR systems in a real-world setting where documents are less polarized. With 452 randomlysampled queries from WIKI-BALANCESynthetic, we scrape publicly-available natural web documents from the top 10 results of Google Search in October 2023. To reduce noise, we keep only HTML documents’ body text.

heading: 3.3 WIKI-BALANCE Quality Estimation
text: Human Evaluation. To measure the quality of WIKI-BALANCESynthetic, we recruit domain experts3 from Upwork to blindly evaluate 10-20 random query-document pairs. First, evaluators consider the subjectiveness and topical relevance of the query. A subjective question does not have a single correct answer, and a relevant query relates to the topic in an interesting and well-specified way. Table 1 shows that queries are sufficiently subjective and highly relevant to their respective topics. Annotators also score our synthetic documents for faithfulness (Durmus et al., 2020), coherence (Dang, 2005), relevance, and fluency, showing that they are high in each of these respects.
Safety. Although WIKI-BALANCE is centered around controversial issues, we want to reduce risk by measuring and addressing any toxic, harmful, or otherwise unsafe content contained in the documents. Using the OpenAI Moderation API, we determine that no document contains hate, harassment, self-harm, or unwarranted sexual content with a model confidence score larger than 0.09.
3For Entertainment, History, Religion, and Sports, we recruit a Graduate Student with a B.S. in Journalism. For Politics, Sexuality, Law, and Media, we find a Public Policy Graduate Student with a B.A. in Political Science. For Psychiatry, Technology, and People we enlist a Nurse in Clinical Behavioral Health with a B.A. in Psychology. And for Science and Environment, we recruit a former CDC health communication specialist with a B.S. in Public Health and an M.S. in Health Education. For Languages and Philosophy, we hire a former writing expert at Grammarly with an M.F.A.
We manually verified that the most triggering documents were benign cases of almost journalistic reporting on their respective topics (e.g., a debate on the key ingredients of sexual gratification).

heading: 4 Indexical Bias Metrics

heading: 4.1 Prior Metrics
text: Indexical bias arises when documents of class A receive greater visibility in search results than do documents of class B. Due to primacy effects (Ho and Imai, 2008), the document’s rank index can serve as a proxy for visibility, as higher-ranked documents will be more visible (Joachims et al., 2007; Pan et al., 2007; Baeza-Yates, 2018), and thus more frequently clicked (Insights, 2013). Discounted Cumulative Gain (DCG) assumes a document’s visibility, or the attention it receives, is inversely proportional to the logarithm of its index. Although log-based decay may not exactly reflect user attention (Ghosh et al., 2021; Sapiezynski et al., 2019), this has become the standard in IR. DCG is thus defined according to Equation 1.
DCG(r, u) = |r|∑
i=1
u(i, r) log2 i (1)
where u(i, r) is the utility of ranking r up to and including document i. Researchers can normalize the Discounted Cumulative Gain by setting
nDCG(r, u) = DCG(r, u)−minr′{DCG(r′, u)}
maxr′{DCG(r′, u)} −minr′{DCG(r′, u)} (2)
That is, we set the minimum to zero and divide by the metric’s highest attainable value for the given number of items |r| and metric parameters. This means all measures will reach their best, most fair value at 0, and their worst value at 1.
Following the group fairness literature (Pedreschi et al., 2009; Pedreshi et al., 2008), we can define u(i, r) as statistical parity in the visibility of a protected group (Pitoura et al., 2022). In the standard formulation of Yang and Stoyanovich (2017), they set rND = nDCG(r, uND) with
uND(i, r) = Pg@i− Pg@ |r| (3)
which is the difference between the proportion of protected group g members in the top i against the proportion of protected group g members in the population (i.e., the full ranking). The corresponding rND metric is convex and continuous, but not differentiable at zero. To further smooth this metric, Yang and Stoyanovich (2017) also consider the KL divergence between protected group membership in the top i vs. the full ranking, giving rKL = nDCG(r, uKL) with
uKL(i, r) = −Pg@i log ( Pg@ |r| Pg@i )
− ( 1− Pg@i log ( 1− Pg@ |r| 1− Pg@i )) (4)
To apply fairness metrics rND and rKL to the most general case of indexical bias, we can abstract group membership g to indicate whether a document is polarized in a particular direction. We can safely ignore any prior metric for which this generalization would not apply, such as the
keyword-bias metric of Rekabsaz et al. (2021) and the Normalized Discounted Ratio of Yang and Stoyanovich (2017), which assumes g is a minority group (Pg@|r| < 0.5). We will also focus entirely on ranking bias metrics, and ignore Kulshrestha et al. (2019) and others who measure the bias of individual documents.

heading: 4.2 The DUO Bias Metric
text: Clearly, rND and rKL can be used only in cases where document polarization labels are known, such as when: (1) we have manually annotated the corpus according to the target axis, or (2) when we generate a controllable synthetic corpus like WIKI-BALANCESynthetic. Option (1) is not scalable, especially with thousands of distinct axes of controversy in §3. Option (2) does not apply to evaluations in real-world settings. This motivates us to build an unsupervised metric for indexical bias that can operate automatically, even in real-world settings, using scalable knowledge from LLMs.
Our proposed bias metric is the “Discounted Uniformity of Opinions” (DUO). This unsupervised metric critically depends on our synthetic data to determine the axis of polarization for each query.4 Given a topic t represented by a query qt, we pull from WIKI-BALANCESynthetic a set of |r| synthetic documents that argue each opposing perspective for qt following §3.1. We use a transformer-based document encoder to embed each document into a dense vector, and by running PCA, we project document embeddings into scalar polarization scores pj ∈ R, with an average score of p̄ = 1|r| ∑|r| i=1 pj .
4It is important to note that even for our DUO computations on the Natural corpus, we rely on WIKI-BALANCESynthetic here to determine the axis of bias.
This allows us to define
uV (i, r) = 1
i
i∑
j=1
(pj − p̄)2 (5)
so that the utility of a ranked subset is the variance of its polarization scores. Now this utility is the complement of a bias metric: more variance in the polarization scores indicates a better balance of ideas and less indexical bias. Thus for convenience in this paper, we will refer to our normalized DUO(r) metric by the following equation:
DUO(r) = 1− nDCG(r, uV ) (6)

heading: 5 Validating DUO
text: Experimental evidence demonstrates the validity of the DUO metric. Our first experiment shows that the unsupervised polarization score that grounds DUO can accurately partition a corpus of polarized documents into their respective viewpoints (§5.1). Our second experiment shows that DUO can help predict the Search Engine Manipulation Effect in a real behavioral manipulation (§5.2). Together, these results demonstrate both the fundamental and psychological validity of DUO.

heading: 5.1 Validation with Synthetic Data
text: Since DUO depends on reliable polarization scores to compute viewpoint variance, the first validation step is to evaluate the accuracy of these scores. Accurate polarization scores should partition a set of related documents into contrasting subsets for each viewpoint, where documents with positive scores endorse Perspective 1, and documents with negative scores endorse Perspective 2 for a given query
(or vice versa). With viewpoint labels from §3.1 as ground truth, we compute the maximum accuracy for each query, and average over all queries in WIKI-BALANCESynthetic.
Accuracy depends on our document embedding model, so we evaluate all 124 of the models associated with Sentence BERT (Reimers and Gurevych, 2019), a transformer-based document similarity metric widely used in the IR community. We also evaluate the recent voyage-02 embeddings from Voyage AI.5 Table 2 provides the worst accuracy (73%), best accuracy (95%), and the median (87%) and mean (86%) accuracy across all 124 evaluations. These validate our approach since, in the best case, with every third query there is only one incorrect document label among the 8 documents retrieved. For more detailed results, see Tables 6 and 7 in Appendix A.3.

heading: 5.2 Validation with A Behavioral Study
text: Experimental Design. A behavioral study can demonstrate the psychological validity of the DUO metric as predictive of the Search Engine Manipulation Effect (SEME). For each query q ∈ WIKI-BALANCE, a randomly sampled participant will have some opinion oprior ∈ Z. SEME predicts that, after this participant considers a biased list of search results relevant to the given query, their final opinion oposterior ∈ Z will be shifted with a magnitude proportional to the magnitude of the indexical bias. Our null hypothesis is that DUO is unrelated to the effect, so the coefficient β2 is zero in the following regression:
oposterior = β0 + β1oprior + β2µ + DUO + ϵ
To validate DUO, we would reject the null hypothesis. Here, µ+DUO is a signed copy of DUO (see Appendix A.2 for a derivation), which by default measures only the magnitude of the bias and not its direction. We need a sign to indicate the direction
5https://www.voyageai.com/

heading: SEARCH
text: of the bias because we expect that shift will move towards the favored perspective.
Experiment. American adult participants are recruited from Prolific. Each participant interacts with an interface like that shown in Figure 3. At the top is a query and a given perspective, like anarchism is a viable alternative to the coercive power of the state. The participant then provides their initial opinion on a 7-point Likert scale, oprior ∈ {−3,−2,−1, 0, 1, 2, 3}, where −3 indicates strong disagreement with the listed perspective, and 3 indicates strong agreement. After entering this prior, the participant uses our manipulated search engine, which retrieves a list of up to 10 search results. Participants are randomly assigned to the experimental manipulation: either results are ordered (1) with the maximum bias or (2) minimum bias, according to the DUO metric. We log any article links the participant clicks, assuming participants are motivated to read these results, since it can assist them in the penultimate task question: summarizing their opinion and quoting evidence. The participant concludes the task after providing oposterior on a similar Likert scale.
cle link ( Behavior=Clicked ), we observe significant (p < 0.05) positive β2 coefficients, leading us to conclude that DUO helps predict the SEME in cases of article click-through, and thus validating our method.
Results. We run our experiment for N = 200 evaluations on each of our two corpora, as shown in Table 3. By logging click behavior, we discover that participants are not as often motivated to read search results in the Synthetic corpus as they are in the Natural corpus. With Synthetic, 10% of users clicked at least one article, while in the Natural, half of users clicked at least one article. Differential click-through behavior effects our findings. When we limit our regression to only those trials where a user clicked at least one link (Behavior: Clicked), we can reject the null hypothesis with statistical significance. So only in cases of article click-through, DUO significantly helps predict the Search Engine Manipulation Effect with an R2 effect size greater than 0.48 in both the Natural and Combined corpora, p < 0.05. These findings are robust, as they replicate with the best and average embedding models (see Table 9 in Appendix A.6). From this, we conclude that DUO is psychologically valid as it helps predict SEME.

heading: 6 IR Bias Audits
text: With DUO (§5) and evaluation corpora (§3), we can audit the indexical bias of leading IR models. The labeled synthetic corpus shows us how DUO compares with prior bias metrics. However, synthetic results may not reflect end-model behavior on real document distributions. For more realistic results, we evaluate on the Natural corpus, where SEME behavioral experiments clearly validate DUO as predictive. The results in Tables 4 and 5 use both standard relevance metrics and the bias metrics from §4, and are averaged over 3 random seeds. Along with concurrent work (Zhao et al., 2024), PAIR is among the first cross-domain audits of indexical biases in open-source IR systems.

heading: 6.1 Models
text: Following BEIR (Thakur et al., 2021), we use BM25 as a strong lexical baseline, and evaluate six additional open-source neural systems, as well as one industrial system. For sparse models, we evaluate SPARTA (Zhao et al., 2021) and SPLADE (Formal et al., 2021). Next, we consider three dense models, ANCE (Xiong et al., 2020), SBERT (Reimers and Gurevych, 2019), and Use-QA (Yang et al., 2021). Our late-interaction model is ColBERT (Khattab and Zaharia, 2020). Finally, we evaluate Google’s search engine. Our document embedding model is the best-performing sentence-t5-xl.

heading: 6.2 Aggregate Bias Results
text: Here, the most relevant models are not always the least biased. Although SPLADE produces the most relevant results, it also introduces the most indexical bias in the natural evaluation setting (DUO= 0.62). On the other hand, Use-QA is the least relevant model, yet it produces the least biased rankings in both the natural and synthetic evaluation (DUO= 0.64, 0.57).
Most importantly, bias results validate our DUO metric, as DUO highly correlates with the supervised metrics rND and rKL, with Spearman correlations of 0.80 and 0.83 respectively (p < 0.05). Thus DUO gives us similar conclusions about model bias without the need for human annotation. On the synthetic data, Use-QA and SPARTA are the least biased (DUO≤ 0.58; rKL = 0.60), followed by ColBERT, SPLADE, and BM-25 (DUO= 0.60). The most biased models are ANCE and SBERT (DUO≥ 0.61; rKL≥ 0.62). These results are stable; even if we compute DUO using a less accurate embedding model, the relative model order is roughly preserved (ρ = 0.72; see Appendix A.6).
Model bias on the synthetic corpus also weakly predicts its bias on the natural corpus, with a strong Spearman correlation of 0.64 between synthetic and natural DUO scores. ANCE remains the most biased model, while Use-QA remains the least biased open-source model. Google search is the least biased overall (DUO = 0.63). We conclude that synthetic evaluation may be used as a surrogate in this way to quickly evaluate IR systems across a wide range of domains. However, natural data remains the gold standard and should not be replaced by synthetic evaluations alone. Discrepancies between the synthetic and natural results may reflect differences between the distributions
indexical bias in the Natural setting. Best results are green , and worst results are red . †N/A indicates that the metric is not applicable because it requires external human labels.
of their document polarizations. Synthetic data follows a highly-polarized bimodal distribution, while natural bias scores are both more neutral, and also normally distributed (see Appendix A.4). The respective evaluations are mutually complementary.
Overall, the orderings between models above are relatively stable, even when we consider alternative embedding models, or when we apply debiasing methods to the embedding process to remove possible spurious correlations that arise from WIKI-BALANCESynthetic (see Appendix A.6 for more details on Experimental Replications, and
Appendix A.5 for methods to remove spurious artifacts).

heading: 6.3 Aggregate Relevance Results
text: Our relevance results confirm prior work (Thakur et al., 2021). Table 4 shows BM-25 is the strongest baseline for relevant retrieval on both WIKI-BALANCE corpora, and that BM-25 beats out models of greater complexity like ANCE and SPARTA. ColBERT also achieves the top relevance scores on the WIKI-BALANCESynthetic corpus, which also aligns with Thakur et al. (2021) and sanity-checks our results. The nDCG@10 rel-
evance scores are all higher in the left side of the table, showing unsurprisingly that our synthetic corpus is an easier task than natural web retrieval.

heading: 6.4 Domain-Level Results
text: One of PAIR’s benefits is the ability to evaluate models domain-specific biases. Small aggregate differences in bias performance may not sway industry leaders and practitioners to adopt an entirely new IR system, but if an operational system demonstrates weakness in a particular domain, that can become a focal point for bias mitigation, like debiasing embeddings. Here, PAIR can serve as a precise instrument for diagnosing and addressing localized indexical biases, as in this section.
Table 5 decomposes aggregate bias results into focal domains, revealing weaknesses in even the best models. The best open model, Use-QA, still falls short of Google Search in three key domains: Psychiatry (+0.12 more bias than Google), Politics (+0.06 DUO), and Law (+0.05 DUO). Since indexical bias in political search results can affect voting behavior (Epstein and Robertson, 2015), practitioners may have strong incentives to mitigate such biases in open source systems.
In politics as in the aggregate, Google Search performs with the least bias on natural web data. However, given the power of any prominent search engine to influence countless users, it could be strategic for such search companies to invest greater attention towards bias mitigation at the weakest points. For Google, a weakness is the Environment (+0.04 more biased than the best Use-QA model).

heading: 7 Applications and Extensions
text: The PAIR framework is general, and future work can explore its extensions outside of search in other domains where information follows a rank order, such as a chatbot’s probability-ranked utterances, a politician’s most common phrases, or even the ordered paragraphs of a news article (i.e., framing; Ziems and Yang, 2021). Indexical bias evaluation will become increasingly relevant to mitigate harms in more recent generative methods which rely on Retrieval Augmented Generation for grounding knowledge (Lewis et al., 2020; Shuster et al., 2022; Jiang et al., 2023; Khattab et al., 2022).
A natural extension of the DUO could also handle issues with more than two sides, simultaneously incorporating multiple axes of semantic variation. Our formalization is readily prepared for
such an extension. If we generate up to m different viewpoint axes for each issue in WIKIBALANCESynthetic, we could simply increase the dimensionality of the PCA projection in §4 such that polarization scores become polarization vectors Pj ∈ Rm. This would allow a separate variance utility computation uxv(i, r) = 1 i ∑i j=1(P x j −P̄ x)2 and thus a separate score DUOx(r) for each viewpoint axis x. Depending on the application, one could aggregate across axes x, for example by taking the maximum bias score as in the multi-group Attention Bias Ratio of Ghosh et al. (2021). For more discussion of multi-group extensions of indexical bias metrics, see the Group Relevance Framework of Sakai et al. (2023).
Additionally, there is still much to be learned about the formal and mathematical properties of the DUO metric, especially in the context of optimization and reranking. Prior works suggest that fluctuations in fairness metrics like DUO can lead to unstable training (Rekabsaz et al., 2021). This may pose a challenge to the integration of DUO with current systems. One other bottleneck that may prevent the widespread adoption of DUO in reranking is the expensive computation for normalization. Our code includes a stochastic approximation, but the code could be further optimized.

heading: 8 Conclusion
text: From web search to personal assistants, IR systems have the potential to skew users’ opinions on a wide range of topics, from media and entertainment to political issues and scientific insights. Before one can address the problematic outcomes of such manipulation, one should first expect to measure indexical bias, the root of this psychological effect. PAIR is the first completely automatic method for evaluating indexical bias in IR systems without the need for manual human annotations. PAIR is built on two bias evaluation corpora, which support our DUO metric. Since DUO requires no human supervision, it can serve as a scalable evaluation metric and, in future work, as an automatic reranking criterion. We demonstrate the psychological validity of DUO using a controlled experiment. After proving its validity, we use DUO to run an extensive audit over the biases in current IR technologies, both open and closed-source. Together, these contributions provide a basis for future efforts to measure and address indexical bias in IR.

heading: Limitations
text: Complementary Notions of Fairness. All of the methodology we introduced in this work was focused on fairness of exposure (Diaz et al., 2020), balancing rankings to ensure equal visibility (Pessach and Shmueli, 2022) between groups or ideas. However, the appropriateness of fair exposure depends on the context (Singh and Joachims, 2018). It may not always be socially desirable to balance certain viewpoints, as this may elevate or amplify hate, espouse misinformation, or jeopardize personal or collective well-being. For any practitioners interested in applying PAIR or using the WIKIBALANCE corpus, we strongly encourage a more careful selection of the topics and domains over which DUO balance is optimized.
Other related works have measured the complementary objective of fairness through neutrality (Zerveas et al., 2022), where systems are encouraged to preferentially retrieve more factual, neutral, and unbiased documents and to omit the most polarized documents entirely. Fairness through neutrality is critical and should not be ignored, especially for highly sensitive or ideological domains and settings where users may be most susceptible to confirmation bias (Del Vicario et al., 2017) and its negative outcomes.
Still, we note that fairness through neutrality is not always attainable with a polarized corpus, nor is the notion of neutrality applicable to all queries (Krieg et al., 2023; Zerveas et al., 2022). For example, undecided voters may query a factual, encyclopediac corpus of Wikipedia articles on the biographies of candidates in an upcoming election. Voters opinions can shift by mere exposure to preferentially ranked biographies (Epstein and Robertson, 2015). Here there is no truly neutral document, despite the academic, factual tone they carry.
On the other hand, DUO applies both to highly polarized and more neutral corpora. This is because DUO measures indexical bias as a relative quantity—how biased the ranked results are relative to the most imbalanced possible ranking of those same documents. In doing so, we disentangle system bias from any document biases in the corpus itself (Kulshrestha et al., 2017).
Methodological Bias. This work seeks to measure bias in IR systems. However, it is important to acknowledge potential biases in the PAIR evaluation process itself. It is a non-trivial task to create an unbiased IR evaluation corpus. Just as tradi-
tional crowd-annotated datasets are prone to subjectivity biases in the document selection, relevance scoring, and other steps in the annotation pipeline, so also are synthetic methods vulnerable to such viewpoint biases, which may derive from the distribution of the LLM pretraining corpus, the prompt design, the seed topics used for prompting, or other related variables. The topics represented in WIKIBALANCE were drawn automatically from English Wikipedia and extrapolated with language models. The authors of this paper did not hand-select any topics or documents, nor do we endorse any particular viewpoints in these resources. In the Appendix A.1, we more thoroughly discuss biases in the seed topics, and we encourage future work to carefully consider and expand on this discussion.

heading: Ethics
text: This study has been approved by the Institutional Review Board (IRB) at the researchers’ institution, and participant consent was obtained using the standard institutional consent form. For the annotation process, we included a warning in the instructions that the content might be offensive or upsetting. Annotators were also encouraged to stop the labeling process if they were overwhelmed, and regardless of how may tasks they completed, participants were paid a fair stipend of $20 per hour for their time.

heading: Acknowledgements
text: We are thankful to the members of SALT Lab and the Stanford NLP Group for their helpful feedback on the draft. We especially appreciated suggestions from Tiziano Piccardi, Jing Huang, Faye Holt, Dora Zhao, Julia Kruk, and Michael Ryan. The work has been supported by a grant from Meta. Caleb Ziems is supported by the NSF Graduate Research Fellowship under Grant No. DGE-2039655.

heading: A.1 Potential Biases in WIKI-BALANCE
text: Controversial Wikipedia article titles are topically diverse, covering 15 broad domains, including politics, history, religion, science. Given Wikipedia’s global scope, we unsurprisingly find that each domain has wide coverage. Still, biases in the editor pool mean the distribution of topics in each domain is skewed. For example, in politics, local issues typically concern Anglophone countries (71.69% of political topics are about countries in which English is either a official, majority, or secondary language). In Western politics, the United States is over-represented, appearing in 62% of Western issues (see Figure 4). Neither fact is surprising, since topics come from English Wikipedia, and a plurality English-language Wikipedians edit from the United States.6 See §8 for a more in-depth discussion on the impacts of skewed data.
6 https://en.wikipedia.org/wiki/Wikipedia:Who_writes_
Wikipedia

heading: A.2 Measuring the Directionality of DUO Bias
text: By default, DUO metric measures only the magnitude of the bias and not its direction, as is the case with previous metrics (Gezici et al., 2021). Unlike prior metrics, our methodology allows for an unsupervised computation of a sign to indicate the direction of the bias. If we consider the signum function of a real number
sgn(x) =    −1 x < 0 0 x = 0
1 x > 0
we can extend this definition to a set A ⊂ R by
sgn(A) = 2× 1 [( |{ai ∈ A : ai > 0}|
|A|
) > 0.5 ] + 1
this gives us sgn(A) = −1 when A contains more negative values than non-negative values, and sgn(A) = 1 otherwise. If we modify the utility to encode the sign on the set of polarization scores {pj} as follows
u+V (i, r) = sgn({pj}ij=1)uV (i, r)
Then we can set
µ+DUO(r) = sgn ( DUO(r, u+V ) ) (DUO(r, uV ))
which assigns the existing DUO magnitude an appropriate polarity. Our signed µ+DUO(r) value will be useful for understanding how biased rankings can shift a reader’s opinion towards the perspective favored by the ranking (see §5.2).

heading: A.3 Expanded Accuracy of Polarization Embeddings
text: Here in Tables 6 and 7, we enumerate the unsupervised polarization label accuracy for each Transformer-based embedding model in the Sentence BERT library. Table 6 gives the 13 models with greater than 90% accuracy, while Table 7 enumerates the remaining models in order of their accuracies. With a worst accuracy of 73%, a best accuracy of 95%, and a median accuracy of 87%, these results strongly validate our approach, and show its robustness across document embedding implementation.

heading: A.4 Differing Distributions: Natural And Synthetic
text: Unsurprisingly, Figure 5 shows how natural Google search web articles are distributed differently than
our synthetic corpus. Whereas natural bias scores (bottom) are normally distributed around a neutral mean of zero, synthetic data (top) follows a highlypolarized bimodal distribution, and includes some extreme outliers. We can conclude that any discrepancies between the synthetic and natural results in §6 are largely due to these differences. The respective evaluations are mutually complementary.

heading: A.5 Removing Spurious Correlations from Polarization Embeddings
text: DUO depends on WIKI-BALANCESynthetic for the |r| synthetic documents on which polarization scores pj are computed via PCA. The synthetic data can introduce spurious correlations into the DUO metric. For example, with the following query
How can the legal system effectively enforce copyright laws in BitTorrentenabled piracy?
we have two perspectives: (1) from the perspective of the Entertainment Industry, and (2) from the perspective of BitTorrent users. Naturally, any synthetic documents generated for Perspective 1 will contain more legal language. This means that even neutral documents could be marked by our embedding method as supporting Perspective 1 if they contain legal language (e.g., a purportedly neutral Wikipedia article). Here, we will describe how we worked to understand and remove such spurious correlations in DUO.
Following prior works on the removal of bias in word embeddings (Bolukbasi et al., 2016; Lauscher et al., 2020), we opt to identify in the document embedding space some subspace in which the spurious correlation lies; then we can effectively project
away this subspace. Let b be the principle bias axis in our embedding space, and xj be the raw document embedding for document j. The debiased document embedding will then be
x̃j = xj − ⟨xj , b⟩ b
As before, we can fit PCA on X̃ = [x̃1; ...; x̃8] to compute de-biased polarization scores p̃j .
To identify b, we used the following process, extracting the axis of spurious correlation automatically from a set of “distractor documents.” First we generate these distractor documents to represent spurious correlations in the data. These distractors are the outputs of a pipeline similar to that of §3.1 for creating WIKI-BALANCESynthetic. For a given document dit, which answers some query qt (e.g., “Did Edison steal patents from Tesla?”) about a high-level topic t (e.g., Nikola Tesla) by endorsing one particular perspective P it (e.g., “Edison stole patents from Tesla.”), we will generate a distractor document d̃it that use similar style and vocabulary as dit without ever answering qt. The document d̃it will neither endorse nor deny P i t . To this end, we retrieve a distractor question q̃it, which is a query semantically distinct from qit, but it concerns the same high-level topic (e.g., the query, “How did Tesla’s personal eccentricities influence his
work?”). With the distractor question q̃it, we generate d̃it by prompting gpt-3.5-turbo, “<dit> Using as many words and phrases from the paragraph above as possible, try to answer: <q̃it>.” To ensure that d̃it is neutral with respect to qt, we follow up with, “Rewrite the above paragraph but remove any sentences that have to do with the idea: <P it >.”
Now for each original document dit we have a distractor d̃it. Now we want to identify a spurious decision boundary between each perspective, so we partition the distractors according to the perspective of the document they stylistically emulate:
D1 = {d̃it : P it = 1} D2 = {d̃it : P it = 2} If each set of documents has its own average document embedding µD1 , µD2 , we can set the principle axis of spurious correlation to be
b = µD1 − µD2 This difference of means is a proven reliable method for identifying the semantic direction between binary concepts (Marks and Tegmark, 2023).

heading: A.6 Experimental Replications
text: In this section, we replicate our studies with different parameters to demonstrate the robustness of our experimental results in both the SEME Behavioral Study in §5.2 and the Bias Audit in §6.
Bias Audit. Table 8 gives the experimental replications for the Bias Audit, using different embedding methods. The MiniLM columns indicate the use of all-MiniLM-L6-v2, a weaker embedding model, as its accuracy in Table 7 was only 83%, compared with the stronger 95% performance of sentence-t5-xl. The MiniLM synthetic results have a correlation of ρ = 0.72 with the T5-XL results. Separately, in the T5-XL-Debiased column, we replicate our findings using the debiasing methods from §A.5 and find a strong correlation of ρ = 0.89 with the synthetic T5-XL results, and ρ = 0.94 with the natural T5-XL results. In each case, debiasing preserves the relative model ordering at the bottom of Table 8. For example, on WIKI-BALANCESynthetic, we have Use-QA ≻ SPARTA ⪰ BM-25 ≻ ColBERT ⪰ SPLADE ≻ SBERT ≻ ANCE. We can conclude that our PAIR methodology largely induces a stable relative model ordering from the DUO metric, even when we consider alternative embedding models or apply debiasing methods to the embedding process to remove any potential spurious correlations.
SEME. Finally, we run experimental replications for the SEME in which we try both different embedding systems and different DUO step sizes. In Table 7, we determined that clip-ViT-B-32-multilingual-v1 gives the lowest polarization score accuracy of 73%. Now in Table 9, we find that this polarization accuracy is not high enough to produce a significant effect in the SEME experiment. However, we can replicate our significant findings for a mid-performance model, all-MiniLM-L6-v2, which has 84% polarization accuracy, and the topperformance model, sentence-t5-xl which has 95% polarization accuracy. We can also replicate these experiments when we increase the DUO step size from 1 to 2 as in prior work (Yang and Stoyanovich, 2017). To increase the step size, one can effectively substitute the DCG in Equation 1 with
DCG(r, u) = |r|∑
i=2,4,...
u(i, r)
log2 i
where i increments in steps of 2. In all such replications, we observe significant (p < 0.05) positive β2 coefficients, leading us to conclude that DUO helps predict the SEME in cases of article click-through, and thus validating our method.

heading: A.7 Parameters and Computing Budget
text: All experiments were performed on a Ubuntu Linux machine with 6 Nvidia GeForce RTX 2080 Ti GPUs. Each model evaluation took around 4 hours. All embedding models were run with the Sentence
Corpus Behavior Step DUO Emb Model N β2 P (β2 = 0) R2
BERT (Reimers and Gurevych, 2019) package on default parameters. All IR models were run with the BEIR (Thakur et al., 2021) package on default parameters.

heading: Future Work