Metrics for What, Metrics for Whom: Assessing Actionability of Bias Evaluation Metrics in NLP (2024)

Abstract: This paper introduces the concept of actionability in the context of bias measures in natural language processing (NLP). We define actionability as the degree to which a measure’s results enable informed action and propose a set of desiderata for assessing it. Building on existing frameworks such as measurement modeling, we argue that actionability is a crucial aspect of bias measures that has been largely overlooked in the literature.We conduct a comprehensive review of 146 papers proposing bias measures in NLP, examining whether and how they provide the information required for actionable results. Our findings reveal that many key elements of actionability, including a measure’s intended use and reliability assessment, are often unclear or entirely absent.This study highlights a significant gap in the current approach to developing and reporting bias measures in NLP. We argue that this lack of clarity may impede the effective implementation and utilization of these measures. To address this issue, we offer recommendations for more comprehensive and actionable metric development and reporting practices in NLP bias research.

## introduction as the landscape of bias measures in natural language processing (nlp) has expanded, so too has the literature examining and interrogating these measures (e.g., @xcite @xcite @xcite @xcite . in particular, increasingly rich reflections within and beyond nlp have offered vocabularies and frameworks for navigating this landscape; for example, the framework of measurement modeling from the quantitative social sciences disentangles what is measured (a theoretical construct) from how it is measured (its operationalization), and offers the vocabulary * joint first authors. across the literature proposing and examining bias measures, talk about measures is often informally tied to talk about what can be done with results produced by measures-i.e., measures' results are often used in decision-making, and good measures should not only exhibit characteristics such as validity and reliability, but should also facilitate decision-making or intervention. for example, natural language generation practitioners use the results of automated metrics to select which models should undergo human evaluation @xcite , while other measures' results might guide policies for model release and deployment @xcite . together, this suggests another piece of vocabulary with which we might assess bias measures. in this paper, we seek to formalize this intuition by introducing actionability-the degree to which a measure's results enable informed action-and outlining a set of desiderata for actionability-what information is required of a bias measure in order to act based on its results. at the same time, while the measurement modeling literature has shown the importance of clearly conceptualizing bias and establishing bias measures' validity and reliability, it has also shown that the nlp literature routinely fails to do so. for example, bias in the nlp literature is often underspecified @xcite , and measures are often poorly matched to the constructs they are intended to measure @xcite or lack sufficient description to establish a match altogether @xcite . hypothesizing that the literature may similarly seldom assess what bias measures can be used for, and whether enough information is provided to facilitate that use, we conduct a review of 146 papers proposing bias measures, examining whether and how papers provide the information required to act based on the proposed measures' results. we find that many desiderata for actionability, such as a bias measure's intended use or an assessment of its reliability, are often not clearly provided or go unstated altogether. we argue that this lack of clear information may hinder bias measures' effective implementation and use, and offer suggestions for improving the development and dissemination of bias measures in nlp research. ## actionability in this section, we introduce and formalize actionability, draw connections between actionability and other concepts related to the trustworthy nlp literature, and provide an example of a bias measure and the actions it facilitated. we introduce actionability in order to answer the question: what is required of a bias measure in order to take informed actions based on its results? following @xcite , we define a bias measure as an "evaluation standard that includes a metric(s) applied to a dataset" which is applied to measure "bias," itself a contested and often underspecified construct @xcite . throughout the paper we use "bias" to refer expansively to the wide range of concerns, impacts, and harms that work in the nlp literature has sought to measure under the term "bias." actionability refers to the degree to which a measure's results enable decision-making or intervention; that is, results from actionable bias measures should facilitate informed actions with respect to the bias under measurement. such results might communicate aspects of the measured bias such as who is impacted or harmed by a system, the degree and scale of impact or harm, or potential sources of the issue. in turn, the decisions or interventions that these results enable might include targeted improvements to training or fine-tuning processes (e.g., @xcite @xcite @xcite , deployment of appropriate safeguards (e.g., @xcite @xcite , decisions to re-design or not to deploy @xcite , or changes in regulation or policy @xcite . the ability to act on bias measure results may not be equally distributed among stakeholders, as power or organizational dynamics can shape their ability to intervene. for example, while some results may suggest that retraining a model or delaying a system's deployment would be effective interventions, stakeholders might not be equally empowered to take such actions. stakeholders such as consumers may only be in a position to opt out of using or providing data for a system @xcite , while regulators may choose to sanction-e.g., by issuing fines or outright banning uses that are not compliant with regulation-or allow particular applications. to better situate actionability, we consider it against other concepts the responsible nlp literature-specifically, accountability, transparency, interpretability, and validity-beginning with accountability. evaluations or audits of ai systems are often conducted (implicitly or explicitly) with the goals of "establish[ing] informed and consequential judgments of... ai systems" @xcite )-e.g., whether a system's behavior is legally compliant-and holding ai providers accountable-i.e., "responsible or answerable for a system, its behavior and its potential impacts" @xcite . however, as birhane et al. write, in practice "ai audit studies do not consistently translate into more concrete objectives to regulate system outcomes." thus, we see the actionability of a bias measure as a component for ensuring that results from bias measures can translate into action that shapes system outcomes and policy and holds providers responsible. research on the transparency of ai systems has argued for the importance of "develop[ing] more trustworthy ai" @xcite . using liao and wortman @xcite definition of (informational) transparency as "what information about a model [or system] should be disclosed to enable appropriate understanding," we see transparency as required for actionability. that is, it is impossible for stakeholders to act upon the results of a bias measure without crucial knowledge about a system's design and deployment. the interpretability of a model attends to whether we understand the process by which the model produces an output @xcite . analogously, the interpretability of a bias measure attends to whether we understand the process by which a bias measure arrives at a result. unlike actionability, interpretability does not attend to whether that result enables informed interventions-even if understanding why a measure produces a result can help facilitate interventions @xcite . the aspects of validity most closely related to actionability are the three addressing the use and utility of a measure's results: consequential validity, predictive validity, and hypothesis validity. 1 consequential validity involves "identifying and evaluating the consequences of using the measurements obtained from a measurement model" @xcite . for bias measures, for example, using a certain measure may make harm to some populations more visible than others, depending on which populations the measure was designed for, or else a measure's uptake-and the subsequent optimization of models towards it-may have unintended effects. thus, consequential validity is related to actionability, as one consequence of using a bias measure's results is precisely the decisions or interventions that might be made on the basis of those results; therefore, in developing actionable metrics, practitioners should consider the consequences of the decisions and interventions that those metrics facilitate. meanwhile, predictive validity captures "the extent to which measurements obtained from a measurement model are predictive of measurements of any relevant observable properties... thought to be related to the construct purported to be measured," while hypothesis validity captures "the extent to which the measurements obtained from a measurement model support substantively interesting hypotheses about the construct purported to be measured" @xcite . 2 we argue that for bias measures, actionability is very closely related to predictive and hypothesis validity, as bias measure results that enable decisions or interventions also implicitly or explicitly support a particular type of hypothesisi.e., a hypothesis that some decision(s) or intervention(s) can meaningfully address the bias under measurement. while actionability can thus be understood as a narrower form of hypothesis validity, we propose it as its own concept to draw attention to the specific types of hypotheses-i.e., about meaningful decisions or interventions-that we argue bias measures should support. while other types of validity-face, content, convergent, and divergent validity-appear less directly related to actionability conceptually, we see 1 validity has been conceptualized in several ways; we use the conceptualization from jacobs and wallach (2021). 2 we consider predictive and hypothesis validity together because, as jacobs and wallach (2021) point out, "the main distinction between predictive validity and hypothesis validity hinges on the definition of 'substantively interesting hypotheses,"' and that "distinction is not always clear cut." them as no less important; bias measures that do not capture all relevant aspects of the bias to be measured, or whose results are implausible or fail to correlate with other measures' results @xcite , are unlikely to enable informed action. similarly, measures that are not reliable are unlikely to be actionable, as their results may not provide a sufficient basis for making well-informed decisions. in this desideratum we include test-retest reliability-i.e., whether similar inputs yield similar results @xcite -as well as the reporting of a measure's margins of error; statistical tests used to assess results' significance @xcite ; and other analyses of possible sources of uncertainty of results @xcite , such as variation due to choices of seed words @xcite or templates @xcite . example. in 2014, amazon sought to develop an ai system for screening candidate resumés, which was ultimately discontinued in 2018 because it ranked female candidates lower than male candidates @xcite . while we do not know the exact details of the bias measure(s) amazon used to assess the system, the results did facilitate understanding of who might have been impactedpeople who had attended women's colleges used the word "women's" on their resumés, or did not use words "more commonly found on male engineers" resumés, such as 'executed"'-all disproportionately women and gender minorities @xcite . we also know that the results enabled at least three actions: first, amazon attempted to mitigate the issue, "edit[ing] the programs to make them neutral to [the terms mentioned]"; second, amazon discontinued the use of the system for ranking candidates and "disbanded the team [building the system]"; and finally, amazon moved towards using a "'much-watered down version"' for to help with 'rudimentary chores,' including culling duplicate candidate profiles" @xcite . this example illustrates how results from bias measures can facilitate various actions from various stakeholders, including mitigation attempts by system developers and decisions to discontinue or to use alternate versions for different purposes by (presumably) amazon leadership. it further illustrates the importance of transparency-specifically, the lack of external transparency with respect to results of amazon's bias measures, and to the active use of the system between 2015 and 2018. had the biases of the system been public knowledge, stakeholders outside the project team and amazon leadership would have been able take action-e.g., individuals would have been able to withdraw applications or choose not to apply, while regulators would have the ability to sanction the use of a system that was in breach of regulation around gender discrimination in hiring. insofar that the results of a bias measure of the system are not disclosed to the public and regulators, both are precluded from informed and meaningful action. ## desiderata for actionability what, concretely, makes a bias measure actionable? in this section, we outline desiderata for bias measures-i.e., information that a measure should provide and justify to enhance its actionability. we draw these desiderata from prior literature related to responsible nlp, including work on fairness in machine learning and nlp @xcite , measurement @xcite , and ai auditing and algorithmic accountability @xcite . we will also use these desiderata as the basis for our taxonomy and survey in the remainder of this paper. motivation. the motivation for a proposed bias measure specifies what need the measure is intended to address, e.g., measuring direct discrimination @xcite , adapting to new socio-cultural contexts @xcite , or extending to new languages @xcite . a clearly described motivation can increase a measure's actionability by helping people using the measure to assess whether the bias they seek to measure and the system and context of use for which they seek to measure bias are well-matched to the need the measure is intended to address. underlying bias construct. drawing on measurement modeling, we view bias as an unobservable theoretical construct operationalized via bias measures @xcite . under this view, a proposed bias measure is always accompanied, implicitly or explicitly, by an underlying theoretical understanding of what constitutes bias. however, these theoretical understandings are not always clearly specified or conceptualized; blodgett et al. (2020) illustrate that "bias" in the nlp literature is often underspecified, and @xcite argue that disagreements in the ai fairness literature often arise because authors rarely make explicit their theoretical understandings of fairness, which has many "context-dependent, and sometimes even conflicting" understandings. we argue that clarity in the conceptualization of a bias measure's underlying construct can increase the measure's actionability, as a bias construct articulates the measure's scope-e.g., what impacts or harms the measure is intended to capture, for which populations those impacts or harms are intended to be captured, or what constitutes impact or harm. if the bias construct is not clearly specified and conceptualized, it becomes unclear how the measure's results speak to any impacts or harms, and is therefore unlikely that those results can facilitate informed action. interval and ideal result. understanding, and therefore acting, on the results of a measure requires clearly articulated information about the values a result can take on; these values inform the statistical analyses that can be performed and the interpretations that can be made. minimally, actionability requires descriptions of: first, the numerical domain of the result (natural, real, or rational); foot_0 second, the interval a measure operates on-i.e., the values the result can take on-which may or may not be bounded (log-likelihood-based measures being an example of the latter @xcite ); and third, the scale of the intervalfor example, for measures on a logarithmic scale a result of 10 might be much worse than 3, but not that much better than 20. the numerical domain and the bounds and scale of the interval are necessary for interpreting the result, as they allow people using the measure to estimate how far the result is from the interval bounds and what it might mean relative to other possible results. proposed bias measures should also specify an ideal result. the choice of an ideal result is inherently normative, as it reflects a measure creator's perspective on what constitutes desired system behavior and how that is expressed in the measure's result. 4 specifying an ideal result can facilitate a measure's actionability by providing people using the measure with a clear goal or requirement, particularly if the choice is explicitly connected to the underlying bias construct and its wider socio-historical context-e.g., for hiring, an ideal result might be adherence to the four-fifths rule, a guideline for assessing what constitutes discrimination in employment in the u.s. @xcite . 5 intended use. proposed bias measures should specify under what circumstances or conditions the measure should be expected to produce meaningful results. this can include, for example, what types of models or additional data are required to be used in conjunction with the measure or which hyperparameters govern the behavior of the measure. broadly, intended use seeks to describe a wide variety of conditions that may be mechanistic-e.g., models, data, or hyper-parameters-or social, e.g., particular social settings in which the result of a measure is considered meaningful. for example, in some measures, the metric is closely tied to a specific dataset or dataset format-e.g., stereoset's @xcite stereotyping score aggregates a model's preferences for stereotypical versus anti-stereotypical completions and therefore requires a dataset containing such stereotype/anti-stereotype pairs. moreover, stereoset's particular construction-i.e., its use of log-likelihood and pseudo-perplexity to measure stereotyping-are designed for use with masked language models and auto-regressive language models respectively. by contrast,crows-pairs @xcite , a similar measure, only uses pseudo-perplexity and can therefore only be used with masked language models.thus, the construction of stereoset and crows-pairs limits their applicability to certain dataset and model characteristics, and they may therefore be poorly matched with other settings. providing descriptions of the mechanical conditions and socio-historical context and that render the result of a measure meaningful facilitates actionability by bounding a measure's application space, thereby giving potential users of a measure the information needed to assess whether the measure is appropriate for their use cases. in particular, when metrics and datasets are introduced together to propose a new measure, specifying the intended use can help to clarify how the dataset and metric together make the measure fit-for-purpose, as well as what other data the metric might potentially be appropriately applied to, and vice versa. reliability. as we discusss in section 2, we view the reliability of a bias measure as a prerequisite for actionability. thus, proposed bias measures should explain how their reliability was assessed. ## literature review and analysis to identify current trends and existing gaps in the field, we conduct a literature review, examining how papers proposing bias metrics engage with our desiderata for actionability. while previous studies (e.g., @xcite @xcite have explored how responsible nlp concerns (including bias) and measures of those concerns are described in the nlp literature, to the best of our knowledge, this is the first review specifically focused on the actionability of bias measures. ## threats to actionability measures' stated motivations rarely linked to their use. we read and categorized all free-form text passages describing motivations into a categorical schema using an inductive process. for 20% of papers, we were unable to identify any text passage with a clear motivation for introducing a new measure. in all other cases, we were able to identify clear motivations such as extending the measure to another language, setting, or modality. a subset of papers providing a motivation are motivated by improving existing measures, e.g., @xcite measure that "allow(s) for better identification of gender bias," or by addressing reliability or reproducibility concerns. although 80% of the papers provide a motivation, the degree to which that motivation is clear and specific varies, leaving a large subset of papers either vaguely gesturing towards a motivation. for example, @xcite motivate their work on measuring bias in llms due to the existence of "langchain," an underspecified "threat." ## discussion a considerable number of papers fail to provide crucial details about what motivates a bias measure and how it should be used. therefore, we offer several suggestions for the development and dissemination of new bias measures. indeed, any proposed measure is accompanied, implicitly or explicitly, with an intended use; most papers introduce measures in the context of their use for some model or system. what many papers leave unstated is to which other settings a measure may be applied-e.g., other models, domains, or (social) contexts, if any. therefore, practitioners are, more often than not, unable to assess a measure's suitability for their use cases. similarly, providing explicit reasoning about the the construct a measure is intended to capture can help prevent conflation between the conceptualization and the operationalization of a construct (e.g., @xcite . we argue that clearly articulating the underlying construct can additionally help in defining the measure's intended scope and use. this is particularly important given that recent work has shown that bias measures are often so closely tied to specific use cases that they cannot be reused across tasks, datasets, or languages @xcite . lack of clarity in motivation, intended use, or bias construct may lead practitioners to adopt measures that are poorly matched to their own use cases (e.g., without realizing that some of the design choices for the measure are tied to particular intended uses), or forego measures that would have been appropriate due to insufficient knowledge of their applicability. moreover, although stakeholders may be-in principle-equally able to take an action on the basis of some bias measures, which actions they are afforded may differ based on their ability to directly intervene in the system. for instance, individuals' actions may be limited to refusal @xcite and collective action as means for changing a system, while developers, companies, and regulators can engage in more direct processes. developers can address biases in models and systems; companies can allocate resources for addressing them, delay deployment, or retire models and systems entirely; and regulators can engage in regulatory processes to develop new regulation or apply existing regulation. when developing a bias measure, it may therefore also be appropriate to consider which stakeholder(s) the measure should enable to take action. ## conclusion we introduce actionability of bias measures, identify several desiderata for the actionability, and annotate 146 papers in the nlp literature for these desiderata, finding that much information required for actionability is under-specified or unstated. this finding suggests that current measures may not enable practitioners to meaningfully act on their results. we provide recommendations for future work that we hope can support the development of actionable bias measures, and believe that our desiderata can serve as a starting point for broader discussions on how we assess bias in models and systems, and more broadly help minimize the disparity between research artifacts and their practical uptake. moreover, although bias measures have been the focal point of our intervention, further work could explore how our framework might extend to other measurement instruments. such measurement instruments may facilitate different possible actions or interventions than bias measures, and actionability for those instruments may demand different desiderata.