Understanding and Meeting Practitioner Needs When Measuring Representational Harms Caused by LLM -Based Systems (2025)

Abstract: The NLP research community has made publicly available numerous instruments for measuring representational harms caused by large language model (LLM)-based systems. These instruments have taken the form of datasets, metrics, tools, and more. In this paper, we examine the extent to which such instruments meet the needs of practitioners tasked with evaluating LLM-based systems. Via semi-structured interviews with 12 such practitioners, we find that practitioners are often unable to use publicly available instruments for measuring representational harms. We identify two types of challenges. In some cases, instruments are not useful because they do not meaningfully measure what practitioners seek to measure or are otherwise misaligned with practitioner needs. In other cases, instruments-even useful instruments-are not used by practitioners due to practical and institutional barriers impeding their uptake. Drawing on measurement theory and pragmatic measurement, we provide recommendations for addressing these challenges to better meet practitioner needs.

## introduction representational harms @xcite occur when a system "represents some social groups in a less favorable light than others, demeans them, or fails to recognize their existence altogether" @xcite . numerous studies have documented representational harms caused by large language model (llm)-based systems (e.g., @xcite @xcite @xcite . it is important to measure and mitigate such harms-especially for systems that will be deployed in real-world contexts. however, this is known to be a challenging task. like many other concepts related to the capabilities, * work conducted during a microsoft research internship. to facilitate measuring representational harms, the nlp research community has produced numerous publicly available foot_0 measurement instruments, including datasets, metrics, tools, benchmarks, foot_1 and annotation instructions. in this paper, we investigate whether these instruments meet the needs of practitioners tasked with evaluating llm-based systems. as studies examining the uptake of other responsible ai artifacts have found, practitioner needs as assumed in the research literature are often different from those actually voiced by practitioners @xcite @xcite . this potential mismatch presents a critical opportunity for researchers and practitioners to engage with one another. if measurement instruments do not meet the needs of practitioners tasked with evaluating llm-based systems, those instruments will not be used by such practitioners-causing developers and deployers of llm-based systems either to not engage in measurement or to rely on bespoke instruments that might not be publicly understood. through a series of semi-structured interviews with 12 practitioners tasked with evaluating llmbased systems for representational harms, we find that practitioners are often unable to use publicly available measurement instruments, despite a desire to do so. we identify two types of challenges that lead to this. in some cases, instruments are not useful: they do not meaningfully measure what practitioners seek to measure or are otherwise developing measurement instruments that are simultaneously useful and used is not a new challenge. measurement theory from the social sciences has long been concerned with designing instruments that are useful, i.e., those that meaningfully measure what they purport to measure @xcite @xcite . pragmatic measurement builds on measurement theory to focus on designing measurement instruments that are both useful and used in practice, i.e., that are aligned with practitioner needs and designed to overcome barriers impeding their uptake @xcite . drawing on work from measurement theory and pragmatic measurement, we identify opportunities to improve measurement instruments and their uptake among practitioners. ## related work the nlp research community has made publicly available numerous instruments for measuring representational harms caused by llm-based systems. we provide examples of such instruments in assessments of existing measurement instruments have identified potential limitations to their usefulness. prior work has shown that the concepts such instruments seek to measure are often poorly motivated, unclear, or not meaningfully measured by those instruments @xcite @xcite @xcite @xcite . researchers have also found that instruments can be highly sensitive to implementation choices that should not affect measurement outcomes @xcite @xcite @xcite . furthermore, instruments often fail to produce measurements that enable informed actions @xcite , or that are useful for measuring representational harms in real-world deployment contexts @xcite @xcite . in this paper, we explore the extent to which these assessments of existing measurement instruments reflect the needs of practitioners, which are often both implicit and impacted by practical constraints @xcite . we build on prior work from hci showing that practitioners often struggle to use artifacts developed by researchers, in part because these artifacts are misaligned with practitioner needs @xcite @xcite @xcite . this work focused on the needs of practitioners who seek to measure allocative harms caused by predictive models. to our knowledge, we are the first to explore whether and to what extent publicly available measurement instruments meet the real-world needs of practitioners who seek to measure representational harms caused by llm-based systems. pid role employer p01 research engineer big tech company p02 applied scientist big tech company p03 research scientist big tech company p04 research engineer big tech company p05 consultant ai startup p06 scientist big tech company p07 research scientist ai startup p08 data engineer big non-tech company p09 nlp specialist big non-tech company p10 research scientist ai startup p11 researcher ai nonprofit p12 researcher big tech company ## methods to understand the extent to which publicly available measurement instruments meet the needs of practitioners, we conducted a series of semistructured interviews with 12 practitioners tasked with evaluating llm-based systems. participants, whom we refer to throughout by ids p1-p12, worked on llm-based systems (e.g., search engines, chatbots) and content moderation tools for such systems. see interviews. to scaffold the interviews, we identified a set of desiderata for measurement instruments: validity, reliability, specificity, extensibility, scalability, interpretability, and actionability (see ## practitioner challenges participants reported being aware of a range of publicly available measurement instruments, including tools designed to identify unsafe or toxic text (e.g., deberta @xcite and llama guard @xcite ), as well as datasets and benchmarks (e.g., stereoset @xcite and crows-pairs @xcite ). however, these measurement instruments were used by at most one or two participants-none were more widely used. more importantly, all participants discussed facing challenges that prevented them from using publicly available measurement instruments. specifically, participants experienced two types of challenges that left them unable to use publicly available measurement instruments, even when they had a desire to do so. first, instruments are not useful when they do not meaningfully measure what practitioners seek to measure or are otherwise misaligned with practitioner needs ( §4.1). second, instruments are not used when practitioners face barriers impeding their uptake-even if the instruments are useful ( §4.2). these challenges are context-dependent and determined by practitioner needs; the same instrument may be useful for one practitioner in one context but not for another. we found that the desiderata identified in ## addressing practitioner challenges in this section, we draw on measurement theory from the social sciences and pragmatic measurement to improve the usefulness and uptake of measurement instruments. as we mentioned in §1, measurement theory has long been concerned with designing instruments that are useful, while pragmatic measurement builds on measurement theory to focus on designing measurement instruments that are both useful and used in practice. designers of measurement instruments can draw on measurement theory to improve the validity and reliability of their instruments, and can draw on pragmatic measurement to improve other aspects of the usefulness and uptake of their instruments. this is not to suggest that designers of measurement instruments are solely responsible for meeting practitioner needs. rather, practitioners who seek to use measurement instruments are responsible for adapting those instruments to meet their specific needs. however, designers can facilitate this by designing extensible measurement instruments, which can help address usefulness and uptake challenges, as we explain below. regulators and organizations that develop and deploy llm-based systems can also play a role in removing both practical and institutional barriers. finally, we briefly discuss trade-offs inherent to attempting to meet multiple desiderata simultaneously. measurement theory provides a framework with which to improve the usefulness of measurement instruments. all participants reported concerns about whether measurement instruments meaningfully measure the concepts that they are intended to measure-often because those concepts were not clearly defined. these concerns were exacerbated by the fact that representational harms are abstract and contested, meaning that different measurement instruments may operationalize different definitions of representational harms. measurement theory offers a framework with which these concerns can be better understood and addressed. measurement theory provides a framework for obtaining measurements (e.g., scores calculated using a benchmark) of abstract concepts (e.g., "stereotypes") through the processes of systematization, operationalization, application, and interrogation @xcite . systematization is the process of formulating a specific, often theoretically grounded, definition of the concept of interest. this definition-the systematized concept-is, as noted by multiple participants and in prior work, often absent from instruments for measuring representational harms. instead, designers of measurement instruments often jump straight to operationalization, which is the process of developing one or more instruments for measuring the concept of interest. 7 designers of measurement instruments should draw on measurement theory to ensure that their instruments are valid and reliable. jumping straight to operationalization not only creates uncertainty concerning what precisely measurement instruments are intended to measure, but also renders it impossible to pose the question of "how well" a concept of interest is being measured when that concept has multiple competing meanings. for example, stereotypes can be negative, neutral, or positive in sentiment. an instrument that performs well at measuring stereotypes with negative sentiment may perform poorly if its validity is interrogated with respect to a broader definition of stereotyping that encompasses other sentiments. we therefore recommend that designers of measurement instruments do not skip systematization (or conflate it with operationalization) and clearly document systematized concepts as part of making measurement instruments publicly available. once a concept has been systematized, questions of how accurately the systematized concept is being measured can be answered using different lenses of validity and reliability @xcite . because participants reported perceived threats to the validity of measurement instruments, we recommend that designers of measurement instruments use these lenses to rigorously interrogate the validity and reliability of their instruments, providing evidence from these interrogations when making their instruments publicly available (see @xcite . lastly, we recommend that designers of measurement instruments release resources (e.g., guidelines, code) that practitioners can use to interrogate validity and reliability in different contexts. pragmatic measurement suggests additional ways to improve the usefulness of measurement instruments. all participants struggled to use measurement instruments that were misaligned with their needs. we argue that drawing on pragmatic measurement, which builds on measurement theory and has historically been concerned with improving the real-world effectiveness of clinical research, can help designers of measurement instruments to improve aspects of the usefulness of their instruments beyond validity and reliability @xcite . designers of measurement instruments should draw on pragmatic measurement to ensure that their instruments are interpretable, actionable, and scalable in different contexts. pragmatic measurement suggests ways to improve the interpretability, actionability, and scalability of measurement instruments, all of which contribute to their usefulness. many participants reported struggling to use measurement instruments that lack interpretabilty. pragmatic measurement offers several ways to improve the interpretability of measurement instruments. for example, where possible, designers of measurement instruments should publish distributions of measurements produced by their instruments for known datasets @xcite . 8 additionally, designers of measurement instruments should publish information about how to interpret the measurements produced by their instruments (e.g., what does a measurement of 0.3 mean vs. a measurement of 0.8?) @xcite . these suggestions are simple, evidencebased ways to improve interpretability. furthermore, they are complementary to the recommendations for improving actionability recently proposed by @xcite . see @xcite , @xcite , and @xcite for other ways to further improve the usefulness of measurement instruments. practitioners are responsible for adapting measurement instruments to meet their specific needs-but designers of measurement instruments can help by ensuring that their instruments are extensible. pragmatic measurement emphasizes the importance of extensibility, i.e., whether an instrument can be adapted for different systems, use cases, and deployment contexts. 9 in pragmatic measurement, extensibility is a low-effort, high-impact way to make measurement instruments useful: rather than designing multiple measurement instruments to meet different needs, designers can focus on extensible instruments that practitioners can individually tailor @xcite . based on our findings in §4 and the pragmatic measurement literature, we identify key criteria that need to be met for measurement instruments to be considered extensible. first, we recommend making them open source or otherwise modifiable. second, we recommend that they be modular, i.e., composed of discrete, interconnected pieces @xcite . designers of measurement instruments and practitioners should both ensure that measurement instruments remain valid and reliable when adapted to new contexts. a measurement instrument may be sufficiently valid and reliable in one context, but not in another. to ensure that extensible measurement instruments remain valid and reliable, we recommend that designers of measurement instruments be explicit about what is being measured (the systematized concept) and how it is being measured. separating systematization from operationalization can enable measurement instruments to be more easily modified. designers of measurement instruments should also be explicit about the impacts on validity and reliability of modifying different aspects of their instruments when adapting them to new contexts @xcite . we also emphasize that any time a measurement instrument is to be used in a new context, its validity and reliability must be re-interrogated @xcite . to facilitate this, we reiterate that designers of measurement instruments should release resources (e.g., guidelines, code) that practitioners can use to interrogate validity and reliability in different contexts. designers should also make it easy for practitioners to compare measurements produced by instruments with and without adaptations @xcite . extensibility can help overcome practical barriers impeding the uptake of measurement instruments. participants noted practical barriers to using measurement instruments, including security and data licensing. extensible measurement instruments that can adapted to support local use may help mitigate some security concerns. it should also be possible to adapt extensible measurement instruments to incorporate newly obtained, proprietary, or other sources of data in order to overcome data licensing issues. trade-offs. designing measurement instruments that are simultaneously valid, reliable, specific, extensible, scalable, interpretable, and actionable and are actually used by practitioners is a tall order. trade-offs are inevitable: for example, as measurement instruments become more specific to a particular context, they are likely to become less extensible. we emphasize the importance of interrogating validity and reliability, and, beyond that, we caution against over-indexing on any particular desideratum. for example, designers of measurement instruments who optimize for scalability without considering other desiderata may face unwanted trade-offs. by analogy, blocklists, which are commonly used to mitigate representational harms, are relatively low effort to deploy and scale, but do not wholly and fully capture the contextual nature of representational harms. as a result, they produce false positives when social groups engage in actions like reclaiming slurs, which can lead to over-moderation or erasure of those groups @xcite . ## conclusion we found that practitioners are often unable to use publicly available instruments for measuring representational harms and identified two types of challenges. in some cases, instruments are not useful because they do not meaningfully measure what practitioners seek to measure or are otherwise mis-aligned with practitioner needs. in other cases, instruments-even useful instruments-are not used by practitioners due to practical and institutional barriers impeding their uptake. furthermore, both types of challenges can be exacerbated by the specifics of measuring representational harms or evaluating llm-based systems. drawing on measurement theory and pragmatic measurement, we provided recommendations for addressing these challenges to better meet practitioner needs.