Thesis Proposal: D etecting Agency Attribution (2024)

Abstract: We explore computational methods for perceived agency attribution in natural language data. We consider ‘agency’ as the freedom and capacity to act, and the corresponding Natural Language Processing (NLP) task involves automatically detecting attributions of agency to entities in text. Our theoretical framework draws on semantic frame analysis, role labelling and related techniques. In initial experiments, we focus on the perceived agency of AI systems. To achieve this, we analyse a dataset of English-language news coverage of AI-related topics, published within one year surrounding the release of the Large Language Model-based service ChatGPT, a milestone in the general public’s awareness of AI. Building on this, we propose a schema to annotate a dataset for agency attribution and formulate additional research questions to answer by applying NLP models.

## introduction the value of studying power relations through the lens of language has been investigated in various contexts from online communities to film plots, see e.g. @xcite ; danescu- @xcite ; @xcite . across different fields, agency has a range of definitions, which highlight different aspects of the concept. within the context of this work, agency is taken as the freedom and capacity of an entity to act, corresponding to one facet of power. perceived agency is then the agency that we project on other entities while interpreting a description of a situation. perceived agency is important because it signals autonomy and independence, but also moral accountability: it is hard to imagine a hero or villain who is always a victim of circumstance. the perception of agency also influences how we assign blame or praise. for example, the actor-observer cognitive bias @xcite is the general tendency to explain other individuals' behaviours as an effect of their personalities, i.e. as something they cannot help doing given who they are, and our own behaviours as the rational response to our current situation. for example, someone else's slow driving may be attributed to their age or gender, but when we drive slowly we attribute it to specific reasons such as worn tyres. if we feel we deserve credit, we can frame our behaviour as an active choice; if we want to avoid guilt, we can emphasise the external pressure. to assign, e.g. responsibility, agency attribution is frequently manipulated in political discourse and partisan reporting to affect the audience @xcite . for example, there are discursive techniques to humanise or dehumanise migrants that draw on agency @xcite : portraying incoming migrants as independent agents (e.g. asylum seekers) in opposition to more passive roles (refugees) naturally affects public perception and can influence the assumptions underlying political decisions @xcite . however, even without an explicit political intention, the wording may suggest how agency is assigned. wikipedia, for example, is widely known to have its guidelines built around a 'neutral point of view', but the editorial bias (not limited to agency) that the guidelines seek to eliminate remains @xcite . in the general audience media, it is not uncommon to observe agency shifts between collectives and individuals ('the company has decided to lay off' vs. 'the board has decided' vs. 'the ceo has decided') and between individuals and artefacts ('the car crashed into the bridge' vs. 'the driver crashed his car into the bridge') (te brömmelstroet, 2020). furthermore, perceived agency plays a key role when discussing entities that are specifically designed to appear intelligent. most prominently -the various technologies referred to as ai, but also, for example, toys, voice recognition systems, and non-playable video game characters. therefore, perceived agency is created by a mixture of intentional and unintentional messaging. in this proposal, we focus on perceived agency as a prediction target for computational models, and as a direction for linguistic analysis in computational social sciences. if framed as a question-answering task, the prompt would be 'who or what acts or can act independently and intentionally in this situation?'. from this, we derive two research objectives. the first is to investigate the computational approach to studying attributed agency (rq1), in particular, the efficacy of different computational models in predicting perceived agency. rq1: what linguistic features and computational models are most suitable for predicting perceived agency? the second goal is to apply nlp methods as part of studies in computational social science to understand agency in specific contexts. we are interested in learning whether: (i) it is feasible to use such computational models to measure the public's perception of agency, (ii) the models generalise across narrow topics, and (iii) these models can be part of topic-specific social scientific studies and combined with qualitative approaches. in short: rq2: to what extent can automated prediction of perceived agency in text answer questions from social science? what questions can it answer? ## prior computational work there is limited research on nlp methods for agency attribution. the closest work is likely that by @xcite on perceived responsibility, which is part of their broader work on detecting perspectivisation @xcite using framenet-based annotations @xcite . they project perceived responsibility in a sentence onto three axes: blame, focus, and cause, and demonstrate that text features account for some of the differences in these facets of perceived responsibility. in particular, they study italian news reporting of femicides and dutch articles reporting traffic accidents @xcite . the latter is based on the study by te brömmelstroet (2020) that investigates news headlines about traffic accidents. the authors annotated articles into agentive and non-agentive categories based on the phrasing. this group of works focus on the attribution of responsibility ('who or what is blamed in this situation?'), rather than specifically on agency as we describe it above. a larger group of related work is focused on the (perceived) semantic roles that entities play in a text. this approach goes back to the folklorist studies of @xcite and focuses on identifying archetypes, such as 'hero', 'villain' or 'victim'. computational applications of propp's ideas vary from directly applying components of his grammar to new texts @xcite to using large language models (llms) for zero-shot role labelling @xcite . in terms of domains of application, news articles are prominent @xcite , but role prediction has also been applied in other settings containing political discussions. for example, identifying semantic roles in memes has been used as a shared task and prompted both text-based and multimodal solutions @xcite . finally, semantic role labelling, as well as framenet-style annotations, are used in the field of emotion detection @xcite . as in agency attribution, emotion detection assumes a choice of perspective (i.e. that of the writer or reader) before making predictions. a particularly relevant study by @xcite is related to both of these groups of works and introduces frames of agency and power to investigate the subtler types of gender bias in modern films. their study focuses on establishing the agency of characters throughout a longer narrative (compared to shorter messages we are interested in) and emphasises authority as one of the main indicators of agency, but even with these differences in approach, it remains one of the closest points of reference for this proposal. ## application areas we consider two application areas, namely the agency ascribed to ai systems and the examination of bias in news reporting. ai anthropomorphism our first domain of interest, to which most of our preliminary work has been dedicated, is the ongoing discourse on systems claiming to be artificial intelligence (ai). recent developments of llms and their branding as 'ai' reinforce the anthropomorphisation of the technology. generative models, especially those used in chatbots, tend to emulate first-person human speech, and end users are intended to project higher levels of agency on these systems. this may have positive effects in some contexts (sheehan et al., 2020) but also have highly adversarial effects, e.g. a dangerous over-reliance on the system @xcite . more broadly, the degree to which ai systems are viewed as active agents is reflected in public conversation and news coverage. the impact of alternative phrasings is illustrated in reporting bias our second domain of interest is attributed agency bias towards marginalised groups in news reporting. rhetoric plays a significant role in advancing political agendas and through the "correct" linguistic choices, stereotypical qualities can be ascribed to individuals. for example, the labelling of individuals as opportunistic or immoral has been demonstrated to influence public opinions and migration policies @xcite @xcite . agency attribution is an important aspect of this phenomenon and a focus of our investigation. in both of these domains, language reflects how the media or the public interprets specific technological and social issues. these interpretations by the mass media inform and influence those who make policies and regulations, thus translating perceptions into reality. ## proposed work this section outlines the preliminary and proposed future work that will go into the thesis. ## conclusion when we interpret stories, make decisions based on them, or place responsibility or blame, we rely on our perception of agency to understand whose intentions are driving events. language choices can intentionally and unintentionally influence this perception and, ultimately, our reactions. in this proposal, we put forward perceived agency detection as an nlp task and outline our preliminary and planned work on creating and annotating a perceived agency dataset. our focus is on two topical areas: a narrow one (perceived agency of ai) and a broader one (perceived agency in news headlines, with an eye towards bias). we describe some of our planned computational experiments, which will evolve as we learn from our findings, and aim for computational social science applications. we hope this proposal brings focus to the notion of perceived agency and highly welcome all types of feedback to further improve it.