Riveter: Measuring Power and Social Dynamics Between Entities (2023)

Abstract: Riveter provides a complete easy-to-use pipeline for analyzing verb connotations associated with entities in text corpora. We prepopulate the package with connotation frames of sentiment, power, and agency, which have demonstrated usefulness for capturing social phenomena, such as gender bias, in a broad range of corpora. For decades, lexical frameworks have been foundational tools in computational social science, digital humanities, and natural language processing, facilitating multifaceted analysis of text corpora. But working with verb-centric lexica specifically requires natural language processing skills, reducing their accessibility to other researchers. By organizing the language processing pipeline, providing complete lexicon scores and visualizations for all entities in a corpus, and providing functionality for users to target specific research questions, Riveter greatly improves the accessibility of verb lexica and can facilitate a broad range of future research.

## introduction language is a powerful medium that intricately encodes social dynamics between people, such as perspectives, biases, and power differentials @xcite . when writing, authors choose how to portray or frame each person in a text, highlighting certain features @xcite to form larger arguments @xcite . for example, in the screenplay for the 2009 film sherlock holmes, the authors dramatize a sudden reversal of power by playing on gender stereotypes. first, they describe how "the man with the roses beckons irene forward" (figure 1 ), which portrays the character irene adler as being lured by the man. after she is trapped, "she slices upward with a razor-sharp knife," reversing the power dynamic. here, specific word choices shape and then challenge the viewers' expectations about how the interaction is presumed to unfold. more broadly, an author's word choices can not only communicate important details about a character or narrative but can also reveal larger social attitudes and biases @xcite , shape readers' opinions and beliefs about social groups @xcite , as well as act as powerful mechanisms to persuade or to induce empathy @xcite . studying these word choices across large datasets can illuminate domainspecific patterns of interest to scholars in the humanities and social sciences. examining verbs-who does what to whom?-is one established approach for measuring the textual portrayal of people and groups of people. each verb carries connotations that can indicate the social dynamics at play between the subject and object of the verb. connotation frames @xcite capture these dynamics by coding verbs with directional scores. for example, these frames might label verbs with who holds power and who lacks power, or who is portrayed with positive or negative sentiment. in the sherlock holmes scene description, "another man steps in behind her, trapping her," the verb to trap implies that "the man" has more power over "her" (figure 1 ; @xcite . these verb lexica have been used successfully to study portrayals in many diverse contexts including films @xcite , online forums @xcite ), text books @xcite , and news articles @xcite . lexica in general are extremely popular among social science and digital humanities scholars. they are interpretable and intuitive @xcite , especially when compared with black-box classification models, and continue to be a go-to resource (e.g., liwc; @xcite . however, verb-based lexica pose specific technical hurdles for those less experienced in software engineering and natural language processing (nlp). these lexica require core nlp skills such as traversing parse trees, identifying named entities and references, and lemmatizing verbs to identify matches. at the same time, the larger research questions motivating their usage require deep domain expertise from the social sciences and humanities. to meet this need, we introduce riveter , 1 which includes tools to use, evaluate, and visualize verb lexica, enabling researchers to measure power and other social dynamics between entities in text. this package includes a pipeline system for importing a lexicon, parsing a dataset, identifying people or entities, resolving coreferences, and measuring patterns across those entities. it also includes evaluation and visualization methods to promote grounded analyses within a targeted dataset. we release riveter as a python package, along with jupyter notebook demonstrations and extensive documentation aimed at social science and humanities researchers. to showcase the usefulness of this package, we describe two case studies: (1) power differentials and biases in gpt-3 generated stories and (2) gender-based patterns in the novel pride and prejudice. the first study provides a proof-of-concept; dyads with predetermined power differentials are used to generate stories, and we are able to detect these distribution shifts using riveter. the second study zooms in on a particular author, text, 1 the name riveter is inspired by "rosie the riveter," the allegorical figure who came to represent american women working in factories and at other industrial jobs during world war ii. rosie the riveter has become an iconic symbol of power and shifting gender roles-subjects that the riveter package aims to help users measure and explore by combining (or riveting) components into a pipeline. rashkin et al. (2016) sap et al. (2017) agent theme effect(a) effect(t) state(a) state(t) value(a) value(t) verb agency(a) persp(a↔t) power(a↔t) wrtpersp(a) wrtpersp(t) rdrpersp(a) rdrpersp(t) figure 2: a verb predicate can connote various sentiment, power, and agency levels for its agent and theme; connotation frames distill these into six relation types ( §2.3; each type is colored in a different hue). and social setting, examining how a 19th century novelist both portrayed and subverted gender roles. these case studies highlight the diverse contexts and research questions for which this package can be used across human and machine-generated text, and across the social sciences and the humanities. ## ethics and broader impact riveter comes with some risks and limitations. this package is targeted only at english-language texts; we have not included non-english lexica in the tool nor do we expect the parsing, named entity recognition, and coreference resolution to directly translate to other languages. while related lexica exist for non-english languages (e.g., @xcite (german), @xcite (extension to 10 european languages)), the generalizability of riveter is limited to english-language settings. the results of riveter are only as reliable as the corpora and lexica used as input (and their relationships to one another). we have emphasized interpretability in designing this package, encourag-ing users to examine their results at different levels of granularity. however, there are still dangers of biases "baked-in" to the data, via its sampling and curation, or to the lexica, in the choice of terms and their annotations by human workers. lexica that are useful in one setting are not always useful in other settings, and we encourage researchers to validate their lexica on their target corpora. drawing from a framework describing the roles for computational tools in social change @xcite , we believe that riveter can fill multiple important roles. first, it can act as a diagnostic, measuring social biases and hierarchies across large corpora, as in @xcite where dehumanization of lgbtq+ people was measured across news datasets and time. riveter can also act as a formalizer, allowing researchers to examine the specific words used by authors, adding concrete and fine-grained evidence to the constructions of broader patterns, as in @xcite where the supporting words were used to characterize healthcare roles during childbirth. finally, riveter can act as a synecdoche by bringing new attention and perspectives to long-recognized problems, as in @xcite where renewed attention was given to gender biases in film.