Cross-Dialect Social Media Dependency Parsing for Social Scientific Entity Attribute Analysis (2022)

Abstract: In this paper, we utilize recent advancements in social media natural language processing to obtain state-of-the-art syntactic dependency parsing results for social media English. We observe performance gains of 3.4 UAS and 4.0 LAS against the previous state-of-the-art as well as less disparity between African-American and Mainstream American English dialects. We demonstrate the computational social scientific utility of this parser for the task of socially embedded entity attribute analysis: for a specified entity, derive its semantic relationships from parses’ rich syntax, and accumulate and compare them across social variables. We conduct a case study on politicized views of U.S. official Anthony Fauci during the COVID-19 pandemic.

## introduction corpora of social media text contain wide ranges of beliefs that researchers may seek to analyze. but numerous studies have found significant challenges in applying natural language processing (nlp) techniques to social media, ranging from inconsistent spelling practices to continuously evolving terminology @xcite . under the now-ubiquitous modeling paradigm of pretrained transformers @xcite @xcite , it is crucial to include social media content in a language model pretraining corpus. bertweet @xcite , a language model trained entirely on english twitter, has shown state-of-theart results in classification @xcite , part-of-speech (pos) tagging @xcite , and named entity recognition (ner) @xcite on social media english. in addition, treebanks have been annotated to cover this specific variety of english. tweebank v2 @xcite consists of 3,550 english tweets annotated according to universal dependencies @xcite , and @xcite add ner tags following the four-class conll 2003 guidelines @xcite . other work has considered the impact of demographic and dialectical factors on social media nlp. @xcite investigate linguistic variation of african-american english (aae) on twitter from aggregate user demographics, developing a small 500 tweet universal dependencies corpus half of which consists of tweets heavily using aae. on this aae subset, dependency parsers encounter worse performance than on mainstream american english (mae), and a similar aae-mae dialect disparity is widespread in other areas of nlp (e.g. @xcite . social media nlp advances could enable novel techniques in computational social science. retrieval and representation of the beliefs and opinions of various groups and ideologies is of clear importance to many social sciences, with applications ranging from misinformation studies @xcite to political science and economics @xcite . with these goals in mind, we train a state-of-the-art social media dependency parser, evaluating social media english performance, as well as aae dialect disparity, among eleven alternative pretrained models ( §3). to illustrate dependency parsing's utility for social media analysis, we implement a rule-based semantic attribute extractor to analyze authors' views toward an entity (figure 1 ; §4), and evaluate it in a case study of political narratives surrounding the u.s. official dr. anthony fauci during the covid-19 pandemic-we compare extractions against the authors' social variable of geolocated election results ( §5). we find our tweetie system has better yield and higher precision for this task, compared to using previous open information extraction systems. ## related work: social semantic extraction natural language processing has been used to extract social insight from corpora in humanistic and social scientific study. @xcite ; @xcite analyze the economic impact of dependency parse-extracted adjective modification from product reviews and seller feedback, associating perceived attributes with monetary prices. narrative analysis of fictional characters has used dependency parses to extract attributes associated with character archetypes @xcite ; our semantic relation extractor follows and extends their approach. these dependency-based systems can be viewed as expanding on widely used collocation methods that tabulate words appearing near an entity @xcite ; for example, @xcite use words directly before an entity (a rough adjective modifier extractor) to analyze attributes ascribed to immigrants in political discourse. in the nlp context, outside of computational social science, open information extraction (oie) is a related semantic approach that extracts relational tuples without a predefined schema, often applied to large heterogenous corpora, such as web data @xcite , typically using off-the-shelf nlp technologies such as part-of-speech (pos) tagging, named entity recognition (ner), semantic role labelling, and dependency parsing @xcite . our tweetie information extractor uses a rule system working directly from dependency parses, following the approach of argument extraction and normalization systems props @xcite and @xcite ; the latter performs well on oie benchmarks @xcite . we share predpatt's motivation to rely on univerisal dependencies parses, which have coverage and availability across many language varieties, including social media english here. this contrasts favorably to the domain-dependent limitations of machine-learned semantic role labeling @xcite and semantic dependency parsing @xcite . ## tweetie: belief extraction from dependencies a well-performing social media dependency parser, along with pre-existing pos and ner taggers, enable novel applications for computational social science. we apply these technologies for a belief extraction system, which decodes these syntactic structures into simple semantic representations and presents information applicable for computational social scientific purposes, specifically the delin-eation of beliefs to communities represented by social variables. we call this system tweetie. ## case study: covid-19 polarization a key source of variation in opinion is with respect to political ideology, and social media is rife with arguments about political figures specifically. in this section, we show tweetie's ability to capture the ideological attributes of said figures, specifically the attributes social media users ascribe to dr. anthony fauci, director of the national institute of allergy and infectious diseases, who is a key figure in united states covid-19 discourse. while tweetie could be used to study a network of entities and their relations, we find focusing on a single entity is a useful and insightful first step. ## conclusion and future work the annotations from tweebank v2 and the performance improvements from bertweet have lead to significant advancements in social media dependency parsing, with performance gains of 3.4 uas and 4.0 las, as well as significantly lessening how much performance lags for the non-standard language variety of african-american english. these achievements enable downstream applications of syntactic parsing on social media data, of which we note information extraction as being especially utilizable for computational social scientific means. we outline a process to decode these dependency parses into aggregatable semantic structures, for comparisons with social variables that one may seek to study. we show how one can model political narratives with respect to named entities with a case study on elements and actions attributed to dr. anthony fauci on social media during the covid-19 pandemic. through this, we replicate findings in social scientific literature on the topic, and we have similar extractions to pre-existing open information extraction yet with increased yield, enabling more substantial computational social scientific analyses. future work can build upon these foundations by extending these techniques to beliefs spanning multiple entities, by considering additional social variables, or by taking into account temporal effects through timestamps. this could allow for the observation of more complex phenomena, such as actions from an entity towards another entity or the adoption and decline of beliefs over time.