Modeling Framing in Immigration Discourse on Social Media (2021)

Abstract: The framing of political issues can influence policy and public opinion. Even though the public plays a key role in creating and spreading frames, little is known about how ordinary people on social media frame political issues. By creating a new dataset of immigration-related tweets labeled for multiple framing typologies from political communication theory, we develop supervised models to detect frames. We demonstrate how users’ ideology and region impact framing choices, and how a message’s framing influences audience responses. We find that the more commonly-used issue-generic frames obscure important ideological and regional patterns that are only revealed by immigration-specific frames. Furthermore, frames oriented towards human interests, culture, and politics are associated with higher user engagement. This large-scale analysis of a complex social and linguistic phenomenon contributes to both NLP and social science research.

## introduction framing selects particular aspects of an issue and makes them salient in communicating a message @xcite . framing can impact how people understand issues, attribute responsibility @xcite , and endorse possible solutions, thus having major implications for public opinion and policy decisions @xcite . while past work has studied framing by the news media and the political elite, little is known about how ordinary people frame political issues. yet, framing by ordinary people can influence others' perspectives and may even shape elites' rhetoric @xcite . to shed light on this important topic, we focus on one issue-immigration-and develop a new methodology to computationally analyze its framing on twitter. our work highlights unique insights that social media data offers. the massive amount of available social media content enables us to compare framing strategies across countries and political ideologies. furthermore, social media provides unique insights into how messages resonate with audiences through interactive signals such as retweets and favorites. by jointly analyzing the production and reception of frames on twitter, we provide an in-depth analysis of immigration framing by and on the public. political communications research has identified numerous typologies of frames, such as issuegeneric policy, immigration-specific, and narrative. each of these frame types can significantly shape the audience's perceptions of an issue @xcite @xcite , but prior nlp work seeking to detect frames in mass media (e.g. @xcite @xcite has largely been limited to a single issue-generic policy typology. multiple dimensions of framing must be considered in order to better understand the structure of immigration discourse and its effect on public opinion and attitudes. we thus create a novel dataset of immigration-related tweets containing labels for each typology to facilitate more nuanced computational analyses of framing. this work combines political communication theory with nlp to model multiple framing strategies and analyze how the public on twitter frames immigration. our contributions are as follows: ## framing in the media framing serves four functions: (i) defining problems, (ii) diagnosing causes, (ii) making evaluative judgments, and (iv) suggesting solutions @xcite . framing impacts what people notice about an issue, making it a key mechanism by which a text influences its audience. framing typologies we draw upon distinct typologies of frames that can be applied to the issue of immigration: (1) issue-specific, which identify aspects of a particular issue, or (2) issue-generic, which appear across a variety of issues and facilitate cross-issue comparison @xcite . issue-generic frames include policy frames that focus on aspects of issues important for policymaking, such as economic consequences or fairness and equality @xcite . other generic frames focus on a text's narrative; news articles use both episodic frames, which highlight specific events or individuals, and thematic frames, which place issues within a broader social context. the use of episodic versus thematic frames can influence the audience's attitudes. for example, episodic frames lead audiences to attribute responsibility for issues such as poverty to individual citizens while thematic frames lead them to hold the government responsible @xcite . issue-specific frames for immigration focus on the portrayal of immigrants. our analysis uses @xcite 's set of issue-specific frames, which represent immigrants as heroes (cultural diversity, integration, good workers), victims (humanitarian, global economy, discrimination), and threats (to jobs, public order, taxpayers, cultural values). both issue-specific and generic frames provide unique insights but present advantages and drawbacks. while issue-specific frames analysis are specific and detailed, they are hard to generalize and replicate across studies, which is a key advantage for generic frames @xcite . framing effects studies of framing typically focus on either frame-building or framesetting @xcite . frame-building is the process by which external factors, such as a journalist's ideology or economic pressures, influence what frames are used; frame-building studies thus treat framing as the dependent variable. frame-setting studies treat frames as independent variables that impact how an audience interprets and evaluates issues. prior analyses of frame-building in immigration news highlight region and ideology as particularly important factors. right-leaning media from conservative regions are more likely to frame immigrants as intruders @xcite , and as threats to the economy and public safety @xcite . framing also differs across countries; while the us press emphasizes public order, discrimination, and humanitarian concerns, the french press more frequently frames immigrants as victims of global inequality @xcite . frame-setting has also been studied in the context of immigration. for example, experimental work has shown that frames eliciting angry or enthusiastic emotions impact participants' opinions on immigration @xcite . while past work has analyzed linguistic framing in twitter immigration discourse (e.g., @xcite , little is known about how such framing affects users' interactive behaviors such as resharing content, which is a key objective of frame setting. ## computational approaches to framing because many people now generate and consume political content on social media, scholars have increasingly used automated techniques to study framing on social media. large-scale research of framing on twitter has commonly focused on unsupervised approaches. (e.g., @xcite @xcite . such approaches, including those focused on hashtag analysis, can reveal interesting framing patterns. for instance, @xcite shows that frame usage varies across events. similarly, topic models have been used to compare "refugee crisis" media discourses across the european countries @xcite , and to uncover differences in attitudes towards migrants @xcite . although lexicon analysis and topic models can provide insights about immigration discourse, here, we adopt a supervised approach to ground our work in framing research and to enable robust evaluation. we draw inspiration from a growing body of nlp research that uses supervised approaches to detect issue-generic policy frames in news articles, a task popularized by the media frames corpus @xcite , which contains issuegeneric policy frame labels for articles across several issues @xcite . using this corpus, prior work has detected frames with techniques including logistic regression @xcite , and narrative @xcite frames with brief descriptions. 2016), recurrent neural networks @xcite , lexicon induction @xcite , and fine-tuning pretrained language models @xcite . @xcite further extracted subcategories of issue-generic policy frames in newspaper coverage using a weakly-supervised approach. finally, issue-generic frames have also been computationally studied in other media, including online fora and politicians' tweets @xcite . we build upon this literature by incorporating additional frame typologies that reflect important dimensions of media discourse with real-world consequences @xcite @xcite . beyond detecting frames, we computationally analyze frame-building and frame-setting among social media users; though well-studied in traditional news media, little is known about how social media users frame immigration or its effects @xcite . noting that issue-generic policy frames obscure important linguistic differences, several works stud-ied issue-specific frames in news media for issues such as missile defense and gun violence @xcite . we extend issuespecific frame analyses to immigration by adopting an immigration-specific typology developed by political communication scholars @xcite . in contrast to prior nlp work focused on traditional media or political elites @xcite , we highlight the role that social media publics play in generating and propagating frames. furthermore, we provide a new computational model of narrative framing @xcite , that together with models for issue-generic policy and issue-specific frames, provides complementary views on the framing of immigration. finally, our large-scale analysis of frame-setting illustrates the potential for using nlp to understand how a message's framing shapes its audience behavior. ## data we first collect a large dataset of immigrationrelated tweets, and then annotate a subset of this full dataset for multiple types of frames. ## frame detection we formulate frame detection as a multilabel classification problem for each of the three typologies, using our dataset to train supervised models. experimental setup our proposed model is a roberta model @xcite trained using binary cross-entropy on the cls token. we consider both (i) a model trained using the roberta-base parameters and (ii) a second model that has first been fine-tuned on our full set of immigration tweets using masked-language modeling. fine tuning was performed for 60 epochs. in both models, early stopping is used to avoid overfitting. models are compared with two baselines: random prediction, and logistic regression with unigram and bigram features. each model was trained five times with different random seeds and we report bootstrapped mean performance. ## frame-building analysis in writing about an issue, individuals are known to select particular frames-a process known as frame-building-based on numerous factors, such as exposure to politicians' rhetoric or their own identity @xcite . here, we focus on two specific identity attributes affecting frame building: (i) political ideology and (ii) country/region. the political, social, and historical contexts of an one's nation-state can impact how they frame immigration @xcite . immigration has a long history in the usa relative to europe, and former european colonial powers (e.g. the uk) have longer immigration histories than other countries (e.g. norway) @xcite . cross-country variation in news framing also arise from differences in immigration policies @xcite , media systems @xcite , journalis-tic norms @xcite , geographic proximity to immigrant populations or points of entry @xcite , and immigrants' race/ethnicity @xcite . at the same time, increased globalization may result in a uniform transnational immigration discourse @xcite . framing variations across countries has implications for government policies and initiatives, particularly in determining what solutions could be applied internationally or tailored to each country @xcite . prior studies on the role of ideology in framebuilding have focused on the newspapers or political movements, showing patterns in frames like morality and security by political affiliation in european immigration discourse @xcite or in use of economic frames by american newspapers @xcite . however, it remains unclear whether these patterns observed for elite groups can generalize to the effect of individual people's political dispositions. ## audience response to frames chong and druckman (2007, p. 116) assert that a "challenge for future work concerns the identification of factors that make a frame strong." studies of frame-setting-i.e., how a message's framing affects its audience's emotions, beliefs, and opinions-have largely been restricted to smallscale experimental studies because responses to news media framing cannot be directly observed @xcite . however, twitter provides insight into the frame-setting process via interactive signals: favorites and retweets. while related, these two actions can have distinct underlying motivations: favoriting often indicates positive alignment between the author and the reader; in contrast, retweeting may also be driven by other motivations, such as the desire to inform or entertain others @xcite . different audience interactions have been shown to exhibit distinct patterns in political communication on twitter @xcite . here, we test how a message's framing impacts both the favorites and retweets that it receives. experimental setup we fit hierarchical linear mixed effects models with favorites and retweets (log-transformed) as the dependent variable on us ## conclusion users' exposure to political information on social media can have immense consequences. by leveraging multiple theory-informed typologies, our computational analysis of framing enables us to better understand public discourses surrounding immigration. we furthermore show that framing on twitter affects how audience interactions with messages via favoriting and retweeting behaviors. this work has implications for social media platforms, who may wish to improve users' experiences by enabling them to discover content with a diversity of frames. by exposing users to a wide range of perspectives, this work can help lay foundations for more cooperative and effective online discussions. all code, data, annotation guidelines, and pretrained models are available at https: //github.com/juliamendelsohn/framing.