Investigating Controversy Framing across Topics on Social Media (2025)

Abstract: Controversial discourse is abundant on social media. Understanding how controversial problems are framed in online discourse is crucial for gaining insights into public opinion formation and for addressing misinformation and polarization. This paper presents a novel method for discovering and articulating framing of controversial problems, enabling the investigation of how controversy is framed across several diverse topics. The promising results, made possible by recent advances in Large Language Models, indicate that discovering framings across topics is feasible. The discovered frames offer valuable insights into how and why controversial problems are discussed on social media.

## introduction pew research center @xcite indicates that social media now plays a crucial role in shaping public discourse -especially among younger adults -by serving as a primary source for news and opinion shaping. in this digital public sphere, how issues are understood and opinions formed depends not only on the content that is shared, but also on the way that content is framed @xcite @xcite @xcite . as defined by @xcite "to frame is to select some aspects of a perceived reality and make them more salient in a communicating text, in such a way as to promote problem definition, causal interpretation, moral evaluation, and/or treatment recommendation for the item described." most previous work (e.g. @xcite ; @xcite ; @xcite ; @xcite ; @xcite ; @xcite ) focused only on the automatic discovery of framing aspects which are addressed throughout social media posts. figure 1 shows how, for the topic of immigration, 27 different aspects/problems (a few illustrated in the figure ) were automatically detected in posts by @xcite when the identification of frame aspects addressed in posts was cast as a multi-label classification problem. although the identification of the frame aspects addressed in posts is very important, we claim it does not inform us how each of those controversial aspects is framed. we know only which reality aspects/problems are highlighted by the frames, but we are not aware of how these controversial aspects are interpreted or framed. entman's definition indicates that the causal interpretation of the topic's problems reveals how the problems are framed. therefore, we propose to automatically generate the frames of communication (focs) that are (1) evoked from posts and (2) provide an interpretation of the framing of the topic's highlighted aspects/problems. in this way, as shown in figure 1 , following the framework introduced in @xcite , we consider that (a) each foc may be evoked by multiple posts; and (b) each foc provides a different interpretation of a topic's aspect; while (c) multiple posts may address the same topic's aspect. the fact that each topic aspect affords multiple interpretations, provided by the focs evoked in the posts that address it, indicates that these topic aspects correspond to controversial problems (cps) associated with a topic. an example of a post that evokes an foc is illustrated in figure 2 . the same post addresses a cp, for which the foc provides an interpretation. ## the datasets we relied on five different datasets: two reference datasets and three additional datasets used for testing and evaluation of frame discovery and articulation method introduced in this paper. the first reference dataset covers the topic of covid-19 vaccines (hereafter, c19), and the second one covers immigration (hereafter, imm). both datasets include social media posts from twitter / x along with the annotated frames of communication they evoke. additionally, for each frame, annotations are available of the underlying problems interpreted and definitions for those problems. the three testing and evaluation datasets consist of posts from twitter / x discussing the controversial topics of abortion (ab), climate change (cc), and feminism (fm). dataset for covid-19 vaccines (c19): consists of the only available dataset of social media posts annotated with articulated frames, namely covaxframes @xcite . this dataset includes frames related to covid-19 vaccination hesitancy. vaccine hesitancy is characterized by seven factors, or problems, that increase or decrease an individual's likelihood of getting vaccinated @xcite , these problems, along with their definitions, are provided in appendix a. for the present study, we use the test portion of covaxframeswhich includes 2,815 posts from twitter / x with 113 evoked frames and the corresponding problem interpretations. dataset for immigration (imm): extends the dataset introduced by @xcite , which consists of 2,730 posts from twitter / x annotated with any of the 27 generic, narrative, and immigration-specific problems they address. the analysis of immigration framing in @xcite relied on the 14 issue-generic frame problems introduced by the policy frames codebook @xcite and the media frames corpus @xcite , and expanded these problems by introducing 11 issue-specific problems @xcite ) and 2 narrative-focused problems @xcite surrounding the topic of immigration. the definition for each problem is provided in appendix a. we have further annotated the 72 focs evoked in the posts of this dataset. the focs were produced by communication expert-led frame analysis, in the same way as in @xcite . abortion (ab), climate change (cc), and feminism (fm): the semeval-2016 task 6 dataset @xcite was widely employed to train and evaluate automatic stance detection methods on five controversial topics @xcite @xcite @xcite . each post, sourced from twitter / x, is annotated with the stance expressed towards one of five controversial topics: abortion, atheism, climate change, feminism, and hillary clinton. the topics of abortion, climate change, and feminism were selected for our study due to extensive prior work investigating the controversial problems addressed for these topics. appendix a lists these controversial problems for each topic, along with their definitions and sources of the definitions. we utilize all social media posts from each topic, as provided by tweeteval @xcite , which consists of 933 posts about abortion (ab), 564 posts discussing climate change (cc), and 949 posts about feminism (fm). it is important to note that these three test datasets, ab, cc, and fm do not have available annotations of focs evoked in their posts, and thus can only serve as evaluation datasets. ## the method our top-down approach for discovering frames of communication (focs) was inspired by how communication experts perform framing analysis @xcite @xcite . framing analysis consists of a complex series of inductive, then deductive judgments on an entire corpus, from which expert inference of the articulation of focs emerges @xcite @xcite . in the inductive phase, the discovery and articulation of focs results from expert reasoning with the cps addressed across multiple texts derived from corpus-wide inspection to arrive at a consistent, causal interpretation of each cp. the deductive phase identifies the texts from the entire corpus where the focs are evoked, revealing the relevance of each foc, cf. @xcite . foc discovery and articulation is very difficult, even for trained communication experts who rely on codebooks emerging from their reasoning and painful inspection of large quantities of texts @xcite @xcite . to our knowledge, corpus-wide automatic discovery and articulation of focs (corresponding to the inductive phase) has not been attempted before. however, methods for automatic identification of foc evocation (corresponding to the deductive phase) have been reported in @xcite . therefore, we decided that it is crucial for our new top-down approach to focus only on the automatic discovery and articu-lation of focs. for this purpose, we designed the corpus-wide discovery & articulation of focs (cwda-foc) method, which leverages some recent advances made possible by newer llms, answering the following questions. q1: why did recent llms facilitate corpus-wide discovery and articulation of focs? first, long-context llms, such as gpt-4o @xcite , o1 @xcite , and o3-mini (ope-nai, 2025) allow us to encode entire corpora of social media posts in their contexts, unlike llms with smaller context sizes. for the datasets introduced in section 2, we require llms capable of encoding contexts that approach 100,000 tokenssignificantly more than models such as bert (512) @xcite , llama 2 (4,096) @xcite , and even gpt-3.5 (16,385) @xcite and earlier versions of gpt-4 (32,768) (openai, 2023). second, newer llms allow structured prompting. strict structured prompting, which arose as a technique that enables a strict format to be specified for llm outputs @xcite , is ideal for specifying (1) how focs are expected to be articulated and ( 2 ) what format demonstrations presented to the llm should follow. the strict output format is imposed using a constrained decoding methodology, which ensures the llm generates text in a specified format, such as json. constrained decoding deterministically changes the next-token probabilities of an llm, such that all non-acceptable tokens (based on the specified format) are assigned probability zero. third, llms must also support long output contexts that are sufficient to articulate potentially hundreds of discovered frames, while also supporting strict structured prompting. q2: how can we take advantage of existing corpora annotated with cps and focs for discovering and articulating focs from new corpora? in section 2 we introduced two corpora of social media posts annotated with the addressed cps and the evoked focs, namely the c19 corpus and a new one, introduced by this paper, namely imm. any of these corpora can be considered as possible examples of a reference corpus. formally, a reference corpus r(t 1 ) ={p 1 1 , p 1 2 , ..., p 1 m } of posts, discussing a topic t 1 , has annotations from ( 1 ) a list of cps: {c 1 1 , c 1 2 , ..., c 1 q } and their definitions; (2) a list of articulated focs {f 1 1 , f 1 2 , ..., f 1 x }, with (3) additional information linking each foc f 1 i to the cp (or cps) c i,1 j (or {c i,1 j , ..., c i,1 k } it is interpreting. this formalization of the reference corpus informs the design of novel corpus-wide one-shot (cwos) prompting of llms to enable our cwda-foc method. the cwos prompting method, illustrated in figure 3 , uses a prompt that consists of three different contexts: 212 a reference context r c ; 222 a discovery context d c ; and 232 a test context t c . to generate r c a context template is used, having slots for: (a) the topic name; (b) the name of each cp and their definitions; as well as (c) the corpus of posts. the context template is filled with information from the reference collection of posts, r(t 1 ) by a function f r . the function f r fills the slot for the topic name with the name of t 1 . similarly, f r fills the slots for the name and definition of each cp of r(t 1 ) with the name of each c 1 i and its definition. the slot of the corpus of posts is filled with the text from {p 1 1 , p 1 2 , ..., p 1 m }. the reference corpus r(t 1 ) also informs d c , through a discovery template, having slots of (a) each foc discovered and articulated from {p 1 1 , p 1 2 , ..., p 1 m }, the posts of r(t 1 ); as well as (b) sub-slots for cps interpreted by each foc. the discovery template is filled with information from r(t 1 ) by a function f d . the function f d fills each foc slot with a f i from r(t 1 ). it also fills the sub-slots for the cps interpreted by each foc f i with {c i j , ..., c i k } if f i interprets all those cps. it is important to note that d c does not contain any information indicating in which posts any foc f i from r(t 1 ) is evoked. when a new corpus of posts discussing another topic t 2 , n (t 2 ) is considered, we formalize it as n (t 2 ) ={p 2 1 , p 2 2 , ..., p 2 n }, where each p 2 j is a post from the n (t 2 ) corpus. however, this test corpus has no annotations of focs. but, we are aware that when discussing the topic t 2 , the list of cps that are addressed are: {c 2 1 , c 2 2 , ..., c 2 n }. to discover and articulate focs from n (t 2 ), t c is generated by using again the context template. this time the context template is populated by the function f r operating on n (t 2 ), thus (a) filling the topic name slot with the name of t 2 ; (b) filling the slots for the name and definition of each cp of n (t 2 ) with their name of each c 2 i and its definition; and (c) the slot of the corpus of posts is filled with {p 2 1 , p 2 2 , ..., p 2 n }. as shown in figure 3 , the prompt consisting of a concatenation of r c , d c , and t c is presented to an llm. while the r c provides the necessary information to perform foc discovery and articulation, d c is key in teaching the llm how it should perform this complex task, and together r c and d c serve as a single one-shot demonstration. t c then invites the llm to ( 1 ) generate {f 2 1 , f 2 2 , ..., f 2 y }, the focs evoked in n (t 2 ); and (2) to provide information linking each foc f 2 i to the cp (or cps) c i,2 j (or {c i,2 j , ..., c i,2 r }) it is interpreting. in summary, the cwda-foc method relies on prompting new llms using the cwos prompting method when (a) considering a reference corpus discussing a topic t 1 ; and (b) discovering the focs from a new corpus discussing a different topic t 2 . figure 4 illustrates the cwda-foc method and the details of the cwos prompting method when using as reference corpus c19 and the new corpus imm. all details of the cwos prompting methods are presented in appendix b. ## evaluation results we considered the only two methods proposed for foc discovery and articulation: a bottom-up approach, and our top-down corpus-wide discovery and articulation of frames of communication (cwda-foc) approach, introduced in section 3. ## discussion discussion of the quantitative evaluation: the advantages of cwda-foc are apparent as (1) it is the first and only method capable of performing cross-topic foc discovery and articulation, and (2) cwda-foc performs extremely well across multiple topics, as illustrated in tables 1, 2, 3, 4, and 5. on the covid-19 vaccines dataset, the cwda-foc method, as well as the prior cot-icacl approach, achieve strong scores on soundness and clarity, as provided in the cot-icacl method achieved high performance by leveraging multiple chain-of-thought demonstrations for the same covid-19 vaccine topic; however, its applicability is limited to scenarios where the reference demonstrations and test posts discuss the same controversial topic. in contrast, the top-down cwda-foc method overcomes this limitation by employing a single oneshot demonstration drawn from a reference corpus discussing a different topic. cwda-foc thereby offers greater flexibility and scalability across topics. we also found that, interestingly, some of the best-performing systems were llms trained to perform reasoning, such as o1 and o3-mini. we attribute this improved performance to the one-shot nature of our task, relying on a single demonstra-tion. this indicates that reasoning models have more of an opportunity to "think" about this single demonstration. for topics where a comprehensive reference set of focs is unavailable (abortion, climate change, and feminism as shown in tables 3 , 4 , and 5), the evaluation focused solely on soundness and clarity, showing that that cwda-foc is effective in articulating evoked focs. although performance varies with the underlying llm, models like o3mini and o1 consistently produce focs that are both sound and clear, underscoring the robustness of cwda-foc across a multitude of controversial topics. we also find that the reference topic of immigration produced more sound and clearly articulated focs on the similarly political topics of abortion and feminism, as shown in discussion of the qualitative evaluation: the analysis of the focs articulated by the bestperforming cwda-foc configurations on each evaluation dataset (ab, fm, and cc) reveals insights into the quality of focs automatically discovered on these topics. when immigration was used as the reference topic, o3-mini articulated for the topic of abortion f oc 1 : "fetuses do not consent to death, and therefore abortion is murder." f oc 1 was also identified by o3-mini as interpreting the problems of "fetal rights", "sanctity of life", and "language & terminology". for the topic of feminism, o1 articulated f oc 2 : "accusations of 'feminazis' or 'ugly feminists' in media are used to dismiss women's issues as trivial or extremist." f oc 2 was noted as interpreting the problem of "media representation and backlash" by o1. when the topic of covid-19 vaccines was used as a reference topic, o3-mini articulated for the topic of climate change f oc 3 : "technical discussions on biodiversity and habitat loss, though data-driven, can render climate change an abstract and distant concern for the average person" and f oc 4 : "unseasonably cool july temperatures are invoked to question long-term climate trends, feeding claims that climate science remains uncertain and abstract." the o3-mini model also identified that both f oc 3 and f oc 4 interpret the problem of "psychological distance & abstraction", while f oc 4 also interprets the problem of "scientific uncertainty & climate denial". ## related work early research on framing in social media relied heavily on unsupervised approaches @xcite @xcite ) that uncovered lexical patterns and network structures suggestive of framing. while these methods revealed interesting trends, they did not articulate the underlying focs or identify the specific problems that these focs address. other studies have developed classifiers to detect frameinvoking language @xcite , yet these approaches did not capture the causal interpretations embedded within focs. a growing body of work has applied supervised nlp methods, often leveraging the media frames corpus (mfc) @xcite , to detect frame-interpreted problems. techniques ranging from logistic regression @xcite and recurrent neural networks @xcite to lexicon induction @xcite and fine-tuning pre-trained language models @xcite have been used to analyze news and social media. more recently, weakly-supervised methods have been employed to extract subcategories of policy frame dimensions @xcite . notably, @xcite identified immigration policy problems in social media posts using multilabel classification based on roberta @xcite . however, all of these prior methods focused solely on discovering frame problems rather than articulating them with explicit causal rationales. more recent research using llms has started to expand beyond frame problems. @xcite utilized existing focs to perform counterfactual reasoning with llms to perform zero-shot stance detection, while @xcite demonstrated that stance detection towards focs has both theoretical and empirical benefits over stance detection towards topics or claims. islam and goldwasser (2025) uncovered latent arguments from posts on social media using llms, which constructed themes (problems) and talking points for each theme. ## conclusion this paper presents a new, top-down method capable of discovering and articulating frames of communication from social media. by leveraging a single one-shot demonstration drawn from an external reference dataset, corpus-wide one-shot (cwos) prompting overcomes the limitations of methods that require topic-specific demonstrations, thereby enabling effective cross-domain framing analysis. extensive evaluations across multiple topics demonstrate that cwos not only articulates a substantial portion of known frames but also uncovers many new frames that are both clearly articulated and sound.